fix(langchain): handle Anthropic cache_creation nested dict in usage

[hassiebp]

Summary

Anthropic's extended prompt caching returns a nested cache_creation dict in the usage payload:

{
    "input_tokens": 9454,
    "output_tokens": 380,
    "cache_read_input_tokens": 0,
    "cache_creation": {
        "ephemeral_1h_input_tokens": 0,
        "ephemeral_5m_input_tokens": 2048,
    },
}

_parse_usage_model in the LangChain callback handler ended with a isinstance(v, int) filter that silently discarded the nested dict, so cache-creation token counts were lost entirely (on v2.x this same payload caused a Pydantic ValidationError that dropped the whole generation end() event — see #1697).

Fix

Before the final int filter, flatten the nested dict into per-tier keys (cache_creation_ephemeral_5m_input_tokens, cache_creation_ephemeral_1h_input_tokens) and set an aggregated cache_creation_input_tokens total via setdefault, so an API-provided aggregate is never overwritten.

Parsed result for the payload above:

{
    "cache_read_input_tokens": 0,
    "input": 9454,
    "output": 380,
    "cache_creation_ephemeral_1h_input_tokens": 0,
    "cache_creation_ephemeral_5m_input_tokens": 2048,
    "cache_creation_input_tokens": 2048,
}

Tests

• test_anthropic_cache_creation_nested_dict — reproduces the reported payload, asserts flattened tiers + aggregate and that no non-int value survives
• test_anthropic_cache_creation_keeps_existing_aggregate — API-provided cache_creation_input_tokens wins over the computed sum
• test_anthropic_cache_creation_all_zero — zero tiers are kept, no aggregate is invented
• uv run pytest tests/unit/test_parse_usage_model.py → 9 passed; ruff + mypy clean

Fixes #1697
Fixes LFE-10230

🤖 Generated with Claude Code

Greptile Summary

This PR fixes silent loss of Anthropic extended prompt-caching token counts in the LangChain callback handler. When cache_creation is a nested dict (new API format), the existing isinstance(v, int) final filter was silently discarding it; on pydantic v2 it went further and raised a ValidationError that dropped the entire end() event.

• Adds a flattening step in _parse_usage_model that pops the cache_creation dict, creates per-tier flattened keys (e.g. cache_creation_ephemeral_5m_input_tokens), and computes an aggregate cache_creation_input_tokens via setdefault so an API-provided value is never overwritten.
• Adds three unit tests covering the standard case, the pre-existing aggregate case, and the all-zero tiers case.

Confidence Score: 4/5

The change is narrowly scoped to flattening a previously-discarded nested dict; all existing code paths are unaffected and the only observable change is that Anthropic extended-caching token counts now appear in the usage model instead of being silently dropped.

The flattening logic is correct and the three new unit tests cover the main behavioural contracts. The one gap is that the per-tier key assignment unconditionally overwrites any pre-existing flat key of the same name, but this requires a payload structure that does not appear in any known Anthropic API response today.

No files require special attention; the two changed files are self-contained and the test coverage matches the documented fix.

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
langfuse/langchain/CallbackHandler.py:1700-1701
**Flattened tier keys silently overwrite pre-existing flat keys**

If `usage_model` already contains a top-level `cache_creation_ephemeral_5m_input_tokens` key (e.g. from a future API response that sends both formats), `usage_model[f"cache_creation_{key}"] = value` will overwrite it without warning. The existing pattern for `prompt_tokens_details` and `input_token_details` has the same behaviour, so this is consistent, but it is worth noting if Anthropic ever starts sending mixed payloads. A `setdefault` here (mirroring the aggregate line below) would avoid the silent overwrite.

_{Reviews (1): Last reviewed commit: "fix(langchain): handle Anthropic cache_c..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[github-actions]

@claude review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 782c0398e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep cache-creation buckets mutually exclusive

When Anthropic/ChatAnthropic returns both the top-level cache_creation_input_tokens and the nested cache_creation tiers, this adds per-tier cache_creation_*_input_tokens buckets while the aggregate is still retained. Langfuse stores flat usage_details verbatim and expects every non-total key to be a mutually exclusive bucket; because raw Anthropic usage has no total_tokens, the derived total/UI input usage will count cache writes once via cache_creation_input_tokens and again via the new tier keys, inflating usage and any matching inferred costs. Please emit either the aggregate or the tier buckets, not both.

Useful? React with 👍 / 👎.

[claude]

LGTM — targeted fix that flattens Anthropic's nested cache_creation dict following the same pattern already used for other nested token details in this function.

Extended reasoning...

Overview

The PR patches _parse_usage_model in langfuse/langchain/CallbackHandler.py to handle Anthropic's extended prompt caching payload, which returns cache_creation as a nested dict keyed by cache tier (ephemeral_5m_input_tokens, ephemeral_1h_input_tokens). Previously the final isinstance(v, int) filter silently dropped the nested dict, losing the token counts (and on v2.x triggering a Pydantic ValidationError that discarded the whole generation event). Three unit tests are added covering the reported payload, aggregate preservation, and the all-zero edge case.

Security risks

None. This is a telemetry/usage-metric parsing change with no auth, crypto, permission, or user-input-boundary implications.

Level of scrutiny

Low. The change is a self-contained ~20-line addition inside a single function, follows the exact same flatten-and-optionally-adjust pattern already used for prompt_tokens_details, output_token_details, candidates_tokens_details, and cache_tokens_details a few lines above. Uses setdefault to avoid clobbering an API-provided aggregate, and the value-typed filter (isinstance(value, int)) matches the surrounding style.

Other factors

• Bug hunting system found no issues.
• Test coverage is proportionate to the fix: happy path, existing-aggregate precedence, and zero-tier edge case; also asserts no non-int value survives the final filter.
• Fixes a known issue (#1697 / LFE-10230) with a clear reproduction in the PR description.
• No outstanding reviewer comments; only the automated @claude review trigger.