False `miss_avoidable`: the TTL clock runs from the previous call, not from the entry the turn would have hit

[philmerrell]

classify_cache_status decides "avoidable" vs "TTL expired" using the gap to the immediately-previous call row. When consecutive calls in a session used different prefixes, that is the wrong clock, and a genuine TTL expiry gets booked as avoidable waste.

Mechanism

The caller takes the previous row with a Limit=1 descending scan — always the last call, whatever prefix it used (backend/src/apis/shared/sessions/metadata.py:405-427). classify_cache_status then compares that single gap against the TTL (prompt_cache.py:127):

if gap_seconds is None or gap_seconds > CACHE_TTL_SECONDS:
    return CacheStatus.MISS_TTL_EXPIRED
return CacheStatus.MISS_AVOIDABLE

But the entry a turn would have hit is the last call with the same prefix, which can be much older than the last call overall.

Measured instance

Dev session bf9481e7-62ec-4a14-817f-546a2953588d, during the #741 verification. An @-mention sits between two plain turns, so the plain turns' prefix and the mention's prefix interleave:

#	turn	time	status	read	write	gap	toolCfgHash
2	plain	22:29:34	hit	2306	65	27	4f53cda1
3	mention	22:30:16	miss_avoidable	0	3942	41	98890bac
4	plain	22:34:42	miss_avoidable	0	2720	266	4f53cda1

Row 4's cacheGapSeconds is 266 (against row 3, the mention) so it lands under the 300s TTL and is called avoidable. But row 4's prefix matches row 2, written at 22:29:34 — 308 seconds earlier. That entry had already expired. The re-write was unavoidable, and it was booked as $0.006256 of wastedUsd.

Why it matters

bump_session_aggregates (metadata.py:1497) rolls cacheStatus == "miss_avoidable" into per-session avoidableMissCount and wastedUsd. Those are the numbers the cost dashboard reports, and this whole layer exists to make a specific bug class — nondeterministic prefix assembly — visible. Inflating it with unavoidable expiries erodes exactly the signal it was built to provide.

In that six-turn test session, three rows were miss_avoidable: two were deliberate mention swaps and one was this misclassification. Zero were the bug the metric is for. That ratio gets worse, not better, as @-mentions get used — every mention makes the prefixes interleave.

Fix direction

The fingerprints needed to do this correctly are already on the row. Rather than "the previous call", find the most recent previous call whose toolConfigHash + systemPromptHash match this one, and measure the gap to that. Options:

1. Query a small window (say Limit=10) descending and pick the newest same-fingerprint row; fall back to today's behavior when none matches within the window. Cheap, no schema change.
2. Keep Limit=1 but treat a fingerprint change between consecutive calls as evidence that the relevant entry is older than the measured gap, and classify conservatively as miss_ttl_expired rather than miss_avoidable unless a same-prefix predecessor is known to be inside the TTL.

(1) is more accurate; (2) is a two-line change that removes the false positives at the cost of some false negatives. Both are better than the current rule, which is confidently wrong in the one scenario the platform has just started producing routinely.

Whichever lands, compute_wasted_usd follows automatically — it already returns 0.0 for anything that is not MISS_AVOIDABLE.

Not the same as the mention dimension

Distinct from the follow-up noted on #741. That one is about turns classified correctly — a mention really does change the prefix — that are nonetheless deliberate, and want a dimension saying so. This issue is about turns classified incorrectly: the classifier asserts a live cache entry existed when it had already expired. Fixing either does not fix the other.

Related

• @-mention forks conversation history: the mention turn and the plain turns run on two different cached agents #741 / fix(inference): one session is one conversation, whatever agent runs a turn #750 — the work that surfaced this
• The pre-existing "false miss_avoidable after below-threshold turns" nit from the original observability PR (fix: preserve Bedrock prompt-cache hits across turns (deterministic ordering + byte-stable compaction) #697) is the same family; the previous_cached_prefix_tokens <= 0 guard at prompt_cache.py:121 handles that case only.

🤖 Generated with Claude Code

[philmerrell]

Fixed by #754 (fix/cache-status-ttl-clock, merged to develop as 95851c7). The TTL clock now runs from the same-prefix predecessor rather than from the previous call, so a turn is no longer classified miss_avoidable against an entry it could never have hit.

Closing by hand: Fixes #753 did not auto-close because GitHub only fires that on merge to the default branch, and every PR here targets develop.

Note the distinct remaining gap, now tracked as #756: this fixed turns classified incorrectly. Turns classified correctly but deliberately — an @-mention swap — still book as waste, because nothing on the C# row records that the turn was a mention.