Skip to content

feat: preserve RLM call lineage across retries - #2385

Closed
hallerite wants to merge 2 commits into
feat/acp-response-metadatafrom
feat/rlm-lineage
Closed

feat: preserve RLM call lineage across retries#2385
hallerite wants to merge 2 commits into
feat/acp-response-metadatafrom
feat/rlm-lineage

Conversation

@hallerite

@hallerite hallerite commented Aug 17, 2026

Copy link
Copy Markdown
Member

Summary

  • add a generic, harness-bound logical-call resolver for stable identity, private request headers, and namespaced ModelCall.metadata
  • replay completed non-streaming calls and coalesce concurrent attempts by opaque logical ID while preserving the existing Stainless retry fallback
  • bind every logical ID to route, request bytes, and stable metadata so changed bodies or lineage cannot receive a stale replay
  • move every X-RLM-* constant, parser, schema rule, and stripping decision into the RLM harness protocol module
  • keep Trace.nodes authoritative for training branches; runtime lineage remains descriptive call metadata

Unrelated harnesses cannot activate RLM behavior merely by sending X-RLM-* headers.

Depends on #2384.

Validation

  • uv run pytest tests/v1/test_rlm_lineage.py tests/v1/test_trace.py -q
  • complete deterministic v1 suite passed as part of the stack
  • Ruff, format, and Ty passed locally and in CI

Comment thread verifiers/v1/clients/eval.py Outdated
Comment thread verifiers/v1/interception/server.py Outdated
@hallerite
hallerite force-pushed the feat/acp-response-metadata branch from 779fff8 to bfd6c6b Compare August 17, 2026 19:48
Comment thread verifiers/v1/session.py
class LogicalCallReplay:
binding: tuple[str, bytes, bytes]
response: dict | None = None
inflight: "asyncio.Future[dict | None] | None" = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High v1/session.py:102

A cancelled coalesced request cancels the shared LogicalCallReplay.inflight future, so the original exchange cannot publish its result and all other retry handlers receive CancelledError. Await this shared future through asyncio.shield so cancellation of one waiter cannot cancel synchronization for the other attempts.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/session.py around line 102:

A cancelled coalesced request cancels the shared `LogicalCallReplay.inflight` future, so the original exchange cannot publish its result and all other retry handlers receive `CancelledError`. Await this shared future through `asyncio.shield` so cancellation of one waiter cannot cancel synchronization for the other attempts.

@hallerite

Copy link
Copy Markdown
Member Author

Superseded by #2386, which now contains the full RLM ACP contract as one PR against main.

@hallerite hallerite closed this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant