Skip to content

feat: support RLM training over ACP - #2386

Draft
hallerite wants to merge 1 commit into
mainfrom
feat/rlm-acp-contract
Draft

feat: support RLM training over ACP#2386
hallerite wants to merge 1 commit into
mainfrom
feat/rlm-acp-contract

Conversation

@hallerite

@hallerite hallerite commented Aug 17, 2026

Copy link
Copy Markdown
Member

Summary

  • express the RLM runtime as typed harness configuration and send its strict ai.prime.rlm/runtime-v1 payload through ACP session/new
  • carry opaque ACP session/close metadata through the generic runner and let the RLM harness validate its own final ai.prime.rlm/session-v1 snapshot into training metrics
  • honor the standard HTTP Idempotency-Key in the interception server by coalescing in-flight retries and replaying completed non-streaming responses within one rollout
  • namespace the key before forwarding it upstream so separate rollouts cannot collide in a provider-global idempotency cache
  • pin the live RLM E2E to Nano commit 7a26d3c2cd2f38e11807ef7b88ce4ea197552ea8 until the companion Nano PR merges

Verifiers does not model RLM lineage, compaction, subagents, or ACP lifecycle artifacts. Those semantics stay inside Nano; Verifiers only transports standard ACP close metadata and standard HTTP idempotency.

Contract

This intentionally requires the new Nano contract and contains no backwards-compatibility path. Land PrimeIntellect-ai/nano-rlm#134 before this PR, then remove the temporary CI commit pin once Nano main contains the contract.

Validation

  • Ruff check and format passed on all touched Python files
  • pytest tests/v1/test_idempotency.py -q — passed against a real loopback interception server
  • complete deterministic v1 suite — passed
  • Prime VM contract E2E — passed, covering two ACP prompts, MCP resume, concurrent recursive children, kernel/subprocess environment isolation, and final close metrics
  • pre-push hooks — markdownlint, Ruff, format, and Ty all passed

Comment thread verifiers/v1/harnesses/rlm/harness.py Outdated
Comment thread verifiers/v1/harnesses/rlm/harness.py Outdated
Comment thread verifiers/v1/acp/runner.py Outdated
@hallerite
hallerite changed the base branch from feat/rlm-lineage to main August 18, 2026 21:45
@hallerite
hallerite force-pushed the feat/rlm-acp-contract branch from 3f8ca38 to 4b4b1ac Compare August 18, 2026 22:50
@hallerite hallerite changed the title feat: use the RLM ACP training contract feat: support RLM training over ACP Aug 18, 2026
Comment thread verifiers/v1/acp/runner.py Outdated
@hallerite
hallerite force-pushed the feat/rlm-acp-contract branch 2 times, most recently from eb232a8 to 1b401b7 Compare August 19, 2026 19:23
Comment thread verifiers/v1/interception/server.py
@hallerite
hallerite force-pushed the feat/rlm-acp-contract branch from 1b401b7 to dcadf55 Compare August 19, 2026 21:20
@@ -253,9 +261,21 @@ async def _stop(self, *, graceful: bool) -> None:
return
try:
if graceful and reader is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium acp/__init__.py:263

Graceful close swallows exceptions from acp_close_metrics and trace.record_metrics, so invalid close metadata or metrics make the session appear to close successfully while final RLM metrics are silently absent. Limit the best-effort except to shutdown transport, allowing metric validation and recording failures to propagate.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/acp/__init__.py around line 263:

Graceful close swallows exceptions from `acp_close_metrics` and `trace.record_metrics`, so invalid close metadata or metrics make the session appear to close successfully while final RLM metrics are silently absent. Limit the best-effort `except` to shutdown transport, allowing metric validation and recording failures to propagate.

)
if idempotent is None:
idempotent = IdempotentRequest(binding=binding)
session.idempotent_requests[idempotency_key] = idempotent

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High interception/server.py:438

Each distinct client-supplied Idempotency-Key adds an entry to session.idempotent_requests that retains the full response.raw for the rollout, so a long-running harness with fresh keys grows memory without bound and can OOM the shared interception server. Add a bounded or expiring replay cache, or evict entries once their replay guarantee is no longer needed.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @verifiers/v1/interception/server.py around line 438:

Each distinct client-supplied `Idempotency-Key` adds an entry to `session.idempotent_requests` that retains the full `response.raw` for the rollout, so a long-running harness with fresh keys grows memory without bound and can OOM the shared interception server. Add a bounded or expiring replay cache, or evict entries once their replay guarantee is no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant