perf(checkpoint): reduce Gym control-plane overhead

[macandro96]

Summary

• return and consume completion receipts inline with /run
• batch and retry idempotent completion acknowledgements with reserved control-plane HTTP capacity
• coalesce controller ACK obligations without holding the data-plane mutation barrier during network I/O
• commit independent Gym participants concurrently within dependency-ordered stages

Stack

• depends on fix(checkpoint): harden generation-prefix recovery #4320
• Gym counterpart: perf(checkpoint): scale recovery control paths Gym#3755
• next: sharded Gym recovery

Validation

• ruff check passed on the reconciled optimization files
• Python compilation passed
• scale benchmarks and cluster checkpoint suites still need to run

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 411df78 (PR #4321 from amahishi/rl-checkpoint-scale-optimizations)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of amahishi/prefix-recovery-hardening branch (fast-forward)

All submodule changes look good! ✨