feat(checkpoint): support sharded generation-prefix recovery

[macandro96]

Summary

• export each Gym shard's active generation-cut index
• validate that every prepared shard contributes one digest-bound cut fragment
• union peer cut indexes into one authoritative shared policy-model ledger commit
• restore sharded generation prefixes while preserving shard-local agent/resource state
• add sharded prefix recovery and failure-path coverage

Stack

• depends on feat(checkpoint): support sharded Gym turn recovery #4322
• Gym counterpart: feat(checkpoint): union sharded generation cuts Gym#3756

Validation

• ruff check and ruff format --check passed on the final stack
• Python compilation passed for the checkpoint orchestration modules
• the two-shard prefix-recovery functional test still needs to run on the cluster

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

❌ Submodule Fast-Forward Check Failed

Check based on commit: a058431 (PR #4323 from amahishi/prefix-recovery-sharded-prefix)

❌ Submodules that need attention:

Gym: ❌ Commits have DIVERGED from a common ancestor
TARGET (amahishi/prefix-recovery-sharded-gym branch): https://github.com/NVIDIA-NeMo/Gym/commits/e95d57f7d91af426a17336a1ae03398d42de0479/
CURRENT (PR #4323 from amahishi/prefix-recovery-sharded-prefix): https://github.com/NVIDIA-NeMo/Gym/commits/126a8adcbb2a90e42d5681bf99ccbe2bc2e54497/

Please ensure all submodule commits are fast-forwards of the amahishi/prefix-recovery-sharded-gym branch before merging.

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 36b14b2 (PR #4323 from amahishi/prefix-recovery-sharded-prefix)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of amahishi/prefix-recovery-sharded-gym branch (fast-forward)

All submodule changes look good! ✨