perf: enable HybridEP in remaining GB200 MoE performance recipes

[seonjinn]

Summary

Enable Flex + HybridEP in the remaining Qwen3-30B-A3B and Nemotron3 Super GB200 performance recipes: Sync / Async-1off × BF16 / MXFP8 rollout. Six YAML edits cover eight workloads through inheritance.

Set NVL72 topology explicitly, per-domain ranks matching EP16 (EP8 for Qwen Async), and packed-input prepadding. Preserve existing batch sizes, parallelism, precision, allocator/refit settings, and dense-model recipes. No dependency or submodule changes.

Measured performance — BF16 only

BF16 validation: four of seven matched comparisons completed 20 steps and final validation. Qwen3-30B-A3B Sync and Async-1off, and Nemotron3 Super Async-1off completed. Qwen3-235B-A22B Async-1off also completed as an additional benchmark of an already-enabled recipe (not a new YAML change in this PR). Nemotron3 Super Sync remains blocked by OOM; Qwen3-235B-A22B Sync 16n4g and 32n4g lack completed comparisons. MXFP8 results and further MXFP8 validation are excluded from this report; this description update does not change the YAML diff.

Actual training steps 2–20, 19 observations per reported metric; scheduled validation is included in E2E time. Values are AlltoAll → HybridEP (change). Both policy training and rollout use BF16; all Async runs are Async-1off.

Step time — seconds

Workload	GB200 GPUs	E2E	Policy training	Logprob	W&B
Qwen3-30B-A3B / Sync / BF16	16	180.69 → 160.72 (-11.1%)	80.63 → 64.84 (-19.6%)	19.88 → 15.44 (-22.4%)	Baseline / HybridEP
Qwen3-30B-A3B / Async-1off / BF16	16	242.77 → 210.18 (-13.4%)	188.18 → 160.74 (-14.6%)	39.87 → 32.44 (-18.6%)	Baseline / HybridEP
Nemotron3 Super / Async-1off / BF16¹	128	54.01 → 53.25 (-1.4%)	10.65 → 7.75 (-27.3%)	4.69 → 4.19 (-10.8%)	Baseline / HybridEP
Qwen3-235B-A22B / Async-1off / BF16	128	127.16 → 120.27 (-5.4%)	48.79 → 39.56 (-18.9%)	15.90 → 12.90 (-18.9%)	Baseline / HybridEP

Throughput — logged tokens/sec/GPU

Workload	E2E	Policy training	Logprob
Qwen3-30B-A3B / Sync / BF16	2305.50 → 2594.52 (+12.5%)	5151.42 → 6403.53 (+24.3%)	20898.17 → 26895.26 (+28.7%)
Qwen3-30B-A3B / Async-1off / BF16	1727.27 → 2014.46 (+16.6%)	4407.28 → 5162.22 (+17.1%)	20800.45 → 25570.85 (+22.9%)
Nemotron3 Super / Async-1off / BF16¹	142.91 → 166.35 (+16.4%)	1196.35 → 1631.53 (+36.4%)	2617.49 → 2990.57 (+14.3%)
Qwen3-235B-A22B / Async-1off / BF16	222.40 → 253.74 (+14.1%)	926.37 → 1142.39 (+23.3%)	2839.57 → 3499.34 (+23.2%)

Qwen3-30B-A3B uses 4 × 4 GB200 GPUs: Sync colocated, Async-1off split 8 policy + 8 generation. Nemotron3 Super Async-1off uses 32 × 4 GB200 GPUs, split 64 + 64. Qwen3-235B-A22B Async-1off also uses 32 × 4 GB200 GPUs, split 64 policy + 64 generation: training TP4/PP4/CP1/EP16, generation TP8, sequence length 8192, batch size 512. Original recipe memory and batch settings are preserved; its two configurations differ only in dispatcher/backend and logging destinations. Resolved W&B configurations within each pair differ only in HybridEP options and logging destinations.

¹ Both Nemotron3 Super arms use gpu_memory_utilization=0.6 instead of default 0.7; this override is not in this diff. Qwen3-30B-A3B has no memory override. These are matched-config runs, not identical-token replays. Throughput values are means of logged per-step rates, not inverses of mean times. Nemotron3 Super mean tokens/sample change 2902 → 3021 and exposed generation 19.13 → 21.63 s. Its whole-window E2E throughput is 111.91 → 117.98 (+5.4%); the table's +16.4% is not a 16.4% wall-time reduction.

Accuracy — short-run observations only

Workload	Mean reward	Mean gen_kl_error	Step-20 validation accuracy
Qwen3-30B-A3B / Sync / BF16	0.52760 → 0.52572	0.001886 → 0.001893	55.47% → 54.30%
Qwen3-30B-A3B / Async-1off / BF16	0.52552 → 0.52614	0.001893 → 0.001890	54.30% → 52.73%
Nemotron3 Super / Async-1off / BF16¹	0.56867 → 0.56908	0.004272 → 0.003307	63.67% → 60.16%
Qwen3-235B-A22B / Async-1off / BF16	0.58337 → 0.58563	0.006932 → 0.006922	66.67% → 60.00%

Tracked reward/KL/loss/gradient/ratio metrics are finite across all 19 measured steps. Qwen3-30B-A3B and Nemotron3 Super validation uses 256 samples and one generation per prompt, without step-0 evaluation. Qwen3-235B-A22B retains its original, smaller validation configuration; its validation scores are not directly comparable across models. Its final accuracy declines by 6.67 percentage points despite similar mean reward and KL. Observed BF16 validation declines remain unresolved: one short pair cannot distinguish run variation from a systematic effect. Finite metrics and close rewards do not prove accuracy or convergence equivalence.

Remaining failures and validation boundaries

• Nemotron3 Super / Sync / BF16: original 0.7 reproduces vLLM KV-cache OOM with both dispatchers. 0.6 passes generation initialization but encounters host OOM. Disabling policy NUMA memory binding reaches reference logprob, then exceeds Ray's host-memory threshold (~883/909 GiB). No completed comparison.
• Qwen3-235B-A22B / Sync / BF16 (16n4g and 32n4g): both AlltoAll baselines hit NCCL ALLGATHER watchdog timeouts and eventually the four-hour job limit. Dependent HybridEP jobs did not start. No speedup or HybridEP failure conclusion is available; the underlying timeout cause is not established. Baseline W&B: 16n4g / 32n4g.
• Source boundary: experiments use NeMo-RL 90a2a212 with recipe overlays and DeepEP 17cfb817; this PR is based on d633032b. Intervening changes include shared W&B logging behavior. The additional Qwen3-235B-A22B runs use experiment commit 022fe8a746 on the same runtime baseline. These are not exact-PR-head runtime validation.
• Aggregation correction: earlier values used actual steps 1–19 while labeled 2–20. These tables are corrected to actual steps 2–20, checked against the runtime logging code and complete history.

Tests / readiness

Configuration regression tests: 102 passed, covering all 18 GB200 MoE recipes, dense exclusions, EP-dependent topology, prepadding eligibility, and preserved environment settings. This is not GPU runtime evidence.

uv run --no-project --with pytest --with hydra-core python -m pytest \
  --confcutdir=tests/unit/tools \
  tests/unit/tools/test_hybridep_default_8g_recipes.py -o addopts='' -q

• Update existing recipes directly; configuration tests pass.
• Complete Qwen3-30B-A3B Sync and Async-1off BF16 matched 20-step comparisons.
• Complete Qwen3-235B-A22B Async-1off BF16 matched 20-step comparison.
• Record performance, W&B links, and accuracy limitations.
• Resolve Nemotron3 Super Sync BF16 OOM and complete its matched comparison.
• Diagnose Qwen3-235B-A22B Sync baseline timeouts and complete both matched comparisons.
• Complete exact-PR-head runtime validation before declaring merge-ready.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[copy-pr-bot]

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[seonjinn]

/ok to test 3841bcc