feat(trtllm): Qwen3.5 routed-expert block-FP8/MXFP8 refit

[hchings]

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[hchings]

Team review: Qwen3.5 routed-expert block-FP8 / MXFP8 refit

Reviewed by a team of 4 agents plus a leader re-verification pass over the full diff, with every upstream claim checked against the TensorRT-LLM commit this repo pins rather than main. 8 comments below.

The quantization module itself is good, and that's the bulk of the work here. Worth saying explicitly:

• quantization/fp8.py imports only math/re/torch and takes TRT-LLM types as parameters (configure_fp8_moe_backend(llm_kwargs, moe_config_type, ...)) rather than importing them. That's why the 223-line test file exercises the real quantization math on CPU with a 6-line stub and no engine mocking — a strictly better shape than the vllm/quantization/fp8.py analog, which routes the same decisions through a module-level mutable global.
• We checked the numerics against the pinned consumer and they're right: the VANILLA per-expert split is what Qwen3_5MoeHfWeightMapper expects, the [ceil(N/128), ceil(K/128)] FP32 scale orientation and dequant = fp8 * scale_inv match create_weights, the MXFP8 plain-uint8 [..., K/32] layout matches MXFP8CutlassFusedMoEMethod (which does its own packing and swizzle, so not pre-swizzling is correct), and all 13 modules_to_not_convert patterns translate correctly through upstream's Qwen3.5 normalizer. The block-FP8 caster was validated on CPU across aligned, unaligned, batched and non-contiguous inputs.
• The multi-worker result aggregation fix in trtllm_worker_async.py is a genuine improvement — the old results[0] masked failures on ranks > 0, and the updated tests cover it.

The findings share one root cause, which is why several read similarly: this was developed against a newer TensorRT-LLM than the repo pins, and the branch appears not to have been run against the pinned build. Three of the comments below are consequences of that (recompute_active_requests, the FP8 refit hooks, the torch.compile hooks); one is an unrelated rebase artifact that breaks all TRT-LLM runs (engine_cfg); and three are test/doc drift. None of them touch the design.

Generated by Claude Code