fix(megatron): bind gradient callbacks to runtime model config - #4116
Open
DanialTaheri wants to merge 1 commit into
Open
DanialTaheri wants to merge 1 commit into
DanialTaheri wants to merge 1 commit into
Conversation
Use get_model_config(model) when installing runtime callbacks from NeMo-RL's own setup path. Providers such as Nemotron Omni may copy their configuration during construction, so the MCore scheduler must receive callbacks on the constructed model's configuration. Cover copied/shared configs, nested wrappers, overlap on/off, and training versus forward-only finalizer dispatch. Port the NeMo-RL changes from df6b2d7d3102048d85ac6afead41dd4813352e59 without the local Bridge backport gitlink or other experiment changes. The existing Bridge callback installer supports this fix unchanged. Signed-off-by: smohsenitahe <smohsenitahe@nvidia.com>
Contributor
Author
|
/ok to test ebfb72e |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Bind NeMo-RL's Megatron runtime callbacks to the constructed model's configuration, not the original provider configuration.
finalize_megatron_setup()invokes Bridge's_update_model_config_funcs()directly. Providers such as Nemotron Omni can copy their configuration during model construction, so installing callbacks onmegatron_cfg.modelleaves the runtime configuration read by the MCore scheduler without them. Useget_model_config(model)to unwrap the model and bind callbacks to the configuration actually used for training.The regression tests cover shared and copied configurations, nested model wrappers, gradient/parameter overlap enabled and disabled, and training versus forward-only execution. They check gradient-finalizer dispatch once after three training microbatches and no dispatch during forward-only execution.
Issues
Related: NVIDIA-NeMo/Megatron-Bridge#6067
Bridge's PR fixes its own setup call site; NeMo-RL has a separate call site and needs this companion change. The callback installer itself is unchanged, so this NeMo-RL change works with the target branch's existing Bridge pin. No dependency bump is included.
Usage
No configuration changes are required. The corrected callback binding is applied during Megatron policy setup.
Before your PR is "Ready for review"
TestFinalizeMegatronSetup.Additional Information