Skip to content

feat(ppo): add DeepSeek R1 7B single-controller recipe - #4119

Open
snowmanwwg wants to merge 3 commits into
mainfrom
feat/ppo-deepseek-r1-7b-single-controller-recipe
Open

snowmanwwg wants to merge 3 commits into
mainfrom
feat/ppo-deepseek-r1-7b-single-controller-recipe

Conversation

@snowmanwwg

Copy link
Copy Markdown
Contributor

Summary

  • add an 8-node x 4-GPU SingleController PPO recipe for DeepSeek-R1-Distill-Qwen-7B on DAPOMath
  • allocate four nodes to non-colocated vLLM generation and four nodes to colocated policy/value training
  • use 2,048 prompts per RL step with policy and critic global batch sizes of 2,048, satisfying the current SingleController one-optimizer-step-per-RL-step constraint
  • configure an 18,432-token context, sequence packing, and no system prompt

Testing

  • git diff --check
  • YAML syntax parsed successfully
  • GPU training not run as part of this PR preparation

Signed-off-by: Wenwen Gao <94138584+snowmanwwg@users.noreply.github.com>
@snowmanwwg
snowmanwwg requested a review from a team as a code owner September 13, 2026 04:29
@copy-pr-bot

copy-pr-bot Bot commented Sep 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@snowmanwwg

Copy link
Copy Markdown
Contributor Author

/ok to test 54d8d7f

Signed-off-by: Wenwen Gao <94138584+snowmanwwg@users.noreply.github.com>
@snowmanwwg

Copy link
Copy Markdown
Contributor Author

/ok to test 003db41

Signed-off-by: Wenwen Gao <94138584+snowmanwwg@users.noreply.github.com>
@snowmanwwg

Copy link
Copy Markdown
Contributor Author

/ok to test e0a4514

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant