Skip to content

add modular blockset for diffusion gemma - #14978

Open
akshan-main wants to merge 2 commits into
huggingface:mainfrom
akshan-main:diffusion-gemma-modular
Open

akshan-main wants to merge 2 commits into
huggingface:mainfrom
akshan-main:diffusion-gemma-modular

Conversation

@akshan-main

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a modular blockset for DiffusionGemma: DiffusionGemmaBlocks + DiffusionGemmaModularPipeline, with a text encoder step, a generation setup step (canvas layout, KV cache, EOS state), a scheduler setup step that splits the budget into predictor and corrector steps, a canvas loop that prefills, denoises and appends each block, and a decode step that trims at EOS and detokenizes. It is the first modular pipeline with text output. The checkpoint keeps its weights at the repository root, so the docs build the components by hand as in the standard usage.

Outputs match DiffusionGemmaPipeline token for token with shared components and seed, on the tiny components and on google/diffusiongemma-26B-A4B-it in bf16 (one and two canvases, temperature sampling, a batch of two prompts of different lengths, and the static cache path). The tests include the batched adaptive-stopping cases from #14386. Test components live in akshan-main/tiny-diffusion-gemma-modular-pipe.

The first commit is the loader fix from #14661 (a transformers model folder that shadows a pipeline folder resolved to the wrong library, which broke save and reload here) and drops out once that lands. #14661 also makes the scheduler accept CPU generators on CUDA, which the GPU runs of the modular tests need.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline?
  • Did you read our philosophy doc (important for complex PRs)?
  • Was this discussed/approved via a GitHub issue or the forum? Discussed on Slack with @sayakpaul @yiyixuxu
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

@yiyixuxu @sayakpaul

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests modular-pipelines utils pipelines size/L PR with diff > 200 LOC labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Hi @akshan-main, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an issue (or gets the no-issue-needed label), you can ignore this message — it stays here as a comment, but it no longer applies.

@yiyixuxu yiyixuxu added the no-issue-needed for PRs that do not require link to an issue label Oct 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation modular-pipelines no-issue-needed for PRs that do not require link to an issue pipelines size/L PR with diff > 200 LOC tests utils

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants