Repository navigation
[TPU] TorchTPU backend integration - eager / torch.compile / tp - #14039
Open
JingyaHuang wants to merge 74 commits into
Open
JingyaHuang wants to merge 74 commits into
JingyaHuang wants to merge 74 commits into
Conversation
github-actions
Bot
added
documentation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
JingyaHuang
changed the title
JingyaHuang
marked this pull request as ready for review
September 7, 2026 16:23
…ster tpu.md; drop redundant execution_device check - Propagate the text_encoder.device-based fix (introduced for TPU CPU-offload support) from FluxPipeline/Flux2KleinPipeline/WanPipeline into their `# Copied from` copies (flux/*, flux2_klein_inpaint, visualcloze, anyflow, chronoedit, lucy_edit, skyreels_v2/*). SDXL-family copies of StableDiffusionXLPipeline.encode_prompt are intentionally left untouched; they'll be handled in a follow-up PR that fixes device placement for every pipeline component (not just text encoders). - Register docs/source/en/optimization/tpu.md in _toctree.yml (was breaking the docs build: "not present in the table of contents"). - Remove the redundant "prefer non-CPU, non-meta component" loop from DiffusionPipeline._execution_device: PR huggingface#14383 already fixed this in DiffusionPipeline.device, which _execution_device falls back to. Verified on TPU hardware that _execution_device still resolves correctly for a split-placement pipeline after the removal. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
sayakpaul
reviewed
Oct 5, 2026
Member
There was a problem hiding this comment.
Nice, thanks! The changes are minimal and we should be able to merge soon.
(Ran the updated tests on 2 A10Gs and they worked: https://huggingface.co/jobs/sayakpaul/6ac30cecfbc85ba6823a4556)
| image.save("output.png") | ||
| ``` | ||
|
|
||
| If the text encoder alone is too large for a single chip(eg. FLUX.2-dev's Mistral-3-Small is ~45GB), |
Member
There was a problem hiding this comment.
I think we should provide a working example here. Or we could use transformers' support for TP?
Also, the same argument (a model to be placed on a TPU chip is too large) applies to the DiT. I think we could make an entirely separate section to talk about TP, etc.
Or, we could include a statement like "For details on how to shard the denoising module, refer to the "## Tensor parallelism" section."
Contributor
Author
There was a problem hiding this comment.
Yeah, it would be better just show a simple example with everything ran on tpu w/o tp in this section, and have an example with TP on the TP dedicated section.
stevhliu
reviewed
Oct 6, 2026
sayakpaul
reviewed
Oct 7, 2026
sayakpaul
left a comment
sayakpaul
left a comment
Member
There was a problem hiding this comment.
Thanks! I left some further comments.
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
…n the TPU examples
…step, thus we can use it for the tpu.md compile mode snippet
…/diffusers into add-torchtpu-support
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Need the fix #14739 and perferrably merge the sharding improvement PR #14544 first.
torch.compileThis is a preparation based on TorchTPU beta before the official release.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.