Repository navigation
[LTX-2] Add decode_audio to skip the audio decode in modular pipelines - #15045
Open
christopher5106 wants to merge 1 commit into
Open
christopher5106 wants to merge 1 commit into
christopher5106 wants to merge 1 commit into
Conversation
`LTX2AutoAudioDecoderStep` wraps `LTX2AudioDecoderStep` in a `ConditionalPipelineBlocks` that runs it unless `decode_audio=False`, in every LTX-2 and LTX-2.5 decoder chain. A video-only output (e.g. an IC-LoRA alpha matte) then skips the audio VAE and vocoder and gets `audio=None`; the video is unchanged, since the audio latents are still denoised jointly with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds a per-call
decode_audioinput (defaultTrue) to the LTX-2 / LTX-2.5 modular pipelines.pipe(..., decode_audio=False)skips the audio VAE decode and vocoder and returnsaudio=None, for video-only outputs such as an IC-LoRA alpha matte (Lightricks/LTX-2.5-22b-IC-LoRA-Alpha-Gen). The video is unchanged, since the audio latents are still denoised jointly with it.LTX2AutoAudioDecoderStep(ConditionalPipelineBlocks,modular_blocks_ltx2.py): runsLTX2AudioDecoderStepunlessdecode_audio=False, otherwise skipped. It is the same pattern asLTX2AutoPromptEnhancerStep.LTX2AudioDecoderStepinLTX2DecoderStep,LTX2ConditionDecoderStep,LTX25DecoderStepandLTX25ConditionDecoderStep, under the sameaudio_decodename: workflow layouts and the existing workflow tests are unchanged.LTX2AudioDecoderStepitself only gains thedecode_audioinput declaration, so the input shows up in the resolved workflows' inputs; it has no new branch.decode_audio=Falsegivesaudio=Noneand bit-identical videos, for LTX-2 text2video and LTX-2.5 in_context.utils/modular_auto_docstring.py, thendoc-builder style --max_len 119.Fixes #15044
Before submitting
self-reviewskill on the diff?docs/source/en/api/pipelines/ltx2.mdcovers the non-modular pipelines only)Self-review notes
Blocking: none.
Non-blocking, left for review:
LTX2AudioDecoderStepdeclaresdecode_audiobut does not read it (decoders.py).get_workflow()resolves conditional blocks down to their leaf, so declaring it there is what makes the input appear in a workflow's inputs. This mirrorsenable_prompt_enhancementon the prompt-enhancer leaves. The alternative is overridinginputsonLTX2AutoAudioDecoderStep, but resolved workflows would then not list it.select_blockkeys on the value of a user input (decode_audio is False), not on its presence. The note inConditionalPipelineBlocks.select_blockrestricts value-based selection for intermediate outputs only, andLTX2AutoPromptEnhancerStepdoes the same withenable_prompt_enhancement.audio_latents/audio_num_framesas optional: a conditional block without a default has no required inputs.LTX2AudioDecoderStepstill requires them whenever it runs.Dead code: none.
Verdict: READY.
Who can review?
@yiyixuxu @DN6