Skip to content

[LTX-2] Add decode_audio to skip the audio decode in modular pipelines - #15045

Open
christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_ltx2_audio_decode_toggle
Open

christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_ltx2_audio_decode_toggle

Conversation

@christopher5106

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a per-call decode_audio input (default True) to the LTX-2 / LTX-2.5 modular pipelines. pipe(..., decode_audio=False) skips the audio VAE decode and vocoder and returns audio=None, for video-only outputs such as an IC-LoRA alpha matte (Lightricks/LTX-2.5-22b-IC-LoRA-Alpha-Gen). The video is unchanged, since the audio latents are still denoised jointly with it.

  • New LTX2AutoAudioDecoderStep (ConditionalPipelineBlocks, modular_blocks_ltx2.py): runs LTX2AudioDecoderStep unless decode_audio=False, otherwise skipped. It is the same pattern as LTX2AutoPromptEnhancerStep.
  • It replaces LTX2AudioDecoderStep in LTX2DecoderStep, LTX2ConditionDecoderStep, LTX25DecoderStep and LTX25ConditionDecoderStep, under the same audio_decode name: workflow layouts and the existing workflow tests are unchanged.
  • LTX2AudioDecoderStep itself only gains the decode_audio input declaration, so the input shows up in the resolved workflows' inputs; it has no new branch.
  • Tests: decode_audio=False gives audio=None and bit-identical videos, for LTX-2 text2video and LTX-2.5 in_context.
  • Docstrings regenerated with utils/modular_auto_docstring.py, then doc-builder style --max_len 119.

Fixes #15044

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case. [LTX-2] Modular pipelines: let callers skip the audio decode for video-only outputs聽#15044
  • Did you make sure to update the documentation with your changes? (generated block docstrings; docs/source/en/api/pipelines/ltx2.md covers the non-modular pipelines only)
  • Did you write any new necessary tests?
  • Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Self-review notes

Blocking: none.

Non-blocking, left for review:

  1. Input declared but not read by the leaf. LTX2AudioDecoderStep declares decode_audio but does not read it (decoders.py). get_workflow() resolves conditional blocks down to their leaf, so declaring it there is what makes the input appear in a workflow's inputs. This mirrors enable_prompt_enhancement on the prompt-enhancer leaves. The alternative is overriding inputs on LTX2AutoAudioDecoderStep, but resolved workflows would then not list it.
  2. Selection on a value. select_block keys on the value of a user input (decode_audio is False), not on its presence. The note in ConditionalPipelineBlocks.select_block restricts value-based selection for intermediate outputs only, and LTX2AutoPromptEnhancerStep does the same with enable_prompt_enhancement.
  3. Generated docstrings change. The decoder chains now list audio_latents / audio_num_frames as optional: a conditional block without a default has no required inputs. LTX2AudioDecoderStep still requires them whenever it runs.

Dead code: none.
Verdict: READY.

Who can review?

@yiyixuxu @DN6

`LTX2AutoAudioDecoderStep` wraps `LTX2AudioDecoderStep` in a `ConditionalPipelineBlocks` that runs it unless
`decode_audio=False`, in every LTX-2 and LTX-2.5 decoder chain. A video-only output (e.g. an IC-LoRA alpha matte)
then skips the audio VAE and vocoder and gets `audio=None`; the video is unchanged, since the audio latents are still
denoised jointly with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LTX-2] Modular pipelines: let callers skip the audio decode for video-only outputs

1 participant