Skip to content

[LTX-2] Modular pipelines: let callers skip the audio decode for video-only outputs #15044

Open

Description

@christopher5106

Is your feature request related to a problem? Please describe.

The LTX-2 / LTX-2.5 modular pipelines always run LTX2AudioDecoderStep (audio VAE decode + vocoder), even when the caller only wants the video. For some IC-LoRAs, decoding audio makes no sense: Lightricks/LTX-2.5-22b-IC-LoRA-Alpha-Gen turns a reference video into its alpha matte, and the soundtrack decoded alongside it has no meaning; it is thrown away.

Today the only ways to avoid it are:

  • output_type="latent", which also skips the video decode;
  • removing the block (sub_blocks.pop("audio_decode") in each decoder chain), which changes the blocks for every later call. That is not usable when one loaded pipeline serves both video-only and audio-visual requests.

Describe the solution you'd like.

A per-call input, decode_audio (default True): pipe(..., decode_audio=False) skips the audio VAE and vocoder and returns audio=None. The video is unchanged: the audio latents are still denoised jointly with it.

Following the modular guide (a new block rather than a branch inside an existing one), LTX2AudioDecoderStep stays as is and is wrapped in a ConditionalPipelineBlocks, LTX2AutoAudioDecoderStep. That wrapper runs it unless decode_audio=False, the same pattern as LTX2AutoPromptEnhancerStep with enable_prompt_enhancement. It replaces LTX2AudioDecoderStep in the LTX-2 and LTX-2.5 decoder chains under the same audio_decode name, so workflow layouts do not change.

Describe alternatives you've considered.

  • An if not decode_audio: branch inside LTX2AudioDecoderStep. It is simpler, but it is the specializing branch the modular guide advises against.
  • A presence-based AutoPipelineBlocks trigger (skip_audio_decode) selecting a no-op block. Any non-None value would trigger it, False included.
  • A video-only variant blockset without audio_decode. That gives no per-call choice.
  • Skipping audio generation altogether. This is not possible without changing the video: the transformer always takes audio_hidden_states, and the video attends to the audio tokens.

Additional context.

I have a PR ready and will link it here. On the tiny test checkpoint, decode_audio=False returns audio=None and bit-identical videos for text2video and in_context.

Activity

  1. christopher5106 commented on Oct 15, 2026

    @christopher5106
    ContributorAuthor

    PR: #15045

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions