Skip to content

LTX-2: decouple the RoPE time base from the playback frame rate (Slow-Motion-Control LoRA) #14980

Description

@christopher5106

What
Lightricks/LTX-2.5-22b-LoRA-Slow-Motion-Control expects the diffusion stage to be conditioned on fps / speed while the video keeps its real frame rate. Lightricks' distilled_speed_demo.py does exactly that. Without it, the LoRA gives no speed control.

Why frame_rate alone doesn't cover it
In the LTX-2 pipelines and modular blocks, frame_rate also sizes the generated audio (num_frames / frame_rate) and converts the duration head's seconds into frames. Setting it to fps / speed therefore shortens the audio and inflates the predicted frame count. On the tiny test pipeline, frame_rate=120 gives 9 audio tokens instead of 43, and 121 frames instead of 25 when num_frames is omitted.

Proposal
An optional conditioning_frame_rate (default: frame_rate) used only for the positional time axis: video, keyframe and reference coords, plus the transformer fps. Audio sizing, the duration head and the output keep following frame_rate. This is the split the DFR pipelines already make internally with conditioning_fps.

Implementation proposed in #14967. Does this scope (classic pipelines + modular blocks) look right to you?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions