Physics in 2-Steps: Locking Motion Priors Before Visual Refinement Erases Them

[remyx-ai]

What does this PR do?

Implements PhaseLock ("Physics in 2-Steps: Locking Motion Priors Before Visual Refinement Erases Them") as a training-free, plug-in spectral guidance module and wires it into the AnyFlow denoising loop. extract_motion_phase snapshots the FFT phase of an early few-step latent (the physically-consistent motion prior) and apply_phase_lock re-imposes it on later high-fidelity latents via shortest-arc phase nudging, preserving the magnitude spectrum (visual fidelity) bit-for-bit at full strength.

Motivation

Following the 'Add AnyFlow Video Diffusion Pipelines' and 'Refactor AnyFlow for torch.compile and Standalone Masking' work, the natural next step is improving physical consistency of generated video. PhaseLock preserves valid motion priors from early inference steps without retraining or model changes. It operates purely on the per-step latent — exactly the I/O the AnyFlow (and standard diffusers) callback_on_step_end hook reads and writes back (verified at pipeline_anyflow.py:613-623) — so it wires in as a clean addition with no model/trainer dependency, and no existing module implements spectral phase-locking.

Note on availability: no code repository or model URL surfaced in the paper, recommendation envelope, or arXiv abstract page, so no LICENSE could be evaluated (class: no-code-link, compat: 0.30). Worth confirming an open release before broader adoption.

Changes

Call site: AnyFlowPipeline.__call__ in src/diffusers/pipelines/anyflow/pipeline_anyflow.py — the per-step latent is passed through phase_lock(...) after each scheduler.step when the optional phase_lock argument is provided.

• Implements PhaseLock's core spectral intervention as a training-free, plug-in guidance module (extract_motion_phase / apply_phase_lock), re-imposing the early-step motion-prior phase on later latents while preserving the magnitude spectrum.
• Wires the intervention into the real AnyFlow denoising loop via a new optional phase_lock argument on AnyFlowPipeline.__call__, invoked after each scheduler.step — so calling pipe(prompt, phase_lock=PhaseLockGuidance(...)) exercises it on the production path, with the default None path left byte-for-byte unchanged.
• Exposes the paper's tunable knobs: prior_step (the 2-step prior capture point), lock_strength (Latent Delta Guidance fraction), freq_cutoff_ratio (lock only low-frequency motion, leave high-frequency appearance detail untouched), and cutoff_step_ratio (let final steps refine freely).
• Exports PhaseLockGuidance from diffusers.pipelines.anyflow, documents it in the AnyFlow API page with a runnable example, and adds unit + integration tests asserting the spectral guarantees (magnitude preserved, phase steered) and that the guidance measurably alters real pipeline output above the CPU noise floor.

Out of scope

• No physical-consistency benchmark or quantitative evaluation reproducing the paper's '+6.2 points' / '~18% phase erosion' measurements (would require I2V eval datasets and physics-scoring infra the repo doesn't host).
• No image-to-video (I2V) conditioning path — the paper frames PhaseLock for I2V, but it is wired into the existing T2V AnyFlowPipeline.__call__ since the latent-spectrum operation is modality-agnostic and that is the verified call site.
• No training, distillation, or model changes — consistent with the paper being training-free; the prior is captured purely from the live denoising trajectory.
• No automatic enabling/default heuristics or auto-tuning of prior_step/lock_strength; the user must opt in and choose hyperparameters explicitly.

Test plan

⚠️ Tests did not pass. PR opened as draft for review.

../../opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/_pytest/mark/structures.py:404
  /opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/_pytest/mark/structures.py:404: PytestCollectionWarning: cannot collect 'test_load_sharded_checkpoint_device_map_from_hub_subfolder' because it is not a function.
    def __call__(self, *args: object, **kwargs: object):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/models/autoencoders/test_models_autoencoder_oobleck.py
ERROR tests/models/autoencoders/test_models_autoencoder_rae.py
ERROR tests/pipelines/kolors/test_kolors.py - RuntimeError: Failed to import diffusers.pipelines.kolors.tokenizer because of the following error (look up to see its traceback):
No module named 'sentencepiece'
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 3 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
6 warnings, 3 errors in 6.72s

Before submitting

• Did you read the contributor guideline?
• Did you read our philosophy doc?
• Was this discussed/approved via a GitHub issue or the forum?
• Did you make sure to update the documentation?
• Did you write any new necessary tests?

Who can review?

@DN6 @yiyixuxu @sayakpaul