Skip to content

LTX2 HDR reference preprocessing applies an unintended stride-aligned resize #15037

Open

Description

@Shenrui-Ma

preprocess_reference_video_hdr is documented to normalize frames without resizing before HDR's resize/pad step. However, height=None, width=None still selects stride-aligned default dimensions while do_resize=True.

With stride 32, a 1080×1920 (H×W) reference targeting 1088×1920 is first resized to 1056×1920, compressing its height before padding. Inputs smaller than the stride can hit a zero-sized resize.

This reproduces with no model downloads:

import torch
from diffusers.pipelines.ltx2 import LTX2VideoHDRProcessor

video = torch.rand(1, 3, 45, 61, generator=torch.Generator().manual_seed(0))
processor = LTX2VideoHDRProcessor()
actual = processor.preprocess_reference_video_hdr(video, height=64, width=64)
expected = torch.nn.functional.pad(video * 2 - 1, (0, 3, 0, 19), mode="reflect")
expected = expected.permute(1, 0, 2, 3).unsqueeze(0)
torch.testing.assert_close(actual, expected, rtol=0, atol=1e-6)

The assertion fails on the tested revision.

The HDR helper already skips resizing when the target contains the source, so this example should only normalize and pad. A pipeline test with tiny components also confirms that the discrepancy changes output latents.

I have a local fix that disables this preliminary resize on a copy, leaving the original processor unchanged. Would this scope be suitable for a PR?

Environment: Diffusers 0.41.0.dev0, PyTorch 2.14.1, Transformers 5.19.0, Python 3.11.15; macOS arm64.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions