Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ docs/**/modelzoo.md
*.mp4
*.DS_Store
*.png
!tests/assets/
!tests/assets/ltx25/
!tests/assets/ltx25/official_guitar_man.png
*.csv
*.json
*.jpg
Expand Down
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ When adding or porting a pipeline:

- Select the closest maintained pipeline, public example, and tests as structural baselines. Read the relevant
adding-new-example, adding-new-model, adding-new-stage, model-loading, configuration, and service guides.
- Base each new model-family example README on `examples/README_TEMPLATE.md`; keep its required section order and
remove inapplicable optional sections and all placeholders.
- Inventory model-specific classes and configuration fields, then map them to upstream behavior and the selected
baseline.
- Reuse `BasePipeline`, `BaseStage`, `ModuleManager`, existing configuration dataclasses, example contracts, and
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ runtime path, supported workloads, and reproducible real-time gate.

## News 📰

- ✨ **2026-08-19**: Added [**LTX-2.5 Distilled**](examples/ltx25_distilled/README.md) T2V and I2V joint
audio-video generation with a ModuleManager-backed six-stage pipeline, selectable dense attention backends, and
Ulysses sequence parallelism on **1, 2, or 4 x H100** GPUs.
- ✨ **2026-08-10**: Added [**SwiftVR**](examples/swiftvr/README.md) causal video restoration with stateful streaming
and a single-GPU H100 example.
- ✨ **2026-08-05**: Added [**MiniMax H3**](examples/minimax_h3/README.md) T2VA, FL2VA, and Ref2VA joint
Expand Down Expand Up @@ -247,6 +250,7 @@ telefuser/
|----------|------|-------|
| `WanVideo` (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | Main video generation family, including async and service examples in [examples/wan_video/README.md](examples/wan_video/README.md) |
| `LTX Video` | I2V + Audio | Unified audio-video generation via [examples/ltx_video/README.md](examples/ltx_video/README.md) |
| `LTX-2.5 Distilled` | T2V, I2V + Audio | ModuleManager-backed six-stage pipeline with 1/2/4-H100 Ulysses SP; see [examples/ltx25_distilled/README.md](examples/ltx25_distilled/README.md) |
| `MiniMax H3` | T2VA, FL2VA, Ref2VA + Audio | Local 768p joint audio-video generation via [examples/minimax_h3/README.md](examples/minimax_h3/README.md) |
| `LongCat-Video` | T2V, I2V, VC | Long-form generation and continuation via [examples/longcat_video/README.md](examples/longcat_video/README.md) |
| `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | Dense/MoE generation with native CFG/SP and an in-memory base-to-refiner path; see [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |
Expand Down
4 changes: 4 additions & 0 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ TeleFuser 是一个开源的多模态生成与世界模型流式推理和服务

## News 📰

- ✨ **2026-08-19**:新增 [**LTX-2.5 Distilled**](examples/ltx25_distilled/README.md) T2V 和 I2V 联合
音视频生成,采用基于 ModuleManager 的六阶段 Pipeline,支持选择密集注意力后端,并可在
**1、2 或 4 张 H100** 上使用 Ulysses 序列并行。
- ✨ **2026-08-10**:新增 [**SwiftVR**](examples/swiftvr/README.md) 因果视频修复、有状态流式推理及单卡 H100 示例。
- ✨ **2026-08-05**:新增 [**MiniMax H3**](examples/minimax_h3/README.md) T2VA、FL2VA 和 Ref2VA 联合
音视频生成,并支持标准 `telefuser serve` 服务模式。在相同的 768p、5 秒、50 步 T2VA 请求和一次预热
Expand Down Expand Up @@ -233,6 +236,7 @@ telefuser/
|----------|------|------|
| `WanVideo` (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | 主力视频生成家族,含异步和服务示例,见 [examples/wan_video/README.md](examples/wan_video/README.md) |
| `LTX Video` | I2V + Audio | 统一音视频生成,见 [examples/ltx_video/README.md](examples/ltx_video/README.md) |
| `LTX-2.5 Distilled` | T2V, I2V + Audio | 基于 ModuleManager 的六阶段 Pipeline,支持 1/2/4 张 H100 的 Ulysses SP,见 [examples/ltx25_distilled/README.md](examples/ltx25_distilled/README.md) |
| `MiniMax H3` | T2VA, FL2VA, Ref2VA + Audio | 本地 768p 联合音视频生成,见 [examples/minimax_h3/README.md](examples/minimax_h3/README.md) |
| `LongCat-Video` | T2V, I2V, VC | 长视频生成与续写,见 [examples/longcat_video/README.md](examples/longcat_video/README.md) |
| **NEW** `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | 支持原生 CFG/SP 的 Dense/MoE 生成与内存直传 base-to-refiner,见 [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |
Expand Down
4 changes: 4 additions & 0 deletions docs/en/adding_new_example.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ Pipeline examples are standalone Python scripts that demonstrate how to use Tele
3. Compatible with the TeleFuser server (`telefuser serve`)
4. Well-documented with clear naming conventions

Each model-family directory must include a `README.md` based on
[`examples/README_TEMPLATE.md`](../../examples/README_TEMPLATE.md). Keep the required section order, remove unused
optional sections, and replace all template placeholders before submitting the example.

## File Structure and Naming

### Directory Organization
Expand Down
1 change: 1 addition & 0 deletions docs/en/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ Reusable stages, model configs, schedulers, and pipeline orchestration.
|-------|-------|-------------|
| WanVideo (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | Video generation and editing |
| LTX Video | I2V + Audio | Video generation with audio |
| LTX-2.5 Distilled | T2V, I2V + Audio | ModuleManager-backed six-stage pipeline with 1/2/4-H100 Ulysses SP; see the [example guide](../../examples/ltx25_distilled/README.md) |
| MiniMax H3 | T2VA, FL2VA, Ref2VA + Audio | Local 768p joint audio-video generation |
| FlashVSR | VSR | Video super-resolution |
| SwiftVR | Causal video restoration | Stateful restoration with BF16, compile, FP8Linear, Ulysses SP, and stage-parallel options; see the [example guide](../../examples/swiftvr/README.md) |
Expand Down
1 change: 1 addition & 0 deletions docs/zh/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ AdaTaylorCache 和运行时缓存控制,面向重复生成工作负载。
|------|------|------|
| WanVideo (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | 视频生成和编辑 |
| LTX Video | I2V + Audio | 视频生成 + 音频 |
| LTX-2.5 Distilled | T2V、I2V + Audio | 基于 ModuleManager 的六阶段流水线,支持 1/2/4 张 H100 的 Ulysses SP,见[示例文档](../../examples/ltx25_distilled/README.md) |
| MiniMax H3 | T2VA, FL2VA, Ref2VA + Audio | 本地 768p 音视频联合生成 |
| FlashVSR | VSR | 视频超分辨率 |
| SwiftVR | 因果视频修复 | 支持 BF16、torch.compile、FP8Linear、Ulysses SP 和 stage-parallel,见 [示例文档](../../examples/swiftvr/README.md) |
Expand Down
192 changes: 192 additions & 0 deletions examples/README_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
<!--
Copy this file to examples/{example_directory}/README.md and replace every
{PLACEHOLDER}. Delete guidance comments and optional sections that do not apply.

Keep the required section order so readers can scan all example READMEs in the
same way. Commands must be runnable from the repository root. Document only
options and behavior that exist in the checked-in example scripts.
-->

# {MODEL_FAMILY} Examples

{ONE_OR_TWO_SENTENCES_DESCRIBING_THE_SUPPORTED_MODELS_TASKS_AND_OUTPUTS}

## Model Source

<!-- Required. Add one row per checkpoint or auxiliary model. Use N/A when a source is unavailable. -->

| Model | HuggingFace | ModelScope | Purpose |
| --- | --- | --- | --- |
| `{MODEL_NAME}` | [{HF_REPOSITORY}]({HF_URL}) | [{MODELSCOPE_REPOSITORY}]({MODELSCOPE_URL}) | {MODEL_PURPOSE} |

## Feature Support

<!--
Required. Keep only relevant rows and add model/version columns when support
differs. Use Supported, Unsupported, Partial, or N/A; explain Partial below the
table. Do not use an ambiguous question mark for unverified support.
-->

| Feature | Support | Notes |
| --- | --- | --- |
| {TASK_OR_FEATURE} | Supported | {CONSTRAINTS_OR_VARIANTS} |
| Multi-GPU inference | {SUPPORT_STATUS} | {PARALLEL_STRATEGY_AND_VALID_DEGREES} |
| LoRA | {SUPPORT_STATUS} | {SUPPORTED_VARIANTS} |
| Quantization | {SUPPORT_STATUS} | {DTYPES_OR_FORMATS} |
| CPU offload | {SUPPORT_STATUS} | {OFFLOAD_MODES} |
| Feature cache | {SUPPORT_STATUS} | {CACHE_IMPLEMENTATION} |
| Server API | {SUPPORT_STATUS} | {SERVE_OR_STREAM_SERVE} |

## Requirements

<!-- Required. State minimum hardware and extra dependencies beyond the normal TeleFuser installation. -->

- GPU: {GPU_MODEL_OR_MINIMUM_VRAM}
- Software: {CUDA_PYTORCH_OR_EXTRA_PACKAGE_REQUIREMENTS}
- Input assets: {REQUIRED_INPUT_FORMATS_OR_NONE}

Install TeleFuser by following the [development setup](../../CONTRIBUTING.md#development-setup). Then install any
example-specific dependencies:

```bash
{INSTALL_COMMANDS_OR_COMMENT_STATING_NO_EXTRA_DEPENDENCIES}
```

## Model Directory

<!-- Required. Show the exact layout expected by the scripts. Omit branches that are downloaded automatically. -->

```text
{MODEL_ROOT}/
|-- {CHECKPOINT_OR_DIRECTORY}
\-- {AUXILIARY_CHECKPOINT_OR_DIRECTORY}
```

Set the model root if the examples use `TF_MODEL_ZOO_PATH`:

```bash
export TF_MODEL_ZOO_PATH=/path/to/model_zoo
```

## Quick Start

<!-- Required. Lead with the smallest representative command that produces an output. -->

```bash
python examples/{example_directory}/{representative_script}.py \
--model_root /path/to/model \
--prompt "{EXAMPLE_PROMPT}" \
--output_path work_dirs/{OUTPUT_FILE}
```

The command writes {OUTPUT_DESCRIPTION} to `work_dirs/{OUTPUT_FILE}`.

## Examples

<!--
Required. Group scripts by task when the directory contains multiple tasks.
Repeat the task and script blocks as needed. Use script names as headings.
-->

### {TASK_NAME}

#### `{script_name.py}`

{ONE_SENTENCE_PURPOSE_AND_WHEN_TO_USE_THIS_SCRIPT}

```bash
# Basic usage
python examples/{example_directory}/{script_name.py} \
--model_root /path/to/model \
{REQUIRED_ARGUMENTS}

# Multi-GPU or another important variant
python examples/{example_directory}/{script_name.py} \
--gpu_num {GPU_COUNT} \
--model_root /path/to/model \
{VARIANT_ARGUMENTS}
```

Key options:

| Option | Default | Description |
| --- | --- | --- |
| `--model_root` | `{DEFAULT_OR_NONE}` | {MODEL_ROOT_DESCRIPTION} |
| `--gpu_num` | `{DEFAULT_GPU_COUNT}` | {GPU_COUNT_CONSTRAINTS} |
| `{OPTION}` | `{DEFAULT}` | {OPTION_DESCRIPTION} |

Key behavior:

- {IMPORTANT_DEFAULT_OR_MODEL_VARIANT}
- {OUTPUT_SHAPE_FORMAT_OR_LOCATION}
- {LIMITATION_OR_RESOURCE_NOTE}

## Configuration

<!--
Optional. Keep this section only when users must understand non-obvious
parallel, cache, scheduler, or runtime rules.
-->

### {CONFIGURATION_TOPIC}

{EXPLAIN_THE_RULE_ITS_DEFAULT_AND_WHEN_TO_CHANGE_IT}

```python
{MINIMAL_CONFIGURATION_SNIPPET}
```

## Serving

<!-- Optional. Keep only when at least one script implements a serve or stream-serve contract. -->

Start the service:

```bash
telefuser {serve_or_stream-serve} examples/{example_directory}/{server_script}.py \
--port {PORT} \
{OTHER_REQUIRED_OPTIONS}
```

See the [service guide](../../docs/en/service.md) or
[stream server guide](../../docs/en/stream_server.md) for API and deployment details.

## Performance

<!--
Optional. Include only measured results. Never commit TBD values. State enough
environment and workload detail for another developer to reproduce the result.
Use the same metric definition for every row.
-->

Measured with {GPU_COUNT_AND_MODEL}, {SOFTWARE_VERSIONS}, and commit `{GIT_REVISION}`. Results use {PRECISION},
{ATTENTION_BACKEND}, and exclude {EXCLUDED_PHASES_OR_NOTHING}.

| Configuration | GPUs | Resolution | Frames | Steps | Time (s) | Peak VRAM (GiB) |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| {CONFIGURATION_NAME} | {GPU_COUNT} | {RESOLUTION} | {FRAME_COUNT} | {STEP_COUNT} | {ELAPSED_TIME} | {PEAK_VRAM} |

Reproduce the measurement:

```bash
{BENCHMARK_COMMAND}
```

## Troubleshooting

<!-- Optional. Include only failures specific to this example; link shared issues to the relevant guide. -->

### {ERROR_OR_SYMPTOM}

{CAUSE_AND_ACTIONABLE_FIX}

```bash
{DIAGNOSTIC_OR_FIX_COMMAND}
```

## Notes

<!-- Optional. Keep model-specific limitations or output semantics that do not fit above. -->

- {MODEL_SPECIFIC_LIMITATION_OR_COMPATIBILITY_NOTE}
- {OUTPUT_OR_QUALITY_NOTE}
9 changes: 9 additions & 0 deletions examples/data/ltx25/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# LTX-2.5 I2V Reference Input

`official_guitar_man.png` is the image referenced by the Lightricks LTX-2 model card's image-to-video example:

https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png

Its SHA-256 is `e31cbbe4822ce07e1548121b436c0db3a067d1d78f2e75ab3e69375377b57274`. The matching official
prompt is `A man with short gray hair plays a red electric guitar.` Formal LTX-2.5 I2V regressions use this source at
frame 0 with strength 1.0.
Binary file added examples/data/ltx25/official_guitar_man.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
31 changes: 31 additions & 0 deletions examples/example_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,37 @@ pipelines:
ppl_config_overrides:
attn_impl: TORCH_SDPA

# ============================================================
# ltx25_distilled - Joint Audio-Video Generation
# ============================================================

ltx25_distilled_t2v_2gpu:
script: ltx25_distilled/ltx25_distilled_t2v_h100.py
gpu_count: 2
output_type: video
timeout_seconds: 3600
height: 1024
width: 1536
use_run_with_file: true
require_audio: true
ppl_config_overrides:
num_frames: 121
frame_rate: 24.0

ltx25_distilled_i2v_2gpu:
script: ltx25_distilled/ltx25_distilled_i2v_h100.py
gpu_count: 2
output_type: video
timeout_seconds: 3600
height: 512
width: 896
input_image_path: examples/data/ltx25/official_guitar_man.png
use_run_with_file: true
require_audio: true
ppl_config_overrides:
num_frames: 121
frame_rate: 24.0

# ============================================================
# minimax_h3 — Joint Audio-Video Generation
# ============================================================
Expand Down
Loading
Loading