Skip to content

Removed layerwise infra from efficient transformers - #1288

Open
abhishek-singh591 wants to merge 7 commits into
quic:mainfrom
abhishek-singh591:remove-layerwise-infra
Open

abhishek-singh591 wants to merge 7 commits into
quic:mainfrom
abhishek-singh591:remove-layerwise-infra

Conversation

@abhishek-singh591

Copy link
Copy Markdown
Contributor

Remove Layerwise Infrastructure

Summary

This change removes the layerwise export and compilation infrastructure from
QEfficient. Standard full-model ONNX export and compilation remain the supported
deployment path.

What Changed

  • Removed layerwise model-windowing, ONNX splitting, merging, prefixing, and
    per-window compilation helpers.
  • Removed layerwise-specific ONNX export patches and cache-probe handling.
  • Removed the layerwise argument from model loading, export, and compile
    call sites.
  • Simplified Qwen3 MoE, Qwen3.5 MoE, and Qwen3-VL MoE wrappers to use the
    standard export and compilation flow.
  • Removed layerwise-only example scripts and test fixtures.
  • Removed the layerwise onnx-ir==0.2.1 setup requirement from
    examples/README.md.

Removed Interfaces

The following implementation modules are removed and must not be imported by
downstream code:

  • QEfficient.transformers.models._layerwise
  • QEfficient.utils.compile_layerwise
  • QEfficient.utils.layerwise_pipeline

Any caller that passes layerwise=True or layerwise=False must remove that
argument. Layerwise-specific compile options, including internal
_layerwise_cache_probe, are no longer supported.

Migration

Use the existing model load, export, and compile APIs without layerwise
arguments. For example:

from QEfficient import QEFFAutoModelForCausalLM

qeff_model = QEFFAutoModelForCausalLM.from_pretrained(model_id)

qpc_path = qeff_model.compile(
    batch_size=1,
    prefill_seq_len=128,
    ctx_len=4096,
    num_cores=16,
)

The standard export path produces deployment artifacts directly and does not
require a layer-window split/merge pipeline.

Removed Examples

  • examples/disagg_serving/qwen3moe_layerwise.py
  • examples/image_text_to_text/models/qwen3_5_moe/qwen3_5_moe_layerwise.py

Existing disaggregated-serving examples remain available; their layerwise
arguments have been removed.

Compatibility Impact

This is a breaking change only for users of the layerwise APIs, helper modules,
or example scripts. Users of the standard QEfficient export and compile APIs do
not need to change their workflow, except to remove any explicit layerwise
arguments.

Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
abhishek-singh591 marked this pull request as draft August 26, 2026 11:32
Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>
@abhishek-singh591
abhishek-singh591 marked this pull request as ready for review August 27, 2026 15:09
@abhishek-singh591

Copy link
Copy Markdown
Contributor Author

CI-Ready

1 similar comment
@ochougul

Copy link
Copy Markdown
Contributor

CI-Ready

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>
@ochougul

Copy link
Copy Markdown
Contributor

CI-Ready

2 similar comments
@ochougul

Copy link
Copy Markdown
Contributor

CI-Ready

@ochougul

Copy link
Copy Markdown
Contributor

CI-Ready

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. To prevent automatic closure in 5 days, remove the stale label or add a comment. You can reopen a closed pull request at any time.

@github-actions github-actions Bot added the Stale label Sep 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants