whisper: opt-in adaptive encoder window for short-form audio (~3x on short clips)

[random1st]

What

Whisper's encoder always processes a fixed 30 s window, so a 4 s utterance pays for 26 s of silence. Measured on M3 Max (Metal, large-v3-turbo Q8_0): encode cost is a flat ~275 ms per clip regardless of length. Since run_whisper_encoder_on_window already rebuilds the graph per window (it takes n_mel_frames and reallocates on T_enc change), the window can be sized to the audio instead. On 2–6 s utterances this cut end-to-end latency ~3× in our deployment (a voice assistant, where short utterances dominate).

Why opt-in, and why short-form only

• Short-form only. We first tried shrinking the window globally — it re-chunks long-form and destroys it (measured: long-form Russian WER 4.31% → 85.34% at a 20 s window), because the seek/stitch path assumes the native window size. This patch touches only the is_short_form sizing; long-form is bit-for-bit unchanged.
• Opt-in. Positional embeddings were trained at 30 s, so a shorter window is a real accuracy trade. It's gated behind TRANSCRIBE_ADAPTIVE_WINDOW, with TRANSCRIBE_WINDOW_MIN_SECS (default 10) and TRANSCRIBE_WINDOW_MARGIN_SECS (default 2) so each deployment can price the trade. With the default 10 s floor we measured no WER regression on a 26-clip ru/en/code-switching corpus (clean + noisy).
• Default behaviour unchanged. Without the env var, the window stays at the model's native size for both forms.

Validation

• Shipped in production via the Rust transcribe-cpp-sys bindings (patched crate) on macOS/Metal for several weeks of daily use.
• Guarded against the window ever exceeding enc_max_source_positions, and against odd frame counts.
• Uses the existing transcribe::env helpers; no new dependencies.

Happy to adjust knobs/naming or move the gate into run_params if you'd prefer a non-env API.

[cjpais]

I'll get to reviewing this, but I'm not confident I will pull it in. I do appreciate the thought that went into this, and it's not a wide sweeping set of changes. But I generally don't like adding ENV flags as they feel like a hack, and for the most part I am okay with some latency penalty at the cost of correctness. Especially when there are so many other models to choose from for lower latency