ci: fix flaky fast-tier notebook timeout#26
Merged
Conversation
The fast-tier smoke tests intermittently fail when the Frequency Analysis tutorial's brute-force "DFT by correlation" demo (a ~2 min O(n^2) sweep over an 11 kHz audio chord) creeps past the 900 s per-cell cap on a slow/contended shared runner. Raise --nbmake-timeout to 1800 s and cap xdist at -n 2 so the CPU-bound cells aren't oversubscribed. No notebook content changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A first pass also dropped xdist to -n 2, but that cut parallelism and pushed the suite to ~32 min on a slow runner. The 1800 s per-cell timeout is what actually fixes the flaky failure (the brute-force DFT cell finishes well under it), so revert to -n auto to keep wall time reasonable. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Notebook smoke tests (fast tier) job intermittently fails with a per-cell timeout. The culprit is a pre-existing cell in
Tutorials/Signals - Frequency Analysis.ipynb: a brute-force "DFT by correlation" demo that sweeps ~5,500 frequencies (1 Hz step) over an 11 kHz audio chord. Its saved runtime is ~2 min even on a fast machine; on a slow/contended GitHub runner it crept past the old 900 s cap and timed out (PR #25's run).Confirmed flaky: re-running the failed job on un-fixed master passed in 3m24s. The same suite has been observed at ~3 min, ~16 min, and ~32 min purely due to runner variance.
Fix (CI config only — no notebook changes)
--nbmake-timeout=900 → 1800— generous headroom; the ~2 min DFT cell finishes well under 1800 s even on a slow runner. This is the real fix.-n auto(an initial pass tried-n 2, but that cut parallelism and pushed the suite to ~32 min on a slow runner without helping correctness).Optional follow-up
The underlying cell is genuinely slow. It could be made fast and robust by passing a coarser
xcorr_freq_step_sizeto that one demo (the helper already supports it and other cells use it; at a 0–5.5 kHz axis the plot looks identical). Can do separately if wanted.🤖 Generated with Claude Code