pgw#1006: name the autotune overlay — the shared autotune cache is a 3% lever over bytes that are already not reproducible - #528
Merged
Conversation
… filed for is a 3% lever over non-reproducible bytes pgw#1006 was filed on the premise that a large share of the 92-min inductor_compile phase is Triton autotuning. The repo's own banked w8a8 SDXL UNet entry (L4 sm_89, aoti_compile_s=390.3) says 3.1%: CachingAutotuner.benchmark_all_configs = 12.1s over 96 calls, 12.3% of GraphLowering.codegen. At attempt 26's 36-entry scale that is ~5.5 min of a 2h43m mint, and a seeded rig run recovers ~95% of it (1.61s -> 0.081s). That number was reachable only by reading metrics_raw by hand. autotune_s is now a named OVERLAY in both ledgers -- aot_compile_spans.OVERLAY_KEYS for the pooled path, aot_mint._phase_delta for the serial one. An overlay, never a partition member: autotune_at_compile_time resolves True for AOTI, so the autotune block runs inside GraphLowering.codegen and summing them double-counts. The finding that outlives the cache idea: two independent COLD compiles of one graph -- same box, same card, same torch, no cache anywhere -- chose different configs for one kernel, and the choice is baked into the generated wrapper (grid_0 block size, the launchKernel num_warps argument, the embedded cubin). Cell bytes are therefore already not reproducible across mints. Seeding only the prior run's *.best_config reproduced the first choice exactly, 12/12. A shared autotune cache would PIN that hazard, not create it -- handed to the determinism-confirmation design (th#1657) before anything is wired.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
pgw#1006 PHASE 1: measure autotune's real share, and answer the issue's hard acceptance gate (byte-identity) before anything is wired. Both are answered, and both say do not build the cache.
The share — measured, not projected
The repo already banked the number, unnamed. On a real w8a8 SDXL UNet AOTI entry (L4 sm_89,
aoti_compile_s = 390.3,archive/AOT-COMPILE-SPEED-RESEARCH-TORCH.md§0.1):AotCodeCompiler.compile(g++)GraphLowering.codegenCachingAutotuner.benchmark_all_configs(96 calls)async_compile.wait(all Triton kernel compile)Autotune is 12.3% of codegen, 3.1% of an entry. At attempt 26's scale (36 entries, 2h43m, codegen 2674s) that is ~5.5 min of a 163-min mint. A local rig run measured the recoverable fraction: seeding a fresh compile with only the prior run's
*.best_configtook autotune from 1.61s to 0.081s — ~95%. So the ceiling of a perfect fleet-wide autotune cache is ~3% of a mint, against pgw#809/#1000's multiples.The premise in the issue ("a large share of the 92-min
inductor_compileis Triton autotuning") is measurably false.The hard gate — byte-identity FAILS, and not because of the cache
Two independent cold AOTI compiles of one graph — same box, same card, same torch, no cache anywhere — picked different configs for one kernel (
XBLOCK 1024/num_warps 4vs512/8). The choice is baked into the generated wrapper:…plus a different embedded cubin. A third run seeded with only the first run's
*.best_configreproduced the first run's choice exactly (12/12 configs on the second vehicle).So a shared autotune cache does not introduce the hazard — the hazard is already live, and a cache would pin it. That is the determinism-confirmation design's question (th#1657), and it is surfaced there before any enablement, per the issue's own instruction.
What this PR actually changes
Instrumentation only. No cache, no storage wiring, no new config, no env var (
lint_config_readsgreen).autotune_sbecomes a named overlay inaot_compile_spans.OVERLAY_KEYS(pooled path) and inaot_mint._phase_delta(serial path). An overlay, never a partition member:autotune_at_compile_timeresolves True for AOTI, so the autotune block runs insideGraphLowering.codegenand summing the two double-counts.tests/test_autotune_span_pgw1006.pypins the overlay, the non-double-count invariant, the fact that neither autotune key can be absorbed by the substring rule that buildstriton_s, and that an absent autotune is omitted rather than reported as0.0.Previously the number was reachable only by reading
metrics_rawby hand — which is exactly how a 3% term got filed as a lane.Tests
tests/test_autotune_span_pgw1006.py(4 passed) plus the adjacent attribution suitestest_compile_attribution_pgw830.py,test_aot_multigraph_pgw758.py,test_compile_duration_th1322.py— 36 passed.Follow-ups this hands off
aot_export_parallel's stated gate is "a worker's artifact must be byte-identical to the parent's for the same row". That gate can fail nondeterministically for this reason, independent of export parallelism.