Skip to content

pgw#1006: name the autotune overlay — the shared autotune cache is a 3% lever over bytes that are already not reproducible - #528

Merged
PaulFidika merged 1 commit into
masterfrom
pgw1006-autotune-share
Aug 7, 2026
Merged

pgw#1006: name the autotune overlay — the shared autotune cache is a 3% lever over bytes that are already not reproducible#528
PaulFidika merged 1 commit into
masterfrom
pgw1006-autotune-share

Conversation

@PaulFidika

Copy link
Copy Markdown
Contributor

What this is

pgw#1006 PHASE 1: measure autotune's real share, and answer the issue's hard acceptance gate (byte-identity) before anything is wired. Both are answered, and both say do not build the cache.

The share — measured, not projected

The repo already banked the number, unnamed. On a real w8a8 SDXL UNet AOTI entry (L4 sm_89, aoti_compile_s = 390.3, archive/AOT-COMPILE-SPEED-RESEARCH-TORCH.md §0.1):

term s % of entry
AotCodeCompiler.compile (g++) 180.6 46.3%
GraphLowering.codegen 98.5 25.2%
CachingAutotuner.benchmark_all_configs (96 calls) 12.1 3.1%
async_compile.wait (all Triton kernel compile) 5.8 1.5%

Autotune is 12.3% of codegen, 3.1% of an entry. At attempt 26's scale (36 entries, 2h43m, codegen 2674s) that is ~5.5 min of a 163-min mint. A local rig run measured the recoverable fraction: seeding a fresh compile with only the prior run's *.best_config took autotune from 1.61s to 0.081s — ~95%. So the ceiling of a perfect fleet-wide autotune cache is ~3% of a mint, against pgw#809/#1000's multiples.

The premise in the issue ("a large share of the 92-min inductor_compile is Triton autotuning") is measurably false.

The hard gate — byte-identity FAILS, and not because of the cache

Two independent cold AOTI compiles of one graph — same box, same card, same torch, no cache anywhere — picked different configs for one kernel (XBLOCK 1024/num_warps 4 vs 512/8). The choice is baked into the generated wrapper:

<     uint32_t grid_0 = ((xnumel + (1024 - 1)) / (1024));
>     uint32_t grid_0 = ((xnumel + (512 - 1)) / (512));
<     launchKernel(kernels_.triton_poi_fused_addmm_gelu_view_3, grid_0, grid_1, grid_2, 4, 0, ...);
>     launchKernel(kernels_.triton_poi_fused_addmm_gelu_view_3, grid_0, grid_1, grid_2, 8, 0, ...);

…plus a different embedded cubin. A third run seeded with only the first run's *.best_config reproduced the first run's choice exactly (12/12 configs on the second vehicle).

So a shared autotune cache does not introduce the hazard — the hazard is already live, and a cache would pin it. That is the determinism-confirmation design's question (th#1657), and it is surfaced there before any enablement, per the issue's own instruction.

What this PR actually changes

Instrumentation only. No cache, no storage wiring, no new config, no env var (lint_config_reads green).

  • autotune_s becomes a named overlay in aot_compile_spans.OVERLAY_KEYS (pooled path) and in aot_mint._phase_delta (serial path). An overlay, never a partition member: autotune_at_compile_time resolves True for AOTI, so the autotune block runs inside GraphLowering.codegen and summing the two double-counts.
  • tests/test_autotune_span_pgw1006.py pins the overlay, the non-double-count invariant, the fact that neither autotune key can be absorbed by the substring rule that builds triton_s, and that an absent autotune is omitted rather than reported as 0.0.
  • Changelog entry recording the measurement and the byte-identity finding, so the 3% term does not get re-argued as a large share.

Previously the number was reachable only by reading metrics_raw by hand — which is exactly how a 3% term got filed as a lane.

Tests

tests/test_autotune_span_pgw1006.py (4 passed) plus the adjacent attribution suites test_compile_attribution_pgw830.py, test_aot_multigraph_pgw758.py, test_compile_duration_th1322.py — 36 passed.

Follow-ups this hands off

  • th#1657 (trust/determinism lane): cell bytes are not reproducible across mints today. Autotune config selection is a measured cause. Any design that treats a matching cell key as a claim about bytes needs this fact.
  • pgw#1000 / PR pgw#1000: the export-parallel gate is deleted, and the number it waited for was the wrong number #525: aot_export_parallel's stated gate is "a worker's artifact must be byte-identical to the parent's for the same row". That gate can fail nondeterministically for this reason, independent of export parallelism.

… filed for is a 3% lever over non-reproducible bytes

pgw#1006 was filed on the premise that a large share of the 92-min
inductor_compile phase is Triton autotuning. The repo's own banked w8a8 SDXL
UNet entry (L4 sm_89, aoti_compile_s=390.3) says 3.1%:
CachingAutotuner.benchmark_all_configs = 12.1s over 96 calls, 12.3% of
GraphLowering.codegen. At attempt 26's 36-entry scale that is ~5.5 min of a
2h43m mint, and a seeded rig run recovers ~95% of it (1.61s -> 0.081s).

That number was reachable only by reading metrics_raw by hand. autotune_s is
now a named OVERLAY in both ledgers -- aot_compile_spans.OVERLAY_KEYS for the
pooled path, aot_mint._phase_delta for the serial one. An overlay, never a
partition member: autotune_at_compile_time resolves True for AOTI, so the
autotune block runs inside GraphLowering.codegen and summing them
double-counts.

The finding that outlives the cache idea: two independent COLD compiles of one
graph -- same box, same card, same torch, no cache anywhere -- chose different
configs for one kernel, and the choice is baked into the generated wrapper
(grid_0 block size, the launchKernel num_warps argument, the embedded cubin).
Cell bytes are therefore already not reproducible across mints. Seeding only
the prior run's *.best_config reproduced the first choice exactly, 12/12. A
shared autotune cache would PIN that hazard, not create it -- handed to the
determinism-confirmation design (th#1657) before anything is wired.
@PaulFidika
PaulFidika merged commit 77f2641 into master Aug 7, 2026
2 checks passed
@PaulFidika
PaulFidika deleted the pgw1006-autotune-share branch August 7, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant