pgw#1006: name the autotune overlay — the shared autotune cache is a 3% lever over bytes that are already not reproducible

changelog.d/pgw1006.md

@@ -0,0 +1,29 @@
+- **pgw#1006: the compile-time autotune benchmark is a named overlay, and the
+  shared autotune cache it was filed for is measured dead.** pgw#1006 assumed
+  "a large share of the 92-min `inductor_compile` phase is Triton autotuning".
+  The repo's own banked entry says 3.1 %: on a real w8a8 SDXL UNet AOTI compile
+  (L4 sm_89, `aoti_compile_s = 390.3`) `CachingAutotuner.benchmark_all_configs`
+  is **12.1 s over 96 calls** — 12.3 % of `GraphLowering.codegen`, and at
+  attempt 26's 36-entry scale ~5.5 min of a 2 h 43 m mint. A perfect
+  cross-mint cache returns ~95 % of that (rig-measured: 1.61 s -> 0.081 s on a
+  seeded run), so the whole lever is ~3 % of a mint against pgw#809/#1000's
+  multiples.
+- `autotune_s` now rides both ledgers — `aot_compile_spans.OVERLAY_KEYS` for
+  the pooled path and `aot_mint._phase_delta` for the serial one. An OVERLAY,
+  not a partition member: `autotune_at_compile_time` resolves True for AOTI, so
+  the autotune block runs inside `GraphLowering.codegen` and summing the two
+  double-counts. It was previously reachable only by reading `metrics_raw` by
+  hand, which is how a 3 % term got argued about as a large share.
+- **The finding that outlives the cache idea: cell bytes are already not
+  reproducible across mints, and autotune is why.** Two independent COLD
+  compiles of one graph, same box, same card, same torch, no cache anywhere,
+  chose different configs for one kernel (`XBLOCK 1024/num_warps 4` vs
+  `512/8`) — and the selected config is baked into the generated wrapper:
+  `uint32_t grid_0 = ((xnumel + (1024 - 1)) / (1024));` vs `(512 - 1)) / (512)`
+  and the `launchKernel(..., 4, ...)` vs `..., 8, ...` argument, plus a
+  different embedded cubin. Seeding a fresh compile with only the prior run's
+  `*.best_config` reproduced the first run's choice exactly (12/12 configs).
+  So the autotune cache is not a byte-identity hazard being introduced — it is
+  a byte-identity hazard already present, which a cache would PIN rather than
+  create. Handed to the determinism-confirmation design (th#1657) before
+  anything is wired.

src/gen_worker/aot_compile_spans.py

@@ -36,13 +36,22 @@
 
 Overlays vs partition members
 -----------------------------
-``triton_s`` and ``device_lock_wait_s`` are OVERLAYS, not partition members:
-they are known to nest inside the members above them (the triton keys sit
-inside codegen / host compile on a CUDA entry; a device-benchmark flock wait
-happens inside ``aot_compile``). Summing them with the partition was the
-second attribution bug — it inflates "attributed" while leaving the residual
-unexplained. They are reported under their own names so a reader can attribute
-WITHIN a member without double-counting the total.
+``triton_s``, ``autotune_s`` and ``device_lock_wait_s`` are OVERLAYS, not
+partition members: they are known to nest inside the members above them (the
+triton keys sit inside codegen / host compile on a CUDA entry; a
+device-benchmark flock wait happens inside ``aot_compile``). Summing them with
+the partition was the second attribution bug — it inflates "attributed" while
+leaving the residual unexplained. They are reported under their own names so a
+reader can attribute WITHIN a member without double-counting the total.
+
+``autotune_s`` (pgw#1006) is the compile-time Triton autotune benchmark —
+``autotune_at_compile_time`` resolves True for AOTI, so the autotune block runs
+INSIDE ``GraphLowering.codegen``. It is named because it decides two separate
+questions and both were being answered from a residual: how much of a mint a
+shared autotune cache could ever return (measured 3.1 % of a 390 s w8a8 SDXL
+UNet entry — 12.1 s over 96 calls), and whether the selected config, which is
+baked into the generated wrapper's grid expression and ``num_warps``, moved
+between two mints of the same key.
 
 The partition members are the four dynamo keys that are provably disjoint on
 the pin (verified by the containment arithmetic on torch 2.13.0:
@@ -74,8 +83,15 @@
 }
 
 #: Reported, never summed into the partition — see the module docstring.
+#: None of the autotune keys contain "triton"/"async_compile", so ``autotune_s``
+#: and the ``triton_s`` overlay below do not double-count each other.
 OVERLAY_KEYS: Dict[str, Tuple[str, ...]] = {
     "inductor_total_s": ("compile_fx.<locals>.fw_compiler_base",),
+    "autotune_s": (
+        "CachingAutotuner.benchmark_all_configs",
+        "CachingAutotuner.coordinate_descent_tuning",
+        "CachingAutotuner.combo_sequential_autotune",
+    ),
 }
 
 #: The three-level partition, as {total: members}. ``check`` walks this, so
@@ -102,7 +118,7 @@
 #: would delete; ``triton_s`` / ``device_lock_wait_s`` nest inside
 #: ``compile_wall_s``. Anything listed here must be excluded by a reader
 #: summing the table.
-SUBSPANS = ("child_interp_s", "triton_s", "device_lock_wait_s",
+SUBSPANS = ("child_interp_s", "triton_s", "autotune_s", "device_lock_wait_s",
             "inductor_total_s", "parent_stage_s", "parent_spawn_s")
 
 

src/gen_worker/aot_mint.py

@@ -818,6 +818,19 @@ def package_cell(
     ),
 }
 
+#: pgw#1006. NOT a member of ``_PHASE_KEYS`` — these nest INSIDE ``codegen_s``
+#: (``autotune_at_compile_time`` resolves True for AOTI, so the autotune block
+#: runs during codegen), exactly as ``triton_s`` does. Named because it answers
+#: two questions that were being read out of a residual: the whole ceiling of a
+#: shared autotune cache, and whether the selected config moved between two
+#: mints of one key — it is baked into the generated wrapper's grid expression
+#: and ``num_warps``. Same keys as ``aot_compile_spans.OVERLAY_KEYS``.
+_AUTOTUNE_KEYS: Tuple[str, ...] = (
+    "CachingAutotuner.benchmark_all_configs",
+    "CachingAutotuner.coordinate_descent_tuning",
+    "CachingAutotuner.combo_sequential_autotune",
+)
+
 
 def _phase_snapshot() -> Dict[str, float]:
     try:
@@ -835,8 +848,11 @@ def _phase_delta(
 ) -> Dict[str, float]:
     """Named phase seconds spent between two snapshots. ``triton_s`` sums
     every async-compile/triton key so GPU kernel compilation is one
-    labeled number; the remainder of inductor time is NOT invented — the
-    coarse wall clocks around export/compile hold the totals."""
+    labeled number; ``autotune_s`` (pgw#1006) does the same for the
+    compile-time autotune benchmark. Both NEST inside the named phases above
+    them and must not be summed with them. The remainder of inductor time is
+    NOT invented — the coarse wall clocks around export/compile hold the
+    totals."""
     raw = {
         k: round(float(after.get(k, 0.0)) - float(before.get(k, 0.0)), 3)
         for k in set(after) | set(before)
@@ -851,6 +867,9 @@ def _phase_delta(
         if ("async_compile" in k or "triton" in k.lower()) and v > 0), 3)
     if triton:
         out["triton_s"] = triton
+    autotune = round(sum(raw.get(k, 0.0) for k in _AUTOTUNE_KEYS), 3)
+    if autotune > 0:
+        out["autotune_s"] = autotune
     return out
 
 

tests/test_autotune_span_pgw1006.py

@@ -0,0 +1,101 @@
+"""pgw#1006: the compile-time autotune benchmark is a NAMED overlay.
+
+Why this test exists rather than a shared autotune cache. pgw#1006 was filed on
+the premise that "a large share of the 92-min `inductor_compile` phase is Triton
+autotuning". The repo's own banked measurement says otherwise — on a real w8a8
+SDXL UNet AOTI entry (L4 sm_89, `aoti_compile_s = 390.3`),
+`CachingAutotuner.benchmark_all_configs` was **12.1 s over 96 calls, 3.1 %** —
+so the whole ceiling of a perfect cross-mint autotune cache is ~3 % of a mint.
+That number was reachable only by reading `metrics_raw` by hand; nothing named
+it, which is how a 3 % term got argued about as a large share.
+
+It is named now, in both the pooled ledger (`aot_compile_spans`) and the serial
+mint's own table (`aot_mint._phase_delta`), as an OVERLAY: the autotune block
+runs during `GraphLowering.codegen` (`autotune_at_compile_time` resolves True
+for AOTI), so summing it with `codegen_s` double-counts.
+
+The second reason it is named is the byte-identity finding — the selected
+config is baked into the generated wrapper (`grid_0` block size and the
+`launchKernel` `num_warps` argument), so a mint whose autotune picks
+differently emits different bytes for the same key. `autotune_s` moving between
+two mints of one key is the cheapest visible signal that it happened.
+"""
+
+from __future__ import annotations
+
+from typing import Dict
+
+from gen_worker import aot_compile_spans as spans
+from gen_worker import aot_mint
+
+
+AUTOTUNE_METRIC = "CachingAutotuner.benchmark_all_configs"
+
+
+def _snapshot(**values: float) -> Dict[str, float]:
+    return dict(values)
+
+
+def test_autotune_is_an_overlay_in_both_ledgers() -> None:
+    assert "autotune_s" in spans.OVERLAY_KEYS
+    assert AUTOTUNE_METRIC in spans.OVERLAY_KEYS["autotune_s"]
+    assert AUTOTUNE_METRIC in aot_mint._AUTOTUNE_KEYS
+    # An overlay, never a partition member: the autotune block runs inside
+    # codegen, so a reader summing the partition must not see it.
+    assert "autotune_s" not in spans.PARTITION_KEYS
+    assert "autotune_s" not in aot_mint._PHASE_KEYS
+    for members in spans.PARTITIONS.values():
+        assert "autotune_s" not in members
+    assert "autotune_s" in spans.SUBSPANS
+
+
+def test_pooled_ledger_reports_autotune_without_inflating_the_partition() -> None:
+    before = _snapshot()
+    after = _snapshot(**{
+        "GraphLowering.run": 10.0,
+        "GraphLowering.codegen": 100.0,
+        AUTOTUNE_METRIC: 12.1,
+        "AotCodeCompiler.compile": 180.0,
+    })
+    partition, overlays, raw = spans.phase_delta(before, after)
+
+    assert overlays["autotune_s"] == 12.1
+    assert partition["codegen_s"] == 100.0, (
+        "autotune nests inside codegen — naming it must not shrink the member "
+        "it nests in")
+    assert sum(partition.values()) == 290.0, (
+        "the overlay leaked into the partition and double-counted 12.1 s")
+    assert raw[AUTOTUNE_METRIC] == 12.1
+
+
+def test_serial_mint_table_reports_autotune_separately_from_triton() -> None:
+    """The two GPU-touching overlays are different questions.
+
+    `triton_s` is kernel COMPILATION (already fanned out across worker
+    processes, 1.5 % of the banked entry); `autotune_s` is BENCHMARKING on the
+    live card. Neither autotune key contains "triton" or "async_compile", so
+    the two overlays cannot absorb each other — asserted, because the
+    substring rule that builds `triton_s` would silently swallow a future
+    autotune key that happened to be named for triton.
+    """
+    out = aot_mint._phase_delta(_snapshot(), _snapshot(**{
+        "GraphLowering.codegen": 100.0,
+        AUTOTUNE_METRIC: 12.1,
+        "async_compile.wait": 5.8,
+    }))
+    assert out["autotune_s"] == 12.1
+    assert out["triton_s"] == 5.8
+    assert out["codegen_s"] == 100.0
+    for key in aot_mint._AUTOTUNE_KEYS:
+        assert "async_compile" not in key and "triton" not in key.lower()
+
+
+def test_absent_autotune_is_omitted_rather_than_reported_as_zero() -> None:
+    """A CPU entry never autotunes, and a recorded 0.0 would read as a
+    measurement rather than as an absence."""
+    partition, overlays, _ = spans.phase_delta(
+        _snapshot(), _snapshot(**{"GraphLowering.codegen": 3.0}))
+    assert "autotune_s" not in overlays
+    out = aot_mint._phase_delta(
+        _snapshot(), _snapshot(**{"GraphLowering.codegen": 3.0}))
+    assert "autotune_s" not in out