pgw#995: env carries values, never a branch selection — wave 2, the BEHAVIOUR axis, and a rig that can see hub env delivery

[PaulFidika]

pgw#990 deleted GEN_WORKER_PREFER_AOT — the flag that gated both the mint recipe and cell discovery, silently disarmed on a release rebuild, and cost three pod attempts before anyone noticed the whole AOT path was dark. This PR is the class, not the instance.

Why the existing census could not have caught it

pgw#929/931's config_reads_allowlist.txt is a real census — 79 classified (file, variable) pairs. But all six of its classifications (VIOLATION / BOOTSTRAP / IPC / LIBRARY / STANDALONE / TRIPWIRE) answer WHERE a read happens relative to the config pipeline. None answers WHETHER it selects behaviour. Those are orthogonal axes, and the postmortem lives entirely on the second one.

The evidence the axis was missing rather than implicit is in the allowlist itself:

• GEN_WORKER_AOT_EXPORT_PARALLEL and _REUSE were classified LIBRARY. Torch has never heard of either name — they are first-party feature gates that happened to be filed next to TORCHINDUCTOR_CACHE_DIR.
• GEN_WORKER_BG_YIELD, _EAGER_FIRST_BOOT, _MINT_IN_PROCESS were classified STANDALONE — "a CLI/convert path that loads no app config" — while all three are read from executor.py on the serving hot path of every production pod.

A classification scheme that lets three serving-path behaviour switches be labelled "standalone CLI" isn't wrong at any single line; it's measuring the wrong quantity.

The census, on the behaviour axis

env	default	fleet declarations	verdict
GEN_WORKER_BG_YIELD	ON	0	DELETED
GEN_WORKER_EAGER_FIRST_BOOT	ON	0	DELETED
GEN_WORKER_AOT_WRAPPER_SPLIT_OFF	ON	0	DELETED
GEN_WORKER_MINT_IN_PROCESS	ON	0	kept — burn-down, blocker named below
GEN_WORKER_AOT_RUN_IMPL_SPLIT_OFF	ON	5 releases, 1 live entry	KEPT — live
GEN_WORKER_AOT_EXPORT_PARALLEL	OFF	0	KEPT — deleting it is a launch
GEN_WORKER_AOT_EXPORT_REUSE	OFF	0	KEPT — same
GEN_WORKER_HOST_MOVE_GUARD	ON	0	KEPT — ruled exception
GEN_WORKER_PROBE / _PUBLISH_ARMED	OFF	0	KEPT — security boundary
PODGUARD_STATE	—	n/a	KEPT — external producer

The three deletions share one property, and it is the test rather than a hope: their default is ON, so deleting the env makes the CURRENT PRODUCTION BEHAVIOUR unconditional. No pod's behaviour changes.

"Delete every kill switch" is wrong in two separate directions, and saying so is part of the finding:

• AOT_RUN_IMPL_SPLIT_OFF sits in the same file as its deleted sibling and looks identical — and it is LIVE: 5 SDXL releases declare it, 1 endpoint carries a non-deleted entry. Zero declarations is a fact you measure, not one you infer from the code.
• AOT_EXPORT_PARALLEL / _REUSE are default-OFF and dark by their own docstrings. Deleting those gates makes an unproven path unconditional — a launch, not a deletion.

Tests stop using env as a red-verification seam

BG_YIELD=0 and EAGER_FIRST_BOOT=0 drove RED arms in four files. An env kept alive to be a test seam is still an env a release rebuild can flip — the mechanism doesn't care about the intent.

Those arms are replaced by absolute-bound assertions plus source-level guards proving the deleted arm is unreachable, not merely unused. That distinction is load-bearing: deleting a switch and deleting its tests looks identical to deleting a switch and leaving a second reader behind — which is exactly how PREFER_AOT kept a live gate after one was believed removed.

The guard (the bar this PR was held to)

scripts/lint_config_reads.py gains an AST pass that flags any env read feeding a conditional and fails the build unless the (file, variable) pair is in BEHAVIOUR_GATES with the threat it defends against. "It is useful" and "it is off by default" are rejected. Stale exemptions fail too — an exemption for a read that no longer exists is the second carrier this gate exists to prevent.

RED-verified — adding a plausible new switch to executor.py:

src/gen_worker/executor.py:11504 reads GEN_WORKER_SOME_NEW_SWITCH from the
environment and feeds it to a CONDITIONAL. Env vars are for CONFIG and SECRETS,
never logic or behaviour switches. GEN_WORKER_PREFER_AOT was exactly this: it
gated the mint recipe and cell discovery, silently disarmed on a release
rebuild, and cost three pod attempts before anyone noticed the AOT path was dark.

The guard also caught my own over-seeding of BEHAVIOUR_GATES — seven entries I added for reads its conservative detector correctly does not flag. Those were removed rather than the check loosened.

The one remaining defect

GEN_WORKER_MINT_IN_PROCESS is not deleted. Default-ON, zero declarations, and enable_compiled(delegate=False) → caller_forced_in_process is a strictly better parameter seam that already exists, making the env redundant with it. What blocks the deletion is ten test sites across seven files that force the shape via the env and then drive the executor, which reaches the policy with delegate=None. Threading the parameter through those call sites is a real refactor of a 4k-line test file and does not belong in a sweep. Recorded in BEHAVIOUR_GATES as a burn-down, not an exception.

Deliverable 2 — the rig could not see hub env delivery

micro_mint_rig.py (pgw#978) runs the whole mint machinery locally and is why this repo can prove a change before PyPI. But it constructs its own environment — mint_process.child_env for the mint child, dict(os.environ) for the adopting process. Neither is a shape a production pod ever has, so this chain was invisible to every test in the repo:

worker function declares env  ->  release_env_declarations
operator sets a value         ->  endpoint_env_entries
pod launch                    ->  Resolve  ->  pod env  ->  Settings

That chain is what took PREFER_AOT dark. The regression class had exactly one detector: a pod.

• tests/harness/hub_env.py models the hub's rule (an entry reaches the pod only if the release declares its name, plus the reserved-namespace defence) and reports withholdings in th#1650's typed vocabulary. Deliberately small — a big fake hub drifts from the real one and starts certifying its own behaviour, which a test asserts against.
• micro_mint_rig.py --hub-env boots the mint child through it, and strips ambient values so a developer's own shell cannot stand in for a hub-delivered one. That substitution is what the blind spot was made of; a mode that allowed it would be decoration.
• tests/test_hub_env_delivery_pgw995.py drives the real config.load_settings() over hub-resolved environments and reproduces the postmortem in milliseconds: release N declares the name and the value reaches Settings; release N+1 is rebuilt without the declaration, nobody touches the entry, and the value is withheld and says so.

Full HelloAck-shaped boot (Vault, applies_to version/tag matching, the mTLS resolve path) is filed on pgw#995 with an owner.

Verification

• lint_config_reads.py OK — 77 pairs, all classified; behaviour axis green and RED-verified
• lint_http_timeouts.py, lint_unreached_surface.py OK
• ruff check src/gen_worker clean; every changed file clean
• Directly affected suites green: test_mint_gate_pgw677, test_mint_reopen_pgw677, test_mint_vram_budget_pgw737, test_eager_first_boot_pgw671, test_aot_mint_unblock_pgw813_pgw815, test_hub_env_delivery_pgw995 (8 new)

Hub half: th#1650 (tensorhub #882) — the silent continue in EndpointEnvService.Resolve that did the disarming.