fix(chart): un-vacuum the order-of-ops assertion, and make the mysqlClient pin watchable - #714
Conversation
…lient pin watchable Two Bugbot Mediums on the staging promotion PR #713. 1. ORDER-OF-OPS ASSERTION WAS VACUOUS. It required `kubectl rollout restart` before `kubectl annotate` -- but #569 REMOVED that command, and a sibling notMatchRegex forbids it on a command line. So the only text it could still match was the header comments explaining the old mechanism, and it passed regardless of what order the real commands ran in. Measured on the rendered script rather than argued: with the post-re-image annotates hoisted above the first `kubectl set image` -- the actual regress, header comments left in place -- the OLD assertion still passes (True) and the NEW one fails (False). On the real script the new one passes. So a regress that recorded the digest before re-imaging would have frozen every workload on its old image with a green suite. Now anchored to COMMAND lines and to the mechanism that exists: `set image` -> `rollout status` -> `annotate`. (Two earlier attempts at that mutation were themselves broken -- one hoisted the annotates above the comments too, one popped by stale indices after inserting. Both "proved" the wrong thing. The mutation above is verified to have actually moved the lines before its result is quoted.) 2. mysqlClient PIN WAS NEVER WATCHED. `images.mysqlClient` carried a real digest but no `repository` and an empty `tag`, because both live as template defaults. check-digest-drift.sh pairs a pin with the repo/float in its own block, found neither, and classified it UNWATCHABLE -- which exits non-zero, so the DAILY DRIFT WATCH STAYS RED FOREVER while the pin it exists to compare is never checked. A guard that cannot pass teaches everyone to ignore it. Stated in values and consumed by the template, so the value is real config rather than decoration. Both are what the templates already defaulted to. Verified: * render byte-identical across aks/bm/eks values (only POD_TOKEN_SIGNING_SECRET differs, which helm regenerates every run); mysql image line unchanged * helm lint clean; helm unittest 36/36 including the edited suite * drift watch: BEFORE "UNWATCHABLE: images.mysqlClient is pinned to sha256:f546…" AFTER "ok tracebloc/mysql-client:prod sha256:f546e47fb339…"
…any annotate
The order-of-ops assertion only required SOME `kubectl annotate deployment`
after set-image/rollout-status. The flap-counter-reset annotate
(`${ATTEMPT_KEY}- ${FLAP_KEY}-`) already sits there by design, so moving ONLY
the digest-recording annotate (`$annotate_args`) above the re-image kept the
suite green while the digest was recorded BEFORE the image was applied — the
freeze path this guard exists to catch. Require the `$annotate_args`
continuation line so the ordering is tied to the digest write itself, distinct
from the flap reset. Verified by mutation: the loose pattern stayed green with
the digest annotate moved above set-image; the tightened one fails. Bugbot, client#714.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 27371d4. Configure here.
The version-bump-gate (scripts/chart-version-guard.sh) requires a Chart.yaml version bump when client/templates|values change, since the Helm repo only publishes a NEW version — an unbumped edit ships dark or overwrites a published one. develop is at 1.9.39; bump to 1.9.40. backend#1468 unrelated — this is client#714. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
saqlainsyed007
left a comment
There was a problem hiding this comment.
Approve — verified for correctness.
This PR closes two green-but-vacuous guard gaps from #713, plus the required chart bump. I re-derived each independently against the rendered script and the drift watcher rather than the description:
-
Order-of-ops assertion (
image_refresh_test.yaml) — the new(?ms)^\s+kubectl set image.*^\s+kubectl rollout status.*^\s+kubectl annotate deployment[^\n]*\n\s*\$annotate_argsis valid RE2 and correct on two counts. Anchoring to^\s+kubectlmeans the header comments andlog "...set image..."lines can no longer satisfy it — only real command lines. Requiring the$annotate_argscontinuation line ties the ordering to the digest annotate (script lines 507→508); the flap-reset annotate (${ATTEMPT_KEY}- ${FLAP_KEY}-, lines 497–498) carries different args and can no longer stand in. Sincekubectl set imageexists only in the re-image block, hoisting the digest annotate above it leaves noset imagebefore the sole$annotate_argsoccurrence → no match → the freeze regress fails the suite, as intended. -
mysqlClientwatchability (values.yaml+mysql-deployment.yaml) —check-digest-drift.sh's awk pairs the 4-spacerepository:/tag:leaves with the 4-spacedigest:pin within the block; both were previously empty (UNWATCHABLE, non-zero exit → daily watch red forever). Nowrepository: tracebloc/mysql-client+tag: prodmake the pin watchable, and the:prodtag being a single-arch manifest matches the single-arch pin, so it resolvesokrather than a spuriousDRIFT. The template reads.Values.images.mysqlClient.repository | default "tracebloc/mysql-client"; combined with the explicit values it renders byte-identical to the prior hardcode. Only one mysql-client image ref exists on the branch, so no straggler. -
Chart bump 1.9.39 → 1.9.40 — required by the version-bump-gate for chart-content changes; passes.
All 21 checks green (helm lint, helm unittest, 4× template render, drift/source-of-truth, version gates), Bugbot reports no new issues, and the earlier Bugbot order-of-ops thread was already addressed in 27371d4 and resolved.
Two Bugbot Mediums on the staging promotion PR #713.
1. The order-of-operations assertion was vacuous
It required
kubectl rollout restartbeforekubectl annotate— but #569 removed that command, and a siblingnotMatchRegexforbids it on a command line. The only text it could still match was the header comments explaining the old mechanism, so it passed regardless of what order the real commands ran in.Measured on the rendered script rather than argued. Hoisting the post-re-image annotates above the first
kubectl set image— the actual regress, header comments left in place:A regress that recorded the digest before re-imaging would have frozen every workload on its old image with a green suite.
Now anchored to command lines and to the mechanism that exists:
set image→rollout status→annotate.2. The mysqlClient pin was never watched
images.mysqlClientcarried a real digest but norepositoryand an emptytag— both live as template defaults.check-digest-drift.shpairs a pin with the repo/float in its own block, found neither, and classified itUNWATCHABLE, which exits non-zero.So the daily drift watch stays red forever while the pin it exists to compare is never checked. A guard that cannot pass teaches everyone to ignore it.
Now stated in values and consumed by the template, so the value is real config rather than decoration.
Verified
POD_TOKEN_SIGNING_SECRETdiffers — helm regenerates it every run); mysql image line unchangedhelm lintclean;helm unittest36/36, including the edited suiteNote
Low Risk
Changes are chart version bump, stricter unit tests, and values/template wiring that mirrors existing defaults; no intentional runtime behavior change to workloads or image-refresh script logic in this diff.
Overview
Bumps the client chart to 1.9.40 and fixes two CI/guard gaps that let real regressions ship with a green suite.
The image-refresh helm unittest no longer checks ordering via the removed
kubectl rollout restartpath (which could still match header comments). It now requires command-line order:kubectl set image→kubectl rollout status→ digest recording viakubectl annotatewith$annotate_args, so flap-counter annotates cannot satisfy the test if the digest is written before re-imaging.images.mysqlClientnow sets explicitrepositoryandtag: prodinvalues.yaml, and the mysql deployment template reads those fields socheck-digest-drift.shcan pair the pin with repo/tag instead of marking it UNWATCHABLE. Rendered mysql image stays byte-identical to the previous template defaults.Reviewed by Cursor Bugbot for commit 18956ad. Bugbot is set up for automated code reviews on this repo. Configure here.