Let a keyword that failed on infrastructure come back - #216
Merged
Conversation
…n article The anchored gate shipped and the hourly cron wrote its first 162 keywords across 11 sites. The vendor and junk classes are gone. Reading the actual output found two things the fixtures could not have. **Partial matches leaked.** Eight keywords, all one shape: one generic word of a multi-word subject matched, and a word from the generic commercial vocabulary rescued it. "open standards" matched on "open", "software" did the rest, and logicsrc was queued to write about OBS Studio. "supply chain security" matched on "supply" and admitted "industrial automation supply". "online shopping" matched on "online" and admitted "quickbooks online pricing". So a partial match on a multi-word subject may now only be rescued by an anchor the site itself supplied — never by the defaults, which are generic by construction. A complete match is unaffected, including a single-word subject, where "iptv alternatives" is a real query. **And the cross-product floor was publishing constructions.** That one is mine and it was new. Used as per-subject filler it produced "saving money pricing", "deals platform", "coordination d0rz" and "ai content loop" — on-niche, gate-passing, and not topics anybody searches. bl0ggers' niche is "human-in-the-loop AI publishing", so its derived modifiers are literally "human" and "loop", and crossing a subject with those yields nonsense. The crosses' real value was always upstream: as DataForSEO seeds they are what turns "peptide" into "peptide merchant account", and they still seed every expansion. They are simply no longer published on the strength of being grammatically adjacent to the niche. They remain the floor, but site-level rather than per-subject — only a run that would otherwise insert nothing falls back to them, which keeps "a blog with every upstream down still publishes" without letting constructions pad a healthy run. The result reports usedCrossFloor so a site sitting on that for weeks is visible. All eight leaked keywords are now regression cases, with the on-niche keywords from the same run that must still pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013cbcXdpLFUgJZuVfCB4Gzv
Companion to purge-offniche-keywords.ts, for rows written before the gate was tightened. Two verdicts, both computed with the real planner rather than approximated in SQL. The second one needed care. A construction cannot be found by checking for a null search volume, which was the obvious approach: the buyer-journey model also returns keywords with no volume, and those are the best output in the run — "zeek vs suricata for network security monitoring" and "langchain alternatives for agents" both sit in the same null bucket as "deals platform". So a construction is identified by fingerprint against the cross set the planner would build for that site, which is exact rather than heuristic. Ran against the 162 rows from the first production cron: kept 53, removed 95 constructions and 14 partial-match leaks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013cbcXdpLFUgJZuVfCB4Gzv
Publishing stopped on 2026-08-19 and nobody could tell why. All 308 failed
lx_keyword rows carry a null status_reason, because failKeyword() wrote the
reason to console.warn and nothing else — so it lived in the Railway log
buffer and expired there. From the database and the dashboard the outage was
invisible: a number going up, with no cause attached.
Every caller already computes a precise reason ("embedding failed: …",
"quality gate failed after N attempts (slop=…)", "insert failed: …"). It was
simply not being kept. Now it lands on the row, truncated, with a fallback
write so a keyword that cannot record its reason is still marked failed rather
than retried forever.
The quota path gets the same treatment even though it requeues rather than
fails, because it is the one outcome that leaves no trace anywhere: the
keyword looks untouched, no article row is created, nothing reaches the model,
and the dashboard shows a healthy queue while the site quietly stops
publishing. That is the exact shape of this outage.
Diagnosis so far, recorded here because the evidence is easy to lose: article
generation is dying BEFORE the writer model. ai_usage shows zero lx_article
calls since 2026-08-19 while buyer_journey_keyword_research ran today on the
same OpenAI key; credits are not the blocker (3,583 on the account, 20 per
article); and text-embedding-3-small answers normally when called directly.
The next 09:00 run will now say which step it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013cbcXdpLFUgJZuVfCB4Gzv
The 09:00 failures were an OpenAI 429 — "You have no credits remaining" — found in the Railway logs of the deployment that was live at the time. 308 keywords were marked failed for a reason that had nothing to do with their content, and the researched spend behind them was about to be thrown away. REQUEUE=1 emits the update that returns the survivors to the queue instead of the delete that removes the bad rows. Survivors only: of the 308, just 44 pass today's gate, because most predate it. Requeuing all of them would have republished the spam this whole change set exists to stop, so the same gate that decides what may be created decides what may come back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013cbcXdpLFUgJZuVfCB4Gzv
…ality # Conflicts: # scripts/purge-constructed-keywords.ts
ThreatCrush Security Scan40 finding(s) HIGH/CRITICAL: 3 | MEDIUM: 28 | LOW: 9
Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 09:00 failures were an OpenAI 429 — "You have no credits remaining" — found in the Railway logs of the deployment live at the time. 308 keywords were marked failed for a reason that had nothing to do with their content.
REQUEUE=1emits the update that returns the survivors to the queue instead of the delete that removes bad rows. Survivors only: of the 308, just 44 pass today's gate, because most predate it. Requeuing all would have republished the spam this change set exists to stop, so the same gate that decides what may be created decides what may come back.Applied to production: 44 requeued.
🤖 Generated with Claude Code
https://claude.ai/code/session_013cbcXdpLFUgJZuVfCB4Gzv