Skip to content

fix(docs): publish the docs cache through an atomic link swap - #17

Merged
amondnet merged 5 commits into
mainfrom
feat/cache-indirection
Sep 14, 2026
Merged

amondnet merged 5 commits into
mainfrom
feat/cache-indirection

Conversation

@amondnet

@amondnet amondnet commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Closes #14.

The window

unpack moved the verified tree onto <tag> itself. renameSync refuses a populated directory outright, so a publication over an existing tree had to move the old one aside first — and between those two renames the path every caller reads did not exist. A concurrent --no-fetch resolution landing in that window reported the version unavailable while a perfectly good tree was on disk.

The indirection

The bytes now land in a sibling directory named after their verified digest (<tag>.content-<sha12>/), and <tag> becomes a link onto it. Replacing a link is a single atomic rename: a reader arriving mid-publication sees the old tree or the new one, never neither.

docs/boot-3.5.16 -> boot-3.5.16.content-61c6a8ab2d09
docs/boot-3.5.16.content-61c6a8ab2d09/
docs/boot-3.5.16.tag

Keying on the digest also makes a --refresh over unchanged bytes free — the content directory is reused and the link re-points to the name it already had.

The four points the issue asked to decide

  • What path and index report — the link, <tag>, unchanged from before. A caller's stored path stays valid across a refresh, which the resolved target would not; Grep output naming the tag rather than a digest is also the more legible half of that trade.
  • Reclaiming superseded contentsweepLeftovers already reclaimed .staging- and .replaced-; it now covers .content- and .link- on the same TTL. A superseded tree is deliberately not deleted at publication time — a reader that opened it a moment earlier is still inside it. The sweep skips whatever the link currently points at, so a cache that is never refreshed does not delete itself an hour after it was filled.
  • Windows — a junction, not a directory symlink: it needs neither Developer Mode nor elevation. A junction resolves only against an absolute path, so the link is written absolute there and relative everywhere else (a relative link keeps the cache tree movable).
  • No link at all — where neither form can be created, publish falls back to moving the tree into place exactly as before. That reopens the window this change closes, which is the right trade: correctness over atomicity.

A cache written before this change is a real directory at <tag>; rename will not put a link over one, so that one publication still moves it aside, and every later one is the atomic swap.

Verification

  • bun run typecheck, bun run lint, bun run coverage:check, bun run build:skill:check — all pass
  • 217 tests pass; three added for the new behavior (the link layout and what path reports, re-pointing + reclaim on refresh, converting a pre-indirection cache)
  • End-to-end against the live boot-3.5.16 release with the committed .mjs bundle under plain node: first resolve publishes the link, --no-fetch reads _index.md through it, --refresh over identical bytes reuses the content directory and leaves no debris

Summary by cubic

Closes #14. Replaces the docs cache's gap-prone two-step publication over <tag> with an atomic link swap, so concurrent readers see either the old or new content instead of an unavailable path.

Bug Fixes

  • path and index continue to return <tag>, keeping stored paths valid across refreshes.
  • Each publication gets its own random-suffixed content directory, so concurrent publishers of the same digest no longer race over one name.
  • Superseded trees are retired the moment they are moved aside and swept after an hour; every leftover is aged by the entry's own time, not extraction time or a link's target.
  • Windows uses junctions and moves the old entry aside first; link placement is re-attempted only while losing a race, with a bounded budget, before falling back to the direct tree swap, which restores the displaced entry on failure.
  • Existing caches with a real directory at <tag> convert to the new layout on their next publication.
  • Adds coverage for atomic publication, refresh cleanup, stale links, and legacy cache conversion.

Written for commit 54755bb. Summary will update on new commits.

Unpacking moved the verified tree onto `<tag>` itself. `rename` refuses a
populated directory, so publishing had to move the previous tree aside first,
and between those two renames the path callers read did not exist — a
concurrent `--no-fetch` resolution landing in that window reported the version
unavailable while a good tree was on disk.

The bytes now land in a sibling directory keyed by their verified digest and
`<tag>` becomes a link onto it. Replacing a link is one atomic rename: a reader
sees the old tree or the new one, never a gap. `path` and `index` keep naming
`<tag>`, so a caller's stored path survives a refresh.

Windows gets a junction rather than a directory symlink — it needs neither
Developer Mode nor elevation. Where neither form can be created the tree is
moved into place as before, trading the window back for correctness.

Superseded content directories are left in place for readers still inside them
and reclaimed by the existing leftover sweep an hour later; a cache written
before the indirection is converted on its next publication.

Closes #14
@codacy-production

codacy-production Bot commented Sep 14, 2026

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 3 critical · 33 high

Alerts:
⚠ 36 issues (≤ 0 issues of at least minor severity)

Results:
36 new issues

Category Results
ErrorProne 1 high
Security 3 critical
32 high

View in Codacy

🟢 Metrics 0 complexity · 10 duplication

Metric Results
Complexity 0
Duplication 10

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@greptile-apps

greptile-apps Bot commented Sep 14, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 3/5

This PR is not yet safe to merge because concurrent first publication can spuriously fail and a failed legacy-directory conversion can remove the valid public cache path.

Fix All in Claude CodeFindings

  1. P1 Concurrent Publication Can Fail
  2. P1 Failed Swap Removes Cache
  3. P2 Dangling Links Evade Cleanup
Fix with agent prompt
### Issue 1
scripts/docs.ts:224-227
When two initial resolutions of the same archive run concurrently, both can see the digest-keyed content directory as absent. After one resolution renames its extracted tree into place, the other tries to rename onto the now-populated directory and fails, causing that caller to report the version unavailable even though the verified cache was published. Recheck for and reuse a usable tree when this rename loses the race. The bundled implementation in `skills/spring-docs/scripts/docs.mjs` has the same issue.

### Issue 2
scripts/docs.ts:266-277
When converting a legacy or fallback directory, this code first moves the valid target to `displaced`. If the staged-link rename then fails because of a concurrent publisher or filesystem error, the catch discards only the staged link and never restores the displaced tree. This leaves the public cache path missing until another download succeeds. Apply the rollback behavior already used by `swapOnto`. The bundled implementation in `skills/spring-docs/scripts/docs.mjs` has the same issue.

### Issue 3
scripts/docs.ts:363-370
A process killed after creating a staged `.link-*` entry can leave that symlink behind. The sweeper uses `statSync`, which follows the link, so once its content target is absent the stat throws and the swallowed exception prevents `rmSync` from reclaiming the entry. Using link metadata for aging these entries, or removing dangling links when stat fails, would prevent abandoned UUID-named links from accumulating. The bundled implementation in `skills/spring-docs/scripts/docs.mjs` has the same issue.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

  • Preserves stable tag paths while allowing link-based publication and content reuse.
  • Adds cleanup for superseded content and staged links.
  • Adds coverage for link layout, refresh reclamation, and legacy-cache conversion.
  • Updates the committed runtime bundle and documents the new layout.
  • Concurrent initial publication and failed legacy conversion still contain cache-availability failures that should be addressed before merge.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Download archive] --> B[Verify SHA-256]
  B --> C[Extract into staging directory]
  C --> D{Digest content directory usable?}
  D -- No --> E[Rename extracted tree to tag.content-sha12]
  D -- Yes --> F[Reuse existing content tree]
  E --> G[Create staged tag.link-UUID]
  F --> G
  G --> H{Tag is a real directory?}
  H -- Yes --> I[Move legacy tree aside]
  H -- No --> J[Atomically rename staged link onto tag]
  I --> J
  J --> K[Readers access stable tag path]
  K --> L[Sweep expired non-live content and leftovers]
Loading

Reviews (1) · Last reviewed commit: "fix(docs): publish the docs cache throug..."

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces atomic publishing of unpacked documentation trees by utilizing symbolic links (or junctions on Windows) pointing to content directories named after their SHA-256 digests. This ensures readers never encounter a missing or partially written directory during updates. The feedback highlights a high-severity robustness issue in both the TypeScript and JavaScript implementations of linkOnto: if the link swap fails, the pre-existing directory moved to displaced is never restored, leaving the cache in a broken state. It is recommended to wrap the swap in a nested try-catch block to safely restore the displaced directory on failure.

Comment thread scripts/docs.ts
Comment thread skills/spring-docs/scripts/docs.mjs
Comment thread scripts/docs.ts Outdated
Comment thread scripts/docs.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread scripts/docs.ts Outdated
Comment thread scripts/docs.ts Outdated
Comment thread scripts/docs.ts
Comment thread skills/spring-docs/scripts/docs.mjs
Comment thread scripts/__tests__/docs.test.ts Outdated
Comment thread scripts/docs.ts Outdated
Comment thread scripts/docs.ts Outdated
…ndows

Five defects found reviewing the indirection.

Content directories were named by digest alone, so every publisher of the same
bytes contended for one name: the slower of two would find it unusable, delete
the tree the faster one had just linked, and leave a dangling cache path if it
then died. Each publication now gets its own directory — the digest stays as
provenance, a random suffix makes the name private — which also removes the
check-then-delete that made the race possible.

The sweep aged a superseded tree by its extraction time, so a tree that had
been serving for over an hour was reclaimed by the very next run, giving a
reader that entered it just before the swap none of the grace the swap exists
to provide. The swap now re-stamps it, and its hour runs from retirement.

The sweep also read a staged link's age through the link, which is the content
directory's age: a link created moments ago could be reclaimed out from under
the publication in flight, and a dangling one could not be aged at all and
leaked forever. It uses `lstat`.

On Windows `rename` cannot replace a directory, and a junction is one, so
linking over an existing junction fails — and `linkOnto` threw rather than
falling back, turning every refresh of an already-linked cache into
`unavailable`. Windows now moves the old entry aside first, and any failure to
put the link in place returns false to the direct-tree fallback instead of
throwing.

Finally, `linkOnto` deleted only its staged link when the second rename failed,
leaving the cache path missing while the previous tree sat under `.replaced-`.
It restores it, the way `swapOnto` already did.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files (changes from recent commits).

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Re-trigger cubic

Comment thread scripts/docs.ts
Comment thread skills/spring-docs/scripts/docs.mjs
Comment thread scripts/docs.ts
Comment thread scripts/__tests__/docs.test.ts Outdated
Retire superseded trees instead of deleting them. `linkOnto` and `swapOnto`
both deleted the tree they displaced the moment the new one landed, which is
the same cut-out-from-under-a-reader the content-directory grace period exists
to prevent; both now hand it to the sweep. `retire` stamps with `lutimes`, the
side of the link the sweep reads it back from — `swapOnto` can hand it a link
rather than a tree, and following that would age the wrong thing.

Stamp the superseded tree before the swap rather than after, so a concurrent
sweep cannot reclaim an aged tree in the gap between the two.

Retry `linkOnto` once before falling back. Losing the link race is ordinary
rather than exotic on Windows, where every publication moves the old entry
aside, and the loser would otherwise replace the winner's junction with a plain
directory.

`swapOnto` detects an occupied target with `entryExists` rather than
`existsSync`: after a link publication the target can be a link whose tree is
gone, and `existsSync` follows it and reports the path free while `rename`
still has the entry to contend with.

Tests: compare the link as a basename, since a junction stores an absolute
path, and create link fixtures as junctions on Windows, where the symlink
`symlinkSync` defaults to needs Developer Mode.
@amondnet

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an atomic publishing mechanism for documentation caching by utilizing symbolic links (or Windows junctions) to point to unique content directories, preventing temporary path gaps during updates. Superseded directories are retired and cleaned up after a TTL. Feedback was provided regarding the retry logic in publish, where a duplicate expression in a conditional statement (!linkOnto && !linkOnto) causes cognitive overload and should be refactored for clarity.

Comment thread scripts/docs.ts Outdated
`!linkOnto(target, name) && !linkOnto(target, name)` reads as a copy-paste
error and is the shape static analysis flags as a duplicate condition. An
explicit bounded loop says the same thing once.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/docs.ts">

<violation number="1" location="scripts/docs.ts:263">
P2: With three or more Windows refreshes overlapping, both link attempts can lose the target race and fall back to `swapOnto`, replacing the winner's junction with a plain directory and reopening the missing-path window this change is meant to close. Coordinate publication or retry until the link operation succeeds (while separately handling a genuinely unsupported link), rather than using a fixed two-attempt limit.</violation>
</file>

<file name="skills/spring-docs/scripts/docs.mjs">

<violation number="1" location="skills/spring-docs/scripts/docs.mjs:204">
P2: When migrating an old real-directory cache entry, a concurrent sweep can delete the `.replaced-*` tree between the first rename and this retirement call. Stamp `displaced` immediately after moving it aside, before publishing the new link, and apply the same ordering in `swapOnto`.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread scripts/docs.ts Outdated
Comment thread skills/spring-docs/scripts/docs.mjs Outdated
`linkOnto` reported one boolean for two opposite situations. Losing a race is
worth another attempt; a platform that cannot create a link at all fails
identically however many times it is asked, and spending attempts on it was the
reason the attempt count had to stay at two. It now reports which happened, so
`publish` re-attempts only contention — and can afford a real budget instead of
a single spare try. Bounded rather than unbounded: a rename can also fail for
reasons no number of attempts fixes, and a loop that waits for success turns
those into a hang instead of a fallback.

`displaced` is stamped the moment it is moved aside rather than once the new
link lands. Until it is stamped it carries the mtime it was published with, and
a tree that had been serving for days is already past the sweep's cutoff — so a
concurrent sweep could reclaim it in that gap, which is the cut-out-from-under-
a-reader this grace period exists to prevent. Same ordering in `swapOnto`.

Drop the non-null assertion from the new test.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
7.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@amondnet
amondnet merged commit 18ce3c1 into main Sep 14, 2026
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs cache: close the publication window with a stable indirection

1 participant