Skip to content

feat: migrate content layer to comark-content - #2400

Draft
larbish wants to merge 51 commits into
mainfrom
docs/comark-content-migration
Draft

larbish wants to merge 51 commits into
mainfrom
docs/comark-content-migration

Conversation

@larbish

@larbish larbish commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator

Replaces @nuxt/content with comark-content

https://content.comark.dev

Moves content off a build-time SQLite database onto per-commit instances read from GitHub, served with ISR and kept fresh by a push webhook.

Content instances

8 instances, one repo each, one commit sha each. A push on any repo invalidates only the instances reading it.

Instance Repo Branch Content dir Mounted at Dev override
site nuxt/nuxt.com main content/ / (local dir in dev)
examples nuxt/examples main .docs/ /docs/examples NUXT_EXAMPLES_PATH
docs:3.x nuxt/nuxt 3.x docs/ /docs/3.x NUXT_V3_PATH
docs:4.x nuxt/nuxt 4.x docs/ /docs/4.x NUXT_V4_PATH
docs:5.x nuxt/nuxt main docs/ /docs/5.x NUXT_V5_PATH
cli:3.x nuxt/cli 3.x docs/ /docs/3.x/api/commands NUXT_CLI_PATH
cli:4.x nuxt/cli 3.x docs/ /docs/4.x/api/commands NUXT_CLI_PATH
cli:5.x nuxt/cli main docs/ /docs/5.x/api/commands NUXT_CLI_PATH

Each instance also takes a *_BRANCH env override, so the push webhook can be exercised off a non-production branch (warns if one is active in production).

Rendering and invalidation

ISR replaces prerendering (nuxt.config.ts) — no route is prerendered any more. Content routes render on demand and stay on the CDN until purged, with a 300s TTL as the backstop for a missed webhook delivery.

Push webhook (server/api/revalidate.post.ts) — HMAC-verified, registered on the four content repos. Diffs the pushed commits against the previous manifest and purges exactly what changed:

  • the page, its _payload.json, its /raw twin
  • / and /showcase together since each embeds the other
  • global indexes (llms.txt, llms-full.txt, sitemap.xml, sitemap.md, blog/rss.xml, design.md)

The new commit's search artifacts are warmed in the same pass, before the page purges land.

Cross-region ref invalidation (server/utils/content/{cache,refs}.ts) — the repo+branch+dir → sha pointers live in the Vercel Runtime Cache, which is regional. Each entry is written under a cache tag so a forced refresh calls expireTag() and drops the pointer in every region (a plain rewrite only ever reached the region that handled the webhook).

Build snapshots (modules/snapshot/) — the build parses site, examples, docs:4.x and cli:4.x and writes a snapshot into the function bundle, stamped with the commit it was parsed at.

withSnapshot() hydrates a cold start from it instead of walking the content repository, and at a later commit still supplies every unchanged body. The raw source stays the authority for per-file reads, refresh() and watch(). This is the fix for the cold-MISS regression flagged when this PR opened.

Search

Client-side sqlite-wasm FTS5, running in a worker (nuxt-workers), hydrated from each instance's immutable per-commit artifacts.

contentHub() searches the active docs version, its command reference and the examples over one shared database, so a single query ranks across all three — the "CLI reference and examples are out of search" regression is closed. Targets hydrate independently: a sha rotated mid-session on one instance cannot sink the others.

Other changes

Navigation (server/api/navigation/[version].get.ts) — one version's tree, plus the blog and examples subtrees, with stem stripped. app.vue used to fetch every version's tree and filter it down client-side.

Payload caching (server/middleware/payload-etag.ts) — a weak etag on _payload.json for content routes, so client-side navigation gets a 304 from the edge instead of re-downloading the body. Two temporary patches (nuxt, @nuxt/nitro-server) drop crossorigin="anonymous" from same-origin payload preloads, which otherwise makes the preload unusable.

Tracing (server/plugins/otel.ts) — comark-content's parse, cache and source spans land in Vercel Observability.

Highlighting — rangi instead of shiki and twoslash.

Examples — canonicalized to /docs/examples/** (was served under three versioned URLs), with redirects.

Dependencies — removed @nuxt/content, @nuxtjs/mdc, nuxt-content-twoslash, shiki, twoslash, better-sqlite3; added comark-content, rangi, @sqlite.org/sqlite-wasm, nuxt-workers, @octokit/webhooks-methods, @vercel/otel.

Tests — 12 new unit suites covering instance resolution and sources, the revalidate diff, ref cache tags, snapshot assets, content routes and paths, dotted frontmatter, and the multi-instance search targets.

Metrics

Client bundle

From the bundle-size workflow, b82e8fb2 vs 3918b081:

Base (Brotli) PR (Brotli) Δ
Client JavaScript 2.54 MiB 2.05 MiB -497.7 KiB (-19.1%)
Total client assets 2.90 MiB 2.41 MiB -498.8 KiB (-16.8%)

Build

Rows marked † were measured before the build snapshots landed and need a re-measure.

main this PR
Build wall clock 8m 07s 2m 37s †
Content parsing at build 31.7s (17 collections, 1051 files) 4 snapshots (site, examples, docs:4.x, cli:4.x)
Prerendered routes 2492 in 312s 0
Nitro output 91.5 MB (33.2 MB gzip) 56.1 MB (18.9 MB gzip) †
SQLite dump in server bundle 3.06 MB (2.3 MB gzip) —
Build cache 417 MB 291 MB †

A one-word docs typo used to rebuild all 2492 routes. It now purges the one URL that changed.

Time to publish

main this PR
Docs commit → live 8m build webhook purge (~instant)

Runtime (TTFB, /docs/4.x/getting-started/introduction)

Measured before the build snapshots landed; the cold MISS row is what they target.

main this PR
Warm (HIT) 76–87 ms 94–318 ms †
Cold (MISS) n/a (prerendered) TBD

Open items

  • CONTENT_PARSER_VERSION is bumped by hand when the plugin chain changes.
  • The two payload-preload patches drop once the fix is released upstream.

Before merge

Needs VERCEL_BYPASS_TOKEN (build + runtime), WEBHOOK_SECRET, NUXT_OG_IMAGE_SECRET, and push webhooks on nuxt/nuxt.com, nuxt/nuxt, nuxt/examples and nuxt/cli.

@vercel

vercel Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
nuxt Ready Ready Preview Sep 25, 2026 2:47pm UTC

Request Review

@larbish larbish changed the title docs: migrate to comark-content- #2381 feat: migrate content layer to comark-content Aug 31, 2026
# Conflicts:
#	app/pages/docs/[...slug].vue
#	package.json
#	pnpm-lock.yaml
#	pnpm-workspace.yaml
@socket-security

socket-security Bot commented Aug 31, 2026 •

Copy link
Copy Markdown

@nuxt-com-bundle-report

nuxt-com-bundle-report Bot commented Aug 31, 2026 •

Copy link
Copy Markdown

Production bundle

Comparing b82e8fb2 with cf0ba3f3. Compressed sizes are calculated from the emitted production assets.

Metric Base (Brotli) PR (Brotli) Δ Brotli Δ gzip
Client JavaScript 2.54 MiB 2.05 MiB -497.7 KiB (-19.1%) -643.4 KiB (-19.9%)
Client CSS 30.3 KiB 30.3 KiB +7 B (+0.0%) +69 B (+0.2%)
Other client assets 339.3 KiB 339.3 KiB +1 B (+0.0%) -1 B (-0.0%)
Total client assets 2.90 MiB 2.41 MiB -497.7 KiB (-16.8%) -643.3 KiB (-17.6%)

Largest module increases

Module Base (Brotli) PR (Brotli) Δ Brotli
node_modules/shaders/dist/core/lower-BHIbBZvG.js 0 B 143.4 KiB +143.4 KiB
node_modules/shaders/dist/core/FilmStock-DT3jMQ7O.js 0 B 113.9 KiB +113.9 KiB
node_modules/shaders/dist/core/porters-CMOlMI6m.js 0 B 40.6 KiB +40.6 KiB
node_modules/shaders/dist/core/sdf3d-D8apqgo2.js 0 B 26.7 KiB +26.7 KiB
node_modules/shaders/dist/core/agents-mCXmuOM8.js 0 B 24.4 KiB +24.4 KiB
node_modules/eve/dist/src/chunks/use-eve-agent-1NAM6ftx.js 0 B 17.4 KiB +17.4 KiB
node_modules/shaders/dist/core/patterns-Cc6E2ZZl.js 0 B 16.1 KiB +16.1 KiB
node_modules/shaders/dist/core/warpMaps-EGxKq6h8.js 0 B 15.3 KiB +15.3 KiB
node_modules/shaders/dist/core/Irradiance-BoQdvR3u.js 0 B 13.3 KiB +13.3 KiB
node_modules/shaders/dist/core/blur-EIlP7crr.js 0 B 12.8 KiB +12.8 KiB

Module values come from Nuxt’s analyzer and are attribution estimates. This workflow is currently report-only.

Workflow run

larbish and others added 4 commits September 1, 2026 15:24
# Conflicts:
#	app/pages/changelog.vue
#	app/pages/docs/[...slug].vue
#	nuxt.config.ts
#	package.json
#	pnpm-lock.yaml
#	pnpm-workspace.yaml
#	server/mcp/prompts/deploy/deployment-guide.ts
#	server/mcp/tools/blog/get-blog-post.ts
#	server/mcp/tools/deploy/get-deploy-provider.ts
#	server/mcp/tools/docs/get-documentation-page.ts
#	server/mcp/tools/docs/get-getting-started-guide.ts
#	server/mcp/tools/docs/list-documentation-pages.ts
#	server/routes/design.md.get.ts
#	server/routes/raw/index.md.get.ts
#	server/routes/sitemap.md.get.ts
#	server/routes/sitemap.xml.get.ts
#	server/utils/mcp.ts
#	shared/utils/docs.ts
Comment thread app/components/Search.vue Outdated
:search-status="status"
:fuse="fuse"
:loading="status === 'loading'"
:fuse="{ resultLimit: 25 }"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should keep fuse as a defined object here, why disable token search? 🤔

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't useTokenSearch: true by default?

This branch was successfully deployed

1 active deployment
Preview — cf0ba3f3 Deployed Sep 25, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants