Deferred to v0.2.0. Split out of #154, which already took most of the prize.
What is wrong
Starlight indexes the whole build output. It calls index.addDirectory({ path }) with no filter (@astrojs/starlight/dist/integrations/pagefind.js:15), so Pagefind opens and parses every HTML file in dist/.
Only a page that carries data-pagefind-body reaches the index. The docs pages carry it, and so do /id/work/, /id/system/ and /reg/. /id/ref/ and /id/mapping/ do not: they render through src/layouts/RecordPage.astro rather than StarlightPage, which is what #100 asked for. Pagefind therefore opens 87,021 files to index about 624.
Starlight exposes no glob. Its pagefind schema carries indexWeight, ranking and mergeIndex alone.
What it costs
Measured on the full registry, after #154:
|
before #154 |
after #154 |
| Pagefind |
22.5s |
8.4s |
| files scanned |
259,815 |
87,021 |
| files indexed |
~624 |
~624 |
#154 moved the 172,794 /cite/ redirects out of astro build, so Pagefind no longer sees them. The remaining waste is about 8s of a 77s build. That is why this is deferred rather than done.
How to fix it
- Add
src/integrations/pagefind.ts with an astro:build:done hook. Enumerate dist/**/*.html, drop the id/ref/ and id/mapping/ prefixes, and pass the rest to index.addHTMLFile({ sourcePath, url, content }). Write the bundle to dist/pagefind/, the path Search.astro loads.
- Exclude by prefix, never by an allow-list. A forgotten exclusion costs time. A forgotten inclusion drops a page out of site search without a word.
- Set
pagefind: false in the starlight() options to switch off the full-directory scan (@astrojs/starlight/dist/index.js:102).
The catch
pagefind: false empties virtual:starlight/pagefind-config (vite-virtual-modules.js:85), which is how Starlight feeds its tuned ranking weights to the search UI. The UI would fall back to Pagefind's own defaults, and the result order would change with nothing to show for it.
The fix is to serve that module from the same integration with an enforce: 'pre' Vite plugin, carrying the five values from @astrojs/starlight/dist/schemas/pagefind.js. That override fails silently if Starlight renames the module, so the integration should warn when the module was never requested.
Weigh that cost against 8s before taking this on. Reconsider when the registry grows and the number climbs again.
Deferred to v0.2.0. Split out of #154, which already took most of the prize.
What is wrong
Starlight indexes the whole build output. It calls
index.addDirectory({ path })with no filter (@astrojs/starlight/dist/integrations/pagefind.js:15), so Pagefind opens and parses every HTML file indist/.Only a page that carries
data-pagefind-bodyreaches the index. The docs pages carry it, and so do/id/work/,/id/system/and/reg/./id/ref/and/id/mapping/do not: they render throughsrc/layouts/RecordPage.astrorather thanStarlightPage, which is what #100 asked for. Pagefind therefore opens 87,021 files to index about 624.Starlight exposes no glob. Its
pagefindschema carriesindexWeight,rankingandmergeIndexalone.What it costs
Measured on the full registry, after #154:
#154 moved the 172,794
/cite/redirects out ofastro build, so Pagefind no longer sees them. The remaining waste is about 8s of a 77s build. That is why this is deferred rather than done.How to fix it
src/integrations/pagefind.tswith anastro:build:donehook. Enumeratedist/**/*.html, drop theid/ref/andid/mapping/prefixes, and pass the rest toindex.addHTMLFile({ sourcePath, url, content }). Write the bundle todist/pagefind/, the pathSearch.astroloads.pagefind: falsein thestarlight()options to switch off the full-directory scan (@astrojs/starlight/dist/index.js:102).The catch
pagefind: falseemptiesvirtual:starlight/pagefind-config(vite-virtual-modules.js:85), which is how Starlight feeds its tuned ranking weights to the search UI. The UI would fall back to Pagefind's own defaults, and the result order would change with nothing to show for it.The fix is to serve that module from the same integration with an
enforce: 'pre'Vite plugin, carrying the five values from@astrojs/starlight/dist/schemas/pagefind.js. That override fails silently if Starlight renames the module, so the integration should warn when the module was never requested.Weigh that cost against 8s before taking this on. Reconsider when the registry grows and the number climbs again.