diff --git a/apps/docs/content/docs/dev/content-engine/caching.mdx b/apps/docs/content/docs/dev/content-engine/caching.mdx index 8d838e3ce..ebb105d29 100644 --- a/apps/docs/content/docs/dev/content-engine/caching.mdx +++ b/apps/docs/content/docs/dev/content-engine/caching.mdx @@ -158,10 +158,23 @@ Two things worth stating out loud: free. - **A no-op touches nothing.** Publishing something already published transitioned nothing, so a double-clicked button costs one 200 and no cache. + The same holds for an unpublish, and for a restore that changed nothing - + each of those routes answers with `changed`, and the Server Action reads it. Nothing global is ever expired, and one content type's mutation never touches another's tags. + +An **update** that changed nothing is the exception: the generated `PUT` answers +with the row, not with a `changed` flag, so the Server Action cannot tell a +saved-but-identical edit from a real one and treats it as +`update published, same slug`. That expires three tags +stale-while-revalidate - the responses stay served while they refresh, so the +cost is a refresh nobody needed rather than a cache miss. Widening a public +response contract to save it would be the wrong trade; correctness first, hit +rate second. + + [Restore](/docs/dev/content-engine/revisions#restore) has no rules of its own - it lands on the update rows above, because as far as a visitor is concerned a restore *is* an update. It cannot appear on the publish or unpublish rows at @@ -348,6 +361,150 @@ Nothing falls back to Polish, whatever the fallback setting is - so a Polish edi never throws away the English cache. See [Localized public API](/docs/dev/content-engine/localized-public-api#caching). +## Delivery tags + +A content type with [`delivery`](/docs/dev/content-engine/content-delivery) produces +three more scopes, in the same namespace and with the locale in the same position: + +```ts +import { + contentDeliveryRedirectTag, + contentDeliverySitemapTag, + contentDeliveryTag, +} from "@vitnode/core/content"; + +contentDeliveryTag("example.article", 42); +// "content:example.article:delivery:42" + +contentDeliveryTag("example.article", 42, "pl"); +// "content:example.article:delivery:pl:42" + +contentDeliveryRedirectTag("example.article", "stary-slug", "pl"); +// "content:example.article:redirect:pl:stary-slug" + +contentDeliverySitemapTag("example.article", "pl"); +// "content:example.article:sitemap:pl" +``` + +Each answers a different question a page asked, which is why they are separate from +the three above rather than folded into them: + +| Scope | Keyed by | Holds | +| ---------- | ------------- | -------------------------------------------------- | +| `delivery` | the record | canonical path, alternates, SEO metadata | +| `redirect` | the **slug** | "does this address still resolve here" | +| `sitemap` | the locale | one locale's file, and the index that lists them | + +A `generateMetadata` that renders only metadata is tagged `delivery` alone, so an +unrelated field of the record changing does not throw it away. A redirect lookup is +tagged by the **old** address, because that is what a request for a moved page arrives +with. + +### What expires them + +`contentInvalidationTags` takes an optional `delivery` block and derives everything +from the data it already has - the affected locales and every slug the record answered +to across the mutation: + +```ts +contentInvalidationTags({ + contentTypeId, + delivery: { sitemap: true }, + id, + isPublic, + slugs: [previousSlug, currentSlug], + wasPublic, +}); +``` + +```ts +contentInvalidationTags({ + contentTypeId, + delivery: { sitemap: { contentChanged: true, indexChanged: false } }, + id, + isPublic, + slugs: [previousSlug, currentSlug], + wasPublic, +}); +``` + +The sitemap is **two** decisions rather than one, and the reason is ``. A +sitemap entry carries a `lastModified` derived from `updatedAt`, so a plain title edit +on a published record changes the *bytes* of its sitemap file even though the set of +URLs in it is identical: + +- **`contentChanged`** expires the sitemap **file** of each locale the mutation + reached. True for any real mutation of a record that is or was publicly reachable. +- **`indexChanged`** expires the locale-less tag, which for a localized content type is + the *index* of its per-locale files. True only when public reachability flipped, + because an index lists files and counts URLs. + +| Mutation | delivery | redirect (old + new) | sitemap file | sitemap index | +| ----------------------------------- | -------- | -------------------- | ------------ | ------------- | +| Title / SEO edit (still published) | ✅ | ✅ | ✅ | ❌ | +| Slug change (published) | ✅ | ✅ | ✅ | ❌ | +| Publish / unpublish | ✅ | ✅ | ✅ | ✅ | +| Delete (was published) | ✅ | ✅ | ✅ | ✅ | +| Restore that moves a slug | ✅ | ✅ | ✅ | ❌ | +| Translation publish / unpublish | ✅ | ✅ | ✅ | ✅ | +| Translation create / delete | ✅ | ✅ | ✅ | ✅ | +| No-op edit | ❌ | ❌ | ❌ | ❌ | +| Draft edited into another draft | ❌ | ❌ | ❌ | ❌ | + +The first row is the one worth reading twice. **A real update to a published +representation expires that locale's sitemap file, because its `lastModified` changes - +even when the canonical URL stays the same.** Anything else would leave a cached +sitemap serving a timestamp that is no longer true. + +The last two rows are the other half of the same rule: the engine issues no `UPDATE` +for an update that changed nothing, so `updatedAt` does not move and the cached file is +still byte-correct. A draft is in no sitemap either way. + + + Its locale-less tag *is* its sitemap file, so `contentChanged` is what expires it. + The locale-less tag means "the index" only for a localized content type, whose files + are the per-locale ones. + + +### Which locale's sitemap + +Per locale, reusing the Stage 5 fan-out above rather than a second rule: + +```text +PL translation edit → sitemap:pl +EN translation edit → sitemap:en +shared field edit → sitemap:en AND sitemap:pl (the base `updatedAt` is in both) +``` + +A shared edit reaches every locale because a localized entry's `lastModified` is +`max(base.updatedAt, translation.updatedAt)` - so a new base timestamp becomes the +effective value for every published translation. + +One conservative case is worth naming: with `fallback: "default"`, an edit to the +**default** locale's translation also reaches every locale that has no translation of +its own, because that is where their public pages come from. Those locales contribute +no sitemap URL at all - a sitemap never lists a fallback - so expiring their files is a +cache miss rather than a necessity. Following the Stage 5 fan-out is deliberate: one +locale-propagation rule, not two. + + + Omit `delivery` from the input - which is what every content type without the block + does - and `contentInvalidationTags` returns exactly the strings it always returned, + byte for byte. Nothing existing has to be re-tagged, and no warm cache is thrown + away for a feature the content type does not use. A test asserts the exact lists. + + +### Background mutations + +A [scheduled](/docs/dev/content-engine/scheduling) publish reaches the web app through +the same revalidation bridge, with the delivery tags included - there is no second +cross-origin invalidation system, and the same all-origins-must-accept rule applies. + +The bridge's request schema declares `delivery` explicitly, because an object schema +strips what it does not name: a body that carried delivery tags and arrived without +them would leave a stale sitemap behind every background transition. It stays optional, +so an API that has not been redeployed keeps working. + ## Where the Next imports live Exactly one place: `@vitnode/core/content/next`. diff --git a/apps/docs/content/docs/dev/content-engine/canonical-urls.mdx b/apps/docs/content/docs/dev/content-engine/canonical-urls.mdx new file mode 100644 index 000000000..97da9fae2 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/canonical-urls.mdx @@ -0,0 +1,176 @@ +--- +title: Canonical URLs +description: One helper builds every content URL, it is relative on purpose, and the locale is normalized so one page has one cache key. +icon: Link +--- + +A canonical URL is the one address a page admits to living at. Everything else - +redirects, `hreflang`, sitemaps, cache tags - is defined in terms of it, so the +engine builds it in exactly one place. + +```ts +import { contentDeliveryPath } from "@vitnode/core/content"; + +contentDeliveryPath({ definition: articleContentType, slug: "my-article" }); +// "/articles/my-article" + +contentDeliveryPath({ + definition: advancedArticleContentType, + locale: "pl", + slug: "moj-artykul", +}); +// "/pl/articles/moj-artykul" +``` + +## How a path is built + +```text +nonlocalized /{publicApi.path}/{slug} +localized /{locale}/{publicApi.path}/{slug} +``` + +Nothing is configurable here, and that is the point: a resolver has to be able to +parse back what the builder produced, and a per-content-type URL template would +make that a guess. `publicApi.path` is reused rather than duplicated into +`delivery`, so the public API route and the public page cannot disagree about the +prefix. + +## It is relative + +`contentDeliveryPath` never returns an origin, because a content type definition +lives in source control and gets deployed to a preview domain, a staging domain and +production - so an origin baked into it would be wrong in two of the three places. + +Supply one when you need an absolute URL: + +```ts +import { contentDeliveryUrl } from "@vitnode/core/content"; + +contentDeliveryUrl({ + origin: "https://example.com", + path: "/pl/articles/moj-artykul", +}); +// "https://example.com/pl/articles/moj-artykul" +``` + +`https://example.com` and `https://example.com/` produce the same URL - it resolves +rather than concatenates - and a malformed origin comes back `null` rather than a +link with two schemes in it. + +The delivery service takes the same argument: + +```ts +await delivery.findById(42, { locale: "pl", origin: "https://example.com" }); +// { canonicalPath: "/pl/articles/…", canonicalUrl: "https://example.com/pl/articles/…", … } +``` + +`canonicalUrl` is **absent** rather than `null` when no origin was given, so a +consumer never has to tell "no origin was supplied" from "the URL could not be +built". + + + The sitemap protocol only accepts absolute URLs, so `contentSitemapXml` requires + an origin rather than taking one. See [Sitemaps](/docs/dev/content-engine/sitemaps). + + +## The locale is normalized + +```ts +contentDeliveryPath({ definition, locale: "PL", slug: "witaj" }); +contentDeliveryPath({ definition, locale: "pl", slug: "witaj" }); +contentDeliveryPath({ definition, locale: " pl ", slug: "witaj" }); +// all three: "/pl/articles/witaj" +``` + +Same `normalizeContentLocale` the rest of the engine uses. It matters because a +path is also a cache key: three spellings of one locale producing three paths would +produce three cache entries for one page, and expiring one of them would leave the +other two stale forever. + +Slugs are percent-encoded on the way in. A generated slug is already URL-safe - +[`slugify`](/docs/dev/content-engine/slug-field) guarantees it - but a row written +straight into the database is not, and a *path* is what this function promises. + +## Nulls are deliberate + +`contentDeliveryPath` returns `null` rather than a best effort in three cases: + +- **An empty slug.** A canonical URL that points at the list page is worse than no + canonical URL at all. +- **An empty `publicApi.path`.** The content type has no public API. +- **A localized content type with no locale.** A localized record has one URL per + language and no locale-less one, so guessing would hand a reader the wrong + language under a URL that claims otherwise. + +## Parsing a path back + +```ts +import { parseContentDeliveryPath } from "@vitnode/core/content"; + +parseContentDeliveryPath(articleContentType, "/articles/my-article"); +// { locale: null, slug: "my-article" } + +parseContentDeliveryPath(advancedArticleContentType, "/pl/articles/moj-artykul"); +// { locale: "pl", slug: "moj-artykul" } +``` + +The inverse of the builder, and deliberately strict: it accepts exactly the shape +that function produces and refuses everything else. An extra segment, a different +public prefix, a traversal or a malformed escape is `null` rather than a best guess +- a resolver that guessed would answer one content type's URL with another's +record. + +A query string and a fragment are stripped first, because a browser sends them and +they are not part of the identity of a page. + +`delivery.resolvePath()` is this plus the lookup, and it is what a catch-all route +should call: + +```ts +const resolution = await delivery.resolvePath("/pl/articles/stary-slug"); +``` + +## The canonical URL is the *served* locale + +This is the rule most likely to be got wrong, and it comes straight out of +[Stage 5 fallback](/docs/dev/content-engine/localized-public-api): + +```text +requestedLocale = pl +PL translation missing +fallback EN translation exists +``` + +A public `findById()` may return the English copy. The canonical URL of that +response is the **English** one: + +```ts +{ + requestedLocale: "pl", + locale: "en", + isFallback: true, + canonicalPath: "/en/articles/article", +} +``` + +`/pl/articles/article` would be a self-declared canonical that answers 404 - the +Polish translation does not exist, so nothing serves that URL. Reporting the served +locale is what lets a page render `` correctly *and* show a +"not translated yet" notice. + +`findBySlug` and `resolveSlug` remain strict-locale: a URL belongs to the language +it was published under, so they never fall back at all. + +## Registry helpers + +```ts +import { listDeliveryContentTypes } from "@vitnode/core/content"; + +const delivered = listDeliveryContentTypes(core.contentModels); +``` + +Every delivery-enabled content type of an installation, sorted by id so two +processes building the same sitemap index produce the same document. It is what +lets a site-level `/sitemap.xml` enumerate `blog.article`, `docs.page` and +`shop.category` without hardcoding a single plugin name - installing a plugin adds +its content types and removing it takes them out again. diff --git a/apps/docs/content/docs/dev/content-engine/concurrency.mdx b/apps/docs/content/docs/dev/content-engine/concurrency.mdx new file mode 100644 index 000000000..716d647d9 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/concurrency.mdx @@ -0,0 +1,136 @@ +--- +title: Concurrency +description: Who wins when two writers race, what the loser is told, and why a no-op is never a conflict. +icon: GitFork +--- + +Two editors open the same article. One fixes a typo, the other rewrites the +introduction, and both press save within the same second. Exactly one of them +should win, and the other should be told - clearly enough that the AdminCP can +reload the newer record and offer to overwrite it. + +That is the whole of this page, applied to every mutation the engine has. + +## The mechanism + +Two primitives, and which one applies depends on whether the content type has an +editorial workflow. + +**Optimistic locking**, on an editorial content type. Every write carries the +version the caller read, and the write is a single guarded statement: + +```sql +UPDATE "example_articles" + SET "title" = $1, "version" = "version" + 1 + WHERE "id" = $2 AND "version" = $3 +``` + +There is no read-then-write window, because there is no read. Two racing writers +produce one statement that matches and one that does not; the one that does not +gets a [`CONTENT_VERSION_CONFLICT`](/docs/dev/content-engine/editorial) carrying +both versions. + +**A row lock**, on a content type without one. There is no version column to +guard, so `SELECT ... FOR UPDATE` takes the source row before the collection is +read - which makes two concurrent `add` calls run one after the other, and the +second sees the first's result rather than the state they both started from. + +The lock is the database's, not the process's. A second API instance is +serialised by exactly the same primitive. + +## The matrix + +Every row below is a test on two separate PostgreSQL connections. + +| Race | Outcome | +| ---- | ------- | +| update vs update | One winner, one `CONTENT_VERSION_CONFLICT`. Version moves once, one revision. | +| update vs delete | Delete first: the update answers `null` (a 404). Update first: the delete conflicts. Never a resurrection. | +| update vs publish (same version) | One winner. The loser conflicts. | +| update vs publish (no version) | Both may land. A publish writes `status` only, so no field value is reverted. | +| restore vs update | One winner. A restore never silently overwrites a newer edit. | +| restore vs delete | Delete first: the restore answers `null`. It cannot recreate a removed record. | +| scheduled vs manual publish | The second finds nothing to do and is skipped. One revision, one announcement. | +| PL update vs PL update | One winner, one `CONTENT_TRANSLATION_VERSION_CONFLICT`. | +| PL update vs EN update | **Both win.** Separate version domains, separate rows. | +| PL update vs shared update | **Both win.** The translation and the base row are two rows with two versions. | +| PL delete vs stale PL update | No resurrection, either way round. | +| relation add vs add | One winner. Exactly one junction row, at position 0. | +| relation remove vs add | One winner. Positions stay contiguous from zero. | +| reorder vs add / remove | One real mutation. Positions stay contiguous, targets stay unique. | +| repeatable create vs create | One winner (editorial) or both merged (plain service). | +| child update vs reorder | One winner. Child identity survives either way. | +| child delete vs update | One real mutation, and never a resurrected child. | +| collection vs scalar | One winner. Never one writer's categories under another's version. | +| two different records | **Both win.** The lock is per row. | + +## Two rules that surprise people + +### A no-op is not a conflict + +An editorial write that changes nothing succeeds, bumps no version and leaves no +revision - and it does **not** check `expectedVersion`. There is nothing to +overwrite, so there is nothing to conflict about. An editor who pressed save +twice has not created two versions of anything. + +This reaches further than it looks. `repeatable.update` for a child a concurrent +writer already deleted computes a list identical to the stored one, so it is a +successful no-op rather than a conflict. The invariant worth holding is therefore +**one race, one version increment** - not "one race, one rejected promise". + +### A reorder on an unordered relation does nothing + +`field.relation({ multiple: true })` without `ordered: true` is a *set*. The +engine stores it in ascending target order, so `set([9, 2])` and `set([2, 9])` +are the same state - and `reorder` computes the list that is already stored. + +If a sequence matters, say so: + +```ts +relatedArticles: field.relation({ + multiple: true, + ordered: true, // Now `UNIQUE (itemId, position)` makes the order a fact. + self: true, +}), +``` + +## Why positions never collide + +Rewriting an ordered collection cannot be done in place: moving row A from slot 0 +to slot 1 while row B still sits in slot 1 violates `UNIQUE (itemId, position)` +*during* the statement, even though the final state is fine. + +So every surviving row is first parked at a negative slot - a space no settled +row ever occupies - and one final `UPDATE` maps the whole set back to `0..n-1` at +once. No deferrable constraint, no delete-and-recreate, and every child keeps its +identifier through a reorder. + +## Scheduled transitions + +A booked publication is claimed with `SELECT ... FOR UPDATE`, and the lock is +held from the claim through the transition, its revision, the settlement *and* +the queue row that will announce it. Four conditions are re-read from the +database under that lock rather than trusted from the queue payload: the row +exists, it is still `pending`, its generation matches, and its time has come. + +That is what makes a cancel honest. It either wins outright - before the claim - +or waits, then finds the schedule already `completed` and answers a truthful 404. +There is no window in which an administrator is told the cancel worked and then +watches the article go live anyway. + +A stale booking cannot override newer manual state either, because the transition +guards on the *state* it is leaving: a scheduled publish of an already-published +record changes nothing, writes no revision, and announces nothing. + +## Writing concurrent code against the engine + +Three things worth knowing if you call the services directly: + +1. **Pass `expectedVersion`.** Every editorial write requires it, including a + collection mutation, and that is deliberate - defaulting it would make that the + one write which silently overwrites whatever it finds. +2. **Join the transaction, do not open a second one.** Every service method takes + `{ tx }`. Two transactions on two connections is a deadlock waiting for a + quiet afternoon. +3. **Call the effects after the write returns.** Never inside the transaction + callback - see [failure and retries](/docs/dev/content-engine/failure-and-retries). diff --git a/apps/docs/content/docs/dev/content-engine/content-delivery-limitations.mdx b/apps/docs/content/docs/dev/content-engine/content-delivery-limitations.mdx new file mode 100644 index 000000000..9d49d522c --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-delivery-limitations.mdx @@ -0,0 +1,201 @@ +--- +title: Delivery limitations +description: What Content Delivery deliberately does not do, and the reasoning behind each line - so you can tell a gap from a decision. +icon: OctagonAlert +--- + +Delivery is metadata and routing infrastructure. Most of the list below is not +"unfinished" - it is the shape of that decision. + +## It does not render anything + +No page builder, no layout builder, no block renderer, no React page generation. The +engine answers "what is the URL of this, and what should the page say about itself"; what +the page *is* belongs to the application. + +The practical consequence: delivery returns a path and a metadata object, and a plugin or +an app builds `/pl/articles/moj-artykul` from them. It does not assume your route +structure and it will not generate one. + +## No manual redirect manager + +There is no UI, no route and no service for creating a redirect by hand - and none for +deleting one. + +The AdminCP panel is **read-only** on purpose. A redirect is somebody else's incoming +link, so deleting one silently breaks traffic nobody in that dialog can see. That is a +destructive action, and a destructive action needs its own permission, a confirmation +that explains the consequence, and an audit trail. Displaying the history is useful +today; managing it is a product rather than a button. + +Consequences worth knowing: + +- A historical address stays reserved for as long as redirects are enabled. There is no + way to release one through the engine, so + [slug reuse](/docs/dev/content-engine/slug-history-and-redirects#historical-addresses-are-reserved) + by an unrelated record is refused permanently. +- If you genuinely need to release one, delete the row. It is an ordinary table, and the + [migrations guide](/docs/dev/content-engine/content-delivery-migrations) documents its + shape. + +There are also no **wildcard** or **regex** redirects, and no redirects between +arbitrary URLs. Slug history maps one record's old addresses to that record's current +one; a rule engine over paths is a different feature living in a different layer (a +middleware, a CDN, a `next.config` `redirects` array). + +## Redirects require Editorial + +```ts +delivery: { enabled: true, redirects: { enabled: true } } +// ✖ without `editorial: { enabled: true }` +``` + +Slug history has to be written in the same transaction as the slug mutation, its version +check and its revision - and only the editorial mutation paths own such a transaction. +Without `editorial` a content type writes through the plain repository, so +`redirects: { enabled: true }` there would record nothing. + +Refused at definition time rather than downgraded, and **only** `redirects` is affected: +canonical URLs, SEO, alternates, `hreflang`, the sitemap and every delivery read remain +available without Editorial, which keeps Stage 5's "publication and localization without +Editorial" promise intact. See +[Redirects require Editorial](/docs/dev/content-engine/slug-history-and-redirects#redirects-require-editorial). + +Lifting it would mean giving the plain mutation paths a version column and a +transactional history write - which is most of what `editorial` already is. + +## A delivery path is a site-wide namespace + +Two plugins may publish the same `publicApi.path` while neither has `delivery`, because +their API routes are `/api/{pluginId}/content/{path}`. Two **delivery-enabled** content +types may not, because a canonical delivery URL is `/articles/{slug}` with no plugin id +in it - one path would give one public URL two owners. + +The fix is a boot-time check rather than a prefix: adding the plugin id to canonical URLs +would make every public content URL uglier for everybody to avoid a collision almost +nobody hits. Rename one `publicApi.path`, or turn `delivery` off on one of them. + +## No og:image + +`delivery.seo.openGraph` projects a title and a description, and stops there. + +An `og:image` needs an absolute URL, known dimensions and a stable content type for the +file - which is a media subsystem, and Stage 8 does not build one. Emit it from your own +`generateMetadata` alongside the delivery metadata: + +```ts +const metadata = await contentDeliveryMetadata({ … }); + +return { + ...metadata, + openGraph: { ...metadata.openGraph, images: [await coverImageFor(slug)] }, +}; +``` + +## noIndex is shared, not per locale + +`delivery.seo.noIndexField` must be a **shared** boolean, and a localized one is a +definition-time error. + +The reason is that one field drives two consumers - the sitemap exclusion and the +`robots` directive - and they have to agree. A per-locale value would give one record one +answer per language while it has a single canonical decision, and the two consumers could +then disagree about which URLs exist. + +Per-locale indexing is a real thing to want. It is deferred rather than approximated, +because doing it properly means a per-locale sitemap decision *and* a per-locale +`robots`, both derived from the translation actually being served. + +## A localized content type needs a localized slug for redirects + +```ts +// ✖ localized content type, shared slug field +delivery: { enabled: true, redirects: { enabled: true } } +``` + +Every language answers to the same segment, so `/en/x/hello` and `/pl/x/hello` are both +live and one slug change moves both at once. Slug history stores *the URL that was live*, +so one retired row would have to be several paths - and the panel would show one of them +as if it were the address somebody bookmarked. + +Canonical URLs, SEO, alternates and the sitemap all work in that shape. Only the +reservation is ambiguous, so only `redirects` is refused. Mark the slug +`localized: true` and everything is available. + +## 410 Gone is not distinguished from 404 + +A historical URL whose destination is unpublished or deleted answers `not_found`. + +A `410` would be more informative for a deletion - it tells a crawler to forget the URL - +but the engine has no tombstone abstraction that distinguishes "deleted on purpose" from +"unpublished for now", and a `410` that guessed would tell a crawler to forget a URL that +is coming back next week. One documented status, chosen because it is the one that is +always correct. + +## The redirect status is not configurable + +Always `308`. Every historical URL of every content type answers with it, so there is no +per-content-type setting to get wrong and no reason for two of them to disagree. `301` is +not offered: it lets a client rewrite the method to `GET`, and `308` does not. + +## Sitemap frequency and priority are static + +Per content type, not per record. There are no dynamic callbacks: a function that runs +once per URL in a 50,000-URL file is a performance decision disguised as a configuration +option. + +## No site-wide robots.txt + +`delivery.seo.noIndexField` is per record. Site-wide crawl rules - `Disallow`, crawl +delay, sitemap declarations - are application or core configuration, not something a +content type gets to influence. + +## No delivery mutations in the service + +`model.deliveryService` is read-only, and structurally so: slug history is written by the +editorial services inside the transaction that moves the slug, so there is no `reserve` +to call without one. Admin or manual history mutation, if it ever exists, will be a +separate API with its own permission. + +## Delivery metadata is not a revision + +SEO is derived from content fields, so it already participates in +[revisions](/docs/dev/content-engine/revisions) - restoring a revision that had a +different `seo.title` changes the derived metadata on the next read. There is no separate +SEO history, and there will not be one: two revision systems over the same values is two +things to keep in sync. + +## itemId can be absent on a nonlocalized content type + +Delivery metadata reports `itemId: null` for a **nonlocalized** content type whose +`publicApi.fields` withholds `"id"`. + +That is deliberate rather than a gap: delivery reads the **public projection**, so it +cannot report a column the public API declined to publish. Expose `"id"` in the allowlist +and it is always present. A sitemap entry always carries it, because a sitemap row is +built from the row rather than from the projection. + +A **localized** content type has to expose `"id"` - alternates are resolved by +identifier - so its `itemId` is never `null`. + +## Not in Stage 8 at all + +For the avoidance of doubt: no domain management, no CDN configuration, no content +approval workflow, no collaborative editing, no AI SEO generation, no AI translation, no +translation memory, no external TMS, no GraphQL, no semantic search, no analytics, no A/B +testing and no personalized URLs. + +## See also + + + + + diff --git a/apps/docs/content/docs/dev/content-engine/content-delivery-migrations.mdx b/apps/docs/content/docs/dev/content-engine/content-delivery-migrations.mdx new file mode 100644 index 000000000..3a55bc025 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-delivery-migrations.mdx @@ -0,0 +1,184 @@ +--- +title: Delivery migrations +description: One new core table, one deterministic migration, and a clear statement about what history does *not* get backfilled. +icon: Database +--- + +Enabling `delivery` adds no columns to your content tables. It needs one shared core +table, and that is the whole schema change. + +## The migration + +```bash +pnpm drizzle-kit generate --name=add_content_slug_history +pnpm db:migrate +``` + +```sql +CREATE TABLE "core_content_slug_history" ( + "id" serial PRIMARY KEY NOT NULL, + "pluginId" varchar(255) NOT NULL, + "contentTypeId" varchar(100) NOT NULL, + "itemId" integer NOT NULL, + "languageId" integer, + "slug" varchar(160) NOT NULL, + "path" varchar(512) NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + "retiredAt" timestamp +); +ALTER TABLE "core_content_slug_history" ENABLE ROW LEVEL SECURITY; + +CREATE UNIQUE INDEX "core_content_slug_history_shared_unique" + ON "core_content_slug_history" ("contentTypeId","slug") + WHERE "languageId" IS NULL; + +CREATE UNIQUE INDEX "core_content_slug_history_locale_unique" + ON "core_content_slug_history" ("contentTypeId","languageId","slug") + WHERE "languageId" IS NOT NULL; + +CREATE INDEX "core_content_slug_history_item_idx" + ON "core_content_slug_history" ("contentTypeId","itemId","languageId"); + +CREATE INDEX "core_content_slug_history_plugin_id_idx" + ON "core_content_slug_history" ("pluginId"); +``` + +One table for every delivery-enabled content type in the install, for the same reason +`core_content_revisions` is shared: a content table is generated at runtime from a +descriptor, so core's static schema cannot name it - and a per-type history table would +mean a second generated table and a second migration for every plugin. + +### The indexes are the feature + +| Index | What it is for | +| -------------------- | ----------------------------------------------------- | +| `…_shared_unique` | The reservation, for a nonlocalized or shared slug | +| `…_locale_unique` | The reservation, per language | +| `…_item_idx` | One record's history, and retiring the slug it moved off | +| `…_plugin_id_idx` | Ownership, for an audit or a cleanup | + +The two uniques double as the resolver's lookup, which is why they lead with +`(contentTypeId, slug)`: a redirect lookup runs on a public request path for a URL that +is very often a typo, so it has to be an index hit rather than a scan. The PostgreSQL +suite asserts all four exist and that the shared one really is partial. + +Two partial uniques rather than one over a nullable `languageId`, because Postgres treats +every `NULL` as distinct - a single key including it would enforce nothing at all for the +shared case it exists to protect. + +## History is not backfilled + +**Nothing existing gets a history row.** History starts when Stage 8 begins tracking +future public slug changes, and that is a decision rather than an omission: + +```text +existing published article, slug = hello + → no history row + → /articles/hello is canonical, as it always was + → the *next* slug change creates the redirect +``` + +A record's current slug does not need to be history - it is the canonical URL, and the +resolver finds it through the ordinary public read. The first row for a record is written +the next time it is published or the next time its live slug moves. + +### Why revisions are not scanned + +It is tempting to reconstruct history from +[Stage 4 revisions](/docs/dev/content-engine/revisions), and the engine deliberately does +not: + +- **Revision snapshots include draft-only slugs.** A record whose slug was corrected + three times before publication would produce three redirects to URLs nobody ever + visited - and three permanent reservations blocking those addresses. +- **Publication timing is ambiguous.** A snapshot records the values at a version, not + whether that version was ever the *live* one. Reconstructing "was this slug + addressable" from a revision list means guessing. +- **Old schemas differ.** A snapshot taken before a field was renamed does not name the + slug field the content type has today. + +An automatic backfill would therefore create incorrect redirects, and an incorrect +permanent redirect is worse than a missing one: it sends real traffic somewhere wrong and +reserves an address nobody can reclaim. + +### An explicit backfill, if you want one + +If you *know* your data - because you have an access log, an external redirect map, or a +changelog - insert the rows yourself. The shape is documented above and the engine reads +it directly: + +```sql +INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "languageId", "slug", "path", "retiredAt") +VALUES + ('@vitnode/example', 'example.article', 42, NULL, + 'old-slug', '/articles/old-slug', now()); +``` + +Three rules to hold: + +1. **`retiredAt` must be set** for a historical address. A `NULL` means "this is the + record's current slug", and two current rows for one record is a state the engine does + not produce. +2. **`path` must be the URL that was live**, not one rebuilt from today's + `publicApi.path`. That is the whole reason the column exists. +3. **`languageId`** is the language for a localized slug and `NULL` for a shared one - + matching `delivery.slugScope`. Getting it wrong puts the row in the other partial + unique index and the resolver will not find it. + +Verify with the AdminCP delivery panel: it lists exactly what the resolver will use. + +## Changing `publicApi.path` + +```text +/articles → /blog +``` + +This is **source configuration**, not a content mutation, and the engine creates no +redirects for it. Every record's URL changes at once, at deploy time, for a reason no +row in the database records. + +Automating it would mean writing one history row per record on boot - a migration +disguised as a config change, running inside a process that may be one of several +starting at the same moment. So it is left to you, deliberately: + +```sql +-- One row per published record, with the old prefix. +INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "languageId", "slug", "path", "retiredAt") +SELECT + '@vitnode/example', 'example.article', a."id", NULL, + a."slug", '/articles/' || a."slug", now() +FROM "example_articles" a +WHERE a."status" = 'published' AND a."publishedAt" IS NOT NULL +ON CONFLICT DO NOTHING; +``` + +That is safe because the slug is unchanged - only the prefix moved - so the retired +`path` is the old URL and the resolver's destination is the record's current canonical +path under the new prefix. Run it in the same deploy as the config change. + +Stage 8's automatic redirects are for **slug changes**. Route-prefix migrations are a +deployment decision, and they get explicit tooling or explicit SQL. + +## Turning delivery off + +Removing the `delivery` block stops the engine reading or writing history. The table and +its rows stay - which is what you want, because turning it back on restores every +redirect rather than starting from nothing. + +Nothing else about the content type changes: no columns are dropped, no routes disappear +beyond the three delivery ones, and no cache tag it produced was ever a delivery tag. + +## Adding delivery to an existing content type + +Safe and additive: + +1. Add the block. No schema change to your table. +2. Run the core migration if you have not already. +3. Existing published records keep their canonical URLs and gain sitemap entries + immediately. +4. The first slug change on a published record creates the first redirect. + +There is no reindex, no rebuild and no backfill step - which is the practical +consequence of delivery being a projection over data the content type already had. diff --git a/apps/docs/content/docs/dev/content-engine/content-delivery-nextjs.mdx b/apps/docs/content/docs/dev/content-engine/content-delivery-nextjs.mdx new file mode 100644 index 000000000..e6b7594ff --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-delivery-nextjs.mdx @@ -0,0 +1,253 @@ +--- +title: Delivery in Next.js +description: A thin adapter that turns framework-neutral delivery metadata into a generateMetadata return value, a sitemap.ts and a 308 - and nothing more. +icon: FileCode +--- + +The core delivery layer is framework-neutral on purpose, so `@vitnode/core/content/next` +is a translation layer and nothing else: it maps delivery metadata onto the two shapes +Next.js asks for, and issues the redirect the resolver reports. + +It reads over **HTTP** rather than through `model.deliveryService`, because in VitNode's +split deployment the web app is not the process that holds the database. A +single-process install can call the service directly and skip this entirely. + +## generateMetadata + +```tsx title="src/app/[locale]/articles/[slug]/page.tsx" +import { contentDeliveryMetadata } from "@vitnode/core/content/next"; + +import { articleContentType } from "@vitnode/example/content/article"; + +export const generateMetadata = async ({ + params, +}: { + params: Promise<{ locale: string; slug: string }>; +}) => { + const { locale, slug } = await params; + + return await contentDeliveryMetadata({ + definition: articleContentType, + locale, + origin: "https://example.com", + pluginId: "@vitnode/example", + slug, + }); +}; +``` + +That produces, for a published record: + +```ts +{ + title: "My article", + description: "A summary.", + alternates: { + canonical: "https://example.com/en/articles/my-article", + languages: { + en: "https://example.com/en/articles/my-article", + pl: "https://example.com/pl/articles/moj-artykul", + "x-default": "https://example.com/en/articles/my-article", + }, + }, + openGraph: { + title: "My article", + description: "A summary.", + url: "https://example.com/en/articles/my-article", + }, + robots: { index: true, follow: true }, +} +``` + +Every key is **absent** rather than present-and-null when there is no value. Next +renders a `null` title as an empty `` and an absent one not at all, and an empty +`<title>` is worse than none. + +`{}` for a URL that does not resolve, rather than a throw: `generateMetadata` runs +alongside the page, the page is what calls `notFound()`, and a metadata function that +threw would replace a clean 404 with an error boundary. + +### Pass an origin + +Optional, and strongly recommended. Without it every URL in the result is relative - a +relative `canonical` is legal and resolves against the page, and an absolute one is +what every SEO checker asks for. + +### The pure half + +```ts +import { contentDeliveryToNextMetadata } from "@vitnode/core/content/next"; + +contentDeliveryToNextMetadata(deliveryResponse, { origin }); +``` + +Exported separately so a page that already holds the delivery response - because it +fetched the record and its metadata together - can translate it without a second round +trip. It is also what makes the mapping unit-testable without a network. + +## The page: redirect, render, or 404 + +```tsx title="src/app/[locale]/articles/[slug]/page.tsx" +import { contentDeliveryPage } from "@vitnode/core/content/next"; + +const Page = async ({ + params, +}: { + params: Promise<{ locale: string; slug: string }>; +}) => { + const { locale, slug } = await params; + + // Only returns for the current slug: a moved URL has already 308ed, and a missing + // one has already 404ed. + const delivery = await contentDeliveryPage({ + definition: articleContentType, + locale, + pluginId: "@vitnode/example", + slug, + }); + + return <Article delivery={delivery} />; +}; + +export default Page; +``` + +`contentDeliveryPage` is the only helper in the adapter with a side effect, which is why +it lives in its own module: `next/navigation`'s control-flow functions throw to unwind +the render, so a page that only wanted metadata should not be able to reach them by +accident. + +It issues a **308** via `permanentRedirect`, with `RedirectType.replace` so a reader who +follows an old link does not have to press back twice to leave a page they were never +meant to land on. + +A draft, an unpublished record, a deleted one, a slug that never existed and a +historical URL whose destination is no longer public are all the same `notFound()`. A +redirect to hidden content would be a way to confirm it exists. + +<Callout type="info" title="Why not vitnode-frontend/navigation"> + That wrapper is the locale-aware one every app-level redirect should use, and this is + the one place it would be wrong: a delivery location is a **complete** path that + already carries its locale segment - the engine built it - so routing it through + `next-intl` would prefix the locale a second time. It is also a 307, and a canonical + slug change needs the permanent, method-preserving 308. +</Callout> + +### Resolving without acting + +```ts +import { contentDeliveryResolve } from "@vitnode/core/content/next"; + +const resolution = await contentDeliveryResolve({ + definition: articleContentType, + locale, + pluginId: "@vitnode/example", + slug, +}); + +switch (resolution.type) { + case "content": + return resolution; // canonical metadata + case "redirect": + return resolution; // { location, status: 308 } + case "not_found": + return null; +} +``` + +A discriminated union, so a caller branches on `type` rather than inferring which arm it +is holding. The route answers `not_found` as a **200 with a body** rather than a 404, +which is what lets a caller tell "this URL resolves to nothing" from "the delivery API +is unreachable" - and keeps a negative out of the response cache, so publishing the +record makes it resolve immediately. + +## Sitemap + +```ts title="src/app/sitemap.ts" +import { contentSitemapEntries } from "@vitnode/core/content/next"; + +const sitemap = async () => { + const { entries } = await contentSitemapEntries({ + definition: articleContentType, + origin: "https://example.com", + pluginId: "@vitnode/example", + }); + + return entries; +}; + +export default sitemap; +``` + +It pages through the delivery sitemap route until the cursor runs out, so a content type +with 40,000 published records is 40 requests rather than one enormous response. +`maxPages` (default 100) is a backstop, because an unbounded loop against a paginated +API is the one bug in this file that could take a site down - and reaching it is +reported through `truncated` rather than thrown, so a partial sitemap is still a valid +sitemap. + +```ts +const { entries, truncated } = await contentSitemapEntries({ … }); +if (truncated) { + // Split with `generateSitemaps` - see contentSitemapChunks. +} +``` + +Next caps a `sitemap.ts` at 50,000 URLs and splits beyond that with `generateSitemaps`; +[`contentSitemapChunks`](/docs/dev/content-engine/sitemaps#scaling-past-one-file) is the +helper that decides how many files that is. + +For a localized site, one call per locale: + +```ts +const sitemap = async () => { + const locales = ["en", "pl"]; + const pages = await Promise.all( + locales.map(async locale => + ( + await contentSitemapEntries({ + definition: articleContentType, + locale, + origin: "https://example.com", + pluginId: "@vitnode/example", + }) + ).entries, + ), + ); + + return pages.flat(); +}; +``` + +## Cache tags + +Every read here is `cache: "force-cache"` and carries the delivery tag that a mutation +expires: + +| Helper | Tag | +| -------------------------- | ----------------------------------------- | +| `contentDeliveryResolve` | `content:{id}:redirect:{locale?}:{slug}` | +| `contentDeliveryItem` | `content:{id}:delivery:{locale?}:{itemId}` | +| `contentSitemapEntries` | `content:{id}:sitemap:{locale?}` | + +`resolve` is tagged by the **slug** rather than the record, which is what makes a moved +page stop being served from its former URL: a slug change expires the old address's +lookup and the record's metadata at the same moment. + +`contentDeliveryItem` is tagged by the record, which makes it the right call for a page +that already knows which record it is rendering - an edit to the SEO description expires +it, and an unrelated record's publish does not. + +See [Cache behaviour](/docs/dev/content-engine/caching) for the whole tag list and what +expires each one. + +## Metadata types are not in core + +`ContentDeliveryNextMetadata` is a structural type rather than an +`import type { Metadata } from "next"`, so the core package does not grow a +compile-time dependency on the framework's type surface for four keys. It is assignable +to `Metadata`, which is what a `generateMetadata` needs it to be. + +That is the same reason the core engine returns `{ languages, xDefault? }` rather than +Next's `alternates` shape: move to Astro and you write a different forty lines against +the same service. diff --git a/apps/docs/content/docs/dev/content-engine/content-delivery.mdx b/apps/docs/content/docs/dev/content-engine/content-delivery.mdx new file mode 100644 index 000000000..1d982e50b --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-delivery.mdx @@ -0,0 +1,234 @@ +--- +title: Content Delivery +description: Opt a content type into canonical URLs, slug history, redirects, hreflang, SEO metadata and sitemaps - without the engine rendering a single page. +icon: Route +--- + +Stages 1–7 gave content a table, a lifecycle, a public API, translations and a +history. None of them answered the question a frontend actually asks: + +```text +What is the URL of this thing? +``` + +`delivery` is the block that answers it - and the four questions that follow from +it: what was its URL before, should the old one redirect, which other languages +does it exist in, and what should the page put in `<head>`. + +```ts title="src/content/article.ts" +export const articleContentType = defineContentType({ + id: "example.article", + tableName: "example_articles", + + fields: { + title: field.text({ required: true, maxLength: 200 }), + slug: field.slug({ source: "title" }), + excerpt: field.textarea({ maxLength: 500, nullable: true }), + }, + + publication: { enabled: true }, + + publicApi: { + enabled: true, + path: "articles", + fields: ["id", "title", "slug", "excerpt", "publishedAt"], + }, + + // `redirects` below needs this: slug history is written in the same transaction as + // the slug mutation and its revision. + editorial: { enabled: true }, + + delivery: { // [!code highlight] + enabled: true, // [!code highlight] + redirects: { enabled: true }, // [!code highlight] + seo: { // [!code highlight] + titleField: "title", // [!code highlight] + descriptionField: "excerpt", // [!code highlight] + }, // [!code highlight] + sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 }, // [!code highlight] + }, // [!code highlight] + + admin: { label: { plural: "Articles", singular: "Article" } }, +}); +``` + +That is the whole opt-in. Omit the block and **nothing** about the content type +changes: same tables, same routes, same cache tags, same events, same everything. + +## What delivery is not + +Delivery is metadata and routing infrastructure. It is deliberately **not** a page +builder, and the line is worth stating plainly because it explains most of the API: + +- It returns a **path**, not a page. No React, no layout, no blocks. +- It does not assume your route structure. `/pl/articles/x` is what the engine + builds from `publicApi.path`, and a frontend is free to serve it from anywhere. +- It does not know your domain. Canonical paths are relative; you supply an origin + when you want an absolute URL. + +What it gives you is enough typed metadata that a plugin or an app can render +`/pl/articles/moj-artykul` and `/en/articles/my-article` correctly - including the +`hreflang` set, the redirect from the URL that page used to live at, and the +sitemap entry. + +## The blocks + +| Block | What it adds | +| ----------- | ------------------------------------------------------------------ | +| `redirects` | Durable [slug history](/docs/dev/content-engine/slug-history-and-redirects) and automatic 308s. **Needs `editorial`** | +| `seo` | [Title, description, Open Graph and robots](/docs/dev/content-engine/seo) projection | +| `sitemap` | A paginated [sitemap service](/docs/dev/content-engine/sitemaps) | +| `hreflang` | An `x-default` for [localized alternates](/docs/dev/content-engine/localization-and-hreflang) | + +Every one of them is optional. `delivery: { enabled: true }` on its own gives you +canonical URLs and alternates, which is already the hard part. + +## Delivery requires a public API + +```ts +delivery: { enabled: true } +// ✖ without `publicApi: { enabled: true }` +``` + +A content type with no public API has no public URL, so there is nothing for +delivery to be about. That is a **compile error** on the `enabled: true` itself, +not just a boot-time throw: + +```ts +// Type 'true' is not assignable to type 'never'. +``` + +The runtime check stays as well, for a JavaScript caller and for a value that +widened somewhere upstream. Every other delivery rule works the same way - see +[Validation rules](#validation-rules). + +## The service + +`model.deliveryService(c, { pluginId })` is the server API. It is read-only, and +that is structural rather than a convention: slug history is written by the +editorial services inside the transaction that moves the slug, so there is no +`reserve` here to call without one. + +```ts +const delivery = articleContent.deliveryService?.(c, { pluginId }); + +await delivery?.findById(42, { locale: "pl" }); +await delivery?.resolvePath("/pl/articles/stary-slug"); +await delivery?.alternates(42); +await delivery?.sitemap({ locale: "pl", limit: 1_000 }); +await delivery?.history(42); +``` + +`undefined` for a content type without `delivery`, exactly like `publicService` +and `editorialService` - so the check reads naturally in code that does not know +which content type it was handed. + +Every answer is derived from the **public projection**, not from the base row: +`findById` and `resolveSlug` go through `model.publicService`, so the publication +predicate, the field allowlist and the +[fallback rules](/docs/dev/content-engine/localized-public-api) are the ones +already tested rather than a second implementation that agrees on the day it is +written. It is also what makes "SEO cannot leak a private field" true at runtime: +a private column is never fetched, so it is not in the row delivery reads. + +## Generated routes + +A delivery-enabled content type gains three public routes: + +```http +GET /api/{pluginId}/content/{path}/delivery/resolve/{slug} +GET /api/{pluginId}/content/{path}/delivery/item/{id} +GET /api/{pluginId}/content/{path}/delivery/sitemap +``` + +They exist because a frontend is very often **not** the process that holds the +database: VitNode's split deployment runs Next.js against a separate API, so +`generateMetadata`, a catch-all route and a `sitemap.ts` handler all need an HTTP +answer rather than a service call. A single-process install can call the service +directly and never touch them. + +Every path begins with the static `delivery` segment, which is what makes them +impossible to shadow: `/{slug}` is one segment and these are two or three, so a +record whose slug is literally `delivery` still resolves the ordinary way. + +<Callout type="info" title="No staff permission"> + Public delivery resolution is exactly as public as the content it describes. + Requiring a session to learn a canonical URL would be requiring one to render a + page. The [AdminCP route](/docs/dev/content-engine/slug-history-and-redirects#admincp) + that shows historical URLs is a different route, and it does require `can_view`. +</Callout> + +## Validation rules + +Delivery fails at **definition time** rather than at request time, because a +canonical URL that quietly stopped being generated is a page that quietly stopped +being indexable - and that is not a symptom anybody notices. + +| Rule | Result | +| ------------------------------------------------------ | -------------------------- | +| `delivery` without `publicApi` | Compile error + throw | +| `redirects` without `editorial` | Compile error + throw | +| `sitemap` without `publication` | Throw | +| `redirects` on a localized type with a **shared** slug | Throw | +| An SEO field not in `publicApi.fields` | Compile error + throw | +| A `textarea` as `titleField` | Compile error + throw | +| A repeatable leaf in any SEO slot | Compile error + throw | +| A non-boolean `noIndexField` | Compile error + throw | +| A **localized** `noIndexField` | Throw | +| `sitemap.priority` outside `0`–`1` | Throw | +| An unknown `changeFrequency` | Compile error + throw | +| `hreflang` without `localization` | Throw | +| A localized content type withholding `"id"` | Throw | +| A fallback SEO field with no primary | Throw | + +Three are worth a word. `redirects` needs `editorial: { enabled: true }`, because slug +history has to be written in the same transaction as the slug mutation and its revision - +and only the editorial mutation paths own one. Nothing else in `delivery` needs it; see +[Redirects require Editorial](/docs/dev/content-engine/slug-history-and-redirects#redirects-require-editorial). + +A **localized** content type has to expose `"id"` in +`publicApi.fields`, because alternates and `hreflang` are resolved by identifier and +delivery reads the public projection - so without it every localized response would +carry an empty alternate set, which looks exactly like a record with one translation. +A nonlocalized content type has no alternates to resolve and needs nothing. + +And the last one: `fallbackTitleField` without `titleField` is a +configuration that reads as if it does something and does nothing, because the +fallback is only consulted when the primary is empty. Naming only the fallback +means it is never reached, so the engine tells you to name it as the primary +instead. + +## Where to go next + +<Cards> + <Card + href="/docs/dev/content-engine/canonical-urls" + title="Canonical URLs" + description="How a path is built, and why it is relative." + /> + <Card + href="/docs/dev/content-engine/slug-history-and-redirects" + title="Slug history and redirects" + description="When a URL becomes redirectable, and who owns it afterwards." + /> + <Card + href="/docs/dev/content-engine/seo" + title="SEO" + description="Title, description, Open Graph and robots, from public fields." + /> + <Card + href="/docs/dev/content-engine/sitemaps" + title="Sitemaps" + description="A paginated service, an XML helper and a sitemap index." + /> + <Card + href="/docs/dev/content-engine/localization-and-hreflang" + title="Localization and hreflang" + description="Alternates that are real published translations, and nothing else." + /> + <Card + href="/docs/dev/content-engine/content-delivery-nextjs" + title="Next.js helpers" + description="generateMetadata, sitemap.ts and the redirect." + /> +</Cards> diff --git a/apps/docs/content/docs/dev/content-engine/content-engine-observability.mdx b/apps/docs/content/docs/dev/content-engine/content-engine-observability.mdx new file mode 100644 index 000000000..b3cdfeeb7 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-engine-observability.mdx @@ -0,0 +1,272 @@ +--- +title: Observability +description: The two questions an operator asks at three in the morning, and where the Content Engine answers them. +icon: Activity +--- + +This is not a monitoring product. There is no time series, no Prometheus and no +dashboard, because the install has none of those and the Content Engine is not +the place to introduce them. + +What it does have is answers to the two questions that actually get asked when +something looks wrong. + +## Is the search index telling the truth? + +Two storages, two questions. `SearchModel.index` writes the canonical +`core_search_index` row and *then* hands the document to the active provider, so +an Elasticsearch that refuses the second half leaves a canonical table that is +perfectly correct and a search box that is missing results. + +A diagnostic that only looked at the canonical table would call that healthy. So +this one asks both. + +```http +GET /api/@vitnode/core/admin/debug/content/status +``` + +```json +{ + "healthy": false, + "searchHealthy": false, + "effectsHealthy": true, + "contentTypes": [ + { + "contentTypeId": "example.localized-article", + "pluginId": "@vitnode/example", + "search": { + "contentTypeId": "example.localized-article", + "canonicalHealthy": true, + "healthy": false, + "provider": { + "name": "elasticsearch", + "verified": true, + "healthy": false + }, + "locales": [ + { + "locale": "en", + "expected": 421, + "canonicalIndexed": 421, + "canonicalHealthy": true, + "providerIndexed": 421, + "providerHealthy": true + }, + { + "locale": "pl", + "expected": 421, + "canonicalIndexed": 421, + "canonicalHealthy": true, + "providerIndexed": 419, + "providerHealthy": false + } + ] + }, + "schedules": null + } + ] +} +``` + +### The vocabulary + +| Field | Means | +| ----- | ----- | +| `expected` | Published rows - or published translations - the **database** holds. | +| `canonicalIndexed` | Documents in `core_search_index`. | +| `canonicalHealthy` | The canonical table matches the database. | +| `providerIndexed` | Documents the **active provider** holds for this locale. `null` when it cannot say. | +| `providerHealthy` | The provider matches the database for this locale. `null` when nobody looked. | +| `expectedTotal` | Every published row or translation, all locales. | +| `canonicalIndexedTotal` | Every canonical document, all locales. | +| `provider.indexedTotal` | Every provider document, **all locales, including ones nothing knows about**. | +| `verified` | Whether the provider was actually asked. | +| `healthy` | Canonical **and** provider agree, per locale *and* in total. | + +There is deliberately no single `indexed` number. Once there are two storages, +one number can only be a guess about which of them you meant. + +### Why the totals matter as well as the locales + +Per-locale counts can only ask about locales somebody already knows to ask for - +and that list is built from the database and the canonical table. A document that +exists **only** in the provider is invisible to it. + +That is not hypothetical. Deletion runs canonical-first: `SearchModel.delete` +removes the canonical row and then asks the provider. If the provider's half +fails, the document survives in a locale that no longer appears in either source, +so nothing enumerates it: + +```text +database expected pl: 0 +core_search_index pl: 0 +Elasticsearch pl: 1 ← nothing asks about "pl" any more +``` + +An unfiltered total finds it, because it is not built from an enumeration: +`providerTotal > expectedTotal` is enough to say something is wrong even when +every locale anybody checked agreed. The same guard covers a locale that was +removed from the installation entirely, and a content type with **no** rows at +all - where the per-locale list is empty and `[].every(...)` would otherwise say +everything is fine. + +<Callout type="warn" title="Unverified is not healthy"> +A provider that offers no `count` reports `verified: false` and +`providerHealthy: null`, and the content type is **not** healthy. Absence of +evidence is reported as absence of evidence - turning it into a clean bill of +health is exactly how a broken search box hides behind a good canonical table. +</Callout> + +### What each provider does + +| Provider | Behaviour | +| -------- | --------- | +| **Postgres** (bundled) | Its store *is* `core_search_index`, so it declares `canonicalStorage` and is verified without a second query. Canonical and provider always agree, because they are one thing. | +| **Elasticsearch** | Implements `count` with the `_count` API - a number, never fetched documents - filtered by `itemType`, and by `languageCode` when one is given. Omitting the language is the unfiltered total. It creates no index as a side effect: a diagnostic is observational. | +| **Anything else** | Reported as `verified: false` unless it implements `count`. | +| **A provider that throws** | `verified: true`, `healthy: false`, and `error` carries the reason. A failure in either the total or a per-locale count is handled the same way. The status route still answers, and the failure is logged behind `[content-diagnostics]`. | + +The expected side uses the **same** `publishedCondition` the indexer uses. +Re-deriving "published" here would let the diagnostic disagree with the thing it +is diagnosing, which is the one way a health check is worse than none. + +### Teaching your own provider to answer + +```ts +export const MySearchAdapter = (): SearchProviderApiPlugin => ({ + name: "my-engine", + // ... + count: async (c, { itemType, languageCode }) => + await myEngine.count({ itemType, languageCode }), +}); +``` + +Count, do not fetch: a diagnostic over a large collection has to cost the same +as one over an empty one. Honour `languageCode` if your store keeps one document +per translation - a single total cannot say "Polish is missing forty documents" - +and treat an **omitted** `languageCode` as every language, because that call is +what finds a document in a locale nobody thought to ask about. + +Make it observational. Creating an index, warming a cache or writing anything +would let the health check change the thing it is measuring. + +### Repairing drift + +```http +POST /api/@vitnode/core/admin/debug/search/rebuild +{ "itemType": "example.localized-article" } +``` + +Scoped to one collection so a single-collection reindex never wipes the rest of +the index. A rebuild reproduces exactly what live synchronisation would have +written, so repairing drift never changes what search returns beyond making it +correct. + +## Did anything scheduled fail to announce itself? + +A scheduled transition that committed but whose event, index write or cache +expiry did not is recorded on the schedule row - and counted per content type on +the same status route: + +```json +"schedules": { "pending": 3, "withErrors": 0, "failedEffects": 1 } +``` + +| Field | Means | +| ----- | ----- | +| `pending` | Bookings still waiting to fire. Normal. | +| `withErrors` | Pending bookings whose last run threw. The transition has **not** happened and the queue is retrying, so this is visible without being a failure. | +| `failedEffects` | Transitions that **did** happen and were never announced. | + +`failedEffects` is the one that matters: the record *is* published, and nobody +has been told, and no amount of waiting fixes that on its own. The effects task +retries on the queue's backoff, so a non-zero value that stays non-zero is an +outage rather than a blip. Per booking, the AdminCP schedule panel shows the +reason. + +## What "healthy" means + +Three flags, because one would be misleading: + +```ts +{ + healthy: false, // searchHealthy && effectsHealthy + searchHealthy: true, // every searchable content type agrees, canonical and provider + effectsHealthy: false, // nothing committed without being announced +} +``` + +`healthy: true` beside `failedEffects: 15` is worse than no answer - it tells an +operator to stop looking. Splitting the dimensions means a search outage and an +announcement backlog are separately visible, and the headline is simply their +conjunction. + +A pending schedule is **not** unhealthy, and neither is a pending one whose last +attempt threw: in both cases the transition has not happened and the queue is +still working on it. Only `effectsError` - committed, unannounced - moves the +needle. + +## The logs + +Four greppable prefixes in `core_logs`, visible in AdminCP → Advanced → Logs. + +| Prefix | Written when | +| ------ | ------------ | +| `[content-search]` | An index write or delete threw after a committed mutation. | +| `[content-effects]` | An event was emitted and a listener did not receive it. | +| `[content-revalidate]` | A web origin refused or could not be reached. | +| `[content-diagnostics]` | The search provider could not be counted for a health check. | + +Each is a single JSON object behind the prefix, with the content type, the item, +the operation and the underlying error - enough to find the record without +correlating three systems by timestamp. + +Deliberately **not** logged as errors: a `404`, a no-op mutation, and a draft that +was not indexed. All three are the engine working, and a log full of them is a log +nobody reads. + +## Using the diagnostics from code + +Everything the route returns is available directly, so a plugin can build its own +panel or a cron job can alert on it: + +```ts +import { + contentEngineDiagnostics, + contentScheduleHealth, + contentSearchDrift, +} from "@vitnode/core/content/server"; + +// Everything, sorted by content type id so two calls - and two processes - +// report the same order. +const report = await contentEngineDiagnostics(c); + +// Or one content type at a time. `healthy` is canonical *and* provider, so +// check the two separately when you want to tell an index problem from an +// engine outage. +const drift = await contentSearchDrift(c, { model: articleContent }); +if (!drift.canonicalHealthy) { + await c.get("log").error(`[my-plugin] index drift: ${JSON.stringify(drift.locales)}`); +} +if (drift.provider.verified && drift.provider.healthy === false) { + await c.get("log").error(`[my-plugin] ${drift.provider.name}: ${drift.provider.error ?? "documents missing"}`); +} + +// Or just the schedules, in one grouped query for many content types. +const schedules = await contentScheduleHealth(c, ["example.article"]); +``` + +The route is behind `system: can_view`, like the rest of the debug surface. There +is nothing in the response an administrator could not read elsewhere, but there is +nothing in it a visitor should read either. + +## What this deliberately does not do + +- **It does not repair anything.** Drift is detected by counting; correcting it is + a rebuild, and a rebuild is a decision an operator makes. +- **It does not compare documents.** A count is not a checksum: two stale + documents still count as two. It catches the failure that actually happens - a + write that threw, a rebuild that stopped - for the price of two aggregates and + one provider count per locale. +- **It keeps no history.** Every number is computed on demand. Trending them over + time is what a monitoring system is for, and this is not one. diff --git a/apps/docs/content/docs/dev/content-engine/content-engine-security.mdx b/apps/docs/content/docs/dev/content-engine/content-engine-security.mdx new file mode 100644 index 000000000..dc93da8cb --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/content-engine-security.mdx @@ -0,0 +1,159 @@ +--- +title: Security +description: The permission matrix over every generated route, what a public response may contain, and why every bad preview link is the same 404. +icon: Lock +--- + +Three questions, and the engine answers each of them in exactly one place so +there is no second implementation to disagree with the first: + +1. **Who may do this?** A staff permission on every generated route. +2. **What may they see?** An allowlist, applied where the `SELECT` is built. +3. **Who is this anonymous caller?** A signed token, or nobody. + +## The permission matrix + +Every generated route carries an explicit `adminStaffPermission`, checked before +validation - so a request without the permission is a `403` whatever its body +says. The matrix is enumerated from the route builder in the test suite, which is +what stops a new endpoint joining the set without one. + +| Route | Permission | +| ----- | ---------- | +| `GET /` | `can_view` | +| `GET /{id}` | `can_view` | +| `GET /options/{field}` | `can_view` | +| `POST /` | `can_create` | +| `PUT /{id}` | `can_edit` | +| `DELETE /{id}` | `can_delete` | +| `POST /{id}/publish` · `/unpublish` | `can_publish` | +| `GET /{id}/revisions` · `/{revisionId}` | `can_view` | +| `POST /{id}/revisions/{revisionId}/restore` | `can_restore` | +| `POST /{id}/preview` | `can_view` | +| `GET /{id}/schedules` | `can_view` | +| `POST /{id}/schedule` · `/{scheduleId}/cancel` | `can_publish` | +| `GET /{id}/translations` · `/{locale}` · `/public-locales` | `can_view` | +| `POST` · `PUT /{id}/translations/{locale}` | `can_translate` | +| `DELETE /{id}/translations/{locale}` | `can_delete` | +| `POST /{id}/translations/{locale}/publish` · `/unpublish` | `can_publish` | +| `GET /{id}/translations/{locale}/revisions` · `/{revisionId}` | `can_view` | +| `POST /{id}/translations/{locale}/revisions/{revisionId}/restore` | `can_restore` | + +Two choices in there are worth the sentence they take: + +- **`can_publish` for scheduling.** Booking a publication *is* publishing, just + later. A role trusted to write drafts is not automatically trusted to put one + on the internet at 9am on Monday. +- **`can_restore` depends on `can_edit`.** Restoring rewrites many fields at once + from a source the editor did not type, so somebody who may not edit must not + reach the same outcome through the history. + +## The translator + +`can_translate` depends on `can_view` and deliberately **not** on `can_edit`, +which is what makes "writes Polish and nothing else" expressible. Give a role +`can_view + can_translate` and it can: + +- read the record, every locale tab and every locale's history; +- create and edit a translation in any enabled locale. + +It cannot: + +- edit a shared field (`PUT /{id}` is `can_edit`); +- publish or unpublish anything, record or translation (`can_publish`); +- restore a shared revision *or* a locale's own (`can_restore`, which needs + `can_edit`); +- delete the record or a translation (`can_delete`). + +Existing roles simply do not have `can_translate` - permissions are stored as +JSON per role, so a new one denies by default and needs no migration. + +## Advanced collections have no second door + +There is no per-relation or per-repeatable endpoint. A collection is written +through the ordinary `PUT /{id}`, gated on `can_edit`, in the same guarded +transaction as a field edit - so there is no route that hands somebody a write +primitive they could not have used anyway. + +The relation picker (`GET /options/{field}`) is a read of display labels, gated on +`can_view` like every other read, and it writes nothing. + +## Cross-plugin isolation + +Two plugins can name a permission module the same thing - `articles` is not an +unusual choice - and the registry allows it precisely because the plugin id is +part of the key. That only holds because the **route** carries its own plugin id +into the check rather than reading whichever plugin is handling the request. + +Underneath, everything shared is scoped the same way. The revisions table and the +schedules table are shared by every content type in the install, so every +statement filters on `pluginId`, `contentTypeId` *and* `itemId`. A revision id on +its own proves nothing about ownership, and cancelling somebody else's +publication would be a strange way to find that out. + +## Public responses are an allowlist + +`publicApi.fields` is not a filter applied to a full row. It decides the `SELECT` +itself, so a private column is never fetched - which means it cannot be leaked by +a mistake further downstream. + +What a public response contains, exactly: + +- the fields the allowlist names, and nothing else; +- a group carrying only the **leaves** the allowlist named - exposing `seo.title` + does not expose `seo.robots`; +- a repeatable child carrying its public leaves plus its `id`; +- a to-many relation as **identifiers**, never expanded rows - a target has its + own public API, its own allowlist and its own publication state; +- `locale`, on a localized content type, because a response has to say which + language it is. + +What it never contains: `id` unless the allowlist names it, `status`, `version`, +`createdAt`, `updatedAt`, `languageId`, any revision metadata, any private leaf, +any private collection, and any flattened storage column name (`seo.title` is +stored as `seoTitle`, and a response that carried that would publish an internal +detail *and* give a client two spellings of one value). + +## Preview fails closed + +A preview link is an unpublished record behind a short-lived credential, so every +part of it is written to fail rather than to be helpful. + +- **The token is the authorization.** HMAC-SHA256, bound to one plugin, one + content type, one record, one revision and - for a localized content type - one + locale. No session is consulted, which is the point: a reviewer has no account. +- **Every failure is the same 404.** A forged signature, an expired link, a token + for another record, a deleted revision and a record that never existed are + indistinguishable. A 401 or a 403 would confirm the record exists, which is + precisely what a draft URL must not do. +- **No fallback.** A `pl` link opened on the English URL is a 404, not the English + copy. Falling back would hand a reviewer a different language from the one they + were sent. +- **A weak secret disables it entirely.** An install whose `CONTENT_PREVIEW_SECRET` + is missing, too short, or still the published placeholder can have its tokens + forged by anyone, so no token is honoured - and the answer is the same 404, + because "preview is misconfigured here" is not something an anonymous request + needs to learn. +- **Nothing caches it.** `Cache-Control: private, no-store` and + `X-Robots-Tag: noindex, nofollow`, and the response carries no cache tag at all. +- **The projection is the public one.** The same function the detail route uses, + so a field cannot be private on one and public on the other. + +## Cache keys cannot mix public and private + +An AdminCP read forwards the staff cookie and is never cached - no `force-cache`, +no tags. A preview is `no-store`. Only the generated public read is cached, and +only under tags built from `publicApi.path`. + +Locale spellings are normalised where the tag is built, so `pl`, `PL` and `" pl "` +address one cache entry and expire together - a tag is a string comparison, and +hoping every call site agrees on casing is not a plan. + +## Errors say nothing they should not + +A driver error carries the constraint name, often the column, and sometimes the +value that clashed. None of it reaches a response body; every expected failure is +mapped to a status and, where a client has to branch, a stable code. See +[the error contracts](/docs/dev/content-engine/editorial). Anything unrecognised +becomes a bare `500` - the detail goes to `core_logs`, and production answers +`Internal Server Error` and nothing else. diff --git a/apps/docs/content/docs/dev/content-engine/failure-and-retries.mdx b/apps/docs/content/docs/dev/content-engine/failure-and-retries.mdx new file mode 100644 index 000000000..b9394ab75 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/failure-and-retries.mdx @@ -0,0 +1,186 @@ +--- +title: Failure and retries +description: What survives an outage, what gets retried, and why event delivery is at-least-once rather than exactly-once. +icon: TriangleAlert +--- + +A content mutation touches four systems, and only one of them can be rolled +back. The database write either committed or it did not. The event, the search +document and the cache invalidation are calls to things that can be down for a +minute - and once the transaction has closed, nothing can un-send them. + +So the engine draws a hard line, and everything on this page follows from it. + +## Inside the transaction, and after it + +```text +inside the transaction after the commit +------------------------ ---------------------------- +the row lock the event +validation that needs locked state the search document +the content write the cache invalidation +the version increment the remote revalidation +the revision insert +the schedule transition +the collection rows +the queue row that announces it +``` + +Two things about that split are worth stating outright. + +**Nothing external is inside.** No `fetch`, no search call, no `next/cache`. A +rolled-back transaction cannot un-emit an event, so an event emitted inside one +is a lie waiting to happen. + +**The queue row is inside.** The task that announces a scheduled publication is +written in the same transaction as the publication, so it exists if and only if +the transition committed. A crash a millisecond later loses nothing: the row is +durable, and the queue will drain it. + +<Callout type="warn" title="Calling the services directly"> +`syncContentSearch`, `contentEditorialEffects` and `revalidateContent` must be +called **after** the write has returned, never inside a `db.transaction()` +callback. The generated routes already do this; hand-written code has to. +</Callout> + +## When the event transport fails + +`EventsModel.emit` reports rather than throws, so `failures` is the only place a +dead listener or a broker outage is visible at all. + +The write has already committed by then, and two things follow: + +1. **The request still succeeds.** Answering 500 would tell the client its edit + was lost when it was not, and invite a retry that creates a second version of + everything. +2. **The failure is never swallowed.** It goes to `core_logs` behind + `[content-effects]` with the content type, the item and the listener that + failed: + +```text +[content-effects] {"action":"published","contentTypeId":"example.article", +"delivered":0,"eventId":"...","failures":[{"error":"Service unavailable", +"listener":"@acme/plugin:notifications:send-notification"}],"itemId":7} +``` + +Search it in AdminCP → Advanced → Logs. On the scheduled path the same failure is +also written onto the schedule row as `effectsError`, and retried. + +## When the search engine fails + +The index write throws, the mutation stays committed, and the failure is +reported three ways: on the outcome (`ContentSearchSyncOutcome.error`), in +`core_logs` behind `[content-search]`, and - for a scheduled transition - on the +schedule row. + +The index is eventually consistent, and "eventually" is bounded by two things: + +- **the next write** that touches an indexed field on that record, which + rewrites the document; +- **a rebuild**, which rewrites all of them. + +A rebuild reproduces exactly what live synchronisation would have written - +that equality is itself a test - so repairing drift never changes what search +returns beyond making it correct. + +## When a web origin refuses its cache invalidation + +The [revalidation bridge](/docs/dev/content-engine/caching) posts to every +configured web origin, retries each one twice, and **reports rather than +throws**. The caller decides what counts as delivered, and the scheduled effects +task requires *all* of them: + +```text +attempted: 0 nothing needed telling. Not a failure. +delivered === attempted delivered. +delivered < attempted a partial. The run fails and is retried. +``` + +A partial is the dangerous case, not the acceptable one. With two web apps behind +one API, one of them accepting an unpublish while the other does not leaves the +withdrawn page cached and readable - and "at least one worked" would call that a +success and never try the other again. + +## Retrying the announcements without republishing + +A scheduled publication is deliberately **two units of work**: + +```text +content-schedule moves the database. Commits, or does not. +content-schedule-effects announces what committed. Retried on its own. +``` + +Retrying them together would re-run the publish - which is idempotent, so the +second run would find nothing changed and skip the announcements entirely. That +is exactly how a scheduled unpublish ends up permanently serving a cached page it +should have expired, and splitting them is what removes it. + +So a retry re-announces and never re-transitions. After a failed effects run and +a successful retry, the record holds the same version, the same `publishedAt` and +exactly one `publish` revision. + +Every reason is reported, not just the first: + +```text +Scheduled publish of example.article#7 committed, but its effects did not +(event: @acme/plugin:notifications:send (down); search: index refused; + cache: 1/2 web origins accepted the invalidation). +``` + +An operator who fixes one system and retries should not discover the next one on +the following run. + +## At-least-once, and what to do about it + +<Callout type="warn" title="There is no exactly-once event delivery"> +A retried effects run emits its event again. A listener can see the same +`content.example.article.published` twice, and there is no outbox and no +deduplication in the transport. +</Callout> + +Search and cache are idempotent by construction - an upsert and an expiry are the +same operation however many times they run, and the document a retry writes is +byte-identical. Events are not, because a listener can do anything. + +A listener that must act once keys off the identifiers the payload carries: + +```ts +buildEventListener({ + name: "announce-on-slack", + event: "content.example.article.published", + handler: async (c, { payload }) => { + // Present only when a schedule caused this, and stable across every retry + // of that booking. + if (payload.scheduleId && (await alreadyAnnounced(c, payload.scheduleId))) { + return; + } + // ... + }, +}); +``` + +## Idempotency, everywhere else + +| Operation | Repeating it | +| --------- | ------------ | +| publish / unpublish | A no-op. No write, no version bump, no revision, no event, no index write. | +| restore to the current values | A no-op, with `revisionId: null`. | +| relation `add` of a target already present | A no-op. | +| relation `remove` of one that is not there | A no-op. | +| repeatable `set` to the same rows and order | A no-op. | +| search upsert | The same document, rewritten. | +| search delete | Removes nothing that is not there. | +| cache expiry | Expires an already-expired tag. | +| scheduled task delivered twice | The claim refuses anything not still `pending`. | +| effects task retried | Re-announces; never re-transitions. | + +That list falls out of one decision rather than ten special cases: every +collection mutation computes the whole next state and diffs it against the +stored one, so "nothing moved" is a single check that every operation shares. + +## What is left outstanding, and where to see it + +- **Per record:** the AdminCP schedule panel shows `effectsError` on the booking. +- **Per install:** `GET /admin/debug/content/status` counts outstanding + scheduled-effect failures per content type, alongside search index drift. See + [observability](/docs/dev/content-engine/content-engine-observability). diff --git a/apps/docs/content/docs/dev/content-engine/index.mdx b/apps/docs/content/docs/dev/content-engine/index.mdx index 0eaa0e221..3a8158ce7 100644 --- a/apps/docs/content/docs/dev/content-engine/index.mdx +++ b/apps/docs/content/docs/dev/content-engine/index.mdx @@ -112,6 +112,11 @@ one object instead of two that drift apart. title="Database & migrations" description="How Drizzle Kit finds generated tables - and what happens when you rename a field." /> + <Card + href="/docs/dev/content-engine/production-hardening" + title="Production hardening" + description="What the engine guarantees under concurrency, partial failure and scale - and what it does not." + /> <Card href="/docs/dev/content-engine/limitations" title="Limitations" diff --git a/apps/docs/content/docs/dev/content-engine/limitations.mdx b/apps/docs/content/docs/dev/content-engine/limitations.mdx index b5859a61a..da8342567 100644 --- a/apps/docs/content/docs/dev/content-engine/limitations.mdx +++ b/apps/docs/content/docs/dev/content-engine/limitations.mdx @@ -149,11 +149,14 @@ API, not a Content Engine quirk. Everything a handler *itself* produces [is documented](/docs/dev/content-engine/permissions#what-each-route-documents), including the unique-conflict 409. -## Index names are checked across Content Types, not across the schema +## Generated names are checked across Content Types, not across the schema The registry sees every registered content type at once, so it catches two of -them resolving to the same index name - across plugins included. It cannot see -anything else in your database: +them resolving to the same physical name - across plugins included. That covers +every name the engine generates: base tables, translation tables, junction +tables, repeatable child tables, declared indexes, and the primary key, position +and target constraints each generated table carries. It cannot see anything else +in your database: - a collision with a **hand-written table's** index is invisible to it, - so is a collision with an index created directly in a migration. @@ -300,12 +303,19 @@ the rows orphaned by `contentTypeId` - the same story as an orphaned search collection, and with the same fix: a one-line `UPDATE` when it was a rename, and a `DELETE` when it was not. -## The public cursor is always the row id +## Pagination is a position, not a snapshot -`withPagination` paginates on the primary key, so a list sorted by `publishedAt` -still pages by `id`. That is pre-existing behaviour shared with every admin -list, not a public-API quirk - but it means two rows with the same -`publishedAt` can order differently between pages. +The cursor encodes the **ordered tuple** - the sort column's value and the row's +identifier - so any orderable column pages exactly, ties included and nulls +included. What it deliberately does not do is freeze the collection: rows +inserted behind the cursor are not seen until the next pass. Holding a +transaction open across requests is the only alternative, and it is a worse one. + +The cursor is opaque and belongs to the ordering that produced it. Replaying one +against a different `orderBy` is a `400` rather than a page of wrong rows, and a +legacy numeric cursor is honoured only where the identifier really is the whole +tuple - a list ordered by `id`. See +[performance and scaling](/docs/dev/content-engine/performance-and-scaling#cursor-pagination). ## Content types are code @@ -320,3 +330,10 @@ migrations. Content events are emitted after a successful write, in-process, with no outbox and no retries. If a listener must not be missed, do the work in the same request or push it onto the [queue](/docs/dev/advanced/queue). + +A listener that fails does not fail the request - the write has already +committed, and reporting 500 would say it was lost when it was not - but it is +never swallowed either: the failure is logged behind `[content-effects]` with the +content type, the item and the listener, and on the scheduled path it is also +recorded on the booking and retried. See +[failure and retries](/docs/dev/content-engine/failure-and-retries). diff --git a/apps/docs/content/docs/dev/content-engine/localization-and-hreflang.mdx b/apps/docs/content/docs/dev/content-engine/localization-and-hreflang.mdx new file mode 100644 index 000000000..4b58fa8e0 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/localization-and-hreflang.mdx @@ -0,0 +1,216 @@ +--- +title: Localization and hreflang +description: Alternates are real published translations and nothing else - so an hreflang never points at a 404, and a fallback never fabricates a URL. +icon: Languages +--- + +A localized record has one URL per language, and a page has to announce the others. +`delivery.alternates()` is that list, and its defining property is what it leaves +out. + +```ts +await delivery.alternates(42); +// [ +// { locale: "en", path: "/en/articles/my-article" }, +// { locale: "pl", path: "/pl/articles/moj-artykul" }, +// ] +``` + +## An alternate is a promise that a URL resolves + +So it is included only when **all** of this holds: + +```text +the base row is published +AND the translation is published +AND publishedAt <= now +AND the installation still serves that language +``` + +That is the same subordinated predicate the +[localized public read](/docs/dev/content-engine/localized-public-api) applies - not +a second implementation of it - so an alternate can never describe something the +public API would refuse to serve. + +Ordered by locale, so two processes rendering the same `hreflang` set produce the +same markup. + +<Callout type="warn" title="A localized content type must expose id"> + Alternates are resolved **by identifier** - the query enumerates a record's published + translations - and delivery reads the public projection, so a localized content type + that withholds `"id"` from `publicApi.fields` is a definition-time error. Without it + every localized response would carry an empty alternate set, which looks exactly like + a record with one translation. A nonlocalized content type has no alternates and needs + nothing. +</Callout> + +## Fallback never fabricates an alternate + +This is the rule to internalise: + +```text +Article #42 + EN published + PL published + DE draft +``` + +```text +alternates: /en/articles/… /pl/articles/… + (no DE) +``` + +A German reader with `fallback: "default"` will be *served* the English copy - that +is what fallback is for. But German has no URL of its own, so listing +`/de/articles/…` would announce an `hreflang` pointing at a 404 and invite a crawler +to index the same content twice under two addresses. + +Fallback decides **which translation answers a request**. It never creates a URL. + +## hreflang + +```ts +const metadata = await delivery.findById(42, { locale: "pl" }); + +metadata.hreflang; +// { +// languages: { en: "/en/articles/my-article", pl: "/pl/articles/moj-artykul" }, +// xDefault: "/en/articles/my-article", +// } +``` + +Framework-neutral by design: `{ languages, xDefault? }` rather than a Next.js +`Metadata` object, because the core engine has no business knowing which framework +renders it. The [Next.js adapter](/docs/dev/content-engine/content-delivery-nextjs) +turns it into `alternates.languages` in one line, and an Astro or Remix adapter would +do the same. + +## x-default + +```ts +delivery: { + enabled: true, + hreflang: { xDefault: "defaultLocale" }, +} +``` + +`"defaultLocale"` is the only supported value, and that is deliberate: an `x-default` +has to point at a URL that actually resolves, and the default locale's canonical path +is the one URL a localized record is guaranteed to have whenever it is public at all. + +It is emitted **only when that language is genuinely published**: + +```text +EN published, PL published → x-default = /en/articles/… +EN unpublished, PL published → no x-default at all +``` + +An `x-default` pointing at a translation the record does not have would be a hint to +crawl a 404 - worse than emitting nothing. + +Omit the block and no `x-default` is emitted. The engine will not invent a +locale-less route it does not serve. + +<Callout type="info" title="It needs localization"> + `delivery.hreflang` on a content type without `localization` is a definition-time + error. One language has no alternates, so there is nothing for an `x-default` to be + the default of. +</Callout> + +## Per-locale slug history + +Each locale's redirects are its own, because `languageId` is part of the history key: + +```text +EN: /en/articles/hello → /en/articles/hello-world +PL: /pl/articles/witaj (unchanged, no redirect created) +``` + +Changing the English URL retires an English address and reserves an English one. The +Polish history is not read and not written. + +The same slug may be retired independently in two locales - `/en/x/shared` and +`/pl/x/shared` are two URLs, so two different records may each own one of them: + +```sql +UNIQUE (contentTypeId, languageId, slug) WHERE languageId IS NOT NULL +``` + +A nonlocalized content type uses `languageId = NULL` and the other partial index. + +### A localized slug is required for redirects + +```ts +// ✖ localized content type, shared slug +fields: { + slug: field.slug({ source: "title" }), // shared + title: field.text({ localized: true, required: true }), +} +delivery: { enabled: true, redirects: { enabled: true } } +``` + +Every language would answer to the same segment, so `/en/x/hello` and `/pl/x/hello` +are both live and one slug change moves both at once. Slug history stores *the URL +that was live*, so one retired row would have to be several paths - and the AdminCP +would show one of them as if it were the address somebody bookmarked. + +Canonical URLs, SEO, alternates and the sitemap all work fine in that shape. Only the +reservation is ambiguous, so only `redirects` is refused. Mark the slug +`localized: true` and everything is available. + +## Unpublishing one language + +```text +EN translation unpublished + /en/articles/hello-world not_found + /en/articles/hello (retired) not_found + /pl/articles/witaj still canonical +``` + +One language going dark is not the record going dark. The resolver reads the live +subordinated publication state per locale, so nothing else is affected - and +republishing the English translation brings its redirects back. + +## Deleting one translation + +The translation's history is **kept**, exactly as a deleted record's is: the URL +existed, and the resolver answers `not_found` for it by finding no live translation +rather than by having forgotten it. + +## The locale is normalized everywhere + +`PL`, `pl` and `" pl "` produce one path, one cache tag and one history lookup. The +canonical spelling always comes back off `core_languages.code`, never the caller's +casing - see [Canonical URLs](/docs/dev/content-engine/canonical-urls#the-locale-is-normalized). + +## Localized sitemaps + +Each language is its own sitemap file, and a draft translation contributes nothing: + +```text +Article #42 EN published PL published DE draft + +/en/articles/article +/pl/articles/artykul +``` + +No fallback URLs, for the same reason there are no fallback alternates. See +[Sitemaps](/docs/dev/content-engine/sitemaps#localized-sitemaps). + +## Events + +A translation slug change emits the delivery events with the locale attached: + +```ts +{ + contentId: 42, + locale: "pl", + previousSlug: "stary-slug", + slug: "nowy-slug", + previousPath: "/pl/articles/stary-slug", + canonicalPath: "/pl/articles/nowy-slug", +} +``` + +They arrive **alongside** `translation_updated`, never instead of it. See +[Slug history and redirects](/docs/dev/content-engine/slug-history-and-redirects#events). diff --git a/apps/docs/content/docs/dev/content-engine/meta.json b/apps/docs/content/docs/dev/content-engine/meta.json index 604d40ee6..71e167fdc 100644 --- a/apps/docs/content/docs/dev/content-engine/meta.json +++ b/apps/docs/content/docs/dev/content-engine/meta.json @@ -36,10 +36,26 @@ "advanced-modeling-public-api", "advanced-modeling-migrations", "advanced-modeling-limitations", + "content-delivery", + "canonical-urls", + "slug-history-and-redirects", + "seo", + "localization-and-hreflang", + "sitemaps", + "content-delivery-nextjs", + "content-delivery-migrations", + "content-delivery-limitations", "admincp", "permissions", "events", "overriding-admincp", + "production-hardening", + "concurrency", + "failure-and-retries", + "content-engine-security", + "content-engine-observability", + "performance-and-scaling", + "migration-hardening", "limitations" ] } diff --git a/apps/docs/content/docs/dev/content-engine/migration-hardening.mdx b/apps/docs/content/docs/dev/content-engine/migration-hardening.mdx new file mode 100644 index 000000000..c81d978a0 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/migration-hardening.mdx @@ -0,0 +1,175 @@ +--- +title: Migration hardening +description: Proving an upgrade is safe before you run it against an install that has rows in it. +icon: DatabaseZap +--- + +A fresh install gets its schema from `drizzle-kit` and that is the end of it. An +existing install is the interesting case: it has rows, and the upgrade has to +move them onto the newer shape without a destructive step nobody reviewed. + +The patterns are on +[localization migrations](/docs/dev/content-engine/localization-migrations) and +[advanced modeling migrations](/docs/dev/content-engine/advanced-modeling-migrations). +This page is about **proving** them, and about the invariants each one has to +hold. + +## The rule every pattern shares + +> The destructive statement lives in a different migration from the copy. + +Two files, because between them somebody has to look at the counts. A backfill +that silently dropped three rows and then deleted its source is not something +anybody can notice afterwards - the evidence went with the source. + +That is why the to-one → to-many pattern is two migrations, why the JSON array → +repeatable pattern is two, and why the localization pattern has a verification +step that aborts rather than continuing. + +## The invariants + +Every upgrade in the test suite checks the same six things. Yours should too. + +| Invariant | How to check it | +| --------- | --------------- | +| **Row count** | `count(*)` on the source and the destination, before the drop. | +| **Nullability** | Every new `NOT NULL` column has a default, or the `ALTER` fails on a populated table. | +| **Foreign keys** | Add them *after* the copy, so a bad reference is a named failure with the data intact. | +| **Uniqueness** | Same: a pre-existing duplicate surfaces at the index, not halfway through a backfill. | +| **Order** | `WITH ORDINALITY` for a JSON array; `position` starting at zero, which is where the engine reads from. | +| **Source intact** | The old column is still there when the first migration ends. | + +## What PostgreSQL gives you for free + +DDL is transactional. A migration file that fails on its third statement leaves +**nothing** behind - not the rows it inserted, and not the table it created. So a +failed migration is always safe to fix and re-run, and the verification step can +`RAISE EXCEPTION` knowing it will take the whole file with it: + +```sql +DO $$ +DECLARE source_count integer; copied_count integer; +BEGIN + SELECT count(*) INTO source_count FROM "example_articles"; + SELECT count(*) INTO copied_count FROM "example_articles_translations"; + + IF source_count <> copied_count THEN + RAISE EXCEPTION 'Copied % of % rows; refusing to drop the source columns.', + copied_count, source_count; + END IF; +END $$; +``` + +<Callout type="warn" title="The exception"> +`CREATE INDEX CONCURRENTLY` cannot run inside a transaction at all, so a +migration that uses one is **not** atomic - a failure leaves an invalid index +behind, and you have to drop it by hand. Nothing the Content Engine generates +uses `CONCURRENTLY`; if you add one for a large table, put it in a migration of +its own and know what it costs. +</Callout> + +## The upgrades that are covered + +Each of these runs against a populated table in +`plugins/example/src/database/migration-postgres.test.ts`. + +### Adding a structured group + +Additive and safe, because `defineContentType` already requires every leaf of an +optional group to be nullable or defaulted: + +```sql +ALTER TABLE "example_articles" + ADD COLUMN "syndicationIndexable" boolean DEFAULT true NOT NULL, + ADD COLUMN "syndicationPriority" integer DEFAULT 5 NOT NULL; +``` + +Every existing row gets the default. A `NOT NULL` column with no default is a +`23502` on a table with rows in it - which is the same rule the definition +enforces, arrived at from the other side. + +### Regrouping columns you already have + +Moving `seoTitle` and `seoDescription` into a `seo` group needs **no data +migration**: `field.text()` named `seoTitle` and `seo.title` compile to the same +column. Generate the migration and check it is empty before believing it. + +### To-one → to-many + +Copy at position 0, verify, then drop in a second migration: + +```sql +INSERT INTO "example_articles_categories" ("itemId", "relatedItemId", "position") +SELECT "id", "category", 0 FROM "example_articles" WHERE "category" IS NOT NULL; +``` + +The junction's foreign key goes on afterwards, so a stale reference is a `23503` +you can look at rather than a half-finished copy. + +### JSON array → repeatable + +`WITH ORDINALITY` is what carries the order across, and `ordinality - 1` is what +puts it where the engine reads from: + +```sql +INSERT INTO "example_articles_faq" ("itemId", "position", "question", "answer") +SELECT a."id", entry.ordinality - 1, + entry.value ->> 'question', entry.value ->> 'answer' +FROM "example_articles" a, + jsonb_array_elements(a."faqJson") WITH ORDINALITY AS entry(value, ordinality) +WHERE a."faqJson" IS NOT NULL; +``` + +Get that wrong and nothing fails - somebody's FAQ is silently reordered. + +### Non-localized → localized + +The longest one, and the only one with a mandatory verification step. Six stages: +create the table, resolve the language id, copy, **verify**, constrain, drop. + +Two details the tests pin: + +- **The timestamps travel with the values.** A translation stamped `now()` would + tell every editor the whole collection was rewritten on deployment day. +- **The language is resolved, not hardcoded.** A literal `1` is right on the + machine it was written on and wrong on every other install. + +And one consequence worth expecting: uniqueness moves from *global* to *per +language*. The base table's old unique slug index goes away with the column, and +`/en/about` and `/pl/about` become two legal rows. + +## Before you run it anywhere real + +1. **Against a copy of production first.** The verification step turns a data + loss into a failed migration, which is the whole point - but a failed + migration is still better discovered on a copy. +2. **Check the counts by hand between the two files.** That pause is the entire + safety mechanism. Automating it away is a product decision made by a script. +3. **Take constraint and index names from the generated file.** They are derived + from your table name and clamped to 63 characters; a name that does not match + makes every future diff noisy. +4. **Never regenerate an applied migration.** It bumps the journal's `when` and + the migrator replays the whole file, which fails on `relation already exists` + at best. Add a new one. + +## Testing your own migration + +Copy the shape of the example suite: build the old table, put rows in it that are +awkward on purpose, run your migration script, and assert the invariants before +the destructive step. + +```ts +it("copies every entry and preserves its order", async () => { + await migrate(CREATE_CHILD); + + const [{ expected }] = await sql` + SELECT coalesce(sum(jsonb_array_length("faqJson")), 0)::int AS expected + FROM "legacy_articles" + `; + expect(await countOf("legacy_articles_faq")).toBe(expected); +}); +``` + +The awkward rows are the ones worth having: a `NULL` array, a duplicate that will +collide with the new unique index, a row whose title normalises to nothing. Every +one of those has been a real migration bug in something, somewhere. diff --git a/apps/docs/content/docs/dev/content-engine/performance-and-scaling.mdx b/apps/docs/content/docs/dev/content-engine/performance-and-scaling.mdx new file mode 100644 index 000000000..5006f6a04 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/performance-and-scaling.mdx @@ -0,0 +1,347 @@ +--- +title: Performance and scaling +description: What a page costs in round trips, why that number does not move when the page grows, and where cursor pagination is exact. +icon: Gauge +--- + +None of the numbers here are milliseconds. A wall-clock figure says more about +the machine than about the code, and a test built on one fails on a busy CI +runner for reasons nobody can act on. + +What is measured instead is **algorithmic**: how many round trips one page costs, +whether that number moves when the page grows, and whether a lookup seeks on an +index or reads the whole table. Those are properties of the SQL, and they are the +ones that decide whether a collection stays usable at a hundred thousand rows. + +## A page costs a bounded number of queries + +The invariant an N+1 breaks is not "few queries" - it is "the *same* number of +queries for five rows and for sixty". + +| Read | Round trips per page | +| ---- | -------------------- | +| Admin list, labels included | bounded - one count, one select, joins inside it | +| Public list | bounded | +| Localized public list | bounded, whatever the locale | +| Public list with advanced collections | bounded, **plus one batch per exposed collection** | +| Revision history page | bounded | +| Search rebuild page | bounded | + +Two things make that hold. + +**Labels are joined, not looked up.** One `LEFT JOIN` per reference field +resolves every display name in the same round trip - there is no per-row lookup +anywhere in the list path. + +**Collections are batched by parent id.** A to-many relation is deliberately +absent from `ContentSelect` for exactly this reason: a list that carried one +would issue a query per row, and a table of 25 rows with two collections would be +50 round trips. `loadMany` takes the whole page's ids and issues one query per +collection field. + +## Only what the projection asks for + +A public read loads the collections the allowlist actually exposes, and no +others. `relatedArticles` is private on the example content type, so a public +list never touches its junction table at all - querying it to discard the rows +afterwards is work with no answer attached. + +The same rule runs through search: `contentSearchAdvancedValues` loads only the +collections the search configuration names. A content type that indexes none - +every Stage 1-5 one - pays a boolean check and no query. + +## Shared collections are loaded once, not once per locale + +The localized rebuild emits one document per published translation, so a record +with three languages appears three times on a page. Its FAQ is *shared*, and +loading it three times would be an N+1 hiding behind a perfectly correct result. + +So the batch loader deduplicates parent ids before it queries. Three translations +of one record read the FAQ once. + +## Cursor pagination + +Pages are keyset reads rather than `OFFSET`, so a page deep into a large +collection seeks on the index instead of counting past every earlier row. + +The cursor is the **ordered tuple**: the sort column's value *and* the row's +identifier. That is not a detail - it is the whole correctness argument. A list +ordered by `title` and a cursor that is only an identifier describe two +different sequences, and a page boundary between them skips rows permanently, +because a short page looks exactly like the end of a collection. + +```text +ORDER BY updatedAt DESC, id DESC + +WHERE updatedAt < :cursorValue + OR (updatedAt = :cursorValue AND id < :cursorId) +``` + +Ascending flips both comparisons; backward pagination (`last`) runs the whole +thing in reverse and flips the page back afterwards. The `ORDER BY` and the +predicate are built from the same tuple in the same direction, which is the +invariant everything else rests on. + +<Callout title="The cursor is opaque, and self-contained"> +It is `base64url(JSON)` carrying the column, the value and the identifier - so +it is meaningless outside the ordering that produced it, and it says so: a +cursor minted while a list was ordered by `updatedAt` and replayed against the +same list ordered by `title` is a `400`, not a page of wrong rows. Hand it back +unchanged; never parse it, and never build one. + +Nothing re-reads the row it came from. A cursor is the position **as it stood +when the page was generated**, so editing or deleting the row that happened to +sit on the boundary does not move it. Re-reading would mean one edit silently +skips every row the ordering used to have between the old position and the new +one - a page of results nobody ever sees, with no error and no short page to +notice it by. +</Callout> + +### The value is captured by the query that returned the row + +**The cursor value comes from the same `SQL` statement as the row it describes. +There is no second lookup of the boundary row.** The page query projects it +alongside the ordinary columns: + +```sql +SELECT title, slug, updatedAt::text AS "__cursorValue" +FROM articles +ORDER BY updatedAt ASC, id ASC +LIMIT 26 +``` + +The reason is the same one that makes a cursor self-contained, applied half a +step earlier. A cursor has to name **the exact ordered tuple the row occupied at +issuance**, and a second `SELECT` after the page has come back is a +time-of-check / time-of-use gap another writer can walk through: + +```text +page query returns id=42, updatedAt=10:00 + ← another writer sets id=42 to 14:00 +boundary lookup id=42, updatedAt=14:00 +cursor issued (14:00, 42) ← the row was never there +``` + +The next page then starts after 14:00 and 10:01, 11:00, 12:00 and 13:00 are gone +for good. A `DELETE` in the same window was worse: the lookup found nothing, the +value became `null`, and `null` is not "no position" - for a nullable ordering it +is a *real* one inside the null block, so the walk jumped there and abandoned the +rest of the collection. + +One statement closes the window rather than locking it. `__cursorValue` is +internal: it is stripped before a row reaches a handler, so it appears in no +response, no OpenAPI schema, no search document and no revision snapshot. A +projection that names only `title` and `slug` still pages by `updatedAt` +correctly, because pagination selects what it needs without widening the +allowlist. + +### Timestamps travel as the database wrote them + +A Postgres `timestamp` keeps microseconds; a JavaScript `Date` keeps +milliseconds. A boundary value that had been through a `Date` would be strictly +*smaller* than the one still in the table, and the next page would exclude the +whole millisecond it came from - which, since `now()` stamps every row in one +statement identically, would end a bulk-imported collection's walk after page +one. + +So a temporal cursor carries the column's own `::text` and the predicate binds it +back with an explicit cast. The value in the cursor is byte-identical to the +value the next comparison is parsed from. + +### A tampered cursor is refused, not coerced + +The cursor is opaque but not signed, so every field is checked against the +column it claims to describe: + +| Column | Accepted | +| ------ | -------- | +| number | a JSON number, finite | +| bigint | a decimal integer string | +| boolean | a JSON boolean | +| string | a JSON string | +| `date` | `2026-08-09` - a day, and nothing more | +| `time` | `10:00:00`, `10:00:00.123456`, plus an offset only if the column has a zone | +| `timestamp`, `timestamptz` | a day, optionally a time, optionally an offset - `2026-08-09 10:00:00.123456+00` | +| any | `null`, which is a real position | + +Coercion is the failure mode being avoided, not an inelegance: `Boolean("false")` +is `true`, `Number("")` is `0`, and `BigInt("nonsense")` throws a `SyntaxError` +that would leave the route answering `500`. Every mismatch is a `400`. + +**A malformed or impossible timestamp cursor is rejected as `400` before it +reaches PostgreSQL.** Shape is not enough for a temporal value, because a shape +check cannot tell a day from a date that looks like one: + +```text +2026-13-01 2026-00-01 2026-02-30 2025-02-29 +2026-01-32 2026-08-09 24:00:00 2026-08-09 23:60:00 +2026-08-09 23:59:61 2026-08-09 10:00:00+25:00 +``` + +Every one of those matches `YYYY-MM-DD HH:MM:SS` and none of them is a moment, +so Postgres answers the cast with `invalid input syntax` - a `500` produced by a +query string. So the components are range-checked as well: month 1-12, the day +count for that month *in that year* (2024-02-29 is a day, 2025-02-29 is not), +hours 0-23, minutes and seconds 0-59, and an offset within ±15:59:59. + +The grammar is derived from the column's SQL type rather than from Drizzle's +JavaScript one, because the two disagree exactly where it matters: `date()` and +`time()` hand back plain strings, so a value bound straight through would reach +Postgres as `'nonsense'::date`. Validation is deliberately **stricter** than +Postgres in one place - `24:00:00` is a valid input to Postgres and something its +own `::text` never writes, so a cursor carrying one did not come from a row. + +The original string is what gets bound; nothing is reformatted, so microseconds +survive validation digit for digit. + +<Callout title="The supported temporal domain"> +Temporal cursor pagination intentionally supports **AD years 1 through 294276** +and does not support PostgreSQL BC-era representations. Values outside that +domain are rejected with `400`. + +This is a deliberate limit on the accepted grammar, not an inability to produce +one: cursor values are read straight from `orderColumn::text`, so a row genuinely +holding a BC-era or out-of-range timestamp would mint a cursor the next request +then refuses. Narrowing the domain keeps validation something that can be read +and reasoned about; widening it is a decision, not a fix. +</Callout> + +**What is guaranteed:** + +- **any** orderable column pages exactly - a title, a nullable `publishedAt`, a + custom field, whatever its relationship to the identifier; +- the boundary is **stable**: updating or deleting the row the cursor named does + not move it, and every row that was after that position is still reachable - + including when the change lands *between* the page query and the cursor being + minted, because there is nothing in between; +- rows sharing a sort value are returned exactly once, because the identifier is + the tiebreaker in both the ordering and the predicate; +- a nullable order column walks through its null block correctly - Postgres sorts + `NULLS LAST` ascending and `NULLS FIRST` descending, and the predicate names + the block rather than letting `column > NULL` end the walk early; +- a row already returned never comes back, and the cursor always advances, so a + loop always terminates; +- a page never claims a neighbour it cannot hand out a cursor for; +- a public page is capped at the server's ceiling however large a `first` an + anonymous caller sends. + +**What is not:** + +<Callout type="warn" title="A cursor is a position, not a snapshot"> +Rows inserted behind the cursor are not seen until the next pass, and a row that +is *edited* so that it moves from behind the cursor to ahead of it is seen a +second time. That is the trade every keyset pagination makes, and it is the right +one - the alternative is holding a transaction open across requests. What never +happens is a row being skipped because something *else* moved. +</Callout> + +### Invalid input is refused, not repaired + +`first=0` used to clamp its way into a one-row page that reported +`hasNextPage: true`; `first=abc` became `NaN` and fell through to the default +page size. Both are requests nobody made, answered as if they had. Every one of +these is now a `400`, most of them from the route's own schema: + +```text +first=0 last=0 first=-1 last=-1 +first=abc last=abc first=1.5 +first=5&last=5 +cursor=<anything that is not a cursor> +``` + +A legacy numeric cursor still works in the one place it was ever correct - +a list ordered by its identifier, where the number really is the whole tuple. +Anywhere else it is refused rather than guessed at. + +## Indexes + +Everything a generated query needs is generated with it, and the definitions are +asserted rather than assumed: + +| Query | Index | +| ----- | ----- | +| `findById` | the primary key | +| `findBySlug` | the slug's unique index | +| localized slug lookup | `UNIQUE (languageId, slug)` on the translation table | +| public list | `(status, publishedAt)` from the publication block | +| revision history | `UNIQUE (contentTypeId, itemId, version)` | +| translation history | the partial unique index on `(…, languageId, version)` | +| relation membership `EXISTS` | the junction's primary key `(itemId, relatedItemId)` | +| reverse relation lookup | `(relatedItemId)` - Postgres does not index the child side of a foreign key on its own | +| repeatable `loadMany` | `UNIQUE (itemId, position)` | +| schedule claims | `(status, scheduledFor)` and `(contentTypeId, itemId)` | + +The plans are checked too, on a table large enough for the planner to have a real +choice: a slug lookup and an identifier lookup both seek, and a revision history +page does not scan. Nothing asserts a *cost* - planner costs move between majors +for reasons that are none of the engine's business. + +## Adding an index of your own + +```ts +indexes: [ + { on: ["status", "createdAt"] }, + // Leaf paths work, and compile against the generated column. + { on: ["syndication.priority"] }, +], +``` + +Names are derived from the table and clamped to Postgres' 63 characters with a +fingerprint, and a collision anywhere in the schema is a **boot failure** rather +than a migration that fails at deploy time. That includes the names nobody wrote +down: a junction's primary key, its position constraint, its target index and a +repeatable's position constraint. + +## A rebuild walks by key, not by offset + +Both generated indexers page with a cursor: `(itemId, languageId)` for the +localized one, `id` for the other. `OFFSET` was wrong twice over - it re-reads +and discards every earlier row, so a deep page pays for every page before it, +and the offset counts rows in a set that is *moving*. A record unpublished after +page one shifts everything behind it forward by one, and the next `OFFSET 100` +steps straight over a row nobody ever indexed. + +<Callout type="warn" title="A rebuild is not a snapshot"> +It walks the collection as it stands. A row published **ahead** of the cursor is +picked up by the same pass; one published behind it is not, and is indexed by its +own publish instead. A row unpublished before the walk reaches it is simply never +read; one unpublished after it was read has already had its document written, and +the live unpublish is what removes it. Nothing is skipped that the walk had not +already passed. +</Callout> + +## Reads stay page-bound + +A page is a page whatever the table holds behind it: `first: 25` materialises 25 +rows, and `totalCount` is an aggregate rather than a fetch. Nothing in the list +path reads a collection whole. + +The one read that deliberately loads everything is `findDetail`, which is what an +edit form needs - one record, with its collections attached. A list must never +call it, and does not. + +## Where the limits actually are + +- **A repeatable is a handful of rows a person edits in one form.** The ceiling + is 200 and the default is 50; model a content type for anything larger. +- **A to-many relation is capped** for the same reason - it is edited as a set in + a picker, not paged. +- **A rebuild is a queue task**, paged by key rather than by `OFFSET`, so page + five hundred costs what page one costs. It is not free; run it when you need it + rather than on a schedule. +- **`totalCount` is a `COUNT(*)`** on every list page. On a very large table with + a filter that no index covers, that is the expensive part of the request - and + the fix is an index on the filter, not a change to the pagination. + +## Running the scale suite + +```bash +DATABASE_TEST_URL=postgres://postgres:postgres@localhost:5432/vitnode_test \ + pnpm --filter @vitnode/example test +``` + +`performance-postgres.test.ts` seeds a few thousand rows rather than the ten +thousand a plan might suggest, because every property above is visible at any size +above "a handful" - and a suite nobody waits for is a suite nobody runs. Where +scale genuinely matters, because a sequential scan is cheaper than an index on a +tiny table, the fixture is grown until the planner has a real choice to make. diff --git a/apps/docs/content/docs/dev/content-engine/production-hardening.mdx b/apps/docs/content/docs/dev/content-engine/production-hardening.mdx new file mode 100644 index 000000000..ea0e42967 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/production-hardening.mdx @@ -0,0 +1,108 @@ +--- +title: Production hardening +description: What the Content Engine guarantees under concurrency, partial failure and scale - and, just as importantly, what it does not. +icon: ShieldCheck +--- + +Stages 1-6 built the Content Engine. This page is the other half of the story: +what happens when two editors press save at the same second, when the search +engine is down, when a table has a hundred thousand rows in it, and when the +definition a revision was written under no longer exists. + +None of it is new functionality. It is the set of promises the engine makes, each +one written down next to the test that proves it - because a guarantee nobody +can point at is a hope. + +## The promises, in one place + +| Area | Guarantee | +| ---- | --------- | +| **Concurrency** | Two writers carrying the same `expectedVersion` produce exactly one winner and one structured conflict. | +| **Atomicity** | The content write, its version bump, its revision and its collection rows are one transaction. | +| **Effects** | Events, search and cache invalidation run **after** the commit and can never undo it. | +| **Delivery** | At-least-once where a retry is involved, best-effort otherwise. Never exactly-once. | +| **Search** | The database is the source of truth. Live synchronisation and a rebuild produce the same document. | +| **Cache** | A mutation expires exactly the tags it touched, per locale, and nothing global. | +| **Security** | Every generated route carries a staff permission; a public response is an allowlist, not a filter. | +| **Errors** | Every expected constraint failure has a stable status and code. No SQLSTATE reaches a client. | +| **Pagination** | The cursor is the ordered tuple, so any orderable column pages exactly - ties and nulls included. | +| **Revisions** | History is append-only and per-record monotonic, across deletes and recreations. | +| **Boot** | A generated table, index or constraint name that would collide is a boot failure, not a Tuesday. | + +Each one has a page of its own: + +- [Concurrency](/docs/dev/content-engine/concurrency) - the race matrix and who wins. +- [Failure and retries](/docs/dev/content-engine/failure-and-retries) - what survives an outage, and what gets retried. +- [Security](/docs/dev/content-engine/content-engine-security) - permissions, privacy, preview. +- [Observability](/docs/dev/content-engine/content-engine-observability) - what an operator can diagnose. +- [Migration hardening](/docs/dev/content-engine/migration-hardening) - upgrading an install that has rows in it. +- [Performance and scaling](/docs/dev/content-engine/performance-and-scaling) - pagination, query counts, N+1. + +## What is *not* promised + +Worth reading before the rest, because most production surprises come from a +guarantee somebody assumed rather than one that broke. + +<Callout type="warn" title="No exactly-once event delivery"> +There is no outbox. A retried scheduled effect re-emits its event, so a listener +can see the same `published` twice. Payloads carry a `scheduleId` precisely so a +listener that must act once can key off it. See +[failure and retries](/docs/dev/content-engine/failure-and-retries). +</Callout> + +- **No frozen dataset across pages.** A cursor is a position in an ordering, not + a snapshot. Rows inserted behind the cursor are not seen; rows already returned + never come back. The *ordering itself* is exact for any orderable column - the + cursor encodes the whole ordered tuple. +- **No cross-request transaction.** A mutation is atomic. A *workflow* made of + several mutations is not, and nothing here pretends otherwise. +- **No automatic schema migration.** Migrations are generated by `drizzle-kit` + and committed. The engine creates no schema at runtime. +- **No field-level permissions.** Permissions are per content type and per + operation. A translator is limited by *locale*, not by field. +- **No repair of a search index that is merely stale.** Drift is *detected* by + counting; correcting it is a rebuild, which is a decision an operator makes. + +## Running the hardening suites + +Everything on these pages is tested against a real PostgreSQL, because none of +it can be shown with a mock - a lock wait, a guarded `UPDATE` that matches +nothing and a `DELETE` that commits mid-transaction are all database behaviour. + +```bash +DATABASE_TEST_URL=postgres://postgres:postgres@localhost:5432/vitnode_test \ + pnpm --filter @vitnode/example test +``` + +<Callout type="warn" title="It wipes the database"> +The suites drop and recreate the schema, and refuse to start unless the database +name contains "test". +</Callout> + +The files, and what each one is for: + +| File | Covers | +| ---- | ------ | +| `concurrency-postgres.test.ts` | Every race in the matrix, on two connections. | +| `resilience-postgres.test.ts` | Event, search and cache failures; idempotency; drift. | +| `integrity-postgres.test.ts` | Delete cascades, schema-evolution restore, disabled locales. | +| `performance-postgres.test.ts` | Pagination, query counts, index use, batch loading. | +| `migration-postgres.test.ts` | The documented upgrade patterns, against real rows. | + +Without `DATABASE_TEST_URL` they skip, loudly, rather than passing quietly. + +## Supported PostgreSQL versions + +PostgreSQL 17 and 18 are both supported, and the suites are **version-aware** +rather than lenient. The one difference that reaches an API contract: + +```text +ON DELETE RESTRICT violated + PostgreSQL 18 and later -> SQLSTATE 23001 (restrict_violation) + earlier majors -> SQLSTATE 23503 (foreign_key_violation) +``` + +Both map to the same `409`, so upgrading the database does not change what a +client sees. The tests assert the *correct* code for the server they are running +against - "one of these two" would still pass if a future major stopped refusing +the delete entirely. diff --git a/apps/docs/content/docs/dev/content-engine/seo.mdx b/apps/docs/content/docs/dev/content-engine/seo.mdx new file mode 100644 index 000000000..fdc76d4a6 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/seo.mdx @@ -0,0 +1,232 @@ +--- +title: SEO metadata +description: Project a title, a description, Open Graph and a robots directive out of fields the public API already exposes - with explicit fallbacks and no invented text. +icon: Search +--- + +`delivery.seo` names which **public** fields become which piece of page metadata. +It projects; it never invents. + +```ts +delivery: { + enabled: true, + seo: { + titleField: "seo.title", + fallbackTitleField: "title", + descriptionField: "seo.description", + fallbackDescriptionField: "excerpt", + noIndexField: "syndication.noIndex", + openGraph: { + titleField: "seo.title", + descriptionField: "seo.description", + }, + }, +} +``` + +## Every field has to be public + +```ts +seo: { titleField: "internalNote" } +// ✖ Type error, and a definition-time throw +``` + +A `<title>` is rendered into a public page, so it has to be something the public API +would already have said out loud. That is a **compile error** as well as a runtime +one - `ContentDeliveryTitleField` extracts from `publicApi.fields`, so an +unexposed name is not in the union at all. + +It is also true at runtime for free, and this is the part worth understanding: SEO +is projected from the **public projection** rather than from the base row. A private +column is never fetched, so it is not in the object the projection returns - the +metadata cannot reach it even by mistake. + +## Field kinds + +| Slot | Kinds | Why | +| -------------------------- | -------------------- | ---------------------------------------------- | +| `titleField` | `text` | A `<title>` is one line, not a paragraph | +| `descriptionField` | `text`, `textarea` | Prose is exactly what a description is | +| `noIndexField` | `boolean`, **shared** | Two answers, one canonical decision | + +A **repeatable leaf** is refused in every slot: a page has one title and a +repeatable has many values. A **group leaf** is accepted everywhere - `seo.title` is +one column under a generated name, so it is one value. + +```ts +fields: { + seo: field.group({ + localized: true, + nullable: true, + fields: { + title: field.text({ nullable: true, maxLength: 200 }), + description: field.textarea({ nullable: true, maxLength: 500 }), + }, + }), +} +``` + +Paths use the same canonical dotted form the rest of the engine speaks - see +[Structured fields](/docs/dev/content-engine/structured-fields). There is no second +flatten/unflatten implementation here; delivery reads the projected row through +`readContentPath`, exactly as search does. + +## Fallbacks are explicit + +```ts +seo: { + titleField: "seo.title", + fallbackTitleField: "title", +} +``` + +The fallback is consulted when the primary resolves to `null` or to whitespace - a +`<title>` of three spaces is a missing title with extra steps. That covers the common +case exactly: nobody writes an SEO title twice, so `seo.title` is usually empty and +the article's real `title` is what should appear. + +There is deliberately **no** "derive a description from the first 160 characters of +the body". A summary somebody did not write is a summary nobody reviewed, and it +would silently become the description of every page that forgot to set one. + +<Callout type="warn" title="A fallback with no primary is refused"> + `fallbackTitleField` without `titleField` reads as if it does something and does + nothing: the fallback is only reached when the primary is empty, so on its own it + is never consulted. Name it as `titleField` instead. +</Callout> + +## The result + +```ts +const metadata = await delivery.findById(42, { locale: "pl" }); + +metadata.seo; +// { title: "Mój artykuł", description: "…" } +``` + +The shape is stable whether or not the block was configured - a content type that +names nothing gets `{ title: null, description: null }` - so a frontend never +branches on "was SEO set up", only on "did a value come back". + +## Open Graph + +```ts +seo: { + titleField: "seo.title", + openGraph: { titleField: "social.title" }, +} +``` + +`null` when the content type configured none, and that is a different fact from "it +did, and this page has no title" - a renderer treats them differently, because the +first emits no tags at all. + +Each Open Graph slot falls back to the ordinary SEO one, which makes the common case +- the same title in both places - a two-line config: + +```ts +openGraph: {} // inherits titleField and descriptionField +``` + +<Callout type="info" title="No og:image"> + Stage 8 does not implement a media subsystem, and an `og:image` needs one: an + absolute URL, known dimensions and a stable content type for the file. Emit it from + your own `generateMetadata` alongside the delivery metadata - see + [Limitations](/docs/dev/content-engine/content-delivery-limitations). +</Callout> + +## Robots and noindex + +```ts +seo: { noIndexField: "syndication.noIndex" } +``` + +```ts +metadata.robots; +// { index: false, follow: true } +``` + +One boolean drives **two** consumers, and that is the whole reason it exists as a +single field: a record excluded from the sitemap and a record reporting `index: +false` have to be the same record. Two settings would eventually disagree. + +`follow` is always `true`. "Do not list this page" and "do not follow the links on +it" are different instructions, and a content type that asked for the first has not +asked for the second - a `noindex, nofollow` page is a dead end for a crawler walking +the site, which is a decision for site-wide robots configuration rather than for one +record. + +`null` when no `noIndexField` is configured, so a content type that never thought +about indexing emits no `robots` meta tag rather than an affirmative "yes, index +this". + +### It has to be shared + +```ts +// ✖ a localized group's leaf +seo: { noIndexField: "flags.noIndex" } +``` + +A localized boolean would give one record one answer per language while it has a +single canonical decision - and the sitemap exclusion and the `robots` directive +would then be able to disagree. Delivery refuses it at definition time. + +Per-locale indexing is a real thing to want; it is deferred rather than approximated. +See [Limitations](/docs/dev/content-engine/content-delivery-limitations). + +## Localized SEO + +A localized group gives every language its own copy: + +```ts +fields: { + seo: field.group({ localized: true, nullable: true, fields: { … } }), +} +``` + +```text +en: { title: "English SEO", description: "English summary" } +pl: { title: null, description: null } → falls back to the + Polish `title` +``` + +The fallback is **that language's own** field, never English's. The projection reads +one row - the translation being served - so it has nothing else to reach for. + +On a fallback read the metadata reports the locale it actually served, and the +canonical URL follows it: + +```ts +{ + requestedLocale: "pl", + locale: "en", + isFallback: true, + canonicalPath: "/en/articles/article", + seo: { title: "The English title", … }, +} +``` + +See [Canonical URLs](/docs/dev/content-engine/canonical-urls#the-canonical-url-is-the-served-locale). + +## SEO has no revision history of its own + +SEO is derived from content fields, so it already participates in +[revisions](/docs/dev/content-engine/revisions): + +```text +restore a revision that had seo.title = "Old heading" +→ seo.title is "Old heading" again +→ the delivery metadata says so on the next read +``` + +There is no second history to keep in sync, and restoring SEO is not a separate +operation. That is the reason `delivery.seo` names fields rather than storing values. + +## Search + +Changing an SEO field already triggers +[search synchronization](/docs/dev/content-engine/search) when the field is one +`search` indexes - there is no second indexing path, and delivery adds none. What +delivery guarantees is narrower and worth stating: a search document carries the +**current** canonical URL, and a historical URL never becomes a second document +competing with the page it redirects to. diff --git a/apps/docs/content/docs/dev/content-engine/sitemaps.mdx b/apps/docs/content/docs/dev/content-engine/sitemaps.mdx new file mode 100644 index 000000000..21952d9ac --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/sitemaps.mdx @@ -0,0 +1,293 @@ +--- +title: Sitemaps +description: A cursor-paginated service that lists what is public right now, plus pure helpers that turn its entries into valid XML and a sitemap index. +icon: Map +--- + +```ts +delivery: { + enabled: true, + sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 }, +} +``` + +That is the whole configuration. The engine does not build a file - it answers "which +URLs are public right now", one page at a time, and leaves serialization to a pure +helper. + +## Querying and serializing are separate + +Deliberately, and it is the design decision that makes both testable: "which URLs are +public" is a keyset scan over two tables, and "what does a sitemap file look like" is +a string. Folded into one function, the XML would be untestable without a database +and the pagination untestable without parsing XML. + +```ts +// the query +const page = await delivery.sitemap({ locale: "pl", limit: 1_000 }); + +// the serialization +import { contentSitemapXml } from "@vitnode/core/content"; + +const xml = contentSitemapXml({ + entries: page.entries, + origin: "https://example.com", +}); +``` + +## One page of entries + +```ts +await delivery.sitemap({ cursor, limit, locale }); +``` + +```ts +{ + entries: [ + { + itemId: 42, + locale: "pl", + path: "/pl/articles/moj-artykul", + lastModified: new Date("2026-01-02T03:04:05.000Z"), + changeFrequency: "weekly", + priority: 0.7, + }, + ], + nextCursor: 42, // pass back as `cursor`; null on the last page +} +``` + +### Cursors, never offsets + +`cursor` is the last `itemId` of the previous page, and pagination is a keyset over +the primary key. An `OFFSET` deep into a large table both slows down linearly *and* +skips rows when something is published between two pages - and a sitemap is +regenerated from scratch every time a crawler asks, so both matter. + +Ordering is `ORDER BY id ASC`, which makes the output deterministic: no duplicates, no +gaps, and the same document from two processes. + +`limit` defaults to 1,000 and is capped at the protocol's 50,000. A page is one keyset +query plus one batched read, so 1,000 rows is a response a serverless function can +hold without thinking about it; a caller that wants a whole 50,000-URL file asks for +it explicitly. + +### Only what is public right now + +The publication predicate is not a parameter: + +```text +nonlocalized the base row published +localized the base row AND the translation published +``` + +A draft, an unpublished record and a `publishedAt` in the future are all simply +absent. The localized form is the same subordination the +[public read](/docs/dev/content-engine/localized-public-api) applies. + +## Localized sitemaps + +Each published translation is one URL, and each language is its own file: + +```text +Article #42 EN published PL published DE draft + +/en/articles/article +/pl/articles/artykul +``` + +No DE, and **no fallback URLs**. A locale served English through +`fallback: "default"` has no URL of its own, so listing one would put the same +content in the sitemap twice under two addresses. + +A locale that names no language this install serves gets an empty page rather than an +error - a crawler asking for `/sitemaps/blog.article-de.xml` on a site with no German +should get a valid empty document. + +## lastModified + +| Content type | Value | +| -------------- | -------------------------------------------- | +| nonlocalized | `base.updatedAt` | +| localized | `max(base.updatedAt, translation.updatedAt)` | + +The localized rule is the interesting one: both halves are rendered into the page, so +a **shared** field moving changes what every language's document says even though no +translation row was touched. Taking the translation's timestamp alone would tell a +crawler nothing had changed. + +<Callout type="info" title="Timestamps are read through the column's own decoder"> + Drizzle turns off the driver's timestamp parsing so its column mappers can treat a + naive `timestamp` as UTC. A raw `sql` fragment has no mapper, so the driver's + fallback parses the same value as *local* time - the two disagree by the server's + offset. The `greatest()` expression borrows the column's decoder with `.mapWith`, + which is why a localized `lastmod` is not hours out. The PostgreSQL suite asserts + it. +</Callout> + +## changeFrequency and priority + +Static, per content type, and validated: + +```ts +sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 } +``` + +`changeFrequency` must be one of the seven values the protocol defines - `always`, +`hourly`, `daily`, `weekly`, `monthly`, `yearly`, `never`. A crawler ignores an +unknown value silently, so a typo has to be a compile error or it is a hint nobody +ever receives. + +`priority` must be between `0` and `1` inclusive, and is emitted at one decimal place. + +Both are omitted from the XML when unset, which is valid. There are deliberately no +per-record dynamic callbacks: a function that runs once per URL in a 50,000-URL file +is a performance decision disguised as a configuration option. + +### It is also what the cache tag follows + +Because `lastModified` comes from `updatedAt`, a real edit to a published record changes +that locale's sitemap file even when its URL does not move - so the file's cache tag is +expired for a plain title or SEO edit, not only for a publish or a slug change: + +```text +title edit on a published record +→ updatedAt moves +→ <lastmod> moves +→ sitemap file tag expired (the index is not) +``` + +A no-op edit writes no `UPDATE`, so `updatedAt` does not move and the cached file is +still byte-correct. See [Caching](/docs/dev/content-engine/caching#delivery-tags) for +the full matrix and for the file-versus-index distinction. + +## Excluding one record + +```ts +seo: { noIndexField: "syndication.noIndex" } +``` + +`noIndex = true` removes the record from the sitemap **and** reports +`robots: { index: false }` - one boolean behind both, so they cannot disagree. It is a +single clause in the query rather than a post-filter, so a page of 1,000 entries is +1,000 listed URLs rather than however many survived. + +The field must be a shared boolean. See [SEO](/docs/dev/content-engine/seo#robots-and-noindex). + +## The XML helper + +```ts +import { contentSitemapXml } from "@vitnode/core/content"; + +contentSitemapXml({ entries, origin: "https://example.com" }); +``` + +```xml +<?xml version="1.0" encoding="UTF-8"?> +<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> + <url> + <loc>https://example.com/articles/my-article</loc> + <lastmod>2026-01-02T03:04:05.000Z</lastmod> + <changefreq>weekly</changefreq> + <priority>0.7</priority> + </url> +</urlset> +``` + +`origin` is required rather than optional: the protocol only accepts absolute URLs, so +this is the one place delivery cannot stay origin-agnostic. An entry whose path will +not resolve against it is **dropped** rather than emitted - one malformed `<loc>` is a +document a crawler may reject whole. + +XML's five predefined entities are escaped, with `&` first: escaping it after `<` would +turn the `<` just produced into `&lt;`. + +### hreflang inside a sitemap + +```ts +contentSitemapXml({ + alternates: await readDeliveryAlternatesMany({ c, itemIds, model }), + entries, + origin: "https://example.com", +}); +``` + +```xml +<url> + <loc>https://example.com/en/articles/my-article</loc> + <xhtml:link rel="alternate" hreflang="en" href="https://example.com/en/articles/my-article" /> + <xhtml:link rel="alternate" hreflang="pl" href="https://example.com/pl/articles/moj-artykul" /> +</url> +``` + +Opt-in, and standards-compliant: the namespace is declared on the root element, and +every alternate of a group is repeated inside **each** of its `<url>` entries - +including the entry's own. That last rule is the one implementations get wrong, and it +is why alternates are supplied per entry rather than derived: the caller has already +resolved which translations are published, and the serializer does not go looking. + +`readDeliveryAlternatesMany` batches a whole page into one query rather than one per +URL. + +## Scaling past one file + +```ts +import { contentSitemapChunks, contentSitemapIndexXml } from "@vitnode/core/content"; + +const { pages, size } = contentSitemapChunks({ total, size: 1_000 }); +``` + +```text +/sitemap.xml the index +/sitemaps/blog.article-1.xml +/sitemaps/blog.article-2.xml +``` + +```ts +contentSitemapIndexXml({ + entries: Array.from({ length: pages }, (_, page) => ({ + path: `/sitemaps/blog.article-${page + 1}.xml`, + })), + origin: "https://example.com", +}); +``` + +`pages` is at least `1` even for an empty content type, and that is on purpose: an +index that lists a file which does not exist is a broken index, and a content type +with nothing published today will have something tomorrow. `size` is clamped to the +protocol's 50,000-URL ceiling, so a caller cannot ask for one enormous invalid file. + +`contentSitemapIndexXml` emits `<sitemapindex>` with `<sitemap>` children - a separate +function from `contentSitemapXml` because it is a separate document type, and because +an index whose entries were `<url>` elements is the single most common way to publish a +sitemap no crawler reads. + +## A site-level sitemap + +```ts +import { listDeliveryContentTypes } from "@vitnode/core/content"; + +const delivered = listDeliveryContentTypes(core.contentModels); +``` + +Every delivery-enabled content type of the installation, sorted by id. Build one index +entry per content type per locale per chunk, and no plugin name is ever hardcoded - +installing a plugin adds its URLs and removing it takes them out again. + +## Memory + +Nothing here loads a content type whole: + +- one keyset page at a time, bounded by `limit`; +- one batched translation read per page, never one per row; +- one batched alternates read per page, when alternates are asked for; +- `noIndex` as a `WHERE` clause rather than a post-filter. + +The PostgreSQL suite pages a fixture in twos and asserts every record appears exactly +once, in ascending key order, with the cursor ending at `null`. + +## Next.js + +`contentSitemapEntries` pages through the delivery route and returns entries a +`sitemap.ts` can return directly. See +[Next.js helpers](/docs/dev/content-engine/content-delivery-nextjs#sitemap). diff --git a/apps/docs/content/docs/dev/content-engine/slug-history-and-redirects.mdx b/apps/docs/content/docs/dev/content-engine/slug-history-and-redirects.mdx new file mode 100644 index 000000000..4c7390353 --- /dev/null +++ b/apps/docs/content/docs/dev/content-engine/slug-history-and-redirects.mdx @@ -0,0 +1,387 @@ +--- +title: Slug history and redirects +description: Every URL a record was ever reachable at is recorded, reserved, and redirected to the current one - with no chains and no stolen addresses. +icon: CornerDownRight +--- + +Change the slug of a published article and its old URL stops existing. Every link +to it, every bookmark, every search result: gone. `delivery.redirects` is the block +that fixes that. + +```ts +delivery: { + enabled: true, + redirects: { enabled: true }, +}, + +// Required. See below. +editorial: { enabled: true }, +``` + +From then on: + +```text +current: /articles/stary-slug +update: slug = nowy-slug + +after commit: + /articles/nowy-slug canonical + /articles/stary-slug 308 -> /articles/nowy-slug +``` + +## Redirects require Editorial + +```ts +delivery: { enabled: true, redirects: { enabled: true } } +// ✖ without `editorial: { enabled: true }` +``` + +Slug history has to be written in the **same transaction** as the slug mutation, its +version check and its revision - otherwise a committed slug change could leave the old +URL unreserved, or a reservation could survive a rolled-back write. The only mutation +paths that own such a transaction are the editorial ones; without `editorial` a content +type writes through the plain repository, which has no version to guard and no history +to write. + +So `redirects` without `editorial` would be a feature that silently records nothing. +It is a **compile error** and a definition-time throw rather than a silent downgrade to +`redirects: { enabled: false }` - an author who asked for redirects and quietly got none +would find out from a broken link months later. + +The restriction is narrow. Everything else in `delivery` is a read over data the content +type already has, and stays available without `editorial`: + +| Feature | Needs Editorial? | +| -------------------------------- | ---------------- | +| Canonical URLs | ❌ | +| SEO / Open Graph / robots | ❌ | +| Alternates and `hreflang` | ❌ | +| Sitemap | ❌ | +| Delivery reads and the AdminCP panel | ❌ | +| **Slug history and redirects** | ✅ | + +The same rule applies to a localized content type, and for the same reason: localized +slug history is written by `translation-editorial-service`, which a content type without +`editorial` does not have either. + +## When a slug becomes redirectable + +This is the rule the whole feature rests on, so it is stated exactly: + +> A slug becomes redirectable only if it was **previously used by an addressable +> public version** - that is, the record (or the translation) was published while +> that slug was current. + +The consequence is the useful part: + +| Situation | Redirect? | +| ------------------------------------------------------ | --------- | +| Published as `a`, changed to `b` | ✅ `a → b` | +| Draft created as `a`, corrected to `b`, then published | ❌ none | +| Published as `a`, unpublished, changed to `b` | ❌ none *yet* | +| …then republished | ✅ `a → b` | + +A draft whose slug was corrected three times before anybody saw it produces no +redirects at all, because none of those URLs was ever live. Without that rule a +content type would accumulate a redirect per typo, and each one would be a +permanent claim on an address nobody had visited. + +## What is stored + +One shared table, `core_content_slug_history`: + +```text +id +pluginId +contentTypeId +itemId +languageId NULL for a shared slug +slug +path the URL exactly as it was live +createdAt +retiredAt NULL while this slug is the record's current address +``` + +Both states are stored - current *and* retired - and that is what makes the +uniqueness below a **reservation** rather than only a log. + +`path` is recorded rather than rebuilt on read, because it is the one thing the +engine cannot recompute later: a path is built from `publicApi.path`, which is +source configuration a developer may change. The URL that was live is a historical +fact, so it is kept as one - and the AdminCP shows exactly the address somebody's +bookmark holds. + +<Callout type="info" title="No foreign key to the record"> + Same reasoning as `core_content_revisions`: the target table is generated at + runtime, so core's static schema cannot name it. Every query is scoped by + `(contentTypeId, itemId)`, and a URL's history stays true after the record is + gone. +</Callout> + +## Historical addresses are reserved + +```text +Article 1: old slug = hello current = hello-world +Article 2: slug = hello ✖ CONTENT_DELIVERY_SLUG_RESERVED +``` + +`hello` is free on the content table - Article 1 moved off it - so without the +reservation Article 2 could take it, and `/articles/hello` would silently stop +redirecting to the article it belonged to and start resolving to an unrelated one. +Every link, bookmark and search result pointing at it would change meaning. + +So a historical public slug stays reserved for the **content type and locale** that +retired it, for as long as redirects are enabled. Two partial unique indexes enforce +it: + +```sql +UNIQUE (contentTypeId, slug) WHERE languageId IS NULL +UNIQUE (contentTypeId, languageId, slug) WHERE languageId IS NOT NULL +``` + +Two indexes rather than one over a nullable column, because Postgres treats every +`NULL` as distinct - a single key including `languageId` would enforce nothing at +all for the shared case it exists to protect. + +The refusal is a structured **409**, not a raw constraint error: + +```json +{ + "code": "CONTENT_DELIVERY_SLUG_RESERVED", + "contentTypeId": "example.article", + "locale": null, + "slug": "hello" +} +``` + +It carries no owning-record id on purpose: a 409 on a public-facing address must not +become a way to enumerate records the caller cannot read. + +A record may always take **its own** retired address back - moving from `b` to `a` +re-activates its own row rather than colliding with it. + +<Callout type="warn" title="A draft's slug is checked, not reserved"> + Taking a new slug on a draft checks the reservations - so an editor hears "that + address is taken" at save time rather than at publish time - but claims nothing. + A draft has no public URL, and reserving one would refuse a live address to + somebody who wants it. +</Callout> + +## Resolution collapses chains + +```text +a → b +b → c + +request a → c (one hop) +request b → c (one hop) +``` + +The database keeps the chronology - three rows, two retired - and the *resolver* is +what collapses it. It never follows the history: it looks the address up, finds the +record it belongs to, and reads that record's **current** slug. There is no second +hop to make. + +```ts +await delivery.resolvePath("/articles/a"); +// { type: "redirect", status: 308, location: "/articles/c" } +``` + +## 308, and only 308 + +`308 Permanent Redirect` rather than `301`, and the difference is not cosmetic: a +`301` lets a client rewrite the method to `GET`, a `308` does not. Both behave +identically for the `GET` a content page is read with - and only one of them still +behaves correctly the day somebody `POST`s to a form under a moved path. + +It is **not configurable**. Every historical URL of every content type answers with +this, so there is no per-content-type setting to get wrong and no reason for two of +them to disagree. + +## Unpublished and deleted destinations + +A historical URL must never become a way to reach content that is not public. + +```text +record unpublished → old URLs answer not_found, history retained +record republished → old URLs redirect again +record deleted → old URLs answer not_found, history retained +``` + +The resolver checks the **live** publication state rather than the history, which is +why this needs no extra bookkeeping: an unpublished record simply has no current +canonical path to redirect to, so the answer is `not_found`. + +`not_found` rather than `410 Gone`, and that is a decision rather than an omission: +the engine has no abstraction that distinguishes "deleted on purpose" from +"unpublished for now", and a `410` that guessed would tell a crawler to forget a URL +that is coming back next week. + +History is **kept** on delete. An incoming link to a deleted article is exactly the +diagnostic somebody will want, and the resolver answers 404 for it by reading the +live record rather than by having forgotten the URL. + +## Restore + +A [revision restore](/docs/dev/content-engine/revisions) can move a slug, and it +integrates with history like any other edit: + +```text +current slug: new-name +restored revision: old-name + +after restore: + /articles/old-name canonical + /articles/new-name 308 -> /articles/old-name +``` + +The two addresses swap roles. A restore that changes no slug writes nothing at all - +the diff proves nothing moved before the delivery step runs. + +## Localized history + +Each locale's history is its own, because `languageId` is part of the key: + +```text +EN: /en/articles/hello → /en/articles/hello-world +PL: /pl/articles/witaj (untouched) +``` + +Changing the English URL creates no Polish redirect and retires no Polish address. +The same historical slug may be retired independently in two locales, because +`/en/x/shared` and `/pl/x/shared` are two URLs. + +See [Localization and hreflang](/docs/dev/content-engine/localization-and-hreflang). + +<Callout type="warn" title="A localized content type needs a localized slug"> + `delivery.redirects` refuses a localized content type whose `publicApi.slugField` + is **shared**. Every language would answer to the same segment, so one retired + address would belong to several URLs at once - and slug history stores the URL + that was live. Canonical URLs, SEO, alternates and the sitemap all work fine in + that shape; only the reservation is ambiguous, so only it is refused. +</Callout> + +## Transactions + +The slug write and its reservation are one transaction: + +```text +BEGIN + lock the row (the guarded UPDATE does it) + verify expectedVersion + update the slug + retire the old address + reserve the new one + write the revision +COMMIT + +emit the delivery events +invalidate the cache tags +sync the search index +``` + +The order matters twice. The reservation runs **after** the guarded write, so a +writer holding a stale `expectedVersion` fails first and leaves the history exactly +as it found it. And the old address is retired **before** the new one is reserved, +or a move from `a` to `b` and back to `a` would hit its own live reservation. + +Everything after `COMMIT` is outside the transaction, for the reason every other +stage states: a rollback cannot un-emit an event or un-expire a cache tag. + +## Concurrency + +Two editors racing on one slug produce one winner and one structured +[version conflict](/docs/dev/content-engine/editorial#optimistic-locking) - the +guarded `UPDATE` is the whole mechanism, and the history follows it: + +```text +version 3, slug = A +writer 1: A → B +writer 2: A → C + +one commits; the other gets 409 CONTENT_VERSION_CONFLICT +history: exactly one retirement and one new reservation +``` + +Two *different* records racing for the same retired address both lose: the +reservation lookup takes a row lock, so they serialise rather than race, and the +address belongs to neither of them. + +## Events + +Two events, each gated on a fact rather than an operation: + +```text +content.<id>.delivery_slug_changed +content.<id>.delivery_redirect_created +``` + +```ts +{ + contentId: 42, + locale: "pl", + previousSlug: "stary-slug", + slug: "nowy-slug", + previousPath: "/pl/articles/stary-slug", + canonicalPath: "/pl/articles/nowy-slug", +} +``` + +They are emitted **alongside** `updated` or `restored`, never instead of one: the +field mutation and the URL change are different facts with different audiences. A +listener that mirrors content wants the first; one that warms a CDN, tells an +external search engine or writes to an edge redirect table wants the second, and +would otherwise have to inspect `changedFields` for a slug field whose name it +cannot know. + +`delivery_redirect_created` fires only when the old address had genuinely been live, +so a corrected draft emits nothing. There is deliberately no sitemap event - every +mutation that changes a sitemap line already emits one of these or a publication +event. + +Both are documented in +[Built-in events](/docs/dev/events/built-in-events). + +### When nobody hears them + +Both are emitted **after** the transaction commits, and `emit()` reports a dead +listener rather than throwing one - so a broker outage cannot roll a slug change +back, and the request still answers 200. It is not swallowed either: the failure +goes to `core_logs` behind `[content-effects]` with the content type, the item, the +locale and the listener that failed, exactly like the publication events. See +[Observability](/docs/dev/content-engine/content-engine-observability#the-logs). + +That log line is the one worth alerting on. A listener that purges a CDN or writes +an edge redirect table missing a `delivery_slug_changed` leaves the old address +404ing at the edge while the origin is entirely correct - which is the failure +nobody notices from the inside. + +## AdminCP + +Every delivery-enabled content type gets a read-only delivery panel on its row +action: + +```text +Delivery + +Canonical URL +/pl/articles/moj-artykul + +Status +Published + +Historical URLs +/pl/articles/stary-slug → redirects to the current URL +/pl/articles/jeszcze-starszy → redirects to the current URL +``` + +Gated by `can_view` and nothing narrower. It reports what the slug mutations already +did, so the permission that allowed the mutation is the only one it needs - +inventing a `can_manage_redirects` for a screen that manages nothing would be a +permission every install has to configure for no decision it can make. + +**Read-only is the deliberate scope.** A redirect is somebody else's incoming link, +so deleting one silently breaks traffic nobody in that dialog can see. That is a +destructive action, and a destructive action needs its own permission, a +confirmation that explains the consequence, and an audit trail. Displaying the +history is useful today; managing it is a product rather than a button. diff --git a/apps/docs/content/docs/dev/events/built-in-events.mdx b/apps/docs/content/docs/dev/events/built-in-events.mdx index 48ebb666c..522971ecd 100644 --- a/apps/docs/content/docs/dev/events/built-in-events.mdx +++ b/apps/docs/content/docs/dev/events/built-in-events.mdx @@ -275,7 +275,33 @@ content.example.article.translation_unpublished (with publication) content.example.article.translation_restored (with editorial) ``` -Every one of them carries `locale` and `languageId`. They are deliberately +And one that opts into +[`delivery`](/docs/dev/content-engine/content-delivery) emits two more: + +```text +content.example.article.delivery_slug_changed +content.example.article.delivery_redirect_created +``` + +These arrive **alongside** `updated` (or `restored`, or `translation_updated`), never +instead of one: a field moving and a URL moving are different facts with different +audiences. A listener that mirrors content wants the first; one that warms a CDN, tells +an external search engine or writes to an edge redirect table wants the second, and +would otherwise have to inspect `changedFields` for a slug field whose name it cannot +know. + +`delivery_redirect_created` fires only when the old address had genuinely been +**publicly addressable** - so an article whose slug was corrected three times while it +was still a draft emits nothing, and a published article that moves emits exactly one. +That is the difference between "a URL now needs a redirect" and "somebody edited a +field". Both payloads carry `previousPath` and `canonicalPath`, and `locale` is `null` +when the slug is shared. + +There is deliberately no sitemap event: every mutation that changes a sitemap line +already emits one of these or a publication event, and a third carrying no new +information would be one more thing to keep consistent for no listener's benefit. + +Every translation event carries `locale` and `languageId`. They are deliberately **not** folded into `updated`: a shared update and a Polish translation update are different domain facts with different consequences - one invalidates every language, the other invalidates one - and a listener that had to inspect @@ -316,6 +342,21 @@ core event - `changedFields` narrows to that content type's own field names. description: "Restored only - the revision the values were taken from.", type: "number", }, + previousSlug: { + description: + "Delivery only - the slug the record answered to before this mutation.", + type: "string", + }, + previousPath: { + description: + "Delivery only - the full path it answered to before, e.g. `/pl/articles/stary-slug`.", + type: "string", + }, + canonicalPath: { + description: + "Delivery only - the path it answers to now, and where the historical one redirects.", + type: "string", + }, locale: { description: "Translation events only - the canonical core_languages.code the mutation was made in. Always present, so a listener never has to go and ask which language.", diff --git a/apps/docs/migrations/0032_add_content_slug_history.sql b/apps/docs/migrations/0032_add_content_slug_history.sql new file mode 100644 index 000000000..cb6e056e6 --- /dev/null +++ b/apps/docs/migrations/0032_add_content_slug_history.sql @@ -0,0 +1,17 @@ +CREATE TABLE "core_content_slug_history" ( + "id" serial PRIMARY KEY NOT NULL, + "pluginId" varchar(255) NOT NULL, + "contentTypeId" varchar(100) NOT NULL, + "itemId" integer NOT NULL, + "languageId" integer, + "slug" varchar(160) NOT NULL, + "path" varchar(512) NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + "retiredAt" timestamp +); +--> statement-breakpoint +ALTER TABLE "core_content_slug_history" ENABLE ROW LEVEL SECURITY;--> statement-breakpoint +CREATE UNIQUE INDEX "core_content_slug_history_shared_unique" ON "core_content_slug_history" USING btree ("contentTypeId","slug") WHERE "languageId" IS NULL;--> statement-breakpoint +CREATE UNIQUE INDEX "core_content_slug_history_locale_unique" ON "core_content_slug_history" USING btree ("contentTypeId","languageId","slug") WHERE "languageId" IS NOT NULL;--> statement-breakpoint +CREATE INDEX "core_content_slug_history_item_idx" ON "core_content_slug_history" USING btree ("contentTypeId","itemId","languageId");--> statement-breakpoint +CREATE INDEX "core_content_slug_history_plugin_id_idx" ON "core_content_slug_history" USING btree ("pluginId"); \ No newline at end of file diff --git a/apps/docs/migrations/0033_add_example_article_no_index.sql b/apps/docs/migrations/0033_add_example_article_no_index.sql new file mode 100644 index 000000000..cab8ed154 --- /dev/null +++ b/apps/docs/migrations/0033_add_example_article_no_index.sql @@ -0,0 +1 @@ +ALTER TABLE "example_advanced_articles" ADD COLUMN "syndicationNoIndex" boolean DEFAULT false NOT NULL; \ No newline at end of file diff --git a/apps/docs/migrations/meta/0032_snapshot.json b/apps/docs/migrations/meta/0032_snapshot.json new file mode 100644 index 000000000..386f622ef --- /dev/null +++ b/apps/docs/migrations/meta/0032_snapshot.json @@ -0,0 +1,4029 @@ +{ + "id": "b7094309-91e5-43f2-b9f9-d5666d73f0f4", + "prevId": "c3a84fce-ca99-43a8-8b83-a8be82faeed9", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.core_admin_permissions": { + "name": "core_admin_permissions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "unrestricted": { + "name": "unrestricted", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "permissions": { + "name": "permissions", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + } + }, + "indexes": { + "core_admin_permissions_role_id_idx": { + "name": "core_admin_permissions_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_admin_permissions_user_id_idx": { + "name": "core_admin_permissions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_permissions_roleId_core_roles_id_fk": { + "name": "core_admin_permissions_roleId_core_roles_id_fk", + "tableFrom": "core_admin_permissions", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_admin_permissions_userId_core_users_id_fk": { + "name": "core_admin_permissions_userId_core_users_id_fk", + "tableFrom": "core_admin_permissions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_admin_sessions": { + "name": "core_admin_sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "lastSeen": { + "name": "lastSeen", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "deviceId": { + "name": "deviceId", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_admin_sessions_token_idx": { + "name": "core_admin_sessions_token_idx", + "columns": [ + { + "expression": "token", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_admin_sessions_user_id_idx": { + "name": "core_admin_sessions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_sessions_userId_core_users_id_fk": { + "name": "core_admin_sessions_userId_core_users_id_fk", + "tableFrom": "core_admin_sessions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_admin_sessions_deviceId_core_sessions_known_devices_id_fk": { + "name": "core_admin_sessions_deviceId_core_sessions_known_devices_id_fk", + "tableFrom": "core_admin_sessions", + "tableTo": "core_sessions_known_devices", + "columnsFrom": [ + "deviceId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_admin_sessions_token_unique": { + "name": "core_admin_sessions_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_revisions": { + "name": "core_content_revisions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "operation": { + "name": "operation", + "type": "varchar(20)", + "primaryKey": false, + "notNull": true + }, + "snapshot": { + "name": "snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "changedFields": { + "name": "changedFields", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "actorType": { + "name": "actorType", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true, + "default": "'system'" + }, + "actorUserId": { + "name": "actorUserId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "restoredFromRevisionId": { + "name": "restoredFromRevisionId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_content_revisions_item_version_unique": { + "name": "core_content_revisions_item_version_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_translation_version_unique": { + "name": "core_content_revisions_translation_version_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_language_idx": { + "name": "core_content_revisions_language_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_plugin_id_idx": { + "name": "core_content_revisions_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_actor_user_id_idx": { + "name": "core_content_revisions_actor_user_id_idx", + "columns": [ + { + "expression": "actorUserId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_content_revisions_actorUserId_core_users_id_fk": { + "name": "core_content_revisions_actorUserId_core_users_id_fk", + "tableFrom": "core_content_revisions", + "tableTo": "core_users", + "columnsFrom": [ + "actorUserId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_schedules": { + "name": "core_content_schedules", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true + }, + "scheduledFor": { + "name": "scheduledFor", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "generation": { + "name": "generation", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "status": { + "name": "status", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "createdBy": { + "name": "createdBy", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completedAt": { + "name": "completedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "lastError": { + "name": "lastError", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "effectsError": { + "name": "effectsError", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_content_schedules_active_unique": { + "name": "core_content_schedules_active_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "action", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "status = 'pending'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_due_idx": { + "name": "core_content_schedules_due_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "scheduledFor", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_item_idx": { + "name": "core_content_schedules_item_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_plugin_id_idx": { + "name": "core_content_schedules_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_created_by_idx": { + "name": "core_content_schedules_created_by_idx", + "columns": [ + { + "expression": "createdBy", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_content_schedules_createdBy_core_users_id_fk": { + "name": "core_content_schedules_createdBy_core_users_id_fk", + "tableFrom": "core_content_schedules", + "tableTo": "core_users", + "columnsFrom": [ + "createdBy" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_slug_history": { + "name": "core_content_slug_history", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "path": { + "name": "path", + "type": "varchar(512)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "retiredAt": { + "name": "retiredAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_content_slug_history_shared_unique": { + "name": "core_content_slug_history_shared_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_locale_unique": { + "name": "core_content_slug_history_locale_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_item_idx": { + "name": "core_content_slug_history_item_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_plugin_id_idx": { + "name": "core_content_slug_history_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_cron": { + "name": "core_cron", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "lastRun": { + "name": "lastRun", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "module": { + "name": "module", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "nextRun": { + "name": "nextRun", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "schedule": { + "name": "schedule", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_admin_dashboard": { + "name": "core_admin_dashboard", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "widgets": { + "name": "widgets", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_admin_dashboard_user_id_idx": { + "name": "core_admin_dashboard_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_dashboard_userId_core_users_id_fk": { + "name": "core_admin_dashboard_userId_core_users_id_fk", + "tableFrom": "core_admin_dashboard", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_admin_dashboard_userId_unique": { + "name": "core_admin_dashboard_userId_unique", + "nullsNotDistinct": false, + "columns": [ + "userId" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_files": { + "name": "core_files", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "key": { + "name": "key", + "type": "varchar(512)", + "primaryKey": false, + "notNull": true + }, + "folder": { + "name": "folder", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "mimeType": { + "name": "mimeType", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "size": { + "name": "size", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_files_user_id_idx": { + "name": "core_files_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_files_userId_core_users_id_fk": { + "name": "core_files_userId_core_users_id_fk", + "tableFrom": "core_files", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_files_key_unique": { + "name": "core_files_key_unique", + "nullsNotDistinct": false, + "columns": [ + "key" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_languages": { + "name": "core_languages", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "code": { + "name": "code", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "timezone": { + "name": "timezone", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true, + "default": "'UTC'" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "default": { + "name": "default", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "time24": { + "name": "time24", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": { + "core_languages_code_idx": { + "name": "core_languages_code_idx", + "columns": [ + { + "expression": "code", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_languages_name_idx": { + "name": "core_languages_name_idx", + "columns": [ + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_languages_code_unique": { + "name": "core_languages_code_unique", + "nullsNotDistinct": false, + "columns": [ + "code" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_languages_words": { + "name": "core_languages_words", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "languageCode": { + "name": "languageCode", + "type": "varchar", + "primaryKey": false, + "notNull": true + }, + "pluginCode": { + "name": "pluginCode", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "value": { + "name": "value", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tableName": { + "name": "tableName", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "variable": { + "name": "variable", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_languages_words_lang_code_idx": { + "name": "core_languages_words_lang_code_idx", + "columns": [ + { + "expression": "languageCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_languages_words_languageCode_core_languages_code_fk": { + "name": "core_languages_words_languageCode_core_languages_code_fk", + "tableFrom": "core_languages_words", + "tableTo": "core_languages", + "columnsFrom": [ + "languageCode" + ], + "columnsTo": [ + "code" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_logs": { + "name": "core_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "varchar(10)", + "primaryKey": false, + "notNull": true + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(45)", + "primaryKey": false, + "notNull": true + }, + "method": { + "name": "method", + "type": "varchar(10)", + "primaryKey": false, + "notNull": true, + "default": "'GET'" + }, + "path": { + "name": "path", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'localhost'" + }, + "userAgent": { + "name": "userAgent", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "statusCode": { + "name": "statusCode", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 500 + }, + "userId": { + "name": "userId", + "type": "bigint", + "primaryKey": false, + "notNull": false + }, + "test123": { + "name": "test123", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": {}, + "foreignKeys": { + "core_logs_userId_core_users_id_fk": { + "name": "core_logs_userId_core_users_id_fk", + "tableFrom": "core_logs", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_moderators_permissions": { + "name": "core_moderators_permissions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "unrestricted": { + "name": "unrestricted", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "permissions": { + "name": "permissions", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + } + }, + "indexes": { + "core_moderators_permissions_role_id_idx": { + "name": "core_moderators_permissions_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_moderators_permissions_user_id_idx": { + "name": "core_moderators_permissions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_moderators_permissions_roleId_core_roles_id_fk": { + "name": "core_moderators_permissions_roleId_core_roles_id_fk", + "tableFrom": "core_moderators_permissions", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_moderators_permissions_userId_core_users_id_fk": { + "name": "core_moderators_permissions_userId_core_users_id_fk", + "tableFrom": "core_moderators_permissions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_queue": { + "name": "core_queue", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "queue": { + "name": "queue", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true, + "default": "'default'" + }, + "status": { + "name": "status", + "type": "varchar(20)", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "payload": { + "name": "payload", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "maxAttempts": { + "name": "maxAttempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 3 + }, + "availableAt": { + "name": "availableAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "reservedAt": { + "name": "reservedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "lastError": { + "name": "lastError", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completedAt": { + "name": "completedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_queue_status_available_at_idx": { + "name": "core_queue_status_available_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "availableAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_roles": { + "name": "core_roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "default": { + "name": "default", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "root": { + "name": "root", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "guest": { + "name": "guest", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "color": { + "name": "color", + "type": "varchar(50)", + "primaryKey": false, + "notNull": false + }, + "allowUploadFiles": { + "name": "allowUploadFiles", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "totalMaxStorage": { + "name": "totalMaxStorage", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "maxStorageForSubmit": { + "name": "maxStorageForSubmit", + "type": "integer", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_search_index": { + "name": "core_search_index", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "itemType": { + "name": "itemType", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageCode": { + "name": "languageCode", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "authorId": { + "name": "authorId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "search_vector": { + "name": "search_vector", + "type": "tsvector", + "primaryKey": false, + "notNull": true, + "generated": { + "as": "setweight(to_tsvector(CASE lower(split_part(\"core_search_index\".\"languageCode\", '-', 1)) WHEN 'da' THEN 'danish'::regconfig WHEN 'de' THEN 'german'::regconfig WHEN 'en' THEN 'english'::regconfig WHEN 'es' THEN 'spanish'::regconfig WHEN 'fi' THEN 'finnish'::regconfig WHEN 'fr' THEN 'french'::regconfig WHEN 'hu' THEN 'hungarian'::regconfig WHEN 'it' THEN 'italian'::regconfig WHEN 'nl' THEN 'dutch'::regconfig WHEN 'no' THEN 'norwegian'::regconfig WHEN 'pl' THEN 'polish'::regconfig WHEN 'pt' THEN 'portuguese'::regconfig WHEN 'ro' THEN 'romanian'::regconfig WHEN 'ru' THEN 'russian'::regconfig WHEN 'sv' THEN 'swedish'::regconfig WHEN 'tr' THEN 'turkish'::regconfig ELSE 'simple'::regconfig END, coalesce(\"core_search_index\".\"title\", '')), 'A') || setweight(to_tsvector(CASE lower(split_part(\"core_search_index\".\"languageCode\", '-', 1)) WHEN 'da' THEN 'danish'::regconfig WHEN 'de' THEN 'german'::regconfig WHEN 'en' THEN 'english'::regconfig WHEN 'es' THEN 'spanish'::regconfig WHEN 'fi' THEN 'finnish'::regconfig WHEN 'fr' THEN 'french'::regconfig WHEN 'hu' THEN 'hungarian'::regconfig WHEN 'it' THEN 'italian'::regconfig WHEN 'nl' THEN 'dutch'::regconfig WHEN 'no' THEN 'norwegian'::regconfig WHEN 'pl' THEN 'polish'::regconfig WHEN 'pt' THEN 'portuguese'::regconfig WHEN 'ro' THEN 'romanian'::regconfig WHEN 'ru' THEN 'russian'::regconfig WHEN 'sv' THEN 'swedish'::regconfig WHEN 'tr' THEN 'turkish'::regconfig ELSE 'simple'::regconfig END, coalesce(\"core_search_index\".\"content\", '')), 'B')", + "type": "stored" + } + }, + "containerType": { + "name": "containerType", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "containerId": { + "name": "containerId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "isPublic": { + "name": "isPublic", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "indexedAt": { + "name": "indexedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_search_index_search_vector_idx": { + "name": "core_search_index_search_vector_idx", + "columns": [ + { + "expression": "search_vector", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + }, + "core_search_index_created_at_idx": { + "name": "core_search_index_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_author_id_idx": { + "name": "core_search_index_author_id_idx", + "columns": [ + { + "expression": "authorId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_item_type_idx": { + "name": "core_search_index_item_type_idx", + "columns": [ + { + "expression": "itemType", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_language_code_idx": { + "name": "core_search_index_language_code_idx", + "columns": [ + { + "expression": "languageCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_is_public_idx": { + "name": "core_search_index_is_public_idx", + "columns": [ + { + "expression": "isPublic", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_search_index_authorId_core_users_id_fk": { + "name": "core_search_index_authorId_core_users_id_fk", + "tableFrom": "core_search_index", + "tableTo": "core_users", + "columnsFrom": [ + "authorId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_search_index_item_unique": { + "name": "core_search_index_item_unique", + "nullsNotDistinct": false, + "columns": [ + "itemType", + "itemId", + "languageCode" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_sessions": { + "name": "core_sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "deviceId": { + "name": "deviceId", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_sessions_user_id_idx": { + "name": "core_sessions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_sessions_userId_core_users_id_fk": { + "name": "core_sessions_userId_core_users_id_fk", + "tableFrom": "core_sessions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_sessions_deviceId_core_sessions_known_devices_id_fk": { + "name": "core_sessions_deviceId_core_sessions_known_devices_id_fk", + "tableFrom": "core_sessions", + "tableTo": "core_sessions_known_devices", + "columnsFrom": [ + "deviceId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_sessions_token_unique": { + "name": "core_sessions_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_sessions_known_devices": { + "name": "core_sessions_known_devices", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "publicId": { + "name": "publicId", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "userAgent": { + "name": "userAgent", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "lastSeen": { + "name": "lastSeen", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_sessions_known_devices_ip_address_idx": { + "name": "core_sessions_known_devices_ip_address_idx", + "columns": [ + { + "expression": "ipAddress", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_sessions_known_devices_publicId_unique": { + "name": "core_sessions_known_devices_publicId_unique", + "nullsNotDistinct": false, + "columns": [ + "publicId" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users": { + "name": "core_users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "nameCode": { + "name": "nameCode", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "password": { + "name": "password", + "type": "varchar", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "newsletter": { + "name": "newsletter", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "avatarColor": { + "name": "avatarColor", + "type": "varchar(6)", + "primaryKey": false, + "notNull": true + }, + "emailVerified": { + "name": "emailVerified", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "birthday": { + "name": "birthday", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "language": { + "name": "language", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'en'" + } + }, + "indexes": { + "core_users_name_code_idx": { + "name": "core_users_name_code_idx", + "columns": [ + { + "expression": "nameCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_name_idx": { + "name": "core_users_name_idx", + "columns": [ + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_email_idx": { + "name": "core_users_email_idx", + "columns": [ + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_roleId_core_roles_id_fk": { + "name": "core_users_roleId_core_roles_id_fk", + "tableFrom": "core_users", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "core_users_language_core_languages_code_fk": { + "name": "core_users_language_core_languages_code_fk", + "tableFrom": "core_users", + "tableTo": "core_languages", + "columnsFrom": [ + "language" + ], + "columnsTo": [ + "code" + ], + "onDelete": "set default", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_nameCode_unique": { + "name": "core_users_nameCode_unique", + "nullsNotDistinct": false, + "columns": [ + "nameCode" + ] + }, + "core_users_name_unique": { + "name": "core_users_name_unique", + "nullsNotDistinct": false, + "columns": [ + "name" + ] + }, + "core_users_email_unique": { + "name": "core_users_email_unique", + "nullsNotDistinct": false, + "columns": [ + "email" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_confirm_emails": { + "name": "core_users_confirm_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "core_users_confirm_emails_userId_core_users_id_fk": { + "name": "core_users_confirm_emails_userId_core_users_id_fk", + "tableFrom": "core_users_confirm_emails", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_confirm_emails_token_unique": { + "name": "core_users_confirm_emails_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_forgot_password": { + "name": "core_users_forgot_password", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "core_users_forgot_password_userId_core_users_id_fk": { + "name": "core_users_forgot_password_userId_core_users_id_fk", + "tableFrom": "core_users_forgot_password", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_forgot_password_userId_unique": { + "name": "core_users_forgot_password_userId_unique", + "nullsNotDistinct": false, + "columns": [ + "userId" + ] + }, + "core_users_forgot_password_token_unique": { + "name": "core_users_forgot_password_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_secondary_roles": { + "name": "core_users_secondary_roles", + "schema": "", + "columns": { + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_users_secondary_roles_user_id_idx": { + "name": "core_users_secondary_roles_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_secondary_roles_role_id_idx": { + "name": "core_users_secondary_roles_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_secondary_roles_userId_core_users_id_fk": { + "name": "core_users_secondary_roles_userId_core_users_id_fk", + "tableFrom": "core_users_secondary_roles", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_users_secondary_roles_roleId_core_roles_id_fk": { + "name": "core_users_secondary_roles_roleId_core_roles_id_fk", + "tableFrom": "core_users_secondary_roles", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "core_users_secondary_roles_userId_roleId_pk": { + "name": "core_users_secondary_roles_userId_roleId_pk", + "columns": [ + "userId", + "roleId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_sso": { + "name": "core_users_sso", + "schema": "", + "columns": { + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "providerId": { + "name": "providerId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "providerAccountId": { + "name": "providerAccountId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_users_sso_user_id_idx": { + "name": "core_users_sso_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_sso_userId_core_users_id_fk": { + "name": "core_users_sso_userId_core_users_id_fk", + "tableFrom": "core_users_sso", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.blog_categories": { + "name": "blog_categories", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "color": { + "name": "color", + "type": "varchar(50)", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.blog_posts": { + "name": "blog_posts", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "categoryId": { + "name": "categoryId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "authorId": { + "name": "authorId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "blog_posts_categoryId_blog_categories_id_fk": { + "name": "blog_posts_categoryId_blog_categories_id_fk", + "tableFrom": "blog_posts", + "tableTo": "blog_categories", + "columnsFrom": [ + "categoryId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "blog_posts_authorId_core_users_id_fk": { + "name": "blog_posts_authorId_core_users_id_fk", + "tableFrom": "blog_posts", + "tableTo": "core_users", + "columnsFrom": [ + "authorId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles": { + "name": "example_advanced_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "syndicationIndexable": { + "name": "syndicationIndexable", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "syndicationPriority": { + "name": "syndicationPriority", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 5 + } + }, + "indexes": { + "example_advanced_articles_syndication_priority_idx": { + "name": "example_advanced_articles_syndication_priority_idx", + "columns": [ + { + "expression": "syndicationPriority", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_created_at_idx": { + "name": "example_advanced_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_updated_at_idx": { + "name": "example_advanced_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_status_published_at_idx": { + "name": "example_advanced_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_categories": { + "name": "example_advanced_articles_categories", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "relatedItemId": { + "name": "relatedItemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "example_advanced_articles_categories_position_key": { + "name": "example_advanced_articles_categories_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_categories_related_item_id_idx": { + "name": "example_advanced_articles_categories_related_item_id_idx", + "columns": [ + { + "expression": "relatedItemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_categories_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_categories_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_categories", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_categories_relatedItemId_example_categories_id_fk": { + "name": "example_advanced_articles_categories_relatedItemId_example_categories_id_fk", + "tableFrom": "example_advanced_articles_categories", + "tableTo": "example_categories", + "columnsFrom": [ + "relatedItemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_categories_pk": { + "name": "example_advanced_articles_categories_pk", + "columns": [ + "itemId", + "relatedItemId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_faq": { + "name": "example_advanced_articles_faq", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "question": { + "name": "question", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "answer": { + "name": "answer", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_advanced_articles_faq_position_key": { + "name": "example_advanced_articles_faq_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_faq_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_faq_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_faq", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_related_articles": { + "name": "example_advanced_articles_related_articles", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "relatedItemId": { + "name": "relatedItemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "example_advanced_articles_related_articles_position_key": { + "name": "example_advanced_articles_related_articles_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_related_articles_related_item_id_idx": { + "name": "example_advanced_articles_related_articles_related_item_id_idx", + "columns": [ + { + "expression": "relatedItemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_related_articles_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_related_articles_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_related_articles", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_related_articles_relatedItemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_related_articles_relatedItemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_related_articles", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "relatedItemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_related_articles_pk": { + "name": "example_advanced_articles_related_articles_pk", + "columns": [ + "itemId", + "relatedItemId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_translations": { + "name": "example_advanced_articles_translations", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "seoTitle": { + "name": "seoTitle", + "type": "varchar(200)", + "primaryKey": false, + "notNull": false + }, + "seoDescription": { + "name": "seoDescription", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "example_advanced_articles_translations_language_id_status_idx": { + "name": "example_advanced_articles_translations_language_id_status_idx", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_translations_language_id_slug_key": { + "name": "example_advanced_articles_translations_language_id_slug_key", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_translations_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_translations_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_translations", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_translations_languageId_core_languages_id_fk": { + "name": "example_advanced_articles_translations_languageId_core_languages_id_fk", + "tableFrom": "example_advanced_articles_translations", + "tableTo": "core_languages", + "columnsFrom": [ + "languageId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_translations_item_id_language_id_pk": { + "name": "example_advanced_articles_translations_item_id_language_id_pk", + "columns": [ + "itemId", + "languageId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_articles": { + "name": "example_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "code": { + "name": "code", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "excerpt": { + "name": "excerpt", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "views": { + "name": "views", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "featured": { + "name": "featured", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "author": { + "name": "author", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "category": { + "name": "category", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_articles_status_created_at_idx": { + "name": "example_articles_status_created_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_slug_key": { + "name": "example_articles_slug_key", + "columns": [ + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_code_key": { + "name": "example_articles_code_key", + "columns": [ + { + "expression": "code", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_author_idx": { + "name": "example_articles_author_idx", + "columns": [ + { + "expression": "author", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_category_idx": { + "name": "example_articles_category_idx", + "columns": [ + { + "expression": "category", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_created_at_idx": { + "name": "example_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_updated_at_idx": { + "name": "example_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_status_published_at_idx": { + "name": "example_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_articles_author_core_users_id_fk": { + "name": "example_articles_author_core_users_id_fk", + "tableFrom": "example_articles", + "tableTo": "core_users", + "columnsFrom": [ + "author" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + }, + "example_articles_category_example_categories_id_fk": { + "name": "example_articles_category_example_categories_id_fk", + "tableFrom": "example_articles", + "tableTo": "example_categories", + "columnsFrom": [ + "category" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_categories": { + "name": "example_categories", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "name": { + "name": "name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_categories_created_at_idx": { + "name": "example_categories_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_categories_updated_at_idx": { + "name": "example_categories_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_localized_articles": { + "name": "example_localized_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "featured": { + "name": "featured", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": { + "example_localized_articles_created_at_idx": { + "name": "example_localized_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_updated_at_idx": { + "name": "example_localized_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_status_published_at_idx": { + "name": "example_localized_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_localized_articles_translations": { + "name": "example_localized_articles_translations", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "body": { + "name": "body", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_localized_articles_translations_language_id_status_idx": { + "name": "example_localized_articles_translations_language_id_status_idx", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_translations_language_id_slug_key": { + "name": "example_localized_articles_translations_language_id_slug_key", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_localized_articles_translations_itemId_example_localized_articles_id_fk": { + "name": "example_localized_articles_translations_itemId_example_localized_articles_id_fk", + "tableFrom": "example_localized_articles_translations", + "tableTo": "example_localized_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_localized_articles_translations_languageId_core_languages_id_fk": { + "name": "example_localized_articles_translations_languageId_core_languages_id_fk", + "tableFrom": "example_localized_articles_translations", + "tableTo": "core_languages", + "columnsFrom": [ + "languageId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_localized_articles_translations_item_id_language_id_pk": { + "name": "example_localized_articles_translations_item_id_language_id_pk", + "columns": [ + "itemId", + "languageId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + } + }, + "enums": {}, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} \ No newline at end of file diff --git a/apps/docs/migrations/meta/0033_snapshot.json b/apps/docs/migrations/meta/0033_snapshot.json new file mode 100644 index 000000000..e1ce93871 --- /dev/null +++ b/apps/docs/migrations/meta/0033_snapshot.json @@ -0,0 +1,4036 @@ +{ + "id": "318c5944-3dbe-4646-a80e-fd047f44db84", + "prevId": "b7094309-91e5-43f2-b9f9-d5666d73f0f4", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.core_admin_permissions": { + "name": "core_admin_permissions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "unrestricted": { + "name": "unrestricted", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "permissions": { + "name": "permissions", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + } + }, + "indexes": { + "core_admin_permissions_role_id_idx": { + "name": "core_admin_permissions_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_admin_permissions_user_id_idx": { + "name": "core_admin_permissions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_permissions_roleId_core_roles_id_fk": { + "name": "core_admin_permissions_roleId_core_roles_id_fk", + "tableFrom": "core_admin_permissions", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_admin_permissions_userId_core_users_id_fk": { + "name": "core_admin_permissions_userId_core_users_id_fk", + "tableFrom": "core_admin_permissions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_admin_sessions": { + "name": "core_admin_sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "lastSeen": { + "name": "lastSeen", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "deviceId": { + "name": "deviceId", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_admin_sessions_token_idx": { + "name": "core_admin_sessions_token_idx", + "columns": [ + { + "expression": "token", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_admin_sessions_user_id_idx": { + "name": "core_admin_sessions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_sessions_userId_core_users_id_fk": { + "name": "core_admin_sessions_userId_core_users_id_fk", + "tableFrom": "core_admin_sessions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_admin_sessions_deviceId_core_sessions_known_devices_id_fk": { + "name": "core_admin_sessions_deviceId_core_sessions_known_devices_id_fk", + "tableFrom": "core_admin_sessions", + "tableTo": "core_sessions_known_devices", + "columnsFrom": [ + "deviceId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_admin_sessions_token_unique": { + "name": "core_admin_sessions_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_revisions": { + "name": "core_content_revisions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "operation": { + "name": "operation", + "type": "varchar(20)", + "primaryKey": false, + "notNull": true + }, + "snapshot": { + "name": "snapshot", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "changedFields": { + "name": "changedFields", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "actorType": { + "name": "actorType", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true, + "default": "'system'" + }, + "actorUserId": { + "name": "actorUserId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "restoredFromRevisionId": { + "name": "restoredFromRevisionId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_content_revisions_item_version_unique": { + "name": "core_content_revisions_item_version_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_translation_version_unique": { + "name": "core_content_revisions_translation_version_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_language_idx": { + "name": "core_content_revisions_language_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "version", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_plugin_id_idx": { + "name": "core_content_revisions_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_revisions_actor_user_id_idx": { + "name": "core_content_revisions_actor_user_id_idx", + "columns": [ + { + "expression": "actorUserId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_content_revisions_actorUserId_core_users_id_fk": { + "name": "core_content_revisions_actorUserId_core_users_id_fk", + "tableFrom": "core_content_revisions", + "tableTo": "core_users", + "columnsFrom": [ + "actorUserId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_schedules": { + "name": "core_content_schedules", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true + }, + "scheduledFor": { + "name": "scheduledFor", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "generation": { + "name": "generation", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "status": { + "name": "status", + "type": "varchar(16)", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "createdBy": { + "name": "createdBy", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completedAt": { + "name": "completedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "lastError": { + "name": "lastError", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "effectsError": { + "name": "effectsError", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_content_schedules_active_unique": { + "name": "core_content_schedules_active_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "action", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "status = 'pending'", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_due_idx": { + "name": "core_content_schedules_due_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "scheduledFor", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_item_idx": { + "name": "core_content_schedules_item_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_plugin_id_idx": { + "name": "core_content_schedules_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_schedules_created_by_idx": { + "name": "core_content_schedules_created_by_idx", + "columns": [ + { + "expression": "createdBy", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_content_schedules_createdBy_core_users_id_fk": { + "name": "core_content_schedules_createdBy_core_users_id_fk", + "tableFrom": "core_content_schedules", + "tableTo": "core_users", + "columnsFrom": [ + "createdBy" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_content_slug_history": { + "name": "core_content_slug_history", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "contentTypeId": { + "name": "contentTypeId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "path": { + "name": "path", + "type": "varchar(512)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "retiredAt": { + "name": "retiredAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_content_slug_history_shared_unique": { + "name": "core_content_slug_history_shared_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_locale_unique": { + "name": "core_content_slug_history_locale_unique", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"languageId\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_item_idx": { + "name": "core_content_slug_history_item_idx", + "columns": [ + { + "expression": "contentTypeId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_content_slug_history_plugin_id_idx": { + "name": "core_content_slug_history_plugin_id_idx", + "columns": [ + { + "expression": "pluginId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_cron": { + "name": "core_cron", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "lastRun": { + "name": "lastRun", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "module": { + "name": "module", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "nextRun": { + "name": "nextRun", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "schedule": { + "name": "schedule", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_admin_dashboard": { + "name": "core_admin_dashboard", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "widgets": { + "name": "widgets", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_admin_dashboard_user_id_idx": { + "name": "core_admin_dashboard_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_admin_dashboard_userId_core_users_id_fk": { + "name": "core_admin_dashboard_userId_core_users_id_fk", + "tableFrom": "core_admin_dashboard", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_admin_dashboard_userId_unique": { + "name": "core_admin_dashboard_userId_unique", + "nullsNotDistinct": false, + "columns": [ + "userId" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_files": { + "name": "core_files", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "key": { + "name": "key", + "type": "varchar(512)", + "primaryKey": false, + "notNull": true + }, + "folder": { + "name": "folder", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "mimeType": { + "name": "mimeType", + "type": "varchar(255)", + "primaryKey": false, + "notNull": false + }, + "size": { + "name": "size", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_files_user_id_idx": { + "name": "core_files_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_files_userId_core_users_id_fk": { + "name": "core_files_userId_core_users_id_fk", + "tableFrom": "core_files", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_files_key_unique": { + "name": "core_files_key_unique", + "nullsNotDistinct": false, + "columns": [ + "key" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_languages": { + "name": "core_languages", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "code": { + "name": "code", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "timezone": { + "name": "timezone", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true, + "default": "'UTC'" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "default": { + "name": "default", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "time24": { + "name": "time24", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": { + "core_languages_code_idx": { + "name": "core_languages_code_idx", + "columns": [ + { + "expression": "code", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_languages_name_idx": { + "name": "core_languages_name_idx", + "columns": [ + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_languages_code_unique": { + "name": "core_languages_code_unique", + "nullsNotDistinct": false, + "columns": [ + "code" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_languages_words": { + "name": "core_languages_words", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "languageCode": { + "name": "languageCode", + "type": "varchar", + "primaryKey": false, + "notNull": true + }, + "pluginCode": { + "name": "pluginCode", + "type": "varchar(50)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "value": { + "name": "value", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "tableName": { + "name": "tableName", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "variable": { + "name": "variable", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_languages_words_lang_code_idx": { + "name": "core_languages_words_lang_code_idx", + "columns": [ + { + "expression": "languageCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_languages_words_languageCode_core_languages_code_fk": { + "name": "core_languages_words_languageCode_core_languages_code_fk", + "tableFrom": "core_languages_words", + "tableTo": "core_languages", + "columnsFrom": [ + "languageCode" + ], + "columnsTo": [ + "code" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_logs": { + "name": "core_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "type": { + "name": "type", + "type": "varchar(10)", + "primaryKey": false, + "notNull": true + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(45)", + "primaryKey": false, + "notNull": true + }, + "method": { + "name": "method", + "type": "varchar(10)", + "primaryKey": false, + "notNull": true, + "default": "'GET'" + }, + "path": { + "name": "path", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'localhost'" + }, + "userAgent": { + "name": "userAgent", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "statusCode": { + "name": "statusCode", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 500 + }, + "userId": { + "name": "userId", + "type": "bigint", + "primaryKey": false, + "notNull": false + }, + "test123": { + "name": "test123", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": {}, + "foreignKeys": { + "core_logs_userId_core_users_id_fk": { + "name": "core_logs_userId_core_users_id_fk", + "tableFrom": "core_logs", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_moderators_permissions": { + "name": "core_moderators_permissions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "unrestricted": { + "name": "unrestricted", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "permissions": { + "name": "permissions", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + } + }, + "indexes": { + "core_moderators_permissions_role_id_idx": { + "name": "core_moderators_permissions_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_moderators_permissions_user_id_idx": { + "name": "core_moderators_permissions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_moderators_permissions_roleId_core_roles_id_fk": { + "name": "core_moderators_permissions_roleId_core_roles_id_fk", + "tableFrom": "core_moderators_permissions", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_moderators_permissions_userId_core_users_id_fk": { + "name": "core_moderators_permissions_userId_core_users_id_fk", + "tableFrom": "core_moderators_permissions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_queue": { + "name": "core_queue", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "queue": { + "name": "queue", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true, + "default": "'default'" + }, + "status": { + "name": "status", + "type": "varchar(20)", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "payload": { + "name": "payload", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "attempts": { + "name": "attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "maxAttempts": { + "name": "maxAttempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 3 + }, + "availableAt": { + "name": "availableAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "reservedAt": { + "name": "reservedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "lastError": { + "name": "lastError", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completedAt": { + "name": "completedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "core_queue_status_available_at_idx": { + "name": "core_queue_status_available_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "availableAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_roles": { + "name": "core_roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "protected": { + "name": "protected", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "default": { + "name": "default", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "root": { + "name": "root", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "guest": { + "name": "guest", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "color": { + "name": "color", + "type": "varchar(50)", + "primaryKey": false, + "notNull": false + }, + "allowUploadFiles": { + "name": "allowUploadFiles", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "totalMaxStorage": { + "name": "totalMaxStorage", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "maxStorageForSubmit": { + "name": "maxStorageForSubmit", + "type": "integer", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_search_index": { + "name": "core_search_index", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "pluginId": { + "name": "pluginId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "itemType": { + "name": "itemType", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageCode": { + "name": "languageCode", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "authorId": { + "name": "authorId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "title": { + "name": "title", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "content": { + "name": "content", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + }, + "search_vector": { + "name": "search_vector", + "type": "tsvector", + "primaryKey": false, + "notNull": true, + "generated": { + "as": "setweight(to_tsvector(CASE lower(split_part(\"core_search_index\".\"languageCode\", '-', 1)) WHEN 'da' THEN 'danish'::regconfig WHEN 'de' THEN 'german'::regconfig WHEN 'en' THEN 'english'::regconfig WHEN 'es' THEN 'spanish'::regconfig WHEN 'fi' THEN 'finnish'::regconfig WHEN 'fr' THEN 'french'::regconfig WHEN 'hu' THEN 'hungarian'::regconfig WHEN 'it' THEN 'italian'::regconfig WHEN 'nl' THEN 'dutch'::regconfig WHEN 'no' THEN 'norwegian'::regconfig WHEN 'pl' THEN 'polish'::regconfig WHEN 'pt' THEN 'portuguese'::regconfig WHEN 'ro' THEN 'romanian'::regconfig WHEN 'ru' THEN 'russian'::regconfig WHEN 'sv' THEN 'swedish'::regconfig WHEN 'tr' THEN 'turkish'::regconfig ELSE 'simple'::regconfig END, coalesce(\"core_search_index\".\"title\", '')), 'A') || setweight(to_tsvector(CASE lower(split_part(\"core_search_index\".\"languageCode\", '-', 1)) WHEN 'da' THEN 'danish'::regconfig WHEN 'de' THEN 'german'::regconfig WHEN 'en' THEN 'english'::regconfig WHEN 'es' THEN 'spanish'::regconfig WHEN 'fi' THEN 'finnish'::regconfig WHEN 'fr' THEN 'french'::regconfig WHEN 'hu' THEN 'hungarian'::regconfig WHEN 'it' THEN 'italian'::regconfig WHEN 'nl' THEN 'dutch'::regconfig WHEN 'no' THEN 'norwegian'::regconfig WHEN 'pl' THEN 'polish'::regconfig WHEN 'pt' THEN 'portuguese'::regconfig WHEN 'ro' THEN 'romanian'::regconfig WHEN 'ru' THEN 'russian'::regconfig WHEN 'sv' THEN 'swedish'::regconfig WHEN 'tr' THEN 'turkish'::regconfig ELSE 'simple'::regconfig END, coalesce(\"core_search_index\".\"content\", '')), 'B')", + "type": "stored" + } + }, + "containerType": { + "name": "containerType", + "type": "varchar(100)", + "primaryKey": false, + "notNull": false + }, + "containerId": { + "name": "containerId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "isPublic": { + "name": "isPublic", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "indexedAt": { + "name": "indexedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_search_index_search_vector_idx": { + "name": "core_search_index_search_vector_idx", + "columns": [ + { + "expression": "search_vector", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "gin", + "with": {} + }, + "core_search_index_created_at_idx": { + "name": "core_search_index_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_author_id_idx": { + "name": "core_search_index_author_id_idx", + "columns": [ + { + "expression": "authorId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_item_type_idx": { + "name": "core_search_index_item_type_idx", + "columns": [ + { + "expression": "itemType", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_language_code_idx": { + "name": "core_search_index_language_code_idx", + "columns": [ + { + "expression": "languageCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_search_index_is_public_idx": { + "name": "core_search_index_is_public_idx", + "columns": [ + { + "expression": "isPublic", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_search_index_authorId_core_users_id_fk": { + "name": "core_search_index_authorId_core_users_id_fk", + "tableFrom": "core_search_index", + "tableTo": "core_users", + "columnsFrom": [ + "authorId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_search_index_item_unique": { + "name": "core_search_index_item_unique", + "nullsNotDistinct": false, + "columns": [ + "itemType", + "itemId", + "languageCode" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_sessions": { + "name": "core_sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "deviceId": { + "name": "deviceId", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "core_sessions_user_id_idx": { + "name": "core_sessions_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_sessions_userId_core_users_id_fk": { + "name": "core_sessions_userId_core_users_id_fk", + "tableFrom": "core_sessions", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_sessions_deviceId_core_sessions_known_devices_id_fk": { + "name": "core_sessions_deviceId_core_sessions_known_devices_id_fk", + "tableFrom": "core_sessions", + "tableTo": "core_sessions_known_devices", + "columnsFrom": [ + "deviceId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_sessions_token_unique": { + "name": "core_sessions_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_sessions_known_devices": { + "name": "core_sessions_known_devices", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "publicId": { + "name": "publicId", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "userAgent": { + "name": "userAgent", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "lastSeen": { + "name": "lastSeen", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_sessions_known_devices_ip_address_idx": { + "name": "core_sessions_known_devices_ip_address_idx", + "columns": [ + { + "expression": "ipAddress", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_sessions_known_devices_publicId_unique": { + "name": "core_sessions_known_devices_publicId_unique", + "nullsNotDistinct": false, + "columns": [ + "publicId" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users": { + "name": "core_users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "nameCode": { + "name": "nameCode", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "email": { + "name": "email", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "password": { + "name": "password", + "type": "varchar", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "newsletter": { + "name": "newsletter", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "avatarColor": { + "name": "avatarColor", + "type": "varchar(6)", + "primaryKey": false, + "notNull": true + }, + "emailVerified": { + "name": "emailVerified", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "birthday": { + "name": "birthday", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "language": { + "name": "language", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'en'" + } + }, + "indexes": { + "core_users_name_code_idx": { + "name": "core_users_name_code_idx", + "columns": [ + { + "expression": "nameCode", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_name_idx": { + "name": "core_users_name_idx", + "columns": [ + { + "expression": "name", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_email_idx": { + "name": "core_users_email_idx", + "columns": [ + { + "expression": "email", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_roleId_core_roles_id_fk": { + "name": "core_users_roleId_core_roles_id_fk", + "tableFrom": "core_users", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "core_users_language_core_languages_code_fk": { + "name": "core_users_language_core_languages_code_fk", + "tableFrom": "core_users", + "tableTo": "core_languages", + "columnsFrom": [ + "language" + ], + "columnsTo": [ + "code" + ], + "onDelete": "set default", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_nameCode_unique": { + "name": "core_users_nameCode_unique", + "nullsNotDistinct": false, + "columns": [ + "nameCode" + ] + }, + "core_users_name_unique": { + "name": "core_users_name_unique", + "nullsNotDistinct": false, + "columns": [ + "name" + ] + }, + "core_users_email_unique": { + "name": "core_users_email_unique", + "nullsNotDistinct": false, + "columns": [ + "email" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_confirm_emails": { + "name": "core_users_confirm_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "core_users_confirm_emails_userId_core_users_id_fk": { + "name": "core_users_confirm_emails_userId_core_users_id_fk", + "tableFrom": "core_users_confirm_emails", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_confirm_emails_token_unique": { + "name": "core_users_confirm_emails_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_forgot_password": { + "name": "core_users_forgot_password", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "token": { + "name": "token", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "ipAddress": { + "name": "ipAddress", + "type": "varchar(40)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "expiresAt": { + "name": "expiresAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "core_users_forgot_password_userId_core_users_id_fk": { + "name": "core_users_forgot_password_userId_core_users_id_fk", + "tableFrom": "core_users_forgot_password", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "core_users_forgot_password_userId_unique": { + "name": "core_users_forgot_password_userId_unique", + "nullsNotDistinct": false, + "columns": [ + "userId" + ] + }, + "core_users_forgot_password_token_unique": { + "name": "core_users_forgot_password_token_unique", + "nullsNotDistinct": false, + "columns": [ + "token" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_secondary_roles": { + "name": "core_users_secondary_roles", + "schema": "", + "columns": { + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "roleId": { + "name": "roleId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_users_secondary_roles_user_id_idx": { + "name": "core_users_secondary_roles_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "core_users_secondary_roles_role_id_idx": { + "name": "core_users_secondary_roles_role_id_idx", + "columns": [ + { + "expression": "roleId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_secondary_roles_userId_core_users_id_fk": { + "name": "core_users_secondary_roles_userId_core_users_id_fk", + "tableFrom": "core_users_secondary_roles", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "core_users_secondary_roles_roleId_core_roles_id_fk": { + "name": "core_users_secondary_roles_roleId_core_roles_id_fk", + "tableFrom": "core_users_secondary_roles", + "tableTo": "core_roles", + "columnsFrom": [ + "roleId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "core_users_secondary_roles_userId_roleId_pk": { + "name": "core_users_secondary_roles_userId_roleId_pk", + "columns": [ + "userId", + "roleId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.core_users_sso": { + "name": "core_users_sso", + "schema": "", + "columns": { + "userId": { + "name": "userId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "providerId": { + "name": "providerId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "providerAccountId": { + "name": "providerAccountId", + "type": "varchar(255)", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "core_users_sso_user_id_idx": { + "name": "core_users_sso_user_id_idx", + "columns": [ + { + "expression": "userId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "core_users_sso_userId_core_users_id_fk": { + "name": "core_users_sso_userId_core_users_id_fk", + "tableFrom": "core_users_sso", + "tableTo": "core_users", + "columnsFrom": [ + "userId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.blog_categories": { + "name": "blog_categories", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "color": { + "name": "color", + "type": "varchar(50)", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.blog_posts": { + "name": "blog_posts", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "categoryId": { + "name": "categoryId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "authorId": { + "name": "authorId", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "blog_posts_categoryId_blog_categories_id_fk": { + "name": "blog_posts_categoryId_blog_categories_id_fk", + "tableFrom": "blog_posts", + "tableTo": "blog_categories", + "columnsFrom": [ + "categoryId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "blog_posts_authorId_core_users_id_fk": { + "name": "blog_posts_authorId_core_users_id_fk", + "tableFrom": "blog_posts", + "tableTo": "core_users", + "columnsFrom": [ + "authorId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles": { + "name": "example_advanced_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "syndicationIndexable": { + "name": "syndicationIndexable", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "syndicationNoIndex": { + "name": "syndicationNoIndex", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "syndicationPriority": { + "name": "syndicationPriority", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 5 + } + }, + "indexes": { + "example_advanced_articles_syndication_priority_idx": { + "name": "example_advanced_articles_syndication_priority_idx", + "columns": [ + { + "expression": "syndicationPriority", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_created_at_idx": { + "name": "example_advanced_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_updated_at_idx": { + "name": "example_advanced_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_status_published_at_idx": { + "name": "example_advanced_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_categories": { + "name": "example_advanced_articles_categories", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "relatedItemId": { + "name": "relatedItemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "example_advanced_articles_categories_position_key": { + "name": "example_advanced_articles_categories_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_categories_related_item_id_idx": { + "name": "example_advanced_articles_categories_related_item_id_idx", + "columns": [ + { + "expression": "relatedItemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_categories_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_categories_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_categories", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_categories_relatedItemId_example_categories_id_fk": { + "name": "example_advanced_articles_categories_relatedItemId_example_categories_id_fk", + "tableFrom": "example_advanced_articles_categories", + "tableTo": "example_categories", + "columnsFrom": [ + "relatedItemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_categories_pk": { + "name": "example_advanced_articles_categories_pk", + "columns": [ + "itemId", + "relatedItemId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_faq": { + "name": "example_advanced_articles_faq", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "question": { + "name": "question", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "answer": { + "name": "answer", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_advanced_articles_faq_position_key": { + "name": "example_advanced_articles_faq_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_faq_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_faq_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_faq", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_related_articles": { + "name": "example_advanced_articles_related_articles", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "relatedItemId": { + "name": "relatedItemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "position": { + "name": "position", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "example_advanced_articles_related_articles_position_key": { + "name": "example_advanced_articles_related_articles_position_key", + "columns": [ + { + "expression": "itemId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "position", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_related_articles_related_item_id_idx": { + "name": "example_advanced_articles_related_articles_related_item_id_idx", + "columns": [ + { + "expression": "relatedItemId", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_related_articles_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_related_articles_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_related_articles", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_related_articles_relatedItemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_related_articles_relatedItemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_related_articles", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "relatedItemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_related_articles_pk": { + "name": "example_advanced_articles_related_articles_pk", + "columns": [ + "itemId", + "relatedItemId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_advanced_articles_translations": { + "name": "example_advanced_articles_translations", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "seoTitle": { + "name": "seoTitle", + "type": "varchar(200)", + "primaryKey": false, + "notNull": false + }, + "seoDescription": { + "name": "seoDescription", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "example_advanced_articles_translations_language_id_status_idx": { + "name": "example_advanced_articles_translations_language_id_status_idx", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_advanced_articles_translations_language_id_slug_key": { + "name": "example_advanced_articles_translations_language_id_slug_key", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_advanced_articles_translations_itemId_example_advanced_articles_id_fk": { + "name": "example_advanced_articles_translations_itemId_example_advanced_articles_id_fk", + "tableFrom": "example_advanced_articles_translations", + "tableTo": "example_advanced_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_advanced_articles_translations_languageId_core_languages_id_fk": { + "name": "example_advanced_articles_translations_languageId_core_languages_id_fk", + "tableFrom": "example_advanced_articles_translations", + "tableTo": "core_languages", + "columnsFrom": [ + "languageId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_advanced_articles_translations_item_id_language_id_pk": { + "name": "example_advanced_articles_translations_item_id_language_id_pk", + "columns": [ + "itemId", + "languageId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_articles": { + "name": "example_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "code": { + "name": "code", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + }, + "excerpt": { + "name": "excerpt", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "views": { + "name": "views", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "featured": { + "name": "featured", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "author": { + "name": "author", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "category": { + "name": "category", + "type": "integer", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_articles_status_created_at_idx": { + "name": "example_articles_status_created_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_slug_key": { + "name": "example_articles_slug_key", + "columns": [ + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_code_key": { + "name": "example_articles_code_key", + "columns": [ + { + "expression": "code", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_author_idx": { + "name": "example_articles_author_idx", + "columns": [ + { + "expression": "author", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_category_idx": { + "name": "example_articles_category_idx", + "columns": [ + { + "expression": "category", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_created_at_idx": { + "name": "example_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_updated_at_idx": { + "name": "example_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_articles_status_published_at_idx": { + "name": "example_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_articles_author_core_users_id_fk": { + "name": "example_articles_author_core_users_id_fk", + "tableFrom": "example_articles", + "tableTo": "core_users", + "columnsFrom": [ + "author" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "cascade" + }, + "example_articles_category_example_categories_id_fk": { + "name": "example_articles_category_example_categories_id_fk", + "tableFrom": "example_articles", + "tableTo": "example_categories", + "columnsFrom": [ + "category" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_categories": { + "name": "example_categories", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "name": { + "name": "name", + "type": "varchar(100)", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_categories_created_at_idx": { + "name": "example_categories_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_categories_updated_at_idx": { + "name": "example_categories_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_localized_articles": { + "name": "example_localized_articles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "serial", + "primaryKey": true, + "notNull": true + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "featured": { + "name": "featured", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + } + }, + "indexes": { + "example_localized_articles_created_at_idx": { + "name": "example_localized_articles_created_at_idx", + "columns": [ + { + "expression": "createdAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_updated_at_idx": { + "name": "example_localized_articles_updated_at_idx", + "columns": [ + { + "expression": "updatedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_status_published_at_idx": { + "name": "example_localized_articles_status_published_at_idx", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "publishedAt", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + }, + "public.example_localized_articles_translations": { + "name": "example_localized_articles_translations", + "schema": "", + "columns": { + "itemId": { + "name": "itemId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "languageId": { + "name": "languageId", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "version": { + "name": "version", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 1 + }, + "createdAt": { + "name": "createdAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updatedAt": { + "name": "updatedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "publishedAt": { + "name": "publishedAt", + "type": "timestamp", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "varchar(32)", + "primaryKey": false, + "notNull": true, + "default": "'draft'" + }, + "title": { + "name": "title", + "type": "varchar(200)", + "primaryKey": false, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "varchar(160)", + "primaryKey": false, + "notNull": true + }, + "body": { + "name": "body", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": { + "example_localized_articles_translations_language_id_status_idx": { + "name": "example_localized_articles_translations_language_id_status_idx", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "example_localized_articles_translations_language_id_slug_key": { + "name": "example_localized_articles_translations_language_id_slug_key", + "columns": [ + { + "expression": "languageId", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "slug", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "example_localized_articles_translations_itemId_example_localized_articles_id_fk": { + "name": "example_localized_articles_translations_itemId_example_localized_articles_id_fk", + "tableFrom": "example_localized_articles_translations", + "tableTo": "example_localized_articles", + "columnsFrom": [ + "itemId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "cascade" + }, + "example_localized_articles_translations_languageId_core_languages_id_fk": { + "name": "example_localized_articles_translations_languageId_core_languages_id_fk", + "tableFrom": "example_localized_articles_translations", + "tableTo": "core_languages", + "columnsFrom": [ + "languageId" + ], + "columnsTo": [ + "id" + ], + "onDelete": "restrict", + "onUpdate": "cascade" + } + }, + "compositePrimaryKeys": { + "example_localized_articles_translations_item_id_language_id_pk": { + "name": "example_localized_articles_translations_item_id_language_id_pk", + "columns": [ + "itemId", + "languageId" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": true + } + }, + "enums": {}, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} \ No newline at end of file diff --git a/apps/docs/migrations/meta/_journal.json b/apps/docs/migrations/meta/_journal.json index 230eb9f15..1f44b074f 100644 --- a/apps/docs/migrations/meta/_journal.json +++ b/apps/docs/migrations/meta/_journal.json @@ -225,6 +225,20 @@ "when": 1786181800826, "tag": "0031_add_example_advanced_articles", "breakpoints": true + }, + { + "idx": 32, + "version": "7", + "when": 1786194085698, + "tag": "0032_add_content_slug_history", + "breakpoints": true + }, + { + "idx": 33, + "version": "7", + "when": 1786195724174, + "tag": "0033_add_example_article_no_index", + "breakpoints": true } ] } \ No newline at end of file diff --git a/apps/docs/src/examples/data-table.tsx b/apps/docs/src/examples/data-table.tsx index d0d86a2c0..0de593d86 100644 --- a/apps/docs/src/examples/data-table.tsx +++ b/apps/docs/src/examples/data-table.tsx @@ -73,8 +73,9 @@ export default function DataTableExample() { pageInfo={{ hasNextPage: false, hasPreviousPage: false, - startCursor: 1, - endCursor: 1, + // Opaque cursors, as the API mints them - never row identifiers. + startCursor: null, + endCursor: null, count: 1, totalCount: 1, }} diff --git a/packages/elasticsearch/src/index.test.ts b/packages/elasticsearch/src/index.test.ts index 864f74194..d207eb526 100644 --- a/packages/elasticsearch/src/index.test.ts +++ b/packages/elasticsearch/src/index.test.ts @@ -6,6 +6,7 @@ const { bulk, index, deleteByQuery, + countDocs, ping, search, exists, @@ -15,6 +16,7 @@ const { const bulk = vi.fn(); const del = vi.fn(); const deleteByQuery = vi.fn(); + const countDocs = vi.fn(); const ping = vi.fn(); const search = vi.fn(); const exists = vi.fn(); @@ -25,6 +27,7 @@ const { bulk, delete: del, deleteByQuery, + count: countDocs, ping, search, indices: { exists, create }, @@ -44,6 +47,7 @@ const { bulk, index, deleteByQuery, + countDocs, ping, search, exists, @@ -448,3 +452,105 @@ describe("ElasticsearchSearchAdapter.ping", () => { expect(await ElasticsearchSearchAdapter(config).ping?.(c)).toBe(false); }); }); + +/** + * Provider-level diagnostics. + * + * The canonical `core_search_index` and this index are two storages, and only + * one of them is what a visitor actually searches. A drift diagnostic that could + * not ask this one would report a perfectly healthy canonical table while the + * search box was missing results - which is the failure this API exists to make + * visible. + */ +describe("ElasticsearchSearchAdapter.count", () => { + it("declares itself countable, so diagnostics can verify it", () => { + const adapter = ElasticsearchSearchAdapter(config); + + expect(typeof adapter.count).toBe("function"); + // And it is *not* the canonical storage: it mirrors, so it can drift. + expect(adapter.capabilities?.canonicalStorage).toBeUndefined(); + }); + + it("counts one collection without fetching a single document", async () => { + countDocs.mockResolvedValue({ count: 42 }); + + const total = await ElasticsearchSearchAdapter(config).count?.(c, { + itemType: "blog_post", + }); + + expect(total).toBe(42); + // `_count`, never `_search`: a diagnostic over a large index has to cost + // the same as one over an empty one. + expect(search).not.toHaveBeenCalled(); + expect(countDocs.mock.calls[0][0]).toMatchObject({ + index: "test", + query: { bool: { filter: [{ term: { itemType: "blog_post" } }] } }, + }); + }); + + it("narrows to one language when asked", async () => { + // Per-locale is the whole point: "Polish is missing forty documents" is not + // something a single total can say. + countDocs.mockResolvedValue({ count: 7 }); + + await ElasticsearchSearchAdapter(config).count?.(c, { + itemType: "blog_post", + languageCode: "pl", + }); + + expect(countDocs.mock.calls[0][0].query.bool.filter).toEqual([ + { term: { itemType: "blog_post" } }, + { term: { languageCode: "pl" } }, + ]); + }); + + it("reads an index that does not exist yet as empty", async () => { + // An install that has never rebuilt has no index. That is drift to report, + // not a crash in the status route. + countDocs.mockResolvedValue({}); + + await expect( + ElasticsearchSearchAdapter(config).count?.(c, { itemType: "blog_post" }), + ).resolves.toBe(0); + expect(countDocs.mock.calls[0][1]).toMatchObject({ ignore: [404] }); + }); + + it("counts every language when none is named", async () => { + // The unfiltered total, which is what catches a document left behind in a + // locale the content type no longer has - per-locale counts can only ask + // about locales somebody already knows to ask for. + countDocs.mockResolvedValue({ count: 9 }); + + await ElasticsearchSearchAdapter(config).count?.(c, { + itemType: "blog_post", + }); + + expect(countDocs.mock.calls[0][0].query.bool.filter).toEqual([ + { term: { itemType: "blog_post" } }, + ]); + }); + + it("creates no index as a side effect", async () => { + // A diagnostic is observational. Creating the index here would make the + // health check change the thing it is measuring - and would report a + // never-rebuilt install as a healthy empty one. + countDocs.mockResolvedValue({ count: 0 }); + + await ElasticsearchSearchAdapter(config).count?.(c, { + itemType: "blog_post", + }); + + expect(create).not.toHaveBeenCalled(); + expect(exists).not.toHaveBeenCalled(); + }); + + it("lets a transport failure surface, so the caller can report it", async () => { + // Swallowing it here would turn "Elasticsearch is down" into "zero + // documents", which reads as drift rather than as an outage. + countDocs.mockRejectedValue(new Error("connect ECONNREFUSED")); + + await expect( + ElasticsearchSearchAdapter(config).count?.(c, { itemType: "blog_post" }), + ).rejects.toThrow("connect ECONNREFUSED"); + }); +}); diff --git a/packages/elasticsearch/src/index.ts b/packages/elasticsearch/src/index.ts index 31947ba5d..08fee597f 100644 --- a/packages/elasticsearch/src/index.ts +++ b/packages/elasticsearch/src/index.ts @@ -294,6 +294,39 @@ export const ElasticsearchSearchAdapter = ( languageScopedDelete: true, }, + /** + * How many documents this index holds for one collection. + * + * `_count` rather than a search: it returns a number without fetching a + * single document, so a diagnostic over a large index costs the same as one + * over an empty one. `languageCode` narrows it to a single translation, + * which is what makes per-locale drift visible - "Polish is missing forty + * documents" is not something a total can say. + * + * A missing index means zero rather than an error: an install that has never + * rebuilt has no index yet, and that is drift to report, not a crash. + */ + count: async (_c, { itemType, languageCode }) => { + const response = await getClient().count( + { + index, + query: { + bool: { + filter: [ + { term: { itemType } }, + ...(languageCode === undefined + ? [] + : [{ term: { languageCode } }]), + ], + }, + }, + }, + { ignore: [404] }, + ); + + return response.count ?? 0; + }, + index: async (_c, doc) => { await ensureIndex(); await getClient().index({ diff --git a/packages/vitnode/src/api/adapters/search/postgres.ts b/packages/vitnode/src/api/adapters/search/postgres.ts index 88193f36f..3a4ee863f 100644 --- a/packages/vitnode/src/api/adapters/search/postgres.ts +++ b/packages/vitnode/src/api/adapters/search/postgres.ts @@ -75,11 +75,15 @@ const buildFilters = (params: SearchQueryParams): SQL | undefined => { export const PostgresSearchAdapter = (): SearchProviderApiPlugin => ({ name: "postgres", - // `languageScopedDelete` is true by construction: this provider's store *is* - // `core_search_index`, and `SearchModel.delete` already narrows that table by - // `languageCode` before it gets here. There is no second copy to keep in step. + // Two capabilities that are both true for the same reason: this provider's + // store *is* `core_search_index`. `languageScopedDelete` because + // `SearchModel.delete` already narrows that table by `languageCode` before it + // gets here, and `canonicalStorage` because there is no second copy to drift + // from - so diagnostics can report the canonical count as the provider count + // rather than asking the same table twice. capabilities: { authorBoost: false, + canonicalStorage: true, facets: false, languageScopedDelete: true, timeDecay: false, diff --git a/packages/vitnode/src/api/lib/pagination-cursor.test.ts b/packages/vitnode/src/api/lib/pagination-cursor.test.ts new file mode 100644 index 000000000..b475575d5 --- /dev/null +++ b/packages/vitnode/src/api/lib/pagination-cursor.test.ts @@ -0,0 +1,507 @@ +import { bigint, date, pgTable, time, timestamp } from "drizzle-orm/pg-core"; +// @vitest-environment node +import { HTTPException } from "hono/http-exception"; +import { describe, expect, it } from "vitest"; + +import { core_users } from "@/database/users"; + +import { + cursorValueForColumn, + cursorValueIsCanonicalText, + cursorValueOf, + decodePaginationCursor, + encodePaginationCursor, + isCursorSortableColumn, +} from "./pagination-cursor"; + +/** The column shapes core itself has no example of, but a plugin may. */ +const probes = pgTable("cursor_probes", { + big: bigint({ mode: "bigint" }), + clock: time(), + clockTz: time({ withTimezone: true }), + day: date(), + moment: timestamp(), +}); + +/** + * The cursor is the ordered tuple, or it is nothing. + * + * An identifier on its own only describes a position when the list is ordered + * by the identifier. For any other column it names a row whose place in the + * sequence nobody knows - which is how a list ordered by `updatedAt` used to + * skip every row whose id happened to fall on the wrong side of it. + */ + +const statusOf = (error: unknown): number => + error instanceof HTTPException ? error.status : 0; + +describe("encoding", () => { + it("round-trips a cursor", () => { + const cursor = { + column: "updatedAt", + id: 42, + value: "2026-08-08T12:00:00.000Z", + }; + + expect( + decodePaginationCursor(encodePaginationCursor(cursor), { + column: "updatedAt", + primaryKey: "id", + }), + ).toEqual(cursor); + }); + + it("is opaque, so nothing downstream starts parsing it", () => { + const encoded = encodePaginationCursor({ + column: "updatedAt", + id: 42, + value: "2026-08-08T12:00:00.000Z", + }); + + // base64url: URL-safe, and not a number somebody will be tempted to read. + expect(encoded).toMatch(/^[A-Za-z0-9_-]+$/); + expect(Number.isNaN(Number(encoded))).toBe(true); + }); + + it("round-trips a null order value, which is a real position", () => { + const cursor = { column: "publishedAt", id: 9, value: null }; + + expect( + decodePaginationCursor(encodePaginationCursor(cursor), { + column: "publishedAt", + primaryKey: "id", + }), + ).toEqual(cursor); + }); + + it.each([ + ["a string", "Zebra"], + ["a number", 12], + ["a boolean", true], + ])("round-trips %s value", (_why, value) => { + const cursor = { column: "name", id: 3, value }; + + expect( + decodePaginationCursor(encodePaginationCursor(cursor), { + column: "name", + primaryKey: "id", + }).value, + ).toEqual(value); + }); +}); + +describe("decoding refuses what it cannot trust", () => { + const decode = (raw: string, column = "updatedAt") => + decodePaginationCursor(raw, { column, primaryKey: "id" }); + + it.each([ + ["garbage", "not-a-cursor!!"], + ["an empty string", " "], + [ + "valid base64 that is not JSON", + Buffer.from("hello").toString("base64url"), + ], + [ + "JSON that is not an object", + Buffer.from("[1,2,3]").toString("base64url"), + ], + [ + "an object with no id", + Buffer.from(JSON.stringify({ column: "updatedAt", value: 1 })).toString( + "base64url", + ), + ], + [ + "a non-integer id", + Buffer.from( + JSON.stringify({ column: "updatedAt", id: 1.5, value: 1 }), + ).toString("base64url"), + ], + [ + "a zero id", + Buffer.from( + JSON.stringify({ column: "updatedAt", id: 0, value: 1 }), + ).toString("base64url"), + ], + [ + "an object value", + Buffer.from( + JSON.stringify({ column: "updatedAt", id: 1, value: { a: 1 } }), + ).toString("base64url"), + ], + ])("answers 400 for %s", (_why, raw) => { + expect(() => decode(raw)).toThrow(HTTPException); + try { + decode(raw); + } catch (error) { + expect(statusOf(error)).toBe(400); + } + }); + + it("refuses a cursor minted for another ordering", () => { + // The two describe different sequences, so the position means nothing - + // and using it anyway is exactly how rows get skipped. + const encoded = encodePaginationCursor({ + column: "updatedAt", + id: 42, + value: "2026-08-08T12:00:00.000Z", + }); + + expect(() => decode(encoded, "title")).toThrow(/different ordering/); + }); +}); + +describe("legacy numeric cursors", () => { + it("still works when the list is ordered by its identifier", () => { + // There the identifier really is the whole ordered tuple, so an old + // bookmark keeps working. + expect( + decodePaginationCursor("42", { column: "id", primaryKey: "id" }), + ).toEqual({ column: "id", id: 42, value: 42 }); + }); + + it("is refused for any other ordering rather than guessed at", () => { + // The regression this whole change exists for: a bare number says nothing + // about where `updatedAt` was, so interpreting it as one would silently + // skip rows. + expect(() => + decodePaginationCursor("42", { column: "updatedAt", primaryKey: "id" }), + ).toThrow(/cannot be used with the "updatedAt" ordering/); + }); +}); + +describe("column values", () => { + it("keeps a timestamp as text on both sides", () => { + // Deliberately *not* a `Date` round trip. A `Date` holds milliseconds and + // Postgres holds microseconds, so turning the text back into one would + // truncate the value the next comparison is parsed from - and exclude the + // whole millisecond the cursor came from. The predicate casts this text + // back to the column's type instead, and lets Postgres do the parsing. + const flattened = cursorValueOf( + core_users.createdAt, + "2026-08-08 12:00:00.123456", + ); + + expect(flattened).toBe("2026-08-08 12:00:00.123456"); + expect(cursorValueForColumn(core_users.createdAt, flattened)).toBe( + "2026-08-08 12:00:00.123456", + ); + }); + + it("keeps a string a string", () => { + expect(cursorValueOf(core_users.name, "Ada")).toBe("Ada"); + expect(cursorValueForColumn(core_users.name, "Ada")).toBe("Ada"); + }); + + it("keeps a number a number", () => { + expect(cursorValueOf(core_users.id, 7)).toBe(7); + expect(cursorValueForColumn(core_users.id, 7)).toBe(7); + }); + + it("treats null as null on both sides", () => { + expect(cursorValueOf(core_users.createdAt, null)).toBeNull(); + expect(cursorValueForColumn(core_users.createdAt, null)).toBeNull(); + }); + + it("refuses an unparseable date rather than letting Postgres fail the cast", () => { + expect(() => + cursorValueForColumn(core_users.createdAt, "not-a-date"), + ).toThrow(HTTPException); + }); + + it("accepts the sortable column kinds", () => { + expect(isCursorSortableColumn(core_users.id)).toBe(true); + expect(isCursorSortableColumn(core_users.name)).toBe(true); + expect(isCursorSortableColumn(core_users.createdAt)).toBe(true); + expect(isCursorSortableColumn(core_users.newsletter)).toBe(true); + expect(isCursorSortableColumn(probes.big)).toBe(true); + }); +}); + +/** + * A cursor is opaque, not signed. A client can edit it. + * + * So every field is hostile input, checked against the column it claims to + * describe. Coercion is the failure mode to avoid, not just an inelegance: + * `Boolean("false")` is `true`, `Number("")` is `0`, and `BigInt("nonsense")` + * throws a `SyntaxError` that would leave the route as a 500. Each of those is + * a wrong page or a wrong status code handed to somebody who asked for neither. + */ +describe("a tampered cursor value is refused, never coerced", () => { + const refuses = ( + column: Parameters<typeof cursorValueForColumn>[0], + value: unknown, + ) => { + expect(() => cursorValueForColumn(column, value as never)).toThrow( + HTTPException, + ); + try { + cursorValueForColumn(column, value as never); + } catch (error) { + expect(statusOf(error)).toBe(400); + } + }; + + describe("boolean", () => { + it('refuses the string "false", which coercion would read as true', () => { + refuses(core_users.newsletter, "false"); + }); + + it.each([ + ["a number", 0], + ["a string", "true"], + ["an empty string", ""], + ])("refuses %s", (_why, value) => { + refuses(core_users.newsletter, value); + }); + + it("accepts a real boolean, and null", () => { + expect(cursorValueForColumn(core_users.newsletter, false)).toBe(false); + expect(cursorValueForColumn(core_users.newsletter, true)).toBe(true); + expect(cursorValueForColumn(core_users.newsletter, null)).toBeNull(); + }); + }); + + describe("number", () => { + it.each([ + ["a numeric string", "42"], + ["an empty string", ""], + ["a boolean", true], + ["nonsense", "not-a-number"], + ])("refuses %s", (_why, value) => { + refuses(core_users.id, value); + }); + + it("accepts a finite number, and null", () => { + expect(cursorValueForColumn(core_users.id, 42)).toBe(42); + expect(cursorValueForColumn(core_users.id, null)).toBeNull(); + }); + }); + + describe("bigint", () => { + it("refuses a value that would make BigInt() throw", () => { + // The one that used to escape as a native `SyntaxError`, and therefore + // as a 500. + refuses(probes.big, "not-a-bigint"); + }); + + it.each([ + ["a fractional string", "1.5"], + ["an empty string", ""], + ["a number", 12], + ["a boolean", false], + ["whitespace", " 12 "], + ])("refuses %s", (_why, value) => { + refuses(probes.big, value); + }); + + it("accepts a decimal integer string, signed or not, and null", () => { + expect(cursorValueForColumn(probes.big, "9007199254740993")).toBe( + 9007199254740993n, + ); + expect(cursorValueForColumn(probes.big, "-4")).toBe(-4n); + expect(cursorValueForColumn(probes.big, null)).toBeNull(); + }); + }); + + describe("timestamp", () => { + it.each([ + ["nonsense", "not-a-date"], + ["a number", 1_700_000_000], + ["a boolean", true], + ["a half-written date", "2026-08"], + ["an injection attempt", "2026-08-09'; DROP TABLE users; --"], + ])("refuses %s", (_why, value) => { + // Reaching Postgres with any of these would be an invalid-cast 500 + // rather than a 400 - and the last one has no business getting near a + // cast at all. + refuses(core_users.createdAt, value); + }); + + it.each([ + ["a Postgres timestamp", "2026-08-09 10:00:00.123456"], + ["a Postgres timestamptz", "2026-08-09 10:00:00.123456+00"], + ["a plain date", "2026-08-09"], + ["an ISO string", "2026-08-09T10:00:00.123Z"], + ])("accepts %s, unchanged", (_why, value) => { + // Unchanged is the point: the predicate casts this text back to the + // column's type, so Postgres parses it at the precision it stored. + expect(cursorValueForColumn(core_users.createdAt, value)).toBe(value); + }); + + it("is the one kind bound as text plus a cast", () => { + expect(cursorValueIsCanonicalText(core_users.createdAt)).toBe(true); + expect(cursorValueIsCanonicalText(core_users.id)).toBe(false); + expect(cursorValueIsCanonicalText(core_users.name)).toBe(false); + }); + + /** + * The gap a pattern alone leaves open. + * + * Every one of these has the shape of a timestamp and is not a moment, so a + * shape check waves it through and Postgres answers the cast with + * `invalid input syntax` - a 500 produced by a query string, on a route + * whose whole promise is that it does not do that. + */ + describe("impossible values that still look like timestamps", () => { + it.each([ + ["month 13", "2026-13-01"], + ["month 0", "2026-00-01"], + ["a day past the end of the month", "2026-02-30"], + ["a day past the end of a short month", "2026-04-31"], + ["29 February in a common year", "2025-02-29"], + ["29 February in a century that is not a leap year", "1900-02-29"], + ["day 0", "2026-08-00"], + ["day 32", "2026-01-32"], + ["hour 24", "2026-08-09 24:00:00"], + ["hour 99", "2026-08-09 99:00:00"], + ["minute 60", "2026-08-09 23:60:00"], + ["second 61", "2026-08-09 23:59:61"], + ["year 0", "0000-01-01"], + ])("refuses %s", (_why, value) => { + refuses(core_users.createdAt, value); + }); + + it.each([ + ["an offset past the maximum", "2026-08-09 10:00:00+25:00"], + ["an offset with 99 minutes", "2026-08-09 10:00:00+12:99"], + ["an offset that is not a number", "2026-08-09 10:00:00+ab"], + ["a seven-digit fraction", "2026-08-09 10:00:00.1234567"], + ["trailing rubbish", "2026-08-09 10:00:00 OR 1=1"], + [ + "an era suffix, which is outside the supported domain", + "2026-08-09 BC", + ], + ])("refuses %s", (_why, value) => { + refuses(core_users.createdAt, value); + }); + + it.each([ + ["29 February in a leap year", "2024-02-29"], + ["29 February in a leap century", "2000-02-29"], + ["the last second of a day", "2026-08-09 23:59:59"], + ["the first second of a day", "2026-08-09 00:00:00"], + ["31 December", "2026-12-31"], + ["microseconds", "2026-08-09 10:00:00.123456"], + ["a whole-hour offset", "2026-08-09 10:00:00+02"], + ["a half-hour offset", "2026-08-09 10:00:00+05:30"], + ["a compact offset", "2026-08-09 10:00:00-0400"], + ["a negative offset", "2026-08-09 10:00:00-04"], + ])("accepts %s", (_why, value) => { + expect(cursorValueForColumn(core_users.createdAt, value)).toBe(value); + }); + + it("preserves microseconds through validation, digit for digit", () => { + // The reason the value is kept as text at all. Anything that reformats + // it here is a truncation the next comparison inherits. + const value = "2026-08-09 10:00:00.000001"; + + expect(cursorValueForColumn(core_users.createdAt, value)).toBe(value); + }); + }); + }); + + /** + * A `date` and a `time` column look like strings to Drizzle - `dataType` says + * `"string"` - but Postgres still has to parse them. Classifying from the SQL + * type is what stops `'nonsense'::date` being a 500. + */ + describe("other temporal columns", () => { + it("holds a date column to a date, and nothing more", () => { + expect(cursorValueForColumn(probes.day, "2026-08-09")).toBe("2026-08-09"); + refuses(probes.day, "2026-08-09 10:00:00"); + refuses(probes.day, "2026-02-30"); + refuses(probes.day, "not-a-date"); + }); + + it("holds a time column to a time, and refuses a zone it has not got", () => { + expect(cursorValueForColumn(probes.clock, "10:00:00")).toBe("10:00:00"); + expect(cursorValueForColumn(probes.clock, "10:00:00.123456")).toBe( + "10:00:00.123456", + ); + refuses(probes.clock, "10:00:00+02"); + refuses(probes.clock, "24:00:00"); + refuses(probes.clock, "2026-08-09"); + }); + + it("lets a time-with-zone column carry its zone", () => { + expect(cursorValueForColumn(probes.clockTz, "10:00:00+02")).toBe( + "10:00:00+02", + ); + refuses(probes.clockTz, "10:00:00+25"); + }); + + it("binds every temporal column as text plus a cast", () => { + for (const column of [probes.day, probes.clock, probes.clockTz]) { + expect(cursorValueIsCanonicalText(column)).toBe(true); + expect(isCursorSortableColumn(column)).toBe(true); + } + }); + }); + + describe("string", () => { + it.each([ + ["a number", 12], + ["a boolean", true], + ])("refuses %s", (_why, value) => { + refuses(core_users.name, value); + }); + + it("accepts a string, and null", () => { + expect(cursorValueForColumn(core_users.name, "Ada")).toBe("Ada"); + expect(cursorValueForColumn(core_users.name, null)).toBeNull(); + }); + }); + + it("never lets a native parser error escape", () => { + // Whatever is thrown, it is an `HTTPException` - not a `SyntaxError`, a + // `RangeError`, or anything else that would surface as a 500. + const hostile = [ + [probes.big, "nope"], + [core_users.createdAt, "nope"], + [core_users.newsletter, "nope"], + [core_users.id, "nope"], + ] as const; + + for (const [column, value] of hostile) { + try { + cursorValueForColumn(column, value); + throw new Error(`Expected ${column.name} to refuse ${value}.`); + } catch (error) { + expect(error).toBeInstanceOf(HTTPException); + } + } + }); +}); + +describe("minting keeps the database's own representation", () => { + it("keeps a Postgres timestamp string exactly as it was read", () => { + // Microseconds and all: this is the value the next comparison is parsed + // from, so anything lost here is lost from the ordering. + expect( + cursorValueOf(core_users.createdAt, "2026-08-09 10:00:00.123456"), + ).toBe("2026-08-09 10:00:00.123456"); + }); + + it("rewrites a Date into the form the column would have been read in", () => { + // A `Date` only arrives when a caller mints from a value it is holding - + // the paginated path reads `::text`. Writing it the way Postgres writes a + // `timestamp` keeps minting and validation speaking one grammar, so a + // cursor this module produced can never be one it later refuses. + expect( + cursorValueOf(core_users.createdAt, new Date("2026-08-09T10:00:00.123Z")), + ).toBe("2026-08-09 10:00:00.123"); + }); + + it("refuses a Date that is not a moment, rather than minting nonsense", () => { + expect(() => + cursorValueOf(core_users.createdAt, new Date("not-a-date")), + ).toThrow(/invalid date/i); + }); + + it("carries a bigint as a decimal string, which JSON can hold", () => { + expect(cursorValueOf(probes.big, 9007199254740993n)).toBe( + "9007199254740993", + ); + }); +}); diff --git a/packages/vitnode/src/api/lib/pagination-cursor.ts b/packages/vitnode/src/api/lib/pagination-cursor.ts new file mode 100644 index 000000000..289222d30 --- /dev/null +++ b/packages/vitnode/src/api/lib/pagination-cursor.ts @@ -0,0 +1,488 @@ +import type { PgColumn } from "drizzle-orm/pg-core"; + +import { HTTPException } from "hono/http-exception"; + +/** + * The opaque cursor a paginated list hands out, and takes back. + * + * Two properties, and both of them are load-bearing. + * + * **It is the ordered tuple.** A cursor has to describe a position in an + * ordering, and an ordering is `(orderColumn, id)` - so the cursor is that pair. + * An identifier on its own is only a position when the list is ordered by the + * identifier; for any other column it names a row whose place in the sequence + * nobody knows. + * + * **It is self-contained.** The value it carries *is* the boundary, and nothing + * re-reads the row it came from. That is the difference between a cursor and a + * pointer: a cursor is the position as it stood when the page was generated, and + * editing or deleting the row that happened to sit on the boundary must not move + * it. Re-reading would mean an edit to one row silently skips every row the + * ordering used to have between the old position and the new one. + * + * The wire form is `base64url(JSON)`: opaque, so no client starts depending on + * the shape, and self-describing, so a cursor minted for one order column is + * refused by a request that has since changed to another. + * + * It is **not signed**, so every field is treated as hostile input and validated + * against the column it claims to describe - see {@link cursorValueForColumn}. + */ + +/** What an order column's value can be, once it has been through JSON. */ +export type PaginationCursorValue = boolean | null | number | string; + +export interface PaginationCursor { + /** The order column this cursor was minted for. */ + column: string; + /** The row's primary key - the tiebreaker half of the ordered tuple. */ + id: number; + /** The order column's value on that row. `null` is a real position. */ + value: PaginationCursorValue; +} + +/** + * How one column's values travel in a cursor. + * + * Named per kind rather than inferred, because "how do I serialise this" and + * "what am I willing to accept back" are the same question asked twice, and + * answering it in one place is what stops the second answer being looser than + * the first. + */ +type CursorKind = "bigint" | "boolean" | "number" | "string" | "temporal"; + +const KIND_BY_DATA_TYPE: Record<string, CursorKind> = { + bigint: "bigint", + boolean: "boolean", + date: "temporal", + number: "number", + string: "string", +}; + +const badRequest = (message: string): HTTPException => + new HTTPException(400, { message }); + +/** The one message a tampered or stale cursor ever produces. */ +const INVALID_CURSOR = "Invalid pagination cursor."; + +/** + * The three shapes a temporal value comes in, keyed by what Postgres will parse. + * + * Classified from the **SQL** type rather than the JavaScript one, because the + * two disagree in exactly the case that matters: `date()` and `time()` hand back + * plain strings, so `dataType` calls them `"string"` - and a string cursor bound + * straight into `column > $1` would reach Postgres as `'nonsense'::date` and + * come back as a 500 rather than a 400. + */ +type TemporalType = "date" | "time" | "timestamp"; + +const temporalTypeOf = (column: PgColumn): null | TemporalType => { + const sqlType = column.getSQLType().toLowerCase(); + + // Order matters: "timestamp with time zone" also starts with "time". + if (sqlType.startsWith("timestamp")) return "timestamp"; + if (sqlType.startsWith("time")) return "time"; + if (sqlType.startsWith("date")) return "date"; + + return null; +}; + +const hasTimeZone = (column: PgColumn): boolean => + column.getSQLType().toLowerCase().includes("with time zone"); + +/** + * Whether a column can be paged through at all. + * + * A `json`, `array` or custom column has no total order Postgres and JavaScript + * agree on, so a cursor over one would be a value the next page cannot compare + * against. Refused rather than approximated. + */ +export const isCursorSortableColumn = (column: PgColumn): boolean => + temporalTypeOf(column) !== null || column.dataType in KIND_BY_DATA_TYPE; + +const kindOf = (column: PgColumn): CursorKind => { + if (temporalTypeOf(column)) return "temporal"; + + const kind = KIND_BY_DATA_TYPE[column.dataType]; + if (!kind) { + throw badRequest( + `The "${column.name}" column cannot be used as a pagination cursor.`, + ); + } + + return kind; +}; + +/** + * The grammar of a Postgres temporal value, as `::text` renders it. + * + * One pattern per SQL type, because a `date` column and a `timestamp` column do + * not accept the same strings and pretending they do is how a cursor for one + * ends up being parsed as the other: + * + * | SQL type | accepted | + * | --------------------------- | ------------------------------------------- | + * | `date` | `2026-08-09` | + * | `time` | `10:00:00`, `10:00:00.123456` | + * | `time with time zone` | the above, optionally `+02` / `Z` | + * | `timestamp` | a date, optionally a time, optionally a zone | + * | `timestamp with time zone` | the same, and that is what `::text` writes | + * + * A `T` separator and a `Z` designator are accepted alongside the space-and- + * offset form Postgres writes, because a JavaScript `Date` is the one input this + * module takes that has no database text behind it. + * + * Matching the shape is only half of it. These patterns cannot tell `2026-02-30` + * from `2026-02-28`, so every capture is range-checked afterwards - see + * {@link isRealTemporal}. + */ +const TEMPORAL_GRAMMAR: Record<TemporalType, RegExp> = { + date: /^(?<year>\d{4,6})-(?<month>\d{2})-(?<day>\d{2})$/, + time: /^(?<hour>\d{2}):(?<minute>\d{2}):(?<second>\d{2})(?:\.\d{1,6})?(?<zone>.*)$/, + timestamp: + /^(?<year>\d{4,6})-(?<month>\d{2})-(?<day>\d{2})(?:[ T](?<hour>\d{2}):(?<minute>\d{2}):(?<second>\d{2})(?:\.\d{1,6})?(?<zone>.*))?$/, +}; + +/** + * Whatever the trailing group swallowed, checked rather than trusted. + * + * `(?<zone>.*)` is deliberately greedy: it catches a seventh fractional digit, + * an era suffix and `OR 1=1` alike, and hands all of them here to be refused. + */ +type TemporalParts = Partial< + Record< + "day" | "hour" | "minute" | "month" | "second" | "year" | "zone", + string + > +>; + +/** `Z`, or `±HH`, `±HH:MM`, `±HHMM`, `±HH:MM:SS` - the forms Postgres writes. */ +const ZONE = /^([+-])(\d{2})(?::?(\d{2}))?(?::?(\d{2}))?$/; + +const DAYS_IN_MONTH = [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]; + +const isLeapYear = (year: number): boolean => + (year % 4 === 0 && year % 100 !== 0) || year % 400 === 0; + +const isRealZone = (raw: string, allowed: boolean): boolean => { + if (raw === "") return true; + if (!allowed) return false; + if (raw === "Z") return true; + + const match = ZONE.exec(raw); + if (!match) return false; + + const [, , hours, minutes = "0", seconds = "0"] = match; + + // Postgres refuses anything past ±15:59:59, and so does this. + return Number(hours) <= 15 && Number(minutes) <= 59 && Number(seconds) <= 59; +}; + +/** + * Whether a shaped temporal string is a moment that exists. + * + * The reason a pattern is not enough: `2026-02-30`, `2025-02-29`, `2026-13-01` + * and `2026-08-09 23:60:00` all match the shape and all make Postgres raise + * `invalid input syntax`, which is a 500 arriving from a query string. Every one + * of them is refused here instead, before anything is bound. + * + * Deliberately stricter than Postgres in one place: Postgres reads `24:00:00` as + * the following midnight, but its own `::text` never writes it, so a cursor + * carrying one did not come from a row. + */ +const isRealTemporal = ( + column: PgColumn, + temporal: TemporalType, + value: string, +): boolean => { + const match = TEMPORAL_GRAMMAR[temporal].exec(value); + if (!match) return false; + + const { day, hour, minute, month, second, year, zone } = + (match.groups as TemporalParts | undefined) ?? {}; + + if (year !== undefined) { + const [y, m, d] = [Number(year), Number(month), Number(day)]; + if (y < 1 || y > 294276 || m < 1 || m > 12) return false; + + const limit = m === 2 && isLeapYear(y) ? 29 : DAYS_IN_MONTH[m - 1]; + if (d < 1 || d > limit) return false; + } + + // A bare `date`, or a `timestamp` written as one: no time to check. + if (hour === undefined) return true; + + if (Number(hour) > 23 || Number(minute) > 59 || Number(second) > 59) { + return false; + } + + // A `timestamp` takes an offset and throws it away, which is what lets a + // `Date`-minted value carry `+00`. A `time` without a zone does not. + return isRealZone( + zone ?? "", + temporal === "timestamp" || hasTimeZone(column), + ); +}; + +const DECIMAL_INTEGER = /^-?\d+$/; + +const pad = (value: number, width = 2): string => + String(value).padStart(width, "0"); + +/** + * A `Date` written the way Postgres writes the column it belongs to. + * + * Only reachable when a caller mints a cursor from a value it already holds + * rather than from a row - the paginated path selects `::text` and never sees a + * `Date`. Even so it goes through the same grammar as everything else, so a + * minted cursor and an accepted cursor can never disagree about what a value + * looks like. + */ +const canonicalFromDate = (column: PgColumn, value: Date): string => { + const time = value.getTime(); + if (!Number.isFinite(time)) { + throw new Error( + `Cannot build a pagination cursor from an invalid date on "${column.name}".`, + ); + } + + const zone = hasTimeZone(column) ? "+00" : ""; + const date = `${pad(value.getUTCFullYear(), 4)}-${pad(value.getUTCMonth() + 1)}-${pad(value.getUTCDate())}`; + const clock = `${pad(value.getUTCHours())}:${pad(value.getUTCMinutes())}:${pad(value.getUTCSeconds())}.${pad(value.getUTCMilliseconds(), 3)}`; + + switch (temporalTypeOf(column)) { + case "date": + return date; + case "time": + return `${clock}${zone}`; + default: + return `${date} ${clock}${zone}`; + } +}; + +/** + * One column value, flattened into the cursor's canonical representation. + * + * Per kind, and deliberately not a generic coercion: + * + * | kind | carried as | + * | --------- | --------------------------------------------- | + * | number | a JSON number | + * | boolean | a JSON boolean | + * | string | a JSON string | + * | bigint | a decimal string, because JSON has no bigint | + * | temporal | the database's own `::text`, microseconds and all | + * | null | `null` | + * + * The temporal row is the one worth reading twice. A Postgres `timestamp` keeps + * microseconds and a JavaScript `Date` keeps milliseconds, so a value that has + * been through a `Date` is *strictly smaller* than the one still in the table - + * and comparing against it would exclude the entire millisecond it came from. + * Since `now()` stamps every row in one statement identically, that is not an + * edge case: it would end a bulk-imported collection's walk after page one. So + * the page query selects `column::text` and this function keeps it exactly as + * Postgres wrote it. + */ +export const cursorValueOf = ( + column: PgColumn, + value: unknown, +): PaginationCursorValue => { + if (value === null || value === undefined) return null; + + switch (kindOf(column)) { + case "bigint": { + if (typeof value === "bigint") return value.toString(); + if (typeof value === "number") return String(value); + break; + } + case "boolean": { + if (typeof value === "boolean") return value; + break; + } + case "number": { + if (typeof value === "number") return value; + break; + } + case "temporal": { + // Already `::text` from the database on the paginated path. A `Date` only + // reaches here when a caller mints from a value it is holding, and it is + // rewritten into the form Postgres would have written. + if (value instanceof Date) return canonicalFromDate(column, value); + if (typeof value === "string") return value; + break; + } + default: { + if (typeof value === "string") return value; + break; + } + } + + // The value came off a row of the very column it is being minted for, so + // anything else is a wiring bug rather than bad input - and quietly writing + // `"[object Object]"` into a cursor would hide it until somebody turned a + // page. + throw new Error( + `Cannot build a pagination cursor from a ${typeof value} value of "${column.name}".`, + ); +}; + +/** + * The cursor value, validated against the column it claims to describe. + * + * Validation rather than coercion, because the cursor is opaque but not signed: + * a client can edit it. `Boolean("false")` is `true`, `Number("")` is `0`, and + * `BigInt("nonsense")` throws a `SyntaxError` that would surface as a 500 - so + * every kind checks the shape it expects and refuses anything else with a 400. + * + * Returns the value in the form the SQL comparison needs: a real `boolean`, + * `number` or `bigint` for those kinds, and for a temporal column the + * **canonical text**, which the predicate binds with an explicit cast so + * Postgres parses it at full precision. + * + * A temporal value is checked for more than shape. `2026-02-30` and + * `2026-08-09 23:60:00` look like timestamps and are not moments, and Postgres + * answers a cast of either with `invalid input syntax` - a 500 produced by a + * query string. Both are refused here, so nothing impossible is ever bound. + */ +export const cursorValueForColumn = ( + column: PgColumn, + value: PaginationCursorValue, +): unknown => { + if (value === null) return null; + + switch (kindOf(column)) { + case "bigint": { + // A decimal string, and nothing else: `BigInt("1.5")` and `BigInt("")` + // are a `SyntaxError` and a `0` respectively, and neither is an answer. + if (typeof value !== "string" || !DECIMAL_INTEGER.test(value)) { + throw badRequest(INVALID_CURSOR); + } + + return BigInt(value); + } + case "boolean": { + if (typeof value !== "boolean") throw badRequest(INVALID_CURSOR); + + return value; + } + case "number": { + if (typeof value !== "number" || !Number.isFinite(value)) { + throw badRequest(INVALID_CURSOR); + } + + return value; + } + case "temporal": { + const temporal = temporalTypeOf(column); + if ( + typeof value !== "string" || + !temporal || + !isRealTemporal(column, temporal, value) + ) { + throw badRequest(INVALID_CURSOR); + } + + // Kept as text. The predicate casts it back to the column's own type, so + // Postgres does the parsing - at the precision it stored. + return value; + } + default: { + if (typeof value !== "string") throw badRequest(INVALID_CURSOR); + + return value; + } + } +}; + +/** + * Whether this column's value travels as canonical text plus a cast. + * + * True for every temporal type, and it decides two things at once: the page + * query selects `column::text` rather than the column, and the predicate binds + * the cursor back with an explicit cast. Both halves exist so the microseconds + * Postgres stored survive a round trip that JavaScript's millisecond `Date` + * would otherwise truncate. + */ +export const cursorValueIsCanonicalText = (column: PgColumn): boolean => + kindOf(column) === "temporal"; + +export const encodePaginationCursor = (cursor: PaginationCursor): string => + Buffer.from(JSON.stringify(cursor), "utf8").toString("base64url"); + +/** A bare integer, which is what every cursor was before this. */ +const LEGACY_CURSOR = /^[1-9]\d{0,14}$/; + +const isCursorValue = (value: unknown): value is PaginationCursorValue => + value === null || + typeof value === "boolean" || + typeof value === "number" || + typeof value === "string"; + +/** + * Reads a cursor, or refuses the request. + * + * Three ways this says no, and each of them is a `400` rather than a page of + * wrong rows: + * + * 1. **garbage** - not base64url, not JSON, or not the shape; + * 2. **the wrong column** - a cursor minted while the list was ordered by + * `updatedAt`, replayed against a list now ordered by `title`. The two + * describe different sequences, so the position means nothing; + * 3. **a legacy numeric cursor on a non-primary-key ordering** - the exact case + * that used to skip rows. A bare number is still accepted when the list is + * ordered by its identifier, because there it really is the whole tuple. + * + * The *value* is checked separately, against the column - see + * {@link cursorValueForColumn} - because only the caller knows which column this + * request is ordered by. + */ +export const decodePaginationCursor = ( + raw: string, + { column, primaryKey }: { column: string; primaryKey: string }, +): PaginationCursor => { + const trimmed = raw.trim(); + if (trimmed === "") throw badRequest(INVALID_CURSOR); + + if (LEGACY_CURSOR.test(trimmed)) { + if (column !== primaryKey) { + throw badRequest( + `This cursor cannot be used with the "${column}" ordering. Start from the first page.`, + ); + } + const id = Number(trimmed); + + return { column, id, value: id }; + } + + let parsed: unknown; + try { + parsed = JSON.parse( + Buffer.from(trimmed, "base64url").toString("utf8"), + ) as unknown; + } catch { + throw badRequest(INVALID_CURSOR); + } + + if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) { + throw badRequest(INVALID_CURSOR); + } + + const candidate = parsed as Record<string, unknown>; + const id = candidate.id; + if ( + typeof candidate.column !== "string" || + typeof id !== "number" || + !Number.isSafeInteger(id) || + id <= 0 || + !isCursorValue(candidate.value) + ) { + throw badRequest(INVALID_CURSOR); + } + + if (candidate.column !== column) { + throw badRequest( + `This cursor was issued for a different ordering. Start from the first page.`, + ); + } + + return { column, id, value: candidate.value }; +}; diff --git a/packages/vitnode/src/api/lib/with-pagination.ts b/packages/vitnode/src/api/lib/with-pagination.ts index 5a0964d8f..99a9823c3 100644 --- a/packages/vitnode/src/api/lib/with-pagination.ts +++ b/packages/vitnode/src/api/lib/with-pagination.ts @@ -8,65 +8,222 @@ import type { import type { Context } from "hono"; import { z } from "@hono/zod-openapi"; -import { and, asc, count, desc, gt, ilike, lt, or } from "drizzle-orm"; +import { + and, + asc, + count, + desc, + eq, + gt, + ilike, + isNotNull, + isNull, + lt, + or, + sql, +} from "drizzle-orm"; +import { HTTPException } from "hono/http-exception"; +import type { PaginationCursor } from "./pagination-cursor"; + +import { + cursorValueForColumn, + cursorValueIsCanonicalText, + cursorValueOf, + decodePaginationCursor, + encodePaginationCursor, + isCursorSortableColumn, +} from "./pagination-cursor"; + +/** Nobody may ask for more than this in one page, whatever they send. */ +const MAX_PAGE_SIZE = 100; + +/** + * The column a page query carries purely so its rows can be turned into cursors. + * + * Selected by the **same statement** that returns the rows, and removed again + * before anything leaves this module. It exists because a cursor has to describe + * the position the returned row actually occupied, and the only way to be + * certain of that is to read the two out of one snapshot. + * + * Prefixed so it cannot collide with a column name, and stripped rather than + * documented, because it is pagination's business and nobody else's. + */ +export const PAGINATION_CURSOR_FIELD = "__cursorValue"; + +/** What a page query must spread into its projection. */ +export type PaginationCursorSelection = Record< + typeof PAGINATION_CURSOR_FIELD, + PgColumn | SQL<string> +>; + +/** + * Reads `first`, `last` and `cursor`, or refuses the request with a 400. + * + * Refusing rather than repairing is the change worth noting. `first=0` used to + * clamp its way into a one-row page that reported `hasNextPage: true`, and + * `first=abc` became `NaN` and fell through to the default page size - both of + * them a request nobody made, answered as if they had. Every one of these is now + * a stable 400, and the route schema rejects most of them a step earlier. + */ function parsePaginationParams(params: { query: { cursor?: string; first?: string; last?: string }; -}): { cursor?: number; first?: number; last?: number } { - const cursor = params.query.cursor - ? parseInt(params.query.cursor, 10) - : undefined; - const first = params.query.first - ? Math.min(parseInt(params.query.first, 10), 100) - : undefined; - const last = params.query.last - ? Math.min(parseInt(params.query.last, 10), 100) - : undefined; +}): { cursor?: string; first?: number; last?: number } { + const size = (raw: string | undefined, name: string): number | undefined => { + if (raw === undefined || raw === "") return undefined; + + const parsed = Number(raw); + if (!Number.isSafeInteger(parsed) || parsed < 1) { + throw new HTTPException(400, { + message: `"${name}" must be a whole number greater than zero.`, + }); + } + + return Math.min(parsed, MAX_PAGE_SIZE); + }; + + const first = size(params.query.first, "first"); + const last = size(params.query.last, "last"); if (first !== undefined && last !== undefined) { - throw new Error("Cannot specify both first and last"); - } - if (first !== undefined && first < 0) { - throw new Error("first must be positive"); - } - if (last !== undefined && last < 0) { - throw new Error("last must be positive"); + throw new HTTPException(400, { + message: 'Use either "first" or "last", not both.', + }); } - return { cursor, first, last }; + const cursor = params.query.cursor?.trim(); + + return { cursor: cursor === "" ? undefined : cursor, first, last }; } -function getOrderFn( +/** + * Which way the rows really come back. + * + * Backward pagination runs the query in reverse and flips the page afterwards, + * so the *effective* SQL direction is not the one the caller asked for - and + * the cursor predicate has to describe the effective one, or it would be reading + * a sequence the `ORDER BY` is not producing. + */ +function effectiveDirection( isForward: boolean, order: "asc" | "desc", -): typeof asc | typeof desc { - if (isForward) { - return order === "asc" ? asc : desc; - } +): "asc" | "desc" { + if (isForward) return order; - return order === "asc" ? desc : asc; + return order === "asc" ? "desc" : "asc"; } -function buildWhereWithCursor< - Primary extends ColumnBaseConfig<"number", string>, ->( - baseWhere: SQL | undefined, - cursor: number | undefined, - isForward: boolean, - order: "asc" | "desc", - table: PgTable, - primaryCursor: PgColumn<Primary>, -): SQL | undefined { - if (!cursor) return baseWhere; +/** + * `and`/`or` given at least one defined condition always produce SQL. + * + * Stated as a check rather than a non-null assertion: the assertion would be a + * claim about code somewhere else, and this is a claim about the two lines above + * it - which is the kind that stays true. + */ +function required(value: SQL | undefined): SQL { + if (!value) throw new Error("Expected a pagination condition."); + + return value; +} + +/** + * "Strictly after this position, in this direction." + * + * The whole keyset, written out. `(column, id)` is the ordered tuple, so the + * predicate is the tuple comparison - not a comparison of one half of it: + * + * ```sql + * column > :value OR (column = :value AND id > :id) -- ascending + * column < :value OR (column = :value AND id < :id) -- descending + * ``` + * + * `:value` comes from the **cursor** and nowhere else. That is the invariant a + * cursor exists to provide: it is the position as it stood when the page was + * generated, so editing the row that happened to sit on the boundary must not + * move it. Reading the row's current value instead would mean one edit silently + * skips every row the ordering used to have between the old position and the + * new one - and deleting it would leave no position at all. + * + * The `NULL` branches are the part that is easy to get wrong. Postgres sorts + * `NULLS LAST` for `ASC` and `NULLS FIRST` for `DESC`, and `column > NULL` is + * `NULL` rather than true - so a nullable order column needs the null block + * named explicitly, or a page boundary landing on it would end the walk early + * and silently. + */ +function buildCursorCondition({ + column, + cursor, + direction, + isPrimaryOrder, + primary, +}: { + column: PgColumn; + cursor: PaginationCursor; + direction: "asc" | "desc"; + isPrimaryOrder: boolean; + primary: PgColumn; +}): SQL { + const after = direction === "asc" ? gt : lt; + + // The identifier is the whole tuple when the list is ordered by it, so there + // is no second half to compare and no null block to worry about. + if (isPrimaryOrder) return after(primary, cursor.id); + + const boundary = boundaryValue(column, cursor); + + if (direction === "asc") { + // NULLS LAST: a null cursor is inside the trailing block, and everything + // that is not null is already behind us. + if (cursor.value === null) { + return required(and(isNull(column), gt(primary, cursor.id))); + } + + return required( + or( + gt(column, boundary), + and(eq(column, boundary), gt(primary, cursor.id)), + isNull(column), + ), + ); + } + + // NULLS FIRST: a null cursor is inside the *leading* block, so the rest of + // that block comes first and every non-null row follows it. + if (cursor.value === null) { + return required( + or(and(isNull(column), lt(primary, cursor.id)), isNotNull(column)), + ); + } + + return required( + or(lt(column, boundary), and(eq(column, boundary), lt(primary, cursor.id))), + ); +} - const cursorFilter = - (isForward && order === "asc") || (!isForward && order === "desc") - ? gt - : lt; +/** + * The cursor's own value, bound so Postgres compares it at full precision. + * + * Two shapes, because two kinds of value survive a round trip differently: + * + * - a **temporal** value travels as the database's own `::text` and is bound + * back with an explicit cast, so Postgres parses the microseconds it wrote. + * Binding a JavaScript `Date` here would silently truncate to milliseconds and + * exclude the whole millisecond the cursor came from. + * - **everything else** - a number, a string, a boolean, a bigint - is exact in + * JavaScript already, so it goes through the column's own encoder. + * + * `getSQLType()` is derived from the schema rather than from the request, which + * is what makes `sql.raw` safe here; the value itself is always a bound + * parameter. + */ +function boundaryValue(column: PgColumn, cursor: PaginationCursor): SQL { + const value = cursorValueForColumn(column, cursor.value); - const cursorWhere = cursorFilter(table[primaryCursor.name], cursor); + if (cursorValueIsCanonicalText(column)) { + return sql`${String(value)}::${sql.raw(column.getSQLType())}`; + } - return baseWhere ? and(baseWhere, cursorWhere) : cursorWhere; + return sql`${sql.param(value, column)}`; } function buildSearchWhere( @@ -122,6 +279,14 @@ export async function withPagination< }; primaryCursor: PgColumn<Primary>; query: (args: { + /** + * Spread this into the projection: `.select({ ...fields, ...cursorSelection })`. + * + * Not optional in practice. It is how the cursor value is read out of the + * same statement as the row, and a query that omits it can only be paged by + * a column it happens to have selected itself. + */ + cursorSelection: PaginationCursorSelection; limit: number | Placeholder<string, unknown>; orderBy: SQL; where: SQL | undefined; @@ -130,21 +295,57 @@ export async function withPagination< table: Omit<PgTableWithColumns<T>, "enableRLS">; where?: SQL; }): Promise<{ - edges: QueryMin[]; + edges: Omit<QueryMin, typeof PAGINATION_CURSOR_FIELD>[]; pageInfo: { count: number; - endCursor: null | number; + /** An opaque cursor. Hand it back as `cursor`; never parse it. */ + endCursor: null | string; hasNextPage: boolean; hasPreviousPage: boolean; - startCursor: null | number; + startCursor: null | string; totalCount: number; }; }> { - const { cursor, first, last } = parsePaginationParams(params); + const { cursor: rawCursor, first, last } = parsePaginationParams(params); const isForward = last === undefined; - const orderFn = getOrderFn(isForward, orderByFromParams.order); - const orderBy: SQL = orderFn(table[orderByFromParams.column.name]); + const direction = effectiveDirection(isForward, orderByFromParams.order); + const orderFn = direction === "asc" ? asc : desc; + + const primary = table[primaryCursor.name]; + const orderName = orderByFromParams.column.name; + const orderColumn = table[orderName] as PgColumn; + const isPrimaryOrder = orderName === primaryCursor.name; + + // A column with no total order Postgres and JavaScript agree on cannot be + // paged at all, so it is refused rather than served for one page and then + // quietly wrong on the next. + if (!isCursorSortableColumn(orderColumn)) { + throw new HTTPException(400, { + message: `Results cannot be ordered by "${orderName}".`, + }); + } + + /** + * The ordered tuple, `(requested column, identifier)`. + * + * The tiebreaker is not decoration: without it the ordering is partial, so + * every row sharing an `updatedAt` sits wherever Postgres feels like putting + * it, and a page boundary landing inside a tie skips or repeats rows. With it + * the ordering is total - and the cursor predicate below compares the *same* + * tuple, which is the invariant this whole module rests on. + */ + const orderBy: SQL = isPrimaryOrder + ? orderFn(primary) + : sql`${orderFn(orderColumn)}, ${orderFn(primary)}`; + + const cursor = + rawCursor === undefined + ? undefined + : decodePaginationCursor(rawCursor, { + column: orderName, + primaryKey: primaryCursor.name, + }); const searchWhere = buildSearchWhere(search, params.query.search); const baseWhere = @@ -152,54 +353,195 @@ export async function withPagination< ? and(whereFromParams, searchWhere) : (whereFromParams ?? searchWhere); - const where = buildWhereWithCursor( - baseWhere, - cursor, - isForward, - orderByFromParams.order, - table, - primaryCursor, - ); + const cursorWhere = cursor + ? buildCursorCondition({ + column: orderColumn, + cursor, + direction, + isPrimaryOrder, + primary, + }) + : undefined; + const where = + baseWhere && cursorWhere + ? and(baseWhere, cursorWhere) + : (baseWhere ?? cursorWhere); const totalCount = await fetchTotalCount(c, table, baseWhere); + /** + * The cursor value, projected by the page query itself. + * + * A temporal column goes through `::text` so no microsecond is lost on the way + * out; everything else is exact in JavaScript already and is selected as it + * is. Either way it rides along with the row, which is the point: a cursor + * minted from a *second* read would describe wherever the boundary row had got + * to by then, not where it was when it was chosen for this page. + */ + const cursorSelection: PaginationCursorSelection = { + [PAGINATION_CURSOR_FIELD]: cursorValueIsCanonicalText(orderColumn) + ? sql<string>`${orderColumn}::text` + : orderColumn, + }; + const limit = (first ?? last ?? 50) + 1; - const edges = await query({ limit, where, orderBy }); + const edges = await query({ cursorSelection, limit, where, orderBy }); const requested = first ?? last ?? edges.length; const hasMore = edges.length > requested; const slicedEdges = edges.slice(0, requested); const finalEdges = isForward ? slicedEdges : slicedEdges.reverse(); - const startCursor: null | number = - (finalEdges[0]?.[primaryCursor.name] as number) ?? null; - const endCursor: null | number = - (finalEdges.at(-1)?.[primaryCursor.name] as number) ?? null; + const boundaries = cursorsFrom({ + edges: finalEdges, + orderColumn, + orderName, + primaryName: primaryCursor.name, + }); return { pageInfo: { totalCount, count: finalEdges.length, - hasNextPage: isForward ? hasMore : !!cursor, - hasPreviousPage: isForward ? !!cursor : hasMore, - startCursor, - endCursor, + // An empty page has nothing to page from, so it never advertises a + // neighbour it cannot hand out a cursor for. + hasNextPage: + finalEdges.length === 0 ? false : isForward ? hasMore : Boolean(cursor), + hasPreviousPage: + finalEdges.length === 0 ? false : isForward ? Boolean(cursor) : hasMore, + ...boundaries, }, - edges: finalEdges, + edges: finalEdges.map(withoutCursorField), }; } +/** + * The row as the caller asked for it, with pagination's own column taken back. + * + * The internal value is projected for one purpose and has no business in an + * admin response, a public response, an OpenAPI schema, a search document or a + * revision snapshot - all of which are built from what this returns. + */ +function withoutCursorField<QueryMin extends Record<string, unknown>>( + row: QueryMin, +): Omit<QueryMin, typeof PAGINATION_CURSOR_FIELD> { + if (!(PAGINATION_CURSOR_FIELD in row)) return row; + + const { [PAGINATION_CURSOR_FIELD]: _cursorValue, ...rest } = row; + + return rest; +} + +/** + * The two cursors a page hands back, read off the page itself. + * + * No query. That is the entire design: the value and the row come out of one + * `SELECT`, so the tuple a cursor names is the tuple that actually decided where + * the row sat. + * + * It used to be a second `SELECT` of the boundary rows by id, which looked + * harmless and was not. Between the page query and that lookup another writer + * can move the boundary row - so a row chosen at `(10:00, 42)` would be handed + * back as a cursor saying `(14:00, 42)`, and the next page would start after + * 14:00 and skip everything in between. A `DELETE` in the same window was worse: + * the lookup returned nothing, the value became `null`, and for a nullable + * ordering `null` is a *real* position inside the null block - so the walk + * jumped there and abandoned the rest of the collection. Both are gone by + * construction rather than by locking. + */ +function cursorsFrom({ + edges, + orderColumn, + orderName, + primaryName, +}: { + edges: readonly Record<string, unknown>[]; + orderColumn: PgColumn; + orderName: string; + primaryName: string; +}): { endCursor: null | string; startCursor: null | string } { + const first = edges[0]; + const last = edges.at(-1); + if (!first || !last) return { endCursor: null, startCursor: null }; + + /** + * Where the boundary value comes from, in order of preference. + * + * The projected field is the answer for every query built through this module. + * A query that omits it can still be paged by a column it selected itself - + * exact for a number, a string or a boolean, and from the same statement, so + * the invariant holds. A temporal column is the one case with no safe + * fallback: the row carries a `Date` that has already dropped the microseconds + * the next comparison needs, so minting from it would hand out a cursor that + * silently re-reads part of the page it came from. + */ + const valueOf = (row: Record<string, unknown>): unknown => { + if (PAGINATION_CURSOR_FIELD in row) return row[PAGINATION_CURSOR_FIELD]; + if (!cursorValueIsCanonicalText(orderColumn) && orderName in row) { + return row[orderName]; + } + + throw new Error( + `The page query for "${orderName}" must spread \`cursorSelection\` into its projection, so the cursor value is read from the same statement as the row.`, + ); + }; + + const mint = (row: Record<string, unknown>): string => + encodePaginationCursor({ + column: orderName, + id: Number(row[primaryName]), + value: cursorValueOf(orderColumn, valueOf(row)), + }); + + return { endCursor: mint(last), startCursor: mint(first) }; +} + +/** A positive whole number, as a query string carries it. */ +const zodPageSize = z + .string() + .regex(/^\d+$/, "Must be a whole number.") + .refine(value => Number(value) >= 1, "Must be greater than zero.") + .refine(value => Number.isSafeInteger(Number(value)), "Too large."); + export const zodPaginationPageInfo = z.object({ totalCount: z.number(), count: z.number(), hasNextPage: z.boolean(), hasPreviousPage: z.boolean(), - startCursor: z.number().nullable(), - endCursor: z.number().nullable(), + /** + * Opaque. It encodes the ordered tuple the next page continues from, so it is + * meaningless outside the ordering that produced it - hand it back unchanged. + */ + startCursor: z.string().nullable(), + endCursor: z.string().nullable(), }); -export const zodPaginationQuery = z.object({ - cursor: z.string().optional(), - first: z.string().optional(), - last: z.string().optional(), -}); +/** + * The pagination half of a list route's query, validated at the edge. + * + * Every rule that can be stated here is stated here rather than left to the + * internals, so a bad page size is a 400 from the route's own contract - and + * appears in the OpenAPI document - instead of something the handler discovers + * later. `parsePaginationParams` re-checks all of it, because a service can be + * called directly and a plugin can build a route without this schema. + * + * The cursor is only shape-checked here: it is opaque, so "looks like a cursor" + * is all a request schema can honestly say. Whether it decodes, and whether it + * belongs to *this* ordering, is decided where the ordering is known. + */ +export const zodPaginationQuery = z + .object({ + cursor: z + .string() + .min(1) + .max(512) + // base64url, or a legacy numeric cursor. Anything else cannot be one. + .regex(/^[A-Za-z0-9_-]+$/, "Invalid cursor.") + .optional(), + first: zodPageSize.optional(), + last: zodPageSize.optional(), + }) + .refine( + query => query.first === undefined || query.last === undefined, + 'Use either "first" or "last", not both.', + ); diff --git a/packages/vitnode/src/api/models/search.test.ts b/packages/vitnode/src/api/models/search.test.ts index c38cfe079..f9f9a034c 100644 --- a/packages/vitnode/src/api/models/search.test.ts +++ b/packages/vitnode/src/api/models/search.test.ts @@ -7,6 +7,7 @@ import { core_search_index } from "@/database/search"; import type { SearchDocument, SearchProviderApiPlugin } from "./search"; +import { PostgresSearchAdapter } from "../adapters/search/postgres"; import { assertSearchProviderCapabilities, normalizeSearchIndexerPage, @@ -376,3 +377,49 @@ describe("assertSearchProviderCapabilities", () => { ).not.toThrow(); }); }); + +/** + * The provider half of a search diagnostic. + * + * `SearchModel.index` writes the canonical row and *then* hands the document to + * the provider, so the two can disagree - and a diagnostic that cannot ask the + * provider would report the canonical table's health as the whole story. + */ +describe("provider diagnostics", () => { + const modelFor = (provider: SearchProviderApiPlugin) => + new SearchModel({ + get: (key: string) => + key === "core" ? { search: { adapter: provider } } : undefined, + } as never); + + it("reports the bundled Postgres provider as canonical storage", () => { + // Its store *is* `core_search_index`, so a diagnostic can use the canonical + // count rather than paying for a second one over the same rows. + expect(modelFor(PostgresSearchAdapter()).isCanonicalStorage()).toBe(true); + }); + + it("reports a mirroring provider as not canonical", () => { + expect(modelFor(createProvider()).isCanonicalStorage()).toBe(false); + }); + + it("answers null when the provider offers no count", async () => { + // `null` is not zero and not healthy - it means nobody looked, and the + // caller has to report that as unverified. + await expect( + modelFor(createProvider()).countDocuments({ itemType: "blog_post" }), + ).resolves.toBeNull(); + }); + + it("passes the item type and language straight through", async () => { + const count = vi.fn().mockResolvedValue(12); + const model = modelFor({ ...createProvider(), count }); + + await expect( + model.countDocuments({ itemType: "blog_post", languageCode: "pl" }), + ).resolves.toBe(12); + expect(count.mock.calls[0][1]).toEqual({ + itemType: "blog_post", + languageCode: "pl", + }); + }); +}); diff --git a/packages/vitnode/src/api/models/search.ts b/packages/vitnode/src/api/models/search.ts index cdd0e98e6..4d3db855b 100644 --- a/packages/vitnode/src/api/models/search.ts +++ b/packages/vitnode/src/api/models/search.ts @@ -86,6 +86,17 @@ export interface SearchResult { export interface SearchProviderCapabilities { authorBoost: boolean; + /** + * Whether the provider's store **is** `core_search_index`. + * + * True only for the bundled Postgres provider, which queries the canonical + * table directly rather than mirroring it. Diagnostics use this to skip a + * second count of the same rows: canonical and provider are one storage, so + * asking twice would cost a query to learn something already known. + * + * A mirroring provider - anything with its own store - must leave it unset. + */ + canonicalStorage?: boolean; facets: boolean; /** * Whether {@link SearchProviderApiPlugin.delete} honours its `languageCode`. @@ -285,6 +296,22 @@ export interface SearchProviderApiPlugin { bulkIndex: (c: Context, docs: SearchDocument[]) => Promise<void>; capabilities?: SearchProviderCapabilities; clear: (c: Context, itemType?: string) => Promise<void>; + /** + * How many documents the provider holds for one collection. + * + * Optional, and its absence is meaningful: a provider that cannot be counted + * is reported as **unverified** rather than healthy, because "we did not look" + * and "we looked and it was fine" are different answers and only one of them + * is worth acting on. + * + * It must count rather than fetch - `_count` on Elasticsearch, `COUNT(*)` on a + * table - and honour `languageCode` where the provider stores one document per + * translation. Omitting the language means every language. + */ + count?: ( + c: Context, + args: { itemType: string; languageCode?: string }, + ) => Promise<number>; /** * Removes one item's documents. * @@ -444,6 +471,23 @@ export class SearchModel { await this.provider().clear(this.c, itemType); } + /** + * How many documents the **provider** holds, or `null` when it cannot say. + * + * `null` is not zero and not healthy: it means the provider offers no + * diagnostics, and a caller has to report that as unverified rather than + * turning an absence of evidence into a clean bill of health. + */ + async countDocuments(args: { + itemType: string; + languageCode?: string; + }): Promise<null | number> { + const provider = this.provider(); + if (!provider.count) return null; + + return await provider.count(this.c, args); + } + /** * Removes one item from the index, in one language or in all of them. * @@ -484,6 +528,16 @@ export class SearchModel { await this.provider().index(this.c, clean); } + /** + * Whether the active provider's store is the canonical table itself. + * + * Diagnostics ask this before counting twice - see + * {@link SearchProviderCapabilities.canonicalStorage}. + */ + isCanonicalStorage(): boolean { + return this.provider().capabilities?.canonicalStorage === true; + } + name(): string { return this.provider().name; } diff --git a/packages/vitnode/src/api/modules/admin/advanced/cron/routes/get.route.ts b/packages/vitnode/src/api/modules/admin/advanced/cron/routes/get.route.ts index b49a25c9c..d12a9d01a 100644 --- a/packages/vitnode/src/api/modules/admin/advanced/cron/routes/get.route.ts +++ b/packages/vitnode/src/api/modules/admin/advanced/cron/routes/get.route.ts @@ -1,3 +1,4 @@ +import { getTableColumns } from "drizzle-orm"; import z from "zod"; import { buildRoute } from "@/api/lib/route"; @@ -56,10 +57,10 @@ export const getCronsRoute = buildRoute({ }, c, primaryCursor: core_cron.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") - .select() + .select({ ...getTableColumns(core_cron), ...cursorSelection }) .from(core_cron) .where(where) .orderBy(orderBy) diff --git a/packages/vitnode/src/api/modules/admin/advanced/queue/routes/get.route.ts b/packages/vitnode/src/api/modules/admin/advanced/queue/routes/get.route.ts index 40181afa9..affd8a4ae 100644 --- a/packages/vitnode/src/api/modules/admin/advanced/queue/routes/get.route.ts +++ b/packages/vitnode/src/api/modules/admin/advanced/queue/routes/get.route.ts @@ -1,4 +1,4 @@ -import { inArray } from "drizzle-orm"; +import { getTableColumns, inArray } from "drizzle-orm"; import z from "zod"; import { buildRoute } from "@/api/lib/route"; @@ -73,10 +73,10 @@ export const getQueueTasksRoute = buildRoute({ c, primaryCursor: core_queue.id, where: statuses.length ? inArray(core_queue.status, statuses) : undefined, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") - .select() + .select({ ...getTableColumns(core_queue), ...cursorSelection }) .from(core_queue) .where(where) .orderBy(orderBy) diff --git a/packages/vitnode/src/api/modules/admin/debug/debug.admin.module.ts b/packages/vitnode/src/api/modules/admin/debug/debug.admin.module.ts index dc77d8d7d..dd9af296c 100644 --- a/packages/vitnode/src/api/modules/admin/debug/debug.admin.module.ts +++ b/packages/vitnode/src/api/modules/admin/debug/debug.admin.module.ts @@ -1,6 +1,7 @@ import { CONFIG_PLUGIN } from "../../../../config"; import { buildModule } from "../../../lib/module"; import { clearSearchDebugAdminRoute } from "./routes/clear-search.route"; +import { contentStatusDebugAdminRoute } from "./routes/content-status.route"; import { integrationsDebugAdminRoute } from "./routes/integrations.route"; import { logsDebugAdminRoute } from "./routes/logs.route"; import { queueDebugAdminRoute } from "./routes/queue.route"; @@ -20,6 +21,7 @@ export const debugAdminModule = buildModule({ searchStatusDebugAdminRoute, rebuildSearchDebugAdminRoute, clearSearchDebugAdminRoute, + contentStatusDebugAdminRoute, sendTestEmailDebugAdminRoute, testAiDebugAdminRoute, testStorageUploadDebugAdminRoute, diff --git a/packages/vitnode/src/api/modules/admin/debug/routes/content-status.route.ts b/packages/vitnode/src/api/modules/admin/debug/routes/content-status.route.ts new file mode 100644 index 000000000..fe7c2ae66 --- /dev/null +++ b/packages/vitnode/src/api/modules/admin/debug/routes/content-status.route.ts @@ -0,0 +1,112 @@ +import { z } from "zod"; + +import { buildRoute } from "@/api/lib/route"; +import { CONFIG_PLUGIN } from "@/config"; +import { contentEngineDiagnostics } from "@/content/server/diagnostics"; + +const localeDriftSchema = z.object({ + /** Documents `core_search_index` holds for this locale. */ + canonicalIndexed: z.number(), + canonicalHealthy: z.boolean(), + /** Published rows - or published translations - the database holds. */ + expected: z.number(), + /** `""` for a content type that is not localized. */ + locale: z.string(), + /** `null` when the provider offers no diagnostics - unverified, not healthy. */ + providerHealthy: z.boolean().nullable(), + providerIndexed: z.number().nullable(), +}); + +const contentTypeSchema = z.object({ + contentTypeId: z.string(), + features: z.object({ + editorial: z.boolean(), + localization: z.boolean(), + publicApi: z.boolean(), + publication: z.boolean(), + scheduling: z.boolean(), + search: z.boolean(), + }), + pluginId: z.string(), + /** `null` for a content type without `search`. */ + search: z + .object({ + canonicalHealthy: z.boolean(), + /** Documents `core_search_index` holds, every locale. */ + canonicalIndexedTotal: z.number(), + contentTypeId: z.string(), + /** Published rows - or translations - the database holds, every locale. */ + expectedTotal: z.number(), + /** Canonical **and** provider both agree. Unverified is not healthy. */ + healthy: z.boolean(), + locales: z.array(localeDriftSchema), + provider: z.object({ + /** Why the provider could not be counted, when that is the answer. */ + error: z.string().optional(), + healthy: z.boolean().nullable(), + /** + * Every document the provider holds, in any locale. + * + * The guard against a document left behind in a locale the database no + * longer knows about, which per-locale counts can never ask for. + */ + indexedTotal: z.number().nullable(), + name: z.string(), + /** Whether the provider was actually asked. */ + verified: z.boolean(), + }), + }) + .nullable(), + /** `null` for a content type without scheduling. */ + schedules: z + .object({ + /** Transitions that committed but were never announced. */ + failedEffects: z.number(), + pending: z.number(), + withErrors: z.number(), + }) + .nullable(), +}); + +/** + * What the Content Engine looks like from the outside, right now. + * + * Sits beside `/search/status` under the same `system: can_view` permission, + * and answers the questions that one cannot: `/search/status` reports what is + * *in* the index, and this reports what the **database** says should be there. + * A collection can be 100% covered by the first and still be missing every + * Polish document, because coverage is measured against the indexer's own count + * and drift is measured against the rows. + * + * Aggregates only - two counts per content type - so it is safe to open on an + * install with a large table. Nothing here mutates anything; repairing drift is + * `/search/rebuild`, which is a separate decision and a separate route. + */ +export const contentStatusDebugAdminRoute = buildRoute({ + pluginId: CONFIG_PLUGIN.pluginId, + adminStaffPermission: { module: "system", permission: "can_view" }, + route: { + method: "get", + description: + "Report every registered content type, its search index drift per locale, and its outstanding scheduled-effect failures.", + path: "/content/status", + responses: { + 200: { + content: { + "application/json": { + schema: z.object({ + contentTypes: z.array(contentTypeSchema), + /** No scheduled transition committed without being announced. */ + effectsHealthy: z.boolean(), + /** `searchHealthy && effectsHealthy`. */ + healthy: z.boolean(), + searchHealthy: z.boolean(), + }), + }, + }, + description: "Content Engine status", + }, + }, + }, + handler: async c => c.json(await contentEngineDiagnostics(c)), +}); diff --git a/packages/vitnode/src/api/modules/admin/debug/routes/logs.route.ts b/packages/vitnode/src/api/modules/admin/debug/routes/logs.route.ts index 74266f343..0ec2eb213 100644 --- a/packages/vitnode/src/api/modules/admin/debug/routes/logs.route.ts +++ b/packages/vitnode/src/api/modules/admin/debug/routes/logs.route.ts @@ -66,10 +66,11 @@ export const logsDebugAdminRoute = buildRoute({ query, }, primaryCursor: core_logs.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_logs.id, pluginId: core_logs.pluginId, type: core_logs.type, diff --git a/packages/vitnode/src/api/modules/admin/files/routes/list.route.ts b/packages/vitnode/src/api/modules/admin/files/routes/list.route.ts index 873605bf3..5ce24159c 100644 --- a/packages/vitnode/src/api/modules/admin/files/routes/list.route.ts +++ b/packages/vitnode/src/api/modules/admin/files/routes/list.route.ts @@ -90,10 +90,11 @@ export const listFilesAdminRoute = buildRoute({ c, primaryCursor: core_files.id, search: [core_files.name], - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_files.id, name: core_files.name, key: core_files.key, diff --git a/packages/vitnode/src/api/modules/admin/roles/routes/list.route.ts b/packages/vitnode/src/api/modules/admin/roles/routes/list.route.ts index efe9f5138..4985eba3b 100644 --- a/packages/vitnode/src/api/modules/admin/roles/routes/list.route.ts +++ b/packages/vitnode/src/api/modules/admin/roles/routes/list.route.ts @@ -100,10 +100,11 @@ export const listRolesAdminRoute = buildRoute({ ) : undefined, primaryCursor: core_roles.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_roles.id, color: core_roles.color, protected: core_roles.protected, diff --git a/packages/vitnode/src/api/modules/admin/staff/routes/admins.route.ts b/packages/vitnode/src/api/modules/admin/staff/routes/admins.route.ts index f1bfbd19f..1b5b4d786 100644 --- a/packages/vitnode/src/api/modules/admin/staff/routes/admins.route.ts +++ b/packages/vitnode/src/api/modules/admin/staff/routes/admins.route.ts @@ -38,10 +38,11 @@ export const listAdminsStaffAdminRoute = buildRoute({ query, }, primaryCursor: core_admin_permissions.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_admin_permissions.id, roleId: core_admin_permissions.roleId, userId: core_admin_permissions.userId, diff --git a/packages/vitnode/src/api/modules/admin/staff/routes/moderators.route.ts b/packages/vitnode/src/api/modules/admin/staff/routes/moderators.route.ts index 1ac4f1eb7..b0bcb0353 100644 --- a/packages/vitnode/src/api/modules/admin/staff/routes/moderators.route.ts +++ b/packages/vitnode/src/api/modules/admin/staff/routes/moderators.route.ts @@ -38,10 +38,11 @@ export const listModeratorsStaffAdminRoute = buildRoute({ query, }, primaryCursor: core_moderators_permissions.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_moderators_permissions.id, roleId: core_moderators_permissions.roleId, userId: core_moderators_permissions.userId, diff --git a/packages/vitnode/src/api/modules/admin/users/routes/list.route.ts b/packages/vitnode/src/api/modules/admin/users/routes/list.route.ts index bd04bd9dc..6ddc5104a 100644 --- a/packages/vitnode/src/api/modules/admin/users/routes/list.route.ts +++ b/packages/vitnode/src/api/modules/admin/users/routes/list.route.ts @@ -90,10 +90,11 @@ export const listUsersAdminRoute = buildRoute({ search: [core_users.name, core_users.email, core_users.nameCode], where: roleIds.length ? inArray(core_users.roleId, roleIds) : undefined, primaryCursor: core_users.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_users.id, name: core_users.name, email: core_users.email, diff --git a/packages/vitnode/src/api/modules/admin/users/routes/users.route.ts b/packages/vitnode/src/api/modules/admin/users/routes/users.route.ts index a2e6fe2cc..cb9f70af2 100644 --- a/packages/vitnode/src/api/modules/admin/users/routes/users.route.ts +++ b/packages/vitnode/src/api/modules/admin/users/routes/users.route.ts @@ -59,10 +59,11 @@ export const usersAdminRoute = buildRoute({ query, }, primaryCursor: core_users.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_users.id, name: core_users.name, email: core_users.email, diff --git a/packages/vitnode/src/api/modules/search/routes/search.route.ts b/packages/vitnode/src/api/modules/search/routes/search.route.ts index ee4433064..0256d91ed 100644 --- a/packages/vitnode/src/api/modules/search/routes/search.route.ts +++ b/packages/vitnode/src/api/modules/search/routes/search.route.ts @@ -3,10 +3,24 @@ import { z } from "@hono/zod-openapi"; import { CONFIG_PLUGIN } from "@/config"; import { buildRoute } from "../../../lib/route"; -import { - zodPaginationPageInfo, - zodPaginationQuery, -} from "../../../lib/with-pagination"; +import { zodPaginationQuery } from "../../../lib/with-pagination"; + +/** + * The search index's own page info. + * + * Deliberately not `zodPaginationPageInfo`: that one describes a keyset walk + * over a table and hands out an opaque cursor for the ordered tuple. A search + * page is not that - a relevance-sorted one walks by offset and an ordinary one + * by row id - so it keeps the numeric cursors it has always had. + */ +const zodSearchPageInfo = z.object({ + totalCount: z.number(), + count: z.number(), + hasNextPage: z.boolean(), + hasPreviousPage: z.boolean(), + startCursor: z.number().nullable(), + endCursor: z.number().nullable(), +}); export const zodSearchHitSchema = z.object({ id: z.number(), @@ -57,7 +71,11 @@ export const searchRoute = buildRoute({ "application/json": { schema: z.object({ edges: z.array(zodSearchHitSchema), - pageInfo: zodPaginationPageInfo, + // The search index has its own pagination - a relevance-sorted + // page walks by offset, and an ordinary one by row id - so it + // keeps the numeric cursors it has always had rather than the + // opaque keyset cursor `withPagination` mints for a table. + pageInfo: zodSearchPageInfo, }), }, }, diff --git a/packages/vitnode/src/api/modules/users/files/routes/list.route.ts b/packages/vitnode/src/api/modules/users/files/routes/list.route.ts index d9a40c7d4..dce44b888 100644 --- a/packages/vitnode/src/api/modules/users/files/routes/list.route.ts +++ b/packages/vitnode/src/api/modules/users/files/routes/list.route.ts @@ -72,10 +72,11 @@ export const listUserFilesRoute = buildRoute({ primaryCursor: core_files.id, search: [core_files.name], where: eq(core_files.userId, user.id), - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: core_files.id, name: core_files.name, key: core_files.key, diff --git a/packages/vitnode/src/components/table/content.test.tsx b/packages/vitnode/src/components/table/content.test.tsx index 5099e6650..ce4aa3ebb 100644 --- a/packages/vitnode/src/components/table/content.test.tsx +++ b/packages/vitnode/src/components/table/content.test.tsx @@ -32,10 +32,11 @@ const edges: DemoUser[] = [ const pageInfo = { count: edges.length, - endCursor: 2, + // Opaque strings, as `withPagination` mints them - never row ids. + endCursor: "eyJjb2x1bW4iOiJpZCIsImlkIjoyLCJ2YWx1ZSI6Mn0", hasNextPage: false, hasPreviousPage: false, - startCursor: 1, + startCursor: "eyJjb2x1bW4iOiJpZCIsImlkIjoxLCJ2YWx1ZSI6MX0", totalCount: edges.length, }; diff --git a/packages/vitnode/src/components/table/pagination.tsx b/packages/vitnode/src/components/table/pagination.tsx index 8badd5ff9..461ca4e21 100644 --- a/packages/vitnode/src/components/table/pagination.tsx +++ b/packages/vitnode/src/components/table/pagination.tsx @@ -24,10 +24,10 @@ export const PaginationDataTable = ({ }: { pageInfo: { count: number; - endCursor: null | number; + endCursor: null | string; hasNextPage: boolean; hasPreviousPage: boolean; - startCursor: null | number; + startCursor: null | string; totalCount: number; }; }) => { @@ -93,7 +93,7 @@ export const PaginationDataTable = ({ const params = new URLSearchParams(searchParams.toString()); params.set("last", `${Number(pageSize)}`); if (startCursor) { - params.set("cursor", `${startCursor}`); + params.set("cursor", startCursor); } else { params.delete("cursor"); } @@ -122,7 +122,7 @@ export const PaginationDataTable = ({ const params = new URLSearchParams(searchParams.toString()); params.set("first", `${Number(pageSize)}`); if (endCursor) { - params.set("cursor", `${endCursor}`); + params.set("cursor", endCursor); } else { params.delete("cursor"); } diff --git a/packages/vitnode/src/content/cache.delivery.test.ts b/packages/vitnode/src/content/cache.delivery.test.ts new file mode 100644 index 000000000..d641e3833 --- /dev/null +++ b/packages/vitnode/src/content/cache.delivery.test.ts @@ -0,0 +1,276 @@ +import { describe, expect, it } from "vitest"; + +import { + contentDeliveryRedirectTag, + contentDeliverySitemapTag, + contentDeliveryTag, + contentInvalidationTags, + contentPublicItemTag, + contentPublicListTag, + contentPublicSlugTag, +} from "./cache"; + +/** + * The delivery cache tags, and the promise that a content type without `delivery` + * produces exactly the tags it always produced. + * + * That second half is the important one and is why the assertions are exact strings + * rather than "some revalidation happened": the whole of Stage 8's opt-in claim at + * this layer is that an existing content type's tag list does not move, and only a + * byte comparison can show it. + */ + +const ID = "example.article"; + +describe("delivery tag builders", () => { + it("follows the existing namespace, with the locale after the scope", () => { + expect(contentDeliveryTag(ID, 42)).toBe( + "content:example.article:delivery:42", + ); + expect(contentDeliveryTag(ID, 42, "pl")).toBe( + "content:example.article:delivery:pl:42", + ); + + expect(contentDeliveryRedirectTag(ID, "old-slug")).toBe( + "content:example.article:redirect:old-slug", + ); + expect(contentDeliveryRedirectTag(ID, "stary-slug", "pl")).toBe( + "content:example.article:redirect:pl:stary-slug", + ); + + expect(contentDeliverySitemapTag(ID)).toBe( + "content:example.article:sitemap", + ); + expect(contentDeliverySitemapTag(ID, "pl")).toBe( + "content:example.article:sitemap:pl", + ); + }); + + it("normalizes the locale, so PL and pl expire together", () => { + for (const locale of ["PL", "pl", " pl "]) { + expect(contentDeliveryTag(ID, 1, locale)).toBe( + "content:example.article:delivery:pl:1", + ); + expect(contentDeliverySitemapTag(ID, locale)).toBe( + "content:example.article:sitemap:pl", + ); + } + }); +}); + +describe("contentInvalidationTags without delivery", () => { + it("is byte-identical to the Stage 1-7 output for a flat mutation", () => { + expect( + contentInvalidationTags({ + contentTypeId: ID, + id: 42, + isPublic: true, + slugs: ["old", "new"], + wasPublic: true, + }), + ).toStrictEqual([ + contentPublicListTag(ID), + contentPublicItemTag(ID, 42), + contentPublicSlugTag(ID, "old"), + contentPublicSlugTag(ID, "new"), + ]); + }); + + it("is byte-identical for a localized mutation", () => { + expect( + contentInvalidationTags({ + contentTypeId: ID, + id: 7, + isPublic: true, + locales: [ + { + isPublic: true, + locale: "pl", + slugs: ["stary", "nowy"], + wasPublic: true, + }, + ], + slugs: [], + wasPublic: true, + }), + ).toStrictEqual([ + contentPublicListTag(ID, "pl"), + contentPublicItemTag(ID, 7, "pl"), + contentPublicSlugTag(ID, "stary", "pl"), + contentPublicSlugTag(ID, "nowy", "pl"), + ]); + }); + + it("still returns nothing for a draft edited into another draft", () => { + expect( + contentInvalidationTags({ + contentTypeId: ID, + id: 1, + isPublic: false, + slugs: ["a", "b"], + wasPublic: false, + }), + ).toStrictEqual([]); + }); +}); + +describe("contentInvalidationTags with delivery", () => { + it("adds the delivery metadata tag and one redirect tag per slug", () => { + expect( + contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + id: 42, + isPublic: true, + slugs: ["old", "new"], + wasPublic: true, + }), + ).toStrictEqual([ + contentPublicListTag(ID), + contentPublicItemTag(ID, 42), + contentPublicSlugTag(ID, "old"), + contentPublicSlugTag(ID, "new"), + contentDeliveryTag(ID, 42), + contentDeliveryRedirectTag(ID, "old"), + contentDeliveryRedirectTag(ID, "new"), + ]); + }); + + it("expires the sitemap file whenever its bytes moved", () => { + // A nonlocalized content type's locale-less tag *is* its one sitemap file, so + // `contentChanged` is what expires it - including for a plain title edit, whose + // `<lastmod>` moved even though the URL did not. + const withSitemap = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: true, indexChanged: false } }, + id: 42, + isPublic: true, + slugs: ["same"], + wasPublic: true, + }); + + expect(withSitemap).toContain(contentDeliverySitemapTag(ID)); + + const untouched = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + id: 42, + isPublic: true, + slugs: ["new"], + wasPublic: true, + }); + + expect(untouched).not.toContain(contentDeliverySitemapTag(ID)); + }); + + it("emits the sitemap tag once for a nonlocalized content type", () => { + // `contentChanged` and `indexChanged` name the same tag here, because a + // nonlocalized content type has one file and no index. + const tags = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: true, indexChanged: true } }, + id: 42, + isPublic: true, + slugs: ["new"], + wasPublic: false, + }); + + expect( + tags.filter(tag => tag === contentDeliverySitemapTag(ID)), + ).toHaveLength(1); + }); + + it("expires each locale's sitemap and the index that lists them", () => { + const tags = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: true, indexChanged: true } }, + id: 7, + isPublic: true, + locales: [ + { isPublic: true, locale: "en", slugs: ["hello"], wasPublic: true }, + { isPublic: true, locale: "pl", slugs: ["witaj"], wasPublic: false }, + ], + slugs: [], + wasPublic: true, + }); + + expect(tags).toContain(contentDeliverySitemapTag(ID, "en")); + expect(tags).toContain(contentDeliverySitemapTag(ID, "pl")); + // The locale-less one too, because a language gaining a page changes how many + // files the index lists. + expect(tags).toContain(contentDeliverySitemapTag(ID)); + }); + + it("expires a locale's file without its index on an ordinary edit", () => { + // The rule §3.6 asks for: a title edit rewrites bytes inside an existing file and + // changes neither which files exist nor how many. + const tags = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: true, indexChanged: false } }, + id: 7, + isPublic: true, + locales: [ + { isPublic: true, locale: "pl", slugs: ["witaj"], wasPublic: true }, + ], + slugs: [], + wasPublic: true, + }); + + expect(tags).toContain(contentDeliverySitemapTag(ID, "pl")); + expect(tags).not.toContain(contentDeliverySitemapTag(ID)); + expect(tags).not.toContain(contentDeliverySitemapTag(ID, "en")); + }); + + it("keeps one locale's delivery tags out of another's", () => { + const tags = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + id: 7, + isPublic: true, + locales: [ + { + isPublic: true, + locale: "pl", + slugs: ["stary", "nowy"], + wasPublic: true, + }, + ], + slugs: [], + wasPublic: true, + }); + + expect(tags).toContain(contentDeliveryTag(ID, 7, "pl")); + expect(tags).toContain(contentDeliveryRedirectTag(ID, "stary", "pl")); + expect(tags).not.toContain(contentDeliveryTag(ID, 7, "en")); + expect(tags).not.toContain(contentDeliveryRedirectTag(ID, "stary", "en")); + }); + + it("touches nothing at all for a draft that stayed a draft", () => { + // The delivery tags follow the public ones: a mutation that changed no public + // response should not throw away a warm cache for a feature it did not reach. + expect( + contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + id: 1, + isPublic: false, + slugs: ["a", "b"], + wasPublic: false, + }), + ).toStrictEqual([]); + }); + + it("drops an empty slug rather than tagging a redirect for it", () => { + const tags = contentInvalidationTags({ + contentTypeId: ID, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + id: 42, + isPublic: true, + slugs: ["", "new"], + wasPublic: false, + }); + + expect(tags).not.toContain(contentDeliveryRedirectTag(ID, "")); + expect(tags).toContain(contentDeliveryRedirectTag(ID, "new")); + }); +}); diff --git a/packages/vitnode/src/content/cache.ts b/packages/vitnode/src/content/cache.ts index c7a33d73a..cc09755e7 100644 --- a/packages/vitnode/src/content/cache.ts +++ b/packages/vitnode/src/content/cache.ts @@ -79,6 +79,51 @@ export const contentPublicSlugTag = ( locale?: string, ): string => tag(contentTypeId, "slug", ...localeParts(locale), slug); +/** + * The delivery metadata of one record, in one locale. + * + * Separate from {@link contentPublicItemTag} even though both are keyed by + * identifier, because the two hold different responses: the item tag covers the + * public projection a page renders, and this one covers the canonical path, the + * alternates and the SEO metadata its `<head>` is built from. A page that reads + * both is tagged with both; one that renders only metadata - a `generateMetadata` + * that does not fetch the body - is tagged with this alone and is not thrown away + * when an unrelated field of the record changes. + */ +export const contentDeliveryTag = ( + contentTypeId: string, + id: number, + locale?: string, +): string => tag(contentTypeId, "delivery", ...localeParts(locale), id); + +/** + * One historical URL's redirect lookup, in one locale. + * + * Keyed by the **old** slug, which is what a request for a moved page arrives + * with. The locale is load-bearing for the same reason it is on the slug tag: two + * languages routinely retire the same slug, and a locale-less tag would make one + * language's slug change expire the other language's redirect. + */ +export const contentDeliveryRedirectTag = ( + contentTypeId: string, + slug: string, + locale?: string, +): string => tag(contentTypeId, "redirect", ...localeParts(locale), slug); + +/** + * One content type's sitemap - the whole thing, or one locale's share of it. + * + * Both forms exist and both are expired together by a mutation that changes what + * is listed: a localized content type has one sitemap per language *and* an index + * that enumerates them, and publishing a Polish translation changes the Polish + * file and the number of files. A content type that is not localized only ever + * produces the three-segment form. + */ +export const contentDeliverySitemapTag = ( + contentTypeId: string, + locale?: string, +): string => tag(contentTypeId, "sitemap", ...localeParts(locale)); + /** * How hard a mutation expires the tags it touched. * @@ -107,8 +152,56 @@ export interface ContentLocaleInvalidation { wasPublic: boolean; } +/** + * The delivery half of one mutation's invalidation. + * + * Absent for every content type without `delivery`, which is what makes Stage 8 + * opt-in at the cache layer too: `contentInvalidationTags` returns exactly the + * strings it always returned when this is `undefined`, byte for byte, so nothing + * existing has to be re-tagged and no warm cache is thrown away for a feature the + * content type does not use. + * + * Nothing in here names a locale or a slug of its own: both are already on the + * input - `locales[].slugs` carries the old and the new URL of every locale the + * mutation reached - and deriving the delivery tags from the same data is what + * keeps the public tags and the delivery tags from disagreeing about what moved. + */ +export interface ContentDeliveryInvalidation { + /** What this mutation did to the sitemap. See {@link ContentSitemapChange}. */ + sitemap: ContentSitemapChange; +} + +/** + * How one mutation changed a sitemap, split into the two things a tag can cache. + * + * One boolean is not enough, and the reason is `<lastmod>`. A sitemap entry carries + * `lastModified`, derived from `updatedAt` - so a plain title edit on a published + * record changes the **bytes** of that locale's sitemap file even though the set of + * URLs in it is identical. Treating "the sitemap changed" as "membership changed" + * leaves a cached file serving a stale `<lastmod>` for as long as the tag lives. + * + * The two are separate because they cache different documents: + * + * - **`contentChanged`** - the sitemap *file* for this locale is no longer + * byte-identical. True for any real mutation of a record that is or was publicly + * reachable, whether what moved was a URL, a title or an SEO field. + * - **`indexChanged`** - the set of sitemap files, or how many of them there are, + * moved. True only when public reachability flipped, because an index lists files + * and their count follows the number of URLs. A title edit changes neither. + * + * Declared here rather than next to the write path because `cache.ts` is the + * client-safe layer and must not import from `server/` - the same reason the tag + * builders are plain strings a directory up from Drizzle. + */ +export interface ContentSitemapChange { + contentChanged: boolean; + indexChanged: boolean; +} + export interface ContentInvalidationInput { contentTypeId: string; + /** Delivery tags, for a content type with `delivery: { enabled: true }`. */ + delivery?: ContentDeliveryInvalidation; id: number; /** Whether the row is publicly reachable *after* the mutation. */ isPublic: boolean; @@ -154,6 +247,7 @@ const slugTags = ( */ export const contentInvalidationTags = ({ contentTypeId, + delivery, id, isPublic, locales, @@ -161,13 +255,21 @@ export const contentInvalidationTags = ({ wasPublic, }: ContentInvalidationInput): string[] => { if (locales !== undefined) { - return locales - .filter(entry => entry.wasPublic || entry.isPublic) - .flatMap(entry => [ + const reached = locales.filter(entry => entry.wasPublic || entry.isPublic); + + return [ + ...reached.flatMap(entry => [ contentPublicListTag(contentTypeId, entry.locale), contentPublicItemTag(contentTypeId, id, entry.locale), ...slugTags(contentTypeId, entry.slugs, entry.locale), - ]); + ]), + ...deliveryTags({ + contentTypeId, + delivery, + id, + locales: reached, + }), + ]; } if (!wasPublic && !isPublic) return []; @@ -176,6 +278,75 @@ export const contentInvalidationTags = ({ contentPublicListTag(contentTypeId), contentPublicItemTag(contentTypeId, id), ...slugTags(contentTypeId, slugs), + ...deliveryTags({ + contentTypeId, + delivery, + id, + locales: [{ isPublic, locale: undefined, slugs, wasPublic }], + }), + ]; +}; + +/** + * The delivery tags one mutation touched, per locale it reached. + * + * Three scopes, and each answers a different question a page asked: + * + * - **delivery metadata**, keyed by identifier, because a `generateMetadata` reads + * the canonical path and the alternates of one record; + * - **redirect lookups**, keyed by every slug the record answered to across the + * mutation, because a resolver caches "this old URL points there" and a second + * slug change moves the destination; + * - **the sitemap**, per locale and as a whole, but only when the set of listed + * URLs actually changed. + * + * Empty when the content type has no delivery layer, which is the whole of Stage + * 8's opt-in promise at this layer. + */ +const deliveryTags = ({ + contentTypeId, + delivery, + id, + locales, +}: { + contentTypeId: string; + delivery: ContentDeliveryInvalidation | undefined; + id: number; + locales: readonly { + isPublic: boolean; + locale: string | undefined; + slugs: readonly string[]; + wasPublic: boolean; + }[]; +}): string[] => { + if (delivery === undefined) return []; + + const tags = locales.flatMap(entry => [ + contentDeliveryTag(contentTypeId, id, entry.locale), + ...[...new Set(entry.slugs)] + .filter(slug => slug !== "") + .map(slug => + contentDeliveryRedirectTag(contentTypeId, slug, entry.locale), + ), + // The sitemap *file* this locale is listed in. For a content type that is not + // localized `entry.locale` is `undefined`, so this is the locale-less tag - which + // is that content type's only sitemap file rather than an index of files. + ...(delivery.sitemap.contentChanged + ? [contentDeliverySitemapTag(contentTypeId, entry.locale)] + : []), + ]); + + // The locale-less tag on its own means the *index* of a localized content type's + // per-locale files, so it is expired only when the set of files or their count + // moved - never for a title edit, which rewrites bytes inside one existing file. + // De-duplicated, because a content type that is not localized produces only this + // form and the line above already emitted it. + return [ + ...new Set( + delivery.sitemap.indexChanged + ? [...tags, contentDeliverySitemapTag(contentTypeId)] + : tags, + ), ]; }; diff --git a/packages/vitnode/src/content/conflicts.ts b/packages/vitnode/src/content/conflicts.ts index 3e69966b5..aee8f39c4 100644 --- a/packages/vitnode/src/content/conflicts.ts +++ b/packages/vitnode/src/content/conflicts.ts @@ -2,11 +2,15 @@ import { z } from "zod"; import { CONTENT_CONFLICT_CODES, + CONTENT_DELIVERY_CODES, CONTENT_SCHEDULE_CODES, CONTENT_TRANSLATION_CONFLICT_CODES, CONTENT_UNPROCESSABLE_CODES, } from "./const"; +export type ContentDeliveryCode = + (typeof CONTENT_DELIVERY_CODES)[keyof typeof CONTENT_DELIVERY_CODES]; + export type ContentConflictCode = (typeof CONTENT_CONFLICT_CODES)[keyof typeof CONTENT_CONFLICT_CODES]; @@ -103,6 +107,46 @@ export const parseContentTranslationConflict = ( } }; +/** + * The 409 body a write refused by the slug reservation answers with. + * + * Its own schema rather than a third member of {@link zodContentConflict}: that + * union is the contract Stage 4 editorial routes already publish, and widening it + * would change a response schema every generated client is built from. A route + * that can hit the reservation declares this one **alongside** it, so a client + * that only knows the older union still parses the arms it knows. + * + * `locale` is `null` for a content type whose slug is shared, and the locale code + * when the slug is localized - which is exactly the scope the reservation covers. + * There is deliberately no owning-record id: a 409 on a public-facing address must + * not become a way to enumerate records the caller cannot read. + */ +export const zodContentDeliveryConflict = z.object({ + code: z.literal(CONTENT_DELIVERY_CODES.slugReserved), + contentTypeId: z.string(), + locale: z.string().nullable(), + slug: z.string(), +}); + +export type ContentDeliveryConflict = z.infer< + typeof zodContentDeliveryConflict +>; + +/** Reads a delivery conflict out of a response body, or `null`. */ +export const parseContentDeliveryConflict = ( + body: string | undefined, +): ContentDeliveryConflict | null => { + if (body === undefined || body === "") return null; + + try { + const parsed = zodContentDeliveryConflict.safeParse(JSON.parse(body)); + + return parsed.success ? parsed.data : null; + } catch { + return null; + } +}; + /** The 422 body a restore answers with when the snapshot no longer fits. */ export const zodContentUnprocessable = z.object({ code: z.literal(CONTENT_UNPROCESSABLE_CODES.notRestorable), diff --git a/packages/vitnode/src/content/const.ts b/packages/vitnode/src/content/const.ts index 594de9ed5..1db79fcab 100644 --- a/packages/vitnode/src/content/const.ts +++ b/packages/vitnode/src/content/const.ts @@ -450,6 +450,112 @@ export const CONTENT_SCHEDULE_CODES = { unsupported: "CONTENT_SCHEDULE_UNSUPPORTED", } as const; +// --------------------------------------------------------------------------- +// Content delivery (Stage 8) +// --------------------------------------------------------------------------- + +/** + * Field kinds `delivery.seo.titleField` may name. + * + * `text` only, and the same reasoning `CONTENT_SEARCH_TITLE_KINDS` gives: a + * `<title>` is one line, a `textarea` in that slot puts a paragraph in a browser + * tab, and a slug is already in the URL the title accompanies. + */ +export const CONTENT_DELIVERY_TITLE_KINDS = ["text"] as const; + +/** Field kinds `delivery.seo.descriptionField` may name. */ +export const CONTENT_DELIVERY_DESCRIPTION_KINDS = ["text", "textarea"] as const; + +/** + * Field kinds `delivery.seo.noIndexField` may name. + * + * `boolean` only: "should a crawler index this" has two answers, and a truthy + * string would make the sitemap's exclusion rule depend on what somebody typed. + */ +export const CONTENT_DELIVERY_NO_INDEX_KINDS = ["boolean"] as const; + +/** + * The `changefreq` values the sitemap protocol defines. + * + * Validated rather than passed through: a crawler ignores an unknown value + * silently, so a typo would be a hint nobody ever receives. + */ +export const CONTENT_SITEMAP_CHANGE_FREQUENCIES = [ + "always", + "hourly", + "daily", + "weekly", + "monthly", + "yearly", + "never", +] as const; + +const sitemapChangeFrequencies: ReadonlySet<string> = new Set( + CONTENT_SITEMAP_CHANGE_FREQUENCIES, +); + +export const isContentSitemapChangeFrequency = ( + value: unknown, +): value is (typeof CONTENT_SITEMAP_CHANGE_FREQUENCIES)[number] => + typeof value === "string" && sitemapChangeFrequencies.has(value); + +/** + * The sitemap protocol's own ceiling: 50,000 URLs in one file. + * + * A delivery sitemap page never returns more than this, and the index helper + * chunks by it - so a content type with a million records produces a sitemap + * index rather than an invalid document. + */ +export const CONTENT_SITEMAP_MAX_URLS = 50_000; + +/** + * How many URLs one `sitemap.list` page returns by default. + * + * Far below the protocol ceiling on purpose: a page is one keyset query plus one + * batched translation read, and 1,000 rows is a response a serverless function + * can hold without thinking about it. A caller that wants a whole 50,000-URL + * file asks for it explicitly. + */ +export const CONTENT_SITEMAP_DEFAULT_PAGE_SIZE = 1_000; + +/** + * The redirect a moved canonical URL answers with. + * + * `308` rather than `301`, and the difference is not cosmetic: `301` lets a + * client rewrite the method to `GET`, `308` does not. A content URL is read with + * `GET` today, so the two behave identically now - and only one of them still + * behaves correctly the day somebody `POST`s to a form under a moved path. + * + * One status, not a configuration knob: every historical URL of every content + * type answers with this, so there is no per-content-type setting to get wrong + * and no reason for two of them to disagree. + */ +export const CONTENT_DELIVERY_REDIRECT_STATUS = 308; + +/** + * How a delivery resolution came out. + * + * `not_found` rather than a `gone` tombstone: the engine has no abstraction that + * distinguishes "deleted on purpose" from "unpublished for now", and a `410` + * that guessed would tell a crawler to forget a URL that is coming back. + */ +export const CONTENT_DELIVERY_RESOLUTIONS = [ + "content", + "not_found", + "redirect", +] as const; + +/** `core_content_slug_history.path` is `varchar(512)`. */ +export const CONTENT_DELIVERY_PATH_MAX_LENGTH = 512; + +/** Machine-readable reasons a delivery write or read was refused. */ +export const CONTENT_DELIVERY_CODES = { + invalidUrl: "CONTENT_DELIVERY_INVALID_URL", + notEnabled: "CONTENT_DELIVERY_NOT_ENABLED", + redirectConflict: "CONTENT_DELIVERY_REDIRECT_CONFLICT", + slugReserved: "CONTENT_DELIVERY_SLUG_RESERVED", +} as const; + /** * Every content type gets the first four staff permissions. `can_publish` is * generated only for content types with `publication: { enabled: true }`, diff --git a/packages/vitnode/src/content/define.ts b/packages/vitnode/src/content/define.ts index db0bb63c3..9be48f575 100644 --- a/packages/vitnode/src/content/define.ts +++ b/packages/vitnode/src/content/define.ts @@ -1,6 +1,11 @@ import type { AnyContentTypeDefinition, ContentAdminConfig, + ContentDeliveryConfig, + ContentDeliveryDescriptionField, + ContentDeliveryEnabled, + ContentDeliveryNoIndexField, + ContentDeliveryTitleField, ContentEditorialConfig, ContentEditorialEnabled, ContentFieldDescriptor, @@ -21,6 +26,7 @@ import type { ContentSearchTitleField, ContentTypeDefinition, ResolvedContentAdminConfig, + ResolvedContentDeliveryConfig, ResolvedContentEditorialConfig, ResolvedContentLocalizationConfig, ResolvedContentPublicApiConfig, @@ -64,6 +70,7 @@ import { CONTENT_TABLE_NAME_PATTERN, isFilterableFieldKind, } from "./const"; +import { resolveContentDelivery } from "./delivery"; import { ContentEngineError } from "./errors"; import { resolveContentIndexes } from "./indexes"; import { @@ -1410,8 +1417,23 @@ export const defineContentType = < TLocalization extends ContentLocalizationConfig | { enabled: false } = { enabled: false; }, + // The whole `delivery` argument, inferred as one type, for the same two reasons + // `TSearch` and `TEditorial` are. Its *constraint* is what enforces the field + // rules - a constraint is checked once `TPublicField` and `TPublicEnabled` are + // resolved, which is what makes "delivery needs a public API" and "an SEO field + // has to be public" compile errors rather than boot-time ones. + TDelivery extends + | ContentDeliveryConfig< + TPublicEnabled, + ContentEditorialEnabled<TEditorial>, + ContentDeliveryTitleField<TFields, TPublicField>, + ContentDeliveryDescriptionField<TFields, TPublicField>, + ContentDeliveryNoIndexField<TFields, TPublicField> + > + | { enabled: false } = { enabled: false }, >({ admin, + delivery, editorial, fields, id, @@ -1427,6 +1449,13 @@ export const defineContentType = < TPublication, ContentEditorialEnabled<TEditorial> >; + /** + * Opts into the delivery layer: canonical URLs, slug history, automatic + * redirects, localized alternates, `hreflang`, SEO projection and sitemap + * entries. Needs `publicApi`, and every SEO field it names has to be in + * `publicApi.fields`. Omit it and nothing about the content type changes. + */ + delivery?: TDelivery; /** * Opts into the editorial workflow: a `version` column, optimistic locking * and revision history, plus optional preview and scheduling. Omit it and @@ -1471,7 +1500,8 @@ export const defineContentType = < ContentEditorialEnabled<TEditorial>, ContentPreviewEnabled<TEditorial>, ContentSchedulingEnabled<TEditorial>, - ContentLocalizationEnabled<TLocalization> + ContentLocalizationEnabled<TLocalization>, + ContentDeliveryEnabled<TDelivery> > => { if (!CONTENT_ID_PATTERN.test(id)) { throw new ContentEngineError( @@ -1643,6 +1673,27 @@ export const defineContentType = < tableName, }); + // After localization, because "which language owns a historical URL" is read + // off the field partition, and after `publicApi`, because every canonical path + // and every SEO field is stated in terms of the resolved public allowlist. + const resolvedDelivery = resolveContentDelivery({ + // The `{ enabled: false }` arm exists only so an explicit literal typechecks - + // the same widening `publicApi`, `search`, `editorial` and `localization` do. + delivery: delivery as ContentDeliveryConfig | undefined, + // Read off the *resolved* editorial config rather than the argument, so the + // redirect check sees exactly what `resolveEditorial` decided. + editorial: resolvedEditorial.enabled, + fields: fieldMap, + id, + localization: { + defaultLocale: resolvedLocalization.defaultLocale, + enabled: resolvedLocalization.enabled, + }, + localizedFields, + publicApi: resolvedPublicApi, + publication: publicationEnabled, + }); + const definition: ContentTypeDefinition< TId, TFields, @@ -1653,10 +1704,14 @@ export const defineContentType = < ContentEditorialEnabled<TEditorial>, ContentPreviewEnabled<TEditorial>, ContentSchedulingEnabled<TEditorial>, - ContentLocalizationEnabled<TLocalization> + ContentLocalizationEnabled<TLocalization>, + ContentDeliveryEnabled<TDelivery> > = { admin: resolvedAdmin, advanced: resolvedAdvanced, + delivery: resolvedDelivery as ResolvedContentDeliveryConfig< + ContentDeliveryEnabled<TDelivery> + >, editorial: resolvedEditorial as ResolvedContentEditorialConfig< ContentEditorialEnabled<TEditorial>, ContentPreviewEnabled<TEditorial>, @@ -1688,7 +1743,8 @@ export const defineContentType = < ContentEditorialEnabled<TEditorial>, ContentPreviewEnabled<TEditorial>, ContentSchedulingEnabled<TEditorial>, - ContentLocalizationEnabled<TLocalization> + ContentLocalizationEnabled<TLocalization>, + ContentDeliveryEnabled<TDelivery> > >({ admin: resolvedAdmin, diff --git a/packages/vitnode/src/content/delivery.test-d.ts b/packages/vitnode/src/content/delivery.test-d.ts new file mode 100644 index 000000000..29f038812 --- /dev/null +++ b/packages/vitnode/src/content/delivery.test-d.ts @@ -0,0 +1,343 @@ +import { assertType, describe, expectTypeOf, it } from "vitest"; + +import { + testArticleContentType, + testPostContentType, +} from "@/tests/content-fixtures"; + +import type { ContentEventsFor } from "./events"; +import type { + AnyContentTypeDefinition, + ContentSitemapChangeFrequency, + DeliverableContentTypeDefinition, + ResolvedContentDeliveryConfig, +} from "./types"; + +import { defineContentType } from "./define"; +import { field } from "./fields"; + +/** + * Stage 8 at the type level. + * + * The rules worth a compile error rather than a boot-time one are the ones an author + * gets wrong while typing: naming a private field as an SEO title, putting prose in a + * title slot, or reaching for a delivery service a content type does not have. Every + * `@ts-expect-error` below is a mistake the editor catches before the file is saved. + */ + +const fields = { + excerpt: field.textarea({ maxLength: 500, nullable: true }), + /** Declared but never exposed - the private half of every check below. */ + internalCode: field.text({ nullable: true }), + seo: field.group({ + fields: { + description: field.textarea({ nullable: true }), + title: field.text({ nullable: true }), + }, + nullable: true, + }), + slug: field.slug({ source: "title" }), + title: field.text({ maxLength: 200, required: true }), + views: field.number({ integer: true, defaultValue: 0 }), +}; + +const shared = { + admin: { label: { plural: "Articles", singular: "Article" } }, + fields, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug", "excerpt", "seo.title", "seo.description"], + path: "articles", + }, +} as const; + +const deliveredType = defineContentType({ + ...shared, + id: "typed.delivered", + editorial: { enabled: true }, + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { + descriptionField: "seo.description", + fallbackDescriptionField: "excerpt", + fallbackTitleField: "title", + openGraph: { descriptionField: "excerpt", titleField: "title" }, + titleField: "seo.title", + }, + sitemap: { changeFrequency: "weekly", enabled: true, priority: 0.7 }, + }, + tableName: "typed_delivered", +}); + +const plainType = defineContentType({ + ...shared, + id: "typed.plain", + tableName: "typed_plain", +}); + +describe("delivery configuration", () => { + it("keeps the `enabled` literal, so every conditional resolves", () => { + expectTypeOf(deliveredType.delivery.enabled).toEqualTypeOf<true>(); + expectTypeOf(plainType.delivery.enabled).toEqualTypeOf<false>(); + }); + + // The whole Stage 8 type design rests on this: an eleventh type parameter on + // `ContentTypeDefinition` must not break the erased form every relation thunk, + // registry and route builder is written against. + it("stays assignable to AnyContentTypeDefinition", () => { + expectTypeOf<typeof deliveredType>().toExtend<AnyContentTypeDefinition>(); + assertType<AnyContentTypeDefinition>(deliveredType); + assertType<AnyContentTypeDefinition>(plainType); + }); + + it("narrows to DeliverableContentTypeDefinition only with delivery", () => { + expectTypeOf< + typeof deliveredType + >().toExtend<DeliverableContentTypeDefinition>(); + expectTypeOf< + typeof plainType + >().not.toExtend<DeliverableContentTypeDefinition>(); + }); + + it("accepts a group leaf and a plain field in the SEO slots", () => { + expectTypeOf(deliveredType.delivery.seo.titleField).toEqualTypeOf< + null | string + >(); + expectTypeOf( + deliveredType.delivery.sitemap.changeFrequency, + ).toEqualTypeOf<ContentSitemapChangeFrequency | null>(); + }); + + it("records the slug scope", () => { + expectTypeOf(deliveredType.delivery.slugScope).toEqualTypeOf< + "localized" | "none" | "shared" + >(); + }); +}); + +describe("delivery requires a public API", () => { + it("refuses `enabled: true` without one", () => { + defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "typed.no-public", + // @ts-expect-error - delivery needs `publicApi: { enabled: true }`: without a + // public allowlist there is no canonical URL for delivery to be about. + delivery: { enabled: true }, + fields, + publication: { enabled: true }, + tableName: "typed_no_public", + }); + }); + + it("still accepts an explicit `enabled: false`", () => { + const off = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "typed.off", + delivery: { enabled: false }, + fields, + publication: { enabled: true }, + tableName: "typed_off", + }); + + expectTypeOf(off.delivery.enabled).toEqualTypeOf<false>(); + }); +}); + +describe("redirects require editorial", () => { + it("refuses `redirects: { enabled: true }` without editorial", () => { + defineContentType({ + ...shared, + id: "typed.no-editorial", + delivery: { + enabled: true, + // @ts-expect-error - slug history has to be written in the same transaction + // as the slug mutation and its revision, and only the editorial mutation + // paths own one. Without `editorial` this would record nothing. + redirects: { enabled: true }, + }, + tableName: "typed_no_editorial", + }); + }); + + it("still accepts an explicit `redirects: { enabled: false }`", () => { + const off = defineContentType({ + ...shared, + id: "typed.redirects-off", + delivery: { enabled: true, redirects: { enabled: false } }, + tableName: "typed_redirects_off", + }); + + expectTypeOf(off.delivery.enabled).toEqualTypeOf<true>(); + }); + + it("accepts redirects once editorial is enabled", () => { + const on = defineContentType({ + ...shared, + id: "typed.redirects-on", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + tableName: "typed_redirects_on", + }); + + expectTypeOf(on.delivery.enabled).toEqualTypeOf<true>(); + }); + + it("leaves every other delivery block available without editorial", () => { + // The rule is narrow on purpose: only slug history needs a transaction. + const reads = defineContentType({ + ...shared, + id: "typed.reads-only", + delivery: { + enabled: true, + seo: { descriptionField: "excerpt", titleField: "title" }, + sitemap: { changeFrequency: "daily", enabled: true, priority: 0.5 }, + }, + tableName: "typed_reads_only", + }); + + expectTypeOf(reads.delivery.enabled).toEqualTypeOf<true>(); + expectTypeOf(reads.editorial.enabled).toEqualTypeOf<false>(); + }); +}); + +describe("SEO field references", () => { + it("refuses a field the public allowlist withholds", () => { + defineContentType({ + ...shared, + id: "typed.private-seo", + delivery: { + enabled: true, + // @ts-expect-error - `internalCode` is a text field, but it is not in + // `publicApi.fields`, and a `<title>` is rendered into a public page. + seo: { titleField: "internalCode" }, + }, + tableName: "typed_private_seo", + }); + }); + + it("refuses prose in a title slot", () => { + defineContentType({ + ...shared, + id: "typed.prose-title", + delivery: { + enabled: true, + // @ts-expect-error - `excerpt` is a textarea. A `<title>` is one line, and a + // paragraph in a browser tab is not a heading. + seo: { titleField: "excerpt" }, + }, + tableName: "typed_prose_title", + }); + }); + + it("refuses a number in a description slot", () => { + defineContentType({ + ...shared, + id: "typed-bad.description", + delivery: { + enabled: true, + // @ts-expect-error - `views` is a number, and it is private besides. + seo: { descriptionField: "views" }, + }, + tableName: "typed_bad_description", + }); + }); + + it("refuses a nested path the content type does not declare", () => { + defineContentType({ + ...shared, + id: "typed.bad-path", + delivery: { + enabled: true, + // @ts-expect-error - `seo.heading` is not a leaf of the `seo` group. + seo: { titleField: "seo.heading" }, + }, + tableName: "typed_bad_path", + }); + }); + + it("accepts a valid nested group path", () => { + const nested = defineContentType({ + ...shared, + id: "typed.nested", + delivery: { + enabled: true, + seo: { descriptionField: "seo.description", titleField: "seo.title" }, + }, + tableName: "typed_nested", + }); + + expectTypeOf(nested.delivery.enabled).toEqualTypeOf<true>(); + }); + + it("refuses an unknown change frequency", () => { + defineContentType({ + ...shared, + id: "typed.bad-freq", + delivery: { + enabled: true, + // @ts-expect-error - not one of the seven values the protocol defines. + sitemap: { changeFrequency: "fortnightly", enabled: true }, + }, + tableName: "typed_bad_freq", + }); + }); +}); + +describe("the resolved config is generic over `enabled`", () => { + it("pins `true` for a delivered content type", () => { + expectTypeOf(deliveredType.delivery).toExtend< + ResolvedContentDeliveryConfig<true> + >(); + }); + + it("pins `false` for one without", () => { + expectTypeOf(plainType.delivery).toExtend< + ResolvedContentDeliveryConfig<false> + >(); + }); +}); + +describe("Stage 1-7 backward compatibility", () => { + it("leaves the existing fixtures assignable and unchanged", () => { + assertType<AnyContentTypeDefinition>(testArticleContentType); + assertType<AnyContentTypeDefinition>(testPostContentType); + expectTypeOf( + testArticleContentType.delivery.enabled, + ).toEqualTypeOf<false>(); + expectTypeOf(testPostContentType.delivery.enabled).toEqualTypeOf<false>(); + }); +}); + +describe("delivery events", () => { + it("adds both keys for a content type with redirects", () => { + expectTypeOf<ContentEventsFor<typeof deliveredType>>().toHaveProperty( + "content.typed.delivered.delivery_slug_changed", + ); + expectTypeOf<ContentEventsFor<typeof deliveredType>>().toHaveProperty( + "content.typed.delivered.delivery_redirect_created", + ); + }); + + it("adds neither for a content type without delivery", () => { + // The keys are gated on `delivery: { enabled: true }`, so a listener for one + // cannot even be registered - which is what keeps every Stage 1-7 event map + // byte-identical. + expectTypeOf<ContentEventsFor<typeof plainType>>().not.toHaveProperty( + "content.typed.plain.delivery_slug_changed", + ); + expectTypeOf<ContentEventsFor<typeof plainType>>().not.toHaveProperty( + "content.typed.plain.delivery_redirect_created", + ); + }); + + it("keeps the ordinary events in place alongside them", () => { + expectTypeOf<ContentEventsFor<typeof deliveredType>>().toHaveProperty( + "content.typed.delivered.updated", + ); + expectTypeOf<ContentEventsFor<typeof deliveredType>>().toHaveProperty( + "content.typed.delivered.published", + ); + }); +}); diff --git a/packages/vitnode/src/content/delivery.test.ts b/packages/vitnode/src/content/delivery.test.ts new file mode 100644 index 000000000..6b69aaf1f --- /dev/null +++ b/packages/vitnode/src/content/delivery.test.ts @@ -0,0 +1,824 @@ +import { describe, expect, it } from "vitest"; + +import { defineContentType } from "./define"; +import { + contentDeliveryHreflang, + contentDeliveryOpenGraph, + contentDeliveryPath, + contentDeliveryRobots, + contentDeliverySeo, + contentDeliveryUrl, + contentSitemapDefaults, + isDeliverableContentType, + listDeliveryContentTypes, + parseContentDeliveryPath, +} from "./delivery"; +import { field } from "./fields"; + +/** + * Stage 8 definition validation and the pure delivery projections. + * + * Everything here runs without a database, because everything here is a rule about + * a *definition* or a pure function over a public row - and the rules are the half + * of Stage 8 that has to fail loudly at boot rather than quietly at request time. + */ + +const base = { + admin: { label: { plural: "Articles", singular: "Article" } }, + publication: { enabled: true } as const, + tableName: "delivery_articles", +} as const; + +const publicApi = { + enabled: true, + fields: ["id", "title", "slug", "excerpt", "publishedAt"], + path: "articles", +} as const; + +const fields = { + excerpt: field.textarea({ maxLength: 500, nullable: true }), + hidden: field.boolean({ defaultValue: false }), + /** A text field the public allowlist deliberately withholds. */ + internalCode: field.text({ nullable: true }), + slug: field.slug({ source: "title" }), + title: field.text({ maxLength: 200, required: true }), + views: field.number({ integer: true, defaultValue: 0 }), +}; + +const articleType = defineContentType({ + ...base, + id: "delivery.article", + // `redirects` needs `editorial`: slug history has to be written in the same + // transaction as the slug mutation and its revision. + editorial: { enabled: true }, + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { descriptionField: "excerpt", titleField: "title" }, + sitemap: { changeFrequency: "weekly", enabled: true, priority: 0.7 }, + }, + fields, + publicApi, +}); + +const plainType = defineContentType({ + ...base, + id: "delivery.plain", + fields, + publicApi, + tableName: "delivery_plain", +}); + +describe("delivery definition validation", () => { + it("defaults to disabled, so a Stage 1-7 content type is unchanged", () => { + expect(plainType.delivery).toStrictEqual({ + enabled: false, + hreflang: { xDefault: null }, + redirects: { enabled: false }, + seo: { + descriptionField: null, + fallbackDescriptionField: null, + fallbackTitleField: null, + noIndexField: null, + openGraph: null, + titleField: null, + }, + sitemap: { changeFrequency: null, enabled: false, priority: null }, + slugScope: "none", + }); + }); + + it("refuses delivery without a public API", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.private", + // Refused by the types too - see `delivery.test-d.ts`. Cast here because + // this asserts the *runtime* guard, which a JavaScript caller still reaches. + delivery: { enabled: true as never }, + fields, + tableName: "delivery_private", + }), + ).toThrow(/delivery needs `publicApi/); + }); + + it("refuses an SEO field that is not publicly exposed", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.private-seo", + // A text field, so the kind check passes - and absent from + // `publicApi.fields`, so a `<title>` built from it would publish something + // the public API deliberately withholds. + delivery: { + enabled: true, + seo: { titleField: "internalCode" as never }, + }, + fields, + publicApi, + tableName: "delivery_private_seo", + }), + ).toThrow(/not in publicApi.fields/); + }); + + it("refuses an unsupported SEO field kind", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.bad-kind", + // `excerpt` is a textarea, which is a description and never a title. + delivery: { enabled: true, seo: { titleField: "excerpt" as never } }, + fields, + publicApi, + tableName: "delivery_bad_kind", + }), + ).toThrow(/of kind "textarea"/); + }); + + it("refuses an unknown SEO field", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.unknown-seo", + delivery: { enabled: true, seo: { titleField: "nope" as never } }, + fields, + publicApi, + tableName: "delivery_unknown_seo", + }), + ).toThrow(/references unknown field "nope"/); + }); + + it("refuses a repeatable leaf as a title", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.repeatable-seo", + delivery: { + enabled: true, + seo: { titleField: "faq.question" as never }, + }, + fields: { + ...fields, + faq: field.repeatable({ + fields: { question: field.text({ required: true }) }, + }), + }, + publicApi: { + ...publicApi, + fields: [...publicApi.fields, "faq.question"], + }, + tableName: "delivery_repeatable_seo", + }), + ).toThrow(/many values rather than one/); + }); + + it("accepts a group leaf as a title and a description", () => { + const withGroup = defineContentType({ + ...base, + id: "delivery.group-seo", + delivery: { + enabled: true, + seo: { + descriptionField: "seo.description", + fallbackTitleField: "title", + titleField: "seo.title", + }, + }, + fields: { + ...fields, + seo: field.group({ + fields: { + description: field.textarea({ nullable: true }), + title: field.text({ nullable: true }), + }, + nullable: true, + }), + }, + publicApi: { + ...publicApi, + fields: [...publicApi.fields, "seo.title", "seo.description"], + }, + tableName: "delivery_group_seo", + }); + + expect(withGroup.delivery.seo).toMatchObject({ + descriptionField: "seo.description", + fallbackTitleField: "title", + titleField: "seo.title", + }); + }); + + it("refuses a fallback with no primary, which would never be read", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.orphan-fallback", + delivery: { + enabled: true, + seo: { fallbackTitleField: "title" as never }, + }, + fields, + publicApi, + tableName: "delivery_orphan_fallback", + }), + ).toThrow(/without `titleField`/); + }); + + it("refuses a sitemap priority outside 0-1", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.bad-priority", + delivery: { enabled: true, sitemap: { enabled: true, priority: 7 } }, + fields, + publicApi, + tableName: "delivery_bad_priority", + }), + ).toThrow(/between 0 and 1/); + }); + + it("refuses an unknown change frequency", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.bad-freq", + delivery: { + enabled: true, + sitemap: { + // A crawler ignores an unknown value silently, so a typo has to be + // caught here or it is a hint nobody ever receives. + changeFrequency: "fortnightly" as never, + enabled: true, + }, + }, + fields, + publicApi, + tableName: "delivery_bad_freq", + }), + ).toThrow(/sitemap protocol defines/); + }); + + it("refuses a non-boolean noIndexField", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.bad-noindex", + delivery: { enabled: true, seo: { noIndexField: "title" as never } }, + fields, + publicApi, + tableName: "delivery_bad_noindex", + }), + ).toThrow(/Expected one of: boolean/); + }); + + it("refuses x-default without localization", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.bad-xdefault", + delivery: { enabled: true, hreflang: { xDefault: "defaultLocale" } }, + fields, + publicApi, + tableName: "delivery_bad_xdefault", + }), + ).toThrow(/delivery.hreflang needs `localization/); + }); + + it("records the slug scope so history knows which language owns a URL", () => { + expect(articleType.delivery.slugScope).toBe("shared"); + expect(localizedType.delivery.slugScope).toBe("localized"); + }); + + it("refuses redirects on a localized content type with a shared slug", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.shared-slug", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + fields: { + body: field.textarea({ localized: true, required: true }), + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + localization: { defaultLocale: "en", enabled: true }, + publicApi: { + enabled: true, + fields: ["title", "slug", "body"], + path: "articles", + }, + tableName: "delivery_shared_slug", + }), + ).toThrow(/needs a localized slug field/); + }); + + it("refuses redirects without editorial", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.no-editorial", + // No `editorial`, so the only mutation path is the plain repository - which + // has no version to guard and no history to write. Accepting this would be + // accepting a redirect feature that silently records nothing. + delivery: { + enabled: true, + redirects: { enabled: true as never }, + }, + fields, + publicApi, + tableName: "delivery_no_editorial", + }), + ).toThrow(/delivery.redirects needs `editorial/); + }); + + it("refuses localized redirects without editorial", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.localized-no-editorial", + delivery: { + enabled: true, + redirects: { enabled: true as never }, + }, + fields: { + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "delivery_localized_no_editorial", + }), + ).toThrow(/delivery.redirects needs `editorial/); + }); + + it("accepts redirects with editorial", () => { + const withEditorial = defineContentType({ + ...base, + id: "delivery.with-editorial", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + fields, + publicApi, + tableName: "delivery_with_editorial", + }); + + expect(withEditorial.delivery.redirects.enabled).toBe(true); + }); + + it("accepts delivery without redirects and without editorial", () => { + // Everything except slug history is a read over data the content type already + // has, so none of it needs a transactional mutation path. + const withoutEditorial = defineContentType({ + ...base, + id: "delivery.reads-only", + delivery: { + enabled: true, + seo: { descriptionField: "excerpt", titleField: "title" }, + sitemap: { changeFrequency: "weekly", enabled: true, priority: 0.7 }, + }, + fields, + publicApi, + tableName: "delivery_reads_only", + }); + + expect(withoutEditorial.delivery).toMatchObject({ + enabled: true, + redirects: { enabled: false }, + seo: { descriptionField: "excerpt", titleField: "title" }, + sitemap: { changeFrequency: "weekly", enabled: true, priority: 0.7 }, + }); + }); + + it("accepts localized delivery reads without editorial", () => { + // Stage 5 supports publication and localization without editorial, and Stage 8 + // must not take that away - only `redirects` needs the extra dependency. + const localizedReads = defineContentType({ + ...base, + id: "delivery.localized-reads", + delivery: { + enabled: true, + hreflang: { xDefault: "defaultLocale" }, + seo: { fallbackTitleField: "title", titleField: "seo.title" }, + sitemap: { enabled: true }, + }, + fields: { + seo: field.group({ + fields: { title: field.text({ nullable: true }) }, + localized: true, + nullable: true, + }), + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true, fallback: "default" }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug", "seo.title"], + path: "articles", + }, + tableName: "delivery_localized_reads", + }); + + expect(localizedReads.delivery).toMatchObject({ + enabled: true, + hreflang: { xDefault: "defaultLocale" }, + redirects: { enabled: false }, + sitemap: { enabled: true }, + slugScope: "localized", + }); + }); + + it("leaves a content type without delivery untouched by the rule", () => { + // No `editorial`, no `delivery` - the Stage 1-7 shape, still accepted. + expect(plainType.delivery.enabled).toBe(false); + expect(plainType.editorial.enabled).toBe(false); + }); + + it("refuses a localized content type that withholds id", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.no-id", + delivery: { enabled: true }, + fields: { + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true }, + publicApi: { + enabled: true, + // No `id`, so alternates and `hreflang` could not be resolved - and an + // empty `hreflang` looks exactly like a record with one translation. + fields: ["title", "slug"], + path: "articles", + }, + tableName: "delivery_no_id", + }), + ).toThrow(/needs "id" in publicApi.fields/); + }); + + it("does not require id of a nonlocalized content type", () => { + // It has no alternates to resolve, so there is nothing the identifier is needed + // for - `itemId` simply comes back `null` on its delivery metadata. + const withoutId = defineContentType({ + ...base, + id: "delivery.no-id-flat", + delivery: { enabled: true }, + fields, + publicApi: { enabled: true, fields: ["title", "slug"], path: "articles" }, + tableName: "delivery_no_id_flat", + }); + + expect(withoutId.delivery.enabled).toBe(true); + expect(withoutId.publicApi.fields).not.toContain("id"); + }); + + it("refuses a localized noIndexField", () => { + expect(() => + defineContentType({ + ...base, + id: "delivery.localized-noindex", + delivery: { + enabled: true, + seo: { noIndexField: "flags.noIndex" as never }, + }, + fields: { + // A localized group's leaves live on the translation table, so the value + // would differ per language while the record has one canonical decision. + flags: field.group({ + fields: { noIndex: field.boolean({ defaultValue: false }) }, + localized: true, + }), + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true }, + publicApi: { + enabled: true, + // `id` because a localized delivery content type has to expose it - see + // "refuses a localized content type that withholds id" below. + fields: ["id", "title", "slug", "flags.noIndex"], + path: "articles", + }, + tableName: "delivery_localized_noindex", + }), + ).toThrow(/has to be shared/); + }); +}); + +const localizedType = defineContentType({ + ...base, + id: "delivery.localized", + editorial: { enabled: true }, + delivery: { + enabled: true, + hreflang: { xDefault: "defaultLocale" }, + redirects: { enabled: true }, + seo: { + descriptionField: "seo.description", + fallbackTitleField: "title", + titleField: "seo.title", + }, + sitemap: { changeFrequency: "daily", enabled: true, priority: 0.5 }, + }, + fields: { + seo: field.group({ + fields: { + description: field.textarea({ nullable: true }), + title: field.text({ nullable: true }), + }, + localized: true, + nullable: true, + }), + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true, fallback: "default" }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug", "seo.title", "seo.description"], + path: "articles", + }, + tableName: "delivery_localized", +}); + +describe("contentDeliveryPath", () => { + it("has no locale segment for a nonlocalized content type", () => { + expect( + contentDeliveryPath({ definition: articleType, slug: "my-article" }), + ).toBe("/articles/my-article"); + }); + + it("prefixes the locale for a localized content type", () => { + expect( + contentDeliveryPath({ + definition: localizedType, + locale: "pl", + slug: "moj-artykul", + }), + ).toBe("/pl/articles/moj-artykul"); + }); + + it("normalizes the locale, so one URL has one cache key", () => { + const paths = ["PL", "pl", " pl "].map(locale => + contentDeliveryPath({ definition: localizedType, locale, slug: "witaj" }), + ); + + expect(new Set(paths).size).toBe(1); + expect(paths[0]).toBe("/pl/articles/witaj"); + }); + + it("refuses to build a localized path with no locale", () => { + expect( + contentDeliveryPath({ definition: localizedType, slug: "witaj" }), + ).toBeNull(); + }); + + it("is null for an empty slug rather than pointing at the list page", () => { + expect( + contentDeliveryPath({ definition: articleType, slug: " " }), + ).toBeNull(); + }); + + it("percent-encodes a slug that was written straight into the database", () => { + expect( + contentDeliveryPath({ definition: articleType, slug: "a b/c" }), + ).toBe("/articles/a%20b%2Fc"); + }); +}); + +describe("contentDeliveryUrl", () => { + it("resolves a path against an origin, with or without a trailing slash", () => { + for (const origin of ["https://example.com", "https://example.com/"]) { + expect(contentDeliveryUrl({ origin, path: "/articles/x" })).toBe( + "https://example.com/articles/x", + ); + } + }); + + it("is null for a malformed origin rather than a URL with two schemes", () => { + expect(contentDeliveryUrl({ origin: "not a url", path: "/x" })).toBeNull(); + }); + + it("passes a null path straight through", () => { + expect( + contentDeliveryUrl({ origin: "https://example.com", path: null }), + ).toBeNull(); + }); +}); + +describe("parseContentDeliveryPath", () => { + it("round-trips the path it builds", () => { + expect( + parseContentDeliveryPath(articleType, "/articles/my-article"), + ).toStrictEqual({ locale: null, slug: "my-article" }); + + expect( + parseContentDeliveryPath(localizedType, "/pl/articles/moj-artykul"), + ).toStrictEqual({ locale: "pl", slug: "moj-artykul" }); + }); + + it("decodes the slug and normalizes the locale", () => { + expect( + parseContentDeliveryPath(localizedType, "/PL/articles/a%20b"), + ).toStrictEqual({ locale: "pl", slug: "a b" }); + }); + + it("strips a query string and a fragment", () => { + expect( + parseContentDeliveryPath(articleType, "/articles/x?utm=1#top"), + ).toStrictEqual({ locale: null, slug: "x" }); + }); + + it("refuses a path that belongs to another content type", () => { + expect(parseContentDeliveryPath(articleType, "/news/x")).toBeNull(); + }); + + it("refuses the wrong number of segments", () => { + for (const path of ["/articles", "/articles/a/b", "/pl/articles/a"]) { + expect(parseContentDeliveryPath(articleType, path)).toBeNull(); + } + }); + + it("refuses a traversal and a malformed escape", () => { + expect(parseContentDeliveryPath(articleType, "/articles/..")).toBeNull(); + expect(parseContentDeliveryPath(articleType, "/articles/%zz")).toBeNull(); + }); + + it("refuses a path longer than the stored column", () => { + expect( + parseContentDeliveryPath(articleType, `/articles/${"a".repeat(600)}`), + ).toBeNull(); + }); +}); + +describe("SEO projection", () => { + it("reads the configured fields off a public row", () => { + expect( + contentDeliverySeo(articleType, { + excerpt: "A summary.", + title: "My article", + }), + ).toStrictEqual({ description: "A summary.", title: "My article" }); + }); + + it("falls back only when the primary is empty", () => { + expect( + contentDeliverySeo(localizedType, { + seo: { description: null, title: " " }, + title: "The heading", + }), + ).toStrictEqual({ description: null, title: "The heading" }); + + expect( + contentDeliverySeo(localizedType, { + seo: { description: "d", title: "SEO heading" }, + title: "The heading", + }), + ).toStrictEqual({ description: "d", title: "SEO heading" }); + }); + + it("never invents a description from other content", () => { + expect( + contentDeliverySeo(articleType, { excerpt: null, title: "T" }), + ).toStrictEqual({ description: null, title: "T" }); + }); + + it("cannot read a field the public row does not carry", () => { + // The row is the public projection, so a private field is absent from the + // object entirely rather than merely skipped. + expect(contentDeliverySeo(articleType, { views: 9 })).toStrictEqual({ + description: null, + title: null, + }); + }); + + it("is a stable shape for a content type that configured nothing", () => { + expect(contentDeliverySeo(plainType, { title: "T" })).toStrictEqual({ + description: null, + title: null, + }); + }); +}); + +describe("Open Graph projection", () => { + it("is null when the content type configured none", () => { + expect(contentDeliveryOpenGraph(articleType, { title: "T" })).toBeNull(); + }); + + it("falls back to the ordinary SEO slots", () => { + const withOg = defineContentType({ + ...base, + id: "delivery.og", + delivery: { + enabled: true, + seo: { openGraph: {}, titleField: "title" }, + }, + fields, + publicApi, + tableName: "delivery_og", + }); + + expect( + contentDeliveryOpenGraph(withOg, { title: "Shared heading" }), + ).toStrictEqual({ description: null, title: "Shared heading" }); + }); +}); + +describe("robots projection", () => { + it("is null without a noIndexField", () => { + expect(contentDeliveryRobots(articleType, {})).toBeNull(); + }); + + it("reads the boolean and always allows following", () => { + const withNoIndex = defineContentType({ + ...base, + id: "delivery.noindex", + delivery: { enabled: true, seo: { noIndexField: "hidden" } }, + fields, + publicApi: { ...publicApi, fields: [...publicApi.fields, "hidden"] }, + tableName: "delivery_noindex", + }); + + expect(contentDeliveryRobots(withNoIndex, { hidden: true })).toStrictEqual({ + follow: true, + index: false, + }); + expect(contentDeliveryRobots(withNoIndex, { hidden: false })).toStrictEqual( + { + follow: true, + index: true, + }, + ); + }); +}); + +describe("contentDeliveryHreflang", () => { + const alternates = [ + { locale: "en", path: "/en/articles/my-article" }, + { locale: "pl", path: "/pl/articles/moj-artykul" }, + ]; + + it("maps alternates to a language map", () => { + expect( + contentDeliveryHreflang({ alternates, definition: localizedType }), + ).toStrictEqual({ + languages: { + en: "/en/articles/my-article", + pl: "/pl/articles/moj-artykul", + }, + xDefault: "/en/articles/my-article", + }); + }); + + it("omits x-default when the default locale is not published", () => { + expect( + contentDeliveryHreflang({ + alternates: [alternates[1]], + definition: localizedType, + }), + ).toStrictEqual({ languages: { pl: "/pl/articles/moj-artykul" } }); + }); + + it("emits no x-default when the content type did not ask for one", () => { + expect( + contentDeliveryHreflang({ alternates, definition: articleType }), + ).toStrictEqual({ + languages: { + en: "/en/articles/my-article", + pl: "/pl/articles/moj-artykul", + }, + }); + }); +}); + +describe("registry helpers", () => { + it("lists only delivery-enabled content types, in a stable order", () => { + const entries = [ + { definition: localizedType, pluginId: "b" }, + { definition: plainType, pluginId: "a" }, + { definition: articleType, pluginId: "a" }, + ]; + + expect( + listDeliveryContentTypes(entries).map(entry => entry.definition.id), + ).toStrictEqual(["delivery.article", "delivery.localized"]); + }); + + it("narrows a definition to a deliverable one", () => { + expect(isDeliverableContentType(articleType)).toBe(true); + expect(isDeliverableContentType(plainType)).toBe(false); + }); + + it("reports the sitemap defaults, or nothing", () => { + expect(contentSitemapDefaults(articleType)).toStrictEqual({ + changeFrequency: "weekly", + priority: 0.7, + }); + expect(contentSitemapDefaults(plainType)).toBeNull(); + }); +}); diff --git a/packages/vitnode/src/content/delivery.ts b/packages/vitnode/src/content/delivery.ts new file mode 100644 index 000000000..629eb56fd --- /dev/null +++ b/packages/vitnode/src/content/delivery.ts @@ -0,0 +1,783 @@ +import type { + AnyContentTypeDefinition, + ContentDeliveryConfig, + ContentFieldDescriptor, + ContentFieldMap, + ContentSitemapChangeFrequency, + DeliverableContentTypeDefinition, + ResolvedContentDeliveryConfig, + ResolvedContentPublicApiConfig, +} from "./types"; + +import { + CONTENT_DELIVERY_DESCRIPTION_KINDS, + CONTENT_DELIVERY_NO_INDEX_KINDS, + CONTENT_DELIVERY_PATH_MAX_LENGTH, + CONTENT_DELIVERY_TITLE_KINDS, + isContentSitemapChangeFrequency, +} from "./const"; +import { ContentEngineError } from "./errors"; +import { normalizeContentLocale } from "./locale"; +import { readContentPath, splitContentFieldPath } from "./paths"; + +/** + * The Content Delivery layer: what a public URL *is*, rather than what a record + * contains. + * + * Everything in this module is pure and client-safe. It answers four questions + * and nothing else - what is the canonical path of this record in this language, + * which languages does it also exist in, what should the page put in `<head>`, + * and is a given path the current one - so a frontend can render a page, an + * `hreflang` set and a sitemap entry from data the engine already has. + * + * It deliberately does **not** render anything. There is no layout here, no + * React, no Next.js and no `Metadata`: those belong to the application, and the + * `content/next` adapter is the thin translation layer between the two. + */ + +/** Kinds the three SEO slots accept, as runtime sets. */ +const titleKinds: ReadonlySet<string> = new Set(CONTENT_DELIVERY_TITLE_KINDS); +const descriptionKinds: ReadonlySet<string> = new Set( + CONTENT_DELIVERY_DESCRIPTION_KINDS, +); +const noIndexKinds: ReadonlySet<string> = new Set( + CONTENT_DELIVERY_NO_INDEX_KINDS, +); + +/** The disabled default every content type without `delivery` carries. */ +export const contentDeliveryDisabled: ResolvedContentDeliveryConfig<false> = { + enabled: false, + hreflang: { xDefault: null }, + redirects: { enabled: false }, + seo: { + descriptionField: null, + fallbackDescriptionField: null, + fallbackTitleField: null, + noIndexField: null, + openGraph: null, + titleField: null, + }, + sitemap: { changeFrequency: null, enabled: false, priority: null }, + slugScope: "none", +}; + +/** + * Resolves one SEO field name to the descriptor it addresses, or `null`. + * + * A leaf path resolves through its **group**, and a repeatable is deliberately + * not resolvable here: `assertSeoField` needs to tell "this leaf is a column on + * the row" from "this leaf is a column on a child row", and only the first can + * be one page's title. + */ +const resolveSeoTarget = ( + fields: ContentFieldMap, + name: string, +): null | { + container: "group" | "repeatable" | "row"; + descriptor: ContentFieldDescriptor; +} => { + const path = splitContentFieldPath(name); + if (!path) { + const fieldValue = fields[name]; + + return fieldValue ? { container: "row", descriptor: fieldValue } : null; + } + + const [owner, leaf] = path; + const container = fields[owner]; + if (container?.kind !== "group" && container?.kind !== "repeatable") { + return null; + } + + const leafValue = (container as { fields: ContentFieldMap }).fields[leaf]; + + return leafValue + ? { container: container.kind, descriptor: leafValue } + : null; +}; + +/** + * Checks one configured SEO field name. + * + * The public-exposure rule is the important one, and it is what makes "SEO + * metadata cannot leak a private value" a property of the definition rather than + * of every consumer: a `<title>` is rendered into a public page, so it has to be + * something the public API would already have said out loud. + */ +const assertSeoField = ({ + exposed, + fields, + id, + kinds, + label, + localizedFields, + name, + shared = false, +}: { + exposed: ReadonlySet<string>; + fields: ContentFieldMap; + id: string; + kinds: ReadonlySet<string>; + label: string; + localizedFields: ContentFieldMap; + name: string; + /** Whether the slot refuses a localized field. Only `noIndexField` does. */ + shared?: boolean; +}): void => { + const target = resolveSeoTarget(fields, name); + if (!target) { + throw new ContentEngineError( + `${label} references unknown field "${name}".`, + { contentTypeId: id }, + ); + } + + if (target.container === "repeatable") { + throw new ContentEngineError( + `${label} names the repeatable leaf "${name}", which is many values rather than one. A page has one title, one description and one indexing decision.`, + { contentTypeId: id }, + ); + } + + if (!kinds.has(target.descriptor.kind)) { + throw new ContentEngineError( + `${label} names "${name}" of kind "${target.descriptor.kind}". Expected one of: ${[...kinds].sort().join(", ")}.`, + { contentTypeId: id }, + ); + } + + if (!exposed.has(name)) { + throw new ContentEngineError( + `${label} names "${name}", which is not in publicApi.fields. Delivery metadata is rendered into a public page, so every field it is built from has to be publicly readable already.`, + { contentTypeId: id }, + ); + } + + if (shared) { + const path = splitContentFieldPath(name); + const owner = path ? path[0] : name; + if (localizedFields[owner] !== undefined) { + throw new ContentEngineError( + `${label} names the localized field "${name}". This slot has to be shared: sitemap inclusion and the \`robots\` metadata must agree, and a per-locale value would give one record one answer per language while it has a single canonical decision.`, + { contentTypeId: id }, + ); + } + } +}; + +/** + * Checks and fills in `delivery`. + * + * Runs after `resolvePublicApi` and after the field partition, because every rule + * here is stated in terms of both: the public allowlist decides which fields may + * be projected, and the partition decides which language a historical URL belongs + * to. + * + * Nothing is silently ignored. An invalid delivery block fails at definition + * time - a canonical URL that quietly stopped being generated is a page that + * quietly stopped being indexable, and that is not a symptom anybody notices. + */ +export const resolveContentDelivery = ({ + delivery, + editorial, + fields, + id, + localization, + localizedFields, + publicApi, + publication, +}: { + delivery: ContentDeliveryConfig | undefined; + /** Whether the content type opted into the editorial workflow. */ + editorial: boolean; + fields: ContentFieldMap; + id: string; + localization: { defaultLocale: string; enabled: boolean }; + localizedFields: ContentFieldMap; + publicApi: ResolvedContentPublicApiConfig; + publication: boolean; +}): ResolvedContentDeliveryConfig => { + if (!delivery?.enabled) return contentDeliveryDisabled; + + if (!publicApi.enabled) { + throw new ContentEngineError( + "delivery needs `publicApi: { enabled: true, path, fields }`. A content type with no public API has no public URL, so there is no canonical path, no redirect and no sitemap entry for delivery to produce.", + { contentTypeId: id }, + ); + } + + const slugField = publicApi.slugField; + if (slugField === "") { + throw new ContentEngineError( + "delivery needs an exposed slug field. `publicApi` already requires exactly one, so this content type is misconfigured upstream.", + { contentTypeId: id }, + ); + } + + const redirects = delivery.redirects?.enabled === true; + const sitemapConfig = + delivery.sitemap?.enabled === true ? delivery.sitemap : null; + const slugScope = + localizedFields[slugField] === undefined ? "shared" : "localized"; + + // Slug history has to be written in the same transaction as the slug mutation, the + // version check and the revision - and the only mutation paths that own such a + // transaction are `editorial-service` and `translation-editorial-service`. Without + // `editorial` a content type writes through the plain repository, which has neither + // a version to guard nor a history to write, so accepting this would be accepting a + // feature that records nothing. + // + // Refused rather than downgraded to `redirects: { enabled: false }`: an author who + // asked for redirects and silently got none would find out from a broken link + // months later. The type system refuses it too - see `ContentDeliveryConfig`. + if (redirects && !editorial) { + throw new ContentEngineError( + "delivery.redirects needs `editorial: { enabled: true }`. Redirect history has to be written atomically with the slug mutation and its version and revision, and only the editorial mutation paths own that transaction. Delivery without `redirects` - canonical URLs, SEO, alternates and the sitemap - works without editorial.", + { contentTypeId: id }, + ); + } + + // A localized content type whose slug is *shared* has one URL segment and several + // URLs - `/en/articles/hello` and `/pl/articles/hello` are both live, and a slug + // change moves all of them at once. Slug history stores the URL that was live, so + // one retired row would have to be several paths, and the AdminCP would show one + // of them as if it were the address somebody bookmarked. Canonical URLs, SEO, + // alternates and the sitemap all work fine in that shape - only the redirect + // reservation is ambiguous, so only it is refused. + if (redirects && localization.enabled && slugScope === "shared") { + throw new ContentEngineError( + `delivery.redirects needs a localized slug field on a localized content type, but "${slugField}" is shared. Every language answers to the same segment, so one retired address would belong to several URLs at once. Mark the slug \`localized: true\`, or drop \`redirects\`.`, + { contentTypeId: id }, + ); + } + + // Restated even though `publicApi` already requires publication: a sitemap + // lists what anonymous readers can reach, and "what can be reached" is exactly + // the publication lifecycle. Without it every row would be in the sitemap from + // the moment it was created. + if (sitemapConfig && !publication) { + throw new ContentEngineError( + "delivery.sitemap needs `publication: { enabled: true }`. A sitemap lists what is publicly reachable, and without the lifecycle every row would be listed the moment it was created.", + { contentTypeId: id }, + ); + } + + if (sitemapConfig?.priority !== undefined) { + const { priority } = sitemapConfig; + if (!Number.isFinite(priority) || priority < 0 || priority > 1) { + throw new ContentEngineError( + `delivery.sitemap.priority is ${priority}; the sitemap protocol defines it between 0 and 1 inclusive.`, + { contentTypeId: id }, + ); + } + } + + if ( + sitemapConfig?.changeFrequency !== undefined && + !isContentSitemapChangeFrequency(sitemapConfig.changeFrequency) + ) { + throw new ContentEngineError( + `delivery.sitemap.changeFrequency is "${String(sitemapConfig.changeFrequency)}", which is not one of the values the sitemap protocol defines. A crawler ignores an unknown one silently, so a typo would be a hint nobody ever receives.`, + { contentTypeId: id }, + ); + } + + if (delivery.hreflang !== undefined) { + if (delivery.hreflang.xDefault !== "defaultLocale") { + throw new ContentEngineError( + `delivery.hreflang.xDefault is "${String(delivery.hreflang.xDefault)}"; the only supported value is "defaultLocale". An x-default has to point at a URL that actually resolves.`, + { contentTypeId: id }, + ); + } + + if (!localization.enabled) { + throw new ContentEngineError( + "delivery.hreflang needs `localization: { enabled: true, defaultLocale }`. A content type with one language has no alternates, so there is nothing for an x-default to be the default of.", + { contentTypeId: id }, + ); + } + } + + const exposed = new Set(publicApi.fields); + + // Alternates and `hreflang` are resolved by identifier - the query enumerates a + // record's published translations - and delivery reads the **public projection**, + // so a localized content type that withholds `id` would silently produce an empty + // `hreflang` set from `resolveSlug`. Refused loudly here rather than left as a + // quiet gap: an empty `hreflang` looks exactly like a record with one translation. + // + // Not required of a nonlocalized content type, which has no alternates to resolve. + if (localization.enabled && !exposed.has("id")) { + throw new ContentEngineError( + 'delivery on a localized content type needs "id" in publicApi.fields. Alternates and `hreflang` are resolved by identifier, and delivery reads the public projection - so without it every localized response would carry an empty alternate set.', + { contentTypeId: id }, + ); + } + + const seo = delivery.seo ?? {}; + + for (const [label, name] of [ + ["delivery.seo.titleField", seo.titleField], + ["delivery.seo.fallbackTitleField", seo.fallbackTitleField], + ["delivery.seo.openGraph.titleField", seo.openGraph?.titleField], + ] as const) { + if (name === undefined) continue; + + assertSeoField({ + exposed, + fields, + id, + kinds: titleKinds, + label, + localizedFields, + name, + }); + } + + for (const [label, name] of [ + ["delivery.seo.descriptionField", seo.descriptionField], + ["delivery.seo.fallbackDescriptionField", seo.fallbackDescriptionField], + [ + "delivery.seo.openGraph.descriptionField", + seo.openGraph?.descriptionField, + ], + ] as const) { + if (name === undefined) continue; + + assertSeoField({ + exposed, + fields, + id, + kinds: descriptionKinds, + label, + localizedFields, + name, + }); + } + + if (seo.noIndexField !== undefined) { + assertSeoField({ + exposed, + fields, + id, + kinds: noIndexKinds, + label: "delivery.seo.noIndexField", + localizedFields, + name: seo.noIndexField, + shared: true, + }); + } + + // A fallback with no primary is a configuration that reads as if it does + // something and does nothing: the primary is what is consulted first, so + // naming only the fallback means the fallback is never reached. + if (seo.fallbackTitleField !== undefined && seo.titleField === undefined) { + throw new ContentEngineError( + "delivery.seo.fallbackTitleField is set without `titleField`. The fallback is only consulted when the primary is empty, so on its own it would never be read - name it as `titleField` instead.", + { contentTypeId: id }, + ); + } + + if ( + seo.fallbackDescriptionField !== undefined && + seo.descriptionField === undefined + ) { + throw new ContentEngineError( + "delivery.seo.fallbackDescriptionField is set without `descriptionField`. The fallback is only consulted when the primary is empty, so on its own it would never be read.", + { contentTypeId: id }, + ); + } + + return { + enabled: true, + hreflang: { xDefault: delivery.hreflang?.xDefault ?? null }, + redirects: { enabled: redirects }, + seo: { + descriptionField: seo.descriptionField ?? null, + fallbackDescriptionField: seo.fallbackDescriptionField ?? null, + fallbackTitleField: seo.fallbackTitleField ?? null, + noIndexField: seo.noIndexField ?? null, + openGraph: + seo.openGraph === undefined + ? null + : { + descriptionField: seo.openGraph.descriptionField ?? null, + titleField: seo.openGraph.titleField ?? null, + }, + titleField: seo.titleField ?? null, + }, + sitemap: { + changeFrequency: sitemapConfig?.changeFrequency ?? null, + enabled: sitemapConfig !== null, + priority: sitemapConfig?.priority ?? null, + }, + slugScope, + }; +}; + +// --------------------------------------------------------------------------- +// Canonical URLs +// --------------------------------------------------------------------------- + +/** + * The canonical **path** of one record, in one language. + * + * ```text + * /articles/my-article nonlocalized + * /pl/articles/moj-artykul localized + * ``` + * + * Relative, always, and that is the point: a content type definition lives in + * source control and gets deployed to a preview domain, a staging domain and + * production, so an origin baked into it would be wrong in two of the three + * places. {@link contentDeliveryUrl} adds one when a caller has one to add. + * + * The locale segment is **normalized** through `normalizeContentLocale`, so + * `PL`, `pl` and `" pl "` produce one path and therefore one cache key. The slug + * is percent-encoded: a generated slug is already URL-safe, but a row written + * straight into the database is not, and a path is what this function promises. + * + * `null` for an empty slug or an empty public path, rather than a link to + * `/articles/` - a canonical URL that points at a list page is worse than no + * canonical URL at all. + */ +export const contentDeliveryPath = ({ + definition, + locale, + slug, +}: { + definition: AnyContentTypeDefinition; + /** Required for a localized content type, ignored otherwise. */ + locale?: null | string; + slug: string; +}): null | string => { + const path = definition.publicApi.path; + const trimmed = slug.trim(); + if (path === "" || trimmed === "") return null; + + const segments: string[] = []; + + if (definition.localization.enabled) { + const normalized = normalizeContentLocale(locale ?? ""); + // A localized record has one URL per language and no locale-less one. Without + // a locale there is no path to build, and guessing would hand a reader the + // wrong language under a URL that claims otherwise. + if (normalized === "") return null; + + segments.push(encodeURIComponent(normalized)); + } + + segments.push(path, encodeURIComponent(trimmed)); + + return `/${segments.join("/")}`; +}; + +/** + * A canonical path turned absolute, when the caller has an origin. + * + * `origin` is whatever the request or the deployment says it is - a configured + * public URL, `NEXT_PUBLIC_WEB_URL`, a forwarded host. It is separate from the + * path for the reason {@link contentDeliveryPath} explains, and it is validated + * here rather than concatenated: `https://example.com` and + * `https://example.com/` have to produce the same URL, and a malformed origin + * has to be a `null` rather than a link with two schemes in it. + */ +export const contentDeliveryUrl = ({ + origin, + path, +}: { + origin: string; + path: null | string; +}): null | string => { + if (path === null) return null; + + try { + return new URL(path, origin).toString(); + } catch { + return null; + } +}; + +/** One published translation's URL, as `alternates` and `hreflang` report it. */ +export interface ContentDeliveryAlternate { + /** The canonical `core_languages.code`. */ + locale: string; + path: string; +} + +/** + * The `hreflang` set of one record, as a framework-neutral map. + * + * `{ languages, xDefault? }` rather than a Next.js `Metadata` object, because the + * core engine has no business knowing which framework renders it - `content/next` + * turns this into `alternates.languages` in one line, and an Astro or Remix + * adapter would do the same. + * + * Built from {@link ContentDeliveryAlternate}s, which are **real published + * translations** and nothing else. A fallback translation is not an alternate: it + * has no URL in the language that fell back to it, so listing one would announce + * a page that answers 404. + */ +export interface ContentDeliveryHreflang { + languages: Record<string, string>; + /** Present only with `delivery.hreflang.xDefault` and a resolvable default. */ + xDefault?: string; +} + +export const contentDeliveryHreflang = ({ + alternates, + definition, +}: { + alternates: readonly ContentDeliveryAlternate[]; + definition: AnyContentTypeDefinition; +}): ContentDeliveryHreflang => { + const languages: Record<string, string> = {}; + for (const alternate of alternates) + languages[alternate.locale] = alternate.path; + + if (definition.delivery.hreflang.xDefault !== "defaultLocale") { + return { languages }; + } + + // Only when the default locale is genuinely one of the alternates. An + // `x-default` pointing at a language this record has not published would be a + // hint to crawl a 404, which is worse than emitting nothing. + const fallback = alternates.find(alternate => + contentDeliveryLocalesMatch( + alternate.locale, + definition.localization.defaultLocale, + ), + ); + + return fallback === undefined + ? { languages } + : { languages, xDefault: fallback.path }; +}; + +const contentDeliveryLocalesMatch = (a: string, b: string): boolean => + normalizeContentLocale(a) === normalizeContentLocale(b); + +// --------------------------------------------------------------------------- +// SEO projection +// --------------------------------------------------------------------------- + +export interface ContentDeliverySeo { + description: null | string; + title: null | string; +} + +export interface ContentDeliveryRobots { + follow: boolean; + index: boolean; +} + +/** + * Reads one configured SEO slot out of a **public** row. + * + * The row is the public projection - the same object the public API returns - so + * a field the allowlist omits is not merely skipped here, it is absent from the + * object entirely. That is what makes "SEO cannot leak a private field" true at + * runtime as well as at definition time. + * + * A whitespace-only value counts as empty, because a `<title>` of three spaces is + * a missing title with extra steps - and that is exactly when the fallback should + * take over. + */ +const readSeoText = ( + row: Record<string, unknown>, + primary: null | string, + fallback: null | string, +): null | string => { + for (const name of [primary, fallback]) { + if (name === null) continue; + + const value = readContentPath(row, name); + if (typeof value !== "string") continue; + + const trimmed = value.trim(); + if (trimmed !== "") return trimmed; + } + + return null; +}; + +/** + * The `<title>` and `<meta name="description">` of one record. + * + * `{ description: null, title: null }` for a content type whose `delivery.seo` + * names nothing - the shape is stable so a frontend never branches on whether + * the block was configured, only on whether a value came back. + */ +export const contentDeliverySeo = ( + definition: AnyContentTypeDefinition, + row: Record<string, unknown>, +): ContentDeliverySeo => { + const { seo } = definition.delivery; + + return { + description: readSeoText( + row, + seo.descriptionField, + seo.fallbackDescriptionField, + ), + title: readSeoText(row, seo.titleField, seo.fallbackTitleField), + }; +}; + +/** + * The Open Graph pair, or `null` when the content type configured none. + * + * `null` rather than an object of nulls, because "this content type does not + * publish Open Graph metadata" and "it does, and this page has no title" are + * different facts and a renderer treats them differently: the first emits no + * tags at all. + * + * Each slot falls back to the ordinary SEO one, which is what makes the common + * case - the same title in both places - a two-line config rather than four. + */ +export const contentDeliveryOpenGraph = ( + definition: AnyContentTypeDefinition, + row: Record<string, unknown>, +): ContentDeliverySeo | null => { + const { seo } = definition.delivery; + if (seo.openGraph === null) return null; + + const base = contentDeliverySeo(definition, row); + + return { + description: + readSeoText(row, seo.openGraph.descriptionField, null) ?? + base.description, + title: readSeoText(row, seo.openGraph.titleField, null) ?? base.title, + }; +}; + +/** + * The `robots` directive of one record, or `null` without a `noIndexField`. + * + * `follow` is always `true`: "do not list this page" and "do not follow the links + * on it" are different instructions, and a content type that asked for the first + * has not asked for the second. A `noindex, nofollow` page is a dead end for a + * crawler walking the site, which is a decision for site-wide robots + * configuration rather than for one record. + * + * The same field drives the sitemap exclusion, which is what keeps the two from + * disagreeing: a record cannot be absent from the sitemap and `index: true` at + * the same time, because there is one boolean behind both. + */ +export const contentDeliveryRobots = ( + definition: AnyContentTypeDefinition, + row: Record<string, unknown>, +): ContentDeliveryRobots | null => { + const { noIndexField } = definition.delivery.seo; + if (noIndexField === null) return null; + + return { follow: true, index: readContentPath(row, noIndexField) !== true }; +}; + +// --------------------------------------------------------------------------- +// Path parsing +// --------------------------------------------------------------------------- + +/** A public path split into the two things a delivery lookup needs. */ +export interface ContentDeliveryPathParts { + /** `null` for a content type that is not localized. */ + locale: null | string; + slug: string; +} + +/** + * Splits a public path back into its locale and its slug. + * + * The inverse of {@link contentDeliveryPath}, and deliberately strict: it accepts + * exactly the shape that function produces and refuses everything else. A path + * with an extra segment, a different public prefix or a traversal in it is `null` + * rather than a best guess - a resolver that guessed would answer one content + * type's URL with another's record. + * + * A query string and a fragment are stripped first, because a browser sends them + * and they are not part of the identity of a page. + */ +export const parseContentDeliveryPath = ( + definition: AnyContentTypeDefinition, + path: string, +): ContentDeliveryPathParts | null => { + if (path.length > CONTENT_DELIVERY_PATH_MAX_LENGTH) return null; + + const withoutQuery = path.split(/[?#]/)[0] ?? ""; + const segments = withoutQuery + .split("/") + .filter(segment => segment !== "") + .map(segment => { + try { + return decodeURIComponent(segment); + } catch { + // A malformed escape is not a path this engine produced. + return null; + } + }); + + if (segments.some(segment => segment === null)) return null; + + const parts = segments as string[]; + const localized = definition.localization.enabled; + const expected = localized ? 3 : 2; + if (parts.length !== expected) return null; + + const [prefix, slug] = localized + ? [parts[1], parts[2]] + : [parts[0], parts[1]]; + if (prefix !== definition.publicApi.path) return null; + if (slug === "" || slug === "." || slug === "..") return null; + + return { + locale: localized ? normalizeContentLocale(parts[0]) : null, + slug, + }; +}; + +// --------------------------------------------------------------------------- +// Registry +// --------------------------------------------------------------------------- + +/** + * Every delivery-enabled content type of an installation, in a stable order. + * + * What a site-level sitemap index is built from: it enumerates the content types + * that have public URLs at all, so an application never hardcodes plugin names - + * installing a plugin adds its content types to the sitemap and removing it takes + * them out again. + * + * Ordered by content type id, so two processes building the same sitemap index + * produce the same document. + */ +export const listDeliveryContentTypes = < + TEntry extends { definition: AnyContentTypeDefinition; pluginId: string }, +>( + entries: readonly TEntry[], +): TEntry[] => + [...entries] + .filter(entry => entry.definition.delivery.enabled) + .sort((a, b) => a.definition.id.localeCompare(b.definition.id)); + +/** Whether one definition has a delivery layer, as a type guard. */ +export const isDeliverableContentType = ( + definition: AnyContentTypeDefinition, +): definition is DeliverableContentTypeDefinition => + definition.delivery.enabled && definition.publicApi.enabled; + +/** The sitemap defaults of one content type, or `null` when it lists nothing. */ +export const contentSitemapDefaults = ( + definition: AnyContentTypeDefinition, +): null | { + changeFrequency: ContentSitemapChangeFrequency | null; + priority: null | number; +} => { + const { sitemap } = definition.delivery; + if (!definition.delivery.enabled || !sitemap.enabled) return null; + + return { + changeFrequency: sitemap.changeFrequency, + priority: sitemap.priority, + }; +}; diff --git a/packages/vitnode/src/content/errors.ts b/packages/vitnode/src/content/errors.ts index cb036d1b5..25a888d1d 100644 --- a/packages/vitnode/src/content/errors.ts +++ b/packages/vitnode/src/content/errors.ts @@ -317,6 +317,66 @@ export class ContentAdvancedInputError extends ContentInputError { readonly ids: number[]; } +/** + * A slug that another record's public URL history already owns. + * + * A historical public URL stays reserved for the content type and locale that + * retired it, for as long as redirects are enabled, and this is what enforces + * that. Without the reservation `/articles/hello` could stop redirecting to the + * article it belonged to and start resolving to an unrelated one - so every link, + * bookmark and search result pointing at it would silently change meaning. + * + * Per-request, like {@link ContentInputError}, and structured for the same reason + * {@link ContentVersionConflict} is: the AdminCP points at the slug field and says + * which URL is taken, which it cannot do from prose. Everything it carries is the + * caller's own input echoed back plus the content type id, so there is nothing + * internal in it - in particular never the owning record's identifier, which would + * let a public write probe for records it cannot read. + */ +export class ContentDeliverySlugReserved extends ContentEngineError { + constructor({ + contentTypeId, + locale, + slug, + }: { + contentTypeId: string; + locale: null | string; + slug: string; + }) { + super( + locale === null + ? `The address "${slug}" is reserved: another record used it publicly and it still redirects there. Pick a different one.` + : `The address "${slug}" is reserved in "${locale}": another record used it publicly and it still redirects there. Pick a different one.`, + { contentTypeId }, + ); + + this.name = "ContentDeliverySlugReserved"; + this.locale = locale; + this.slug = slug; + } + + readonly locale: null | string; + readonly slug: string; +} + +/** + * A delivery operation on a content type that has no delivery layer. + * + * A configuration bug rather than a per-request one - the model exposes no + * `deliveryService` at all for such a content type, so reaching this means + * somebody built the service by hand. + */ +export class ContentDeliveryNotEnabled extends ContentEngineError { + constructor({ contentTypeId }: { contentTypeId: string }) { + super( + "This content type has no `delivery` block, so it has no canonical URL, no slug history and no sitemap. Add `delivery: { enabled: true }` to generate them.", + { contentTypeId }, + ); + + this.name = "ContentDeliveryNotEnabled"; + } +} + /** * A schedule that does not make sense: a time already past, or an unpublish * that would fire before the publish it is meant to follow. diff --git a/packages/vitnode/src/content/events.ts b/packages/vitnode/src/content/events.ts index 6ff8a271e..531851e66 100644 --- a/packages/vitnode/src/content/events.ts +++ b/packages/vitnode/src/content/events.ts @@ -3,6 +3,8 @@ import type { ContentFieldName, ContentLocalizedFieldName } from "./types"; export type ContentEventAction = | "created" | "deleted" + | "delivery_redirect_created" + | "delivery_slug_changed" | "published" | "restored" | "schedule_cancelled" @@ -270,6 +272,76 @@ type ContentLocalizationEventsFor<TDefinition extends { id: string }> = > : Record<never, never>); +/** + * A record's canonical public URL moved. + * + * Emitted **in addition to** the `updated` (or `restored`) event, not instead of + * it: the field mutation and the URL change are two different facts with two + * different audiences. A listener that mirrors content into another system wants + * the first; one that warms a CDN, tells an external search engine or writes to an + * edge redirect table wants the second, and would otherwise have to inspect + * `changedFields` for a slug field whose name it cannot know. + * + * `locale` is `null` when the slug is shared - a content type that is not + * localized, or a localized one whose slug lives on the base row. + */ +export interface ContentDeliverySlugChangedPayload { + /** The path the record answers to now. */ + canonicalPath: string; + contentId: number; + locale: null | string; + /** The path it answered to before, or `null` when it had no public URL yet. */ + previousPath: null | string; + previousSlug: null | string; + slug: string; +} + +/** + * A historical public URL became a redirect. + * + * Emitted only when the old slug had genuinely been *publicly addressable* - so a + * draft whose slug was corrected three times before it was ever published emits + * nothing, and a published article that moves emits exactly one. That is the + * difference between "a URL exists that needs a redirect" and "somebody edited a + * field", and it is why this is a separate event from the one above rather than a + * boolean on it. + */ +export interface ContentDeliveryRedirectCreatedPayload { + /** Where the historical path now redirects to. */ + canonicalPath: string; + contentId: number; + locale: null | string; + /** The retired path, which now answers with a permanent redirect. */ + previousPath: string; + previousSlug: string; +} + +/** + * The two events the delivery layer adds. + * + * Gated on `delivery: { enabled: true }` exactly like the publication, editorial + * and localization groups, so a content type without it gains no key at all and a + * listener for one cannot be registered - which is what keeps every Stage 1-7 + * event map byte-identical. + * + * Both keys are gated on `delivery` alone rather than the redirect one being gated a + * second time on `redirects`. Whether a content type keeps slug history is a + * *resolved* boolean rather than a literal on the definition's type, so a second gate + * would need another type parameter on `ContentTypeDefinition` to buy one thing: a + * listener nobody can register for an event that would never have fired anyway. + */ +type ContentDeliveryEventsFor<TDefinition extends { id: string }> = + TDefinition extends { delivery: { enabled: true } } + ? Record< + `content.${TDefinition["id"]}.delivery_redirect_created`, + ContentDeliveryRedirectCreatedPayload + > & + Record< + `content.${TDefinition["id"]}.delivery_slug_changed`, + ContentDeliverySlugChangedPayload + > + : Record<never, never>; + /** * The events a content type emits, as a literal-keyed map. * @@ -289,7 +361,8 @@ type ContentLocalizationEventsFor<TDefinition extends { id: string }> = * payloads stay minimal. */ export type ContentEventsFor<TDefinition extends { id: string }> = - ContentEditorialEventsFor<TDefinition> & + ContentDeliveryEventsFor<TDefinition> & + ContentEditorialEventsFor<TDefinition> & ContentLocalizationEventsFor<TDefinition> & ContentPublicationEventsFor<TDefinition> & Record<`content.${TDefinition["id"]}.created`, ContentCreatedPayload> & diff --git a/packages/vitnode/src/content/index.ts b/packages/vitnode/src/content/index.ts index 98b55d524..eebb070dc 100644 --- a/packages/vitnode/src/content/index.ts +++ b/packages/vitnode/src/content/index.ts @@ -28,6 +28,9 @@ export type { ContentFormSpec, } from "./admin/spec"; export { + contentDeliveryRedirectTag, + contentDeliverySitemapTag, + contentDeliveryTag, contentInvalidationTags, contentLocaleInvalidationMode, contentLocaleInvalidations, @@ -39,23 +42,29 @@ export { isContentTranslationPubliclyVisible, } from "./cache"; export type { + ContentDeliveryInvalidation, ContentInvalidationInput, ContentInvalidationMode, ContentLocaleInvalidation, ContentLocaleState, ContentPublicLocaleState, + ContentSitemapChange, } from "./cache"; export { parseContentConflict, + parseContentDeliveryConflict, parseContentTranslationConflict, parseContentUnprocessable, zodContentConflict, + zodContentDeliveryConflict, zodContentTranslationConflict, zodContentUnprocessable, } from "./conflicts"; export type { ContentConflict, ContentConflictCode, + ContentDeliveryCode, + ContentDeliveryConflict, ContentTranslationConflict, ContentTranslationConflictCode, ContentUnprocessable, @@ -66,6 +75,13 @@ export { CONTENT_CACHE_TAG_MAX_LENGTH, CONTENT_CONFLICT_CODES, CONTENT_DEFAULT_PAGE_SIZE, + CONTENT_DELIVERY_CODES, + CONTENT_DELIVERY_DESCRIPTION_KINDS, + CONTENT_DELIVERY_NO_INDEX_KINDS, + CONTENT_DELIVERY_PATH_MAX_LENGTH, + CONTENT_DELIVERY_REDIRECT_STATUS, + CONTENT_DELIVERY_RESOLUTIONS, + CONTENT_DELIVERY_TITLE_KINDS, CONTENT_EDITORIAL_FIELDS, CONTENT_ENUM_DEFAULT_LENGTH, CONTENT_FILTERABLE_FIELD_KINDS, @@ -103,6 +119,9 @@ export { CONTENT_SEARCH_SLUG_PLACEHOLDER, CONTENT_SEARCH_TEXT_KINDS, CONTENT_SEARCH_TITLE_KINDS, + CONTENT_SITEMAP_CHANGE_FREQUENCIES, + CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + CONTENT_SITEMAP_MAX_URLS, CONTENT_SLUG_DEFAULT_LENGTH, CONTENT_SYSTEM_FIELDS, CONTENT_TEXT_DEFAULT_LENGTH, @@ -113,15 +132,39 @@ export { CONTENT_TRANSLATION_TABLE_SUFFIX, CONTENT_UNPROCESSABLE_CODES, isContentPublicationStatus, + isContentSitemapChangeFrequency, isFilterableFieldKind, isLocalizableFieldKind, RESERVED_FILTER_KEYS, } from "./const"; export { defineContentType } from "./define"; +export { + contentDeliveryDisabled, + contentDeliveryHreflang, + contentDeliveryOpenGraph, + contentDeliveryPath, + contentDeliveryRobots, + contentDeliverySeo, + contentDeliveryUrl, + contentSitemapDefaults, + isDeliverableContentType, + listDeliveryContentTypes, + parseContentDeliveryPath, + resolveContentDelivery, +} from "./delivery"; +export type { + ContentDeliveryAlternate, + ContentDeliveryHreflang, + ContentDeliveryPathParts, + ContentDeliveryRobots, + ContentDeliverySeo, +} from "./delivery"; export type { ContentAdvancedCode } from "./errors"; export { ContentAdvancedInputError, ContentDefaultTranslationRequired, + ContentDeliveryNotEnabled, + ContentDeliverySlugReserved, ContentEngineError, ContentInputError, ContentLanguageError, @@ -135,6 +178,8 @@ export { contentEventName } from "./events"; export type { ContentCreatedPayload, ContentDeletedPayload, + ContentDeliveryRedirectCreatedPayload, + ContentDeliverySlugChangedPayload, ContentEventAction, ContentEventsFor, ContentPublishedPayload, @@ -214,6 +259,13 @@ export { contentSearchIndexedFieldNames, contentSearchUrl, } from "./search"; +export { + contentSitemapChunks, + contentSitemapIndexXml, + contentSitemapXml, + escapeXml, +} from "./sitemap"; +export type { ContentSitemapEntry, ContentSitemapIndexEntry } from "./sitemap"; export { slugify } from "./slug"; export type { AnyContentTypeDefinition, @@ -223,6 +275,16 @@ export type { ContentBooleanField, ContentCreateInput, ContentDateTimeField, + ContentDeliveryConfig, + ContentDeliveryDescriptionField, + ContentDeliveryEnabled, + ContentDeliveryHreflangConfig, + ContentDeliveryNoIndexField, + ContentDeliveryOpenGraphConfig, + ContentDeliveryRedirectsConfig, + ContentDeliverySeoConfig, + ContentDeliverySitemapConfig, + ContentDeliveryTitleField, ContentEditorialConfig, ContentEditorialEnabled, ContentEditorialField, @@ -271,6 +333,7 @@ export type { ContentSelect, ContentSharedFieldName, ContentSharedValues, + ContentSitemapChangeFrequency, ContentSlugField, ContentSlugRequired, ContentSystemField, @@ -283,6 +346,7 @@ export type { ContentTypeDefinition, ContentUpdateInput, ContentUserField, + DeliverableContentTypeDefinition, EditorialContentTypeDefinition, FilterableContentFieldKind, FilterableContentFieldName, @@ -291,6 +355,8 @@ export type { PublicContentTypeDefinition, PublicFilterableContentFieldName, ResolvedContentAdminConfig, + ResolvedContentDeliveryConfig, + ResolvedContentDeliverySeoConfig, ResolvedContentEditorialConfig, ResolvedContentIndex, ResolvedContentLocalizationConfig, diff --git a/packages/vitnode/src/content/next/cache-privacy.test.ts b/packages/vitnode/src/content/next/cache-privacy.test.ts new file mode 100644 index 000000000..d46ff8904 --- /dev/null +++ b/packages/vitnode/src/content/next/cache-privacy.test.ts @@ -0,0 +1,188 @@ +// @vitest-environment node +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import { + testEditorialPostContentType, + testLocalizedPageContentType, +} from "@/tests/content-fixtures"; + +/** + * Which responses are allowed into a shared cache, and under which key. + * + * Three questions, and getting any of them wrong is a data leak rather than a + * performance problem: + * + * 1. **Is anything private cached at all?** A preview is an unpublished record + * behind a short-lived credential, and an AdminCP read is a staff response + * carrying private columns. Neither may be stored. + * 2. **Can a private response land on a public key?** Only tagged, cached + * responses can, so this reduces to (1) - but it is asserted from the tag + * side as well, because "no tags" is what makes it true. + * 3. **Do two spellings of one locale share a key?** `PL`, `pl` and ` pl ` + * address the same page, so they have to expire together. A tag is a string + * comparison, so the normalisation has to happen when the tag is built. + */ + +interface FetchArgs { + module: string; + options?: { cache?: string; next?: { tags?: string[] } }; + path: string; + prefixPath?: string; + query?: Record<string, string | string[] | undefined>; +} + +const calls = vi.hoisted(() => [] as FetchArgs[]); + +vi.mock("server-only", () => ({})); +vi.mock("next/headers", () => ({ + cookies: async () => await Promise.resolve({ toString: () => "session=x" }), + headers: async () => await Promise.resolve(new Headers()), +})); + +vi.mock("../../lib/fetcher/raw", () => ({ + rawApiFetch: async (args: FetchArgs) => { + calls.push(args); + + return await Promise.resolve( + new Response(JSON.stringify({ id: 7, title: "Hello" }), { status: 200 }), + ); + }, +})); + +const { contentPublicItemTag, contentPublicListTag, contentPublicSlugTag } = + await import("../cache"); +const { contentPreviewFetch, contentPublicFetch } = + await import("./fetch.server"); +const { contentApiFetch } = await import("../admin/fetch.server"); + +const PLUGIN_ID = "@vitnode/example"; + +const lastCall = (): FetchArgs => { + const call = calls.at(-1); + if (!call) throw new Error("Expected a request."); + + return call; +}; + +beforeEach(() => { + calls.length = 0; +}); + +describe("a private response is never cached and never tagged", () => { + it("reads a preview with no store and no tags", async () => { + await contentPreviewFetch({ + definition: testEditorialPostContentType, + pluginId: PLUGIN_ID, + token: "signed-token", + }); + + const call = lastCall(); + expect(call.options?.cache).toBe("no-store"); + expect(call.options?.next?.tags).toBeUndefined(); + }); + + it("reads a localized preview the same way", async () => { + await contentPreviewFetch({ + definition: testLocalizedPageContentType, + locale: "pl", + pluginId: PLUGIN_ID, + token: "signed-token", + }); + + const call = lastCall(); + expect(call.options?.cache).toBe("no-store"); + expect(call.options?.next?.tags).toBeUndefined(); + }); + + it("reads the AdminCP API with no cache options at all", async () => { + // An admin response carries every private column the content type has. It + // is also per-session: it forwards the staff cookie, so a cached one would + // be one editor's view served to the next. + await contentApiFetch({ + definition: testEditorialPostContentType, + method: "get", + path: "/7", + pluginId: PLUGIN_ID, + }); + + const call = lastCall(); + expect(call.options).toBeUndefined(); + expect(call.prefixPath).toBe("/admin"); + }); + + it("keeps the admin module out of the public tag namespace", async () => { + // Belt and braces: even if an admin read were cached one day, it addresses + // `content/{permissionModule}` under `/admin`, and the public tags are + // built from `publicApi.path`. The two cannot be confused for each other. + await contentApiFetch({ + definition: testEditorialPostContentType, + method: "get", + pluginId: PLUGIN_ID, + }); + const adminCall = lastCall(); + + await contentPublicFetch({ + definition: testEditorialPostContentType, + pluginId: PLUGIN_ID, + }); + const publicCall = lastCall(); + + expect(adminCall.module).not.toBe(publicCall.module); + expect(publicCall.options?.next?.tags).toContain( + contentPublicListTag(testEditorialPostContentType.id), + ); + }); +}); + +describe("a public response is cached under exactly one key per locale", () => { + it.each([["pl"], ["PL"], [" pl "], ["pL"]])( + "normalises %s onto the same tags", + locale => { + const id = testLocalizedPageContentType.id; + + expect(contentPublicListTag(id, locale)).toBe( + contentPublicListTag(id, "pl"), + ); + expect(contentPublicItemTag(id, 7, locale)).toBe( + contentPublicItemTag(id, 7, "pl"), + ); + expect(contentPublicSlugTag(id, "witaj", locale)).toBe( + contentPublicSlugTag(id, "witaj", "pl"), + ); + }, + ); + + it("sends the same tags however the caller spelled the locale", async () => { + const tagsFor = async (locale: string) => { + await contentPublicFetch({ + definition: testLocalizedPageContentType, + locale, + pluginId: PLUGIN_ID, + slug: "witaj", + }); + + return lastCall().options?.next?.tags; + }; + + expect(await tagsFor("PL")).toEqual(await tagsFor("pl")); + }); + + it("keeps two languages on one slug apart", () => { + // The whole reason the locale is in the slug tag: `/en/about` and + // `/pl/about` are two pages, and expiring one must not expire the other. + const id = testLocalizedPageContentType.id; + + expect(contentPublicSlugTag(id, "about", "en")).not.toBe( + contentPublicSlugTag(id, "about", "pl"), + ); + }); + + it("keeps a localized tag out of the locale-less namespace", () => { + // A localized content type has no locale-less public URL, so a tag without + // a locale segment would name a page that does not exist - and one that a + // non-localized content type might legitimately own. + const id = testLocalizedPageContentType.id; + + expect(contentPublicListTag(id, "en")).not.toBe(contentPublicListTag(id)); + }); +}); diff --git a/packages/vitnode/src/content/next/delivery.server.test.ts b/packages/vitnode/src/content/next/delivery.server.test.ts new file mode 100644 index 000000000..ec97c61c6 --- /dev/null +++ b/packages/vitnode/src/content/next/delivery.server.test.ts @@ -0,0 +1,126 @@ +// @vitest-environment node +import { describe, expect, it, vi } from "vitest"; + +// `server-only` throws on import outside a server component, which is exactly its +// job - and exactly what a unit test has to stub, the same way the revalidation +// tests do. +vi.mock("server-only", () => ({})); + +import type { ContentDeliveryResponse } from "./delivery.server"; + +import { contentDeliveryToNextMetadata } from "./delivery.server"; + +/** + * The Next.js metadata mapping, without a network. + * + * `contentDeliveryToNextMetadata` is the whole of the adapter's judgement: which + * delivery fields become which `Metadata` keys, and what an absent value does. Every + * assertion below is about a key being **absent** rather than present-and-null, + * because Next renders a `null` title as an empty `<title>` and an absent one not at + * all - and an empty `<title>` is worse than none. + */ + +const response = ( + overrides: Partial<ContentDeliveryResponse> = {}, +): ContentDeliveryResponse => ({ + alternates: [], + canonicalPath: "/articles/hello", + hreflang: { languages: {} }, + isFallback: false, + itemId: 42, + locale: null, + openGraph: null, + requestedLocale: null, + robots: null, + seo: { description: "A summary.", title: "Hello" }, + ...overrides, +}); + +describe("contentDeliveryToNextMetadata", () => { + it("maps the canonical path relative when no origin is given", () => { + expect(contentDeliveryToNextMetadata(response())).toStrictEqual({ + alternates: { canonical: "/articles/hello" }, + description: "A summary.", + title: "Hello", + }); + }); + + it("makes every URL absolute when an origin is given", () => { + const metadata = contentDeliveryToNextMetadata( + response({ + hreflang: { + languages: { en: "/en/articles/hello", pl: "/pl/articles/witaj" }, + xDefault: "/en/articles/hello", + }, + }), + { origin: "https://example.com" }, + ); + + expect(metadata.alternates).toStrictEqual({ + canonical: "https://example.com/articles/hello", + languages: { + en: "https://example.com/en/articles/hello", + pl: "https://example.com/pl/articles/witaj", + // `x-default` is the standard's own key, so it lives in the same map. + "x-default": "https://example.com/en/articles/hello", + }, + }); + }); + + it("omits a title rather than emitting an empty one", () => { + const metadata = contentDeliveryToNextMetadata( + response({ seo: { description: null, title: null } }), + ); + + expect(metadata).not.toHaveProperty("title"); + expect(metadata).not.toHaveProperty("description"); + }); + + it("omits alternates entirely when there is nothing to say", () => { + const metadata = contentDeliveryToNextMetadata( + response({ canonicalPath: null }), + ); + + expect(metadata).not.toHaveProperty("alternates"); + }); + + it("emits no Open Graph block when the content type configured none", () => { + expect(contentDeliveryToNextMetadata(response())).not.toHaveProperty( + "openGraph", + ); + }); + + it("carries the canonical URL into the Open Graph block", () => { + const metadata = contentDeliveryToNextMetadata( + response({ + openGraph: { description: "Social summary", title: "Social" }, + }), + { origin: "https://example.com" }, + ); + + expect(metadata.openGraph).toStrictEqual({ + description: "Social summary", + title: "Social", + url: "https://example.com/articles/hello", + }); + }); + + it("passes the robots directive through untouched", () => { + expect( + contentDeliveryToNextMetadata( + response({ robots: { follow: true, index: false } }), + ).robots, + ).toStrictEqual({ follow: true, index: false }); + }); + + it("drops an alternate whose path will not resolve against the origin", () => { + const metadata = contentDeliveryToNextMetadata( + response({ hreflang: { languages: { en: "http://", pl: "/pl/x" } } }), + { origin: "https://example.com" }, + ); + + expect(metadata.alternates?.languages).toStrictEqual({ + pl: "https://example.com/pl/x", + }); + }); +}); diff --git a/packages/vitnode/src/content/next/delivery.server.ts b/packages/vitnode/src/content/next/delivery.server.ts new file mode 100644 index 000000000..1f6940876 --- /dev/null +++ b/packages/vitnode/src/content/next/delivery.server.ts @@ -0,0 +1,407 @@ +import "server-only"; + +import type { + ContentDeliveryAlternate, + ContentDeliveryRobots, + ContentDeliverySeo, +} from "../delivery"; +import type { ContentSitemapEntry } from "../sitemap"; +import type { DeliverableContentTypeDefinition } from "../types"; + +import { rawApiFetch } from "../../lib/fetcher/raw"; +import { + contentDeliveryRedirectTag, + contentDeliverySitemapTag, + contentDeliveryTag, +} from "../cache"; +import { CONTENT_SITEMAP_DEFAULT_PAGE_SIZE } from "../const"; +import { contentDeliveryUrl } from "../delivery"; + +/** + * The Next.js side of Content Delivery. + * + * A **thin adapter**, and the thinness is the point: the core engine returns + * framework-neutral delivery metadata, and this module turns it into the two + * shapes Next.js asks for - a `generateMetadata` return value and a `sitemap.ts` + * return value. Nothing here decides anything; move to Astro and you write a + * different forty lines against the same service. + * + * It reads over HTTP rather than through `model.deliveryService`, because in + * VitNode's split deployment the web app is not the process that holds the + * database. A single-process install can still call the service directly and skip + * this entirely. + */ + +/** The delivery metadata of one record, as the API returns it. */ +export interface ContentDeliveryResponse { + alternates: ContentDeliveryAlternate[]; + canonicalPath: null | string; + hreflang: { languages: Record<string, string>; xDefault?: string }; + isFallback: boolean; + /** `null` when the content type's public allowlist withholds `id`. */ + itemId: null | number; + locale: null | string; + openGraph: ContentDeliverySeo | null; + requestedLocale: null | string; + robots: ContentDeliveryRobots | null; + seo: ContentDeliverySeo; +} + +export type ContentDeliveryResolutionResponse = + | (ContentDeliveryResponse & { type: "content" }) + | { location: string; status: number; type: "redirect" } + | { type: "not_found" }; + +/** + * The subset of Next's `Metadata` this adapter produces. + * + * Structural rather than an `import type { Metadata } from "next"`, so the core + * package does not grow a compile-time dependency on the framework's type surface + * for four keys. It is assignable to `Metadata`, which is what a + * `generateMetadata` needs it to be. + */ +export interface ContentDeliveryNextMetadata { + alternates?: { + canonical?: string; + languages?: Record<string, string>; + }; + description?: string; + openGraph?: { + description?: string; + title?: string; + url?: string; + }; + robots?: { follow: boolean; index: boolean }; + title?: string; +} + +const deliveryModule = (definition: DeliverableContentTypeDefinition): string => + `content/${definition.publicApi.path}`; + +/** + * Resolves one public URL through the API, cached and tagged. + * + * Two tags, because one response answers two questions that expire at different + * moments: the record's delivery metadata, and "does this slug still resolve here". + * A slug change invalidates the second for the *old* address and the first for the + * record, and tagging both is what makes a moved page stop being served from its + * former URL. + * + * Only a `200` is stored, and a `not_found` is a `200` with a body - so a URL that + * does not exist yet is not cached as a negative, and publishing the record makes it + * resolve immediately. + */ +export const contentDeliveryResolve = async ({ + definition, + locale, + pluginId, + slug, +}: { + definition: DeliverableContentTypeDefinition; + /** The language to resolve in, for a localized content type. */ + locale?: string; + pluginId: string; + slug: string; +}): Promise<ContentDeliveryResolutionResponse> => { + const effectiveLocale = definition.localization.enabled + ? (locale?.trim() ?? "") === "" + ? definition.localization.defaultLocale + : locale + : undefined; + + const response = await rawApiFetch({ + method: "get", + module: deliveryModule(definition), + options: { + cache: "force-cache", + next: { + tags: [ + contentDeliveryRedirectTag(definition.id, slug, effectiveLocale), + ], + }, + }, + path: `/delivery/resolve/${encodeURIComponent(slug)}`, + pluginId, + query: + effectiveLocale === undefined ? undefined : { locale: effectiveLocale }, + }); + + if (!response.ok) return { type: "not_found" }; + + const payload = (await response.json()) as ContentDeliveryResolutionResponse; + + return payload; +}; + +/** + * Delivery metadata by identifier, cached under the record's delivery tag. + * + * The tag is the *record's*, not the slug's, which is what makes this the right + * call for a page that already knows which record it is rendering: an edit to the + * SEO description expires it, and an unrelated record's publish does not. + */ +export const contentDeliveryItem = async ({ + definition, + id, + locale, + pluginId, +}: { + definition: DeliverableContentTypeDefinition; + id: number; + locale?: string; + pluginId: string; +}): Promise<ContentDeliveryResponse | null> => { + const effectiveLocale = definition.localization.enabled + ? (locale?.trim() ?? "") === "" + ? definition.localization.defaultLocale + : locale + : undefined; + + const response = await rawApiFetch({ + method: "get", + module: deliveryModule(definition), + options: { + cache: "force-cache", + next: { tags: [contentDeliveryTag(definition.id, id, effectiveLocale)] }, + }, + path: `/delivery/item/${id}`, + pluginId, + query: + effectiveLocale === undefined ? undefined : { locale: effectiveLocale }, + }); + + if (!response.ok) return null; + + return (await response.json()) as ContentDeliveryResponse; +}; + +/** + * Delivery metadata as a `generateMetadata` return value. + * + * ```tsx title="src/app/[locale]/articles/[slug]/page.tsx" + * export const generateMetadata = async ({ params }) => { + * const { locale, slug } = await params; + * + * return await contentDeliveryMetadata({ + * definition: articleContentType, + * locale, + * origin: "https://example.com", + * pluginId: "@vitnode/example", + * slug, + * }); + * }; + * ``` + * + * The canonical URL is **absolute when an origin is given and relative otherwise**, + * which is the one place delivery is opinionated: a relative `canonical` is legal + * and resolves against the page, and an absolute one is what every SEO checker asks + * for - so an app that knows its public origin should pass it. + * + * `{}` for a URL that does not resolve, rather than a throw: `generateMetadata` + * runs alongside the page, the page is what calls `notFound()`, and a metadata + * function that threw would replace a clean 404 with an error boundary. + */ +export const contentDeliveryMetadata = async ({ + definition, + locale, + origin, + pluginId, + slug, +}: { + definition: DeliverableContentTypeDefinition; + locale?: string; + /** Turns every URL in the result absolute. Strongly recommended. */ + origin?: string; + pluginId: string; + slug: string; +}): Promise<ContentDeliveryNextMetadata> => { + const resolution = await contentDeliveryResolve({ + definition, + locale, + pluginId, + slug, + }); + + return resolution.type === "content" + ? contentDeliveryToNextMetadata(resolution, { origin }) + : {}; +}; + +/** + * The pure half of {@link contentDeliveryMetadata}: metadata in, `Metadata` out. + * + * Exported separately so a page that already holds the delivery response - because + * it fetched the record and its metadata together - can translate it without a + * second round trip. It is also what makes the mapping unit-testable without a + * network. + */ +export const contentDeliveryToNextMetadata = ( + metadata: ContentDeliveryResponse, + { origin }: { origin?: string } = {}, +): ContentDeliveryNextMetadata => { + const absolute = (path: null | string): string | undefined => { + if (path === null) return undefined; + + return origin === undefined + ? path + : (contentDeliveryUrl({ origin, path }) ?? undefined); + }; + + const canonical = absolute(metadata.canonicalPath); + const languages = Object.fromEntries( + Object.entries(metadata.hreflang.languages).flatMap(([code, path]) => { + const href = absolute(path); + + return href === undefined ? [] : [[code, href]]; + }), + ); + const xDefault = + metadata.hreflang.xDefault === undefined + ? undefined + : absolute(metadata.hreflang.xDefault); + + return { + ...(canonical === undefined && Object.keys(languages).length === 0 + ? {} + : { + alternates: { + ...(canonical === undefined ? {} : { canonical }), + ...(Object.keys(languages).length === 0 + ? {} + : { + languages: { + ...languages, + // `x-default` is the standard's own key, so it goes in the same + // map rather than beside it - which is also how Next emits it. + ...(xDefault === undefined + ? {} + : { "x-default": xDefault }), + }, + }), + }, + }), + ...(metadata.seo.description === null + ? {} + : { description: metadata.seo.description }), + ...(metadata.openGraph === null + ? {} + : { + openGraph: { + ...(metadata.openGraph.description === null + ? {} + : { description: metadata.openGraph.description }), + ...(metadata.openGraph.title === null + ? {} + : { title: metadata.openGraph.title }), + ...(canonical === undefined ? {} : { url: canonical }), + }, + }), + ...(metadata.robots === null ? {} : { robots: metadata.robots }), + ...(metadata.seo.title === null ? {} : { title: metadata.seo.title }), + }; +}; + +/** One entry of a Next.js `sitemap.ts`, as that file's return type wants it. */ +export interface ContentDeliveryNextSitemapEntry { + alternates?: { languages?: Record<string, string> }; + changeFrequency?: ContentSitemapEntry["changeFrequency"]; + lastModified?: Date; + priority?: number; + url: string; +} + +/** + * Every public URL of one content type, in one language, as a Next sitemap. + * + * It pages through the delivery sitemap route until the cursor runs out, so a + * content type with 40,000 published records is 40 requests rather than one + * enormous response - and `maxPages` is a backstop, because an unbounded loop + * against a paginated API is the one bug in this file that could take a site down. + * Reaching it is reported by the return value rather than thrown, so a partial + * sitemap is still a valid sitemap. + * + * Next caps a `sitemap.ts` at 50,000 URLs and splits beyond that with + * `generateSitemaps`; `contentSitemapChunks` is the helper that decides how many + * files that is. + */ +export const contentSitemapEntries = async ({ + definition, + locale, + maxPages = 100, + origin, + pageSize = CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + pluginId, +}: { + definition: DeliverableContentTypeDefinition; + locale?: string; + /** Backstop on the pagination loop. */ + maxPages?: number; + /** Required: the sitemap protocol only accepts absolute URLs. */ + origin: string; + pageSize?: number; + pluginId: string; +}): Promise<{ + entries: ContentDeliveryNextSitemapEntry[]; + /** `true` when `maxPages` stopped the loop before the cursor ran out. */ + truncated: boolean; +}> => { + const effectiveLocale = definition.localization.enabled + ? (locale?.trim() ?? "") === "" + ? definition.localization.defaultLocale + : locale + : undefined; + + const entries: ContentDeliveryNextSitemapEntry[] = []; + let cursor: null | number = null; + let truncated = false; + + for (let visited = 0; visited < maxPages; visited += 1) { + const response = await rawApiFetch({ + method: "get", + module: deliveryModule(definition), + options: { + cache: "force-cache", + next: { + tags: [contentDeliverySitemapTag(definition.id, effectiveLocale)], + }, + }, + path: "/delivery/sitemap", + pluginId, + query: { + ...(cursor === null ? {} : { cursor: String(cursor) }), + ...(effectiveLocale === undefined ? {} : { locale: effectiveLocale }), + limit: String(pageSize), + }, + }); + + if (!response.ok) break; + + const page = (await response.json()) as { + entries: (Omit<ContentSitemapEntry, "lastModified"> & { + lastModified: string; + })[]; + nextCursor: null | number; + }; + + for (const entry of page.entries) { + const url = contentDeliveryUrl({ origin, path: entry.path }); + if (url === null) continue; + + entries.push({ + ...(entry.changeFrequency === null + ? {} + : { changeFrequency: entry.changeFrequency }), + lastModified: new Date(entry.lastModified), + ...(entry.priority === null ? {} : { priority: entry.priority }), + url, + }); + } + + cursor = page.nextCursor; + if (cursor === null) return { entries, truncated }; + } + + truncated = cursor !== null; + + return { entries, truncated }; +}; diff --git a/packages/vitnode/src/content/next/index.ts b/packages/vitnode/src/content/next/index.ts index c918315b6..1d1e10e3f 100644 --- a/packages/vitnode/src/content/next/index.ts +++ b/packages/vitnode/src/content/next/index.ts @@ -8,12 +8,26 @@ * * The cache *tags* live in `@vitnode/core/content`, because they are strings. */ +export { + contentDeliveryItem, + contentDeliveryMetadata, + contentDeliveryResolve, + contentDeliveryToNextMetadata, + contentSitemapEntries, +} from "./delivery.server"; +export type { + ContentDeliveryNextMetadata, + ContentDeliveryNextSitemapEntry, + ContentDeliveryResolutionResponse, + ContentDeliveryResponse, +} from "./delivery.server"; export { contentPreviewFetch, contentPublicFetch, contentPublicItemTags, } from "./fetch.server"; export type { ContentPublicFetchResult } from "./fetch.server"; +export { contentDeliveryPage } from "./redirect.server"; export { POST as contentRevalidateRoute } from "./revalidate-route.server"; export { revalidateContent } from "./revalidate.server"; export type { diff --git a/packages/vitnode/src/content/next/redirect.server.ts b/packages/vitnode/src/content/next/redirect.server.ts new file mode 100644 index 000000000..3e97f68c0 --- /dev/null +++ b/packages/vitnode/src/content/next/redirect.server.ts @@ -0,0 +1,78 @@ +import "server-only"; +// `vitnode-frontend/navigation` is the locale-aware wrapper every app-level redirect +// should use, and this is the one place it would be wrong: a delivery location is a +// **complete** path that already carries its locale segment - the engine built it - +// so routing it through `next-intl` would prefix the locale a second time and send +// `/pl/articles/x` to `/pl/pl/articles/x`. That wrapper is also a 307; a canonical +// slug change needs the permanent, method-preserving 308. +// eslint-disable-next-line no-restricted-imports +import { notFound, permanentRedirect, RedirectType } from "next/navigation"; + +import type { DeliverableContentTypeDefinition } from "../types"; +import type { ContentDeliveryResponse } from "./delivery.server"; + +import { contentDeliveryResolve } from "./delivery.server"; + +/** + * Resolves a public URL and *acts* on the answer: renders, redirects or 404s. + * + * The one helper in the delivery adapter that has a side effect, and it is kept in + * its own module because of what it imports: `next/navigation`'s control-flow + * functions throw to unwind the render, so a page that only wanted metadata should + * not be able to reach them by accident. + * + * ```tsx title="src/app/[locale]/articles/[slug]/page.tsx" + * const Page = async ({ params }) => { + * const { locale, slug } = await params; + * const delivery = await contentDeliveryPage({ + * definition: articleContentType, + * locale, + * pluginId: "@vitnode/example", + * slug, + * }); + * + * // Only reached when the slug is the current one - a moved URL has already + * // redirected and a missing one has already 404ed. + * return <Article delivery={delivery} />; + * }; + * ``` + * + * `permanentRedirect` issues a **308**, which is what the engine's resolver reports + * and the status a canonical slug change deserves: it preserves the request method, + * where a `301` lets a client rewrite it to `GET`. Both behave identically for the + * `GET` a content page is read with - and only one of them still behaves correctly + * the day a form under a moved path is submitted. + * + * `RedirectType.replace`, so a reader who follows an old link does not have to press + * back twice to leave the page they were never meant to land on. + */ +export const contentDeliveryPage = async ({ + definition, + locale, + pluginId, + slug, +}: { + definition: DeliverableContentTypeDefinition; + locale?: string; + pluginId: string; + slug: string; +}): Promise<ContentDeliveryResponse> => { + const resolution = await contentDeliveryResolve({ + definition, + locale, + pluginId, + slug, + }); + + if (resolution.type === "redirect") { + permanentRedirect(resolution.location, RedirectType.replace); + } + + // A draft, an unpublished record, a deleted one, a slug that never existed and a + // historical URL whose destination is no longer public are all the same 404. A + // redirect to hidden content would be a way to confirm it exists, and that is + // precisely what an unpublished URL must not do. + if (resolution.type === "not_found") notFound(); + + return resolution; +}; diff --git a/packages/vitnode/src/content/next/revalidate-route.server.test.ts b/packages/vitnode/src/content/next/revalidate-route.server.test.ts index bbfadac94..ec2cf551c 100644 --- a/packages/vitnode/src/content/next/revalidate-route.server.test.ts +++ b/packages/vitnode/src/content/next/revalidate-route.server.test.ts @@ -81,6 +81,64 @@ describe("the revalidation Route Handler", () => { ]); }); + it("carries the delivery tags across the bridge", async () => { + // The bug this pins down: `zodBody` strips whatever it does not declare, so a + // missing `delivery` member meant a scheduled publish crossed the bridge with its + // delivery tags and arrived with none - leaving a stale sitemap and a stale + // canonical response behind every background transition. + await POST( + request({ + body: JSON.stringify({ + ...body, + delivery: { sitemap: { contentChanged: true, indexChanged: true } }, + }), + }), + ); + + const tags = calls.map(call => call.tag); + + expect(tags).toContain("content:example.article:delivery:7"); + expect(tags).toContain("content:example.article:redirect:hello-world"); + expect(tags).toContain("content:example.article:sitemap"); + }); + + it("expires no sitemap tag when the bridge says it did not move", async () => { + await POST( + request({ + body: JSON.stringify({ + ...body, + delivery: { sitemap: { contentChanged: false, indexChanged: false } }, + }), + }), + ); + + const tags = calls.map(call => call.tag); + + expect(tags).toContain("content:example.article:delivery:7"); + expect(tags).not.toContain("content:example.article:sitemap"); + }); + + it("accepts a body with no delivery member at all", async () => { + // An API that has not been redeployed posts the Stage 1-7 shape, and that body + // still has to be accepted rather than 400. + const response = await POST(request()); + + expect(response.status).toBe(200); + expect(calls.map(call => call.tag)).not.toContain( + "content:example.article:delivery:7", + ); + }); + + it("refuses a malformed delivery member", async () => { + const response = await POST( + request({ + body: JSON.stringify({ ...body, delivery: { sitemap: true } }), + }), + ); + + expect(response.status).toBe(400); + }); + it("honours stale-while-revalidate", async () => { await POST( request({ diff --git a/packages/vitnode/src/content/next/revalidate-route.server.ts b/packages/vitnode/src/content/next/revalidate-route.server.ts index c896939b6..5f65dc587 100644 --- a/packages/vitnode/src/content/next/revalidate-route.server.ts +++ b/packages/vitnode/src/content/next/revalidate-route.server.ts @@ -12,6 +12,23 @@ import { revalidateContent } from "./revalidate.server"; const zodBody = z.object({ contentTypeId: z.string().min(1), + /** + * The delivery share of a mutation, for a content type with `delivery`. + * + * Optional for the same reason `locales` is: an API that has not been redeployed + * posts a body without it, and that body still has to be accepted. Without this + * member the object schema would **strip** it - so a scheduled publish would cross + * the bridge carrying its delivery tags and arrive with none, leaving a stale + * sitemap and a stale canonical response behind every background transition. + */ + delivery: z + .object({ + sitemap: z.object({ + contentChanged: z.boolean(), + indexChanged: z.boolean(), + }), + }) + .optional(), id: z.number().int().positive(), isPublic: z.boolean(), /** diff --git a/packages/vitnode/src/content/registry.test.ts b/packages/vitnode/src/content/registry.test.ts index 541df4c8f..64afd614c 100644 --- a/packages/vitnode/src/content/registry.test.ts +++ b/packages/vitnode/src/content/registry.test.ts @@ -393,3 +393,393 @@ describe("public paths", () => { ).not.toThrow(); }); }); + +/** + * Names nobody wrote down. + * + * A junction or repeatable child table is generated from a field name, snake- + * cased and clamped to Postgres' 63-character limit - so two content types can + * reach the same physical table without either author ever typing it. Postgres + * would not complain: the second `CREATE TABLE` simply never runs, and from then + * on two definitions read and write one table. The same applies one level down, + * to the `UNIQUE (itemId, position)` constraint each of them carries. + */ +describe("generated database identifiers", () => { + const related = ( + id: string, + tableName: string, + fields: Parameters<typeof defineContentType>[0]["fields"], + ) => + defineContentType({ + id, + tableName, + fields, + admin: { + label: { plural: "Things", singular: "Thing" }, + permissionModule: tableName, + form: { fields: Object.keys(fields) }, + }, + }); + + const withRelation = (id: string, tableName: string, field_: string) => + related(id, tableName, { + title: field.text({ required: true }), + [field_]: field.relation({ + multiple: true, + target: () => testCategoryContentType, + }), + }); + + const withRepeatable = (id: string, tableName: string, field_: string) => + related(id, tableName, { + title: field.text({ required: true }), + [field_]: field.repeatable({ + fields: { question: field.text({ required: true }) }, + }), + }); + + it("accepts advanced content types whose generated names differ", () => { + expect(() => + validateContentTypes([ + entry(withRelation("test.one", "test_ones", "tags")), + entry(withRepeatable("test.two", "test_twos", "faq")), + ]), + ).not.toThrow(); + }); + + it("rejects a base table that collides with another type's junction table", () => { + // `test_ones` + `tags` generates `test_ones_tags`, which is exactly the base + // table the second content type declares. Silently sharing it would give one + // definition's rows two owners. + expect(() => + validateContentTypes([ + entry(withRelation("test.one", "test_ones", "tags"), "@vitnode/a"), + entry( + related("test.two", "test_ones_tags", { + title: field.text({ required: true }), + }), + "@vitnode/b", + ), + ]), + ).toThrow(/Table "test_ones_tags" is claimed by both/); + }); + + it("rejects a base table that collides with another type's repeatable table", () => { + expect(() => + validateContentTypes([ + entry( + related("test.two", "test_ones_faq", { + title: field.text({ required: true }), + }), + "@vitnode/b", + ), + entry(withRepeatable("test.one", "test_ones", "faq"), "@vitnode/a"), + ]), + ).toThrow(/Table "test_ones_faq" is claimed by both/); + }); + + it("names the field that generated the colliding table", () => { + // A boot error is only useful if it says which of the two to rename, and the + // generated side is the one that has no obvious name to look for. + expect(() => + validateContentTypes([ + entry(withRelation("test.one", "test_ones", "tags"), "@vitnode/a"), + entry( + related("test.two", "test_ones_tags", { + title: field.text({ required: true }), + }), + "@vitnode/b", + ), + ]), + ).toThrow(/the junction table of "tags"/); + }); + + it("rejects two content types generating the same junction table", () => { + // Different content types, different fields, one physical table: the + // second's junction is named after the first's. + expect(() => + validateContentTypes([ + entry(withRelation("test.one", "test_ones", "tags"), "@vitnode/a"), + entry( + withRelation("test.two", "test_ones_tags", "labels"), + "@vitnode/b", + ), + ]), + ).toThrow(/Table "test_ones_tags" is claimed by both/); + }); + + it("registers the generated position constraint in the index namespace", () => { + // `test_ones` + `faq` generates `test_ones_faq_position_key`. An explicit + // index of that name on another content type would be the same identifier + // in the same schema, and only one of the two would exist. + expect(() => + validateContentTypes([ + entry(withRepeatable("test.one", "test_ones", "faq"), "@vitnode/a"), + entry( + related("test.two", "test_twos", { + title: field.text({ required: true }), + }), + "@vitnode/b", + ), + ]), + ).not.toThrow(); + + expect(() => + validateContentTypes([ + entry(withRepeatable("test.one", "test_ones", "faq"), "@vitnode/a"), + entry( + defineContentType({ + id: "test.two", + tableName: "test_twos", + fields: { title: field.text({ required: true }) }, + indexes: [{ name: "test_ones_faq_position_key", on: ["title"] }], + admin: { + label: { plural: "Twos", singular: "Two" }, + permissionModule: "test_twos", + }, + }), + "@vitnode/b", + ), + ]), + ).toThrow(/Index name "test_ones_faq_position_key" is used by both/); + }); + + it("registers a junction's primary key and target index too", () => { + for (const name of [ + "test_ones_tags_pk", + "test_ones_tags_related_item_id_idx", + ]) { + expect(() => + validateContentTypes([ + entry(withRelation("test.one", "test_ones", "tags"), "@vitnode/a"), + entry( + defineContentType({ + id: "test.two", + tableName: "test_twos", + fields: { title: field.text({ required: true }) }, + indexes: [{ name, on: ["title"] }], + admin: { + label: { plural: "Twos", singular: "Two" }, + permissionModule: "test_twos", + }, + }), + "@vitnode/b", + ), + ]), + ).toThrow(new RegExp(`Index name "${name}" is used by both`)); + } + }); + + it("keeps two long generated table names apart by their fingerprint", () => { + // The clamp is what makes a collision reachable at all; the fingerprint is + // what makes it vanishingly unlikely. Both names fill the limit exactly and + // still differ, so the pair boots. + const base = `t_${"a".repeat(56)}`; + const first = withRepeatable("test.long1", `${base}_x`, "faq"); + const second = withRepeatable("test.long2", `${base}_y`, "faq"); + + const [firstTable, secondTable] = [first, second].map( + definition => definition.advanced.repeatables[0].tableName, + ); + + expect(firstTable).toHaveLength(63); + expect(secondTable).toHaveLength(63); + expect(firstTable).not.toBe(secondTable); + expect(() => + validateContentTypes([ + entry(first, "@vitnode/a"), + entry(second, "@vitnode/b"), + ]), + ).not.toThrow(); + }); +}); + +/** + * Delivery paths are a **site-wide** namespace, unlike the API paths above. + * + * The asymmetry is the whole of this block. A generated API route is + * `/api/{pluginId}/content/{path}`, so two plugins publishing `articles` do not + * collide and Stage 1-7 deliberately allows it. A canonical delivery URL is + * `/articles/{slug}` with no plugin id in it at all, so the same pair really would + * give one public URL two owners: two resolvers claiming it, two sitemaps listing it, + * and one slug reservation table with no way to say whose a retired address was. + */ +describe("delivery paths", () => { + const deliveryWidget = ( + id: string, + tableName: string, + path: string, + { delivery = true }: { delivery?: boolean } = {}, + ) => + defineContentType({ + id, + tableName, + fields: { + title: field.text({ required: true }), + slug: field.slug({ source: "title" }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + path, + fields: ["id", "title", "slug"], + }, + ...(delivery ? { delivery: { enabled: true } } : {}), + admin: { + label: { plural: "Widgets", singular: "Widget" }, + // Distinct, so the permission-module check does not fire first and mask the + // one this block is about. + permissionModule: tableName, + }, + }); + + it("still lets two plugins share a path when neither has delivery", () => { + // The Stage 1-7 promise, restated here so a future delivery change cannot + // quietly turn the API namespace into a global one. + expect(() => + validateContentTypes([ + entry( + deliveryWidget("first.one", "first_ones", "articles", { + delivery: false, + }), + "@acme/one", + ), + entry( + deliveryWidget("second.one", "second_ones", "articles", { + delivery: false, + }), + "@acme/two", + ), + ]), + ).not.toThrow(); + }); + + it("rejects two plugins claiming the same delivery path", () => { + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + entry( + deliveryWidget("news.article", "news_articles", "articles"), + "@acme/news", + ), + ]), + ).toThrow(ContentEngineError); + }); + + it("names both conflicting owners", () => { + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + entry( + deliveryWidget("news.article", "news_articles", "articles"), + "@acme/news", + ), + ]), + ).toThrow( + /Delivery path "articles" is claimed by both @acme\/blog -> blog\.article and @acme\/news -> news\.article/, + ); + }); + + it("says why the namespace is global", () => { + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + entry( + deliveryWidget("news.article", "news_articles", "articles"), + "@acme/news", + ), + ]), + ).toThrow(/site-wide public namespaces and must be globally unique/); + }); + + it("accepts different delivery paths across plugins", () => { + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + entry( + deliveryWidget("news.article", "news_articles", "news"), + "@acme/news", + ), + ]), + ).not.toThrow(); + }); + + it("rejects two content types in one plugin claiming one delivery path", () => { + // The per-plugin API check fires first here, which is correct - both rules are + // violated, and the one that names the narrower fix wins. + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + entry( + deliveryWidget("blog.news", "blog_news", "articles"), + "@acme/blog", + ), + ]), + ).toThrow(ContentEngineError); + }); + + it("does not let a non-delivery route reserve the site namespace", () => { + // A plugin whose `articles` route has no delivery claims nothing site-wide, so a + // delivery-enabled `articles` elsewhere is still free to take it. + expect(() => + validateContentTypes([ + entry( + deliveryWidget("plain.one", "plain_ones", "articles", { + delivery: false, + }), + "@acme/plain", + ), + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + ]), + ).not.toThrow(); + }); + + it("rejects the mixed case whichever order the two arrive in", () => { + const delivered = () => + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ); + const other = () => + entry( + deliveryWidget("news.article", "news_articles", "articles"), + "@acme/news", + ); + + expect(() => validateContentTypes([delivered(), other()])).toThrow( + ContentEngineError, + ); + expect(() => validateContentTypes([other(), delivered()])).toThrow( + ContentEngineError, + ); + }); + + it("leaves one delivery-enabled content type alone", () => { + expect(() => + validateContentTypes([ + entry( + deliveryWidget("blog.article", "blog_articles", "articles"), + "@acme/blog", + ), + ]), + ).not.toThrow(); + }); +}); diff --git a/packages/vitnode/src/content/registry.ts b/packages/vitnode/src/content/registry.ts index 2c71a5723..05230e15b 100644 --- a/packages/vitnode/src/content/registry.ts +++ b/packages/vitnode/src/content/registry.ts @@ -31,6 +31,96 @@ interface IndexOwner { entry: RegisteredContentType; } +/** A physical table name, and the field or content type that generated it. */ +interface TableOwner { + entry: RegisteredContentType; + /** How the name came about, for an error message somebody has to act on. */ + origin: string; +} + +const describeTableOwner = (owner: TableOwner): string => + `${describe(owner.entry)} (${owner.origin})`; + +/** + * Every physical table one content type puts into the schema. + * + * The base table and the translation table are declared; a junction and a + * repeatable child table are **generated** from a field name, clamped to + * Postgres' 63-character limit. Two content types can therefore reach the same + * physical name without either author writing it down - which Postgres would + * accept, because the second `CREATE TABLE` simply never happens and both + * definitions then read and write one table. + */ +const physicalTables = ( + entry: RegisteredContentType, +): { name: string; origin: string }[] => { + const { definition } = entry; + + return [ + { name: definition.tableName, origin: "its base table" }, + ...(definition.localization.enabled + ? [ + { + name: definition.localization.translationTableName, + origin: "its translation table", + }, + ] + : []), + ...definition.advanced.junctions.map(junction => ({ + name: junction.tableName, + origin: `the junction table of "${junction.field}"`, + })), + ...definition.advanced.repeatables.map(repeatable => ({ + name: repeatable.tableName, + origin: `the child table of "${repeatable.field}"`, + })), + ]; +}; + +/** + * Every index and constraint name one content type puts into the schema. + * + * Same namespace problem as the tables, one level down: `UNIQUE (itemId, + * position)` on a junction is named after the junction, which is named after a + * field, which is clamped. Postgres keeps index and constraint names unique per + * schema, so two of them colliding is a migration that fails at deploy time - + * or, with the truncation, one that silently constrains the wrong table. + */ +const physicalIndexes = ( + entry: RegisteredContentType, +): { columns: string[]; name: string }[] => { + const { definition } = entry; + + return [ + ...definition.indexes.map(index => ({ + columns: index.on, + name: index.name, + })), + ...definition.localization.translationIndexes.map(index => ({ + columns: index.on, + name: index.name, + })), + ...definition.advanced.junctions.flatMap(junction => [ + { + columns: [`${junction.field} primary key`], + name: junction.primaryKeyName, + }, + { + columns: [`${junction.field} position`], + name: junction.positionIndexName, + }, + { + columns: [`${junction.field} target`], + name: junction.relatedIndexName, + }, + ]), + ...definition.advanced.repeatables.map(repeatable => ({ + columns: [`${repeatable.field} position`], + name: repeatable.positionIndexName, + })), + ]; +}; + /** * Validates a set of content types coming from one or more plugins. * @@ -43,14 +133,30 @@ interface IndexOwner { * table name, or two content types resolving to the same Postgres index name. * Permission modules and public paths are checked per plugin, because the * plugin id is part of the key each one is addressed by. + * + * **Delivery paths are the one exception**, and the asymmetry is deliberate: an API + * route carries the plugin id and a canonical delivery URL does not, so the second is + * a site-wide namespace where the first is not. See `byDeliveryPath` below. */ export const validateContentTypes = ( entries: RegisteredContentType[], ): RegisteredContentType[] => { const byId = new Map<string, RegisteredContentType>(); - const byTable = new Map<string, RegisteredContentType>(); + const byTable = new Map<string, TableOwner>(); const byPermission = new Map<string, RegisteredContentType>(); const byPublicPath = new Map<string, RegisteredContentType>(); + /** + * Delivery paths, keyed by the path alone. + * + * A **second** map rather than a different key on `byPublicPath`, because the two + * namespaces are genuinely different and both have to be checked. A generated API + * route is `/api/{pluginId}/content/{path}`, so `plugin-a` and `plugin-b` may both + * publish `articles` - and forbidding that would make an app fail to boot over a + * name neither author can see. A **canonical delivery URL** is `/articles/{slug}` + * with no plugin id in it at all, so the same pair really would claim one site-wide + * namespace and `/articles/example` would have two owners. + */ + const byDeliveryPath = new Map<string, RegisteredContentType>(); const byIndexName = new Map<string, IndexOwner>(); for (const entry of entries) { @@ -65,30 +171,22 @@ export const validateContentTypes = ( } byId.set(definition.id, entry); - const duplicateTable = byTable.get(definition.tableName); - if (duplicateTable) { - throw new ContentEngineError( - `Table "${definition.tableName}" is claimed by both ${describe(duplicateTable)} and ${describe(entry)}.`, - { contentTypeId: definition.id }, - ); - } - byTable.set(definition.tableName, entry); - - // The generated translation table shares one namespace with every base - // table, so a content type called `example_articles_translations` and a - // localized `example_articles` would collide - and the shortening clamp - // makes that reachable with two long names that differ only past character - // 63. Both directions are caught by holding one map. - if (definition.localization.enabled) { - const translationTable = definition.localization.translationTableName; - const duplicateTranslationTable = byTable.get(translationTable); - if (duplicateTranslationTable) { + // Base, translation, junction and child tables all in one map: they share + // one Postgres namespace, so a content type called + // `example_articles_translations` and a localized `example_articles` would + // collide - and so would a repeatable called `tags` on `example_articles` + // and a content type whose own table is `example_articles_tags`. The + // shortening clamp makes every one of those reachable with two long names + // that differ only past character 63, and Postgres truncates silently. + for (const { name, origin } of physicalTables(entry)) { + const owner = byTable.get(name); + if (owner) { throw new ContentEngineError( - `Translation table "${translationTable}" is claimed by both ${describe(duplicateTranslationTable)} and ${describe(entry)}. Rename one of the base tables.`, + `Table "${name}" is claimed by both ${describeTableOwner(owner)} and ${describeTableOwner({ entry, origin })}. Two content types cannot share a physical table - rename one of them, or the field that generates it.`, { contentTypeId: definition.id }, ); } - byTable.set(translationTable, entry); + byTable.set(name, { entry, origin }); } // Permission modules are scoped per plugin, so only a collision inside one @@ -122,23 +220,44 @@ export const validateContentTypes = ( ); } byPublicPath.set(pathKey, entry); + + // Delivery is the exception, and only delivery. Its canonical URLs are + // framework-neutral **site** paths - `/articles/my-article`, + // `/pl/articles/moj-artykul` - built from `publicApi.path` with no plugin id in + // them, so two delivery-enabled content types sharing a path would give + // `/articles/example` two owners: two resolvers claiming one URL, two sitemaps + // listing it, and one slug reservation table with no way to say which of them a + // retired address belonged to. + // + // The fix is the check, not a prefix: adding the plugin id to the URL would + // solve the ambiguity by making every public content URL uglier for everybody. + if (definition.delivery.enabled) { + const duplicateDeliveryPath = byDeliveryPath.get(path); + if (duplicateDeliveryPath) { + throw new ContentEngineError( + `Delivery path "${path}" is claimed by both ${describe(duplicateDeliveryPath)} and ${describe(entry)}. Delivery paths are site-wide public namespaces and must be globally unique - give one of them a different \`publicApi.path\`, or turn \`delivery\` off on one of them.`, + { contentTypeId: definition.id }, + ); + } + byDeliveryPath.set(path, entry); + } } // `resolveContentIndexes` already rejects a collision inside one content - // type. Postgres index names are unique per *schema*, though, so two - // content types - from one plugin or from two - cannot share one either. - for (const index of [ - ...definition.indexes, - ...definition.localization.translationIndexes, - ]) { + // type. Postgres index and constraint names are unique per *schema*, + // though, so two content types - from one plugin or from two - cannot share + // one either. The generated junction and repeatable names are in here too: + // they are derived from a field name and clamped, so they are exactly the + // ones nobody wrote down and nobody would think to check. + for (const index of physicalIndexes(entry)) { const owner = byIndexName.get(index.name); if (owner) { throw new ContentEngineError( - `Index name "${index.name}" is used by both ${describeIndexOwner(owner)} and ${describeIndexOwner({ columns: index.on, entry })}. Postgres index names are unique per schema, so rename one of them.`, + `Index name "${index.name}" is used by both ${describeIndexOwner(owner)} and ${describeIndexOwner({ columns: index.columns, entry })}. Postgres index names are unique per schema, so rename one of them.`, { contentTypeId: definition.id }, ); } - byIndexName.set(index.name, { columns: index.on, entry }); + byIndexName.set(index.name, { columns: index.columns, entry }); } } diff --git a/packages/vitnode/src/content/server/delivery-admin-route.test.ts b/packages/vitnode/src/content/server/delivery-admin-route.test.ts new file mode 100644 index 000000000..b47a659ed --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-admin-route.test.ts @@ -0,0 +1,199 @@ +// @vitest-environment node +import { OpenAPIHono } from "@hono/zod-openapi"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import { + testDeliveredPostContentType, + testEditorialPostContentType, +} from "@/tests/content-fixtures"; + +import { createContentModel } from "./model"; +import { buildContentRoutes } from "./routes"; + +let permissionGranted = true; +let requestedPermission: null | { module: string; permission: string } = null; + +// `assertStaffPermission` reads roles out of the database. What matters here is that +// the route *asks* for `can_view` and nothing narrower, so the check itself is +// replaced with a recorder plus a switchable verdict. +vi.mock("../../api/lib/check-staff-permission", () => ({ + assertStaffPermission: async ( + _c: unknown, + args: { module: string; permission: string }, + ) => { + requestedPermission = { module: args.module, permission: args.permission }; + if (!permissionGranted) { + const { HTTPException } = await import("hono/http-exception"); + throw new HTTPException(403, { message: "Forbidden" }); + } + }, +})); + +const delivered = createContentModel(testDeliveredPostContentType); +const editorialPosts = createContentModel(testEditorialPostContentType); + +const PLUGIN_ID = "@vitnode/example"; + +const harness = () => { + const service = { + alternates: vi.fn(), + findById: vi.fn(), + history: vi.fn().mockResolvedValue([]), + resolvePath: vi.fn(), + resolveSlug: vi.fn(), + sitemap: vi.fn(), + }; + + vi.spyOn(delivered, "deliveryService", "get").mockReturnValue(() => service); + + const app = new OpenAPIHono(); + for (const { handler, route } of buildContentRoutes(delivered, { + pluginId: PLUGIN_ID, + })) { + app.openapi(route, handler); + } + + return { app, service }; +}; + +beforeEach(() => { + permissionGranted = true; + requestedPermission = null; +}); + +describe("route generation", () => { + it("adds the delivery route only for a delivery-enabled content type", () => { + const withDelivery = buildContentRoutes(delivered, { + pluginId: PLUGIN_ID, + }).map(entry => entry.route.path); + const without = buildContentRoutes(editorialPosts, { + pluginId: PLUGIN_ID, + }).map(entry => entry.route.path); + + expect(withDelivery).toContain("/{id}/delivery"); + expect(without).not.toContain("/{id}/delivery"); + }); +}); + +describe("admin delivery route", () => { + it("is gated by can_view rather than a permission of its own", async () => { + const { app } = harness(); + + await app.request("/42/delivery"); + + // Read-only, so the permission that allowed the slug mutation is the only one it + // needs. A `can_manage_redirects` would be a permission every install has to + // configure for no decision this screen can make. + expect(requestedPermission).toStrictEqual({ + module: testDeliveredPostContentType.permissionModule, + permission: "can_view", + }); + }); + + it("refuses a request without the permission", async () => { + permissionGranted = false; + const { app } = harness(); + + expect((await app.request("/42/delivery")).status).toBe(403); + }); + + it("reports the canonical URL and the historical ones", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue({ + canonicalPath: "/delivered-posts/current", + locale: null, + }); + service.history.mockResolvedValue([ + { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + itemId: 42, + languageId: null, + path: "/delivered-posts/current", + retiredAt: null, + slug: "current", + }, + { + createdAt: new Date("2025-12-01T00:00:00.000Z"), + itemId: 42, + languageId: null, + path: "/delivered-posts/old", + retiredAt: new Date("2026-01-01T00:00:00.000Z"), + slug: "old", + }, + ]); + + const response = await app.request("/42/delivery"); + const body = (await response.json()) as { + canonicalPath: string; + history: Record<string, unknown>[]; + isPublic: boolean; + }; + + expect(response.status).toBe(200); + expect(body.canonicalPath).toBe("/delivered-posts/current"); + expect(body.isPublic).toBe(true); + expect(body.history).toHaveLength(2); + }); + + it("exposes no storage columns of the history table", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue({ + canonicalPath: "/delivered-posts/current", + locale: null, + }); + service.history.mockResolvedValue([ + { + createdAt: new Date(0), + itemId: 42, + languageId: 2, + path: "/delivered-posts/old", + retiredAt: new Date(0), + slug: "old", + }, + ]); + + const body = (await (await app.request("/42/delivery")).json()) as { + history: Record<string, unknown>[]; + }; + + // `languageId`, `pluginId` and the row id are details of + // `core_content_slug_history`, not part of this contract. + expect(Object.keys(body.history[0]).sort()).toStrictEqual([ + "createdAt", + "path", + "retiredAt", + "slug", + ]); + }); + + it("reports a draft as having no canonical URL rather than inventing one", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue(null); + + const body = (await (await app.request("/42/delivery")).json()) as { + canonicalPath: null | string; + isPublic: boolean; + }; + + // "This is where it *would* live" is a different claim from "this is where it + // lives", and the panel must not make the first one look like the second. + expect(body.canonicalPath).toBeNull(); + expect(body.isPublic).toBe(false); + }); + + it("scopes the read to one language when asked", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue({ canonicalPath: null, locale: "pl" }); + + await app.request("/42/delivery?locale=pl"); + + expect(service.findById).toHaveBeenCalledWith(42, { locale: "pl" }); + expect(service.history).toHaveBeenCalledWith(42, { locale: "pl" }); + }); + + it("rejects an invalid identifier", async () => { + const { app } = harness(); + + expect((await app.request("/abc/delivery")).status).toBe(400); + }); +}); diff --git a/packages/vitnode/src/content/server/delivery-alternates.ts b/packages/vitnode/src/content/server/delivery-alternates.ts new file mode 100644 index 000000000..4fc2a56b9 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-alternates.ts @@ -0,0 +1,170 @@ +import type { + PgColumn, + PgTable, + PgTableWithColumns, + TableConfig, +} from "drizzle-orm/pg-core"; +import type { Context } from "hono"; + +import { and, asc, eq, inArray } from "drizzle-orm"; + +import type { ContentDeliveryAlternate } from "../delivery"; +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentModel } from "./model"; +import type { ContentDatabase } from "./service"; + +import { contentDeliveryPath } from "../delivery"; +import { listContentLanguages } from "./language-resolver"; +import { + contentTranslationPublicationColumns, + publicationColumns, + publishedCondition, +} from "./publication"; + +/** + * The localized alternates of one record: every language it is genuinely + * published in, and its URL there. + * + * "Genuinely" is the whole of it, and it is why this is a query rather than a + * projection of something the public read already returned. An alternate is a + * promise that a URL resolves, so the predicate is the *same* subordinated + * publication rule the public read applies - the base row published, the + * translation published, both dated now or earlier - and a locale that only exists + * through `fallback: "default"` fails it. Fabricating an alternate from a fallback + * would announce `/de/articles/x` for a record with no German translation: an + * `hreflang` pointing at a 404, and an invitation to index the English copy twice. + * + * A language the installation has switched off is filtered out too, in JavaScript + * rather than in SQL - "enabled" is a fact about the app config, not a column on + * `core_languages`, and `listContentLanguages` already holds it for the life of the + * request. + */ +export const readDeliveryAlternates = async < + TDefinition extends AnyContentTypeDefinition, +>({ + c, + itemId, + model, +}: { + c: Context; + itemId: number; + model: ContentModel<TDefinition>; +}): Promise<ContentDeliveryAlternate[]> => { + const batched = await readDeliveryAlternatesMany({ + c, + itemIds: [itemId], + model, + }); + + return batched.get(itemId) ?? []; +}; + +/** + * The same answer for a whole page of records, in one query. + * + * A sitemap with `xhtml:link` alternates needs the alternates of every URL on the + * page, and a per-record query there is the classic N+1 that only becomes visible + * once a site has content. One `IN` and one grouping pass instead. + */ +export const readDeliveryAlternatesMany = async < + TDefinition extends AnyContentTypeDefinition, +>({ + c, + database, + itemIds, + model, +}: { + c: Context; + database?: ContentDatabase; + itemIds: readonly number[]; + model: ContentModel<TDefinition>; +}): Promise<Map<number, ContentDeliveryAlternate[]>> => { + const { columns, definition, translationColumns, translationTable } = model; + const grouped = new Map<number, ContentDeliveryAlternate[]>(); + + if ( + itemIds.length === 0 || + !definition.localization.enabled || + !definition.publicApi.enabled || + !translationTable || + !translationColumns + ) { + return grouped; + } + + const slugField = definition.publicApi.slugField; + const base = publicationColumns(definition, columns); + const translation = contentTranslationPublicationColumns( + definition, + translationColumns, + ); + + // The slug comes off whichever table owns it. A shared slug gives every language + // the same segment, which is a legitimate shape - the locale prefix is what makes + // the two URLs different - so it is read from the base row for all of them. + const slugColumn: PgColumn = + definition.delivery.slugScope === "localized" + ? translationColumns[slugField] + : columns[slugField]; + + const languages = await listContentLanguages(c); + const byId = new Map(languages.map(language => [language.id, language])); + // Widened, not cast: the generated table type carries every column as a literal, + // which Drizzle's `.from()` and `.innerJoin()` overloads cannot resolve through a + // generic. The same widening `buildContentPublicRoutes` documents. + const baseTable: PgTableWithColumns<TableConfig> = model.table; + + const rows = await (database ?? c.get("db")) + .select({ + itemId: translationColumns.itemId, + languageId: translationColumns.languageId, + slug: slugColumn, + }) + .from(translationTable as PgTable) + .innerJoin(baseTable, eq(translationColumns.itemId, columns.id)) + .where( + and( + inArray(translationColumns.itemId, [...itemIds]), + publishedCondition(base), + publishedCondition(translation), + ), + ) + // Deterministic: two processes rendering the same `hreflang` set - or the same + // sitemap - produce the same document, which is what makes a byte comparison a + // usable test rather than a flake. Sorted again by locale below, because the + // canonical code is resolved in JavaScript. + .orderBy( + asc(translationColumns.itemId), + asc(translationColumns.languageId), + ); + + for (const row of rows) { + // The selected keys come back as `unknown` through the generic column map, so + // each one is narrowed here rather than asserted - the same treatment the + // sitemap query gives its own projection. + const itemId = typeof row.itemId === "number" ? row.itemId : null; + const languageId = + typeof row.languageId === "number" ? row.languageId : null; + if (itemId === null || languageId === null) continue; + + const language = byId.get(languageId); + if (!language?.isEnabled) continue; + + const path = contentDeliveryPath({ + definition, + locale: language.locale, + slug: typeof row.slug === "string" ? row.slug : "", + }); + if (path === null) continue; + + const entries = grouped.get(itemId) ?? []; + entries.push({ locale: language.locale, path }); + grouped.set(itemId, entries); + } + + for (const entries of grouped.values()) { + entries.sort((a, b) => a.locale.localeCompare(b.locale)); + } + + return grouped; +}; diff --git a/packages/vitnode/src/content/server/delivery-effects.test.ts b/packages/vitnode/src/content/server/delivery-effects.test.ts new file mode 100644 index 000000000..3f5fc5987 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-effects.test.ts @@ -0,0 +1,335 @@ +import type { Context } from "hono"; + +import { describe, expect, it } from "vitest"; + +import type { ContentDeliveryOutcome } from "./delivery-writes"; + +import { defineContentType } from "../define"; +import { field } from "../fields"; +import { + contentDeliveryEffects, + contentDeliveryInvalidation, +} from "./delivery-effects"; + +/** + * Which delivery events one mutation emits, and which it deliberately does not. + * + * Both events are gated on a *fact* rather than on an operation: the URL moved, and + * the old address had been live. A listener that warms a CDN or writes an edge + * redirect table acts on the second one, so emitting it for a corrected draft would + * make it act on a URL nobody ever visited. + */ + +const articleType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "effects.article", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "effects_articles", +}); + +const plainType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "effects.plain", + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "effects_plain", +}); + +const outcome = ( + overrides: Partial<ContentDeliveryOutcome> = {}, +): ContentDeliveryOutcome => ({ + canonicalPath: "/articles/new", + itemId: 42, + locale: null, + previousPath: "/articles/old", + previousSlug: "old", + redirectCreated: true, + // A slug change on a published record: the file's bytes moved, the index did not. + sitemap: { contentChanged: true, indexChanged: false }, + slug: "new", + slugChanged: true, + ...overrides, +}); + +/** One dead listener, as `EventsModel.emit` reports it rather than throws it. */ +const DEAD_LISTENER = { + error: "Service unavailable", + listener: "warm-edge-cache", + module: "cdn", + pluginId: "@vitnode/edge", +}; + +const buildContext = ({ failing = false }: { failing?: boolean } = {}) => { + const emitted: { name: string; payload: Record<string, unknown> }[] = []; + const logged: string[] = []; + + const c = { + get: (key: string) => { + if (key === "events") { + return { + emit: async (name: string, payload: Record<string, unknown>) => { + emitted.push({ name, payload }); + + return await Promise.resolve({ + delivered: failing ? 0 : 1, + eventId: `event-${emitted.length}`, + failures: failing ? [DEAD_LISTENER] : [], + status: "delivered", + }); + }, + }; + } + + if (key === "log") { + return { + error: async (message: string) => { + logged.push(message); + + return await Promise.resolve(); + }, + }; + } + + return undefined; + }, + } as unknown as Context; + + return { c, emitted, logged }; +}; + +describe("contentDeliveryEffects", () => { + it("emits both events when a live URL moves", async () => { + const { c, emitted } = buildContext(); + + const result = await contentDeliveryEffects(c, articleType, outcome(), { + pluginId: "@vitnode/test", + }); + + expect(emitted.map(entry => entry.name)).toStrictEqual([ + "content.effects.article.delivery_slug_changed", + "content.effects.article.delivery_redirect_created", + ]); + expect(emitted[0].payload).toStrictEqual({ + canonicalPath: "/articles/new", + contentId: 42, + locale: null, + previousPath: "/articles/old", + previousSlug: "old", + slug: "new", + }); + expect(emitted[1].payload).toStrictEqual({ + canonicalPath: "/articles/new", + contentId: 42, + locale: null, + previousPath: "/articles/old", + previousSlug: "old", + }); + expect(result.events).toHaveLength(2); + }); + + it("emits only the slug event when the old address was never live", async () => { + const { c, emitted } = buildContext(); + + await contentDeliveryEffects( + c, + articleType, + outcome({ redirectCreated: false }), + { pluginId: "@vitnode/test" }, + ); + + expect(emitted.map(entry => entry.name)).toStrictEqual([ + "content.effects.article.delivery_slug_changed", + ]); + }); + + it("emits nothing when no URL moved", async () => { + const { c, emitted } = buildContext(); + + await contentDeliveryEffects( + c, + articleType, + outcome({ + previousPath: null, + previousSlug: null, + redirectCreated: false, + slugChanged: false, + }), + { pluginId: "@vitnode/test" }, + ); + + expect(emitted).toStrictEqual([]); + }); + + it("emits nothing for a mutation that reported no delivery outcome", async () => { + const { c, emitted } = buildContext(); + + await contentDeliveryEffects(c, articleType, undefined, { + pluginId: "@vitnode/test", + }); + + expect(emitted).toStrictEqual([]); + }); + + it("emits nothing when the canonical path cannot be built", async () => { + const { c, emitted } = buildContext(); + + // A slug written straight into the database, or a localized content type with a + // shared slug: no single canonical path, so no delivery fact to announce. + await contentDeliveryEffects( + c, + articleType, + outcome({ canonicalPath: null }), + { pluginId: "@vitnode/test" }, + ); + + expect(emitted).toStrictEqual([]); + }); + + it("carries the locale on a localized move", async () => { + const { c, emitted } = buildContext(); + + await contentDeliveryEffects( + c, + articleType, + outcome({ + canonicalPath: "/pl/articles/nowy", + locale: "pl", + previousPath: "/pl/articles/stary", + previousSlug: "stary", + slug: "nowy", + }), + { pluginId: "@vitnode/test" }, + ); + + expect(emitted[0].payload).toMatchObject({ locale: "pl" }); + }); + + /** + * The same post-commit rule the base and translation effects follow. + * + * `EventsModel.emit` reports rather than throws, so `failures` is the only place + * a dead listener is visible - and a missed `delivery_slug_changed` is the most + * expensive one to miss: the listener that purges a CDN or writes an edge + * redirect table never hears the URL moved, so the old address keeps 404ing at + * the edge while the origin is entirely correct. + */ + describe("reporting a delivery failure", () => { + it("logs the failed listener behind the effects prefix", async () => { + const { c, logged } = buildContext({ failing: true }); + + await contentDeliveryEffects(c, articleType, outcome(), { + pluginId: "@vitnode/test", + }); + + // One line per event, and both events fired for this outcome. + expect(logged).toHaveLength(2); + expect(logged[0]).toContain("[content-effects]"); + expect(logged[0]).toContain("effects.article"); + expect(logged[0]).toContain('"itemId":42'); + expect(logged[0]).toContain("warm-edge-cache"); + expect(logged[0]).toContain("Service unavailable"); + }); + + it("names the delivery action, so it is not read as a failed edit", async () => { + const { c, logged } = buildContext({ failing: true }); + + await contentDeliveryEffects(c, articleType, outcome(), { + pluginId: "@vitnode/test", + }); + + expect(logged[0]).toContain("delivery_slug_changed"); + expect(logged[1]).toContain("delivery_redirect_created"); + }); + + it("carries the locale, so a Polish URL is a distinct incident", async () => { + const { c, logged } = buildContext({ failing: true }); + + await contentDeliveryEffects( + c, + articleType, + outcome({ + canonicalPath: "/pl/articles/nowy", + locale: "pl", + previousPath: "/pl/articles/stary", + previousSlug: "stary", + redirectCreated: false, + slug: "nowy", + }), + { pluginId: "@vitnode/test" }, + ); + + expect(logged).toHaveLength(1); + expect(logged[0]).toContain('"locale":"pl"'); + }); + + it("still returns normally, because the write has already committed", async () => { + const { c, logged } = buildContext({ failing: true }); + + const result = await contentDeliveryEffects(c, articleType, outcome(), { + pluginId: "@vitnode/test", + }); + + // The events are still reported back to the caller, failures and all. + expect(result.events).toHaveLength(2); + expect(logged).toHaveLength(2); + }); + + it("writes nothing when every listener heard it", async () => { + const { c, logged } = buildContext(); + + await contentDeliveryEffects(c, articleType, outcome(), { + pluginId: "@vitnode/test", + }); + + // An expected success is not an error, and a log full of them is a log + // nobody reads. + expect(logged).toStrictEqual([]); + }); + }); +}); + +describe("contentDeliveryInvalidation", () => { + it("is undefined for a content type without delivery", () => { + expect(contentDeliveryInvalidation(plainType, outcome())).toBeUndefined(); + }); + + it("passes the sitemap change through unchanged", () => { + expect(contentDeliveryInvalidation(articleType, outcome())).toStrictEqual({ + sitemap: { contentChanged: true, indexChanged: false }, + }); + expect( + contentDeliveryInvalidation( + articleType, + outcome({ sitemap: { contentChanged: true, indexChanged: true } }), + ), + ).toStrictEqual({ sitemap: { contentChanged: true, indexChanged: true } }); + }); + + it("expires no sitemap for a mutation that reported no delivery outcome", () => { + // The delivery metadata tag still goes out - a shared SEO field moving changes + // what every locale's `<head>` renders - but a mutation that touched no + // slug-bearing path has nothing to say about the sitemap. + expect(contentDeliveryInvalidation(articleType, undefined)).toStrictEqual({ + sitemap: { contentChanged: false, indexChanged: false }, + }); + }); +}); diff --git a/packages/vitnode/src/content/server/delivery-effects.ts b/packages/vitnode/src/content/server/delivery-effects.ts new file mode 100644 index 000000000..0736041f8 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-effects.ts @@ -0,0 +1,165 @@ +import type { Context } from "hono"; + +import type { EventEmitResult } from "../../api/models/events"; +import type { ContentDeliveryInvalidation } from "../cache"; +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliveryOutcome } from "./delivery-writes"; + +import { reportContentEventFailures } from "./effects-log"; +import { emitContentEvent } from "./emit"; + +export interface ContentDeliveryEffectsResult { + /** + * What the event transport reported for each delivery event, in the order they + * were emitted. Empty when the mutation moved no URL. + * + * Present rather than discarded for the same reason the editorial effects keep + * theirs: `EventsModel.emit` does not throw, so `failures` is the only place a + * dead listener or a broker outage is visible. + */ + events: EventEmitResult[]; +} + +/** + * The delivery events one mutation owes the rest of the system, after it commits. + * + * Two events at most, and each one is gated on a fact rather than on an operation: + * + * - **`delivery_slug_changed`** whenever the canonical URL is different from what + * it was. Emitted *alongside* `updated` or `restored`, never instead of one: the + * field mutation and the URL change are different facts with different audiences, + * and a listener that warms a CDN or writes an edge redirect table would + * otherwise have to inspect `changedFields` for a slug field whose name it cannot + * know. + * - **`delivery_redirect_created`** only when the old address had genuinely been + * live. A draft whose slug was corrected three times before it was ever published + * emits nothing at all, which is the difference between "a URL now needs a + * redirect" and "somebody edited a field". + * + * There is deliberately no sitemap event. Every mutation that changes a sitemap + * line already emits `published`, `unpublished`, `deleted` or one of the two above, + * and a fifth event carrying no new information would be one more thing to keep + * consistent for no listener's benefit. + * + * **Call it only after the write has returned - never inside the transaction.** A + * rollback cannot un-emit an event. + */ +export const contentDeliveryEffects = async ( + c: Context, + definition: AnyContentTypeDefinition, + delivery: ContentDeliveryOutcome | undefined, + { pluginId }: { pluginId: string }, +): Promise<ContentDeliveryEffectsResult> => { + const events: EventEmitResult[] = []; + + /** + * Emits one delivery event and reports whoever did not hear it. + * + * The reporting is not optional decoration. `EventsModel.emit` reports rather + * than throws, so `failures` is the only place a dead listener is visible - and + * these events are the ones with the most expensive silent failure in the engine: + * a listener that writes an edge redirect table or purges a CDN missing a + * `delivery_slug_changed` leaves a moved URL 404ing at the edge while the origin + * is perfectly correct. The base and translation effects log their own event for + * exactly this reason, and a delivery event that skipped the log would be the one + * announcement nobody could find afterwards. + * + * The write has already committed, so this never fails the request - see + * `reportContentEventFailures`. + */ + const announce = async ( + action: "delivery_redirect_created" | "delivery_slug_changed", + payload: Record<string, unknown>, + { itemId, locale }: { itemId: number; locale: null | string }, + ): Promise<void> => { + const event = await emitContentEvent( + c, + definition, + action, + payload as never, + { pluginId }, + ); + + events.push(event); + await reportContentEventFailures(c, { + action, + contentTypeId: definition.id, + event, + itemId, + // Present only for a localized URL: "nobody heard the Polish article moved" + // is a different incident from "nobody heard the article moved". + ...(locale === null ? {} : { locale }), + }); + }; + + // A canonical path this engine could not build is a URL nobody can visit, so + // there is no delivery fact to announce. It happens for a slug written straight + // into the database, and for a localized content type whose slug is shared - which + // has one segment and several URLs, so no single canonical path. + if ( + delivery === undefined || + !delivery.slugChanged || + delivery.canonicalPath === null + ) { + return { events }; + } + + await announce( + "delivery_slug_changed", + { + canonicalPath: delivery.canonicalPath, + contentId: delivery.itemId, + locale: delivery.locale, + previousPath: delivery.previousPath, + previousSlug: delivery.previousSlug, + slug: delivery.slug, + }, + { itemId: delivery.itemId, locale: delivery.locale }, + ); + + if ( + delivery.redirectCreated && + delivery.previousPath !== null && + delivery.previousSlug !== null + ) { + await announce( + "delivery_redirect_created", + { + canonicalPath: delivery.canonicalPath, + contentId: delivery.itemId, + locale: delivery.locale, + previousPath: delivery.previousPath, + previousSlug: delivery.previousSlug, + }, + { itemId: delivery.itemId, locale: delivery.locale }, + ); + } + + return { events }; +}; + +/** + * The delivery half of a mutation's cache invalidation, or `undefined`. + * + * `undefined` for a content type without `delivery`, which is what makes + * `contentInvalidationTags` return exactly the strings it always returned - and + * therefore what makes Stage 8 opt-in at the cache layer as well as everywhere + * else. + */ +export const contentDeliveryInvalidation = ( + definition: AnyContentTypeDefinition, + delivery: ContentDeliveryOutcome | undefined, +): ContentDeliveryInvalidation | undefined => { + if (!definition.delivery.enabled) return undefined; + + // A content type with delivery whose mutation reported nothing still expires its + // delivery metadata - a shared SEO field moving changes what every locale's + // `<head>` renders even though no URL moved. Only the sitemap is conditional, and + // an absent outcome means the mutation touched no slug-bearing path at all. + return { + sitemap: delivery?.sitemap ?? { + contentChanged: false, + indexChanged: false, + }, + }; +}; diff --git a/packages/vitnode/src/content/server/delivery-routes.test.ts b/packages/vitnode/src/content/server/delivery-routes.test.ts new file mode 100644 index 000000000..a49ba15c3 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-routes.test.ts @@ -0,0 +1,261 @@ +// @vitest-environment node +import { OpenAPIHono } from "@hono/zod-openapi"; +import { describe, expect, it, vi } from "vitest"; + +import { + testDeliveredPostContentType, + testPostContentType, +} from "@/tests/content-fixtures"; + +import { createContentModel } from "./model"; +import { buildContentPublicRoutes } from "./public-routes"; + +/** + * The generated public delivery routes. + * + * Two things are being asserted, and only one of them is about delivery: + * + * 1. The routes answer without any session at all, and their bodies match the + * schemas the OpenAPI document publishes - including the discriminated union, + * whose whole purpose is that a client can branch on `type` rather than guess. + * 2. A content type **without** `delivery` gains no routes whatsoever. That is the + * Stage 1-7 regression assertion at the routing layer: the path list of an + * existing public content type does not move. + */ + +const delivered = createContentModel(testDeliveredPostContentType); + +const PLUGIN_ID = "@vitnode/example"; + +const metadata = { + alternates: [], + canonicalPath: "/delivered-posts/hello-world", + hreflang: { languages: {} }, + isFallback: false, + itemId: 42, + locale: null, + openGraph: { description: "Prose", title: "Hello world" }, + requestedLocale: null, + robots: { follow: true, index: true }, + seo: { description: "Prose", title: "Hello world" }, +}; + +const harness = () => { + const service = { + alternates: vi.fn(), + findById: vi.fn(), + history: vi.fn(), + resolvePath: vi.fn(), + resolveSlug: vi.fn(), + sitemap: vi.fn(), + }; + + vi.spyOn(delivered, "deliveryService", "get").mockReturnValue(() => service); + // The public service is never reached by a delivery route - the delivery service + // is - but the route builder still asks the model for it. + vi.spyOn(delivered, "publicService", "get").mockReturnValue(() => ({ + findById: vi.fn(), + findBySlug: vi.fn(), + findMany: vi.fn(), + })); + + const app = new OpenAPIHono(); + for (const { handler, route } of buildContentPublicRoutes(delivered, { + pluginId: PLUGIN_ID, + })) { + app.openapi(route, handler); + } + + return { app, service }; +}; + +describe("route generation", () => { + it("adds three delivery routes under a static `delivery` segment", () => { + const paths = buildContentPublicRoutes(delivered, { + pluginId: PLUGIN_ID, + }).map(entry => entry.route.path); + + expect(paths).toContain("/delivery/resolve/{slug}"); + expect(paths).toContain("/delivery/item/{id}"); + expect(paths).toContain("/delivery/sitemap"); + }); + + it("adds none at all to a content type without delivery", () => { + const posts = createContentModel(testPostContentType, { + references: { category: () => delivered.table.id }, + }); + + const paths = buildContentPublicRoutes(posts, { pluginId: PLUGIN_ID }).map( + entry => entry.route.path, + ); + + // The Stage 1-7 path list, unchanged. + expect(paths).toStrictEqual(["/", "/{slug}"]); + }); + + it("cannot be shadowed by a record whose slug is `delivery`", () => { + // `/{slug}` is one segment and every delivery path is two or three, so the two + // can never both match whatever order they are registered in. + const paths = buildContentPublicRoutes(delivered, { + pluginId: PLUGIN_ID, + }).map(entry => entry.route.path); + + expect(paths.filter(path => path === "/{slug}")).toHaveLength(1); + expect(paths.every(path => path.split("/").length <= 4)).toBe(true); + }); +}); + +describe("resolve route", () => { + it("answers without any session at all", async () => { + const { app, service } = harness(); + service.resolveSlug.mockResolvedValue({ ...metadata, type: "content" }); + + const response = await app.request("/delivery/resolve/hello-world"); + + expect(response.status).toBe(200); + }); + + it("returns the canonical arm for a current slug", async () => { + const { app, service } = harness(); + service.resolveSlug.mockResolvedValue({ ...metadata, type: "content" }); + + const response = await app.request("/delivery/resolve/hello-world"); + + expect(await response.json()).toMatchObject({ + canonicalPath: "/delivered-posts/hello-world", + itemId: 42, + type: "content", + }); + }); + + it("returns the redirect arm with its status in the body", async () => { + const { app, service } = harness(); + service.resolveSlug.mockResolvedValue({ + location: "/delivered-posts/new", + status: 308, + type: "redirect", + }); + + const response = await app.request("/delivery/resolve/old"); + + // A 200 carrying a redirect, not an HTTP redirect: the *frontend* issues the + // 308, because it owns the URL the reader is on. + expect(response.status).toBe(200); + expect(await response.json()).toStrictEqual({ + location: "/delivered-posts/new", + status: 308, + type: "redirect", + }); + }); + + it("returns the not_found arm as a 200 with a body", async () => { + const { app, service } = harness(); + service.resolveSlug.mockResolvedValue({ type: "not_found" }); + + const response = await app.request("/delivery/resolve/nope"); + + // A 200, so a caller distinguishes "this URL resolves to nothing" from "the + // delivery API is unreachable" - and so a negative is not cached as a 404. + expect(response.status).toBe(200); + expect(await response.json()).toStrictEqual({ type: "not_found" }); + }); + + it("exposes no internal storage fields", async () => { + const { app, service } = harness(); + service.resolveSlug.mockResolvedValue({ ...metadata, type: "content" }); + + const body = (await ( + await app.request("/delivery/resolve/hello-world") + ).json()) as Record<string, unknown>; + + for (const internal of ["languageId", "pluginId", "retiredAt"]) { + expect(body).not.toHaveProperty(internal); + } + }); +}); + +describe("item route", () => { + it("returns the delivery metadata of one record", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue(metadata); + + const response = await app.request("/delivery/item/42"); + + expect(response.status).toBe(200); + expect(await response.json()).toMatchObject({ + canonicalPath: "/delivered-posts/hello-world", + seo: { description: "Prose", title: "Hello world" }, + }); + expect(service.findById).toHaveBeenCalledWith(42, { locale: undefined }); + }); + + it("is a 404 for a record with no public version", async () => { + const { app, service } = harness(); + service.findById.mockResolvedValue(null); + + expect((await app.request("/delivery/item/42")).status).toBe(404); + }); + + it("rejects a non-numeric identifier at validation time", async () => { + const { app } = harness(); + + expect((await app.request("/delivery/item/abc")).status).toBe(400); + }); +}); + +describe("sitemap route", () => { + it("serializes lastModified as an ISO string, matching its schema", async () => { + const { app, service } = harness(); + service.sitemap.mockResolvedValue({ + entries: [ + { + changeFrequency: "weekly", + itemId: 42, + lastModified: new Date("2026-01-02T03:04:05.000Z"), + locale: null, + path: "/delivered-posts/hello-world", + priority: 0.7, + }, + ], + nextCursor: null, + }); + + const response = await app.request("/delivery/sitemap"); + + expect(response.status).toBe(200); + expect(await response.json()).toStrictEqual({ + entries: [ + { + changeFrequency: "weekly", + itemId: 42, + lastModified: "2026-01-02T03:04:05.000Z", + locale: null, + path: "/delivered-posts/hello-world", + priority: 0.7, + }, + ], + nextCursor: null, + }); + }); + + it("passes the cursor and limit through", async () => { + const { app, service } = harness(); + service.sitemap.mockResolvedValue({ entries: [], nextCursor: null }); + + await app.request("/delivery/sitemap?cursor=99&limit=10"); + + expect(service.sitemap).toHaveBeenCalledWith({ + cursor: 99, + limit: 10, + locale: undefined, + }); + }); + + it("rejects a limit above the protocol ceiling", async () => { + const { app } = harness(); + + expect((await app.request("/delivery/sitemap?limit=50001")).status).toBe( + 400, + ); + }); +}); diff --git a/packages/vitnode/src/content/server/delivery-routes.ts b/packages/vitnode/src/content/server/delivery-routes.ts new file mode 100644 index 000000000..3dfaa8b2b --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-routes.ts @@ -0,0 +1,286 @@ +import type { Context } from "hono"; + +import { z } from "@hono/zod-openapi"; +import { HTTPException } from "hono/http-exception"; + +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliveryService } from "./delivery-service"; +import type { ContentModel } from "./model"; + +import { buildRoute } from "../../api/lib/route"; +import { + CONTENT_DELIVERY_REDIRECT_STATUS, + CONTENT_LOCALE_MAX_LENGTH, + CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + CONTENT_SITEMAP_MAX_URLS, +} from "../const"; +import { ContentDeliveryNotEnabled } from "../errors"; +import { resolveContentPublicLocale } from "../locale"; +import { listContentLanguages } from "./language-resolver"; + +/** + * The public delivery routes one content type with `delivery` gets. + * + * ```http + * GET /api/{pluginId}/content/{publicApi.path}/delivery/resolve/{slug} + * GET /api/{pluginId}/content/{publicApi.path}/delivery/item/{id} + * GET /api/{pluginId}/content/{publicApi.path}/delivery/sitemap (delivery.sitemap) + * ``` + * + * They exist because a frontend is very often **not** the process that holds the + * database: VitNode's split deployment runs Next.js against a separate API, so + * `generateMetadata`, a catch-all route and a `sitemap.xml` handler all need an + * HTTP answer rather than a service call. A single-process install can still use + * `model.deliveryService(c)` directly and never touch these. + * + * Every path begins with the static `delivery` segment, which is what makes them + * impossible to shadow: `/{slug}` is one segment and these are two or three, so a + * record whose slug happens to be `delivery` or `sitemap` still resolves the + * ordinary way, whatever order the routes are registered in. + * + * No `adminStaffPermission` and no `/admin/` anywhere in the path - public delivery + * resolution is exactly as public as the content it describes, and requiring a + * session to learn a canonical URL would be requiring one to render a page. + */ +export const buildContentDeliveryRoutes = < + TDefinition extends AnyContentTypeDefinition, + P extends string, +>( + model: ContentModel<TDefinition>, + { pluginId }: { pluginId: P }, +) => { + const { definition } = model; + const label = definition.admin.label; + const localized = definition.localization.enabled; + + const service = (c: Context): ContentDeliveryService => { + const build = model.deliveryService; + if (!build) + throw new ContentDeliveryNotEnabled({ contentTypeId: definition.id }); + + return build(c, { pluginId }); + }; + + const localeQuery = localized + ? { + // Loose on purpose, like `publicParams.slug`: an unknown locale and a + // malformed one are both the same "nothing here", so a stricter pattern + // would only turn one of them into a differently-shaped 400. + locale: z.string().min(1).max(CONTENT_LOCALE_MAX_LENGTH).optional(), + } + : {}; + + /** + * Which language this request is for. + * + * The same resolution the public read routes use, for the same reason: an + * explicit `?locale=` that names no language this install serves is a request for + * something that does not exist, and substituting the default would announce an + * English canonical URL under a Polish one. + */ + const localeFor = async (c: Context) => { + if (!localized) return { locale: undefined, source: "default" as const }; + + const languages = await listContentLanguages(c); + + return resolveContentPublicLocale({ + acceptLanguage: c.req.header("accept-language"), + available: languages + .filter(language => language.isEnabled) + .map(language => language.locale), + defaultLocale: definition.localization.defaultLocale, + explicit: c.req.query("locale"), + }); + }; + + const zodAlternate = z.object({ + locale: z.string(), + path: z.string(), + }); + + const zodSeo = z.object({ + description: z.string().nullable(), + title: z.string().nullable(), + }); + + const zodMetadata = z.object({ + alternates: z.array(zodAlternate), + canonicalPath: z.string().nullable(), + hreflang: z.object({ + languages: z.record(z.string(), z.string()), + xDefault: z.string().optional(), + }), + isFallback: z.boolean(), + // Nullable: delivery metadata is read off the public projection, so a content + // type whose allowlist withholds `id` reports none rather than inventing one. + itemId: z.number().int().nullable(), + locale: z.string().nullable(), + openGraph: zodSeo.nullable(), + requestedLocale: z.string().nullable(), + robots: z.object({ follow: z.boolean(), index: z.boolean() }).nullable(), + seo: zodSeo, + }); + + /** + * The resolution, as a discriminated union. + * + * Three arms rather than a nullable object with an optional `location`, because + * the three outcomes need three different HTTP responses and a client that had to + * infer which one it was holding would eventually redirect to `undefined`. + * + * Nothing internal is in it: no `languageId`, no `pluginId`, no `retiredAt`. Those + * are storage details of `core_content_slug_history`, and a public contract that + * carried them would be a public contract that could not change. + */ + const zodResolution = z.discriminatedUnion("type", [ + zodMetadata.extend({ type: z.literal("content") }), + z.object({ + location: z.string(), + status: z.literal(CONTENT_DELIVERY_REDIRECT_STATUS), + type: z.literal("redirect"), + }), + z.object({ type: z.literal("not_found") }), + ]); + + const zodSitemapEntry = z.object({ + changeFrequency: z.string().nullable(), + itemId: z.number().int(), + lastModified: z.string(), + locale: z.string().nullable(), + path: z.string(), + priority: z.number().nullable(), + }); + + const sitemapQuery = z.object({ + ...localeQuery, + cursor: z.coerce.number().int().positive().optional(), + limit: z.coerce + .number() + .int() + .min(1) + .max(CONTENT_SITEMAP_MAX_URLS) + .optional(), + }); + + const resolve = buildRoute({ + pluginId, + route: { + method: "get", + path: "/delivery/resolve/{slug}", + description: `Resolve one ${label.singular} URL to its canonical form, a redirect, or nothing`, + request: { + params: z.object({ slug: z.string() }), + ...(localized ? { query: z.object(localeQuery) } : {}), + }, + responses: { + 200: { + content: { "application/json": { schema: zodResolution } }, + description: + "The resolution. A `not_found` is a 200 with a body, not a 404", + }, + }, + }, + handler: async c => { + const resolved = await localeFor(c); + // An explicit locale naming no language this install serves resolves to + // nothing rather than to the default - the same rule the public detail route + // follows, and the reason a Polish URL is never answered with English. + if (!resolved) return c.json({ type: "not_found" as const }, 200); + + const resolution = await service(c).resolveSlug(c.req.param("slug"), { + locale: resolved.locale, + }); + + return c.json(resolution, 200); + }, + }); + + const item = buildRoute({ + pluginId, + route: { + method: "get", + path: "/delivery/item/{id}", + description: `Delivery metadata for one published ${label.singular}`, + request: { + params: z.object({ id: z.coerce.number().int().positive() }), + ...(localized ? { query: z.object(localeQuery) } : {}), + }, + responses: { + 200: { + content: { "application/json": { schema: zodMetadata } }, + description: `Canonical URL, alternates and SEO metadata`, + }, + 404: { description: `${label.singular} not found` }, + }, + }, + handler: async c => { + const resolved = await localeFor(c); + if (!resolved) throw notFound(label.singular); + + const id = Number(c.req.param("id")); + const metadata = await service(c).findById(id, { + locale: resolved.locale, + }); + if (!metadata) throw notFound(label.singular); + + return c.json(metadata, 200); + }, + }); + + const sitemap = buildRoute({ + pluginId, + route: { + method: "get", + path: "/delivery/sitemap", + description: `One page of the ${label.plural} sitemap`, + request: { query: sitemapQuery }, + responses: { + 200: { + content: { + "application/json": { + schema: z.object({ + entries: z.array(zodSitemapEntry), + nextCursor: z.number().int().nullable(), + }), + }, + }, + description: `Up to ${CONTENT_SITEMAP_MAX_URLS} public URLs, oldest record first`, + }, + 400: { description: "Invalid query parameters" }, + }, + }, + handler: async c => { + const { cursor, limit } = sitemapQuery.parse(c.req.query()); + const resolved = await localeFor(c); + if (!resolved) return c.json({ entries: [], nextCursor: null }, 200); + + const page = await service(c).sitemap({ + cursor, + limit: limit ?? CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + locale: resolved.locale, + }); + + return c.json( + { + // ISO strings rather than `Date`s, because this crosses a wire: the + // OpenAPI schema says `string` and the runtime has to agree with it. + entries: page.entries.map(entry => ({ + ...entry, + lastModified: entry.lastModified.toISOString(), + })), + nextCursor: page.nextCursor, + }, + 200, + ); + }, + }); + + return [ + resolve, + item, + ...(definition.delivery.sitemap.enabled ? [sitemap] : []), + ]; +}; + +const notFound = (singular: string): HTTPException => + new HTTPException(404, { message: `${singular} not found.` }); diff --git a/packages/vitnode/src/content/server/delivery-service.test.ts b/packages/vitnode/src/content/server/delivery-service.test.ts new file mode 100644 index 000000000..9dfa0210a --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-service.test.ts @@ -0,0 +1,660 @@ +import type { Context } from "hono"; + +import { describe, expect, it } from "vitest"; + +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentModel } from "./model"; + +import { core_content_slug_history } from "../../database/content"; +import { core_languages } from "../../database/languages"; +import { defineContentType } from "../define"; +import { field } from "../fields"; +import { createContentDeliveryService } from "./delivery-service"; + +/** + * The delivery resolver, against the real service, without a database. + * + * The two reads it performs - the public projection and the slug-history lookup - + * are stubbed, and nothing else is: `createContentDeliveryService` is the code under + * test, so the decision it makes (canonical, redirect, or nothing) is the thing + * being asserted rather than a copy of it. That decision is where a mistake becomes + * a permanent 308 to the wrong page, which is exactly why it is worth testing + * without the ceremony of a database. + * + * The queries themselves are covered by the Postgres suite in `plugins/example`. + */ + +const PLUGIN = "@vitnode/test"; + +const articleType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "delivery.article", + editorial: { enabled: true }, + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { descriptionField: "excerpt", titleField: "title" }, + sitemap: { enabled: true, priority: 0.7 }, + }, + fields: { + excerpt: field.textarea({ nullable: true }), + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug", "excerpt"], + path: "articles", + }, + tableName: "delivery_articles", +}); + +const withoutRedirects = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "delivery.no-redirects", + delivery: { enabled: true, sitemap: { enabled: true } }, + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "delivery_no_redirects", +}); + +const localizedType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "delivery.localized", + editorial: { enabled: true }, + delivery: { + enabled: true, + hreflang: { xDefault: "defaultLocale" }, + redirects: { enabled: true }, + seo: { fallbackTitleField: "title", titleField: "seo.title" }, + sitemap: { enabled: true }, + }, + fields: { + seo: field.group({ + fields: { title: field.text({ nullable: true }) }, + localized: true, + nullable: true, + }), + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true, fallback: "default" }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug", "seo.title"], + path: "articles", + }, + tableName: "delivery_localized_articles", +}); + +/** One retired or current address, as `core_content_slug_history` stores it. */ +interface HistoryRow { + itemId: number; + languageId: null | number; + path: string; + retiredAt: Date | null; + slug: string; +} + +/** One public row, and the language it is in. */ +interface PublicRow { + locale?: string; + values: Record<string, unknown>; +} + +type QueryRows = Record<string, unknown>[]; + +/** + * A Drizzle query builder that resolves to whatever the table asks for. + * + * A thenable rather than a promise-returning `limit()`, because the two reads this + * file needs end differently: the language registry awaits straight off `.from()` + * and the history lookup chains `.where().limit(1)` (and sometimes `.for("update")`). + * One thenable satisfies both without the stub having to know which. + */ +const buildDatabase = (rowsFor: (table: unknown) => QueryRows): unknown => { + const select = () => { + let table: unknown; + + const builder = { + for: () => builder, + from: (value: unknown) => { + table = value; + + return builder; + }, + limit: () => builder, + orderBy: () => builder, + then: async ( + resolve: (rows: QueryRows) => unknown, + reject?: (reason: unknown) => unknown, + ) => Promise.resolve(rowsFor(table)).then(resolve, reject), + where: () => builder, + }; + + return builder; + }; + + return { select }; +}; + +/** + * A model whose public service is a map and whose history table is an array. + * + * `findById` mimics the Stage 5 fallback rule rather than re-deriving it: a locale + * with no row of its own is served the default one, and the row says which language + * it is actually in. That is the contract `createContentLocalizedPublicService` + * holds, and reading through it is the whole reason delivery inherits the + * publication predicate and the field allowlist for free. + */ +const buildService = ({ + byId = {}, + bySlug = {}, + definition, + history = [], + languages = [ + { code: "en", id: 1 }, + { code: "pl", id: 2 }, + ], +}: { + byId?: Record<number, PublicRow[]>; + bySlug?: Record<string, { itemId: number; locale?: string }>; + definition: AnyContentTypeDefinition; + history?: HistoryRow[]; + languages?: { code: string; id: number }[]; +}) => { + const localized = definition.localization.enabled; + const defaultLocale = definition.localization.defaultLocale; + + const rowFor = ( + itemId: number, + locale: string | undefined, + ): null | Record<string, unknown> => { + const rows = byId[itemId] ?? []; + if (!localized) return rows[0]?.values ?? null; + + const wanted = (locale ?? defaultLocale).toLowerCase(); + const exact = rows.find(entry => entry.locale === wanted); + if (exact) return { ...exact.values, locale: exact.locale }; + + if (definition.localization.fallback !== "default") return null; + + const fallback = rows.find(entry => entry.locale === defaultLocale); + + return fallback ? { ...fallback.values, locale: fallback.locale } : null; + }; + + const publicService = { + findById: async (id: number, options?: { locale?: string }) => + await Promise.resolve(rowFor(id, options?.locale)), + findBySlug: async (slug: string, options?: { locale?: string }) => { + const hit = bySlug[slug]; + if (!hit) return await Promise.resolve(null); + // Strict-locale, exactly as the real service is: a URL belongs to the + // language it was published under. + if ( + localized && + hit.locale !== (options?.locale ?? defaultLocale).toLowerCase() + ) { + return await Promise.resolve(null); + } + + return await Promise.resolve(rowFor(hit.itemId, hit.locale)); + }, + findMany: async () => + await Promise.resolve({ edges: [], pageInfo: {} as never }), + }; + + const database = buildDatabase(table => { + if (table === core_languages) { + return languages.map(language => ({ + code: language.code, + id: language.id, + isDefault: language.code === defaultLocale, + })); + } + + if (table === core_content_slug_history) { + // The resolver asks for one address at a time, so the stub returns the whole + // set and relies on the service having narrowed it - which it cannot here. + // Each test therefore supplies at most one row. + return history.map(row => ({ createdAt: new Date(0), ...row })); + } + + return []; + }); + + const c = { + get: (key: string) => { + if (key === "db") return database; + if (key === "core") return { i18n: { locales: [] } }; + + return undefined; + }, + } as unknown as Context; + + const model = { + columns: {}, + definition, + publicService: () => publicService, + table: {}, + translationColumns: null, + translationTable: null, + } as unknown as ContentModel<AnyContentTypeDefinition>; + + return createContentDeliveryService({ c, model, pluginId: PLUGIN }); +}; + +const article = (slug: string, id = 42): PublicRow => ({ + values: { excerpt: null, id, slug, title: "T" }, +}); + +const translation = (locale: string, slug: string, id = 7): PublicRow => ({ + locale, + values: { id, seo: { title: null }, slug, title: "T" }, +}); + +describe("createContentDeliveryService", () => { + it("refuses a content type with no delivery block", () => { + const plain = defineContentType({ + admin: { label: { plural: "P", singular: "P" } }, + id: "delivery.none", + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { enabled: true, fields: ["title", "slug"], path: "p" }, + tableName: "delivery_none", + }); + + expect(() => buildService({ definition: plain })).toThrow( + /no `delivery` block/, + ); + }); +}); + +describe("resolveSlug", () => { + it("answers the current slug as canonical content", async () => { + const service = buildService({ + byId: { 42: [article("current")] }, + bySlug: { current: { itemId: 42 } }, + definition: articleType, + }); + + expect(await service.resolveSlug("current")).toMatchObject({ + canonicalPath: "/articles/current", + itemId: 42, + type: "content", + }); + }); + + it("redirects a retired slug to the current canonical path", async () => { + const service = buildService({ + byId: { 42: [article("current")] }, + bySlug: { current: { itemId: 42 } }, + definition: articleType, + history: [ + { + itemId: 42, + languageId: null, + path: "/articles/old", + retiredAt: new Date(), + slug: "old", + }, + ], + }); + + expect(await service.resolveSlug("old")).toStrictEqual({ + location: "/articles/current", + status: 308, + type: "redirect", + }); + }); + + it("collapses a chain: both a and b resolve straight to c", async () => { + for (const retired of ["a", "b"]) { + const service = buildService({ + byId: { 42: [article("c")] }, + bySlug: { c: { itemId: 42 } }, + definition: articleType, + history: [ + { + itemId: 42, + languageId: null, + path: `/articles/${retired}`, + retiredAt: new Date(), + slug: retired, + }, + ], + }); + + // One hop, not two: the resolver reads the record's *current* slug rather + // than the next entry in the chain. + expect(await service.resolveSlug(retired)).toStrictEqual({ + location: "/articles/c", + status: 308, + type: "redirect", + }); + } + }); + + it("is not_found when the destination is no longer public", async () => { + const service = buildService({ + // No public row: an unpublished or deleted record looks like this from here. + byId: {}, + bySlug: {}, + definition: articleType, + history: [ + { + itemId: 42, + languageId: null, + path: "/articles/old", + retiredAt: new Date(), + slug: "old", + }, + ], + }); + + expect(await service.resolveSlug("old")).toStrictEqual({ + type: "not_found", + }); + }); + + it("is not_found for a slug nothing has ever used", async () => { + const service = buildService({ definition: articleType }); + + expect(await service.resolveSlug("never-existed")).toStrictEqual({ + type: "not_found", + }); + }); + + it("never redirects a slug to itself", async () => { + const service = buildService({ + byId: { 42: [article("same")] }, + // The live lookup misses - a stale reservation - and the destination equals + // the address asked for. A redirect loop is worse than a 404. + bySlug: {}, + definition: articleType, + history: [ + { + itemId: 42, + languageId: null, + path: "/articles/same", + retiredAt: null, + slug: "same", + }, + ], + }); + + expect(await service.resolveSlug("same")).toStrictEqual({ + type: "not_found", + }); + }); + + it("never reads the history without redirects", async () => { + const service = buildService({ + byId: { 42: [{ values: { id: 42, slug: "current", title: "T" } }] }, + bySlug: {}, + definition: withoutRedirects, + history: [ + { + itemId: 42, + languageId: null, + path: "/articles/old", + retiredAt: new Date(), + slug: "old", + }, + ], + }); + + expect(await service.resolveSlug("old")).toStrictEqual({ + type: "not_found", + }); + }); +}); + +describe("localized resolveSlug", () => { + it("keeps a locale's redirect inside its own language", async () => { + const service = buildService({ + byId: { 7: [translation("en", "hello-world")] }, + bySlug: {}, + definition: localizedType, + history: [ + { + itemId: 7, + languageId: 1, + path: "/en/articles/hello", + retiredAt: new Date(), + slug: "hello", + }, + ], + }); + + expect(await service.resolveSlug("hello", { locale: "en" })).toStrictEqual({ + location: "/en/articles/hello-world", + status: 308, + type: "redirect", + }); + }); + + it("refuses to point one locale's URL at another language's page", async () => { + const service = buildService({ + // Published in English only. A Polish historical URL must not 308 to the + // English page: that is the wrong language under a URL that says otherwise, + // declared permanent. + byId: { 7: [translation("en", "hello")] }, + bySlug: {}, + definition: localizedType, + history: [ + { + itemId: 7, + languageId: 2, + path: "/pl/articles/witaj", + retiredAt: new Date(), + slug: "witaj", + }, + ], + }); + + expect(await service.resolveSlug("witaj", { locale: "pl" })).toStrictEqual({ + type: "not_found", + }); + }); + + it("resolves a slug strictly, never through the fallback", async () => { + const service = buildService({ + byId: { 7: [translation("en", "hello")] }, + bySlug: { hello: { itemId: 7, locale: "en" } }, + definition: localizedType, + }); + + // `/pl/articles/hello` is not the English article, even though the content type + // falls back to English for a *read*. + expect(await service.resolveSlug("hello", { locale: "pl" })).toStrictEqual({ + type: "not_found", + }); + expect(await service.resolveSlug("hello", { locale: "en" })).toMatchObject({ + canonicalPath: "/en/articles/hello", + type: "content", + }); + }); +}); + +describe("findById", () => { + it("reports the served locale, not the requested one, on a fallback", async () => { + const service = buildService({ + byId: { 7: [translation("en", "hello")] }, + definition: localizedType, + }); + + const metadata = await service.findById(7, { locale: "pl" }); + + // `/pl/articles/hello` would be a self-declared canonical that answers 404. + expect(metadata).toMatchObject({ + canonicalPath: "/en/articles/hello", + isFallback: true, + locale: "en", + requestedLocale: "pl", + }); + }); + + it("is not a fallback when the locale differs only in casing", async () => { + const service = buildService({ + byId: { 7: [translation("en", "hello")] }, + definition: localizedType, + }); + + expect(await service.findById(7, { locale: "EN" })).toMatchObject({ + isFallback: false, + locale: "en", + }); + }); + + it("projects the SEO fallback field when the primary is empty", async () => { + const service = buildService({ + byId: { + 7: [ + { + locale: "en", + values: { + id: 7, + seo: { title: null }, + slug: "hello", + title: "The heading", + }, + }, + ], + }, + definition: localizedType, + }); + + expect((await service.findById(7, { locale: "en" }))?.seo).toStrictEqual({ + description: null, + title: "The heading", + }); + }); + + it("is null for a record with no public version", async () => { + const service = buildService({ definition: articleType }); + + expect(await service.findById(99)).toBeNull(); + }); + + it("adds an absolute URL only when an origin is supplied", async () => { + const service = buildService({ + byId: { 42: [article("hello")] }, + definition: articleType, + }); + + expect(await service.findById(42)).not.toHaveProperty("canonicalUrl"); + expect( + await service.findById(42, { origin: "https://example.com" }), + ).toMatchObject({ canonicalUrl: "https://example.com/articles/hello" }); + }); + + it("carries no alternates for a nonlocalized content type", async () => { + const service = buildService({ + byId: { 42: [article("hello")] }, + definition: articleType, + }); + + expect(await service.findById(42)).toMatchObject({ + alternates: [], + hreflang: { languages: {} }, + }); + }); +}); + +describe("resolvePath", () => { + it("refuses a path that belongs to another content type", async () => { + const service = buildService({ definition: articleType }); + + expect(await service.resolvePath("/news/hello")).toStrictEqual({ + type: "not_found", + }); + }); + + it("resolves a canonical path through the public read", async () => { + const service = buildService({ + byId: { 42: [article("hello")] }, + bySlug: { hello: { itemId: 42 } }, + definition: articleType, + }); + + expect(await service.resolvePath("/articles/hello")).toMatchObject({ + canonicalPath: "/articles/hello", + type: "content", + }); + }); + + it("splits the locale out of a localized path", async () => { + const service = buildService({ + byId: { 7: [translation("pl", "witaj")] }, + bySlug: { witaj: { itemId: 7, locale: "pl" } }, + definition: localizedType, + }); + + expect(await service.resolvePath("/pl/articles/witaj")).toMatchObject({ + canonicalPath: "/pl/articles/witaj", + locale: "pl", + type: "content", + }); + }); + + it("redirects a retired localized path", async () => { + const service = buildService({ + byId: { 7: [translation("pl", "nowy-slug")] }, + bySlug: {}, + definition: localizedType, + history: [ + { + itemId: 7, + languageId: 2, + path: "/pl/articles/stary-slug", + retiredAt: new Date(), + slug: "stary-slug", + }, + ], + }); + + expect(await service.resolvePath("/pl/articles/stary-slug")).toStrictEqual({ + location: "/pl/articles/nowy-slug", + status: 308, + type: "redirect", + }); + }); +}); + +describe("sitemap", () => { + it("is an empty page for a content type that lists nothing", async () => { + const noSitemap = defineContentType({ + admin: { label: { plural: "A", singular: "A" } }, + id: "delivery.no-sitemap", + delivery: { enabled: true }, + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { enabled: true, fields: ["id", "title", "slug"], path: "a" }, + tableName: "delivery_no_sitemap", + }); + + // An empty page rather than a throw: a site-level index enumerates every + // delivery-enabled content type, and one of them opting out is a choice. + expect( + await buildService({ definition: noSitemap }).sitemap(), + ).toStrictEqual({ entries: [], nextCursor: null }); + }); +}); diff --git a/packages/vitnode/src/content/server/delivery-service.ts b/packages/vitnode/src/content/server/delivery-service.ts new file mode 100644 index 000000000..af3c7e2b4 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-service.ts @@ -0,0 +1,440 @@ +import type { Context } from "hono"; + +import type { + ContentDeliveryAlternate, + ContentDeliveryHreflang, + ContentDeliveryRobots, + ContentDeliverySeo, +} from "../delivery"; +import type { ContentSitemapEntry } from "../sitemap"; +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliverySitemapPage } from "./delivery-sitemap"; +import type { ContentModel } from "./model"; +import type { ContentSlugHistoryEntry } from "./slug-history-model"; + +import { CONTENT_DELIVERY_REDIRECT_STATUS } from "../const"; +import { + contentDeliveryHreflang, + contentDeliveryOpenGraph, + contentDeliveryPath, + contentDeliveryRobots, + contentDeliverySeo, + contentDeliveryUrl, + parseContentDeliveryPath, +} from "../delivery"; +import { ContentDeliveryNotEnabled } from "../errors"; +import { contentLocalesMatch, normalizeContentLocale } from "../locale"; +import { readDeliveryAlternates } from "./delivery-alternates"; +import { readContentDeliverySitemapPage } from "./delivery-sitemap"; +import { findContentLanguage } from "./language-resolver"; +import { createContentSlugHistoryModel } from "./slug-history-model"; + +/** + * Everything a page needs to render one record's `<head>`. + * + * Two locales, not one, and the distinction is the whole reason this type exists: + * `requestedLocale` is what the URL asked for and `locale` is what the reader is + * actually being shown. With `localization.fallback: "default"` those differ, and a + * canonical URL built from the first would announce `/pl/articles/x` for an English + * translation - a URL that answers 404, self-referentially declared canonical. + */ +export interface ContentDeliveryMetadata { + /** Real published translations only. Empty for a nonlocalized content type. */ + alternates: ContentDeliveryAlternate[]; + /** + * The canonical path of the version actually being served. + * + * `null` only when the record has no public URL in that language at all, which + * for a resolved record means its slug is empty - a row written straight into the + * database rather than through the engine. + */ + canonicalPath: null | string; + /** Present only when the caller supplied an origin. */ + canonicalUrl?: null | string; + /** Framework-neutral `hreflang`, ready for an adapter to translate. */ + hreflang: ContentDeliveryHreflang; + /** Whether `locale` differs from `requestedLocale`. */ + isFallback: boolean; + /** + * The record's identifier, when the public projection carries one. + * + * `null` for a content type whose `publicApi.fields` withholds `id`, and that is + * deliberate rather than a gap: delivery metadata is read off the **public** + * projection, so it cannot report a column the public API declined to publish. + * Expose `"id"` in the allowlist and it is always present. + * + * A resolution reached through {@link ContentDeliveryService.findById} always + * carries it, because the caller supplied it. + */ + itemId: null | number; + /** The language this response is actually in. */ + locale: null | string; + /** `null` unless `delivery.seo.openGraph` is configured. */ + openGraph: ContentDeliverySeo | null; + /** What was asked for, normalized. `null` for a nonlocalized content type. */ + requestedLocale: null | string; + /** `null` unless `delivery.seo.noIndexField` is configured. */ + robots: ContentDeliveryRobots | null; + seo: ContentDeliverySeo; +} + +/** + * What one public path resolves to. + * + * A discriminated union rather than a nullable metadata object, because the three + * outcomes need three different HTTP responses and a caller that had to infer + * which one it was holding would get it wrong. `redirect` carries its own status + * so a frontend never hardcodes one. + */ +export type ContentDeliveryResolution = + | (ContentDeliveryMetadata & { type: "content" }) + | { location: string; status: 308; type: "redirect" } + | { type: "not_found" }; + +export interface ContentDeliveryReadOptions { + /** The language to read, for a localized content type. */ + locale?: string; + /** Turns every path in the result into an absolute URL as well. */ + origin?: string; +} + +export interface ContentDeliverySitemapArgs { + /** The last `itemId` of the previous page. Keyset, never an offset. */ + cursor?: number; + /** Defaults to `CONTENT_SITEMAP_DEFAULT_PAGE_SIZE`, capped at the protocol's. */ + limit?: number; + /** Required for a localized content type; each language is its own sitemap. */ + locale?: string; +} + +/** + * The read-only delivery layer of one content type. + * + * There is deliberately **no mutation here at all**. Slug history is written by + * the editorial services, inside the transaction that moves the slug, because that + * is the only place the two can be atomic - and exposing a `reserve` here would be + * an invitation to write one without the other. This object answers questions. + * + * Every answer is derived from the **public** projection, not from the base row: + * `findById` and `resolveSlug` go through `model.publicService`, so the publication + * predicate, the field allowlist and the Stage 5 fallback rules are the ones + * already tested rather than a second implementation that agrees on the day it is + * written. That is also what makes "SEO cannot leak a private field" true here for + * free: a private column is never fetched, so it is not in the row this reads. + */ +export interface ContentDeliveryService { + /** + * Every published translation's URL, in a stable order. + * + * Only real ones. A locale served by the fallback has no URL of its own, so it + * is absent - listing it would announce an `hreflang` alternate that answers + * 404 and invite a crawler to index the same content twice. + */ + alternates: (itemId: number) => Promise<ContentDeliveryAlternate[]>; + /** Delivery metadata by identifier, honouring the content type's fallback. */ + findById: ( + itemId: number, + options?: ContentDeliveryReadOptions, + ) => Promise<ContentDeliveryMetadata | null>; + /** + * Every address this record has ever answered to, current one first. + * + * Read-only, and the AdminCP's delivery panel is its only caller today. It needs + * no permission of its own beyond the one that let the reader see the record. + */ + history: ( + itemId: number, + options?: { locale?: string }, + ) => Promise<ContentSlugHistoryEntry[]>; + /** + * Resolves a whole public path: `/pl/articles/stary-slug`. + * + * The one method a catch-all route calls. It parses the path with the same rules + * {@link contentDeliveryPath} builds it by, so a path this engine did not produce + * is `not_found` rather than a guess. + */ + resolvePath: ( + path: string, + options?: { origin?: string }, + ) => Promise<ContentDeliveryResolution>; + /** The same resolution, when the caller has already split locale from slug. */ + resolveSlug: ( + slug: string, + options?: ContentDeliveryReadOptions, + ) => Promise<ContentDeliveryResolution>; + /** One page of sitemap entries. Cursor-paginated and deterministic. */ + sitemap: ( + args?: ContentDeliverySitemapArgs, + ) => Promise<ContentDeliverySitemapPage>; +} + +/** The exposed slug of a public row, or `null` when it has none. */ +const slugOf = ( + definition: AnyContentTypeDefinition, + row: Record<string, unknown>, +): null | string => { + const value = row[definition.publicApi.slugField]; + + return typeof value === "string" && value !== "" ? value : null; +}; + +/** The language a public row is actually in, off the projection's own key. */ +const localeOf = ( + definition: AnyContentTypeDefinition, + row: Record<string, unknown>, +): null | string => { + if (!definition.localization.enabled) return null; + + return typeof row.locale === "string" ? row.locale : null; +}; + +export const createContentDeliveryService = < + TDefinition extends AnyContentTypeDefinition, +>({ + c, + model, + pluginId, +}: { + c: Context; + model: ContentModel<TDefinition>; + pluginId: string; +}): ContentDeliveryService => { + const { definition } = model; + const contentTypeId = definition.id; + + if (!definition.delivery.enabled || !definition.publicApi.enabled) { + throw new ContentDeliveryNotEnabled({ contentTypeId }); + } + + const localized = definition.localization.enabled; + const buildPublic = model.publicService; + if (!buildPublic) throw new ContentDeliveryNotEnabled({ contentTypeId }); + + const slugHistory = createContentSlugHistoryModel({ + c, + definition, + pluginId, + }); + + /** + * The language a historical URL belongs to. + * + * `null` whenever the slug is shared, which covers both a nonlocalized content + * type and a localized one whose slug lives on the base row - in the second case + * every language answers to the same segment, so one reservation is correct for + * all of them. + */ + const historyLanguageId = async ( + locale: null | string, + ): Promise<null | number> => { + if (definition.delivery.slugScope !== "localized" || locale === null) { + return null; + } + + const language = await findContentLanguage(c, locale); + + return language?.id ?? null; + }; + + const metadataFor = async ( + row: Record<string, unknown>, + { + itemId, + origin, + requestedLocale, + }: { + itemId: null | number; + origin?: string; + requestedLocale: null | string; + }, + ): Promise<ContentDeliveryMetadata> => { + const locale = localeOf(definition, row); + const slug = slugOf(definition, row); + const canonicalPath = + slug === null ? null : contentDeliveryPath({ definition, locale, slug }); + const alternates = + localized && itemId !== null ? await readAlternates(itemId) : []; + + return { + alternates, + canonicalPath, + ...(origin === undefined + ? {} + : { + canonicalUrl: contentDeliveryUrl({ origin, path: canonicalPath }), + }), + hreflang: contentDeliveryHreflang({ alternates, definition }), + // Compared on the normalized forms, so `PL` asking and `pl` answering is not + // reported as a fallback. + isFallback: + requestedLocale !== null && + locale !== null && + !contentLocalesMatch(requestedLocale, locale), + itemId, + locale, + openGraph: contentDeliveryOpenGraph(definition, row), + requestedLocale, + robots: contentDeliveryRobots(definition, row), + seo: contentDeliverySeo(definition, row), + }; + }; + + const readAlternates = async ( + itemId: number, + ): Promise<ContentDeliveryAlternate[]> => + localized ? await readDeliveryAlternates({ c, itemId, model }) : []; + + /** + * The record's canonical path **in one specific language**, or `null`. + * + * Strict about the language on purpose. `publicService.findById` may fall back, + * and a redirect must not: sending `/pl/articles/stary-slug` to the English + * canonical would answer a Polish URL with an English page and permanently tell + * a crawler that is correct. So a row that came back in another language is + * treated as "this locale has no published version", which is what it is. + */ + const strictCanonicalPath = async ( + itemId: number, + locale: null | string, + ): Promise<null | string> => { + const row = await buildPublic(c).findById(itemId, { + locale: locale ?? undefined, + }); + if (!row) return null; + + const values = row as Record<string, unknown>; + const served = localeOf(definition, values); + if ( + locale !== null && + served !== null && + !contentLocalesMatch(locale, served) + ) { + return null; + } + + const slug = slugOf(definition, values); + + return slug === null + ? null + : contentDeliveryPath({ definition, locale: served, slug }); + }; + + /** + * The language a read is *actually* for. + * + * The default locale when the caller named none, because that is what the public + * service resolves internally - and the history lookup has to be about the same + * language, or the live branch would search `en` while the redirect branch searched + * the shared rows and found nothing. + */ + const localeFor = (locale: string | undefined): null | string => { + if (!localized) return null; + + return normalizeContentLocale( + locale ?? definition.localization.defaultLocale, + ); + }; + + const resolve = async ( + slug: string, + { locale, origin }: ContentDeliveryReadOptions = {}, + ): Promise<ContentDeliveryResolution> => { + const requestedLocale = localeFor(locale); + + // The live record first, and strictly by slug: a URL belongs to the language + // it was published under, so `findBySlug` never falls back. + const row = await buildPublic(c).findBySlug(slug, { locale }); + if (row) { + const values = row as Record<string, unknown>; + + return { + ...(await metadataFor(values, { + // Only what the public projection actually carries - see + // `ContentDeliveryMetadata.itemId`. + itemId: typeof values.id === "number" ? values.id : null, + origin, + requestedLocale, + })), + type: "content", + }; + } + + if (!definition.delivery.redirects.enabled) return { type: "not_found" }; + + const languageId = await historyLanguageId(requestedLocale); + const owner = await slugHistory.owner({ languageId, slug }); + if (!owner) return { type: "not_found" }; + + // Straight to the record's **current** address, never to the next entry in the + // chain. `a -> b -> c` collapses here rather than in the data: the database + // keeps the chronology, and the resolver answers with one hop. + const destination = await strictCanonicalPath( + owner.itemId, + requestedLocale, + ); + + // Unpublished, deleted, or published only in another language: a historical URL + // must not become a way to reach content that is not public. 404 rather than a + // redirect to a page that would itself 404. + if (destination === null || destination === owner.path) { + return { type: "not_found" }; + } + + return { + location: destination, + status: CONTENT_DELIVERY_REDIRECT_STATUS, + type: "redirect", + }; + }; + + return { + alternates: async itemId => await readAlternates(itemId), + + findById: async (itemId, { locale, origin } = {}) => { + const row = await buildPublic(c).findById(itemId, { locale }); + if (!row) return null; + + return await metadataFor(row, { + itemId, + origin, + requestedLocale: localeFor(locale), + }); + }, + + history: async (itemId, { locale } = {}) => { + const languageId = await historyLanguageId( + locale === undefined ? null : normalizeContentLocale(locale), + ); + + return await slugHistory.list({ + itemId, + // `undefined` - not `null` - when the caller named no locale, so the query + // is unscoped rather than scoped to the shared rows. A shared slug's + // history really is `languageId IS NULL`, and asking for "everything" has + // to stay distinguishable from asking for "the shared ones". + languageId: + definition.delivery.slugScope === "localized" && locale === undefined + ? undefined + : languageId, + }); + }, + + resolvePath: async (path, { origin } = {}) => { + const parts = parseContentDeliveryPath(definition, path); + if (!parts) return { type: "not_found" }; + + return await resolve(parts.slug, { + locale: parts.locale ?? undefined, + origin, + }); + }, + + resolveSlug: async (slug, options) => await resolve(slug, options), + + sitemap: async (args = {}) => + await readContentDeliverySitemapPage({ args, c, model }), + }; +}; + +/** Re-exported so a caller need not reach past this module for the entry type. */ +export type { ContentSitemapEntry }; diff --git a/packages/vitnode/src/content/server/delivery-sitemap.ts b/packages/vitnode/src/content/server/delivery-sitemap.ts new file mode 100644 index 000000000..94e08eb94 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-sitemap.ts @@ -0,0 +1,292 @@ +import type { SQL } from "drizzle-orm"; +import type { + PgColumn, + PgTableWithColumns, + TableConfig, +} from "drizzle-orm/pg-core"; +import type { Context } from "hono"; + +import { and, asc, eq, gt, ne, sql } from "drizzle-orm"; + +import type { ContentSitemapEntry } from "../sitemap"; +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliverySitemapArgs } from "./delivery-service"; +import type { ContentModel } from "./model"; + +import { + CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + CONTENT_SITEMAP_MAX_URLS, +} from "../const"; +import { contentDeliveryPath } from "../delivery"; +import { ContentDeliveryNotEnabled } from "../errors"; +import { splitContentFieldPath } from "../paths"; +import { findContentLanguage } from "./language-resolver"; +import { + contentTranslationPublicationColumns, + publicationColumns, + publishedCondition, +} from "./publication"; + +/** + * One page of a content type's sitemap. + * + * `nextCursor` rather than a page number, and `null` rather than `hasNextPage` on + * its own: a sitemap is regenerated from scratch every time a crawler asks, and an + * `OFFSET` deep into a large table both slows down linearly and skips rows when + * something is published between two pages. A keyset over the primary key does + * neither. + */ +export interface ContentDeliverySitemapPage { + entries: ContentSitemapEntry[]; + /** Pass back as `cursor`. `null` when this was the last page. */ + nextCursor: null | number; +} + +/** + * Where a `noIndex` field is stored, resolved to the column it addresses. + * + * A leaf path (`seo.noIndex`) compiles to a generated column, and the delivery + * resolver has already refused a localized one - so this is always a column on the + * base table and the sitemap predicate is one clause rather than a join. + */ +const noIndexColumn = ( + definition: AnyContentTypeDefinition, + columns: Record<string, PgColumn>, +): null | PgColumn => { + const { noIndexField } = definition.delivery.seo; + if (noIndexField === null) return null; + + const path = splitContentFieldPath(noIndexField); + if (!path) return columns[noIndexField] ?? null; + + const leaf = definition.advanced.leaves.find( + entry => entry.path === noIndexField, + ); + + return leaf === undefined ? null : (columns[leaf.columnName] ?? null); +}; + +/** + * One page of sitemap entries for one content type, in one language. + * + * Everything about this function follows from "a sitemap lists what is public + * right now, and nothing else": + * + * - **The publication predicate is not a parameter.** A nonlocalized entry needs + * the base row published; a localized one needs the base row *and* the + * translation published, which is the same subordination the public read + * applies. A draft, an unpublished record and a future `publishedAt` are all + * simply absent. + * - **No fallback, ever.** Each locale is queried against its own translation, so + * a language served English through `fallback: "default"` contributes no URL - + * it has none of its own, and listing one would put the same content in the + * sitemap twice under two addresses. + * - **`lastModified` is `max(base.updatedAt, translation.updatedAt)`** for a + * localized entry. A shared field moving changes what every language's page + * renders even though no translation row was touched, so taking the + * translation's timestamp alone would tell a crawler nothing had changed. + * - **`noIndex` is one clause**, not a post-filter, so a page of 1,000 entries is + * 1,000 listed URLs rather than however many survived. + */ +export const readContentDeliverySitemapPage = async < + TDefinition extends AnyContentTypeDefinition, +>({ + args, + c, + model, +}: { + args: ContentDeliverySitemapArgs; + c: Context; + model: ContentModel<TDefinition>; +}): Promise<ContentDeliverySitemapPage> => { + const { columns, definition, translationColumns } = model; + const { sitemap } = definition.delivery; + + if (!definition.delivery.enabled || !definition.publicApi.enabled) { + throw new ContentDeliveryNotEnabled({ contentTypeId: definition.id }); + } + + // A content type that lists nothing answers with an empty page rather than + // throwing: a site-level sitemap index enumerates every delivery-enabled content + // type, and one of them opting out of the sitemap is a configuration choice, not + // a caller error. + if (!sitemap.enabled) return { entries: [], nextCursor: null }; + + const limit = Math.max( + 1, + Math.min( + args.limit ?? CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + CONTENT_SITEMAP_MAX_URLS, + ), + ); + const slugField = definition.publicApi.slugField; + const base = publicationColumns(definition, columns); + const exclude = noIndexColumn(definition, columns); + const localized = definition.localization.enabled; + // Widened, not cast - see `readDeliveryAlternatesMany` for why. The translation + // table needs the same treatment for the join below. + const baseTable: PgTableWithColumns<TableConfig> = model.table; + const joinedTranslations: null | PgTableWithColumns<TableConfig> = + model.translationTable; + + const conditions: (SQL | undefined)[] = [ + publishedCondition(base), + args.cursor === undefined ? undefined : gt(columns.id, args.cursor), + // `ne(..., true)` rather than `eq(..., false)`: the column is `NOT NULL` today, + // and a nullable one added later would silently drop every row whose value was + // never set if this asked for an exact `false`. + exclude === null ? undefined : ne(exclude, true), + ]; + + if (!localized) { + const rows = await c + .get("db") + .select({ + itemId: columns.id, + lastModified: columns.updatedAt, + slug: columns[slugField], + }) + .from(baseTable) + .where( + and(...conditions.filter((part): part is SQL => part !== undefined)), + ) + .orderBy(asc(columns.id)) + .limit(limit + 1); + + return page({ + definition, + limit, + locale: null, + rows: rows.map(row => ({ + itemId: row.itemId as number, + lastModified: row.lastModified as Date, + slug: row.slug, + })), + }); + } + + if (!joinedTranslations || !translationColumns) { + return { entries: [], nextCursor: null }; + } + + // Each language is its own sitemap, so the language is resolved before the query + // rather than joined: a locale that names nothing this install serves has no + // sitemap, which is an empty page rather than an error - a crawler asking for + // `/sitemaps/blog.article-de.xml` on a site with no German should get a valid + // empty document. + const language = await findContentLanguage( + c, + args.locale ?? definition.localization.defaultLocale, + ); + if (!language?.isEnabled) return { entries: [], nextCursor: null }; + + const translation = contentTranslationPublicationColumns( + definition, + translationColumns, + ); + const slugColumn: PgColumn = + definition.delivery.slugScope === "localized" + ? translationColumns[slugField] + : columns[slugField]; + + const rows = await c + .get("db") + .select({ + itemId: columns.id, + // The representation's timestamp, not the row's: both halves are rendered + // into the page, so the later of the two is when it last changed. + // + // `.mapWith` is load-bearing rather than tidy. Drizzle turns off the driver's + // own timestamp parsing so its column mappers can treat a naive `timestamp` + // as UTC - but a raw `sql` fragment has no mapper, so the driver's fallback + // parses the same value as *local* time. The two disagree by the server's + // offset, which would put every localized `lastmod` hours out. Borrowing the + // column's decoder makes this expression read exactly as the column does. + lastModified: + sql<Date>`greatest(${columns.updatedAt}, ${translationColumns.updatedAt})`.mapWith( + columns.updatedAt, + ), + slug: slugColumn, + }) + .from(baseTable) + .innerJoin( + joinedTranslations, + and( + eq(translationColumns.itemId, columns.id), + eq(translationColumns.languageId, language.id), + ), + ) + .where( + and( + ...conditions.filter((part): part is SQL => part !== undefined), + publishedCondition(translation), + ), + ) + .orderBy(asc(columns.id)) + .limit(limit + 1); + + return page({ + definition, + limit, + locale: language.locale, + rows: rows.map(row => ({ + itemId: row.itemId as number, + // `greatest()` comes back as a string on some drivers, so it is normalized + // here rather than trusted - a sitemap `lastmod` of "Invalid Date" is a + // document a crawler rejects. + lastModified: + row.lastModified instanceof Date + ? row.lastModified + : new Date(String(row.lastModified)), + slug: row.slug, + })), + }); +}; + +/** + * Turns one over-fetched page of rows into entries and a cursor. + * + * `limit + 1` is fetched and the extra row is dropped, which is how "is there a + * next page" is answered without a second `COUNT` over a table that may be large. + */ +const page = ({ + definition, + limit, + locale, + rows, +}: { + definition: AnyContentTypeDefinition; + limit: number; + locale: null | string; + rows: readonly { itemId: number; lastModified: Date; slug: unknown }[]; +}): ContentDeliverySitemapPage => { + const visible = rows.slice(0, limit); + const { sitemap } = definition.delivery; + const entries: ContentSitemapEntry[] = []; + + for (const row of visible) { + const path = contentDeliveryPath({ + definition, + locale, + slug: typeof row.slug === "string" ? row.slug : "", + }); + // A row with no buildable path has no URL, so it has no sitemap line. It stays + // out of the entries and still advances the cursor, which is why the cursor is + // taken from `visible` rather than from `entries`. + if (path === null) continue; + + entries.push({ + changeFrequency: sitemap.changeFrequency, + itemId: row.itemId, + lastModified: row.lastModified, + locale, + path, + priority: sitemap.priority, + }); + } + + return { + entries, + nextCursor: rows.length > limit ? (visible.at(-1)?.itemId ?? null) : null, + }; +}; diff --git a/packages/vitnode/src/content/server/delivery-writes.test.ts b/packages/vitnode/src/content/server/delivery-writes.test.ts new file mode 100644 index 000000000..d32b96754 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-writes.test.ts @@ -0,0 +1,418 @@ +import { describe, expect, it } from "vitest"; + +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDatabase } from "./service"; +import type { + ContentSlugHistoryModel, + ContentSlugHistoryTarget, +} from "./slug-history-model"; + +import { defineContentType } from "../define"; +import { ContentDeliverySlugReserved } from "../errors"; +import { field } from "../fields"; +import { applyContentDeliveryWrite } from "./delivery-writes"; + +/** + * When slug history is written, and when it deliberately is not. + * + * The rule this file exists to pin down is the one in §10 of the Stage 8 brief: a + * slug becomes redirectable only if it was **previously used by an addressable + * public version**. That is what separates "a live URL moved and needs a redirect" + * from "somebody fixed a typo in a draft three times before publishing" - and + * getting it wrong means either a pile of redirects nobody asked for, or a moved + * page that 404s. + */ + +const articleType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "writes.article", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "writes_articles", +}); + +const localizedType = defineContentType({ + admin: { label: { plural: "Articles", singular: "Article" } }, + id: "writes.localized", + editorial: { enabled: true }, + delivery: { enabled: true, redirects: { enabled: true } }, + fields: { + slug: field.slug({ localized: true, source: "title" }), + title: field.text({ localized: true, required: true }), + }, + localization: { defaultLocale: "en", enabled: true }, + publication: { enabled: true }, + publicApi: { + enabled: true, + fields: ["id", "title", "slug"], + path: "articles", + }, + tableName: "writes_localized", +}); + +interface Call { + args: ContentSlugHistoryTarget | Omit<ContentSlugHistoryTarget, "locale">; + kind: "assertAvailable" | "reserve" | "retire"; +} + +/** + * A history model that records what it was asked to do. + * + * `retired` is the interesting knob: it is the answer to "was that URL ever live", + * and the whole redirect decision hangs off it. + */ +const recorder = ({ + reserved = null, + retired = true, +}: { reserved?: null | string; retired?: boolean } = {}) => { + const calls: Call[] = []; + + const model: ContentSlugHistoryModel = { + assertAvailable: async (_tx, args) => { + calls.push({ args, kind: "assertAvailable" }); + if (reserved !== null && args.slug === reserved) { + throw new ContentDeliverySlugReserved({ + contentTypeId: "writes.article", + locale: args.locale, + slug: args.slug, + }); + } + + return await Promise.resolve(); + }, + list: async () => await Promise.resolve([]), + owner: async () => await Promise.resolve(null), + reserve: async (_tx, args) => { + calls.push({ args, kind: "reserve" }); + if (reserved !== null && args.slug === reserved) { + throw new ContentDeliverySlugReserved({ + contentTypeId: "writes.article", + locale: args.locale, + slug: args.slug, + }); + } + + return await Promise.resolve({ created: true }); + }, + retire: async (_tx, args) => { + calls.push({ args, kind: "retire" }); + + return await Promise.resolve({ retired }); + }, + }; + + return { calls, model }; +}; + +const tx = {} as ContentDatabase; + +const apply = async ( + definition: AnyContentTypeDefinition, + transition: Parameters<typeof applyContentDeliveryWrite>[0]["transition"], + options?: { reserved?: null | string; retired?: boolean }, +) => { + const { calls, model } = recorder(options); + const outcome = await applyContentDeliveryWrite({ + definition, + slugHistory: model, + transition, + tx, + }); + + return { calls, outcome }; +}; + +describe("a draft", () => { + it("checks its slug but reserves nothing", async () => { + const { calls, outcome } = await apply(articleType, { + isPublic: false, + itemId: 1, + languageId: null, + locale: null, + previousSlug: null, + slug: "hello", + wasPublic: false, + }); + + // Checked, because "that address belongs to an article that moved" is far + // better heard at save time. Not reserved, because a draft has no public URL + // and claiming one would refuse a live address to somebody who wants it. + expect(calls.map(call => call.kind)).toStrictEqual(["assertAvailable"]); + expect(outcome).toMatchObject({ + canonicalPath: "/articles/hello", + redirectCreated: false, + // Neither public before nor after, so no sitemap file lists it either way. + sitemap: { contentChanged: false, indexChanged: false }, + slugChanged: false, + }); + }); + + it("creates no redirect when its slug is corrected before publication", async () => { + const { calls, outcome } = await apply( + articleType, + { + isPublic: false, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "typo", + slug: "fixed", + wasPublic: false, + }, + // Nothing to retire: the old slug was never publicly addressable, so no row + // exists for it. + { retired: false }, + ); + + expect(calls.map(call => call.kind)).toStrictEqual([ + "retire", + "assertAvailable", + ]); + expect(outcome).toMatchObject({ + previousPath: "/articles/typo", + redirectCreated: false, + slugChanged: true, + }); + }); +}); + +describe("publishing", () => { + it("reserves the current address", async () => { + const { calls, outcome } = await apply(articleType, { + isPublic: true, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "hello", + slug: "hello", + wasPublic: false, + }); + + expect(calls).toStrictEqual([ + { + args: { + itemId: 1, + languageId: null, + locale: null, + path: "/articles/hello", + slug: "hello", + }, + kind: "reserve", + }, + ]); + // A publish adds a sitemap line even though no URL moved, and changes how many + // URLs the index counts. + expect(outcome).toMatchObject({ + sitemap: { contentChanged: true, indexChanged: true }, + slugChanged: false, + }); + }); + + it("refuses an address another record's history owns", async () => { + await expect( + apply( + articleType, + { + isPublic: true, + itemId: 2, + languageId: null, + locale: null, + previousSlug: "hello", + slug: "hello", + wasPublic: false, + }, + { reserved: "hello" }, + ), + ).rejects.toThrow(ContentDeliverySlugReserved); + }); +}); + +describe("moving a published URL", () => { + it("retires the old address and reserves the new one, in that order", async () => { + const { calls, outcome } = await apply(articleType, { + isPublic: true, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "old", + slug: "new", + wasPublic: true, + }); + + // Retire first: a move from `a` to `b` and back to `a` would otherwise hit its + // own live reservation. + expect(calls.map(call => call.kind)).toStrictEqual(["retire", "reserve"]); + expect(outcome).toMatchObject({ + canonicalPath: "/articles/new", + previousPath: "/articles/old", + previousSlug: "old", + redirectCreated: true, + // The file's bytes moved - one line now reads a different URL - but the number + // of files an index lists did not. + sitemap: { contentChanged: true, indexChanged: false }, + slugChanged: true, + }); + }); + + it("reports no redirect when the old slug had never been live", async () => { + const { outcome } = await apply( + articleType, + { + isPublic: true, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "old", + slug: "new", + wasPublic: true, + }, + { retired: false }, + ); + + expect(outcome).toMatchObject({ + redirectCreated: false, + slugChanged: true, + }); + }); +}); + +describe("unpublishing and deleting", () => { + it("writes nothing on an unpublish, and keeps the history", async () => { + const { calls, outcome } = await apply(articleType, { + isPublic: false, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "hello", + slug: "hello", + wasPublic: true, + }); + + // No retire (the slug did not move) and no reserve (it is not public). The + // resolver stops redirecting because it reads the live publication state. + expect(calls).toStrictEqual([]); + expect(outcome).toMatchObject({ + sitemap: { contentChanged: true, indexChanged: true }, + slugChanged: false, + }); + }); + + it("writes nothing on a delete, and reports the lost sitemap line", async () => { + const { calls, outcome } = await apply(articleType, { + isPublic: false, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "hello", + slug: null, + wasPublic: true, + }); + + expect(calls).toStrictEqual([]); + expect(outcome).toMatchObject({ + canonicalPath: null, + sitemap: { contentChanged: true, indexChanged: true }, + slug: null, + slugChanged: false, + }); + }); +}); + +describe("a localized slug", () => { + it("carries the language on every write, so histories stay isolated", async () => { + const { calls } = await apply(localizedType, { + isPublic: true, + itemId: 7, + languageId: 2, + locale: "pl", + previousSlug: "stary", + slug: "nowy", + wasPublic: true, + }); + + expect(calls).toStrictEqual([ + { args: { itemId: 7, languageId: 2, slug: "stary" }, kind: "retire" }, + { + args: { + itemId: 7, + languageId: 2, + locale: "pl", + path: "/pl/articles/nowy", + slug: "nowy", + }, + kind: "reserve", + }, + ]); + }); + + it("builds locale-prefixed paths on both sides of the move", async () => { + const { outcome } = await apply(localizedType, { + isPublic: true, + itemId: 7, + languageId: 2, + locale: "pl", + previousSlug: "stary", + slug: "nowy", + wasPublic: true, + }); + + expect(outcome).toMatchObject({ + canonicalPath: "/pl/articles/nowy", + locale: "pl", + previousPath: "/pl/articles/stary", + }); + }); +}); + +describe("delivery without redirects", () => { + it("reports the paths and writes no history at all", async () => { + const withoutRedirects = defineContentType({ + admin: { label: { plural: "A", singular: "A" } }, + id: "writes.no-redirects", + delivery: { enabled: true, sitemap: { enabled: true } }, + fields: { + slug: field.slug({ source: "title" }), + title: field.text({ required: true }), + }, + publication: { enabled: true }, + publicApi: { enabled: true, fields: ["id", "title", "slug"], path: "a" }, + tableName: "writes_no_redirects", + }); + + const outcome = await applyContentDeliveryWrite({ + definition: withoutRedirects, + // `null` is how the caller says "this content type keeps no history". + slugHistory: null, + transition: { + isPublic: true, + itemId: 1, + languageId: null, + locale: null, + previousSlug: "old", + slug: "new", + wasPublic: true, + }, + tx, + }); + + expect(outcome).toMatchObject({ + canonicalPath: "/a/new", + previousPath: "/a/old", + // The URL moved and the file changed - the engine simply cannot redirect the + // old address, because nothing recorded it. + redirectCreated: false, + sitemap: { contentChanged: true, indexChanged: false }, + slugChanged: true, + }); + }); +}); diff --git a/packages/vitnode/src/content/server/delivery-writes.ts b/packages/vitnode/src/content/server/delivery-writes.ts new file mode 100644 index 000000000..551f6ce75 --- /dev/null +++ b/packages/vitnode/src/content/server/delivery-writes.ts @@ -0,0 +1,204 @@ +import type { Context } from "hono"; + +import type { ContentSitemapChange } from "../cache"; +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDatabase } from "./service"; +import type { ContentSlugHistoryModel } from "./slug-history-model"; + +import { contentDeliveryPath } from "../delivery"; +import { createContentSlugHistoryModel } from "./slug-history-model"; + +/** + * What one mutation did to a record's public URLs. + * + * Carried on the editorial outcome so the post-commit effects can emit the delivery + * events and pick the cache tags without re-reading anything: once the transaction + * returns, the *old* URL is gone from the row, and it is the one fact that cannot be + * recovered afterwards - the same reason `previousSlug` is already on the outcome. + * + * Optional on both outcome types rather than required, which is what keeps every + * Stage 1-7 construction site compiling untouched and every content type without + * delivery producing exactly the outcome it always produced. + */ +export interface ContentDeliveryOutcome { + /** The path the record answers to after this mutation. */ + canonicalPath: null | string; + itemId: number; + /** `null` when the slug is shared - see `core_content_slug_history`. */ + locale: null | string; + /** The path it answered to before, when the mutation moved it. */ + previousPath: null | string; + previousSlug: null | string; + /** + * Whether a historical URL became a redirect. + * + * `true` only when the old slug had genuinely been publicly addressable, which is + * the difference between "somebody fixed a typo in a draft" and "a live URL + * moved". It is what the `delivery_redirect_created` event is gated on. + */ + redirectCreated: boolean; + /** + * What this mutation did to the sitemap. + * + * Two booleans rather than one, because a sitemap entry carries a `<lastmod>` + * derived from `updatedAt`: a plain title edit on a published record changes the + * file's bytes without changing which URLs it lists. See + * {@link ContentSitemapChange}. + */ + sitemap: ContentSitemapChange; + /** The slug the record answers to now, or `null` once it is deleted. */ + slug: null | string; + /** Whether the canonical URL is different from what it was. */ + slugChanged: boolean; +} + +/** + * One record's addressability, before and after a mutation. + * + * Supplied by the caller rather than derived here, because only the caller knows: + * it holds the row on both sides of its own guarded write, and re-reading would + * both cost a query and race with a concurrent writer. + */ +export interface ContentDeliveryTransition { + /** Whether the record is publicly reachable *after* the mutation. */ + isPublic: boolean; + itemId: number; + /** `null` when the slug is shared. */ + languageId: null | number; + /** The canonical locale code, or `null` when the slug is shared. */ + locale: null | string; + previousSlug: null | string; + slug: null | string; + /** Whether it was publicly reachable *before*. */ + wasPublic: boolean; +} + +/** + * The delivery half of one slug-bearing mutation, inside its transaction. + * + * The order below is the whole correctness argument, and it is why this is one + * function rather than three calls sprinkled through the editorial services: + * + * 1. **Retire the old address first.** It has to stop being the record's current + * slug before the new one can be reserved, or a move from `a` to `b` and back to + * `a` would hit its own live reservation. + * 2. **Reserve the new one second**, and only when the record is publicly + * reachable. A draft has no public URL, so reserving its slug would hand out a + * permanent claim on a URL that was never live - and then refuse it to somebody + * who wants it. + * 3. **Report, never act.** Nothing here emits an event, writes a cache tag or + * calls the search index. The caller is inside a transaction that may still roll + * back, and a rollback cannot un-send any of those. + * + * `retire` returning `{ retired: false }` is not a failure: it is the answer to + * "was that slug ever a live URL", and a `false` is what keeps a corrected draft + * from creating a redirect nobody asked for. + */ +export const applyContentDeliveryWrite = async ({ + definition, + slugHistory, + transition, + tx, +}: { + definition: AnyContentTypeDefinition; + /** `null` for a content type with `delivery` but no `redirects`. */ + slugHistory: ContentSlugHistoryModel | null; + transition: ContentDeliveryTransition; + tx: ContentDatabase; +}): Promise<ContentDeliveryOutcome> => { + const { + isPublic, + itemId, + languageId, + locale, + previousSlug, + slug, + wasPublic, + } = transition; + const pathFor = (value: null | string): null | string => + value === null + ? null + : contentDeliveryPath({ definition, locale, slug: value }); + + const canonicalPath = pathFor(slug); + const previousPath = pathFor(previousSlug); + const slugChanged = + previousSlug !== null && slug !== null && previousSlug !== slug; + + let redirectCreated = false; + + if (slugHistory !== null) { + if (slugChanged && previousSlug !== null) { + const { retired } = await slugHistory.retire(tx, { + itemId, + languageId, + slug: previousSlug, + }); + // A retired row is proof the URL was live: it is only ever written by a + // publish or by a slug change on an already-public record. + redirectCreated = retired; + } + + if (isPublic && slug !== null && canonicalPath !== null) { + // Reserved whenever the record is publicly reachable *now*, whether or not + // the slug moved: this is also the publish path, where the address becomes + // live for the first time. Idempotent, so a republish of an unchanged slug + // re-activates the row it already has - and it throws when another record + // owns the address, which is the reservation being enforced. + await slugHistory.reserve(tx, { + itemId, + languageId, + locale, + path: canonicalPath, + slug, + }); + } else if (slug !== null && slug !== previousSlug) { + // A draft taking a *new* slug is checked but not reserved. Not reserved, + // because a draft has no public URL and claiming one would refuse a live + // address to somebody who wants it; checked, because telling an editor "that + // address is taken" at save time is far better than at publish time, when + // they have moved on. + await slugHistory.assertAvailable(tx, { + itemId, + languageId, + locale, + slug, + }); + } + } + + return { + canonicalPath, + itemId, + locale, + previousPath: slugChanged ? previousPath : null, + previousSlug: slugChanged ? previousSlug : null, + redirectCreated, + sitemap: { + // Any real mutation of a record that is or was publicly reachable changes the + // file: it gained a line, lost one, moved one, or moved its own `<lastmod>`. + // This function is only ever reached for a real mutation - a no-op update + // returns before the delivery step - so "was or is public" is the whole test. + contentChanged: wasPublic || isPublic, + // Only appearing or disappearing changes how many files an index lists. A slug + // change rewrites one line inside a file; a title edit rewrites a timestamp. + indexChanged: wasPublic !== isPublic, + }, + slug, + slugChanged, + }; +}; + +/** Builds the history model a content type with `delivery` writes through. */ +export const contentSlugHistoryFor = ({ + c, + definition, + pluginId, +}: { + c: Context; + definition: AnyContentTypeDefinition; + pluginId: string; +}): ContentSlugHistoryModel | null => + definition.delivery.enabled && definition.delivery.redirects.enabled + ? createContentSlugHistoryModel({ c, definition, pluginId }) + : null; diff --git a/packages/vitnode/src/content/server/diagnostics.ts b/packages/vitnode/src/content/server/diagnostics.ts new file mode 100644 index 000000000..fdc497836 --- /dev/null +++ b/packages/vitnode/src/content/server/diagnostics.ts @@ -0,0 +1,584 @@ +import type { PgColumn, PgTable } from "drizzle-orm/pg-core"; +import type { Context } from "hono"; + +import { and, count, eq, inArray, sql } from "drizzle-orm"; + +import type { RegisteredContentModel } from "./model"; + +import { core_content_schedules } from "../../database/content"; +import { core_search_index } from "../../database/search"; +import { normalizeContentLocale } from "../locale"; +import { listContentLanguages } from "./language-resolver"; +import { + contentTranslationPublicationColumns, + publicationColumns, + publishedCondition, +} from "./publication"; + +/** + * Operational diagnostics for the Content Engine. + * + * Deliberately small, and deliberately **not** a monitoring product. It answers + * three questions an operator actually asks at three in the morning, and + * nothing else: + * + * 1. *Is the search index telling the truth?* - the database is the source of + * truth, so "how many documents should there be" is a `COUNT` over published + * rows, and "how many are there" is a `COUNT` over `core_search_index`. A + * difference is drift, and drift is repaired by a rebuild. + * 2. *Did anything scheduled fail to announce itself?* - a scheduled transition + * that committed but whose event, index write or cache expiry did not is + * recorded on the schedule row as `effectsError`. One count per content type + * turns "somewhere in the install" into "this content type". + * 3. *Which content types are even in play?* - what is registered, and which of + * the optional subsystems each one has switched on. + * + * No Prometheus, no time series, no dashboards - the install has none of those + * and Stage 7 does not add them. Everything here is a handful of aggregate + * queries, computed on demand, behind an admin permission. + */ + +/** One locale's share of a content type's index, from every storage that has one. */ +export interface ContentSearchDriftLocale { + /** + * `true` when the canonical table matches the database. + * + * A count is not a checksum: two documents can be stale and still count as + * two. It is the cheap check that catches the failure that actually happens - + * a live sync that threw, or a rebuild that stopped halfway - and it costs two + * aggregates rather than a full comparison. + */ + canonicalHealthy: boolean; + /** Documents the canonical `core_search_index` holds for this locale. */ + canonicalIndexed: number; + /** Published rows - or published translations - the database holds. */ + expected: number; + /** + * `""` for a content type that is not localized. + * + * The empty string is what `core_search_index` stores for language-agnostic + * content, so it is the honest key here rather than `null` - it is the value + * the row really holds. + */ + locale: string; + /** + * `true` when the active provider matches, `false` when it does not, and + * `null` when nobody looked. + * + * The three-way answer is the point. A provider that offers no diagnostics + * cannot be called healthy, and calling it healthy anyway is how an + * Elasticsearch outage hides behind a perfectly good canonical table. + */ + providerHealthy: boolean | null; + /** Documents the active provider holds, or `null` when it cannot say. */ + providerIndexed: null | number; +} + +/** What the active search provider could be asked, and what it answered. */ +export interface ContentSearchDriftProvider { + /** Why the provider could not be counted, when that is the answer. */ + error?: string; + /** `null` when unverified - see {@link ContentSearchDriftLocale.providerHealthy}. */ + healthy: boolean | null; + /** + * Every document the provider holds for this content type, in **any** locale. + * + * The guard against ghosts. Per-locale counts can only ask about locales the + * database still knows about, so a document left behind in a locale that has + * since been removed - or one whose canonical row was deleted while the + * provider's delete failed - is invisible to them. A total is not: it is + * larger than `expectedTotal`, and that is enough to say something is wrong. + * + * `null` when the provider offers no diagnostics. + */ + indexedTotal: null | number; + name: string; + /** + * Whether the provider was actually asked. + * + * `false` means it offers no `count`, so nothing about its contents is known. + * `true` with `healthy: false` means it was asked and it disagreed - or it + * threw, and `error` says so. + */ + verified: boolean; +} + +export interface ContentSearchDrift { + /** `true` when `core_search_index` matches the database, per locale and in total. */ + canonicalHealthy: boolean; + /** Documents `core_search_index` holds for this content type, all locales. */ + canonicalIndexedTotal: number; + contentTypeId: string; + /** Published rows - or published translations - the database holds, all locales. */ + expectedTotal: number; + /** + * `true` only when the canonical table **and** the active provider both agree, + * per locale and in total. + * + * An unverified provider is not healthy: absence of evidence is reported as + * absence of evidence, and the operator decides what to do about it. + */ + healthy: boolean; + locales: ContentSearchDriftLocale[]; + provider: ContentSearchDriftProvider; +} + +/** + * Compares what the database says should be indexed against what is - in the + * canonical table **and** in the active search provider. + * + * The two are not the same question, and conflating them was the first gap this + * closes. `SearchModel.index` writes `core_search_index` and then hands the + * document to the provider; an Elasticsearch that refuses the second half leaves + * a canonical table that is perfectly correct and a search box that is missing + * results. + * + * The second gap is the opposite direction, and it needs a different instrument. + * Deletion runs canonical-first: `SearchModel.delete` removes the row and then + * asks the provider. If the provider's half fails, the document survives in a + * locale that no longer appears in either the database or the canonical table - + * so per-locale enumeration, which is built from those two, can never ask about + * it. Hence `indexedTotal`: one unfiltered count that no amount of missing + * enumeration can hide from. + * + * Costs: two aggregates for the canonical side, whatever the collection's size, + * plus one provider total and one provider count per locale. The bundled + * Postgres provider is skipped entirely - its store *is* the canonical table, so + * asking twice would buy nothing. + */ +export const contentSearchDrift = async ( + c: Context, + { model }: Pick<RegisteredContentModel, "model">, +): Promise<ContentSearchDrift> => { + const { definition } = model; + const contentTypeId = definition.id; + + const indexedRows = await c + .get("db") + .select({ + documents: count(), + languageCode: core_search_index.languageCode, + }) + .from(core_search_index) + .where(eq(core_search_index.itemType, contentTypeId)) + .groupBy(core_search_index.languageCode); + + const indexedByLocale = new Map( + indexedRows.map(row => [ + normalizeContentLocale(row.languageCode), + row.documents, + ]), + ); + + const expectedByLocale = await expectedDocuments(c, model); + + const keys = [ + ...new Set([...expectedByLocale.keys(), ...indexedByLocale.keys()]), + ].sort(); + + const sum = (values: Iterable<number>): number => { + let total = 0; + for (const value of values) total += value; + + return total; + }; + const expectedTotal = sum(expectedByLocale.values()); + const canonicalIndexedTotal = sum(indexedByLocale.values()); + + const provider = await providerCounts(c, { + canonicalTotal: canonicalIndexedTotal, + contentTypeId, + locales: keys, + }); + + const locales: ContentSearchDriftLocale[] = keys.map(locale => { + const expected = expectedByLocale.get(locale) ?? 0; + const canonicalIndexed = indexedByLocale.get(locale) ?? 0; + // The canonical count stands in for the provider only when they are one + // storage. Everywhere else an absent count means unverified, never healthy. + const providerIndexed = provider.canonical + ? canonicalIndexed + : (provider.byLocale.get(locale) ?? null); + + return { + canonicalHealthy: expected === canonicalIndexed, + canonicalIndexed, + expected, + locale, + providerHealthy: + providerIndexed === null ? null : providerIndexed === expected, + providerIndexed, + }; + }); + + // The total is part of *both* verdicts, not decoration on the provider one: + // a canonical row in a locale nothing expects is a ghost too, and the grouped + // query above already sees every locale the table holds. + const canonicalHealthy = + canonicalIndexedTotal === expectedTotal && + locales.every(entry => entry.canonicalHealthy); + + const providerHealthy = provider.verified + ? provider.error === undefined && + provider.total === expectedTotal && + locales.every(entry => entry.providerHealthy === true) + : null; + + return { + canonicalHealthy, + canonicalIndexedTotal, + contentTypeId, + expectedTotal, + healthy: canonicalHealthy && providerHealthy === true, + locales, + provider: { + ...(provider.error === undefined ? {} : { error: provider.error }), + healthy: providerHealthy, + indexedTotal: provider.total, + name: provider.name, + verified: provider.verified, + }, + }; +}; + +/** + * Asks the active provider how many documents it holds - in total, and per + * locale. + * + * The total goes first and is asked **unconditionally**, including for a content + * type with no rows and no canonical documents at all. That is the case the + * per-locale loop cannot cover: with nothing to enumerate, `[].every(...)` is + * `true`, and a ghost document would sail through as healthy. + * + * Three outcomes: + * + * - **canonical storage** - the bundled Postgres provider. Verified, and the + * canonical counts are its counts; no query is issued. + * - **no `count`** - a provider that offers no diagnostics. Unverified, which is + * reported as such rather than turned into `healthy: true`. + * - **it threw** - Elasticsearch is down. Verified and unhealthy, with the + * reason attached, and the status route still answers: a diagnostic that + * crashes when the thing it diagnoses is broken is a diagnostic nobody can + * use. A failure in either call is handled the same way. + */ +const providerCounts = async ( + c: Context, + { + canonicalTotal, + contentTypeId, + locales, + }: { canonicalTotal: number; contentTypeId: string; locales: string[] }, +): Promise<{ + byLocale: Map<string, number>; + canonical: boolean; + error?: string; + name: string; + total: null | number; + verified: boolean; +}> => { + const search = c.get("search"); + const name = search.name(); + + if (search.isCanonicalStorage()) { + return { + byLocale: new Map(), + canonical: true, + name, + total: canonicalTotal, + verified: true, + }; + } + + const unverified = { + byLocale: new Map<string, number>(), + canonical: false, + name, + total: null, + verified: false, + }; + + const byLocale = new Map<string, number>(); + try { + // Every locale, including ones nothing here knows the name of. + const total = await search.countDocuments({ itemType: contentTypeId }); + if (total === null) return unverified; + + for (const locale of locales) { + const counted = await search.countDocuments({ + itemType: contentTypeId, + // The empty locale is what a language-agnostic document is stored + // under, and it is a real value to filter on rather than "any". + languageCode: locale, + }); + if (counted === null) return unverified; + byLocale.set(locale, counted); + } + + return { byLocale, canonical: false, name, total, verified: true }; + } catch (error) { + const message = error instanceof Error ? error.message : "unknown error"; + await logDiagnosticFailure( + c, + `${CONTENT_DIAGNOSTICS_LOG_PREFIX} ${JSON.stringify({ + contentTypeId, + error: message, + provider: name, + })}`, + ); + + return { + byLocale: new Map(), + canonical: false, + error: message, + name, + total: null, + verified: true, + }; + } +}; + +/** The prefix a provider diagnostic failure is logged behind. */ +export const CONTENT_DIAGNOSTICS_LOG_PREFIX = "[content-diagnostics]"; + +/** Logging is best effort: it writes to the database, so it can fail too. */ +const logDiagnosticFailure = async ( + c: Context, + message: string, +): Promise<void> => { + try { + await c.get("log").error(message); + } catch { + // eslint-disable-next-line no-console + console.error(`[VitNode] ${message}`); + } +}; + +/** + * How many documents this content type's rows *should* produce, per locale. + * + * `publication` is what makes a row indexable at all, so a content type without + * it has nothing to expect and returns an empty map rather than counting every + * draft - the indexer would not have written them either. + */ +const expectedDocuments = async ( + c: Context, + model: RegisteredContentModel["model"], +): Promise<Map<string, number>> => { + const { definition } = model; + const columns = model.columns; + + if (!definition.publication.enabled) return new Map(); + + const published = publicationColumns(definition, columns); + + if (!definition.localization.enabled) { + const [row] = await c + .get("db") + .select({ value: count() }) + .from(model.table) + .where(publishedCondition(published)); + + // The empty locale, because that is the `languageCode` a non-localized + // document is stored under - not a missing value, an actual `''`. + return new Map([["", row?.value ?? 0]]); + } + + const translationTable: null | PgTable = model.translationTable; + const translationColumns: null | Record<string, PgColumn> = + model.translationColumns; + if (!translationTable || !translationColumns) return new Map(); + + const translation = contentTranslationPublicationColumns( + definition, + translationColumns, + ); + + const rows = await c + .get("db") + .select({ + languageId: translationColumns.languageId, + value: count(), + }) + .from(translationTable) + .innerJoin(model.table, eq(translationColumns.itemId, columns.id)) + .where(and(publishedCondition(published), publishedCondition(translation))) + .groupBy(translationColumns.languageId); + + const languages = await listContentLanguages(c); + const localeOf = new Map( + languages.map(language => [language.id, language.locale]), + ); + + const byLocale = new Map<string, number>(); + for (const row of rows) { + const locale = localeOf.get(row.languageId as number); + // A translation whose language row is gone indexes under no locale, so it is + // expected to produce no document - which is exactly what the indexer does + // with it. Counting it here would report permanent drift nothing can repair. + if (locale === undefined) continue; + + const key = normalizeContentLocale(locale); + byLocale.set(key, (byLocale.get(key) ?? 0) + row.value); + } + + return byLocale; +}; + +export interface ContentScheduleHealth { + /** + * Transitions that committed but whose announcements have not been delivered. + * + * The number worth alerting on: the record *is* published, and nobody has been + * told. The effects task retries on the queue's backoff, so a non-zero value + * that stays non-zero is an outage rather than a blip. + */ + failedEffects: number; + /** Bookings still waiting to fire. */ + pending: number; + /** Pending bookings whose last run threw. */ + withErrors: number; +} + +/** + * Schedule health for a set of content types, in one query. + * + * Grouped rather than looped: an install with thirty schedulable content types + * should cost one aggregate, not thirty round trips. + */ +export const contentScheduleHealth = async ( + c: Context, + contentTypeIds: readonly string[], +): Promise<Map<string, ContentScheduleHealth>> => { + const result = new Map<string, ContentScheduleHealth>(); + if (contentTypeIds.length === 0) return result; + + const rows = await c + .get("db") + .select({ + contentTypeId: core_content_schedules.contentTypeId, + failedEffects: sql<number>`count(*) filter (where ${core_content_schedules.effectsError} is not null)::int`, + pending: sql<number>`count(*) filter (where ${core_content_schedules.status} = 'pending')::int`, + withErrors: sql<number>`count(*) filter (where ${core_content_schedules.status} = 'pending' and ${core_content_schedules.lastError} is not null)::int`, + }) + .from(core_content_schedules) + .where(inArray(core_content_schedules.contentTypeId, [...contentTypeIds])) + .groupBy(core_content_schedules.contentTypeId); + + for (const row of rows) { + result.set(row.contentTypeId, { + failedEffects: row.failedEffects, + pending: row.pending, + withErrors: row.withErrors, + }); + } + + return result; +}; + +export interface ContentTypeDiagnostic { + contentTypeId: string; + features: { + editorial: boolean; + localization: boolean; + publicApi: boolean; + publication: boolean; + scheduling: boolean; + search: boolean; + }; + pluginId: string; + /** `null` for a content type without scheduling, which books nothing. */ + schedules: ContentScheduleHealth | null; + /** `null` for a content type without `search`, which indexes nothing. */ + search: ContentSearchDrift | null; +} + +export interface ContentEngineDiagnostics { + contentTypes: ContentTypeDiagnostic[]; + /** + * Whether anything scheduled committed and was never announced. + * + * A *pending* schedule is normal - it has not fired yet - and so is a pending + * one whose last attempt threw, because the transition has not happened and + * the queue is still retrying it. `effectsError` is the one that matters: the + * record **is** published and nobody was told, and no amount of waiting fixes + * it on its own. + */ + effectsHealthy: boolean; + /** + * `searchHealthy && effectsHealthy`. + * + * Explicit dimensions rather than one number, because `healthy: true` beside + * `failedEffects: 15` is worse than no answer - it tells an operator to stop + * looking. + */ + healthy: boolean; + /** + * Whether every searchable content type agrees with the database - in the + * canonical table **and** in the active provider. + * + * A provider that offers no diagnostics leaves this `false`: unverified is not + * healthy, and the per-content-type `provider.verified` says which it was. + */ + searchHealthy: boolean; +} + +/** + * One pass over every registered content type. + * + * Sorted by id so two calls - and two processes - report the same order, which + * is what makes a diff between them readable. + */ +export const contentEngineDiagnostics = async ( + c: Context, +): Promise<ContentEngineDiagnostics> => { + const registered = [...(c.get("core")?.contentModels ?? [])].sort((a, b) => + a.model.definition.id.localeCompare(b.model.definition.id), + ); + + const schedulable = registered + .filter(entry => entry.model.definition.editorial.scheduling.enabled) + .map(entry => entry.model.definition.id); + const schedules = await contentScheduleHealth(c, schedulable); + + const contentTypes: ContentTypeDiagnostic[] = []; + for (const entry of registered) { + const { definition } = entry.model; + + contentTypes.push({ + contentTypeId: definition.id, + features: { + editorial: definition.editorial.enabled, + localization: definition.localization.enabled, + publicApi: definition.publicApi.enabled, + publication: definition.publication.enabled, + scheduling: definition.editorial.scheduling.enabled, + search: definition.search.enabled, + }, + pluginId: entry.pluginId, + search: definition.search.enabled + ? await contentSearchDrift(c, entry) + : null, + schedules: definition.editorial.scheduling.enabled + ? (schedules.get(definition.id) ?? { + failedEffects: 0, + pending: 0, + withErrors: 0, + }) + : null, + }); + } + + const searchHealthy = contentTypes.every( + entry => entry.search === null || entry.search.healthy, + ); + const effectsHealthy = contentTypes.every( + entry => (entry.schedules?.failedEffects ?? 0) === 0, + ); + + return { + contentTypes, + effectsHealthy, + healthy: searchHealthy && effectsHealthy, + searchHealthy, + }; +}; diff --git a/packages/vitnode/src/content/server/editorial-effects.test.ts b/packages/vitnode/src/content/server/editorial-effects.test.ts index f1c22dd3a..797d5d51f 100644 --- a/packages/vitnode/src/content/server/editorial-effects.test.ts +++ b/packages/vitnode/src/content/server/editorial-effects.test.ts @@ -47,18 +47,44 @@ const harness = ({ failures: [], status: "delivered", }), -}: { contextPlugin?: string; emit?: ReturnType<typeof vi.fn> } = {}) => { + log = vi.fn().mockResolvedValue(undefined), +}: { + contextPlugin?: string; + emit?: ReturnType<typeof vi.fn>; + log?: ReturnType<typeof vi.fn>; +} = {}) => { const store: Record<string, unknown> = { events: { emit }, + // The effects layer writes post-commit failures here. Present on the + // harness because a missing logger is itself a tested fallback, not the + // shape a real request has. + log: { error: log }, plugin: { id: contextPlugin }, }; return { c: { get: (key: string) => store[key] } as unknown as Context, emit, + log, }; }; +/** An emit result with one dead listener on it. */ +const withFailure = () => + vi.fn().mockResolvedValue({ + delivered: 0, + eventId: "event-1", + failures: [ + { + error: "Service unavailable", + listener: "send-notification", + module: "notifications", + pluginId: OWNER, + }, + ], + status: "delivered", + }); + beforeEach(() => { vi.clearAllMocks(); syncContentSearch.mockResolvedValue({ @@ -200,4 +226,94 @@ describe("contentEditorialEffects", () => { }); }); }); + + /** + * A dead listener on an *interactive* mutation has nowhere else to be + * recorded: the scheduled path writes it onto the schedule row and retries, + * and a clicked publish does neither. Without a log line it is invisible. + */ + describe("reporting a delivery failure", () => { + it("logs the failed listener with the record it belongs to", async () => { + const { c, log } = harness({ emit: withFailure() }); + + await contentEditorialEffects( + c, + testEditorialPostContentType, + outcome(), + { pluginId: OWNER }, + ); + + expect(log).toHaveBeenCalledTimes(1); + const message = String(log.mock.calls[0][0]); + expect(message).toContain("[content-effects]"); + expect(message).toContain(testEditorialPostContentType.id); + expect(message).toContain('"itemId":7'); + expect(message).toContain("send-notification"); + expect(message).toContain("Service unavailable"); + }); + + it("names the action, so a failed publish is not read as a failed edit", async () => { + const { c, log } = harness({ emit: withFailure() }); + + await contentEditorialEffects( + c, + testEditorialPostContentType, + outcome({ operation: "unpublish" }), + { pluginId: OWNER }, + ); + + expect(String(log.mock.calls[0][0])).toContain('"action":"unpublished"'); + }); + + it("logs nothing when every listener received it", async () => { + // An expected success is not an error, and a log full of them is a log + // nobody reads. + const { c, log } = harness(); + + await contentEditorialEffects( + c, + testEditorialPostContentType, + outcome(), + { pluginId: OWNER }, + ); + + expect(log).not.toHaveBeenCalled(); + }); + + it("does not fail the mutation when the logger itself is down", async () => { + // The logger writes to the database, so it can fail for the same reason + // the transport did - and the write has already committed either way. + const { c } = harness({ + emit: withFailure(), + log: vi.fn().mockRejectedValue(new Error("core_logs unreachable")), + }); + const console_ = vi + .spyOn(console, "error") + .mockImplementation(() => undefined); + + const result = await contentEditorialEffects( + c, + testEditorialPostContentType, + outcome(), + { pluginId: OWNER }, + ); + + expect(result.event?.failures).toHaveLength(1); + expect(console_).toHaveBeenCalled(); + console_.mockRestore(); + }); + + it("still writes the search document after reporting the failure", async () => { + const { c } = harness({ emit: withFailure() }); + + await contentEditorialEffects( + c, + testEditorialPostContentType, + outcome(), + { pluginId: OWNER }, + ); + + expect(syncContentSearch).toHaveBeenCalledTimes(1); + }); + }); }); diff --git a/packages/vitnode/src/content/server/editorial-effects.ts b/packages/vitnode/src/content/server/editorial-effects.ts index 536551c3b..9913006e7 100644 --- a/packages/vitnode/src/content/server/editorial-effects.ts +++ b/packages/vitnode/src/content/server/editorial-effects.ts @@ -3,10 +3,13 @@ import type { Context } from "hono"; import type { EventEmitResult } from "../../api/models/events"; import type { ContentEventAction } from "../events"; import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliveryEffectsResult } from "./delivery-effects"; import type { ContentEditorialOutcome } from "./editorial-service"; import type { AnyContentModel } from "./model"; import type { ContentSearchSyncOutcome } from "./search-sync"; +import { contentDeliveryEffects } from "./delivery-effects"; +import { reportContentEventFailures } from "./effects-log"; import { emitContentEvent } from "./emit"; import { contentSearchAdvancedValues, @@ -101,6 +104,12 @@ export interface ContentEditorialEffectsOptions { } export interface ContentEditorialEffectsResult { + /** + * The delivery events this mutation emitted, or `undefined` for a content type + * without `delivery` - which is what keeps every existing caller's result shape + * unchanged. + */ + delivery?: ContentDeliveryEffectsResult; /** * What the event transport reported. `null` for a no-op outcome, which emits * nothing at all. @@ -160,11 +169,33 @@ export const contentEditorialEffects = async ( { pluginId }, ); + // Logged here rather than left to each caller. The scheduled path additionally + // records the failure on the schedule row and retries; an interactive route + // does neither, and without this a dead listener on a clicked publish would be + // invisible everywhere. The write has committed either way, so the response + // stays a success - see `reportContentEventFailures`. + await reportContentEventFailures(c, { + action: EVENT_ACTION[outcome.operation], + contentTypeId: definition.id, + event, + itemId: idOf(outcome.row), + }); + + // After the ordinary event, never instead of it: a URL moving and a field moving + // are two facts, and a listener that mirrors content wants the first while one + // that warms a CDN wants the second. + const delivery = definition.delivery.enabled + ? await contentDeliveryEffects(c, definition, outcome.delivery, { + pluginId, + }) + : undefined; + // A localized record is indexed once per published translation, and a mutation // of the *record* moves every one of them: its publication state gates them // all, and a shared field is in all of them. if (definition.localization.enabled && definition.search.enabled) { return { + ...(delivery === undefined ? {} : { delivery }), event, search: null, searchByLocale: model @@ -188,6 +219,7 @@ export const contentEditorialEffects = async ( } return { + ...(delivery === undefined ? {} : { delivery }), event, search: await syncContentSearch(c, definition, { // Read back only when a document is actually made of collection values, diff --git a/packages/vitnode/src/content/server/editorial-service.ts b/packages/vitnode/src/content/server/editorial-service.ts index a18f76114..a0d3ccb93 100644 --- a/packages/vitnode/src/content/server/editorial-service.ts +++ b/packages/vitnode/src/content/server/editorial-service.ts @@ -24,10 +24,12 @@ import type { ContentValuesOf, } from "../types"; import type { ContentAdvancedStore } from "./advanced-store"; +import type { ContentDeliveryOutcome } from "./delivery-writes"; import type { ContentRevisionsModel } from "./revisions-model"; import type { ContentSchedulesModel } from "./schedules-model"; import type { ContentDatabase } from "./service"; +import { isContentPubliclyVisible } from "../cache"; import { CONTENT_EDITORIAL_FIELDS, CONTENT_PUBLICATION_FIELDS, @@ -45,6 +47,10 @@ import { buildContentRepeatableOperations, contentCollectionKinds, } from "./collection-api"; +import { + applyContentDeliveryWrite, + contentSlugHistoryFor, +} from "./delivery-writes"; import { changedPathsToColumns, diffChangedPaths, @@ -70,6 +76,13 @@ export interface ContentEditorialOutcome<TDefinition> { changed: boolean; /** Canonical paths - see {@link ContentUpdateResult.changedFields}. */ changedFields: ContentChangedPath<TDefinition>[]; + /** + * What this mutation did to the record's public URLs, or absent. + * + * Absent for every content type without `delivery` - which is every Stage 1-7 + * one - so the outcome those produce is byte-identical to what it always was. + */ + delivery?: ContentDeliveryOutcome; operation: ContentRevisionOperation; /** The slug the record answered to *before* this mutation, if it has one. */ previousSlug: null | string; @@ -338,6 +351,79 @@ export const createContentEditorialService = < const schedules = definition.editorial.scheduling.enabled ? createContentSchedulesModel({ c, definition, pluginId }) : undefined; + + // Only a **shared** slug is this service's business. A localized slug is a column + // on the translation table, so a base-row mutation cannot move it and + // `translation-editorial-service` owns its history - which is also why a localized + // content type's redirects are per language while a shared slug's are not. + const deliveryEnabled = + definition.delivery.enabled && definition.delivery.slugScope === "shared"; + const slugHistory = contentSlugHistoryFor({ c, definition, pluginId }); + + /** Whether a base row is publicly reachable right now. `false` without publication. */ + const publiclyVisible = (row: null | Record<string, unknown>): boolean => { + if (row === null || !publication) return false; + + return isContentPubliclyVisible({ + publishedAt: row.publishedAt as Date | null | undefined, + status: typeof row.status === "string" ? row.status : undefined, + }); + }; + + /** + * The publication state a row held *before* a transition. + * + * Reconstructed rather than re-read, and it is not a guess: a transition is + * guarded on the state it changes, so a `publish` that returned a row can only + * have found it unpublished, and an `unpublish` can only have found it published. + * A second `SELECT` would race with the writer that just won. + */ + const invert = ( + operation: "publish" | "unpublish", + row: Record<string, unknown>, + ): Record<string, unknown> => ({ + publishedAt: row.publishedAt, + status: operation === "publish" ? "draft" : "published", + }); + + /** + * The delivery half of one mutation, inside its transaction. + * + * `undefined` for a content type without delivery, which is what keeps every + * Stage 1-7 outcome byte-identical - and `before`/`after` are the rows the caller + * already holds on each side of its own guarded write, never a re-read. + */ + const applyDelivery = async ( + tx: ContentDatabase, + { + after, + before, + itemId, + }: { + after: null | Record<string, unknown>; + before: null | Record<string, unknown>; + itemId: number; + }, + ): Promise<ContentDeliveryOutcome | undefined> => { + if (!deliveryEnabled) return undefined; + + return await applyContentDeliveryWrite({ + definition, + slugHistory, + transition: { + isPublic: publiclyVisible(after), + itemId, + // A shared slug belongs to no single language: one reservation covers every + // locale the record appears in, because they all answer to the same segment. + languageId: null, + locale: null, + previousSlug: slugOf(before), + slug: slugOf(after), + wasPublic: publiclyVisible(before), + }, + tx, + }); + }; const { withCreateSlugs, withUpdateSlugs } = createSlugNormalizer( contentTypeId, fields, @@ -553,9 +639,20 @@ export const createContentEditorialService = < version, }); + // The moment an address becomes - or stops being - publicly addressable, which + // is exactly when a slug earns its reservation. A publish reserves the current + // slug; an unpublish leaves the history where it is, because the record is + // coming back and its old URLs should redirect again when it does. + const delivery = await applyDelivery(tx, { + after: row, + before: { ...row, ...invert(operation, row) }, + itemId: id, + }); + return { changed: true, changedFields: [], + ...(delivery === undefined ? {} : { delivery }), operation, previousSlug: slugOf(row), restoredFromRevisionId: null, @@ -713,9 +810,19 @@ export const createContentEditorialService = < version, }); + // A created row is a draft, so it reserves nothing - but its slug is still + // checked against the reservations, because "that address belongs to an + // article that moved" is far better heard now than at publish time. + const delivery = await applyDelivery(tx, { + after: row, + before: null, + itemId: typeof row.id === "number" ? row.id : 0, + }); + return { changed: true, changedFields: allPaths, + ...(delivery === undefined ? {} : { delivery }), operation: "create", previousSlug: null, restoredFromRevisionId: null, @@ -776,9 +883,20 @@ export const createContentEditorialService = < version, }); + // History is deliberately **kept**: an incoming link to a deleted article is + // exactly the diagnostic somebody will want, and the resolver answers 404 + // for it by reading the live record rather than by having forgotten the URL. + // So there is nothing to write here - only a sitemap that has lost a line. + const delivery = await applyDelivery(tx, { + after: null, + before: row, + itemId: id, + }); + return { changed: true, changedFields: [], + ...(delivery === undefined ? {} : { delivery }), operation: "delete", previousSlug: slugOf(row), restoredFromRevisionId: null, @@ -898,9 +1016,20 @@ export const createContentEditorialService = < version, }); + // A restore that brings an older slug back moves the canonical URL exactly + // like an edit does, so it retires the current address and redirects it at + // the restored one. A restore that changed no slug writes nothing - which is + // also why this runs after the guarded write rather than before it. + const delivery = await applyDelivery(tx, { + after: row, + before: current, + itemId: id, + }); + return { changed: true, changedFields, + ...(delivery === undefined ? {} : { delivery }), operation: "restore" as const, previousSlug: slugOf(current), restoredFromRevisionId: revisionId, @@ -1000,9 +1129,19 @@ export const createContentEditorialService = < version, }); + // After the guarded write, so the reservation is only taken by the writer + // that actually won the version race - a loser throws above and leaves the + // history exactly as it found it. + const delivery = await applyDelivery(tx, { + after: row, + before: current, + itemId: id, + }); + return { changed: true, changedFields, + ...(delivery === undefined ? {} : { delivery }), operation: "update" as const, previousSlug: slugOf(current), restoredFromRevisionId: null, diff --git a/packages/vitnode/src/content/server/effects-log.ts b/packages/vitnode/src/content/server/effects-log.ts new file mode 100644 index 000000000..993b9a626 --- /dev/null +++ b/packages/vitnode/src/content/server/effects-log.ts @@ -0,0 +1,81 @@ +import type { Context } from "hono"; + +import type { EventEmitResult } from "../../api/models/events"; + +/** + * The prefix every Content Engine post-commit failure is logged behind. + * + * Greppable on purpose, and distinct from `[content-search]`, which + * `syncContentSearch` already owns: an operator looking for "why did nobody + * hear about this publish" is asking a different question from "why is this + * article missing from search", and one prefix for both would make neither + * answerable. + */ +export const CONTENT_EFFECTS_LOG_PREFIX = "[content-effects]"; + +/** + * Reports listeners that did not receive an event whose mutation **has already + * committed**. + * + * `EventsModel.emit` reports rather than throws, so `failures` is the only place + * a dead listener or a broker outage is visible at all. Two things follow from + * the write having committed, and they are the whole contract: + * + * 1. **The request still succeeds.** The row is in the database; answering 500 + * would tell the client its edit was lost when it was not, and it would + * invite a retry that creates a second version of everything. + * 2. **The failure is never swallowed.** It goes to `core_logs` behind + * {@link CONTENT_EFFECTS_LOG_PREFIX} with the content type, the item and the + * listener that failed, so the AdminCP log viewer can find it and an operator + * can replay whatever the listener was meant to do. + * + * Delivery is **at-least-once** where a retry is involved (the scheduled effects + * task) and best-effort otherwise (an interactive route). There is no outbox and + * no exactly-once guarantee; a listener that must act once keys off the + * identifiers in the payload. + * + * A result with no failures logs nothing - an expected success is not an error, + * and a log full of them is a log nobody reads. + */ +export const reportContentEventFailures = async ( + c: Context, + { + action, + contentTypeId, + event, + itemId, + locale, + }: { + action: string; + contentTypeId: string; + event: EventEmitResult | null; + itemId: number; + /** Present only for a translation mutation. */ + locale?: string; + }, +): Promise<void> => { + if (!event || event.failures.length === 0) return; + + const message = `${CONTENT_EFFECTS_LOG_PREFIX} ${JSON.stringify({ + action, + contentTypeId, + delivered: event.delivered, + eventId: event.eventId, + failures: event.failures.map(failure => ({ + error: failure.error, + listener: `${failure.pluginId}:${failure.module}:${failure.listener}`, + })), + itemId, + ...(locale === undefined ? {} : { locale }), + })}`; + + try { + await c.get("log").error(message); + } catch { + // The logger writes to the database, so it can fail for the same reason the + // transport did. Both are best effort *after* a committed write, and neither + // may turn it into a failed request - so the console is the last resort. + // eslint-disable-next-line no-console + console.error(`[VitNode] ${message}`); + } +}; diff --git a/packages/vitnode/src/content/server/error-contracts.test.ts b/packages/vitnode/src/content/server/error-contracts.test.ts new file mode 100644 index 000000000..1105a74cf --- /dev/null +++ b/packages/vitnode/src/content/server/error-contracts.test.ts @@ -0,0 +1,548 @@ +// @vitest-environment node +import { HTTPException } from "hono/http-exception"; +import { describe, expect, it } from "vitest"; +import { ZodError } from "zod"; + +import { + ContentAdvancedInputError, + ContentDefaultTranslationRequired, + ContentDeliverySlugReserved, + ContentInputError, + ContentLanguageError, + ContentRevisionNotRestorable, + ContentScheduleError, + ContentTranslationExists, + ContentTranslationItemMissing, + ContentTranslationVersionConflict, + ContentVersionConflict, +} from "../errors"; +import { withHttpErrors } from "./http-errors"; +import { withTranslationHttpErrors } from "./translation-http-errors"; + +/** + * What a client is allowed to learn when a write fails. + * + * Two rules, and the second is the one that needs a test rather than a comment: + * + * 1. **Every expected failure has a stable contract** - a status, and for the + * ones a client has to branch on, a `code`. A caller cannot be asked to parse + * English, and it certainly cannot be asked to parse a SQLSTATE. + * 2. **Nothing internal crosses the boundary.** A driver error carries the + * constraint name, often the column, and sometimes the value that clashed. + * None of that may reach a response body - it is a schema description handed + * to whoever asked, and on a public route it is handed to anyone. + */ + +const CONTENT_TYPE_ID = "test.article"; + +/** The whole response, as a client would see it. */ +const responseOf = async ( + run: () => Promise<unknown>, + options: Parameters<typeof withHttpErrors>[2] = {}, +): Promise<{ body: string; status: number }> => { + try { + await withHttpErrors("update", run, { + contentTypeId: CONTENT_TYPE_ID, + ...options, + }); + } catch (error) { + if (!(error instanceof HTTPException)) throw error; + + const res = error.getResponse(); + + return { body: await res.text(), status: res.status }; + } + + throw new Error("Expected the write to fail."); +}; + +const translationResponseOf = async ( + run: () => Promise<unknown>, + action: "create" | "delete" | "read" | "update" = "update", +): Promise<{ body: string; status: number }> => { + try { + await withTranslationHttpErrors(action, run, { + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + locale: "pl", + }); + } catch (error) { + if (!(error instanceof HTTPException)) throw error; + + const res = error.getResponse(); + + return { body: await res.text(), status: res.status }; + } + + throw new Error("Expected the translation write to fail."); +}; + +/** A driver failure, in the shape Drizzle actually wraps one in. */ +const driverError = (code: string, detail: string) => + Object.assign(new Error("Failed query: insert into ..."), { + cause: Object.assign(new Error(detail), { + code, + constraint_name: "example_articles_code_key", + detail, + schema_name: "public", + table_name: "example_articles", + }), + }); + +const throwing = (error: unknown) => async () => { + await Promise.resolve(); + throw error; +}; + +describe("expected database failures map onto stable contracts", () => { + it.each([ + ["23505", 409, "unique violation"], + ["23503", 400, "foreign key violation on a write"], + ["23502", 400, "not-null violation"], + ["23001", 409, "restrict violation"], + ])("turns %s into %i (%s)", async (code, status) => { + const result = await responseOf( + throwing(driverError(code, "Key (code)=(guide-001) already exists.")), + ); + + expect(result.status).toBe(status); + }); + + it("reads the SQLSTATE through the wrapper Drizzle puts around it", async () => { + // `DrizzleQueryError.code` is undefined and the real error is on `cause`, so + // a mapper reading `error.code` alone would turn every constraint failure + // into a 500. + const bare = Object.assign(new Error("duplicate"), { code: "23505" }); + + await expect(responseOf(throwing(bare))).resolves.toMatchObject({ + status: 409, + }); + }); + + it("answers a delete blocked by a reference with 409, not 400", async () => { + // The same SQLSTATE means different things by verb: on a create it is "the + // thing you pointed at is gone", on a delete it is "something still points + // at this". + try { + await withHttpErrors( + "delete", + throwing(driverError("23503", "still referenced")), + { contentTypeId: CONTENT_TYPE_ID }, + ); + } catch (error) { + expect((error as HTTPException).status).toBe(409); + } + }); + + /** + * Postgres 18 reports an explicit `ON DELETE RESTRICT` as `23001` + * (restrict_violation) where earlier majors reported `23503`. Both have to map + * to the same 409, or upgrading the database would change an API contract. + */ + it("answers the same way on both Postgres codes for a blocked delete", async () => { + const statuses = await Promise.all( + ["23001", "23503"].map(async code => { + try { + await withHttpErrors( + "delete", + throwing(driverError(code, "still referenced")), + { contentTypeId: CONTENT_TYPE_ID }, + ); + } catch (error) { + return (error as HTTPException).status; + } + + return 0; + }), + ); + + expect(statuses).toEqual([409, 409]); + }); + + it("rethrows an unrecognised failure for the global handler", async () => { + // A 500 with nothing in it beats a guessed status: `app.onError` logs the + // detail and answers "Internal Server Error" in production. + const unknown = Object.assign(new Error("connection terminated"), { + code: "57P01", + }); + + await expect( + withHttpErrors("update", throwing(unknown), { + contentTypeId: CONTENT_TYPE_ID, + }), + ).rejects.toBe(unknown); + }); +}); + +describe("domain failures map onto their documented codes", () => { + it("answers a stale write with a structured version conflict", async () => { + const result = await responseOf( + throwing( + new ContentVersionConflict({ + contentTypeId: CONTENT_TYPE_ID, + currentVersion: 6, + expectedVersion: 4, + itemId: 7, + }), + ), + { structured: true }, + ); + + expect(result.status).toBe(409); + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_VERSION_CONFLICT", + contentTypeId: CONTENT_TYPE_ID, + currentVersion: 6, + expectedVersion: 4, + itemId: 7, + }); + }); + + it("answers a unique clash with a structured conflict on an editorial route", async () => { + const result = await responseOf( + throwing(driverError("23505", "Key (code)=(guide-001) already exists.")), + { itemId: 7, structured: true }, + ); + + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_UNIQUE_CONFLICT", + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + }); + }); + + /** + * A reserved historical address, which a `23505` could not have explained. + * + * Two constraints can refuse the same write - the live slug index and the + * history reservation - and the driver's code is identical for both. So the + * reservation is checked in the transaction and raised as a domain error, and + * this is the arm it lands on: a 409 that names the slug and the locale rather + * than a SQLSTATE the client would have to guess at. + */ + it("answers a reserved address with its own 409 code", async () => { + const result = await responseOf( + throwing( + new ContentDeliverySlugReserved({ + contentTypeId: CONTENT_TYPE_ID, + locale: null, + slug: "hello-world", + }), + ), + { itemId: 7, structured: true }, + ); + + expect(result.status).toBe(409); + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_DELIVERY_SLUG_RESERVED", + contentTypeId: CONTENT_TYPE_ID, + locale: null, + slug: "hello-world", + }); + }); + + it("keeps the reserved address out of the unique-clash arm", async () => { + // The two share a status and mean different things: a unique clash is + // "another record holds that address now", and this is "another record used + // to hold it and it still redirects there". A client that could not tell them + // apart could not word either one. + const result = await responseOf( + throwing( + new ContentDeliverySlugReserved({ + contentTypeId: CONTENT_TYPE_ID, + locale: "pl", + slug: "stary-slug", + }), + ), + { itemId: 7, structured: true }, + ); + + expect(JSON.parse(result.body)).toMatchObject({ + code: "CONTENT_DELIVERY_SLUG_RESERVED", + locale: "pl", + }); + expect(result.body).not.toContain("CONTENT_UNIQUE_CONFLICT"); + }); + + it("answers an unrestorable revision with 422 and the field names", async () => { + const result = await responseOf( + throwing( + new ContentRevisionNotRestorable({ + contentTypeId: CONTENT_TYPE_ID, + fields: ["category"], + revisionId: 12, + }), + ), + { structured: true }, + ); + + expect(result.status).toBe(422); + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_REVISION_NOT_RESTORABLE", + contentTypeId: CONTENT_TYPE_ID, + fields: ["category"], + revisionId: 12, + }); + }); + + it("answers a refused schedule with 400 and a code", async () => { + const result = await responseOf( + throwing( + new ContentScheduleError("That time has already passed.", { + code: "CONTENT_SCHEDULE_IN_PAST", + contentTypeId: CONTENT_TYPE_ID, + }), + ), + ); + + expect(result.status).toBe(400); + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_SCHEDULE_IN_PAST", + contentTypeId: CONTENT_TYPE_ID, + }); + }); + + it("answers a missing relation target with 400 and the ids the caller sent", async () => { + const result = await responseOf( + throwing( + new ContentAdvancedInputError({ + code: "CONTENT_RELATION_MISSING_TARGET", + contentTypeId: CONTENT_TYPE_ID, + field: "categories", + ids: [99], + message: + 'Relation "categories" references a record that no longer exists: 99.', + }), + ), + ); + + expect(result.status).toBe(400); + // The caller's own input echoed back - nothing internal in it. + expect(result.body).toContain("categories"); + expect(result.body).toContain("99"); + }); + + it("answers a repeatable child that belongs elsewhere with 400", async () => { + const result = await responseOf( + throwing( + new ContentAdvancedInputError({ + code: "CONTENT_REPEATABLE_UNKNOWN_CHILD", + contentTypeId: CONTENT_TYPE_ID, + field: "faq", + ids: [5], + message: + 'Repeatable "faq" was sent an entry that does not belong to this record: 5.', + }), + ), + ); + + expect(result.status).toBe(400); + }); + + it("answers invalid input with 400 and no issue tree", async () => { + const result = await responseOf( + throwing( + new ZodError([ + { + code: "too_small", + minimum: 3, + origin: "string", + path: ["title"], + message: "Too small", + }, + ]), + ), + ); + + expect(result.status).toBe(400); + expect(result.body).toBe("Invalid input data."); + expect(result.body).not.toContain("title"); + }); + + it("keeps a written-for-the-client input error readable", async () => { + const result = await responseOf( + throwing( + new ContentInputError('The slug for "title" normalises to nothing.', { + contentTypeId: CONTENT_TYPE_ID, + }), + ), + ); + + expect(result.status).toBe(400); + expect(result.body).toContain("normalises to nothing"); + }); +}); + +describe("translation failures keep their own union", () => { + it.each([ + [ + "a stale locale write", + new ContentTranslationVersionConflict({ + contentTypeId: CONTENT_TYPE_ID, + currentVersion: 5, + expectedVersion: 2, + itemId: 7, + locale: "pl", + }), + 409, + "CONTENT_TRANSLATION_VERSION_CONFLICT", + ], + [ + "deleting the default translation", + new ContentDefaultTranslationRequired({ + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + locale: "en", + }), + 409, + "CONTENT_DEFAULT_TRANSLATION_REQUIRED", + ], + [ + "a second translation in one locale", + new ContentTranslationExists({ + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + locale: "pl", + }), + 409, + "CONTENT_TRANSLATION_EXISTS", + ], + [ + "a locale this install switched off", + new ContentLanguageError({ + contentTypeId: CONTENT_TYPE_ID, + locale: "de", + reason: "disabled", + }), + 409, + "CONTENT_LANGUAGE_DISABLED", + ], + [ + // In the delivery union rather than translated into the translation one: + // the translation mapper rewrites every 409 the shared mapper produces + // into the unique-clash arm, so this has to be caught before it. + "a localized address another record's history owns", + new ContentDeliverySlugReserved({ + contentTypeId: CONTENT_TYPE_ID, + locale: "pl", + slug: "stary-slug", + }), + 409, + "CONTENT_DELIVERY_SLUG_RESERVED", + ], + ])("answers %s with %i and a code", async (_why, error, status, code) => { + const result = await translationResponseOf(throwing(error)); + + expect(result.status).toBe(status); + expect(JSON.parse(result.body)).toMatchObject({ code }); + }); + + it("answers an unknown locale with 404 rather than a conflict", async () => { + // "There is no such language" and "this install switched it off" want + // different answers: only the second is something an admin can undo. + const result = await translationResponseOf( + throwing( + new ContentLanguageError({ + contentTypeId: CONTENT_TYPE_ID, + locale: "zz", + reason: "missing", + }), + ), + ); + + expect(result.status).toBe(404); + }); + + it("answers a translation of a record that is gone with 404", async () => { + const result = await translationResponseOf( + throwing( + new ContentTranslationItemMissing({ + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + }), + ), + ); + + expect(result.status).toBe(404); + }); + + it("turns a localized unique clash into the translation union, with the locale", async () => { + const result = await translationResponseOf( + throwing( + driverError("23505", "Key (languageId, slug)=(2, hello) exists"), + ), + ); + + expect(result.status).toBe(409); + expect(JSON.parse(result.body)).toEqual({ + code: "CONTENT_TRANSLATION_UNIQUE_CONFLICT", + contentTypeId: CONTENT_TYPE_ID, + itemId: 7, + locale: "pl", + }); + }); +}); + +/** + * The regression that matters most, because its symptom is invisible: a response + * body that happens to contain the constraint name reads fine to a human and + * hands an attacker the schema. + */ +describe("no internal detail crosses the boundary", () => { + const LEAKS = [ + "example_articles_code_key", + "example_articles", + "public", + "23505", + "23503", + "insert into", + "Key (code)=(guide-001)", + ]; + + it.each(["23505", "23503", "23502", "23001"])( + "keeps the driver's detail out of the %s response", + async code => { + const result = await responseOf( + throwing(driverError(code, "Key (code)=(guide-001) already exists.")), + ); + + for (const leak of LEAKS) { + expect(result.body.toLowerCase()).not.toContain(leak.toLowerCase()); + } + }, + ); + + it("keeps it out of the structured editorial body too", async () => { + const result = await responseOf( + throwing(driverError("23505", "Key (code)=(guide-001) already exists.")), + { itemId: 7, structured: true }, + ); + + for (const leak of LEAKS) { + expect(result.body.toLowerCase()).not.toContain(leak.toLowerCase()); + } + }); + + it("keeps it out of the translation body", async () => { + const result = await translationResponseOf( + throwing(driverError("23505", "Key (slug)=(hello) already exists.")), + ); + + for (const leak of LEAKS) { + expect(result.body.toLowerCase()).not.toContain(leak.toLowerCase()); + } + }); + + it("never serialises an Error object into a body", async () => { + // A body of `{}` is what `JSON.stringify(new Error(...))` produces, and a + // body of `{"message":...,"stack":...}` is what a helpful serializer + // produces. Neither is a contract. + const result = await responseOf(throwing(driverError("23505", "boom")), { + itemId: 7, + structured: true, + }); + + expect(result.body).not.toContain("stack"); + expect(JSON.parse(result.body)).not.toHaveProperty("message"); + }); +}); diff --git a/packages/vitnode/src/content/server/http-errors.ts b/packages/vitnode/src/content/server/http-errors.ts index 361bd9a22..98db8ebac 100644 --- a/packages/vitnode/src/content/server/http-errors.ts +++ b/packages/vitnode/src/content/server/http-errors.ts @@ -1,11 +1,20 @@ import { HTTPException } from "hono/http-exception"; import { ZodError } from "zod"; -import type { ContentConflict, ContentUnprocessable } from "../conflicts"; +import type { + ContentConflict, + ContentDeliveryConflict, + ContentUnprocessable, +} from "../conflicts"; import type { ContentScheduleCode } from "../schedules"; -import { CONTENT_CONFLICT_CODES, CONTENT_UNPROCESSABLE_CODES } from "../const"; import { + CONTENT_CONFLICT_CODES, + CONTENT_DELIVERY_CODES, + CONTENT_UNPROCESSABLE_CODES, +} from "../const"; +import { + ContentDeliverySlugReserved, ContentInputError, ContentRevisionNotRestorable, ContentScheduleError, @@ -52,6 +61,19 @@ const jsonError = (status: 400 | 409 | 422, body: unknown): HTTPException => export const contentConflict = (body: ContentConflict): HTTPException => jsonError(409, body); +/** + * A structured 409, for a slug another record's URL history owns. + * + * 409 rather than 400: nothing about the request is malformed, the address is + * simply taken - by a URL that still redirects somewhere, which is a state of the + * system rather than a mistake in the payload. Its own body shape rather than a + * third arm of `zodContentConflict`, so a client generated before Stage 8 still + * parses the arms it knows. + */ +export const contentDeliveryConflict = ( + body: ContentDeliveryConflict, +): HTTPException => jsonError(409, body); + /** A structured 422, for a revision that no longer fits the content type. */ export const contentUnprocessable = ( body: ContentUnprocessable, @@ -104,6 +126,19 @@ export const rethrowAsHttpError = ( }); } + // Before the generic unique-violation mapping below, and before + // `ContentInputError`: a reserved address is a 409 that names the slug and the + // locale, where the driver's own `23505` cannot say which of the two constraints + // - the live slug index or the history reservation - refused the write. + if (error instanceof ContentDeliverySlugReserved) { + throw contentDeliveryConflict({ + code: CONTENT_DELIVERY_CODES.slugReserved, + contentTypeId: error.contentTypeId ?? contentTypeId ?? "", + locale: error.locale, + slug: error.slug, + }); + } + if (error instanceof ContentScheduleError) { throw contentScheduleRejected({ code: error.code, diff --git a/packages/vitnode/src/content/server/index.ts b/packages/vitnode/src/content/server/index.ts index 3e95553ac..47458c9f2 100644 --- a/packages/vitnode/src/content/server/index.ts +++ b/packages/vitnode/src/content/server/index.ts @@ -16,6 +16,46 @@ export { buildTranslationSystemColumns, } from "./column-builders"; export type { ColumnReferenceThunk } from "./column-builders"; +export { + readDeliveryAlternates, + readDeliveryAlternatesMany, +} from "./delivery-alternates"; +export { + contentDeliveryEffects, + contentDeliveryInvalidation, +} from "./delivery-effects"; +export type { ContentDeliveryEffectsResult } from "./delivery-effects"; +export { buildContentDeliveryRoutes } from "./delivery-routes"; +export { createContentDeliveryService } from "./delivery-service"; +export type { + ContentDeliveryMetadata, + ContentDeliveryReadOptions, + ContentDeliveryResolution, + ContentDeliveryService, + ContentDeliverySitemapArgs, +} from "./delivery-service"; +export { readContentDeliverySitemapPage } from "./delivery-sitemap"; +export type { ContentDeliverySitemapPage } from "./delivery-sitemap"; +export { + applyContentDeliveryWrite, + contentSlugHistoryFor, +} from "./delivery-writes"; +export type { + ContentDeliveryOutcome, + ContentDeliveryTransition, +} from "./delivery-writes"; +export { + contentEngineDiagnostics, + contentScheduleHealth, + contentSearchDrift, +} from "./diagnostics"; +export type { + ContentEngineDiagnostics, + ContentScheduleHealth, + ContentSearchDrift, + ContentSearchDriftLocale, + ContentTypeDiagnostic, +} from "./diagnostics"; export { contentEditorialEffects } from "./editorial-effects"; export type { ContentEditorialEffectsOptions, @@ -29,6 +69,10 @@ export type { ContentEditorialService, ContentEditorialWriteOptions, } from "./editorial-service"; +export { + CONTENT_EFFECTS_LOG_PREFIX, + reportContentEventFailures, +} from "./effects-log"; export { emitContentEvent } from "./emit"; export { contentConflict, @@ -179,6 +223,16 @@ export type { ContentServiceOptions, ContentUpdateResult, } from "./service"; +export { + contentSlugHistoryCurrentPaths, + contentSlugHistoryPath, + createContentSlugHistoryModel, +} from "./slug-history-model"; +export type { + ContentSlugHistoryEntry, + ContentSlugHistoryModel, + ContentSlugHistoryTarget, +} from "./slug-history-model"; export { createSlugNormalizer } from "./slugs"; export type { ContentSlugNormalizer } from "./slugs"; export { diff --git a/packages/vitnode/src/content/server/localized-preview-routes.test.ts b/packages/vitnode/src/content/server/localized-preview-routes.test.ts index f8147a0c7..958d88d3e 100644 --- a/packages/vitnode/src/content/server/localized-preview-routes.test.ts +++ b/packages/vitnode/src/content/server/localized-preview-routes.test.ts @@ -66,6 +66,12 @@ const harness = ({ secret = SECRET }: { secret?: string } = {}) => { create: vi.fn(), delete: vi.fn(), exists: vi.fn(), + // Stage 8 reads the base row's publication state to decide whether a + // translation's address is publicly reachable. Resolved as "published" so + // these suites keep exercising what they were written for. + findBasePublication: vi + .fn() + .mockResolvedValue({ publishedAt: new Date(0), status: "published" }), findByLanguageId: vi.fn(), findByLocale: vi.fn().mockResolvedValue(translationRow()), findManyForItem: vi.fn(), diff --git a/packages/vitnode/src/content/server/localized-public-service.ts b/packages/vitnode/src/content/server/localized-public-service.ts index 67cf47cb5..c13d30d26 100644 --- a/packages/vitnode/src/content/server/localized-public-service.ts +++ b/packages/vitnode/src/content/server/localized-public-service.ts @@ -10,6 +10,7 @@ import type { Context } from "hono"; import { and, eq, exists, not, or, sql } from "drizzle-orm"; import { alias } from "drizzle-orm/pg-core"; +import type { PaginationCursorSelection } from "../../api/lib/with-pagination"; import type { AnyContentTypeDefinition, ContentPublicSelect } from "../types"; import type { ContentAdvancedStore } from "./advanced-store"; import type { ContentLanguage } from "./language-resolver"; @@ -414,11 +415,20 @@ export const createContentLocalizedPublicService = < const read = async ( scope: ResolvedLocale, where: SQL | undefined, - { limit, order }: { limit: number; order?: SQL }, + { + cursorSelection, + limit, + order, + }: { + /** Only a paginated read asks for one; a single read has nothing to mint. */ + cursorSelection?: PaginationCursorSelection; + limit: number; + order?: SQL; + }, ): Promise<Record<string, unknown>[]> => { const query = c .get("db") - .select(selection(scope.fallbackTo !== null)) + .select({ ...selection(scope.fallbackTo !== null), ...cursorSelection }) .from(table); if (!scope.fallbackTo) { @@ -582,8 +592,14 @@ export const createContentLocalizedPublicService = < }, table, where, - query: async ({ limit, orderBy: order, where: paged }) => + query: async ({ + cursorSelection, + limit, + orderBy: order, + where: paged, + }) => await read(resolved, paged, { + cursorSelection, limit: typeof limit === "number" ? Math.min(limit, CONTENT_PUBLIC_MAX_PAGE_SIZE + 1) diff --git a/packages/vitnode/src/content/server/model.ts b/packages/vitnode/src/content/server/model.ts index 5329b1667..5c64fe47b 100644 --- a/packages/vitnode/src/content/server/model.ts +++ b/packages/vitnode/src/content/server/model.ts @@ -7,6 +7,7 @@ import type { ResolvedContentLocalizationConfig, } from "../types"; import type { ContentAdvancedStore } from "./advanced-store"; +import type { ContentDeliveryService } from "./delivery-service"; import type { ContentEditorialService } from "./editorial-service"; import type { ContentLocalizedService } from "./localized-service"; import type { ContentPublicService } from "./public-service"; @@ -25,6 +26,7 @@ import type { import { ContentEngineError } from "../errors"; import { createContentAdvancedStore } from "./advanced-store"; import { createContentAdvancedTables } from "./advanced-tables"; +import { createContentDeliveryService } from "./delivery-service"; import { createContentEditorialService } from "./editorial-service"; import { createContentLocalizedPublicService } from "./localized-public-service"; import { createContentLocalizedService } from "./localized-service"; @@ -68,6 +70,21 @@ export interface ContentModel<TDefinition extends AnyContentTypeDefinition> { /** Column name -> Drizzle column, for filters, ordering and custom queries. */ columns: Record<ContentColumnName<TDefinition>, PgColumn>; definition: TDefinition; + /** + * The read-only delivery layer, or `undefined` without a `delivery` block. + * + * `undefined` rather than a throwing stub, matching `publicService` and + * `editorialService`: the check reads naturally in code that does not know which + * content type it was handed, and TypeScript refuses the call until it has been + * made. + * + * `options.pluginId` is required because slug history is stamped with its owner - + * the same reason `editorialService` takes one, and `createContentModel` is + * called from `src/database/*.ts`, which has no reason to know it. + */ + deliveryService: + | ((c: Context, options: { pluginId: string }) => ContentDeliveryService) + | undefined; /** * The transactional editorial repository, or `undefined` when the content * type has no `editorial` block. @@ -273,11 +290,20 @@ export const createContentModel = < }); }; - return { + const model: ContentModel<TDefinition> = { advanced, advancedTables, columns, definition, + // Reads `model` lazily, which is what lets the delivery service be built from + // the finished model without a circular construction: it needs + // `publicService`, the table and the translation table, and every one of them + // is assigned by the time a request calls this. + deliveryService: + definition.delivery.enabled && definition.publicApi.enabled + ? (c: Context, { pluginId }: { pluginId: string }) => + createContentDeliveryService({ c, model, pluginId }) + : undefined, // The plugin id arrives at call time rather than being captured here: a // revision is stamped with its owner, and `createContentModel` is called // from `src/database/*.ts`, which does not otherwise need to know it. Every @@ -389,4 +415,6 @@ export const createContentModel = < translationService: localized ? buildTranslations : undefined, translationTable, }; + + return model; }; diff --git a/packages/vitnode/src/content/server/openapi-parity.test.ts b/packages/vitnode/src/content/server/openapi-parity.test.ts new file mode 100644 index 000000000..292b15d31 --- /dev/null +++ b/packages/vitnode/src/content/server/openapi-parity.test.ts @@ -0,0 +1,1072 @@ +// @vitest-environment node +import type { RouteConfig } from "@hono/zod-openapi"; +import type { MiddlewareHandler } from "hono"; + +import { OpenAPIHono } from "@hono/zod-openapi"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import type { JsonSchemaLike } from "@/tests/openapi-validate"; + +import { + testDeliveredPostContentType, + testEditorialPostContentType, + testLocalizedPageContentType, +} from "@/tests/content-fixtures"; +import { validateAgainstJsonSchema } from "@/tests/openapi-validate"; + +import { + ContentDefaultTranslationRequired, + ContentDeliverySlugReserved, + ContentRevisionNotRestorable, + ContentScheduleError, + ContentTranslationVersionConflict, + ContentVersionConflict, +} from "../errors"; +import { createContentModel } from "./model"; +import { buildContentPublicRoutes } from "./public-routes"; +import { buildContentRoutes } from "./routes"; + +/** + * The document says one thing; the runtime does another. + * + * Every generated route declares its responses in OpenAPI, and a generated + * client is built from exactly that. These tests serve the document the app + * really publishes and check the body the handler really produced against it - + * so a `409` that answers with prose where the document promises a + * discriminated union fails here rather than in somebody's generated client. + * + * Two halves, and both matter: + * + * 1. **the status is declared** - a runtime `409` on a route whose document + * lists only `200` and `404` is a contract break even when the body is fine; + * 2. **the body validates** - against the emitted JSON Schema rather than + * against the Zod object it came from. The two are not interchangeable: + * `z.date()` renders as `{ type: "string", format: "date-time" }`, which is + * exactly what `c.json(row)` sends and exactly what the Zod object rejects. + */ + +vi.mock("../../api/lib/check-staff-permission", () => ({ + assertStaffPermission: async () => await Promise.resolve(), +})); + +const posts = createContentModel(testEditorialPostContentType); +const localized = createContentModel(testLocalizedPageContentType); +const PLUGIN_ID = "@vitnode/example"; + +const adminUser = { + avatarColor: "000000", + birthday: null, + createdAt: new Date(), + email: "test@test.com", + emailVerified: true, + id: 1, + language: "en", + name: "Test", + nameCode: "test", + newsletter: false, + roleId: 1, +}; + +const row = { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + excerpt: null, + id: 7, + publishedAt: null, + slug: "hello-world", + status: "draft" as const, + title: "Hello world", + updatedAt: new Date("2026-01-01T00:00:00.000Z"), + version: 4, + views: 0, +}; + +const revision = { + actorName: null, + actorType: "staff" as const, + actorUserId: null, + changedFields: ["title"], + createdAt: new Date("2026-01-01T00:00:00.000Z"), + id: 3, + operation: "update" as const, + restoredFromRevisionId: null, + version: 4, +}; + +const outcome = (overrides: Record<string, unknown> = {}) => ({ + changed: true, + changedFields: ["title"], + operation: "update" as const, + previousSlug: null, + restoredFromRevisionId: null, + revisionId: 3, + row, + version: 5, + ...overrides, +}); + +const declaredStatuses = (route: RouteConfig): number[] => + Object.keys(route.responses).map(Number); + +interface Suite { + app: OpenAPIHono; + /** The served OpenAPI document, which is what a generated client is built from. */ + document: JsonSchemaLike; + routeOf: (method: string, path: string) => RouteConfig; +} + +/** + * The response schema the **document** publishes for one status. + * + * Not the Zod object the route was built from: `z.date()` renders as + * `{ type: "string", format: "date-time" }`, which is what the handler really + * sends, while the Zod object rejects that string outright. Reading the emitted + * document is the only way to check the contract a client actually consumes. + */ +const documentedSchema = ( + suite: Suite, + route: RouteConfig, + status: number, +): JsonSchemaLike | undefined => { + const paths = suite.document.paths as Record< + string, + Record<string, { responses?: Record<string, JsonSchemaLike> }> + >; + const operation = paths?.[route.path]?.[route.method.toLowerCase()]; + const response = operation?.responses?.[String(status)]; + const content = response?.content as + Record<string, { schema?: JsonSchemaLike }> | undefined; + + return content?.["application/json"]?.schema; +}; + +const mount = ( + built: { handler: unknown; route: RouteConfig }[], + events = { + emit: async () => await Promise.resolve({ delivered: 0, failures: [] }), + }, +): Suite => { + const app = new OpenAPIHono(); + const context: MiddlewareHandler = async (c, next) => { + c.set("admin", { user: adminUser }); + c.set("events", events as never); + c.set("log", { error: async () => await Promise.resolve() } as never); + await next(); + }; + app.use("*", context); + for (const { handler, route } of built) { + app.openapi(route, handler as never); + } + + return { + app, + document: app.getOpenAPIDocument({ + info: { title: "Content Engine", version: "1" }, + openapi: "3.0.0", + }) as unknown as JsonSchemaLike, + routeOf: (method, path) => { + const found = built.find( + entry => + entry.route.method.toUpperCase() === method.toUpperCase() && + entry.route.path === path, + ); + if (!found) throw new Error(`No route for ${method} ${path}.`); + + return found.route; + }, + }; +}; + +/** + * Drives one request and holds the schema its declared status published. + * + * The assertion is deliberately in one helper: "the status is in the document + * and the body parses against it" is the whole contract, and stating it + * twenty-odd times by hand is twenty-odd chances to state it slightly + * differently. + */ +const expectParity = async ( + suite: Suite, + { + body, + expected, + method, + path, + template, + }: { + body?: unknown; + expected: number; + method: string; + path: string; + /** The OpenAPI path, when it differs from the concrete one. */ + template: string; + }, +): Promise<unknown> => { + const res = await suite.app.request(path, { + method, + ...(body === undefined + ? {} + : { + body: JSON.stringify(body), + headers: { "Content-Type": "application/json" }, + }), + }); + + expect([`${method} ${path}`, res.status]).toEqual([ + `${method} ${path}`, + expected, + ]); + + const route = suite.routeOf(method, template); + expect([ + `${method} ${template}`, + declaredStatuses(route).includes(expected), + ]).toEqual([`${method} ${template}`, true]); + + const schema = documentedSchema(suite, route, expected); + if (!schema) return undefined; + + const payload: unknown = await res.json(); + + // The failure message has to name the route, or a red suite says only "one + // of the thirty contracts is wrong". + expect([ + `${method} ${template} -> ${expected}`, + validateAgainstJsonSchema(payload, schema, suite.document), + ]).toEqual([`${method} ${template} -> ${expected}`, []]); + + return payload; +}; + +const adminService = () => ({ + advanced: vi.fn(), + advancedFields: vi.fn(), + create: vi.fn(), + delete: vi.fn(), + findById: vi.fn().mockResolvedValue(row), + findDetail: vi.fn(), + findMany: vi.fn().mockResolvedValue({ + edges: [{ ...row, labels: {} }], + pageInfo: { + count: 1, + // Opaque, as `withPagination` mints them: the ordered tuple, base64url'd. + endCursor: "eyJjb2x1bW4iOiJ2ZXJzaW9uIiwiaWQiOjcsInZhbHVlIjo0fQ", + hasNextPage: false, + hasPreviousPage: false, + startCursor: "eyJjb2x1bW4iOiJ2ZXJzaW9uIiwiaWQiOjcsInZhbHVlIjo0fQ", + totalCount: 1, + }, + }), + options: vi.fn().mockResolvedValue([]), + relations: {}, + repeatable: {}, + update: vi.fn(), +}); + +const editorialStub = () => ({ + create: vi.fn().mockResolvedValue(outcome({ operation: "create" })), + delete: vi.fn().mockResolvedValue(outcome({ operation: "delete" })), + publish: vi.fn().mockResolvedValue(outcome({ operation: "publish" })), + relations: {}, + repeatable: {}, + restore: vi.fn().mockResolvedValue(outcome({ operation: "restore" })), + revisions: { + findById: vi + .fn() + .mockResolvedValue({ ...revision, snapshot: { title: "x" } }), + latest: vi.fn().mockResolvedValue(revision), + list: vi.fn().mockResolvedValue({ + edges: [revision], + pageInfo: { endCursor: 4, hasNextPage: false }, + }), + }, + schedules: { + cancel: vi.fn().mockResolvedValue({ action: "publish" }), + listForItem: vi.fn().mockResolvedValue([]), + schedule: vi.fn().mockResolvedValue({ + generation: 1, + id: 55, + scheduledFor: new Date("2030-01-01T00:00:00.000Z"), + }), + }, + unpublish: vi.fn().mockResolvedValue(outcome({ operation: "unpublish" })), + update: vi.fn().mockResolvedValue(outcome()), +}); + +let editorial: ReturnType<typeof editorialStub>; +let service: ReturnType<typeof adminService>; + +const editorialSuite = (): Suite => { + service = adminService(); + editorial = editorialStub(); + vi.spyOn(posts, "service").mockReturnValue(service as never); + vi.spyOn( + posts as unknown as { editorialService: unknown }, + "editorialService", + "get", + ).mockReturnValue(() => editorial); + + return mount(buildContentRoutes(posts, { pluginId: PLUGIN_ID })); +}; + +beforeEach(() => { + vi.restoreAllMocks(); +}); + +describe("admin routes match their OpenAPI document", () => { + it("publishes nothing about pagination's own column", () => { + // `__cursorValue` is projected by every list query so a cursor can be + // minted from the same statement as the row. It is implementation, not + // contract: it is stripped before the handler sees a row, so it must not + // appear anywhere a generated client would find it. + expect(JSON.stringify(editorialSuite().document)).not.toContain( + "__cursorValue", + ); + }); + + it("lists", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "GET", + path: "/", + template: "/", + }); + }); + + it("reads one record", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "GET", + path: "/7", + template: "/{id}", + }); + }); + + it("answers 404 for a record that is not there", async () => { + const suite = editorialSuite(); + service.findById.mockResolvedValue(null); + + await expectParity(suite, { + expected: 404, + method: "GET", + path: "/7", + template: "/{id}", + }); + }); + + it("creates", async () => { + await expectParity(editorialSuite(), { + body: { title: "Hello world" }, + expected: 201, + method: "POST", + path: "/", + template: "/", + }); + }); + + it("rejects an invalid create with the declared 400", async () => { + await expectParity(editorialSuite(), { + body: { title: "no" }, + expected: 400, + method: "POST", + path: "/", + template: "/", + }); + }); + + it("updates", async () => { + await expectParity(editorialSuite(), { + body: { expectedVersion: 4, values: { title: "Hello again" } }, + expected: 200, + method: "PUT", + path: "/7", + template: "/{id}", + }); + }); + + it("answers a stale update with the documented 409 union", async () => { + const suite = editorialSuite(); + editorial.update.mockRejectedValue( + new ContentVersionConflict({ + contentTypeId: testEditorialPostContentType.id, + currentVersion: 6, + expectedVersion: 4, + itemId: 7, + }), + ); + + const body = await expectParity(suite, { + body: { expectedVersion: 4, values: { title: "Hello again" } }, + expected: 409, + method: "PUT", + path: "/7", + template: "/{id}", + }); + + // The discriminant, spelled out: a client branches on it, so a schema that + // merely admits a string would be a weaker contract than it looks. + expect(body).toMatchObject({ code: "CONTENT_VERSION_CONFLICT" }); + }); + + it("answers a unique clash with the same 409 union", async () => { + const suite = editorialSuite(); + editorial.update.mockRejectedValue( + Object.assign(new Error("duplicate"), { code: "23505" }), + ); + + const body = await expectParity(suite, { + body: { expectedVersion: 4, values: { title: "Hello again" } }, + expected: 409, + method: "PUT", + path: "/7", + template: "/{id}", + }); + + expect(body).toMatchObject({ code: "CONTENT_UNIQUE_CONFLICT" }); + }); + + it("publishes", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "POST", + path: "/7/publish", + template: "/{id}/publish", + }); + }); + + it("unpublishes", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "POST", + path: "/7/unpublish", + template: "/{id}/unpublish", + }); + }); + + it("deletes", async () => { + await expectParity(editorialSuite(), { + body: { expectedVersion: 4 }, + expected: 200, + method: "DELETE", + path: "/7", + template: "/{id}", + }); + }); + + it("lists revisions", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "GET", + path: "/7/revisions", + template: "/{id}/revisions", + }); + }); + + it("reads one revision with its snapshot", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "GET", + path: "/7/revisions/3", + template: "/{id}/revisions/{revisionId}", + }); + }); + + it("restores", async () => { + await expectParity(editorialSuite(), { + body: { expectedVersion: 4 }, + expected: 200, + method: "POST", + path: "/7/revisions/3/restore", + template: "/{id}/revisions/{revisionId}/restore", + }); + }); + + it("answers an unrestorable revision with the documented 422", async () => { + const suite = editorialSuite(); + editorial.restore.mockRejectedValue( + new ContentRevisionNotRestorable({ + contentTypeId: testEditorialPostContentType.id, + fields: ["title"], + revisionId: 3, + }), + ); + + const body = await expectParity(suite, { + body: { expectedVersion: 4 }, + expected: 422, + method: "POST", + path: "/7/revisions/3/restore", + template: "/{id}/revisions/{revisionId}/restore", + }); + + expect(body).toMatchObject({ + code: "CONTENT_REVISION_NOT_RESTORABLE", + fields: ["title"], + }); + }); + + it("lists schedules", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "GET", + path: "/7/schedules", + template: "/{id}/schedules", + }); + }); + + it("books a schedule", async () => { + await expectParity(editorialSuite(), { + body: { + action: "publish", + scheduledFor: new Date(Date.now() + 86_400_000).toISOString(), + }, + expected: 200, + method: "POST", + path: "/7/schedule", + template: "/{id}/schedule", + }); + }); + + it("answers a refused schedule with the documented 400 body", async () => { + const suite = editorialSuite(); + editorial.schedules.schedule.mockRejectedValue( + new ContentScheduleError("That time has already passed.", { + code: "CONTENT_SCHEDULE_IN_PAST", + contentTypeId: testEditorialPostContentType.id, + }), + ); + + const body = await expectParity(suite, { + body: { + action: "publish", + scheduledFor: new Date(Date.now() + 86_400_000).toISOString(), + }, + expected: 400, + method: "POST", + path: "/7/schedule", + template: "/{id}/schedule", + }); + + expect(body).toMatchObject({ code: "CONTENT_SCHEDULE_IN_PAST" }); + }); + + it("cancels a schedule", async () => { + await expectParity(editorialSuite(), { + expected: 200, + method: "POST", + path: "/7/schedule/5/cancel", + template: "/{id}/schedule/{scheduleId}/cancel", + }); + }); + + it("mints a preview link", async () => { + vi.stubEnv("CONTENT_PREVIEW_SECRET", "a".repeat(48)); + const suite = editorialSuite(); + + await expectParity(suite, { + expected: 200, + method: "POST", + path: "/7/preview", + template: "/{id}/preview", + }); + vi.unstubAllEnvs(); + }); +}); + +describe("translation routes match their OpenAPI document", () => { + const translationRow = { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + itemId: 7, + languageId: 1, + locale: "en", + publishedAt: null, + status: "draft" as const, + updatedAt: new Date("2026-01-01T00:00:00.000Z"), + values: { body: "Body", slug: "hello", title: "Hello" }, + version: 2, + }; + + const translationOutcome = (overrides: Record<string, unknown> = {}) => ({ + changed: true, + changedFields: ["title"], + languageId: 1, + locale: "en", + operation: "update" as const, + previousSlug: null, + restoredFromRevisionId: null, + revisionId: 9, + row: translationRow, + version: 3, + ...overrides, + }); + + let translations: Record<string, ReturnType<typeof vi.fn>>; + let translationEditorial: Record<string, unknown>; + + const suite = (): Suite => { + translations = { + exists: vi.fn().mockResolvedValue(true), + findByLanguageId: vi.fn().mockResolvedValue(translationRow), + findByLocale: vi.fn().mockResolvedValue(translationRow), + findManyForItem: vi.fn().mockResolvedValue([ + { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + itemId: 7, + languageId: 1, + locale: "en", + publishedAt: null, + status: "draft", + updatedAt: new Date("2026-01-01T00:00:00.000Z"), + version: 2, + }, + ]), + resolveDefaultLanguage: vi + .fn() + .mockResolvedValue({ id: 1, locale: "en" }), + resolveLanguage: vi.fn().mockResolvedValue({ id: 1, locale: "en" }), + }; + translationEditorial = { + create: vi + .fn() + .mockResolvedValue(translationOutcome({ operation: "create" })), + delete: vi + .fn() + .mockResolvedValue(translationOutcome({ operation: "delete" })), + publish: vi + .fn() + .mockResolvedValue(translationOutcome({ operation: "publish" })), + restore: vi + .fn() + .mockResolvedValue(translationOutcome({ operation: "restore" })), + findRevision: vi + .fn() + .mockResolvedValue({ ...revision, snapshot: { title: "x" } }), + listRevisions: vi.fn().mockResolvedValue({ + edges: [revision], + pageInfo: { endCursor: 2, hasNextPage: false }, + }), + unpublish: vi + .fn() + .mockResolvedValue(translationOutcome({ operation: "unpublish" })), + update: vi.fn().mockResolvedValue(translationOutcome()), + }; + + vi.spyOn( + localized as unknown as { translationService: unknown }, + "translationService", + "get", + ).mockReturnValue(() => translations); + vi.spyOn( + localized as unknown as { translationEditorialService: unknown }, + "translationEditorialService", + "get", + ).mockReturnValue(() => translationEditorial); + vi.spyOn(localized, "service").mockReturnValue(adminService() as never); + + return mount(buildContentRoutes(localized, { pluginId: PLUGIN_ID })); + }; + + it("lists the locales a record exists in", async () => { + await expectParity(suite(), { + expected: 200, + method: "GET", + path: "/7/translations", + template: "/{id}/translations", + }); + }); + + it("reads one translation", async () => { + await expectParity(suite(), { + expected: 200, + method: "GET", + path: "/7/translations/en", + template: "/{id}/translations/{locale}", + }); + }); + + it("creates a translation", async () => { + await expectParity(suite(), { + body: { values: { body: "Cześć", slug: "czesc", title: "Cześć" } }, + expected: 201, + method: "POST", + path: "/7/translations/pl", + template: "/{id}/translations/{locale}", + }); + }); + + it("updates a translation", async () => { + await expectParity(suite(), { + body: { expectedVersion: 2, values: { title: "Hello again" } }, + expected: 200, + method: "PUT", + path: "/7/translations/en", + template: "/{id}/translations/{locale}", + }); + }); + + it("answers a stale translation update with the translation 409 union", async () => { + const built = suite(); + (translationEditorial.update as ReturnType<typeof vi.fn>).mockRejectedValue( + new ContentTranslationVersionConflict({ + contentTypeId: testLocalizedPageContentType.id, + currentVersion: 5, + expectedVersion: 2, + itemId: 7, + locale: "en", + }), + ); + + const body = await expectParity(built, { + body: { expectedVersion: 2, values: { title: "Hello again" } }, + expected: 409, + method: "PUT", + path: "/7/translations/en", + template: "/{id}/translations/{locale}", + }); + + expect(body).toMatchObject({ + code: "CONTENT_TRANSLATION_VERSION_CONFLICT", + locale: "en", + }); + }); + + it("answers a default-translation delete with the documented 409", async () => { + const built = suite(); + (translationEditorial.delete as ReturnType<typeof vi.fn>).mockRejectedValue( + new ContentDefaultTranslationRequired({ + contentTypeId: testLocalizedPageContentType.id, + itemId: 7, + locale: "en", + }), + ); + + const body = await expectParity(built, { + body: { expectedVersion: 2 }, + expected: 409, + method: "DELETE", + path: "/7/translations/en", + template: "/{id}/translations/{locale}", + }); + + expect(body).toMatchObject({ + code: "CONTENT_DEFAULT_TRANSLATION_REQUIRED", + }); + }); + + it("lists one locale's revisions", async () => { + await expectParity(suite(), { + expected: 200, + method: "GET", + path: "/7/translations/en/revisions", + template: "/{id}/translations/{locale}/revisions", + }); + }); +}); + +describe("public routes match their OpenAPI document", () => { + const publicRow = { + excerpt: null, + publishedAt: new Date("2026-01-01T00:00:00.000Z"), + slug: "hello-world", + title: "Hello world", + }; + + const suite = ( + findBySlug: unknown = publicRow, + findById: unknown = publicRow, + ): Suite => { + vi.spyOn( + posts as unknown as { publicService: unknown }, + "publicService", + "get", + ).mockReturnValue(() => ({ + findById: async () => await Promise.resolve(findById), + findBySlug: async () => await Promise.resolve(findBySlug), + findMany: async () => + await Promise.resolve({ + edges: [publicRow], + pageInfo: { + count: 1, + // Opaque, as `withPagination` mints them. + endCursor: "eyJjb2x1bW4iOiJwdWJsaXNoZWRBdCIsImlkIjo3fQ", + hasNextPage: false, + hasPreviousPage: false, + startCursor: "eyJjb2x1bW4iOiJwdWJsaXNoZWRBdCIsImlkIjo3fQ", + totalCount: 1, + }, + }), + })); + + return mount(buildContentPublicRoutes(posts, { pluginId: PLUGIN_ID })); + }; + + it("publishes nothing about pagination's own column", () => { + expect(JSON.stringify(suite().document)).not.toContain("__cursorValue"); + }); + + it("lists", async () => { + await expectParity(suite(), { + expected: 200, + method: "GET", + path: "/", + template: "/", + }); + }); + + it("reads by slug", async () => { + await expectParity(suite(), { + expected: 200, + method: "GET", + path: "/hello-world", + template: "/{slug}", + }); + }); + + it("answers 404 for an unpublished slug", async () => { + await expectParity(suite(null), { + expected: 404, + method: "GET", + path: "/hello-world", + template: "/{slug}", + }); + }); +}); + +/** + * The Stage 8 routes, held to the same contract as everything above. + * + * They are the ones with the most to get wrong: a `Date` that has to leave as an + * ISO string, a discriminated union a frontend branches on to decide between + * rendering a page and issuing a 308, and a third arm on the editorial `409`. A + * generated client is built from the document, so each of those is a promise the + * handler has to keep rather than a schema that merely looks right. + */ +describe("delivery routes match their OpenAPI document", () => { + const delivered = createContentModel(testDeliveredPostContentType); + + const metadata = { + alternates: [], + canonicalPath: "/delivered-posts/hello-world", + hreflang: { languages: {} }, + isFallback: false, + itemId: 42, + locale: null, + openGraph: { description: "Prose", title: "Hello world" }, + requestedLocale: null, + robots: { follow: true, index: true }, + seo: { description: "Prose", title: "Hello world" }, + }; + + const deliveryStub = () => ({ + alternates: vi.fn().mockResolvedValue([]), + findById: vi.fn().mockResolvedValue(metadata), + history: vi.fn().mockResolvedValue([ + { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + itemId: 42, + languageId: null, + path: "/delivered-posts/hello-world", + retiredAt: null, + slug: "hello-world", + }, + { + createdAt: new Date("2025-12-01T00:00:00.000Z"), + itemId: 42, + languageId: null, + path: "/delivered-posts/old-address", + retiredAt: new Date("2026-01-01T00:00:00.000Z"), + slug: "old-address", + }, + ]), + resolvePath: vi.fn(), + resolveSlug: vi.fn().mockResolvedValue({ ...metadata, type: "content" }), + sitemap: vi.fn().mockResolvedValue({ + entries: [ + { + changeFrequency: "weekly", + itemId: 42, + // A `Date` in the service, an ISO string on the wire: exactly the pair + // this suite exists to keep honest. + lastModified: new Date("2026-01-02T03:04:05.000Z"), + locale: null, + path: "/delivered-posts/hello-world", + priority: 0.7, + }, + ], + nextCursor: null, + }), + }); + + let delivery: ReturnType<typeof deliveryStub>; + + /** The public delivery routes: resolve, item and sitemap. */ + const publicSuite = (): Suite => { + delivery = deliveryStub(); + vi.spyOn(delivered, "deliveryService", "get").mockReturnValue( + () => delivery, + ); + vi.spyOn( + delivered as unknown as { publicService: unknown }, + "publicService", + "get", + ).mockReturnValue(() => ({ + findById: vi.fn(), + findBySlug: vi.fn(), + findMany: vi.fn(), + })); + + return mount(buildContentPublicRoutes(delivered, { pluginId: PLUGIN_ID })); + }; + + /** The AdminCP delivery panel's route, plus the editorial routes around it. */ + const adminSuite = ( + editorialOverrides: Record<string, unknown> = {}, + ): Suite => { + delivery = deliveryStub(); + vi.spyOn(delivered, "deliveryService", "get").mockReturnValue( + () => delivery, + ); + vi.spyOn(delivered, "service").mockReturnValue(adminService() as never); + const editorial = { ...editorialStub(), ...editorialOverrides }; + vi.spyOn( + delivered as unknown as { editorialService: unknown }, + "editorialService", + "get", + ).mockReturnValue(() => editorial); + + return mount(buildContentRoutes(delivered, { pluginId: PLUGIN_ID })); + }; + + it("publishes nothing about pagination's own column", () => { + expect(JSON.stringify(publicSuite().document)).not.toContain( + "__cursorValue", + ); + expect(JSON.stringify(adminSuite().document)).not.toContain( + "__cursorValue", + ); + }); + + it("resolves a slug into the content arm", async () => { + const body = await expectParity(publicSuite(), { + expected: 200, + method: "GET", + path: "/delivery/resolve/hello-world", + template: "/delivery/resolve/{slug}", + }); + + expect(body).toMatchObject({ type: "content" }); + }); + + it("resolves a retired slug into the redirect arm", async () => { + const suite = publicSuite(); + delivery.resolveSlug.mockResolvedValue({ + location: "/delivered-posts/hello-world", + status: 308, + type: "redirect", + }); + + const body = await expectParity(suite, { + expected: 200, + method: "GET", + path: "/delivery/resolve/old-address", + template: "/delivery/resolve/{slug}", + }); + + // The discriminant a frontend branches on to issue a 308 rather than render. + expect(body).toMatchObject({ status: 308, type: "redirect" }); + }); + + it("answers an unknown slug with the not_found arm, still a 200", async () => { + const suite = publicSuite(); + delivery.resolveSlug.mockResolvedValue({ type: "not_found" }); + + const body = await expectParity(suite, { + expected: 200, + method: "GET", + path: "/delivery/resolve/nope", + template: "/delivery/resolve/{slug}", + }); + + expect(body).toStrictEqual({ type: "not_found" }); + }); + + it("reads one record's delivery metadata", async () => { + await expectParity(publicSuite(), { + expected: 200, + method: "GET", + path: "/delivery/item/42", + template: "/delivery/item/{id}", + }); + }); + + it("answers 404 for a record with no public version", async () => { + const suite = publicSuite(); + delivery.findById.mockResolvedValue(null); + + await expectParity(suite, { + expected: 404, + method: "GET", + path: "/delivery/item/42", + template: "/delivery/item/{id}", + }); + }); + + it("serves a sitemap page whose lastModified is the documented string", async () => { + const body = await expectParity(publicSuite(), { + expected: 200, + method: "GET", + path: "/delivery/sitemap", + template: "/delivery/sitemap", + }); + + expect(body).toMatchObject({ + entries: [{ lastModified: "2026-01-02T03:04:05.000Z" }], + nextCursor: null, + }); + }); + + it("serves the AdminCP delivery panel, dates and all", async () => { + const body = await expectParity(adminSuite(), { + expected: 200, + method: "GET", + path: "/7/delivery", + template: "/{id}/delivery", + }); + + // The storage columns behind a history row are not part of the contract, and + // the schema is closed, so the document validating is what proves it. + expect(body).toMatchObject({ + canonicalPath: "/delivered-posts/hello-world", + history: [{ slug: "hello-world" }, { slug: "old-address" }], + }); + expect(body).not.toHaveProperty("history.0.languageId"); + }); + + it("answers a reserved address with the documented 409 arm", async () => { + // The write fails the way a taken historical address fails: the slug is free + // on the live table and owned by another record's URL history. + const suite = adminSuite({ + update: vi.fn().mockRejectedValue( + new ContentDeliverySlugReserved({ + contentTypeId: testDeliveredPostContentType.id, + locale: null, + slug: "hello-world", + }), + ), + }); + + const body = await expectParity(suite, { + body: { expectedVersion: 4, values: { title: "Hello again" } }, + expected: 409, + method: "PUT", + path: "/7", + template: "/{id}", + }); + + // The third arm, and the reason it is a union rather than a replacement: a + // client generated before Stage 8 still parses the two it knows. + expect(body).toMatchObject({ + code: "CONTENT_DELIVERY_SLUG_RESERVED", + slug: "hello-world", + }); + }); +}); diff --git a/packages/vitnode/src/content/server/pagination-routes.test.ts b/packages/vitnode/src/content/server/pagination-routes.test.ts new file mode 100644 index 000000000..9909cae10 --- /dev/null +++ b/packages/vitnode/src/content/server/pagination-routes.test.ts @@ -0,0 +1,151 @@ +// @vitest-environment node +import type { MiddlewareHandler } from "hono"; + +import { OpenAPIHono } from "@hono/zod-openapi"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import { + testArticleContentType, + testCategoryContentType, +} from "@/tests/content-fixtures"; + +import { createContentModel } from "./model"; +import { buildContentRoutes } from "./routes"; + +/** + * What a list route does with pagination input it cannot honour. + * + * Every case here used to be answered rather than refused: `first=0` clamped + * its way into a one-row page that reported `hasNextPage: true`, `first=abc` + * became `NaN` and fell through to the default page size, and `first` and + * `last` together threw a bare `Error` that surfaced as a 500. Each of them is + * a request nobody made, answered as if they had. + * + * The schema catches most of them at the edge and `parsePaginationParams` + * catches the rest; both answer 400, which is the only thing a client has to + * know. + */ + +vi.mock("../../api/lib/check-staff-permission", () => ({ + assertStaffPermission: async () => await Promise.resolve(), +})); + +const categories = createContentModel(testCategoryContentType); +const articles = createContentModel(testArticleContentType, { + references: { category: () => categories.table.id }, +}); +const PLUGIN_ID = "@vitnode/example"; + +const adminUser = { + avatarColor: "000000", + birthday: null, + createdAt: new Date(), + email: "test@test.com", + emailVerified: true, + id: 1, + language: "en", + name: "Test", + nameCode: "test", + newsletter: false, + roleId: 1, +}; + +const harness = () => { + const findMany = vi.fn().mockResolvedValue({ + edges: [], + pageInfo: { + count: 0, + endCursor: null, + hasNextPage: false, + hasPreviousPage: false, + startCursor: null, + totalCount: 0, + }, + }); + vi.spyOn(articles, "service").mockReturnValue({ + findMany, + relations: {}, + repeatable: {}, + } as never); + + const app = new OpenAPIHono(); + const context: MiddlewareHandler = async (c, next) => { + c.set("admin", { user: adminUser }); + await next(); + }; + app.use("*", context); + for (const { handler, route } of buildContentRoutes(articles, { + pluginId: PLUGIN_ID, + })) { + app.openapi(route, handler); + } + + return { app, findMany }; +}; + +beforeEach(() => { + vi.restoreAllMocks(); +}); + +describe("pagination input a list route refuses", () => { + it.each([ + ["first=0", "first=0"], + ["last=0", "last=0"], + ["first=-1", "first=-1"], + ["last=-1", "last=-1"], + ["first=abc", "first=abc"], + ["last=abc", "last=abc"], + ["a fractional page size", "first=1.5"], + ["both first and last", "first=5&last=5"], + ["a garbage cursor", "cursor=%21%21not-a-cursor"], + ["an empty cursor", "cursor="], + ])("answers 400 for %s", async (_why, query) => { + const { app } = harness(); + + const res = await app.request(`/?${query}`); + + expect([query, res.status]).toEqual([query, 400]); + }); + + it("never reaches the service with a page size it would have to clamp", async () => { + const { app, findMany } = harness(); + + await app.request("/?first=0"); + + expect(findMany).not.toHaveBeenCalled(); + }); + + it("still accepts a legitimate page", async () => { + const { app, findMany } = harness(); + + const res = await app.request("/?first=25"); + + expect(res.status).toBe(200); + expect(findMany).toHaveBeenCalledWith( + expect.objectContaining({ + query: expect.objectContaining({ first: "25" }), + }), + ); + }); + + it("refuses a legacy numeric cursor on an ordering that is not the identifier", async () => { + // The exact shape of the old bug, refused where the ordering is known: a + // bare number says nothing about where `title` was, so honouring it would + // skip rows. The service raises it; the route passes it through unchanged. + const { app } = harness(); + const { HTTPException } = await import("hono/http-exception"); + vi.spyOn(articles, "service").mockReturnValue({ + findMany: vi.fn().mockImplementation(() => { + throw new HTTPException(400, { + message: 'This cursor cannot be used with the "title" ordering.', + }); + }), + relations: {}, + repeatable: {}, + } as never); + + const res = await app.request("/?orderBy=title&cursor=42"); + + expect(res.status).toBe(400); + }); +}); diff --git a/packages/vitnode/src/content/server/permission-matrix.test.ts b/packages/vitnode/src/content/server/permission-matrix.test.ts new file mode 100644 index 000000000..333051c1c --- /dev/null +++ b/packages/vitnode/src/content/server/permission-matrix.test.ts @@ -0,0 +1,422 @@ +// @vitest-environment node +import type { MiddlewareHandler } from "hono"; + +import { OpenAPIHono } from "@hono/zod-openapi"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import { defineContentType } from "../define"; +import { field } from "../fields"; +import { createContentModel } from "./model"; +import { buildContentRoutes } from "./routes"; + +/** + * Every generated route, against the permission it actually demands. + * + * The existing route suites check permissions one endpoint at a time, which is + * fine until somebody adds an endpoint. This one **enumerates** the routes the + * builder produced and drives each of them, so a new route cannot join the set + * without appearing in the matrix below - and a route with no + * `adminStaffPermission` at all cannot join it silently, because it would answer + * something other than 403 with every permission denied. + * + * The permission check itself is stubbed: it reads roles out of the database, + * and what is under test is which `(module, permission)` each route asks for. + */ + +/** Grants for the request currently in flight. `"module:permission"`. */ +let granted = new Set<string>(); +/** What each request was asked for, in order. */ +let asked: { module: string; permission: string; plugin: string }[] = []; + +vi.mock("../../api/lib/check-staff-permission", () => ({ + assertStaffPermission: async ( + _c: unknown, + args: { module: string; permission: string; plugin: string }, + ) => { + asked.push({ + module: args.module, + permission: args.permission, + plugin: args.plugin, + }); + if (granted.has(`${args.module}:${args.permission}`)) return; + + const { HTTPException } = await import("hono/http-exception"); + throw new HTTPException(403, { message: "Forbidden" }); + }, +})); + +/** + * Everything a content type can switch on, at once. + * + * A maximal fixture on purpose: the matrix is only as complete as the set of + * routes the builder was asked to produce, and a fixture missing `scheduling` + * would quietly drop three endpoints out of the audit. + */ +const kitchenSink = defineContentType({ + id: "test.everything", + tableName: "test_everything", + localization: { enabled: true, defaultLocale: "en", fallback: "default" }, + publication: { enabled: true }, + editorial: { + enabled: true, + revisions: { retention: 10 }, + preview: { enabled: true, expiresInMinutes: 30 }, + scheduling: { enabled: true }, + }, + fields: { + title: field.text({ localized: true, required: true, maxLength: 200 }), + slug: field.slug({ localized: true, source: "title" }), + featured: field.boolean({ defaultValue: false }), + // A reference field, so the picker route exists to be audited. + author: field.user(), + // A collection, so the advanced write path is exercised through the same + // `PUT` an ordinary field edit goes through. + faq: field.repeatable({ + fields: { + question: field.text({ required: true, maxLength: 200 }), + answer: field.textarea({ required: true }), + }, + }), + }, + publicApi: { + enabled: true, + path: "everything", + // `id` is exposed because delivery resolves alternates by identifier off the + // public projection, and a localized delivery content type is refused without + // it. + fields: ["id", "title", "slug", "featured", "publishedAt"], + orderableFields: ["publishedAt"], + }, + // Stage 8, so the delivery route is audited like every other one. `redirects` + // needs `editorial` and a localized slug, and this fixture has both - which is + // the whole reason it is the maximal one rather than a second fixture. + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { titleField: "title" }, + sitemap: { enabled: true }, + }, + admin: { + label: { plural: "Everythings", singular: "Everything" }, + list: { columns: ["featured", "status"] }, + form: { fields: ["faq"] }, + }, +}); + +const model = createContentModel(kitchenSink); +const MODULE = kitchenSink.permissionModule; +const PLUGIN_ID = "@vitnode/example"; + +const adminUser = { + avatarColor: "000000", + birthday: null, + createdAt: new Date(), + email: "test@test.com", + emailVerified: true, + id: 1, + language: "en", + name: "Test", + nameCode: "test", + newsletter: false, + roleId: 1, +}; + +/** Concrete values for the path parameters the generated routes declare. */ +const PARAMS: Record<string, string> = { + field: "author", + id: "7", + locale: "en", + revisionId: "3", + scheduleId: "5", +}; + +const concretePath = (template: string): string => + template.replace(/\{(\w+)\}/g, (_match, name: string) => { + const value = PARAMS[name]; + if (value === undefined) { + throw new Error(`No test value for path parameter "{${name}}".`); + } + + return value; + }); + +/** + * A body wide enough for every write route the builder produces. + * + * Never actually validated in the denial sweep - the permission middleware runs + * first - but a `PUT` with no body would fail for the wrong reason in the + * translator sweep, where some of these routes are allowed through. + */ +const BODY = { + action: "publish" as const, + expectedVersion: 1, + scheduledFor: new Date(Date.now() + 86_400_000).toISOString(), + values: { title: "Hello world" }, +}; + +const routes = buildContentRoutes(model, { pluginId: PLUGIN_ID }); + +const app = (() => { + const instance = new OpenAPIHono(); + const context: MiddlewareHandler = async (c, next) => { + c.set("admin", { user: adminUser }); + c.set("events", { + emit: async () => await Promise.resolve({ failures: [] }), + } as never); + await next(); + }; + instance.use("*", context); + for (const { handler, route } of routes) instance.openapi(route, handler); + + return instance; +})(); + +const request = async (method: string, path: string) => + await app.request(path, { + method, + ...(method === "GET" || method === "DELETE" + ? method === "DELETE" + ? { + body: JSON.stringify(BODY), + headers: { "Content-Type": "application/json" }, + } + : {} + : { + body: JSON.stringify(BODY), + headers: { "Content-Type": "application/json" }, + }), + }); + +/** `"GET /{id}/revisions"` - stable across runs, so the matrix reads as a list. */ +const label = (route: { method: string; path: string }): string => + `${route.method.toUpperCase()} ${route.path}`; + +beforeEach(() => { + granted = new Set(); + asked = []; +}); + +describe("the generated permission matrix", () => { + it("gates every route on a staff permission", async () => { + // Nothing granted, so a route with a permission answers 403 and a route + // without one answers whatever its handler does. The assertion is over the + // whole array rather than a list somebody has to remember to extend. + for (const { route } of routes) { + const res = await request( + route.method.toUpperCase(), + concretePath(route.path), + ); + + expect([label(route), res.status]).toEqual([label(route), 403]); + } + }); + + it("asks for exactly the documented permission on each route", async () => { + const matrix: Record<string, string> = {}; + + for (const { route } of routes) { + asked = []; + await request(route.method.toUpperCase(), concretePath(route.path)); + + // One check per route, not two: a second would mean a route gated twice, + // where only one of the two is visible in the AdminCP permission editor. + expect([label(route), asked.length]).toEqual([label(route), 1]); + expect(asked[0].module).toBe(MODULE); + matrix[label(route)] = asked[0].permission; + } + + expect(matrix).toEqual({ + "DELETE /{id}": "can_delete", + "DELETE /{id}/translations/{locale}": "can_delete", + "GET /": "can_view", + "GET /options/{field}": "can_view", + "GET /{id}": "can_view", + // Read-only: it reports what the slug mutations already did, so the + // permission that allowed the mutation is the only one it needs. There is + // no manual redirect manager to gate separately. + "GET /{id}/delivery": "can_view", + "GET /{id}/public-locales": "can_view", + "GET /{id}/revisions": "can_view", + "GET /{id}/revisions/{revisionId}": "can_view", + "GET /{id}/schedules": "can_view", + "GET /{id}/translations": "can_view", + "GET /{id}/translations/{locale}": "can_view", + "GET /{id}/translations/{locale}/revisions": "can_view", + "GET /{id}/translations/{locale}/revisions/{revisionId}": "can_view", + "POST /": "can_create", + "POST /{id}/preview": "can_view", + "POST /{id}/publish": "can_publish", + "POST /{id}/revisions/{revisionId}/restore": "can_restore", + "POST /{id}/schedule": "can_publish", + "POST /{id}/schedule/{scheduleId}/cancel": "can_publish", + "POST /{id}/translations/{locale}": "can_translate", + "POST /{id}/translations/{locale}/preview": "can_view", + "POST /{id}/translations/{locale}/publish": "can_publish", + "POST /{id}/translations/{locale}/revisions/{revisionId}/restore": + "can_restore", + "POST /{id}/translations/{locale}/unpublish": "can_publish", + "POST /{id}/unpublish": "can_publish", + "PUT /{id}": "can_edit", + "PUT /{id}/translations/{locale}": "can_translate", + }); + }); + + /** + * The role Stage 5 exists for: somebody who writes Polish and nothing else. + * + * `can_translate` depends on `can_view` and deliberately **not** on + * `can_edit`, so this pair is expressible - and the point of the pair is that + * it stops at the language boundary. A translator who could reach `PUT /{id}` + * could rewrite a shared field; one who could reach the base publish routes + * could put an unfinished record on the internet. + */ + describe("translator isolation", () => { + const TRANSLATOR = [`${MODULE}:can_view`, `${MODULE}:can_translate`]; + + const statusFor = async (method: string, path: string) => { + granted = new Set(TRANSLATOR); + + return (await request(method, path)).status; + }; + + it.each([ + ["PUT", "/7"], + ["POST", "/"], + ["DELETE", "/7"], + ["POST", "/7/publish"], + ["POST", "/7/unpublish"], + ["POST", "/7/revisions/3/restore"], + // A shared revision *and* a locale's own: `can_restore` depends on + // `can_edit`, so a translator has neither. + ["POST", "/7/translations/pl/revisions/3/restore"], + ["POST", "/7/schedule"], + ["POST", "/7/schedule/5/cancel"], + ["DELETE", "/7/translations/en"], + ["POST", "/7/translations/en/publish"], + ["POST", "/7/translations/en/unpublish"], + ])("refuses %s %s", async (method, path) => { + await expect(statusFor(method, path)).resolves.toBe(403); + }); + + it.each([ + ["POST", "/7/translations/pl"], + ["PUT", "/7/translations/pl"], + ])("reaches %s %s", async (method, path) => { + // Past the guard is all this asserts. What the handler then does with a + // record that is not there belongs to the translation suites. + await expect(statusFor(method, path)).resolves.not.toBe(403); + }); + + it.each([ + ["GET", "/"], + ["GET", "/7/translations"], + ["GET", "/7/translations/en/revisions"], + ])("still reads %s %s", async (method, path) => { + await expect(statusFor(method, path)).resolves.not.toBe(403); + }); + }); + + /** + * A collection is written through the ordinary `PUT`, and that is the whole + * answer to "can a relation picker be a write primitive". + * + * There is no per-collection mutation endpoint to gate separately, so an + * editor with `can_view` alone cannot add a category by any route - and the + * picker itself is a read of labels, gated on `can_view` like every other + * read. + */ + describe("advanced collections have no second door", () => { + it("exposes no route outside the audited set", () => { + const paths = routes.map(entry => label(entry.route)); + + expect( + paths.filter( + path => path.includes("relations") || path.includes("repeatable"), + ), + ).toEqual([]); + }); + + it("refuses a collection write to a viewer", async () => { + granted = new Set([`${MODULE}:can_view`]); + + const res = await app.request("/7", { + body: JSON.stringify({ + expectedVersion: 1, + values: { faq: [{ answer: "Yes", question: "Really?" }] }, + }), + headers: { "Content-Type": "application/json" }, + method: "PUT", + }); + + expect(res.status).toBe(403); + }); + + it("lets a viewer open the picker, which reads labels and writes nothing", async () => { + granted = new Set([`${MODULE}:can_view`]); + vi.spyOn(model, "service").mockReturnValue({ + options: async () => await Promise.resolve([]), + } as never); + + const res = await app.request("/options/author"); + + expect(res.status).toBe(200); + vi.restoreAllMocks(); + }); + }); + + /** + * Two plugins can name a permission module the same thing - `articles` is not + * an unusual choice - and the registry allows it precisely because the plugin + * id is part of the key. That only holds if the *route* carries its own + * plugin id into the check rather than reading whichever plugin happens to be + * handling the request. + */ + describe("cross-plugin isolation", () => { + it("checks the permission under the route's own plugin", async () => { + for (const { route } of routes) { + asked = []; + await request(route.method.toUpperCase(), concretePath(route.path)); + + expect([label(route), asked[0].plugin]).toEqual([ + label(route), + PLUGIN_ID, + ]); + } + }); + + it("does not follow the plugin the request is being served by", async () => { + // The same model, mounted by a second plugin. Its routes ask under + // `@vitnode/other`, so granting `@vitnode/example`'s module grants + // nothing here - which is what stops one plugin's roles reaching another + // plugin's content through a module name they happen to share. + const other = new OpenAPIHono(); + other.use("*", async (c, next) => { + c.set("admin", { user: adminUser }); + await next(); + }); + for (const { handler, route } of buildContentRoutes(model, { + pluginId: "@vitnode/other", + })) { + other.openapi(route, handler); + } + + asked = []; + granted = new Set([`${MODULE}:can_view`]); + vi.spyOn(model, "service").mockReturnValue({ + findMany: async () => + await Promise.resolve({ edges: [], pageInfo: {} }), + } as never); + const res = await other.request("/"); + vi.restoreAllMocks(); + + // Granted by module name - the module is the same string - and the check + // still ran under the other plugin, which is the fact worth pinning. + expect(res.status).not.toBe(403); + expect(asked[0]).toMatchObject({ + module: MODULE, + permission: "can_view", + plugin: "@vitnode/other", + }); + }); + }); +}); diff --git a/packages/vitnode/src/content/server/public-privacy.test.ts b/packages/vitnode/src/content/server/public-privacy.test.ts new file mode 100644 index 000000000..4cd66419f --- /dev/null +++ b/packages/vitnode/src/content/server/public-privacy.test.ts @@ -0,0 +1,263 @@ +// @vitest-environment node +import { describe, expect, it } from "vitest"; + +import { defineContentType } from "../define"; +import { field } from "../fields"; +import { createContentModel } from "./model"; +import { + contentPublicCollectionFields, + contentPublicSelection, + createContentPublicProjector, +} from "./public-service"; + +/** + * What a public response is allowed to contain, stated as an exact set. + * + * Every other public test asserts that a particular field is present or a + * particular one is absent. This one asserts the **whole** key set, which is the + * only shape of assertion that catches a field nobody thought to check: a leaf + * added to a group later, a system column that started being selected, an + * internal storage name leaking through the flattening. + * + * The fixture is deliberately hostile - every kind has a public member and a + * private sibling, so "the allowlist is a filter" has something to be wrong + * about in each of them. + */ +const contentType = defineContentType({ + id: "test.privacy", + tableName: "test_privacy", + localization: { enabled: true, defaultLocale: "en", fallback: "default" }, + publication: { enabled: true }, + editorial: { enabled: true }, + fields: { + title: field.text({ localized: true, required: true, maxLength: 200 }), + slug: field.slug({ localized: true, source: "title" }), + /** Localized and private: the leak a locale-aware read could produce. */ + internalNotes: field.textarea({ localized: true, nullable: true }), + /** Shared and private. */ + revenue: field.number({ integer: true, defaultValue: 0 }), + featured: field.boolean({ defaultValue: false }), + /** A localized group with one public leaf and one private one. */ + seo: field.group({ + localized: true, + nullable: true, + fields: { + title: field.text({ nullable: true, maxLength: 200 }), + robots: field.text({ nullable: true, maxLength: 100 }), + }, + }), + /** A shared group, entirely private. */ + syndication: field.group({ + fields: { + indexable: field.boolean({ defaultValue: true }), + partnerKey: field.text({ nullable: true, maxLength: 100 }), + }, + }), + /** A repeatable with one public leaf and one private one. */ + faq: field.repeatable({ + fields: { + question: field.text({ required: true, maxLength: 200 }), + answer: field.textarea({ required: true }), + moderatorNote: field.textarea({ nullable: true }), + }, + }), + /** A private to-many relation, and a public one. */ + tags: field.relation({ multiple: true, self: true }), + hiddenLinks: field.relation({ multiple: true, self: true }), + }, + publicApi: { + enabled: true, + path: "privacy", + fields: [ + "title", + "slug", + "featured", + "seo.title", + "faq.question", + "faq.answer", + "tags", + "publishedAt", + ], + orderableFields: ["publishedAt"], + }, + admin: { + label: { plural: "Privacies", singular: "Privacy" }, + list: { columns: ["featured", "status"] }, + form: { fields: ["faq", "tags", "hiddenLinks", "syndication"] }, + }, +}); + +const model = createContentModel(contentType); +const project = createContentPublicProjector(contentType); + +/** + * A raw row carrying **everything** - including the values the projector must + * drop and the flattened storage names it must never surface. + * + * Group leaves arrive already nested, which is what the read layer hands the + * projector; the flat `seoRobots`-style columns are added alongside so a + * projector that copied unknown keys through would be caught here rather than + * in production. + */ +const rawRow = { + createdAt: new Date("2026-01-01T00:00:00.000Z"), + faq: [ + { + answer: "Because.", + id: 11, + moderatorNote: "spam risk", + question: "Why?", + }, + ], + featured: true, + hiddenLinks: [99], + id: 7, + internalNotes: "Do not publish before Friday.", + // The columns the translation and base tables really hold, flattened. + internalNotesColumn: "leak", + languageId: 3, + publishedAt: new Date("2026-02-01T00:00:00.000Z"), + revenue: 12_345, + seo: { robots: "noindex", title: "Public SEO title" }, + seoRobots: "noindex", + seoTitle: "Public SEO title", + slug: "hello-world", + status: "published", + syndication: { indexable: false, partnerKey: "secret-key" }, + syndicationIndexable: false, + syndicationPartnerKey: "secret-key", + tags: [1, 2], + title: "Hello world", + updatedAt: new Date("2026-03-01T00:00:00.000Z"), + version: 9, +}; + +describe("the public projection is an allowlist, not a filter", () => { + const projected = project(rawRow) as Record<string, unknown>; + + it("carries exactly the allowlisted keys", () => { + // `id` is absent because the allowlist does not name it: the cursor needs + // it from the database, and the projector drops it again. + expect(Object.keys(projected).sort()).toEqual([ + "faq", + "featured", + "publishedAt", + "seo", + "slug", + "tags", + "title", + ]); + }); + + it.each([ + ["a private localized scalar", "internalNotes"], + ["a private shared scalar", "revenue"], + ["a wholly private group", "syndication"], + ["a private relation", "hiddenLinks"], + ["the editorial version", "version"], + ["the internal language id", "languageId"], + ["a system timestamp", "createdAt"], + ["a system timestamp", "updatedAt"], + ["the publication state", "status"], + ["the cursor identifier", "id"], + ])("drops %s (%s)", (_why, key) => { + expect(projected).not.toHaveProperty(key); + }); + + it("never surfaces a flattened storage column name", () => { + // `seo.title` is stored as `seoTitle`. A response that carried the column + // name would publish an internal detail *and* give a client two spellings + // of one value. + for (const key of Object.keys(projected)) { + expect(key).not.toMatch(/^(seo|syndication|internalNotes)[A-Z]/); + } + }); + + it("keeps a group to the leaves the allowlist named", () => { + expect(projected.seo).toEqual({ title: "Public SEO title" }); + }); + + it("keeps a repeatable child to its public leaves plus its identity", () => { + // The identifier stays: it is what an editor's `set` matches on, and a + // public consumer needs a stable key per row. The moderator note does not. + expect(projected.faq).toEqual([ + { answer: "Because.", id: 11, question: "Why?" }, + ]); + }); + + it("exposes a relation as identifiers and nothing else", () => { + expect(projected.tags).toEqual([1, 2]); + }); + + it("projects a missing collection as an empty list, not as undefined", () => { + const empty = project({ ...rawRow, faq: undefined, tags: undefined }) as { + faq: unknown; + tags: unknown; + }; + + expect(empty.faq).toEqual([]); + expect(empty.tags).toEqual([]); + }); +}); + +describe("the public read never fetches a private column", () => { + const selection = contentPublicSelection(contentType, model.columns); + + it("selects the allowlist plus the cursor, and nothing else", () => { + expect(Object.keys(selection).sort()).toEqual([ + "featured", + "id", + "publishedAt", + "seo.title", + "slug", + "title", + ]); + }); + + it("leaves every private column out of the SELECT entirely", () => { + // Defence in depth that matters: a private column that is never fetched + // cannot be leaked by a mistake in the projector further downstream. + for (const name of [ + "internalNotes", + "revenue", + "seo.robots", + "syndication.indexable", + "syndication.partnerKey", + "version", + "status", + ]) { + expect(selection).not.toHaveProperty(name); + } + }); + + it("loads only the collections the allowlist exposes", () => { + // A public list must not join a junction table it will then discard, and + // `hiddenLinks` is private - so it is not even a candidate. + expect(contentPublicCollectionFields(contentType).sort()).toEqual([ + "faq", + "tags", + ]); + }); +}); + +describe("the generated public schema agrees with the projection", () => { + it("describes the projected keys plus the one piece of generated metadata", () => { + // The OpenAPI contract and the runtime projection are built from the same + // allowlist, so the only difference between them is `locale` - which the + // route adds because a localized response has to say which language it is, + // and which `publicApi.fields` therefore reserves rather than accepts. + const shape = model.schemas.publicSelectObject.shape; + const projected = Object.keys(project(rawRow)); + + expect(Object.keys(shape).sort()).toEqual([...projected, "locale"].sort()); + }); + + it("parses a projected row once the route has stamped the locale on it", () => { + expect( + model.schemas.publicSelectObject.safeParse({ + ...(project(rawRow) as Record<string, unknown>), + locale: "en", + }).success, + ).toBe(true); + }); +}); diff --git a/packages/vitnode/src/content/server/public-routes.ts b/packages/vitnode/src/content/server/public-routes.ts index 58e792437..652f28200 100644 --- a/packages/vitnode/src/content/server/public-routes.ts +++ b/packages/vitnode/src/content/server/public-routes.ts @@ -38,6 +38,7 @@ import { splitContentFieldPath, } from "../paths"; import { publicOrderableColumns } from "../registry"; +import { buildContentDeliveryRoutes } from "./delivery-routes"; import { findContentLanguage, listContentLanguages } from "./language-resolver"; import { verifyContentPreviewToken } from "./preview-token"; import { @@ -514,8 +515,13 @@ export const buildContentPublicRoutes = < return [ list, // Before `detail` for readability only - the two can never both match, so - // the order carries no meaning. + // the order carries no meaning. The delivery routes are the same: every one of + // them begins with a static `delivery` segment and `/{slug}` is a single + // segment, so a record whose slug is literally "delivery" still resolves. ...(definition.editorial.preview.enabled ? [preview] : []), + ...(definition.delivery.enabled + ? buildContentDeliveryRoutes(model, { pluginId }) + : []), detail, ]; }; diff --git a/packages/vitnode/src/content/server/public-service.ts b/packages/vitnode/src/content/server/public-service.ts index 9ae1949a1..5bd1c32ea 100644 --- a/packages/vitnode/src/content/server/public-service.ts +++ b/packages/vitnode/src/content/server/public-service.ts @@ -465,10 +465,13 @@ export const createContentPublicService = < }, table, where: conditions.length > 1 ? and(...conditions) : conditions[0], - query: async ({ limit, orderBy: order, where }) => + query: async ({ cursorSelection, limit, orderBy: order, where }) => await c .get("db") - .select(selection()) + // The cursor value is projected by this statement and stripped from + // the row before `project` ever sees it, so the public allowlist is + // unchanged: it is pagination's own column, not a field. + .select({ ...selection(), ...cursorSelection }) .from(table) .where(where) .orderBy(order) diff --git a/packages/vitnode/src/content/server/routes.ts b/packages/vitnode/src/content/server/routes.ts index 79b273af1..451bf96bf 100644 --- a/packages/vitnode/src/content/server/routes.ts +++ b/packages/vitnode/src/content/server/routes.ts @@ -18,6 +18,7 @@ import { } from "../../api/lib/with-pagination"; import { zodContentConflict, + zodContentDeliveryConflict, zodContentScheduleRejection, zodContentUnprocessable, } from "../conflicts"; @@ -181,10 +182,21 @@ export const buildContentRoutes = < // a client can tell "someone saved first" from "that value is taken" and act // on the difference. Everything else keeps the plain-text 409 it has always // returned - a Stage 1-3 route's contract does not change. + // A content type with `delivery.redirects` adds a third arm: an address another + // record's URL history still owns. Declared as a union with the editorial pair + // rather than replacing it, so a client generated before Stage 8 still parses the + // two arms it knows and only fails to recognise the new one. + const conflictSchema = + definition.delivery.enabled && definition.delivery.redirects.enabled + ? z.union([zodContentConflict, zodContentDeliveryConflict]) + : zodContentConflict; + const uniqueConflict = editorial ? jsonResponse( - zodContentConflict, - "A record with these values already exists, or the version moved", + conflictSchema, + definition.delivery.redirects.enabled + ? "A record with these values already exists, the version moved, or the address is reserved by a historical URL" + : "A record with these values already exists, or the version moved", ) : { description: "A record with these values already exists" }; @@ -862,6 +874,93 @@ export const buildContentRoutes = < }, }); + /** + * The delivery state of one record: where it lives, and where it used to. + * + * `can_view` rather than a permission of its own, and deliberately so. This is + * read-only - it reports what the slug mutations already did - so the permission + * that allowed the mutation is the only one it needs, and inventing a + * `can_manage_redirects` for a screen that manages nothing would be a permission + * every install has to configure for no decision it can make. + * + * `locale` scopes it to one language on a content type whose slug is localized, + * which is what lets the AdminCP's Polish tab show Polish URLs and nothing else. + */ + const deliveryDetail = buildRoute({ + pluginId, + adminStaffPermission: { module, permission: CONTENT_PERMISSIONS.view }, + route: { + method: "get", + path: "/{id}/delivery", + description: `Canonical URL and historical URLs of one ${label.singular}`, + request: { + params: schemas.params, + query: z.object({ + locale: z.string().min(1).max(CONTENT_LOCALE_MAX_LENGTH).optional(), + }), + }, + responses: { + 200: jsonResponse( + z.object({ + /** `null` while the record has no public URL - a draft, say. */ + canonicalPath: z.string().nullable(), + /** + * Every address it has ever answered to, current one first. + * + * `path` is the URL exactly as it was live, which is what somebody's + * bookmark holds. The storage columns behind it - `languageId`, + * `pluginId`, the row id - are deliberately absent: they are details of + * `core_content_slug_history` rather than part of this contract. + */ + history: z.array( + z.object({ + createdAt: z.date(), + path: z.string(), + retiredAt: z.date().nullable(), + slug: z.string(), + }), + ), + /** Whether the record is publicly reachable in this language now. */ + isPublic: z.boolean(), + locale: z.string().nullable(), + }), + `Delivery state of one ${label.singular}`, + ), + 400: invalidIdentifier, + 404: { description: `${label.singular} not found` }, + }, + }, + handler: async c => { + const build = model.deliveryService; + if (!build) throw notFound(definition); + + const id = identifier(c); + const locale = c.req.query("locale"); + const delivery = build(c, { pluginId }); + + // The canonical path comes from the public read, so a draft reports `null` + // rather than a URL that answers 404 - "this is where it *would* live" is a + // different claim from "this is where it lives", and the panel says so. + const metadata = await delivery.findById(id, { locale }); + const history = await delivery.history(id, { locale }); + + return c.json( + { + canonicalPath: metadata?.canonicalPath ?? null, + history: history.map(entry => ({ + createdAt: entry.createdAt, + path: entry.path, + retiredAt: entry.retiredAt, + slug: entry.slug, + })), + isPublic: metadata !== null, + locale: metadata?.locale ?? null, + }, + 200, + ); + }, + }); + /** * Mints a preview link for the record's newest revision. * @@ -1266,6 +1365,7 @@ export const buildContentRoutes = < : []), ...(editorial ? [revisionList, revisionDetail, restore] : []), ...(previewEnabled ? [previewToken] : []), + ...(definition.delivery.enabled ? [deliveryDetail] : []), ...(definition.editorial.scheduling.enabled ? [scheduleList, scheduleCreate, scheduleCancel] : []), diff --git a/packages/vitnode/src/content/server/schedule-effects.ts b/packages/vitnode/src/content/server/schedule-effects.ts index 67a4a96d6..05dbf15d3 100644 --- a/packages/vitnode/src/content/server/schedule-effects.ts +++ b/packages/vitnode/src/content/server/schedule-effects.ts @@ -212,6 +212,13 @@ export const runContentScheduleEffects = async ( const revalidation = await dispatchContentRevalidation(c, { contentTypeId: definition.id, + // A scheduled transition always flips public reachability, so it changes both the + // file it adds a line to (or removes one from) and the index that counts them. + // Absent for a content type without `delivery`, which keeps its tag list + // byte-identical. + ...(definition.delivery.enabled + ? { delivery: { sitemap: { contentChanged: true, indexChanged: true } } } + : {}), id: payload.itemId, isPublic: isContentRowPublic(row), // A scheduled transition moves the *record*, and the record's publication diff --git a/packages/vitnode/src/content/server/search-indexer.test.ts b/packages/vitnode/src/content/server/search-indexer.test.ts index d16bce27b..e32a8dcb6 100644 --- a/packages/vitnode/src/content/server/search-indexer.test.ts +++ b/packages/vitnode/src/content/server/search-indexer.test.ts @@ -143,7 +143,18 @@ describe("generated content search indexer", () => { expect(opOf(calls, "orderBy")).toBeDefined(); expect(opOf(calls, "limit")).toBe(200); - expect(opOf(calls, "offset")).toBe(400); + }); + + it("never issues a SQL OFFSET, however deep the rebuild has gone", async () => { + // `OFFSET` re-reads and discards every earlier row, and counts rows in a + // set that moves underneath it - a record unpublished after an earlier + // page shifts the rest forward and the next page steps over one. The + // walk is a keyset seek on the primary key instead. + const { c, calls } = createDbMock([[]]); + + await indexerFor(searchable).load(c, 400, 200); + + expect(opOf(calls, "offset")).toBeUndefined(); }); it("maps every row into a document, and stamps the owning plugin", async () => { diff --git a/packages/vitnode/src/content/server/search-indexer.ts b/packages/vitnode/src/content/server/search-indexer.ts index 8d4c906ef..2d117fde6 100644 --- a/packages/vitnode/src/content/server/search-indexer.ts +++ b/packages/vitnode/src/content/server/search-indexer.ts @@ -214,6 +214,16 @@ export const createContentSearchIndexer = < ]), ); + /** + * The keyset cursor, per request. + * + * A `WeakMap` keyed by the Hono context, exactly as the localized indexer + * does: the rebuild task calls `load` repeatedly within one request, and the + * entry is collected with it. A fresh request starts at the beginning, which + * is what a rebuild means. + */ + const cursors = new WeakMap<Context, number>(); + return { itemType: definition.id, @@ -230,23 +240,49 @@ export const createContentSearchIndexer = < return row?.value ?? 0; }, - // Offset paging, which is what the contract exposes. Ordering by the primary - // key keeps pages from overlapping within one rebuild; a row whose - // publication state changes mid-rebuild can still shift, and that is what - // the next publish - or the next rebuild - repairs. - // - // `itemsRead` is the row count, not the document count. A published row with - // no usable title projects to nothing, and reporting that as "no items" would - // end the rebuild before the valid rows after it. + /** + * Keyset paging on `id`, not `OFFSET`. + * + * `OFFSET` was wrong twice over. It re-reads and discards every earlier row, + * so page 500 of a rebuild costs five hundred pages of work - and worse, the + * offset counts rows in a set that is *moving*: a record unpublished after + * page one shifts everything behind it forward by one, and the next + * `OFFSET 100` steps straight over a row nobody ever indexed. A rebuild that + * silently misses rows is the failure a rebuild exists to fix. + * + * `WHERE id > :last` has neither problem. It seeks on the primary key, and + * it is anchored to a value rather than to a position, so rows appearing or + * disappearing behind the cursor cannot move it. + * + * The `offset` argument stays in the signature because the + * {@link SearchIndexer} contract is shared with hand-written indexers; it is + * used only as the "this is a fresh pass" signal, exactly as the localized + * indexer uses it. + * + * `itemsRead` is the row count, not the document count. A published row with + * no usable title projects to nothing, and reporting that as "no items" + * would end the rebuild before the valid rows after it. + */ load: async (c, offset, limit) => { + // The contract's only signal that this is a fresh pass rather than the + // next page of one. + if (offset === 0) cursors.delete(c); + const cursor = cursors.get(c); + const rows = await c .get("db") .select(selection) .from(table) - .where(publishedCondition(published)) + .where( + cursor === undefined + ? publishedCondition(published) + : and(publishedCondition(published), gt(primaryCursor, cursor)), + ) .orderBy(asc(primaryCursor)) - .limit(limit) - .offset(offset); + .limit(limit); + + const last = rows.at(-1); + if (last && typeof last.id === "number") cursors.set(c, last.id); // One batch for the whole page, and only the collections the search // configuration names - never one query per document. diff --git a/packages/vitnode/src/content/server/service.test.ts b/packages/vitnode/src/content/server/service.test.ts index 8123e566a..9d00c24a9 100644 --- a/packages/vitnode/src/content/server/service.test.ts +++ b/packages/vitnode/src/content/server/service.test.ts @@ -326,25 +326,48 @@ describe("content service", () => { it("joins once per reference field instead of querying per row", async () => { const { c, calls } = createDbMock( page([ - { id: 1, label__author: "Ada", label__category: "News" }, - { id: 2, label__author: null, label__category: "News" }, + { + __cursorValue: "2026-01-02 00:00:00", + id: 1, + label__author: "Ada", + label__category: "News", + }, + { + __cursorValue: "2026-01-01 00:00:00", + id: 2, + label__author: null, + label__category: "News", + }, ]), ); await articles.service(c).findMany(); - // `author` and `category` - one join each, and no extra round trips. + // `author` and `category` - one join each, and no per-row lookup. expect(opsOf(calls, "leftJoin")).toHaveLength(2); - expect(opsOf(calls, "select")).toHaveLength(2); // count + page + // Two, and both constant: the count and the page. There is no third + // read to mint the cursors, because the page query already selected the + // value they are made of - which is also what makes a cursor describe + // where the row was rather than where it has since moved. + expect(opsOf(calls, "select")).toHaveLength(2); }); it("splits the joined labels out of the row", async () => { const { c } = createDbMock( - page([{ id: 1, label__author: "Ada", label__category: "News" }]), + page([ + { + __cursorValue: "2026-01-02 00:00:00", + id: 1, + label__author: "Ada", + label__category: "News", + }, + ]), ); const { edges } = await articles.service(c).findMany(); + // No `__cursorValue`: pagination takes its own column back before the + // row reaches anybody. expect(edges[0]).toEqual({ id: 1, labels: { author: "Ada", category: "News" }, @@ -353,7 +376,14 @@ describe("content service", () => { it("reports a missing label as null", async () => { const { c } = createDbMock( - page([{ id: 1, label__author: null, label__category: "News" }]), + page([ + { + __cursorValue: "2026-01-02 00:00:00", + id: 1, + label__author: null, + label__category: "News", + }, + ]), ); const { edges } = await articles.service(c).findMany(); diff --git a/packages/vitnode/src/content/server/service.ts b/packages/vitnode/src/content/server/service.ts index 3e5dc85c7..8feec0465 100644 --- a/packages/vitnode/src/content/server/service.ts +++ b/packages/vitnode/src/content/server/service.ts @@ -66,10 +66,17 @@ export type ContentListRow<TDefinition> = ContentSelect<TDefinition> & { export interface ContentPageInfo { count: number; - endCursor: null | number; + /** + * An opaque cursor for the last row on this page. + * + * It encodes the ordered tuple - the sort column's value *and* the row's + * identifier - so it is meaningless outside the ordering that produced it. + * Hand it back as `cursor`; never parse it. + */ + endCursor: null | string; hasNextPage: boolean; hasPreviousPage: boolean; - startCursor: null | number; + startCursor: null | string; totalCount: number; } @@ -736,10 +743,17 @@ export const createContentService = < }, table, where: combined, - query: async ({ limit, orderBy: order, where: rowWhere }) => { + query: async ({ + cursorSelection, + limit, + orderBy: order, + where: rowWhere, + }) => { // One LEFT JOIN per reference field resolves every label in the same - // round trip - there is no per-row lookup anywhere. - const selection: Record<string, PgColumn> = { + // round trip - there is no per-row lookup anywhere. The cursor value + // rides along in the same statement, which is what makes the cursor a + // record of where the row was rather than where it has since moved. + const selection: Record<string, PgColumn | SQL<string>> = { ...ownSelection(), ...Object.fromEntries( Object.entries(references).map(([name, target]) => [ @@ -747,6 +761,9 @@ export const createContentService = < target.labelColumn, ]), ), + // Last, so a content field can never shadow it and leave the page + // with no way to mint a cursor. + ...cursorSelection, }; let builder = c.get("db").select(selection).from(table).$dynamic(); diff --git a/packages/vitnode/src/content/server/slug-history-model.ts b/packages/vitnode/src/content/server/slug-history-model.ts new file mode 100644 index 000000000..be1ce7b41 --- /dev/null +++ b/packages/vitnode/src/content/server/slug-history-model.ts @@ -0,0 +1,371 @@ +import type { SQL } from "drizzle-orm"; +import type { Context } from "hono"; + +import { and, asc, desc, eq, inArray, isNull, sql } from "drizzle-orm"; + +import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDatabase } from "./service"; + +import { core_content_slug_history } from "../../database/content"; +import { contentDeliveryPath } from "../delivery"; +import { ContentDeliverySlugReserved } from "../errors"; + +/** + * One retired or current public address of one record. + * + * The AdminCP shows these, the resolver reads them and an audit reads them long + * after the record is gone. `retiredAt === null` means "this is the address the + * record answers to now"; anything else is a URL that redirects to it. + */ +export interface ContentSlugHistoryEntry { + createdAt: Date; + itemId: number; + /** `null` for a shared slug - see `core_content_slug_history`. */ + languageId: null | number; + /** The URL that was live, exactly as it was live. */ + path: string; + retiredAt: Date | null; + slug: string; +} + +/** + * One address of one record, as every write names it. + * + * `languageId` is the storage key and `locale` is what an error message says out + * loud - both, because the two are needed at different layers and deriving one + * from the other here would mean a language lookup inside a transaction that + * already knows the answer. + */ +export interface ContentSlugHistoryTarget { + itemId: number; + /** `null` for a shared slug - see `core_content_slug_history`. */ + languageId: null | number; + /** The canonical `core_languages.code`, or `null` when the slug is shared. */ + locale: null | string; + slug: string; +} + +/** + * The persistence half of slug history: reservations in, lookups out. + * + * Every write takes the transaction it should run in, and none of them opens one. + * That is the whole design constraint: the slug mutation, the reservation and the + * revision have to commit or roll back together, so this module can never be the + * thing that decides when that happens. `editorial-service` and + * `translation-editorial-service` own the transaction and call in. + * + * There is deliberately no `delete`. A retired URL is somebody's bookmark, and + * removing the row would let unrelated content inherit it - so the only way + * history shrinks is a deliberate, permissioned AdminCP action, which Stage 8 does + * not ship. + */ +export interface ContentSlugHistoryModel { + /** + * Refuses a slug that another record's history already owns. + * + * Called **before** the write it guards, so an editor is told at save time + * rather than at publish time - and so the failing transaction has done as + * little as possible. A slug this same record already owns is fine: moving from + * `b` back to `a` re-activates its own retired reservation rather than colliding + * with it. + */ + assertAvailable: ( + tx: ContentDatabase, + args: ContentSlugHistoryTarget, + ) => Promise<void>; + /** + * Every address one record has ever had, newest first. + * + * Scoped by language when one is given, which is what makes the AdminCP's Polish + * tab show Polish URLs and nothing else. + */ + list: ( + args: { itemId: number; languageId?: null | number; limit?: number }, + database?: ContentDatabase, + ) => Promise<ContentSlugHistoryEntry[]>; + /** + * The record a retired (or current) address belongs to, or `null`. + * + * The resolver's one lookup, and the reason the two partial unique indexes lead + * with `(contentTypeId, slug)`: this runs on a public request path for a URL that + * is very often a typo, so it has to be an index hit rather than a scan. + */ + owner: ( + args: { languageId: null | number; slug: string }, + database?: ContentDatabase, + ) => Promise<ContentSlugHistoryEntry | null>; + /** + * Records one slug as the record's **current** public address. + * + * Idempotent: a republish of an unchanged slug re-activates the row it already + * has rather than inserting a second one, which is what keeps a retried queue + * task and a double-clicked publish button harmless. + * + * Throws {@link ContentDeliverySlugReserved} when another record owns the + * address. That check is a `SELECT ... FOR UPDATE` inside the caller's + * transaction rather than a caught unique violation, so the error names the slug + * and the locale instead of a Postgres constraint - and so two concurrent + * reservations of the same URL serialise instead of racing. + */ + reserve: ( + tx: ContentDatabase, + args: ContentSlugHistoryTarget & { + /** The path this slug produced, recorded as the historical fact it is. */ + path: string; + }, + ) => Promise<{ created: boolean }>; + /** + * Stamps one of a record's own addresses as no longer current. + * + * `{ retired: true }` only when a row was actually there and actually active, + * which is precisely the "this URL was publicly addressable" test: a draft whose + * slug was corrected three times before it was ever published has no row to + * retire, so it creates no redirect and emits no event. + */ + retire: ( + tx: ContentDatabase, + args: Omit<ContentSlugHistoryTarget, "locale">, + ) => Promise<{ retired: boolean }>; +} + +const HISTORY_LIST_LIMIT = 50; + +/** + * The language predicate, written the one way that is correct for both cases. + * + * `IS NULL` for a shared slug and `=` for a localized one: `languageId = NULL` is + * `NULL` in SQL, never `true`, so an equality comparison would silently match no + * shared row at all - and a shared reservation that matches nothing is a + * reservation that reserves nothing. + */ +const languageCondition = ( + languageId: null | number, + column: typeof core_content_slug_history.languageId, +): SQL => (languageId === null ? isNull(column) : eq(column, languageId)); + +const toEntry = (row: { + createdAt: Date; + itemId: number; + languageId: null | number; + path: string; + retiredAt: Date | null; + slug: string; +}): ContentSlugHistoryEntry => ({ + createdAt: row.createdAt, + itemId: row.itemId, + languageId: row.languageId, + path: row.path, + retiredAt: row.retiredAt, + slug: row.slug, +}); + +const ENTRY_COLUMNS = { + createdAt: core_content_slug_history.createdAt, + itemId: core_content_slug_history.itemId, + languageId: core_content_slug_history.languageId, + path: core_content_slug_history.path, + retiredAt: core_content_slug_history.retiredAt, + slug: core_content_slug_history.slug, +}; + +export const createContentSlugHistoryModel = ({ + c, + definition, + pluginId, +}: { + c: Context; + definition: AnyContentTypeDefinition; + pluginId: string; +}): ContentSlugHistoryModel => { + const contentTypeId = definition.id; + const scope = eq(core_content_slug_history.contentTypeId, contentTypeId); + + const findOwner = async ( + database: ContentDatabase, + { languageId, slug }: { languageId: null | number; slug: string }, + { lock = false }: { lock?: boolean } = {}, + ): Promise<ContentSlugHistoryEntry | null> => { + const query = database + .select(ENTRY_COLUMNS) + .from(core_content_slug_history) + .where( + and( + scope, + eq(core_content_slug_history.slug, slug), + languageCondition(languageId, core_content_slug_history.languageId), + ), + ) + .limit(1); + + const [row] = lock ? await query.for("update") : await query; + + return row ? toEntry(row) : null; + }; + + return { + assertAvailable: async (tx, { itemId, languageId, locale, slug }) => { + const owner = await findOwner(tx, { languageId, slug }); + if (owner === null || owner.itemId === itemId) return; + + throw new ContentDeliverySlugReserved({ contentTypeId, locale, slug }); + }, + + list: async ({ itemId, languageId, limit }, database) => { + const conditions = [scope, eq(core_content_slug_history.itemId, itemId)]; + if (languageId !== undefined) { + conditions.push( + languageCondition(languageId, core_content_slug_history.languageId), + ); + } + + const rows = await (database ?? c.get("db")) + .select(ENTRY_COLUMNS) + .from(core_content_slug_history) + .where(and(...conditions)) + // Current address first, then the retired ones newest to oldest: that is + // the order somebody reading the panel wants, and `id` breaks the tie so + // two rows created in the same millisecond do not swap places between + // reads. + .orderBy( + asc(core_content_slug_history.retiredAt), + desc(core_content_slug_history.id), + ) + .limit(Math.min(limit ?? HISTORY_LIST_LIMIT, HISTORY_LIST_LIMIT)); + + return rows.map(toEntry); + }, + + owner: async (args, database) => + await findOwner(database ?? c.get("db"), args), + + reserve: async (tx, { itemId, languageId, locale, path, slug }) => { + // Locked, so two writers reserving the same address in two transactions + // serialise here rather than both reaching the unique index and one of them + // surfacing a raw `23505`. + const existing = await findOwner( + tx, + { languageId, slug }, + { lock: true }, + ); + + if (existing !== null) { + if (existing.itemId !== itemId) { + throw new ContentDeliverySlugReserved({ + contentTypeId, + locale, + slug, + }); + } + + // Its own row, coming back into service: a slug that moved away and then + // moved back, or a republish of the address it already had. + await tx + .update(core_content_slug_history) + .set({ path, retiredAt: null }) + .where( + and( + scope, + eq(core_content_slug_history.itemId, itemId), + eq(core_content_slug_history.slug, slug), + languageCondition( + languageId, + core_content_slug_history.languageId, + ), + ), + ); + + return { created: false }; + } + + await tx.insert(core_content_slug_history).values({ + contentTypeId, + itemId, + languageId, + path, + pluginId, + slug, + }); + + return { created: true }; + }, + + retire: async (tx, { itemId, languageId, slug }) => { + const rows = await tx + .update(core_content_slug_history) + .set({ retiredAt: sql`now()` }) + .where( + and( + scope, + eq(core_content_slug_history.itemId, itemId), + eq(core_content_slug_history.slug, slug), + languageCondition(languageId, core_content_slug_history.languageId), + // Only an *active* row is retired. A slug already marked historical + // keeps the moment it stopped being live, which is the only timestamp + // that means anything to an audit. + isNull(core_content_slug_history.retiredAt), + ), + ) + .returning({ id: core_content_slug_history.id }); + + return { retired: rows.length > 0 }; + }, + }; +}; + +/** + * The current address of several records at once, keyed by identifier. + * + * Batched rather than one query per record, because the AdminCP list and a sitemap + * page both want a whole page's worth - and the alternative is the classic query + * per row that only shows up as a problem in production. + */ +export const contentSlugHistoryCurrentPaths = async ( + database: ContentDatabase, + { + contentTypeId, + itemIds, + languageId, + }: { + contentTypeId: string; + itemIds: readonly number[]; + languageId: null | number; + }, +): Promise<Map<number, string>> => { + if (itemIds.length === 0) return new Map(); + + const rows = await database + .select({ + itemId: core_content_slug_history.itemId, + path: core_content_slug_history.path, + }) + .from(core_content_slug_history) + .where( + and( + eq(core_content_slug_history.contentTypeId, contentTypeId), + inArray(core_content_slug_history.itemId, [...itemIds]), + languageCondition(languageId, core_content_slug_history.languageId), + isNull(core_content_slug_history.retiredAt), + ), + ); + + return new Map(rows.map(row => [row.itemId, row.path])); +}; + +/** + * The path one slug produces, or the empty string when it produces none. + * + * A thin wrapper over {@link contentDeliveryPath} for the write paths, which have + * to store *something* in a `NOT NULL` column. An unbuildable path means the slug + * was never addressable, so the caller does not reserve it at all - and this + * returning `""` rather than throwing keeps that decision in the caller where the + * surrounding transaction is. + */ +export const contentSlugHistoryPath = ({ + definition, + locale, + slug, +}: { + definition: AnyContentTypeDefinition; + locale: null | string; + slug: string; +}): string => contentDeliveryPath({ definition, locale, slug }) ?? ""; diff --git a/packages/vitnode/src/content/server/translation-advanced-revisions.test.ts b/packages/vitnode/src/content/server/translation-advanced-revisions.test.ts index b665761f9..f10299ddb 100644 --- a/packages/vitnode/src/content/server/translation-advanced-revisions.test.ts +++ b/packages/vitnode/src/content/server/translation-advanced-revisions.test.ts @@ -102,6 +102,12 @@ const translations = () => { create: vi.fn(), delete: vi.fn(), exists: vi.fn(), + // Stage 8 reads the base row's publication state to decide whether a + // translation's address is publicly reachable. Resolved as "published" so + // these suites keep exercising what they were written for. + findBasePublication: vi + .fn() + .mockResolvedValue({ publishedAt: new Date(0), status: "published" }), findByLanguageId: vi.fn(), findByLocale: vi.fn(), findManyForItem: vi.fn(), diff --git a/packages/vitnode/src/content/server/translation-editorial-service.test.ts b/packages/vitnode/src/content/server/translation-editorial-service.test.ts index 1c3da36e4..ffea2d070 100644 --- a/packages/vitnode/src/content/server/translation-editorial-service.test.ts +++ b/packages/vitnode/src/content/server/translation-editorial-service.test.ts @@ -119,6 +119,12 @@ const translations = () => { create: vi.fn(), delete: vi.fn(), exists: vi.fn(), + // Stage 8 reads the base row's publication state to decide whether a + // translation's address is publicly reachable. Resolved as "published" so + // these suites keep exercising what they were written for. + findBasePublication: vi + .fn() + .mockResolvedValue({ publishedAt: new Date(0), status: "published" }), findByLanguageId: vi.fn(), findByLocale: vi.fn(), findManyForItem: vi.fn(), diff --git a/packages/vitnode/src/content/server/translation-editorial-service.ts b/packages/vitnode/src/content/server/translation-editorial-service.ts index f67dd7e7c..72649ab44 100644 --- a/packages/vitnode/src/content/server/translation-editorial-service.ts +++ b/packages/vitnode/src/content/server/translation-editorial-service.ts @@ -14,6 +14,7 @@ import type { ContentLocalizedValues, ContentTranslationRow, } from "../types"; +import type { ContentDeliveryOutcome } from "./delivery-writes"; import type { ContentLanguage } from "./language-resolver"; import type { ContentRevisionPage, @@ -22,6 +23,7 @@ import type { import type { ContentDatabase } from "./service"; import type { ContentTranslationModel } from "./translation-model"; +import { isContentTranslationPubliclyVisible } from "../cache"; import { ContentEngineError, ContentRevisionNotRestorable, @@ -33,6 +35,10 @@ import { contentInnerFields, splitContentFieldPath, } from "../paths"; +import { + applyContentDeliveryWrite, + contentSlugHistoryFor, +} from "./delivery-writes"; import { diffChangedPaths } from "./query"; import { contentTranslationRevisionSnapshot, @@ -54,6 +60,14 @@ export interface ContentTranslationEditorialOutcome<TDefinition> { /** `false` when nothing moved: no write, no revision, no event, no tags. */ changed: boolean; changedFields: ContentLocalizedFieldName<TDefinition>[]; + /** + * What this mutation did to **this locale's** public URL, or absent. + * + * Absent for every content type without `delivery`, and for one whose slug is + * shared - a shared slug is a column on the base row, so a translation mutation + * cannot move it and the base editorial service owns its history. + */ + delivery?: ContentDeliveryOutcome; languageId: number; /** The canonical `core_languages.code`, never the caller's casing. */ locale: string; @@ -232,6 +246,90 @@ export const createContentTranslationEditorialService = < localizedFields, ); + // Only a **localized** slug is this service's business. A shared one is a column + // on the base row, so a translation mutation cannot move it - see the base + // editorial service, which owns that history. + const deliveryEnabled = + definition.delivery.enabled && + definition.delivery.slugScope === "localized"; + const slugHistory = contentSlugHistoryFor({ c, definition, pluginId }); + + /** + * The delivery half of one translation mutation, inside its transaction. + * + * The publication test is the **subordinated** one - the base row published *and* + * this translation published - because that is what makes a localized URL public. + * A published Polish translation of a draft article is not an address anybody can + * reach, so reserving its slug would hand out a permanent claim on a URL that was + * never live. + */ + const applyDelivery = async ( + tx: ContentDatabase, + { + after, + before, + itemId, + languageId, + locale, + }: { + after: ContentTranslationRow<TDefinition> | null; + before: ContentTranslationRow<TDefinition> | null; + itemId: number; + languageId: number; + locale: string; + }, + ): Promise<ContentDeliveryOutcome | undefined> => { + if (!deliveryEnabled) return undefined; + + const base = (await translations.findBasePublication(itemId, { tx })) ?? { + publishedAt: null, + status: undefined, + }; + const visible = ( + row: ContentTranslationRow<TDefinition> | null, + ): boolean => { + if (row === null) return false; + + return isContentTranslationPubliclyVisible({ + base, + translation: { + publishedAt: (row as { publishedAt?: Date | null }).publishedAt, + status: (row as { status?: string }).status, + }, + }); + }; + + return await applyContentDeliveryWrite({ + definition, + slugHistory, + transition: { + isPublic: visible(after), + itemId, + languageId, + locale, + previousSlug: slugOf(before), + slug: slugOf(after), + wasPublic: visible(before), + }, + tx, + }); + }; + + /** + * The publication state one translation held before a transition. + * + * The localized twin of the base service's `invert`, and correct for the same + * reason: a transition is guarded on the state it changes, so a `publish` that + * returned a row can only have found it unpublished. + */ + const invertTranslation = ( + operation: "publish" | "unpublish", + row: ContentTranslationRow<TDefinition>, + ): ContentTranslationRow<TDefinition> => ({ + ...row, + status: operation === "publish" ? "draft" : "published", + }); + /** * One locale's revision model. * @@ -366,9 +464,22 @@ export const createContentTranslationEditorialService = < version: result.version, }); + // Publishing a language is the moment its address becomes live, so this is + // where the reservation is taken. Unpublishing writes nothing: the history + // stays, and the resolver stops redirecting to it because it reads the live + // publication state rather than the history. + const delivery = await applyDelivery(tx, { + after: result.row, + before: invertTranslation(operation, result.row), + itemId: result.row.itemId, + languageId: result.row.languageId, + locale: result.row.locale, + }); + return { changed: true, changedFields: [], + ...(delivery === undefined ? {} : { delivery }), languageId: result.row.languageId, locale: result.row.locale, operation, @@ -412,9 +523,21 @@ export const createContentTranslationEditorialService = < version: row.version, }); + // A new translation starts as a draft, so it reserves nothing - but its slug + // is checked, because "that address belongs to an article that moved" is far + // better heard now than at publish time. + const delivery = await applyDelivery(tx, { + after: row, + before: null, + itemId: row.itemId, + languageId: row.languageId, + locale: row.locale, + }); + return { changed: true, changedFields: localizedPaths, + ...(delivery === undefined ? {} : { delivery }), languageId: row.languageId, locale: row.locale, operation: "create" as const, @@ -449,9 +572,21 @@ export const createContentTranslationEditorialService = < version, }); + // The history of a deleted translation is kept, exactly as a deleted + // record's is: the URL existed, and the resolver answers 404 for it by + // finding no live translation rather than by having forgotten it. + const delivery = await applyDelivery(tx, { + after: null, + before: row, + itemId: row.itemId, + languageId: row.languageId, + locale: row.locale, + }); + return { changed: true, changedFields: [], + ...(delivery === undefined ? {} : { delivery }), languageId: row.languageId, locale: row.locale, operation: "delete" as const, @@ -591,9 +726,21 @@ export const createContentTranslationEditorialService = < version: result.version, }); + // A restore that brings an older localized slug back moves this language's + // canonical URL exactly as an edit does - and one that changed no slug + // writes nothing, which is why it runs after the diff proved something moved. + const delivery = await applyDelivery(tx, { + after: result.row, + before: current, + itemId: result.row.itemId, + languageId: result.row.languageId, + locale: result.row.locale, + }); + return { changed: true, changedFields: result.changedFields, + ...(delivery === undefined ? {} : { delivery }), languageId: result.row.languageId, locale: result.row.locale, operation: "restore" as const, @@ -632,9 +779,20 @@ export const createContentTranslationEditorialService = < version: result.version, }); + // After the guarded write, so the reservation is only taken by the writer + // that actually won this locale's version race. + const delivery = await applyDelivery(tx, { + after: result.row, + before, + itemId: result.row.itemId, + languageId: result.row.languageId, + locale: result.row.locale, + }); + return { changed: true, changedFields: result.changedFields, + ...(delivery === undefined ? {} : { delivery }), languageId: result.row.languageId, locale: result.row.locale, operation: "update" as const, diff --git a/packages/vitnode/src/content/server/translation-effects.ts b/packages/vitnode/src/content/server/translation-effects.ts index 748809bca..fba7e2209 100644 --- a/packages/vitnode/src/content/server/translation-effects.ts +++ b/packages/vitnode/src/content/server/translation-effects.ts @@ -3,10 +3,13 @@ import type { Context } from "hono"; import type { EventEmitResult } from "../../api/models/events"; import type { ContentEventAction } from "../events"; import type { AnyContentTypeDefinition } from "../types"; +import type { ContentDeliveryEffectsResult } from "./delivery-effects"; import type { AnyContentModel } from "./model"; import type { ContentSearchSyncOutcome } from "./search-sync"; import type { ContentTranslationEditorialOutcome } from "./translation-editorial-service"; +import { contentDeliveryEffects } from "./delivery-effects"; +import { reportContentEventFailures } from "./effects-log"; import { emitContentEvent } from "./emit"; import { contentSearchAdvancedValues, @@ -74,6 +77,11 @@ export interface ContentTranslationEffectsOptions { } export interface ContentTranslationEffectsResult { + /** + * The delivery events this translation mutation emitted, or `undefined` for a + * content type without `delivery`. + */ + delivery?: ContentDeliveryEffectsResult; /** * What the event transport reported, or `null` for a no-op outcome. * @@ -131,14 +139,34 @@ export const contentTranslationEffects = async ( { pluginId }, ); - if (!definition.search.enabled || !model) return { event }; + // Same rule as the base effects: the transaction is closed, so a listener that + // never heard about this translation cannot fail the request - but it must not + // vanish either. The locale travels with it, because "nobody heard about the + // Polish copy" is a different incident from "nobody heard about the record". + await reportContentEventFailures(c, { + action: EVENT_ACTION[outcome.operation], + contentTypeId: definition.id, + event, + itemId: outcome.row.itemId, + locale: outcome.locale, + }); + + const delivery = definition.delivery.enabled + ? await contentDeliveryEffects(c, definition, outcome.delivery, { + pluginId, + }) + : undefined; + const withDelivery = delivery === undefined ? {} : { delivery }; + + if (!definition.search.enabled || !model) return { ...withDelivery, event }; // The base row, because a translation's document is built from both halves and // its visibility is subordinate to the record's. const base = await model.service(c).findById(outcome.row.itemId); - if (!base) return { event }; + if (!base) return { ...withDelivery, event }; return { + ...withDelivery, event, // Scoped to the locale that moved. Omitting it would rewrite every other // language's document for a change none of them contains. diff --git a/packages/vitnode/src/content/server/translation-http-errors.ts b/packages/vitnode/src/content/server/translation-http-errors.ts index 78d694414..5bd4f58f0 100644 --- a/packages/vitnode/src/content/server/translation-http-errors.ts +++ b/packages/vitnode/src/content/server/translation-http-errors.ts @@ -4,11 +4,13 @@ import { ZodError } from "zod"; import type { ContentTranslationConflict } from "../conflicts"; import { + CONTENT_DELIVERY_CODES, CONTENT_TRANSLATION_CONFLICT_CODES, CONTENT_UNPROCESSABLE_CODES, } from "../const"; import { ContentDefaultTranslationRequired, + ContentDeliverySlugReserved, ContentInputError, ContentLanguageError, ContentRevisionNotRestorable, @@ -16,7 +18,11 @@ import { ContentTranslationItemMissing, ContentTranslationVersionConflict, } from "../errors"; -import { contentUnprocessable, rethrowAsHttpError } from "./http-errors"; +import { + contentDeliveryConflict, + contentUnprocessable, + rethrowAsHttpError, +} from "./http-errors"; /** A structured 409, in the translation union. */ export const contentTranslationConflict = ( @@ -39,6 +45,7 @@ export const contentTranslationConflict = ( * | version moved | 409 | `CONTENT_TRANSLATION_VERSION_CONFLICT` | * | default translation delete | 409 | `CONTENT_DEFAULT_TRANSLATION_REQUIRED` | * | localized slug taken | 409 | `CONTENT_TRANSLATION_UNIQUE_CONFLICT` | + * | localized slug reserved | 409 | `CONTENT_DELIVERY_SLUG_RESERVED` | * * Anything it does not recognise falls through to {@link rethrowAsHttpError}, * which owns the Postgres constraint codes - so the driver's message, which can @@ -124,6 +131,22 @@ export const withTranslationHttpErrors = async <TResult>( throw new HTTPException(404, { message: error.message }); } + // Answered in the **delivery** union rather than translated into + // `CONTENT_TRANSLATION_UNIQUE_CONFLICT`, and the difference matters to a client: + // a unique clash means another record holds that address *now*, so switching to + // it is impossible; a reservation means another record *used* to hold it and it + // still redirects, which is a different thing to explain and possibly to undo. + // It also has to be caught here rather than left to the fallthrough below, which + // rewrites every 409 the shared mapper produces into the unique-clash arm. + if (error instanceof ContentDeliverySlugReserved) { + throw contentDeliveryConflict({ + code: CONTENT_DELIVERY_CODES.slugReserved, + contentTypeId, + locale: error.locale, + slug: error.slug, + }); + } + // Written for the client on purpose, like the base service's: "send the slug // explicitly" is useless if it never leaves the server. if (error instanceof ContentInputError) { diff --git a/packages/vitnode/src/content/server/translation-model.ts b/packages/vitnode/src/content/server/translation-model.ts index e1e39caa4..1c93a4747 100644 --- a/packages/vitnode/src/content/server/translation-model.ts +++ b/packages/vitnode/src/content/server/translation-model.ts @@ -145,6 +145,23 @@ export interface ContentTranslationModel<TDefinition> { locale: string, options?: ContentTranslationOptions, ) => Promise<boolean>; + /** + * The **base** row's publication state, or `null` when the record is gone. + * + * Exposed because a translation's public reachability is subordinate to the + * record's: a published Polish translation of a draft article is not a public + * URL, so the delivery layer cannot decide whether to reserve an address without + * both halves. It lives here rather than in the editorial layer for the same + * reason `resolveLanguage` does - the base table is this repository's, and a + * second reader would be a second place the two could disagree. + * + * `{ publishedAt: null, status: undefined }` for a content type without + * publication, where a translation is visible as soon as the record is. + */ + findBasePublication: ( + itemId: number, + options?: ContentTranslationOptions, + ) => Promise<null | { publishedAt: Date | null; status: string | undefined }>; findByLanguageId: ( itemId: number, languageId: number, @@ -618,6 +635,30 @@ export const createContentTranslationModel = < return row !== undefined; }, + findBasePublication: async (itemId, options) => { + const baseColumns = table as unknown as Record<string, PgColumn>; + const [row] = await db(options) + .select( + publication + ? { + publishedAt: baseColumns.publishedAt, + status: baseColumns.status, + } + : { publishedAt: baseId, status: baseId }, + ) + .from(table) + .where(eq(baseId, itemId)) + .limit(1); + + if (!row) return null; + if (!publication) return { publishedAt: null, status: undefined }; + + return { + publishedAt: toNullableDate(row.publishedAt), + status: typeof row.status === "string" ? row.status : undefined, + }; + }, + findByLanguageId: async (itemId, languageId, options) => { const row = await readOne(itemId, languageId, db(options)); if (!row) return null; diff --git a/packages/vitnode/src/content/server/translation-publication-routes.test.ts b/packages/vitnode/src/content/server/translation-publication-routes.test.ts index b52c4b2a0..410d9127a 100644 --- a/packages/vitnode/src/content/server/translation-publication-routes.test.ts +++ b/packages/vitnode/src/content/server/translation-publication-routes.test.ts @@ -73,6 +73,12 @@ const harness = () => { create: vi.fn(), delete: vi.fn(), exists: vi.fn(), + // Stage 8 reads the base row's publication state to decide whether a + // translation's address is publicly reachable. Resolved as "published" so + // these suites keep exercising what they were written for. + findBasePublication: vi + .fn() + .mockResolvedValue({ publishedAt: new Date(0), status: "published" }), findByLanguageId: vi.fn(), findByLocale: vi.fn(), findManyForItem: vi.fn(), diff --git a/packages/vitnode/src/content/server/translation-routes.test.ts b/packages/vitnode/src/content/server/translation-routes.test.ts index 547781b78..efc0a803d 100644 --- a/packages/vitnode/src/content/server/translation-routes.test.ts +++ b/packages/vitnode/src/content/server/translation-routes.test.ts @@ -86,6 +86,12 @@ const harness = ({ allow = true }: { allow?: boolean } = {}): Harness => { create: vi.fn(), delete: vi.fn(), exists: vi.fn(), + // Stage 8 reads the base row's publication state to decide whether a + // translation's address is publicly reachable. Resolved as "published" so + // these suites keep exercising what they were written for. + findBasePublication: vi + .fn() + .mockResolvedValue({ publishedAt: new Date(0), status: "published" }), findByLanguageId: vi.fn(), findByLocale: vi.fn(), findManyForItem: vi.fn(), diff --git a/packages/vitnode/src/content/server/translation-routes.ts b/packages/vitnode/src/content/server/translation-routes.ts index a924f0951..ad7c0466b 100644 --- a/packages/vitnode/src/content/server/translation-routes.ts +++ b/packages/vitnode/src/content/server/translation-routes.ts @@ -17,6 +17,7 @@ import type { ContentTranslationModel } from "./translation-model"; import { buildRoute } from "../../api/lib/route"; import { + zodContentDeliveryConflict, zodContentTranslationConflict, zodContentUnprocessable, } from "../conflicts"; @@ -146,9 +147,18 @@ export const buildContentTranslationRoutes = < return value; }; + // A localized content type with `delivery.redirects` can also refuse a slug that + // another record's URL history owns, which answers in the delivery union rather + // than this one - see `withTranslationHttpErrors` for why the two are different + // facts. Declared as a union so both shapes are in the generated document, and a + // client written before Stage 8 still parses the arms it knows. const conflict = jsonResponse( - zodContentTranslationConflict, - "The translation moved, already exists, is the default one, or a localized value is taken", + definition.delivery.enabled && definition.delivery.redirects.enabled + ? z.union([zodContentTranslationConflict, zodContentDeliveryConflict]) + : zodContentTranslationConflict, + definition.delivery.redirects.enabled + ? "The translation moved, already exists, is the default one, a localized value is taken, or the address is reserved by a historical URL" + : "The translation moved, already exists, is the default one, or a localized value is taken", ); const invalidIdentifier = { description: "Invalid identifier or locale" }; const notFound = { diff --git a/packages/vitnode/src/content/sitemap.test.ts b/packages/vitnode/src/content/sitemap.test.ts new file mode 100644 index 000000000..21d9c9a08 --- /dev/null +++ b/packages/vitnode/src/content/sitemap.test.ts @@ -0,0 +1,199 @@ +import { describe, expect, it } from "vitest"; + +import type { ContentSitemapEntry } from "./sitemap"; + +import { + contentSitemapChunks, + contentSitemapIndexXml, + contentSitemapXml, + escapeXml, +} from "./sitemap"; + +/** + * Sitemap serialization, without a database. + * + * A sitemap is a document other people's parsers read, so the assertions here are + * mostly about bytes: valid XML, correct escaping, the elements the protocol + * defines and deterministic output. A malformed `<loc>` is not a cosmetic problem - + * a crawler may reject the whole file. + */ + +const entry = ( + overrides: Partial<ContentSitemapEntry> = {}, +): ContentSitemapEntry => ({ + changeFrequency: "weekly", + itemId: 1, + lastModified: new Date("2026-01-02T03:04:05.000Z"), + locale: null, + path: "/articles/my-article", + priority: 0.7, + ...overrides, +}); + +describe("escapeXml", () => { + it("escapes the five predefined entities", () => { + expect(escapeXml(`&<>"'`)).toBe("&<>"'"); + }); + + it("escapes the ampersand first, so nothing is double-escaped", () => { + // `&` after `<` would turn the `<` this produced into `&lt;`. + expect(escapeXml("<a & b>")).toBe("<a & b>"); + }); +}); + +describe("contentSitemapXml", () => { + it("emits a valid urlset with every configured element", () => { + const xml = contentSitemapXml({ + entries: [entry()], + origin: "https://example.com", + }); + + expect(xml).toBe( + [ + '<?xml version="1.0" encoding="UTF-8"?>', + '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">', + " <url>", + " <loc>https://example.com/articles/my-article</loc>", + " <lastmod>2026-01-02T03:04:05.000Z</lastmod>", + " <changefreq>weekly</changefreq>", + " <priority>0.7</priority>", + " </url>", + "</urlset>", + "", + ].join("\n"), + ); + }); + + it("omits changefreq and priority when the content type set none", () => { + const xml = contentSitemapXml({ + entries: [entry({ changeFrequency: null, priority: null })], + origin: "https://example.com", + }); + + expect(xml).not.toContain("changefreq"); + expect(xml).not.toContain("priority"); + expect(xml).toContain("<loc>https://example.com/articles/my-article</loc>"); + }); + + it("drops an entry whose path will not resolve rather than emitting a bad loc", () => { + const xml = contentSitemapXml({ + entries: [entry(), entry({ itemId: 2, path: "http://" })], + origin: "https://example.com", + }); + + expect(xml.match(/<url>/g)).toHaveLength(1); + }); + + it("declares the xhtml namespace only when alternates are supplied", () => { + const without = contentSitemapXml({ + entries: [entry()], + origin: "https://example.com", + }); + expect(without).not.toContain("xmlns:xhtml"); + + const withAlternates = contentSitemapXml({ + alternates: new Map([ + [ + 1, + [ + { locale: "en", path: "/en/articles/my-article" }, + { locale: "pl", path: "/pl/articles/moj-artykul" }, + ], + ], + ]), + entries: [entry({ locale: "en", path: "/en/articles/my-article" })], + origin: "https://example.com", + }); + + expect(withAlternates).toContain( + 'xmlns:xhtml="http://www.w3.org/1999/xhtml"', + ); + // Every alternate of a group is repeated inside each `<url>` - the rule + // implementations get wrong. + expect(withAlternates).toContain( + '<xhtml:link rel="alternate" hreflang="en" href="https://example.com/en/articles/my-article" />', + ); + expect(withAlternates).toContain( + '<xhtml:link rel="alternate" hreflang="pl" href="https://example.com/pl/articles/moj-artykul" />', + ); + }); + + it("is deterministic, so two processes produce identical bytes", () => { + const entries = [entry(), entry({ itemId: 2, path: "/articles/second" })]; + const first = contentSitemapXml({ entries, origin: "https://example.com" }); + const second = contentSitemapXml({ + entries, + origin: "https://example.com", + }); + + expect(first).toBe(second); + }); + + it("emits an empty but valid document for no entries", () => { + expect( + contentSitemapXml({ entries: [], origin: "https://example.com" }), + ).toBe( + [ + '<?xml version="1.0" encoding="UTF-8"?>', + '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">', + "</urlset>", + "", + ].join("\n"), + ); + }); +}); + +describe("contentSitemapIndexXml", () => { + it("emits a sitemapindex, not a urlset", () => { + const xml = contentSitemapIndexXml({ + entries: [ + { + lastModified: new Date("2026-01-02T03:04:05.000Z"), + path: "/sitemaps/blog.article-1.xml", + }, + { path: "/sitemaps/blog.article-2.xml" }, + ], + origin: "https://example.com", + }); + + expect(xml).toContain("<sitemapindex"); + expect(xml).not.toContain("<urlset"); + expect(xml).toContain( + "<loc>https://example.com/sitemaps/blog.article-1.xml</loc>", + ); + expect(xml).toContain("<lastmod>2026-01-02T03:04:05.000Z</lastmod>"); + // The second entry has no timestamp, so it carries no `lastmod` element. + expect(xml.match(/<lastmod>/g)).toHaveLength(1); + }); +}); + +describe("contentSitemapChunks", () => { + it("is one page for an empty content type, not zero", () => { + // An index that lists a file which does not exist is a broken index, and a + // content type with nothing published today will have something tomorrow. + expect(contentSitemapChunks({ total: 0 })).toStrictEqual({ + pages: 1, + size: 1_000, + }); + }); + + it("divides by the page size and rounds up", () => { + expect(contentSitemapChunks({ size: 100, total: 250 })).toStrictEqual({ + pages: 3, + size: 100, + }); + }); + + it("clamps the page size to the protocol ceiling", () => { + expect( + contentSitemapChunks({ size: 1_000_000, total: 60_000 }), + ).toStrictEqual({ pages: 2, size: 50_000 }); + }); + + it("never accepts a page size below one", () => { + expect(contentSitemapChunks({ size: 0, total: 3 })).toStrictEqual({ + pages: 3, + size: 1, + }); + }); +}); diff --git a/packages/vitnode/src/content/sitemap.ts b/packages/vitnode/src/content/sitemap.ts new file mode 100644 index 000000000..5c4a9cf30 --- /dev/null +++ b/packages/vitnode/src/content/sitemap.ts @@ -0,0 +1,220 @@ +import type { ContentSitemapChangeFrequency } from "./types"; + +import { + CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + CONTENT_SITEMAP_MAX_URLS, +} from "./const"; + +/** + * Sitemap serialization, and nothing else. + * + * Deliberately separate from the queries that produce the entries: "which URLs + * are public right now" is a keyset scan over two tables, and "what does a + * sitemap file look like" is a string. Folding them into one function would make + * the XML untestable without a database and the pagination untestable without + * parsing XML - so the delivery service owns the first and this module owns the + * second. + * + * Client-safe and pure. No Drizzle, no Hono, no `next/*`. + */ + +/** One line of a sitemap, as the delivery service produces it. */ +export interface ContentSitemapEntry { + /** One of the seven `changefreq` values, or `null` to omit the element. */ + changeFrequency: ContentSitemapChangeFrequency | null; + itemId: number; + /** + * When the representation at this URL last changed. + * + * For a localized entry that is `max(base.updatedAt, translation.updatedAt)`, + * because both halves are rendered into the page: a shared field moving changes + * every language's document even though no translation row was touched. + */ + lastModified: Date; + /** The language this URL is in, or `null` for a nonlocalized content type. */ + locale: null | string; + /** Relative, always. An origin is applied at serialization time. */ + path: string; + priority: null | number; +} + +/** + * XML's five predefined entities, escaped in the one order that is correct. + * + * `&` **first**: escaping it after `<` would turn the `<` this function just + * produced into `&lt;`. A slug is percent-encoded by the path builder so this + * is rarely load-bearing, but a sitemap is a document other people's parsers read, + * and "rarely" is not a guarantee. + */ +export const escapeXml = (value: string): string => + value + .replace(/&/g, "&") + .replace(/</g, "<") + .replace(/>/g, ">") + .replace(/"/g, """) + .replace(/'/g, "'"); + +/** + * A path turned into the absolute URL a sitemap has to carry. + * + * The protocol requires absolute URLs, which is the one place delivery cannot + * stay origin-agnostic - so the origin is a required argument here rather than an + * option. A path that will not resolve against it comes back `null` and the entry + * is dropped: a sitemap with one malformed `<loc>` is a sitemap a crawler may + * reject whole. + */ +const absolute = (origin: string, path: string): null | string => { + try { + return new URL(path, origin).toString(); + } catch { + return null; + } +}; + +/** `priority` at the one precision the protocol illustrates, without a float tail. */ +const formatPriority = (priority: number): string => priority.toFixed(1); + +/** + * A `<urlset>` document for one page of entries. + * + * `alternates` are opt-in and, when present, emitted as `xhtml:link` elements - + * the form the sitemap extension for `hreflang` actually defines, with the + * namespace declared on the root element and every alternate of a group repeated + * inside **each** of its `<url>` entries. That last rule is the one implementations + * get wrong, and it is why alternates are supplied per entry rather than derived: + * the caller has already resolved which translations are published, and this + * function does not go looking. + * + * Every entry is emitted in the order it was given, so two processes serializing + * the same page produce byte-identical documents. + */ +export const contentSitemapXml = ({ + alternates, + entries, + origin, +}: { + /** + * The alternates of each entry, keyed by `itemId`. Omit it and no `xhtml:link` + * element is emitted at all, which is a valid sitemap and the right default. + */ + alternates?: ReadonlyMap<number, readonly { locale: string; path: string }[]>; + entries: readonly ContentSitemapEntry[]; + origin: string; +}): string => { + const withAlternates = alternates !== undefined && alternates.size > 0; + const lines: string[] = [ + '<?xml version="1.0" encoding="UTF-8"?>', + withAlternates + ? '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">' + : '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">', + ]; + + for (const entry of entries) { + const loc = absolute(origin, entry.path); + if (loc === null) continue; + + lines.push(" <url>"); + lines.push(` <loc>${escapeXml(loc)}</loc>`); + lines.push( + ` <lastmod>${escapeXml(entry.lastModified.toISOString())}</lastmod>`, + ); + if (entry.changeFrequency !== null) { + lines.push(` <changefreq>${entry.changeFrequency}</changefreq>`); + } + if (entry.priority !== null) { + lines.push(` <priority>${formatPriority(entry.priority)}</priority>`); + } + + for (const alternate of alternates?.get(entry.itemId) ?? []) { + const href = absolute(origin, alternate.path); + if (href === null) continue; + + lines.push( + ` <xhtml:link rel="alternate" hreflang="${escapeXml(alternate.locale)}" href="${escapeXml(href)}" />`, + ); + } + + lines.push(" </url>"); + } + + lines.push("</urlset>"); + + return `${lines.join("\n")}\n`; +}; + +/** One file in a sitemap index. */ +export interface ContentSitemapIndexEntry { + lastModified?: Date; + /** Relative or absolute; a relative one is resolved against the origin. */ + path: string; +} + +/** + * A `<sitemapindex>` document. + * + * What a site serves at `/sitemap.xml` once one file is not enough. It is a + * separate function from {@link contentSitemapXml} because it is a separate + * document type with a separate root element - and because an index whose entries + * were `<url>` elements is the single most common way to publish a sitemap no + * crawler reads. + */ +export const contentSitemapIndexXml = ({ + entries, + origin, +}: { + entries: readonly ContentSitemapIndexEntry[]; + origin: string; +}): string => { + const lines: string[] = [ + '<?xml version="1.0" encoding="UTF-8"?>', + '<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">', + ]; + + for (const entry of entries) { + const loc = absolute(origin, entry.path); + if (loc === null) continue; + + lines.push(" <sitemap>"); + lines.push(` <loc>${escapeXml(loc)}</loc>`); + if (entry.lastModified !== undefined) { + lines.push( + ` <lastmod>${escapeXml(entry.lastModified.toISOString())}</lastmod>`, + ); + } + lines.push(" </sitemap>"); + } + + lines.push("</sitemapindex>"); + + return `${lines.join("\n")}\n`; +}; + +/** + * How many files a given number of URLs needs, and how big each one is. + * + * One `1` for an empty content type rather than `0`: a site that serves + * `/sitemaps/blog.article-1.xml` should get an empty but valid document there + * rather than a 404, because an index that lists a file which does not exist is a + * broken index and a content type with nothing published today will have + * something tomorrow. + * + * `size` is clamped to the protocol's 50,000-URL ceiling, so a caller cannot ask + * for one enormous invalid file by passing a bigger page size. + */ +export const contentSitemapChunks = ({ + size = CONTENT_SITEMAP_DEFAULT_PAGE_SIZE, + total, +}: { + size?: number; + total: number; +}): { pages: number; size: number } => { + const clamped = Math.max( + 1, + Math.min(Math.floor(size), CONTENT_SITEMAP_MAX_URLS), + ); + + return { + pages: Math.max(1, Math.ceil(Math.max(0, total) / clamped)), + size: clamped, + }; +}; diff --git a/packages/vitnode/src/content/types.ts b/packages/vitnode/src/content/types.ts index 9a7538a7d..3e3d89abe 100644 --- a/packages/vitnode/src/content/types.ts +++ b/packages/vitnode/src/content/types.ts @@ -1,4 +1,7 @@ import type { + CONTENT_DELIVERY_DESCRIPTION_KINDS, + CONTENT_DELIVERY_NO_INDEX_KINDS, + CONTENT_DELIVERY_TITLE_KINDS, CONTENT_EDITORIAL_FIELDS, CONTENT_FILTERABLE_FIELD_KINDS, CONTENT_LOCALIZATION_FALLBACKS, @@ -8,6 +11,7 @@ import type { CONTENT_SEARCH_DESCRIPTION_KINDS, CONTENT_SEARCH_TEXT_KINDS, CONTENT_SEARCH_TITLE_KINDS, + CONTENT_SITEMAP_CHANGE_FREQUENCIES, CONTENT_SYSTEM_FIELDS, CONTENT_TRANSLATION_SYSTEM_FIELDS, } from "./const"; @@ -1167,6 +1171,254 @@ export interface ResolvedContentSearchConfig< titleField: string; } +// --------------------------------------------------------------------------- +// Delivery (Stage 8) +// --------------------------------------------------------------------------- + +export type ContentSitemapChangeFrequency = + (typeof CONTENT_SITEMAP_CHANGE_FREQUENCIES)[number]; + +/** + * Field names and group leaf paths `delivery.seo` may name, of one or more kinds. + * + * Three rules, one `Extract`, and they are the same three `ContentSearchTitleField` + * enforces for the same reasons: `TPublicField` is the public allowlist, so a + * private field cannot become a `<title>`; the kind union keeps prose out of a + * title slot and a number out of a description; and a **repeatable** leaf is + * absent, because a page has one title and a repeatable has many values. + */ +export type ContentDeliveryTextField< + TFields, + TPublicField extends string, + TKind extends string, +> = Extract< + TPublicField, + | ContentFieldNamesOfKind<TFields, TKind> + | ContentLeafPathsOfKind<TFields, TKind, "group"> +>; + +/** Field names `delivery.seo.titleField` and its fallback accept. */ +export type ContentDeliveryTitleField< + TFields, + TPublicField extends string, +> = ContentDeliveryTextField< + TFields, + TPublicField, + (typeof CONTENT_DELIVERY_TITLE_KINDS)[number] +>; + +/** Field names `delivery.seo.descriptionField` and its fallback accept. */ +export type ContentDeliveryDescriptionField< + TFields, + TPublicField extends string, +> = ContentDeliveryTextField< + TFields, + TPublicField, + (typeof CONTENT_DELIVERY_DESCRIPTION_KINDS)[number] +>; + +/** Field names `delivery.seo.noIndexField` accepts. */ +export type ContentDeliveryNoIndexField< + TFields, + TPublicField extends string, +> = ContentDeliveryTextField< + TFields, + TPublicField, + (typeof CONTENT_DELIVERY_NO_INDEX_KINDS)[number] +>; + +/** + * Optional Open Graph projection, on top of the SEO one. + * + * Separate fields rather than a flag, because the two audiences differ: a + * `<title>` competes in a search result and an `og:title` competes in a chat + * preview, and an author who wants them identical simply names the same field + * twice. There is deliberately no `imageField` - see + * `apps/docs/.../content-delivery-limitations.mdx`. + */ +export interface ContentDeliveryOpenGraphConfig< + TTitle extends string = string, + TDescription extends string = string, +> { + descriptionField?: TDescription; + titleField?: TTitle; +} + +/** + * What a frontend renders in `<head>`, projected from public fields. + * + * Every slot is optional and every fallback is explicit. There is no "derive a + * description from the first 160 characters of the body": a summary somebody did + * not write is a summary nobody reviewed, and it would silently become the + * description of every page that forgot to set one. + */ +export interface ContentDeliverySeoConfig< + TTitle extends string = string, + TDescription extends string = string, + TNoIndex extends string = string, +> { + descriptionField?: TDescription; + /** Used when `descriptionField` resolves to `null` or an empty string. */ + fallbackDescriptionField?: TDescription; + /** Used when `titleField` resolves to `null` or an empty string. */ + fallbackTitleField?: TTitle; + /** + * A **shared** boolean field that keeps one record out of the sitemap and + * reports `robots: { index: false }`. + * + * Shared rather than localized on purpose: the two consumers have to agree, and + * a per-locale value would make "is this record in the sitemap" a question with + * one answer per language while the record has one canonical decision. A + * localized field here is a definition-time error. + */ + noIndexField?: TNoIndex; + openGraph?: ContentDeliveryOpenGraphConfig<TTitle, TDescription>; + titleField?: TTitle; +} + +/** + * Automatic redirects from a record's historical public URLs. + * + * Needs a slug field, which `publicApi` already guarantees. What it adds is + * persistence: every slug that was ever *publicly addressable* is written to + * `core_content_slug_history`, which is what makes an old URL resolvable after + * the row has moved on - and what reserves it, so unrelated content cannot + * quietly inherit somebody else's incoming links. + */ +export interface ContentDeliveryRedirectsConfig { + enabled: true; +} + +export interface ContentDeliverySitemapConfig { + /** One of the seven `changefreq` values the protocol defines. */ + changeFrequency?: ContentSitemapChangeFrequency; + enabled: true; + /** `0` to `1` inclusive. */ + priority?: number; +} + +/** + * `x-default` for a localized content type. + * + * `"defaultLocale"` is the only supported mapping, and that is deliberate: an + * `x-default` has to point at a URL that actually resolves, and the default + * locale's canonical path is the one URL a localized record is guaranteed to have + * whenever it is public at all. Omit the block and no `x-default` is emitted - + * inventing a locale-less route that the engine does not serve would be worse + * than emitting nothing. + */ +export interface ContentDeliveryHreflangConfig { + xDefault: "defaultLocale"; +} + +/** + * Opts a content type into the delivery layer: canonical URLs, slug history, + * redirects, localized alternates, SEO projection and sitemap entries. + * + * Requires `publicApi: { enabled: true }`, checked at compile time through + * `TPublicEnabled` and again at definition time - a content type with no public + * API has no public URL, so there is nothing for delivery to be about. + * + * `enabled` is literal `true` for the same reason every other opt-in's is: every + * conditional keys off `{ enabled: true }`, and a widened `boolean` would + * silently resolve to "no delivery". + */ +export interface ContentDeliveryConfig< + // Both flags default to `true` rather than `boolean`, which is what keeps the bare + // `ContentDeliveryConfig` usable as a widened parameter type: `boolean extends + // true` is false, so a `boolean` default would resolve `enabled` to `never` and + // make the erased form describe a config nobody can write. + TPublicEnabled extends boolean = true, + TEditorialEnabled extends boolean = true, + TTitle extends string = string, + TDescription extends string = string, + TNoIndex extends string = string, +> { + /** + * Literal `true`, and only when the content type has a public API. + * + * `never` otherwise, which is what turns "delivery needs `publicApi`" into a + * compile error on the `enabled: true` itself rather than a boot-time throw. The + * runtime check stays as well, for a JavaScript caller and for a value that + * widened somewhere upstream. + */ + enabled: TPublicEnabled extends true ? true : never; + hreflang?: ContentDeliveryHreflangConfig; + /** + * Gated on **editorial** as well as on the public API, and the second gate is not + * a taste decision: slug history has to be written in the same transaction as the + * slug mutation, the version check and the revision - and the only mutation paths + * that own such a transaction are the editorial ones. Without `editorial` a + * content type writes through the plain repository, which has no version to guard + * and no history to write, so `redirects: { enabled: true }` there would be a + * feature that silently records nothing. + * + * Only `redirects` is gated. Canonical URLs, SEO, alternates, `hreflang` and the + * sitemap are all reads over data the content type already has, and they remain + * available without `editorial`. + */ + redirects?: TPublicEnabled extends true + ? TEditorialEnabled extends true + ? ContentDeliveryRedirectsConfig | { enabled: false } + : { enabled: false } + : { enabled: false }; + seo?: ContentDeliverySeoConfig<TTitle, TDescription, TNoIndex>; + sitemap?: ContentDeliverySitemapConfig | { enabled: false }; +} + +/** + * Whether a `delivery` argument opted in. + * + * Read back off the argument for the same reason `ContentSearchEnabled` is: the + * whole object is inferred as one type parameter, and an intersection member is + * not an inference site, so this is the only way the literal survives. + */ +export type ContentDeliveryEnabled<TDelivery> = TDelivery extends { + enabled: true; +} + ? true + : false; + +/** `delivery.seo` after `defineContentType` has filled in every default. */ +export interface ResolvedContentDeliverySeoConfig { + descriptionField: null | string; + fallbackDescriptionField: null | string; + fallbackTitleField: null | string; + noIndexField: null | string; + openGraph: null | { + descriptionField: null | string; + titleField: null | string; + }; + titleField: null | string; +} + +/** `delivery` after `defineContentType` has filled in every default. */ +export interface ResolvedContentDeliveryConfig< + TEnabled extends boolean = boolean, +> { + enabled: TEnabled; + hreflang: { xDefault: "defaultLocale" | null }; + redirects: { enabled: boolean }; + seo: ResolvedContentDeliverySeoConfig; + sitemap: { + changeFrequency: ContentSitemapChangeFrequency | null; + enabled: boolean; + priority: null | number; + }; + /** + * Where the slug that addresses this content type lives. + * + * `"localized"` when `publicApi.slugField` is a localized field, `"shared"` + * otherwise - and it is the only thing the whole delivery layer branches on to + * decide which language a historical URL belongs to. A localized slug is + * reserved per language, a shared one once for the content type, and both are + * correct for the URLs they actually produce. + * + * `"none"` for a content type without delivery, which addresses nothing. + */ + slugScope: "localized" | "none" | "shared"; +} + // --------------------------------------------------------------------------- // Editorial // --------------------------------------------------------------------------- @@ -1503,6 +1755,20 @@ export type SchedulableContentTypeDefinition = publication: { enabled: true }; }; +/** + * A content type with a delivery layer: canonical URLs, alternates, SEO and a + * sitemap. + * + * Both halves are pinned, because delivery is defined in terms of the public + * projection: the canonical path is built from `publicApi.path` and the exposed + * slug field, and every SEO field is one of `publicApi.fields`. A content type + * without a public allowlist cannot reach the delivery service at all - which is + * a compile error rather than an empty response. + */ +export type DeliverableContentTypeDefinition = PublicContentTypeDefinition & { + delivery: { enabled: true }; +}; + /** * A content type whose records exist in more than one language. * @@ -1563,10 +1829,17 @@ export interface ContentTypeDefinition< TPreviewEnabled extends boolean = boolean, TSchedulingEnabled extends boolean = boolean, TLocalizationEnabled extends boolean = boolean, + TDeliveryEnabled extends boolean = boolean, > { admin: ResolvedContentAdminConfig; /** Generated junction tables, child tables and the leaf-path mapping. */ advanced: ResolvedContentAdvancedConfig; + /** + * Canonical URLs, slug history, SEO and sitemap - or the disabled default when + * `delivery` is omitted, which is what keeps every Stage 1-7 content type + * byte-identical. + */ + delivery: ResolvedContentDeliveryConfig<TDeliveryEnabled>; /** Editorial workflow, or the disabled default when `editorial` is omitted. */ editorial: ResolvedContentEditorialConfig< TEditorialEnabled, @@ -1598,7 +1871,8 @@ export interface ContentTypeDefinition< TEditorialEnabled, TPreviewEnabled, TSchedulingEnabled, - TLocalizationEnabled + TLocalizationEnabled, + TDeliveryEnabled > >; /** Search synchronization, or the disabled default when `search` is omitted. */ diff --git a/packages/vitnode/src/database/content.ts b/packages/vitnode/src/database/content.ts index 2d0019817..19f226950 100644 --- a/packages/vitnode/src/database/content.ts +++ b/packages/vitnode/src/database/content.ts @@ -10,9 +10,11 @@ import type { import { CONTENT_ACTOR_TYPES, + CONTENT_DELIVERY_PATH_MAX_LENGTH, CONTENT_REVISION_OPERATIONS, CONTENT_SCHEDULE_ACTIONS, CONTENT_SCHEDULE_STATUSES, + CONTENT_SLUG_DEFAULT_LENGTH, } from "../content/const"; import { core_users } from "./users"; @@ -208,6 +210,90 @@ export const core_content_schedules = pgTable( export type ContentScheduleRow = typeof core_content_schedules.$inferSelect; +/** + * Every slug that has ever been a **publicly addressable** URL, for content types + * with `delivery.redirects`. + * + * Shared and foreign-key-free for the same two reasons as + * {@link core_content_revisions}: the target table is generated at runtime so + * core's static schema cannot name it, and a URL's history stays true after the + * record is gone. It is scoped by `(contentTypeId, itemId)` on every query, and a + * delete leaves the history in place - an incoming link to a deleted article is + * exactly the diagnostic somebody will want, and the resolver answers 404 for it + * by reading the live record rather than by having forgotten the URL. + * + * `languageId` is the locale identity, and `NULL` means the slug is **shared**: + * either the content type is not localized, or it is and its slug lives on the + * base row. That single column is what makes `/en/articles/hello` and + * `/pl/articles/hello` two independent histories - changing the English URL + * creates no Polish redirect - while a shared slug stays one reservation covering + * every language it appears in. + * + * `retiredAt` is `NULL` while the slug is the record's *current* address and is + * stamped when it moves away. Both states are stored, which is what makes the + * uniqueness below a **reservation** rather than only a log: a retired URL cannot + * be claimed by unrelated content, so nobody's incoming links quietly change + * meaning. + */ +export const core_content_slug_history = pgTable( + "core_content_slug_history", + t => ({ + id: t.serial().primaryKey(), + pluginId: t.varchar({ length: 255 }).notNull(), + contentTypeId: t.varchar({ length: 100 }).notNull(), + itemId: t.integer().notNull(), + /** `NULL` for a shared slug. See the table comment. */ + languageId: t.integer(), + slug: t.varchar({ length: CONTENT_SLUG_DEFAULT_LENGTH }).notNull(), + /** + * The canonical path this slug produced, e.g. `/pl/articles/stary-slug`. + * + * Stored rather than rebuilt on read, and the reason is that it is the one + * thing the engine cannot recompute later: a path is built from + * `publicApi.path`, which is source configuration a developer may change. The + * URL that was live is a historical fact, so it is recorded as one - and the + * AdminCP shows exactly the address somebody's bookmark holds. + */ + path: t.varchar({ length: CONTENT_DELIVERY_PATH_MAX_LENGTH }).notNull(), + createdAt: t.timestamp().notNull().defaultNow(), + /** When this slug stopped being the record's address. `NULL` while current. */ + retiredAt: t.timestamp(), + }), + t => [ + // The reservation, and the resolver's lookup, in one index each. + // + // Two partial uniques rather than one over a nullable `languageId`, for + // exactly the reason `core_content_revisions` needs two: Postgres treats every + // `NULL` as distinct, so a single key including it would enforce nothing at + // all for the shared case it exists to protect. + // + // No `pluginId` in either key. `validateContentTypes` rejects a duplicate + // content type id across every installed plugin at boot, so an id already + // identifies one content type - adding the owner would widen the index without + // excluding anything. It is still a column, because ownership is what a + // cleanup or an audit keys off. + uniqueIndex("core_content_slug_history_shared_unique") + .on(t.contentTypeId, t.slug) + .where(sql`"languageId" IS NULL`), + uniqueIndex("core_content_slug_history_locale_unique") + .on(t.contentTypeId, t.languageId, t.slug) + .where(sql`"languageId" IS NOT NULL`), + // One record's history, for the AdminCP panel and for retiring the slug a + // mutation just moved away from. The unique indexes above cannot serve it: + // they lead with the slug rather than with the item, and a partial index is + // only usable for queries the planner can prove match its predicate. + index("core_content_slug_history_item_idx").on( + t.contentTypeId, + t.itemId, + t.languageId, + ), + index("core_content_slug_history_plugin_id_idx").on(t.pluginId), + ], +).enableRLS(); + +export type ContentSlugHistoryRow = + typeof core_content_slug_history.$inferSelect; + /** Re-exported so `src/database` consumers need not reach into `content/`. */ export type { ContentAnyRevisionSnapshot, diff --git a/packages/vitnode/src/locales/en.json b/packages/vitnode/src/locales/en.json index 80eb8b581..1042340bd 100644 --- a/packages/vitnode/src/locales/en.json +++ b/packages/vitnode/src/locales/en.json @@ -527,6 +527,23 @@ "unavailable": "Preview is not configured on this deployment. Set CONTENT_PREVIEW_SECRET to at least 32 random bytes and restart the API.", "live": "This record has no saved version yet, so the link shows it live - it will follow any edits made before the reviewer opens it." }, + "delivery": { + "title": "Delivery for this {name}", + "desc": "Where this record lives on the public site, and where it used to.", + "canonical": "Canonical URL", + "no_canonical": "This record has no public URL yet. It gets one when it is published.", + "historical": "Historical URLs", + "no_history": "No previous URLs. Changing the address of a published record adds one here.", + "redirect_active": "redirects to the current URL", + "redirect_inactive": "not redirecting while this is unpublished", + "retired_at": "Replaced", + "inactive_note": "These URLs start redirecting again as soon as this record is published.", + "load_failed": "This record's delivery state could not be read.", + "states": { + "published": "Published", + "not_published": "Not published" + } + }, "schedule": { "title": "Schedule this {name}", "desc": "Publish or unpublish <title> at a set time. Scheduling changes nothing now.", @@ -577,7 +594,8 @@ "forbidden": "You do not have permission to do this.", "version_conflict": "Someone else saved this while you were editing. Your changes are still here.", "unique_conflict": "A record with these values already exists.", - "not_restorable": "This version cannot be restored: {fields} no longer fit this content type." + "not_restorable": "This version cannot be restored: {fields} no longer fit this content type.", + "slug_reserved": "That address is reserved: another record used it publicly and it still redirects there. Pick a different one." }, "translations": { "shared_tab": "Shared", @@ -614,7 +632,8 @@ "unique_conflict": "Another record already uses that address in this language.", "exists": "This language already has a translation. Reload the tab to edit it.", "language_disabled": "This language is switched off on this installation, so its content cannot be written.", - "default_required": "This is the default language, so its translation cannot be deleted." + "default_required": "This is the default language, so its translation cannot be deleted.", + "slug_reserved": "That address is reserved in this language: another record used it publicly and it still redirects there." }, "history": { "show": "Show this language's history", diff --git a/packages/vitnode/src/tests/content-fixtures.ts b/packages/vitnode/src/tests/content-fixtures.ts index 6b6b2c893..cc4082127 100644 --- a/packages/vitnode/src/tests/content-fixtures.ts +++ b/packages/vitnode/src/tests/content-fixtures.ts @@ -483,3 +483,100 @@ export const testAdvancedLocalizedContentType = defineContentType({ list: { columns: ["featured", "status"] }, }, }); + +/** + * The Stage 8 shape: `testPostContentType` plus the whole delivery layer. + * + * A separate fixture rather than a flag on the post, for the same reason the + * searchable and editorial ones are separate: leaving the post exactly as it was is + * what proves a content type without `delivery` produces the same tables, the same + * routes, the same cache tags and the same events it always did. + */ +export const testDeliveredPostContentType = defineContentType({ + id: "test.delivered-post", + tableName: "test_delivered_posts", + fields: { + title: field.text({ required: true, minLength: 3, maxLength: 200 }), + slug: field.slug({ source: "title" }), + excerpt: field.textarea({ maxLength: 500, nullable: true }), + hidden: field.boolean({ defaultValue: false }), + }, + publication: { enabled: true }, + editorial: { enabled: true }, + publicApi: { + enabled: true, + path: "delivered-posts", + fields: ["id", "title", "slug", "excerpt", "hidden", "publishedAt"], + defaultOrderBy: "publishedAt", + }, + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { + titleField: "title", + descriptionField: "excerpt", + noIndexField: "hidden", + openGraph: { titleField: "title", descriptionField: "excerpt" }, + }, + sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 }, + }, + admin: { label: { plural: "Test Delivered", singular: "Test Delivered" } }, +}); + +/** + * A localized delivery content type: locale-prefixed URLs and per-locale history. + * + * Its slug is `localized: true`, which is what `delivery.redirects` requires on a + * localized content type - a shared slug would give every language the same segment, + * so one retired address would belong to several URLs at once. + */ +export const testDeliveredLocalizedContentType = defineContentType({ + id: "test.delivered-localized", + tableName: "test_delivered_localized", + localization: { enabled: true, defaultLocale: "en", fallback: "default" }, + publication: { enabled: true }, + editorial: { enabled: true }, + fields: { + title: field.text({ localized: true, required: true, maxLength: 200 }), + slug: field.slug({ localized: true, source: "title" }), + seo: field.group({ + localized: true, + nullable: true, + fields: { + title: field.text({ nullable: true, maxLength: 200 }), + description: field.textarea({ nullable: true, maxLength: 500 }), + }, + }), + }, + publicApi: { + enabled: true, + path: "delivered-localized", + fields: [ + "id", + "title", + "slug", + "seo.title", + "seo.description", + "publishedAt", + ], + defaultOrderBy: "publishedAt", + }, + delivery: { + enabled: true, + redirects: { enabled: true }, + hreflang: { xDefault: "defaultLocale" }, + seo: { + titleField: "seo.title", + fallbackTitleField: "title", + descriptionField: "seo.description", + }, + sitemap: { enabled: true, changeFrequency: "daily", priority: 0.5 }, + }, + admin: { + label: { + plural: "Test Delivered Localized", + singular: "Test Delivered Localized", + }, + list: { columns: ["status", "updatedAt"] }, + }, +}); diff --git a/packages/vitnode/src/tests/openapi-validate.ts b/packages/vitnode/src/tests/openapi-validate.ts new file mode 100644 index 000000000..4d104c152 --- /dev/null +++ b/packages/vitnode/src/tests/openapi-validate.ts @@ -0,0 +1,185 @@ +/** + * A JSON Schema check over the subset OpenAPI 3.0 documents actually contain. + * + * It exists because "the runtime response matches the OpenAPI schema" cannot be + * asserted with the Zod object the route was built from. `z.date()` renders in + * the document as `{ type: "string", format: "date-time" }` - which is exactly + * what `c.json(row)` puts on the wire - but the Zod object itself rejects that + * string, so parsing with it would report a contract break where the contract is + * kept. The document is what a generated client is built from, so the document + * is what a response has to satisfy. + * + * Deliberately small: `type`, `properties`, `required`, `nullable`, `enum`, + * `format: date-time`, `items`, `additionalProperties`, `oneOf`/`anyOf`/`allOf` + * and `$ref`. That is everything `@hono/zod-openapi` emits for the generated + * Content Engine routes, and anything it does not understand is reported rather + * than quietly passed. + */ + +export type JsonSchemaLike = Record; + +const ISO_DATE_TIME = + /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?(Z|[+-]\d{2}:\d{2})$/; + +const typeOf = (value: unknown): string => { + if (value === null) return "null"; + if (Array.isArray(value)) return "array"; + + return typeof value; +}; + +const resolveRef = ( + schema: JsonSchemaLike, + document: JsonSchemaLike, +): JsonSchemaLike => { + const ref = schema.$ref; + if (typeof ref !== "string") return schema; + + const path = ref.replace(/^#\//, "").split("/"); + let current: unknown = document; + for (const segment of path) { + current = (current as Record | undefined)?.[segment]; + } + + return (current as JsonSchemaLike | undefined) ?? {}; +}; + +/** + * Every way `value` fails `schema`, as dotted paths with a reason. + * + * An empty array means the response is valid. Returning the whole list rather + * than the first problem is deliberate - a response that is wrong in four places + * should say so once, not four runs in a row. + */ +export const validateAgainstJsonSchema = ( + value: unknown, + rawSchema: JsonSchemaLike, + document: JsonSchemaLike = {}, + path = "", +): string[] => { + const schema = resolveRef(rawSchema, document); + const at = path === "" ? "(root)" : path; + const issues: string[] = []; + + const branches = ["oneOf", "anyOf"] as const; + for (const key of branches) { + const options = schema[key]; + if (!Array.isArray(options)) continue; + + const matched = options.some( + option => + validateAgainstJsonSchema( + value, + option as JsonSchemaLike, + document, + path, + ).length === 0, + ); + + return matched ? [] : [`${at}: matched none of ${key}`]; + } + + if (Array.isArray(schema.allOf)) { + for (const option of schema.allOf) { + issues.push( + ...validateAgainstJsonSchema( + value, + option as JsonSchemaLike, + document, + path, + ), + ); + } + } + + if (value === null) { + // OpenAPI 3.0 spells nullability as a sibling flag rather than as a type, + // which is why this is not simply `type.includes("null")`. + return schema.nullable === true || schema.type === undefined + ? issues + : [...issues, `${at}: null, but the document does not allow it`]; + } + + const expected = schema.type; + if (typeof expected === "string") { + const actual = typeOf(value); + const ok = + expected === "integer" + ? actual === "number" && Number.isInteger(value) + : expected === "number" + ? actual === "number" + : actual === expected; + + if (!ok) { + return [...issues, `${at}: expected ${expected}, got ${actual}`]; + } + } + + if (Array.isArray(schema.enum) && !schema.enum.includes(value)) { + issues.push(`${at}: ${JSON.stringify(value)} is not one of the enum`); + } + + if ( + schema.format === "date-time" && + typeof value === "string" && + !ISO_DATE_TIME.test(value) + ) { + issues.push(`${at}: "${value}" is not an ISO date-time`); + } + + if (expected === "array" && Array.isArray(value)) { + const items = schema.items as JsonSchemaLike | undefined; + if (items) { + value.forEach((entry, index) => { + issues.push( + ...validateAgainstJsonSchema( + entry, + items, + document, + `${path}[${index}]`, + ), + ); + }); + } + } + + if (expected === "object" || (expected === undefined && schema.properties)) { + const object = value as Record; + const properties = (schema.properties ?? {}) as Record< + string, + JsonSchemaLike + >; + const required = Array.isArray(schema.required) + ? (schema.required as string[]) + : []; + + for (const name of required) { + if (!(name in object)) + issues.push(`${at}.${name}: missing, but required`); + } + + for (const [name, entry] of Object.entries(object)) { + const property = properties[name]; + if (!property) { + // `additionalProperties: false` is what a strict object emits, and a key + // the document does not describe is a field a generated client will + // silently drop - or, on a public route, a field nobody meant to ship. + if (schema.additionalProperties === false) { + issues.push(`${at}.${name}: not described by the document`); + } + continue; + } + + issues.push( + ...validateAgainstJsonSchema( + entry, + property, + document, + path === "" ? name : `${path}.${name}`, + ), + ); + } + } + + return issues; +}; diff --git a/packages/vitnode/src/views/admin/views/content/actions/delivery-action.tsx b/packages/vitnode/src/views/admin/views/content/actions/delivery-action.tsx new file mode 100644 index 000000000..2261c8b37 --- /dev/null +++ b/packages/vitnode/src/views/admin/views/content/actions/delivery-action.tsx @@ -0,0 +1,107 @@ +"use client"; + +import { LinkIcon } from "lucide-react"; +import { useTranslations } from "next-intl"; +import dynamic from "next/dynamic"; +import React from "react"; + +import { useAdminStaffPermission } from "@/components/staff-permission/provider"; +import { Button } from "@/components/ui/button"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogHeader, + DialogTitle, + DialogTrigger, +} from "@/components/ui/dialog"; +import { Loader } from "@/components/ui/loader"; +import { + Tooltip, + TooltipContent, + TooltipProvider, + TooltipTrigger, +} from "@/components/ui/tooltip"; +import { CONTENT_PERMISSIONS } from "@/content/const"; + +// The panel fetches a record's whole URL history, so it is loaded when the dialog +// is - the same treatment the edit form and the revision history get, and the +// reason a 25-row table costs 25 buttons rather than 25 queries. +const DeliveryPanel = dynamic(async () => + import("./delivery/delivery-panel").then(mod => ({ + default: mod.DeliveryPanel, + })), +); + +/** + * The delivery row action: canonical URL, publication state, historical URLs. + * + * Gated by `can_view`, and by nothing else. It reports what the slug mutations + * already did, so the permission that allowed the mutation is the only one it + * needs - a `can_manage_redirects` for a read-only screen would be a permission + * every install has to configure for no decision it can make. + */ +export const DeliveryContentAction = ({ + contentTypeId, + id, + locale, + permissionModule, + pluginId, + singular, +}: { + contentTypeId: string; + id: number; + /** The language whose URLs to show, when the list is viewed in one. */ + locale?: string; + permissionModule: string; + pluginId: string; + singular: string; +}) => { + const t = useTranslations("core.content.delivery"); + const canView = useAdminStaffPermission({ + module: permissionModule, + permission: CONTENT_PERMISSIONS.view, + plugin: pluginId, + }); + + if (!canView) return null; + + const label = t("title", { name: singular }); + + return ( + + + + + + + } + /> + } + /> + + + + {label} + {t("desc")} + + + }> + + + + + + {label} + + + ); +}; diff --git a/packages/vitnode/src/views/admin/views/content/actions/delivery-api.server.ts b/packages/vitnode/src/views/admin/views/content/actions/delivery-api.server.ts new file mode 100644 index 000000000..e751c0514 --- /dev/null +++ b/packages/vitnode/src/views/admin/views/content/actions/delivery-api.server.ts @@ -0,0 +1,74 @@ +"use server"; + +import { z } from "zod"; + +import { findFrontendContentType } from "@/content/admin/config"; +import { contentApiFetch } from "@/content/admin/fetch.server"; + +/** + * One address a record has answered to. + * + * Exactly what the admin route publishes and not one field more: the storage + * columns behind it - `languageId`, `pluginId`, the row id - are details of + * `core_content_slug_history`, and a panel that displayed them would make them + * part of a contract nobody meant to sign. + */ +const zodDeliveryEntry = z.object({ + createdAt: z.coerce.date(), + path: z.string(), + /** `null` while this is the record's current address. */ + retiredAt: z.coerce.date().nullable(), + slug: z.string(), +}); + +const zodDelivery = z.object({ + canonicalPath: z.string().nullable(), + history: z.array(zodDeliveryEntry), + isPublic: z.boolean(), + locale: z.string().nullable(), +}); + +export type ContentDeliveryPanelData = z.infer; + +export interface ContentDeliveryPanelResult { + data?: ContentDeliveryPanelData; + error?: string; +} + +/** + * Reads one record's delivery state for the AdminCP panel. + * + * A Server Action rather than a fetch in the page, because the panel is lazy: it + * loads when somebody opens the dialog, and a record's URL history is not worth a + * query on every row of a 25-row table. + * + * `can_view` is enforced by the route it calls, not here - the AdminCP's session + * cookie travels with the request and the generated route carries the permission, + * which is the same arrangement every other content action uses. There is + * deliberately no `can_manage_redirects`: this screen manages nothing. + */ +export const readContentDeliveryAction = async ( + contentTypeId: string, + id: number, + locale?: string, +): Promise => { + const entry = findFrontendContentType(contentTypeId); + if (!entry) return { error: "Unknown content type." }; + + const result = await contentApiFetch({ + definition: entry.definition, + method: "get", + path: `/${id}/delivery`, + pluginId: entry.pluginId, + query: locale === undefined ? undefined : { locale }, + schema: zodDelivery, + }); + + if (result.status !== 200 || !result.data) { + return { + error: result.error ?? "This record's delivery state could not be read.", + }; + } + + return { data: result.data }; +}; diff --git a/packages/vitnode/src/views/admin/views/content/actions/delivery/delivery-panel.tsx b/packages/vitnode/src/views/admin/views/content/actions/delivery/delivery-panel.tsx new file mode 100644 index 000000000..ce69d4c5c --- /dev/null +++ b/packages/vitnode/src/views/admin/views/content/actions/delivery/delivery-panel.tsx @@ -0,0 +1,147 @@ +"use client"; + +import { CheckIcon, LinkIcon, XIcon } from "lucide-react"; +import { useTranslations } from "next-intl"; +import React from "react"; + +import { DateFormat } from "@/components/date-format"; +import { Badge } from "@/components/ui/badge"; +import { Loader } from "@/components/ui/loader"; + +import type { ContentDeliveryPanelData } from "../delivery-api.server"; + +import { readContentDeliveryAction } from "../delivery-api.server"; + +/** + * The read-only delivery panel: where a record lives, and where it used to. + * + * Read-only on purpose, and it is the deliberate scope of Stage 8. A redirect is + * somebody else's incoming link, so deleting one silently breaks traffic nobody in + * this dialog can see - that is a destructive action, and a destructive action needs + * a permission of its own, a confirmation that explains the consequence, and an + * audit trail. Displaying the history is useful today; managing it is a product, + * not a button. + */ +export const DeliveryPanel = ({ + contentTypeId, + id, + locale, +}: { + contentTypeId: string; + id: number; + /** The language whose URLs to show, on a content type with localized slugs. */ + locale?: string; +}) => { + const t = useTranslations("core.content.delivery"); + const [state, setState] = React.useState< + | { data: ContentDeliveryPanelData; status: "ready" } + | { message: string; status: "error" } + | { status: "loading" } + >({ status: "loading" }); + + React.useEffect(() => { + let active = true; + + void readContentDeliveryAction(contentTypeId, id, locale).then(result => { + if (!active) return; + + setState( + result.data + ? { data: result.data, status: "ready" } + : { message: result.error ?? t("load_failed"), status: "error" }, + ); + }); + + return () => { + active = false; + }; + }, [contentTypeId, id, locale, t]); + + if (state.status === "loading") return ; + + if (state.status === "error") { + return ( +

+ {state.message} +

+ ); + } + + const { canonicalPath, history, isPublic } = state.data; + const historical = history.filter(entry => entry.retiredAt !== null); + + return ( +
+
+

{t("canonical")}

+ + {canonicalPath === null ? ( +

+ {t("no_canonical")} +

+ ) : ( +

+ + {canonicalPath} +

+ )} + + + {isPublic ? ( + + ) : ( + + )} + {isPublic ? t("states.published") : t("states.not_published")} + +
+ +
+

{t("historical")}

+ + {historical.length === 0 ? ( +

+ {t("no_history")} +

+ ) : ( +
    + {historical.map(entry => ( +
  • + + {entry.path} + + → + + + {canonicalPath === null + ? t("redirect_inactive") + : t("redirect_active")} + + + + {entry.retiredAt === null ? null : ( + + {t("retired_at")} + + + )} +
  • + ))} +
+ )} + + {/* Said out loud, because it is the one thing about this screen somebody + will assume is wrong: an unpublished record's old URLs stop redirecting + and start again when it comes back. */} + {historical.length > 0 && canonicalPath === null ? ( +

+ {t("inactive_note")} +

+ ) : null} +
+
+ ); +}; diff --git a/packages/vitnode/src/views/admin/views/content/actions/mutation-api.server.ts b/packages/vitnode/src/views/admin/views/content/actions/mutation-api.server.ts index 4cd467ce8..806c2d441 100644 --- a/packages/vitnode/src/views/admin/views/content/actions/mutation-api.server.ts +++ b/packages/vitnode/src/views/admin/views/content/actions/mutation-api.server.ts @@ -4,8 +4,10 @@ import { revalidatePath } from "next/cache"; import { z } from "zod"; import type { ContentPublicLocaleState } from "@/content/cache"; +import type { ContentDeliveryInvalidation } from "@/content/cache"; import type { ContentConflict, + ContentDeliveryConflict, ContentScheduleRejection, ContentUnprocessable, } from "@/content/conflicts"; @@ -25,6 +27,7 @@ import { contentApiFetch } from "@/content/admin/fetch.server"; import { isContentPubliclyVisible } from "@/content/cache"; import { parseContentConflict, + parseContentDeliveryConflict, parseContentScheduleRejection, parseContentUnprocessable, } from "@/content/conflicts"; @@ -51,6 +54,15 @@ interface MutationResult { * sentence. */ conflict?: ContentConflict; + /** + * `CONTENT_DELIVERY_SLUG_RESERVED`, naming the address and its locale. + * + * Its own field rather than a third arm of `conflict`, because the two share a + * status and need different words: a unique clash is "another record holds that + * address now", and this is "another record used to hold it and it still + * redirects there". + */ + delivery?: ContentDeliveryConflict; error?: string; /** Why a schedule was refused, when the API said. */ rejection?: ContentScheduleRejection; @@ -66,6 +78,7 @@ const failure = (result: { status: number; }): MutationResult => ({ conflict: parseContentConflict(result.error) ?? undefined, + delivery: parseContentDeliveryConflict(result.error) ?? undefined, error: result.error ?? "", rejection: parseContentScheduleRejection(result.error) ?? undefined, status: result.status, @@ -190,6 +203,7 @@ const invalidate = ( revalidateContent( { contentTypeId: definition.id, + ...deliveryInvalidationFor(definition, before, after, previous, current), id, isPublic: current.isPublic, // Both, so a slug change stops the old URL and starts the new one. @@ -200,6 +214,71 @@ const invalidate = ( ); }; +/** + * The delivery half of a nonlocalized mutation's invalidation. + * + * `{}` for a content type without `delivery`, so spreading it leaves the input - and + * therefore the tag list - exactly as it was. + * + * `contentChanged` is decided by comparing `updatedAt` across the write, which is not + * a proxy for "did the sitemap change" but *the value the sitemap serializes*: a + * sitemap entry's `` is `base.updatedAt`, so the two move together by + * construction. It also answers "was this a no-op" for free - the engine issues no + * `UPDATE` for an update that changed nothing, so the timestamp does not move and the + * cached sitemap is still correct. + * + * `indexChanged` is public reachability flipping, and nothing else: an index lists + * files and counts URLs, so a slug change or a title edit leaves it alone. + */ +const deliveryInvalidationFor = ( + definition: AnyContentTypeDefinition, + before: ContentRow | undefined, + after: ContentRow | undefined, + previous: { isPublic: boolean }, + current: { isPublic: boolean }, +): { delivery?: ContentDeliveryInvalidation } => { + if (!definition.delivery.enabled) return {}; + + const reachable = previous.isPublic || current.isPublic; + + return { + delivery: { + sitemap: { + contentChanged: reachable && timestampMoved(before, after), + indexChanged: previous.isPublic !== current.isPublic, + }, + }, + }; +}; + +/** + * Whether `updatedAt` moved across a write. + * + * `true` when either side is missing - a create or a delete - because the record + * appeared or disappeared and there is no pair to compare. Unparseable values are + * treated the same way: a cached sitemap that might be stale is worse than a cache + * miss. + */ +const timestampMoved = ( + before: ContentRow | undefined, + after: ContentRow | undefined, +): boolean => { + const at = (row: ContentRow | undefined): null | number => { + const value = row?.updatedAt; + if (value instanceof Date) return value.getTime(); + if (typeof value !== "string") return null; + + const parsed = new Date(value).getTime(); + + return Number.isNaN(parsed) ? null : parsed; + }; + + const first = at(before); + const second = at(after); + + return first === null || second === null || first !== second; +}; + export const createContentAction = async ( contentTypeId: string, values: Record, @@ -594,12 +673,28 @@ export const deleteContentAction = async ( // A delete is final, so the question is "was it ever published?" rather // than "was it live a second ago". `publishedAt` survives an unpublish, and // expiring a URL that is now gone forever costs nothing. + const removed = publicStateOf(definition, result.data); + + const wasEverPublic = result.data?.publishedAt != null; + revalidateContent( { contentTypeId: definition.id, + // A delete removes a line from the file and one URL from the index's count, + // whenever the record had one - which is exactly "was it ever published". + ...(definition.delivery.enabled + ? { + delivery: { + sitemap: { + contentChanged: wasEverPublic, + indexChanged: wasEverPublic, + }, + }, + } + : {}), id, isPublic: false, - slugs: [publicStateOf(definition, result.data).slug], + slugs: [removed.slug], wasPublic: result.data?.publishedAt != null, }, // The row is gone. Serving its cached response one more time would be a @@ -660,6 +755,15 @@ const publicationAction = async ( revalidateContent( { contentTypeId: definition.id, + // A real transition flips reachability, so it moves both the file and the + // index that counts its URLs. + ...(definition.delivery.enabled + ? { + delivery: { + sitemap: { contentChanged: true, indexChanged: true }, + }, + } + : {}), id, isPublic, slugs: [slug], diff --git a/packages/vitnode/src/views/admin/views/content/actions/mutation-api.test.ts b/packages/vitnode/src/views/admin/views/content/actions/mutation-api.test.ts index a55801b2a..44da8218a 100644 --- a/packages/vitnode/src/views/admin/views/content/actions/mutation-api.test.ts +++ b/packages/vitnode/src/views/admin/views/content/actions/mutation-api.test.ts @@ -5,6 +5,7 @@ import type { AnyContentTypeDefinition } from "@/content/types"; import { testCategoryContentType, + testDeliveredPostContentType, testEditorialPostContentType, testPostContentType, } from "@/tests/content-fixtures"; @@ -79,6 +80,27 @@ const slugTag = (slug: string) => `content:test.post:slug:${slug}`; const editorialSlugTag = (slug: string) => `content:test.editorial:slug:${slug}`; +/** The delivery fixture's own tags. */ +const DELIVERED = "test.delivered-post"; +const DELIVERY_ITEM = `content:${DELIVERED}:delivery:7`; +const DELIVERY_SITEMAP = `content:${DELIVERED}:sitemap`; +const deliveryRedirectTag = (slug: string) => + `content:${DELIVERED}:redirect:${slug}`; + +/** + * One `updatedAt` per call, monotonically increasing. + * + * A real write moves the timestamp; a no-op does not. That distinction is the whole + * signal the sitemap decision reads, so the fixtures have to be explicit about it + * rather than reusing one constant everywhere. + */ +let clock = Date.parse("2026-01-01T00:00:00.000Z"); +const tick = (): string => { + clock += 60_000; + + return new Date(clock).toISOString(); +}; + const tags = () => cacheCalls.map(call => call.tag); /** Which Next cache API was used - `updateTag` is the immediate one. */ const mode = () => [...new Set(cacheCalls.map(call => call.fn))]; @@ -164,6 +186,203 @@ describe("edit", () => { }); }); +/** + * The sitemap half of delivery invalidation. + * + * A sitemap entry carries ``, derived from `updatedAt` - so a plain title + * edit on a published record changes the **bytes** of its sitemap file even though the + * set of URLs is identical. Treating "the sitemap changed" as "membership changed" + * leaves a cached file serving a stale timestamp, which is what these tests pin down. + */ +describe("delivery sitemap invalidation", () => { + const published = (slug: string, updatedAt: string) => ({ + data: { + id: 7, + publishedAt: past, + slug, + status: "published", + updatedAt, + }, + status: 200, + }); + + beforeEach(() => { + definition = testDeliveredPostContentType; + }); + + it("expires the sitemap for a title edit that moved no URL", async () => { + const before = tick(); + responses = [published("same", before), published("same", tick())]; + + await editContentAction(DELIVERED, 7, { title: "Hello" }); + + // The URL did not move, so nothing was added to or removed from the file - but + // `updatedAt` did, so its `` is different and the cached bytes are stale. + expect(tags()).toContain(DELIVERY_SITEMAP); + }); + + it("expires the sitemap for an SEO-only edit", async () => { + const before = tick(); + responses = [published("same", before), published("same", tick())]; + + await editContentAction(DELIVERED, 7, { excerpt: "A new summary." }); + + expect(tags()).toContain(DELIVERY_SITEMAP); + }); + + it("leaves the sitemap alone for a no-op edit", async () => { + // The engine issues no `UPDATE` for an update that changed nothing, so + // `updatedAt` does not move and the cached sitemap is still byte-correct. + const unchanged = tick(); + responses = [published("same", unchanged), published("same", unchanged)]; + + await editContentAction(DELIVERED, 7, { title: "Hello" }); + + expect(tags()).not.toContain(DELIVERY_SITEMAP); + // The rest of the delivery invalidation still happens: the metadata tag and the + // slug's redirect lookup are expired whether or not the sitemap moved. + expect(tags()).toContain(DELIVERY_ITEM); + expect(tags()).toContain(deliveryRedirectTag("same")); + }); + + it("leaves the sitemap alone for a draft edit", async () => { + const draft = (updatedAt: string) => ({ + data: { + id: 7, + publishedAt: null, + slug: "draft", + status: "draft", + updatedAt, + }, + status: 200, + }); + responses = [draft(tick()), draft(tick())]; + + await editContentAction(DELIVERED, 7, { title: "Hello" }); + + // Not public before or after, so it is in no sitemap file either way. + expect(cacheCalls).toEqual([]); + }); + + it("expires the sitemap on a slug change", async () => { + responses = [published("old", tick()), published("new", tick())]; + + await editContentAction(DELIVERED, 7, { title: "Hello" }); + + expect(tags()).toContain(DELIVERY_SITEMAP); + expect(tags()).toContain(deliveryRedirectTag("old")); + expect(tags()).toContain(deliveryRedirectTag("new")); + }); + + it("expires the sitemap on publish and on unpublish", async () => { + responses = [ + { + data: { + changed: true, + row: { + id: 7, + publishedAt: past, + slug: "hello", + status: "published", + updatedAt: tick(), + }, + }, + status: 200, + }, + ]; + + await publishContentAction(DELIVERED, 7); + + expect(tags()).toContain(DELIVERY_SITEMAP); + }); + + it("expires the sitemap on delete when the record had been published", async () => { + responses = [published("hello", tick())]; + + await deleteContentAction(DELIVERED, 7, 1); + + expect(tags()).toContain(DELIVERY_SITEMAP); + }); + + it("leaves the sitemap alone when deleting a record that was never published", async () => { + responses = [ + { + data: { + id: 7, + publishedAt: null, + slug: "draft", + status: "draft", + updatedAt: tick(), + }, + status: 200, + }, + ]; + + await deleteContentAction(DELIVERED, 7, 1); + + expect(tags()).not.toContain(DELIVERY_SITEMAP); + }); + + it("expires the sitemap for a restore that moved the slug", async () => { + responses = [ + published("current", tick()), + { + data: { + changed: true, + row: { + id: 7, + publishedAt: past, + slug: "restored", + status: "published", + updatedAt: tick(), + }, + }, + status: 200, + }, + ]; + + await restoreContentRevisionAction(DELIVERED, 7, 3, 4); + + expect(tags()).toContain(DELIVERY_SITEMAP); + expect(tags()).toContain(deliveryRedirectTag("current")); + expect(tags()).toContain(deliveryRedirectTag("restored")); + }); + + it("leaves the sitemap alone for a restore that changed nothing", async () => { + const unchanged = tick(); + responses = [ + published("current", unchanged), + { + data: { + changed: false, + row: { + id: 7, + publishedAt: past, + slug: "current", + status: "published", + updatedAt: unchanged, + }, + }, + status: 200, + }, + ]; + + await restoreContentRevisionAction(DELIVERED, 7, 3, 4); + + expect(tags()).not.toContain(DELIVERY_SITEMAP); + }); + + it("adds no delivery tags at all to a content type without delivery", async () => { + // The Stage 1-7 promise: the tag list of an existing content type does not move. + definition = testPostContentType; + responses = [published("same", tick()), published("same", tick())]; + + await editContentAction("test.post", 7, { title: "Hello" }); + + expect(tags()).toEqual([LIST, ITEM, slugTag("same")]); + }); +}); + describe("publish and unpublish", () => { it("expires the list, the item and the slug on publish", async () => { responses = [ diff --git a/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.test.ts b/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.test.ts new file mode 100644 index 000000000..a460ea38f --- /dev/null +++ b/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.test.ts @@ -0,0 +1,286 @@ +// @vitest-environment node +import { beforeEach, describe, expect, it, vi } from "vitest"; + +import type { ContentPublicLocaleState } from "@/content/cache"; + +import { + testDeliveredLocalizedContentType, + testLocalizedPageContentType, +} from "@/tests/content-fixtures"; + +const cacheTags: string[] = []; + +// The real `revalidate.server` runs, so what is asserted is the tag list +// `contentInvalidationTags` actually produces - mocking the layer in between would +// test the mock. +vi.mock("server-only", () => ({})); + +vi.mock("next/cache", () => ({ + revalidatePath: () => undefined, + revalidateTag: (tag: string) => { + cacheTags.push(tag); + }, + updateTag: (tag: string) => { + cacheTags.push(tag); + }, +})); + +vi.mock("@/content/admin/fetch.server", () => ({ + contentApiFetch: async () => await Promise.resolve({ status: 500 }), +})); + +const { invalidateContentLocales } = await import("./public-locale-cache"); + +/** + * The localized half of delivery sitemap invalidation. + * + * A localized sitemap entry's `lastModified` is `max(base.updatedAt, + * translation.updatedAt)`, so a real edit to a **published translation** changes that + * locale's sitemap file even when its URL does not move - and a shared field edit + * changes every published locale's file, because the base timestamp is in all of them. + * + * The distinction these tests pin down is which locale, and whether the *index* moved: + * a title edit rewrites bytes inside one existing file and changes neither which files + * exist nor how many. + */ +const DELIVERED = "test.delivered-localized"; +const sitemapTag = (locale?: string) => + locale === undefined + ? `content:${DELIVERED}:sitemap` + : `content:${DELIVERED}:sitemap:${locale}`; + +/** A locale with its own published translation. */ +const own = (locale: string, slug: string): ContentPublicLocaleState => ({ + hasOwnTranslation: true, + isPublic: true, + locale, + slug, +}); + +/** A locale served the default translation, with none of its own. */ +const fallbackOnly = ( + locale: string, + slug: string, +): ContentPublicLocaleState => ({ + hasOwnTranslation: false, + isPublic: true, + locale, + slug, +}); + +beforeEach(() => { + cacheTags.length = 0; +}); + +describe("a translation update", () => { + it("expires only that locale's sitemap file", () => { + const states = [own("en", "hello"), own("pl", "witaj")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "translation", locale: "pl" }, + ); + + expect(cacheTags).toContain(sitemapTag("pl")); + // English did not move, so its file is still byte-correct. + expect(cacheTags).not.toContain(sitemapTag("en")); + }); + + it("leaves the sitemap index alone", () => { + // A title edit rewrites a `` inside one file. It does not change which + // files exist, so the index that enumerates them is untouched. + const states = [own("en", "hello"), own("pl", "witaj")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "translation", locale: "pl" }, + ); + + expect(cacheTags).not.toContain(sitemapTag()); + }); +}); + +describe("a shared update", () => { + it("expires every published locale's sitemap file", () => { + // The base row's `updatedAt` is part of `max(base, translation)` for every + // language, so a shared edit changes what each of their files serializes. + const states = [own("en", "hello"), own("pl", "witaj")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "shared" }, + ); + + expect(cacheTags).toContain(sitemapTag("en")); + expect(cacheTags).toContain(sitemapTag("pl")); + }); + + it("still leaves the index alone", () => { + const states = [own("en", "hello"), own("pl", "witaj")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "shared" }, + ); + + expect(cacheTags).not.toContain(sitemapTag()); + }); + + it("skips a locale that is not public", () => { + const states = [ + own("en", "hello"), + { hasOwnTranslation: true, isPublic: false, locale: "pl", slug: "witaj" }, + ]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "shared" }, + ); + + expect(cacheTags).toContain(sitemapTag("en")); + // A draft translation is in no sitemap, so nothing about it went stale. + expect(cacheTags).not.toContain(sitemapTag("pl")); + }); +}); + +describe("a default-locale update with a fallback consumer", () => { + it("follows the Stage 5 fan-out", () => { + // `fallback: "default"` makes Polish's *public page* the English translation, so + // Stage 5 reaches Polish - and this reuses that fan-out rather than inventing a + // second locale-propagation rule. + // + // Polish contributes **no sitemap URL**, because a sitemap never lists a fallback + // (the delivery Postgres suite asserts that directly), so expiring its file is + // conservative rather than necessary: a cache miss, never a stale document. + const states = [own("en", "hello"), fallbackOnly("pl", "hello")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "translation", locale: "en" }, + ); + + expect(cacheTags).toContain(sitemapTag("en")); + expect(cacheTags).toContain(sitemapTag("pl")); + }); + + it("does not reach a locale with its own translation", () => { + // Nothing falls back to a language that has its own copy, so a default-locale + // edit leaves it entirely alone. + const states = [own("en", "hello"), own("pl", "witaj")]; + + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + states, + states, + { changed: "translation", locale: "en" }, + ); + + expect(cacheTags).toContain(sitemapTag("en")); + expect(cacheTags).not.toContain(sitemapTag("pl")); + }); +}); + +describe("membership changes", () => { + it("expires the file and the index when a translation is published", () => { + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + [ + own("en", "hello"), + { + hasOwnTranslation: true, + isPublic: false, + locale: "pl", + slug: "witaj", + }, + ], + [own("en", "hello"), own("pl", "witaj")], + { changed: "translation", locale: "pl" }, + ); + + expect(cacheTags).toContain(sitemapTag("pl")); + // A language gained a URL, so how many the index counts moved. + expect(cacheTags).toContain(sitemapTag()); + }); + + it("expires the file and the index when a translation is deleted", () => { + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + [own("en", "hello"), own("pl", "witaj")], + // Absent from the "after" side entirely: the translation is gone. + [own("en", "hello")], + { changed: "translation", locale: "pl" }, + ); + + expect(cacheTags).toContain(sitemapTag("pl")); + expect(cacheTags).toContain(sitemapTag()); + }); + + it("expires every locale's file and the index when the record is unpublished", () => { + invalidateContentLocales( + testDeliveredLocalizedContentType, + 7, + [own("en", "hello"), own("pl", "witaj")], + [ + { + hasOwnTranslation: true, + isPublic: false, + locale: "en", + slug: "hello", + }, + { + hasOwnTranslation: true, + isPublic: false, + locale: "pl", + slug: "witaj", + }, + ], + { changed: "shared" }, + ); + + expect(cacheTags).toContain(sitemapTag("en")); + expect(cacheTags).toContain(sitemapTag("pl")); + expect(cacheTags).toContain(sitemapTag()); + }); +}); + +describe("a localized content type without delivery", () => { + it("produces exactly the Stage 1-7 tag list", () => { + const states = [ + { hasOwnTranslation: true, isPublic: true, locale: "en", slug: "hello" }, + { hasOwnTranslation: true, isPublic: true, locale: "pl", slug: "witaj" }, + ]; + + invalidateContentLocales(testLocalizedPageContentType, 7, states, states, { + changed: "translation", + locale: "pl", + }); + + const id = testLocalizedPageContentType.id; + expect(cacheTags).toStrictEqual([ + `content:${id}:list:pl`, + `content:${id}:item:pl:7`, + `content:${id}:slug:pl:witaj`, + ]); + }); +}); diff --git a/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.ts b/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.ts index f824b4ce3..b2cb16a39 100644 --- a/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.ts +++ b/packages/vitnode/src/views/admin/views/content/actions/public-locale-cache.ts @@ -87,17 +87,46 @@ export const invalidateContentLocales = ( ): void => { if (!definition.publicApi.enabled || !definition.localization.enabled) return; + const states = diffContentPublicLocaleStates(before, after); const reached = contentLocaleInvalidations({ changed, defaultLocale: definition.localization.defaultLocale, fallback: definition.localization.fallback, locale, - states: diffContentPublicLocaleStates(before, after), + states, }); revalidateContent( { contentTypeId: definition.id, + // The delivery tags, for a content type with `delivery`. Absent otherwise, + // which is what keeps a Stage 1-7 content type's tag list byte-identical. + // + // Derived from the locales this mutation actually **reached**, which is the + // Stage 5 fan-out rather than a second locale-propagation rule: a shared field + // reaches every locale, a translation reaches its own, and `sitemap:pl` is + // expired exactly when Polish's public representation moved. That is also what + // makes a plain title edit expire the right file - the sitemap's `` is + // derived from `updatedAt`, so any real edit to a published translation changes + // that file's bytes even though its URL did not move. + ...(definition.delivery.enabled + ? { + delivery: { + sitemap: { + // Every reached locale that is or was public has a file whose bytes + // moved. This helper is only called for a real mutation. + contentChanged: reached.some( + entry => entry.isPublic || entry.wasPublic, + ), + // Only a locale appearing or disappearing changes how many files the + // index lists. + indexChanged: reached.some( + entry => entry.isPublic !== entry.wasPublic, + ), + }, + }, + } + : {}), id, // Not consulted when `locales` is present, and supplied truthfully anyway: // a record is publicly reachable when any of its languages is. diff --git a/packages/vitnode/src/views/admin/views/content/actions/translation-api.server.ts b/packages/vitnode/src/views/admin/views/content/actions/translation-api.server.ts index fb2b46f56..bc0e88fc5 100644 --- a/packages/vitnode/src/views/admin/views/content/actions/translation-api.server.ts +++ b/packages/vitnode/src/views/admin/views/content/actions/translation-api.server.ts @@ -4,6 +4,7 @@ import { revalidatePath } from "next/cache"; import { z } from "zod"; import type { + ContentDeliveryConflict, ContentTranslationConflict, ContentUnprocessable, } from "@/content/conflicts"; @@ -16,6 +17,7 @@ import type { import { findFrontendContentType } from "@/content/admin/config"; import { contentApiFetch } from "@/content/admin/fetch.server"; import { + parseContentDeliveryConflict, parseContentTranslationConflict, parseContentUnprocessable, } from "@/content/conflicts"; @@ -44,6 +46,16 @@ const CONTENT_PAGE_PATH = */ export interface TranslationMutationResult { conflict?: ContentTranslationConflict; + /** + * `CONTENT_DELIVERY_SLUG_RESERVED`, when a localized address is owned by another + * record's URL history. + * + * Its own field rather than a sixth arm of `conflict`, because it is a fact about + * *delivery* rather than about translations - the base routes answer with the same + * shape, and one code for one condition is what lets the AdminCP say the same + * sentence wherever the address was typed. + */ + delivery?: ContentDeliveryConflict; error?: string; status?: number; unprocessable?: ContentUnprocessable; @@ -54,6 +66,7 @@ const failure = (result: { status: number; }): TranslationMutationResult => ({ conflict: parseContentTranslationConflict(result.error) ?? undefined, + delivery: parseContentDeliveryConflict(result.error) ?? undefined, error: result.error ?? "", status: result.status, unprocessable: parseContentUnprocessable(result.error) ?? undefined, diff --git a/packages/vitnode/src/views/admin/views/content/actions/translations/translation-panel.tsx b/packages/vitnode/src/views/admin/views/content/actions/translations/translation-panel.tsx index 5646c0dab..d47fb00c5 100644 --- a/packages/vitnode/src/views/admin/views/content/actions/translations/translation-panel.tsx +++ b/packages/vitnode/src/views/admin/views/content/actions/translations/translation-panel.tsx @@ -184,7 +184,12 @@ export const TranslationPanel = ({ const report = (result: TranslationMutationResult): boolean => { if (result.error === undefined) return true; - const key = conflictMessage(result.conflict); + // The delivery reservation first: it shares a status with the unique clash and + // says a different thing - "that address still redirects to another record" + // rather than "another record holds it now". + const key = result.delivery + ? "slug_reserved" + : conflictMessage(result.conflict); if (result.conflict?.code === "CONTENT_TRANSLATION_VERSION_CONFLICT") { // The form keeps every value the translator typed. Nothing is retried and diff --git a/packages/vitnode/src/views/admin/views/content/lib/mutation-feedback.ts b/packages/vitnode/src/views/admin/views/content/lib/mutation-feedback.ts index c019f1bca..647dc2f6f 100644 --- a/packages/vitnode/src/views/admin/views/content/lib/mutation-feedback.ts +++ b/packages/vitnode/src/views/admin/views/content/lib/mutation-feedback.ts @@ -1,5 +1,6 @@ import type { ContentConflict, + ContentDeliveryConflict, ContentUnprocessable, } from "@/content/conflicts"; @@ -9,6 +10,7 @@ export type ContentErrorKey = | "forbidden" | "not_found" | "not_restorable" + | "slug_reserved" | "unique_conflict" | "validation" | "version_conflict"; @@ -31,9 +33,16 @@ export const contentErrorKey = ( status: number | undefined, structured?: { conflict?: ContentConflict; + delivery?: ContentDeliveryConflict; unprocessable?: ContentUnprocessable; }, ): ContentErrorKey | null => { + // Before the plain conflict, because the two share a status and mean different + // things: a unique clash is "another record holds that address now, so you cannot + // have it", and a reservation is "another record *used* to hold it and it still + // redirects" - which is a different sentence and possibly a different decision. + if (structured?.delivery) return "slug_reserved"; + if (structured?.conflict) { return structured.conflict.code === "CONTENT_VERSION_CONFLICT" ? "version_conflict" diff --git a/packages/vitnode/src/views/admin/views/content/table/content-table-view.tsx b/packages/vitnode/src/views/admin/views/content/table/content-table-view.tsx index 216d64770..4adc5f130 100644 --- a/packages/vitnode/src/views/admin/views/content/table/content-table-view.tsx +++ b/packages/vitnode/src/views/admin/views/content/table/content-table-view.tsx @@ -13,6 +13,7 @@ import { orderableColumns } from "@/content/registry"; import type { ContentRowData } from "./cells"; import { DeleteContentAction } from "../actions/delete-action"; +import { DeliveryContentAction } from "../actions/delivery-action"; import { EditContentAction } from "../actions/edit-action"; import { HistoryContentAction } from "../actions/history-action"; import { PreviewContentAction } from "../actions/preview-action"; @@ -170,6 +171,7 @@ export const ContentTableView = async ({ definition.editorial.enabled ? "w-36" : "", definition.editorial.preview.enabled ? "w-44" : "", definition.editorial.scheduling.enabled ? "w-52" : "", + definition.delivery.enabled ? "w-60" : "", ] .filter(Boolean) .at(-1), @@ -181,6 +183,16 @@ export const ContentTableView = async ({ return ( <> + {definition.delivery.enabled ? ( + + ) : null} {definition.editorial.preview.enabled ? ( { + query: async ({ cursorSelection, limit, where, orderBy }) => { // The title lives in `core_languages_words`, so search resolves matching // category ids from there rather than a column on `blog_categories`. const searchCondition = query.search @@ -99,7 +106,7 @@ export const categoriesRoute = buildRoute({ return await c .get("db") - .select() + .select({ ...getTableColumns(blog_categories), ...cursorSelection }) .from(blog_categories) .where(combinedWhere) .orderBy(orderBy) diff --git a/plugins/blog/src/api/modules/posts/routes/get.route.ts b/plugins/blog/src/api/modules/posts/routes/get.route.ts index 0d98310a5..a50564630 100644 --- a/plugins/blog/src/api/modules/posts/routes/get.route.ts +++ b/plugins/blog/src/api/modules/posts/routes/get.route.ts @@ -77,10 +77,11 @@ export const postsRoute = buildRoute({ query, }, primaryCursor: blog_posts.id, - query: async ({ limit, where, orderBy }) => + query: async ({ cursorSelection, limit, where, orderBy }) => await c .get("db") .select({ + ...cursorSelection, id: blog_posts.id, categoryId: blog_posts.categoryId, createdAt: blog_posts.createdAt, diff --git a/plugins/example/src/const.ts b/plugins/example/src/const.ts index 7e7ff31b6..122c10184 100644 --- a/plugins/example/src/const.ts +++ b/plugins/example/src/const.ts @@ -35,4 +35,12 @@ export const EXAMPLE_MIGRATIONS = [ // table - each with the constraints that make its ordering and its integrity // facts about the database rather than about the service. "0031_add_example_advanced_articles.sql", + // Stage 8. Core again: `core_content_slug_history` is what makes an old public + // URL keep working, and the delivery suites write reservations for both example + // content types - so the table has to exist before either of them publishes. + "0032_add_content_slug_history.sql", + // The shared boolean `delivery.seo.noIndexField` reads, which drives the sitemap + // exclusion and the `robots` metadata together. Additive and defaulted, so every + // existing row becomes indexable rather than silently disappearing from a sitemap. + "0033_add_example_article_no_index.sql", ]; diff --git a/plugins/example/src/content/advanced-article.ts b/plugins/example/src/content/advanced-article.ts index b3f02af42..987fadbb2 100644 --- a/plugins/example/src/content/advanced-article.ts +++ b/plugins/example/src/content/advanced-article.ts @@ -98,6 +98,15 @@ export const advancedArticleContentType = defineContentType({ syndication: field.group({ fields: { indexable: field.boolean({ defaultValue: true }), + /** + * The Stage 8 `noIndexField`, and shared rather than localized on purpose. + * + * Sitemap exclusion and the `robots` metadata are driven by the same + * boolean, so they cannot disagree - and a per-locale value would give one + * record one answer per language while it has a single canonical decision. + * `delivery` refuses a localized field here for exactly that reason. + */ + noIndex: field.boolean({ defaultValue: false }), priority: field.number({ integer: true, min: 0, @@ -136,12 +145,20 @@ export const advancedArticleContentType = defineContentType({ enabled: true, path: "advanced-articles", fields: [ + // Exposed because Stage 8 needs it: alternates and `hreflang` are resolved by + // identifier, and delivery reads the public projection - so a localized + // delivery content type that withheld `id` would carry an empty alternate set. + "id", "title", "slug", "categories", "seo.title", "seo.description", "syndication.priority", + // Public because delivery projects it: `robots: { index: false }` is rendered + // into the page, so the field it comes from has to be something the public API + // would already have said out loud. + "syndication.noIndex", "faq.question", "faq.answer", "publishedAt", @@ -166,6 +183,39 @@ export const advancedArticleContentType = defineContentType({ pathTemplate: "/{locale}/advanced-articles/{slug}", }, + /** + * The Stage 8 reference for a **localized** content type. + * + * Its canonical path carries the locale - `/pl/advanced-articles/moj-artykul` - + * and so does its slug history: the slug is `localized: true`, so each language + * gets its own reservation and changing the English URL creates no Polish + * redirect. + * + * `seo` reads the localized group, so every language has its own title and + * description, with `fallbackTitleField: "title"` filling in when `seo.title` is + * empty - which it usually is, because nobody writes one twice. + * + * `hreflang.xDefault` points at the default locale's canonical path, and only when + * that language is genuinely published: an `x-default` pointing at a translation + * this record does not have would be a hint to crawl a 404. + */ + delivery: { + enabled: true, + redirects: { enabled: true }, + hreflang: { xDefault: "defaultLocale" }, + seo: { + titleField: "seo.title", + fallbackTitleField: "title", + descriptionField: "seo.description", + noIndexField: "syndication.noIndex", + openGraph: { + titleField: "seo.title", + descriptionField: "seo.description", + }, + }, + sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 }, + }, + // Leaf paths, materialised against the generated columns: this compiles to an // index on `syndicationPriority`, exactly as `{ on: ["priority"] }` would have // if `priority` were a top-level field. diff --git a/plugins/example/src/content/article.ts b/plugins/example/src/content/article.ts index ab6c9574d..ae2fa389c 100644 --- a/plugins/example/src/content/article.ts +++ b/plugins/example/src/content/article.ts @@ -45,6 +45,36 @@ export const articleContentType = defineContentType({ pathTemplate: "/articles/{slug}", }, + /** + * The Stage 8 reference for a **nonlocalized** content type. + * + * Its canonical path has no locale segment - `/articles/my-article` - and its slug + * history has no language either: `languageId` is `NULL`, so one reservation + * covers the one URL the record has. + * + * `redirects` is what makes an old address keep working. Change the slug of a + * *published* article and `/articles/old-slug` answers 308 to the new one, for as + * long as the article stays published; change it while it is still a draft and + * nothing is recorded, because the URL was never live. + * + * `seo` projects two fields the public API already exposes. There is no + * `fallbackTitleField` here because `title` is the primary and it is + * `required: true` - a fallback would never be reached. + */ + delivery: { + enabled: true, + redirects: { enabled: true }, + seo: { + titleField: "title", + descriptionField: "excerpt", + // Same fields in both slots, which is the common case: an author who wants a + // different social title names a different field, and one who does not says + // so in two lines rather than four. + openGraph: { titleField: "title", descriptionField: "excerpt" }, + }, + sitemap: { enabled: true, changeFrequency: "weekly", priority: 0.7 }, + }, + editorial: { enabled: true, revisions: { retention: 20 }, diff --git a/plugins/example/src/database/advanced-postgres.test.ts b/plugins/example/src/database/advanced-postgres.test.ts index 632cacc54..e81cd796c 100644 --- a/plugins/example/src/database/advanced-postgres.test.ts +++ b/plugins/example/src/database/advanced-postgres.test.ts @@ -663,6 +663,9 @@ describe.skipIf(!url)("Stage 6 advanced modeling against Postgres", () => { expect(row?.syndication).toStrictEqual({ indexable: false, + // Stage 8 added a third leaf to the group. It is untouched by a write that + // named only `priority`, which is exactly what "partial group update" means. + noIndex: false, priority: 3, }); }); @@ -1406,6 +1409,7 @@ describe.skipIf(!url)("Stage 6 advanced modeling against Postgres", () => { // Nested, never the flattened column names. expect(snapshot.fields.syndication).toStrictEqual({ indexable: true, + noIndex: false, priority: 5, }); }); @@ -1525,6 +1529,7 @@ describe.skipIf(!url)("Stage 6 advanced modeling against Postgres", () => { expect(row?.syndication).toStrictEqual({ indexable: true, + noIndex: false, priority: 7, }); }); diff --git a/plugins/example/src/database/advanced-routes.test.ts b/plugins/example/src/database/advanced-routes.test.ts index 9c0782d4b..629369630 100644 --- a/plugins/example/src/database/advanced-routes.test.ts +++ b/plugins/example/src/database/advanced-routes.test.ts @@ -99,6 +99,9 @@ describe("advanced article: generated routes", () => { expect(Object.keys(shape).sort()).toStrictEqual([ "categories", "faq", + // Stage 8: a localized delivery content type has to expose `id`, because + // alternates and `hreflang` are resolved by identifier. + "id", "locale", "publishedAt", "seo", @@ -107,13 +110,16 @@ describe("advanced article: generated routes", () => { "title", ]); // A private collection is absent from the contract as well as from the - // response - and `syndication` carries only the leaf that was exposed. + // response - and `syndication` carries only the leaves that were exposed. + // `indexable` is still absent, which is the whole point of leaf-level + // allowlisting: `noIndex` joined it in Stage 8 because delivery projects the + // value into a public `robots` directive, and `indexable` did not. expect(shape.relatedArticles).toBeUndefined(); expect( Object.keys( (shape.syndication as unknown as { shape: Record }) .shape, - ), - ).toStrictEqual(["priority"]); + ).sort(), + ).toStrictEqual(["noIndex", "priority"]); }); }); diff --git a/plugins/example/src/database/concurrency-postgres.test.ts b/plugins/example/src/database/concurrency-postgres.test.ts new file mode 100644 index 000000000..9a78457c7 --- /dev/null +++ b/plugins/example/src/database/concurrency-postgres.test.ts @@ -0,0 +1,1185 @@ +import type { Context } from "hono"; + +import { executeContentSchedule } from "@vitnode/core/api/modules/content/helpers/execute-content-schedule"; +import { + ContentTranslationVersionConflict, + ContentVersionConflict, +} from "@vitnode/core/content"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { CONFIG_PLUGIN } from "@/const"; + +import type { ContentTestHarness } from "./harness"; + +import { advancedArticleContent } from "./advanced-articles"; +import { articleContent } from "./articles"; +import { + ACTOR, + clearContentTables, + createContentTestHarness, + DATABASE_TEST_URL, + fulfilledCount, + race, + reasons, +} from "./harness"; +import { localizedArticleContent } from "./localized-articles"; + +/** + * The Stage 7 concurrency matrix, against real Postgres. + * + * Every test here runs two writers on two separate connections at the same + * moment. That is the only way any of it can be shown: a mock cannot produce a + * lock wait, a guarded `UPDATE` that matches nothing, or a `DELETE` that commits + * between another transaction's read and its write. + * + * The invariants, stated once so each test can be read against them: + * + * - **exactly one winner** wherever both writers carry the same + * `expectedVersion`, and the loser is told which version it lost to; + * - **no resurrection** - a record deleted by one writer is never brought back + * by another's write, and neither is a translation; + * - **no partial state** - a losing writer leaves the collections exactly as it + * found them, because the version guard runs before a single junction or child + * row is touched; + * - **monotonic versions** - a race produces one increment, not two, and never + * two revisions at the same version. + */ + +let h: ContentTestHarness; + +const editorial = (on: Context) => { + const build = articleContent.editorialService; + if (!build) throw new Error("example.article has no editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const advanced = (on: Context) => { + const build = advancedArticleContent.editorialService; + if (!build) throw new Error("no advanced editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const plainAdvanced = (on: Context) => advancedArticleContent.service(on); + +const localizedService = (on: Context) => { + const build = localizedArticleContent.localizedService; + if (!build) throw new Error("no localized service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const translationEditorial = (on: Context) => { + const build = localizedArticleContent.translationEditorialService; + if (!build) throw new Error("no translation editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +let categoryId = 0; +let seq = 0; + +/** A published-ready article, at version 1 with one `create` revision. */ +const article = async (overrides: Record = {}) => { + seq += 1; + const outcome = await editorial(h.context).create( + { + category: categoryId, + code: `race-${seq}`, + title: `Race subject ${seq}`, + ...overrides, + }, + { actor: ACTOR }, + ); + + return { id: outcome.row.id, version: outcome.version }; +}; + +const rowOf = async (id: number) => { + const [row] = await h.sql< + { status: string; title: string; version: number }[] + >` + SELECT "title", "status", "version" FROM "example_articles" WHERE "id" = ${id} + `; + + return row; +}; + +const revisionsOf = async (id: number) => + await h.sql<{ operation: string; version: number }[]>` + SELECT "operation", "version" FROM "core_content_revisions" + WHERE "contentTypeId" = 'example.article' AND "itemId" = ${id} + ORDER BY "version" + `; + +const isVersionConflict = (error: unknown): boolean => + error instanceof ContentVersionConflict; + +/** + * How many of a race's sides actually **changed** something. + * + * Not the same as how many succeeded: a collection mutation whose computed next + * state equals the stored one is a successful no-op, and a no-op deliberately + * does not check `expectedVersion` - there is nothing to overwrite, so there is + * nothing to conflict about. Counting real mutations is what pins "one race, + * one version increment". + */ +const changedCount = ( + results: readonly PromiseSettledResult[], +): number => + results.filter( + entry => + entry.status === "fulfilled" && + (entry.value as null | { changed?: boolean })?.changed === true, + ).length; + +describe.skipIf(!DATABASE_TEST_URL)("Content Engine concurrency", () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + }); + + beforeEach(async () => { + await clearContentTables(h.sql); + h.reset(); + + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Races') RETURNING "id" + `; + categoryId = category.id; + }); + + // ------------------------------------------------------------------------- + // Base record races + // ------------------------------------------------------------------------- + + describe("update against update", () => { + it("lets exactly one writer win and tells the other which version it lost to", async () => { + const { id } = await article(); + + const results = await race( + async () => + await editorial(h.context).update( + id, + { title: "Writer A" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await editorial(h.rivalContext).update( + id, + { title: "Writer B" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + expect(reasons(results).every(isVersionConflict)).toBe(true); + expect( + (reasons(results)[0] as ContentVersionConflict).currentVersion, + ).toBe(2); + + const row = await rowOf(id); + expect(row.version).toBe(2); + expect(["Writer A", "Writer B"]).toContain(row.title); + + // One increment and one revision, not two of either. The loser wrote + // nothing at all, so there is no partial mutation to find. + expect((await revisionsOf(id)).map(entry => entry.operation)).toEqual([ + "create", + "update", + ]); + }); + }); + + /** + * Two writers, one of which removes the record. + * + * The order decides which of two shapes the loser sees, and both are stated + * rather than accepted as "either": + * + * - the **update** commits first, so the delete's guarded `DELETE` matches + * nothing and the follow-up read finds version 2: a conflict; + * - the **delete** commits first, so the update's read finds no row at all: + * `null`, which the route turns into a 404. + * + * What never happens is a resurrection - the update's `UPDATE` is guarded by + * both the id and the version, so it cannot recreate a row - and a revision + * for a state that never existed. + */ + describe("update against delete", () => { + it("either refuses the delete or answers the update with nothing", async () => { + const { id } = await article(); + + const results = await race( + async () => + await editorial(h.context).update( + id, + { title: "Edited" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await editorial(h.rivalContext).delete(id, { + actor: ACTOR, + expectedVersion: 1, + }), + ); + + const [updateResult, deleteResult] = results; + + if (deleteResult.status === "fulfilled" && deleteResult.value) { + // The delete won. The update either lost the guard (a conflict) or + // found nothing (`null`) - never a row it went on to rewrite. + expect(await rowOf(id)).toBeUndefined(); + if (updateResult.status === "fulfilled") { + expect(updateResult.value).toBeNull(); + } else { + expect(isVersionConflict(updateResult.reason)).toBe(true); + } + + return; + } + + // The update won, so the delete was refused on the version rather than + // silently removing a record somebody had just edited. + expect(updateResult.status).toBe("fulfilled"); + expect(deleteResult.status).toBe("rejected"); + expect( + isVersionConflict( + deleteResult.status === "rejected" ? deleteResult.reason : null, + ), + ).toBe(true); + expect((await rowOf(id)).title).toBe("Edited"); + }); + + it("never leaves a revision describing a record that was never in that state", async () => { + const { id } = await article(); + + await race( + async () => + await editorial(h.context).update( + id, + { title: "Edited" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await editorial(h.rivalContext).delete(id, { + actor: ACTOR, + expectedVersion: 1, + }), + ); + + const history = await revisionsOf(id); + // Strictly increasing, with no version written twice - which is what the + // partial unique index enforces and what a lost race must not disturb. + expect(history.map(entry => entry.version)).toEqual( + [...history.map(entry => entry.version)].sort((a, b) => a - b), + ); + expect(new Set(history.map(entry => entry.version)).size).toBe( + history.length, + ); + }); + }); + + /** + * A field edit against a publication. + * + * Publishing takes an **optional** `expectedVersion`, because it overwrites no + * field value: requiring one would fail the publish button whenever a + * colleague had fixed a typo, for no protection against a lost update. Both + * halves of that decision are pinned here. + */ + describe("update against publish", () => { + it("lets exactly one win when both carry the same expected version", async () => { + const { id } = await article(); + + const results = await race( + async () => + await editorial(h.context).update( + id, + { title: "Edited first" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await editorial(h.rivalContext).publish(id, { + actor: ACTOR, + expectedVersion: 1, + }), + ); + + expect(fulfilledCount(results)).toBe(1); + expect(reasons(results).every(isVersionConflict)).toBe(true); + expect((await rowOf(id)).version).toBe(2); + }); + + it("overwrites no field value when the publish carries no version", async () => { + // The documented behaviour: an unguarded publish moves `status` and + // nothing else, so a concurrent edit either lands before it or is + // refused - but the title it wrote is never reverted by the publish. + const { id } = await article(); + + const results = await race( + async () => + await editorial(h.context).update( + id, + { title: "Edited alongside" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await editorial(h.rivalContext).publish(id, { actor: ACTOR }), + ); + + const row = await rowOf(id); + expect(row.status).toBe("published"); + + const [updateResult] = results; + if (updateResult.status === "fulfilled" && updateResult.value?.changed) { + expect(row.title).toBe("Edited alongside"); + expect(row.version).toBe(3); + + return; + } + + // Refused, and the title it never wrote is not on the row. + expect(row.title).not.toBe("Edited alongside"); + expect(row.version).toBe(2); + }); + }); + + /** + * A restore is the widest overwrite the engine has - it rewrites many fields + * at once from a source the editor did not type - so it carries the same + * required `expectedVersion` an ordinary update does. + */ + describe("restore against update", () => { + it("never overwrites a newer edit silently", async () => { + const { id } = await article({ title: "Original" }); + await editorial(h.context).update( + id, + { title: "Second" }, + { actor: ACTOR, expectedVersion: 1 }, + ); + + const history = await editorial(h.context).revisions.list(id); + const first = history.edges.find(entry => entry.operation === "create"); + if (!first) throw new Error("Expected a create revision."); + + const results = await race( + async () => + await editorial(h.context).restore(id, first.id, { + actor: ACTOR, + expectedVersion: 2, + }), + async () => + await editorial(h.rivalContext).update( + id, + { title: "Third" }, + { actor: ACTOR, expectedVersion: 2 }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + expect(reasons(results).every(isVersionConflict)).toBe(true); + + const row = await rowOf(id); + expect(row.version).toBe(3); + // Whichever won, the record holds exactly that writer's value - never a + // mixture, and never the loser's. + expect(["Original", "Third"]).toContain(row.title); + }); + + it("writes exactly one new revision, never rewriting the one it restored from", async () => { + const { id } = await article({ title: "Original" }); + await editorial(h.context).update( + id, + { title: "Second" }, + { actor: ACTOR, expectedVersion: 1 }, + ); + + const history = await editorial(h.context).revisions.list(id); + const first = history.edges.find(entry => entry.operation === "create"); + if (!first) throw new Error("Expected a create revision."); + const before = await editorial(h.context).revisions.findById( + id, + first.id, + ); + + await race( + async () => + await editorial(h.context).restore(id, first.id, { + actor: ACTOR, + expectedVersion: 2, + }), + async () => + await editorial(h.rivalContext).update( + id, + { title: "Third" }, + { actor: ACTOR, expectedVersion: 2 }, + ), + ); + + const after = await editorial(h.context).revisions.findById(id, first.id); + expect(after?.snapshot).toEqual(before?.snapshot); + expect(after?.version).toBe(before?.version); + expect((await revisionsOf(id)).length).toBe(3); + }); + }); + + describe("restore against delete", () => { + it("never recreates a record a concurrent delete removed", async () => { + const { id } = await article({ title: "Original" }); + await editorial(h.context).update( + id, + { title: "Second" }, + { actor: ACTOR, expectedVersion: 1 }, + ); + const history = await editorial(h.context).revisions.list(id); + const first = history.edges.find(entry => entry.operation === "create"); + if (!first) throw new Error("Expected a create revision."); + + const results = await race( + async () => + await editorial(h.context).restore(id, first.id, { + actor: ACTOR, + expectedVersion: 2, + }), + async () => + await editorial(h.rivalContext).delete(id, { + actor: ACTOR, + expectedVersion: 2, + }), + ); + + const [restoreResult, deleteResult] = results; + + if (deleteResult.status === "fulfilled" && deleteResult.value) { + // Gone, and it stays gone: a restore reads the live row before it + // writes, so there is no row for it to resurrect. + expect(await rowOf(id)).toBeUndefined(); + if (restoreResult.status === "fulfilled") { + expect(restoreResult.value).toBeNull(); + } else { + expect(isVersionConflict(restoreResult.reason)).toBe(true); + } + + return; + } + + expect(restoreResult.status).toBe("fulfilled"); + expect((await rowOf(id)).title).toBe("Original"); + }); + }); + + // ------------------------------------------------------------------------- + // Scheduled against manual + // ------------------------------------------------------------------------- + + describe("a scheduled transition against a manual one", () => { + const schedules = (on: Context) => { + const model = editorial(on).schedules; + if (!model) throw new Error("example.article has no scheduling"); + + return model; + }; + + const book = async (id: number, action: "publish" | "unpublish") => + await schedules(h.context).schedule({ + action, + actorUserId: null, + itemId: id, + // Inside the past tolerance, so it is due on this tick. + scheduledFor: new Date(Date.now() - 1000), + }); + + it("does nothing when the manual publish got there first", async () => { + const { id } = await article(); + const booked = await book(id, "publish"); + + await editorial(h.context).publish(id, { actor: ACTOR }); + const outcome = await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }); + + // Skipped rather than executed: the state guard on the transition is what + // makes a scheduled publish idempotent, and idempotent is what stops a + // second revision and a second announcement. + expect(outcome).toMatchObject({ + reason: "already in that state", + status: "skipped", + }); + expect( + (await revisionsOf(id)).filter(entry => entry.operation === "publish"), + ).toHaveLength(1); + + const effects = await h.sql` + SELECT "id" FROM "core_queue" WHERE "name" = 'content-schedule-effects' + `; + expect(effects).toHaveLength(0); + }); + + it("does not un-publish a record the editor published after booking the unpublish", async () => { + const { id } = await article(); + await editorial(h.context).publish(id, { actor: ACTOR }); + const booked = await book(id, "unpublish"); + + // Manual unpublish first; the stale booking then finds nothing to do. + await editorial(h.context).unpublish(id, { actor: ACTOR }); + const outcome = await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }); + + expect(outcome.status).toBe("skipped"); + expect( + (await revisionsOf(id)).filter( + entry => entry.operation === "unpublish", + ), + ).toHaveLength(1); + }); + + it("settles the booking either way, so it never runs twice", async () => { + const { id } = await article(); + const booked = await book(id, "publish"); + await editorial(h.context).publish(id, { actor: ACTOR }); + + await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }); + + const [schedule] = await h.sql<{ status: string }[]>` + SELECT "status" FROM "core_content_schedules" WHERE "id" = ${booked.id} + `; + expect(schedule.status).toBe("completed"); + + // A second delivery of the same queue row is a no-op: the claim refuses + // anything that is not still `pending`. + const again = await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }); + expect(again.status).toBe("skipped"); + }); + + it("keeps a scheduled publish and a concurrent edit to one version each", async () => { + const { id } = await article(); + const booked = await book(id, "publish"); + + const results = await race( + async () => + await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }), + async () => + await editorial(h.rivalContext).update( + id, + { title: "Edited while publishing" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ); + + const row = await rowOf(id); + const history = await revisionsOf(id); + + // However they interleaved: no version was written twice, and the row's + // version equals the highest revision. + expect(new Set(history.map(entry => entry.version)).size).toBe( + history.length, + ); + expect(row.version).toBe( + Math.max(...history.map(entry => entry.version)), + ); + + // The publish either committed or was rolled back whole - never half. + const published = history.some(entry => entry.operation === "publish"); + expect(row.status).toBe(published ? "published" : "draft"); + expect(fulfilledCount(results)).toBeGreaterThan(0); + }); + }); + + // ------------------------------------------------------------------------- + // Translations + // ------------------------------------------------------------------------- + + describe("translations", () => { + const guide = async (title: string) => { + const { row } = await localizedService(h.context).create( + { shared: {}, translation: { body: `Body of ${title}`, title } }, + { actor: ACTOR }, + ); + + return row.id; + }; + + const translationRows = async (itemId: number) => + await h.sql<{ languageId: number; title: string; version: number }[]>` + SELECT "languageId", "title", "version" + FROM "example_localized_articles_translations" + WHERE "itemId" = ${itemId} + ORDER BY "languageId" + `; + + it("lets exactly one of two writers on the same locale win", async () => { + const itemId = await guide("Same Locale"); + await translationEditorial(h.context).create( + itemId, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + + const results = await race( + async () => + await translationEditorial(h.context).update( + itemId, + "pl", + { title: "Polski A" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await translationEditorial(h.rivalContext).update( + itemId, + "pl", + { title: "Polski B" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + expect( + reasons(results).every( + error => error instanceof ContentTranslationVersionConflict, + ), + ).toBe(true); + + const rows = await translationRows(itemId); + expect(rows.find(row => row.languageId === 2)?.version).toBe(2); + }); + + it("lets two locales be written at the same time, independently", async () => { + const itemId = await guide("Two Locales"); + await translationEditorial(h.context).create( + itemId, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + + const results = await race( + async () => + await translationEditorial(h.context).update( + itemId, + "pl", + { title: "Polski Nowy" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await translationEditorial(h.rivalContext).update( + itemId, + "en", + { title: "English New" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ); + + // Two version domains, so both win: somebody editing Polish must never be + // told the English copy moved. + expect(fulfilledCount(results)).toBe(2); + + const rows = await translationRows(itemId); + expect(rows.map(row => row.version)).toEqual([2, 2]); + expect(rows.map(row => row.title).sort()).toEqual([ + "English New", + "Polski Nowy", + ]); + }); + + it("lets a locale write and a shared write both succeed", async () => { + const itemId = await guide("Shared And Local"); + await translationEditorial(h.context).create( + itemId, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + + const results = await race( + async () => + await translationEditorial(h.context).update( + itemId, + "pl", + { title: "Polski Zmieniony" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + async () => + await h.rivalDb.execute( + // The shared half, written straight through SQL: what matters here + // is that the two version columns are on two different rows, so + // neither guard can see the other's write. + `UPDATE "example_localized_articles" SET "featured" = true WHERE "id" = ${itemId}`, + ), + ); + + expect(fulfilledCount(results)).toBe(2); + + const [base] = await h.sql<{ featured: boolean }[]>` + SELECT "featured" FROM "example_localized_articles" WHERE "id" = ${itemId} + `; + expect(base.featured).toBe(true); + expect( + (await translationRows(itemId)).find(row => row.languageId === 2) + ?.title, + ).toBe("Polski Zmieniony"); + }); + + it("never resurrects a translation a concurrent delete removed", async () => { + const itemId = await guide("Delete Race"); + await translationEditorial(h.context).create( + itemId, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + + const results = await race( + async () => + await translationEditorial(h.context).delete(itemId, "pl", { + actor: ACTOR, + expectedVersion: 1, + }), + async () => + await translationEditorial(h.rivalContext).update( + itemId, + "pl", + { title: "Stale" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ); + + const rows = await translationRows(itemId); + const polish = rows.find(row => row.languageId === 2); + + const [deleteResult] = results; + if (deleteResult.status === "fulfilled" && deleteResult.value) { + expect(polish).toBeUndefined(); + + return; + } + + // The update won, so the delete was refused - and the translation holds + // the update's value rather than a mixture. + expect(polish?.title).toBe("Stale"); + }); + }); + + // ------------------------------------------------------------------------- + // Advanced collections + // ------------------------------------------------------------------------- + + describe("advanced collections", () => { + let categories: number[] = []; + + const advancedArticle = async () => { + // `title` is localized on this content type, so it is not a shared field + // and never appears in a base create payload. + const outcome = await advanced(h.context).create( + { categories: [] }, + { actor: ACTOR }, + ); + + return { id: outcome.row.id, version: outcome.version }; + }; + + const junctionRows = async (id: number) => + await h.sql<{ position: number; relatedItemId: number }[]>` + SELECT "relatedItemId", "position" + FROM "example_advanced_articles_categories" + WHERE "itemId" = ${id} + ORDER BY "position" + `; + + const relatedRows = async (id: number) => + await h.sql<{ position: number; relatedItemId: number }[]>` + SELECT "relatedItemId", "position" + FROM "example_advanced_articles_related_articles" + WHERE "itemId" = ${id} + ORDER BY "position" + `; + + const versionOfAdvanced = async (id: number) => { + const [row] = await h.sql<{ version: number }[]>` + SELECT "version" FROM "example_advanced_articles" WHERE "id" = ${id} + `; + + return row.version; + }; + + const faqRows = async (id: number) => + await h.sql<{ id: number; position: number; question: string }[]>` + SELECT "id", "position", "question" + FROM "example_advanced_articles_faq" + WHERE "itemId" = ${id} + ORDER BY "position" + `; + + beforeEach(async () => { + const rows = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") + VALUES ('One'), ('Two'), ('Three') RETURNING "id" + `; + categories = rows.map(row => row.id); + }); + + it("refuses one of two racing adds and leaves no half-written set", async () => { + const { id, version } = await advancedArticle(); + + const results = await race( + async () => + await advanced(h.context).relations.categories.add( + id, + categories[0], + { actor: ACTOR, expectedVersion: version }, + ), + async () => + await advanced(h.rivalContext).relations.categories.add( + id, + categories[1], + { actor: ACTOR, expectedVersion: version }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + + // Exactly one target, at position 0 - never two rows written by two + // writers who each thought the set was empty. + const rows = await junctionRows(id); + expect(rows).toHaveLength(1); + expect(rows[0].position).toBe(0); + }); + + it("refuses a remove racing an add", async () => { + const { id, version } = await advancedArticle(); + const seeded = await advanced(h.context).relations.categories.set( + id, + [categories[0]], + { actor: ACTOR, expectedVersion: version }, + ); + const at = seeded?.version ?? version; + + const results = await race( + async () => + await advanced(h.context).relations.categories.remove( + id, + categories[0], + { actor: ACTOR, expectedVersion: at }, + ), + async () => + await advanced(h.rivalContext).relations.categories.add( + id, + categories[1], + { actor: ACTOR, expectedVersion: at }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + + const rows = await junctionRows(id); + // Either the removal happened (empty) or the addition did (two targets) - + // never the removal's result with the addition's row in it. + expect([0, 2]).toContain(rows.length); + expect(rows.map(row => row.position)).toEqual( + rows.map((_row, index) => index), + ); + }); + + /** + * A reorder needs an **ordered** relation to be a mutation at all. + * + * `categories` is unordered: the engine stores it in ascending target order, + * so `reorder` there computes the list that is already stored and is a no-op + * by construction. `relatedArticles` is `ordered: true`, which is what makes + * the author's sequence a fact the database holds - and what makes racing a + * reorder against something else a real contest. + */ + it("keeps positions contiguous when a reorder races an add", async () => { + const { id, version } = await advancedArticle(); + const first = await advancedArticle(); + const second = await advancedArticle(); + const third = await advancedArticle(); + + const seeded = await advanced(h.context).relations.relatedArticles.set( + id, + [first.id, second.id], + { actor: ACTOR, expectedVersion: version }, + ); + const at = seeded?.version ?? version; + + const results = await race( + async () => + await advanced(h.context).relations.relatedArticles.reorder( + id, + [second.id, first.id], + { actor: ACTOR, expectedVersion: at }, + ), + async () => + await advanced(h.rivalContext).relations.relatedArticles.add( + id, + third.id, + { actor: ACTOR, expectedVersion: at }, + ), + ); + + // Exactly one real mutation, so exactly one version increment. The loser + // either lost the guard or found its own computation was a no-op; neither + // writes a junction row. + expect(changedCount(results)).toBe(1); + expect(await versionOfAdvanced(id)).toBe(at + 1); + + const rows = await relatedRows(id); + expect(rows.map(row => row.position)).toEqual( + rows.map((_row, index) => index), + ); + expect(new Set(rows.map(row => row.relatedItemId)).size).toBe( + rows.length, + ); + }); + + it("keeps positions contiguous when a reorder races a remove", async () => { + const { id, version } = await advancedArticle(); + const first = await advancedArticle(); + const second = await advancedArticle(); + const third = await advancedArticle(); + + const seeded = await advanced(h.context).relations.relatedArticles.set( + id, + [first.id, second.id, third.id], + { actor: ACTOR, expectedVersion: version }, + ); + const at = seeded?.version ?? version; + + const results = await race( + async () => + await advanced(h.context).relations.relatedArticles.reorder( + id, + [third.id, second.id, first.id], + { actor: ACTOR, expectedVersion: at }, + ), + async () => + await advanced(h.rivalContext).relations.relatedArticles.remove( + id, + first.id, + { actor: ACTOR, expectedVersion: at }, + ), + ); + + expect(changedCount(results)).toBe(1); + expect(await versionOfAdvanced(id)).toBe(at + 1); + + const rows = await relatedRows(id); + expect([2, 3]).toContain(rows.length); + expect(rows.map(row => row.position)).toEqual( + rows.map((_row, index) => index), + ); + }); + + it("refuses a repeatable delete racing an update of the same child", async () => { + const { id, version } = await advancedArticle(); + const seeded = await advanced(h.context).repeatable.faq.set( + id, + [ + { answer: "A1", question: "Question one" }, + { answer: "A2", question: "Question two" }, + ], + { actor: ACTOR, expectedVersion: version }, + ); + const at = seeded?.version ?? version; + const children = await faqRows(id); + + const results = await race( + async () => + await advanced(h.context).repeatable.faq.delete(id, children[0].id, { + actor: ACTOR, + expectedVersion: at, + }), + async () => + await advanced(h.rivalContext).repeatable.faq.update( + id, + children[0].id, + { question: "Question one edited" }, + { actor: ACTOR, expectedVersion: at }, + ), + ); + + // At most one *real* mutation. The loser either lost the version guard or + // discovered its own computation was a no-op - editing a child that is + // already gone changes nothing, and a no-op is deliberately not a + // conflict, because there is nothing to overwrite. + expect(changedCount(results)).toBe(1); + expect(await versionOfAdvanced(id)).toBe(at + 1); + + const rows = await faqRows(id); + // Either the child is gone or it holds the edit - never a resurrected row + // carrying the pre-edit values. + const first = rows.find(row => row.id === children[0].id); + if (first) expect(first.question).toBe("Question one edited"); + expect(rows.map(row => row.position)).toEqual( + rows.map((_row, index) => index), + ); + }); + + it("refuses a child update racing a reorder", async () => { + const { id, version } = await advancedArticle(); + const seeded = await advanced(h.context).repeatable.faq.set( + id, + [ + { answer: "A1", question: "Question one" }, + { answer: "A2", question: "Question two" }, + ], + { actor: ACTOR, expectedVersion: version }, + ); + const at = seeded?.version ?? version; + const children = await faqRows(id); + + const results = await race( + async () => + await advanced(h.context).repeatable.faq.update( + id, + children[0].id, + { question: "Question one edited" }, + { actor: ACTOR, expectedVersion: at }, + ), + async () => + await advanced(h.rivalContext).repeatable.faq.reorder( + id, + [children[1].id, children[0].id], + { actor: ACTOR, expectedVersion: at }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + + const rows = await faqRows(id); + expect(rows).toHaveLength(2); + expect(rows.map(row => row.position)).toEqual([0, 1]); + // Identity survived whichever way it went: both children are still the + // rows they were, not recreated ones. + expect(new Set(rows.map(row => row.id))).toEqual( + new Set(children.map(row => row.id)), + ); + }); + + it("refuses a collection write racing a scalar write on the same version", async () => { + const { id, version } = await advancedArticle(); + + const results = await race( + async () => + await advanced(h.context).relations.categories.add( + id, + categories[0], + { actor: ACTOR, expectedVersion: version }, + ), + async () => + await advanced(h.rivalContext).update( + id, + { syndication: { indexable: false, priority: 3 } }, + { actor: ACTOR, expectedVersion: version }, + ), + ); + + expect(fulfilledCount(results)).toBe(1); + + const [row] = await h.sql< + { syndicationPriority: number; version: number }[] + >` + SELECT "version", "syndicationPriority" FROM "example_advanced_articles" + WHERE "id" = ${id} + `; + expect(row.version).toBe(version + 1); + + const junction = await junctionRows(id); + // One or the other, never both halves of two different writers. + if (junction.length > 0) { + expect(row.syndicationPriority).toBe(5); + } else { + expect(row.syndicationPriority).toBe(3); + } + }); + + it("keeps two different records independent under load", async () => { + const first = await advancedArticle(); + const second = await advancedArticle(); + + const results = await race( + async () => + await advanced(h.context).relations.categories.add( + first.id, + categories[0], + { actor: ACTOR, expectedVersion: first.version }, + ), + async () => + await advanced(h.rivalContext).relations.categories.add( + second.id, + categories[1], + { actor: ACTOR, expectedVersion: second.version }, + ), + ); + + // The lock is per row, so two records never contend. + expect(fulfilledCount(results)).toBe(2); + expect(await junctionRows(first.id)).toHaveLength(1); + expect(await junctionRows(second.id)).toHaveLength(1); + }); + }); + + // ------------------------------------------------------------------------- + // The plain service, which merges rather than arbitrating + // ------------------------------------------------------------------------- + + describe("the plain service serialises instead of conflicting", () => { + let categories: number[] = []; + + beforeEach(async () => { + const rows = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") + VALUES ('Plain One'), ('Plain Two') RETURNING "id" + `; + categories = rows.map(row => row.id); + }); + + it("keeps both concurrent additions", async () => { + // No version column to guard on, so the row lock does the job instead: + // the second `add` waits, then reads what the first committed. + const outcome = await advanced(h.context).create( + { categories: [] }, + { actor: ACTOR }, + ); + const id = outcome.row.id; + + const results = await race( + async () => + await plainAdvanced(h.context).relations.categories.add( + id, + categories[0], + ), + async () => + await plainAdvanced(h.rivalContext).relations.categories.add( + id, + categories[1], + ), + ); + + expect(fulfilledCount(results)).toBe(2); + + const rows = await h.sql<{ position: number }[]>` + SELECT "position" FROM "example_advanced_articles_categories" + WHERE "itemId" = ${id} ORDER BY "position" + `; + expect(rows.map(row => row.position)).toEqual([0, 1]); + }); + }); +}); diff --git a/plugins/example/src/database/delivery-postgres.test.ts b/plugins/example/src/database/delivery-postgres.test.ts new file mode 100644 index 000000000..b966301fd --- /dev/null +++ b/plugins/example/src/database/delivery-postgres.test.ts @@ -0,0 +1,1655 @@ +import type { SearchDocument } from "@vitnode/core/api/models/search"; +import type { ContentDeliverySitemapPage } from "@vitnode/core/content/server"; +import type { Context } from "hono"; + +import { + ContentDeliverySlugReserved, + ContentVersionConflict, +} from "@vitnode/core/content"; +import { + contentDeliveryEffects, + contentEditorialEffects, + contentTranslationEffects, +} from "@vitnode/core/content/server"; +import { drizzle } from "drizzle-orm/postgres-js"; +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import postgres from "postgres"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { CONFIG_PLUGIN, EXAMPLE_MIGRATIONS } from "@/const"; + +import { advancedArticleContent } from "./advanced-articles"; +import { articleContent } from "./articles"; +import { categoryContent } from "./categories"; + +/** + * Stage 8 against real Postgres. + * + * Everything here is about what the *database* enforces and what the resolver + * actually answers, neither of which a mock can show: + * + * - a historical URL is **reserved** by two partial unique indexes, so an unrelated + * record cannot inherit somebody's incoming links; + * - a redirect chain collapses because the resolver reads the record's current slug + * rather than the next entry in the chain; + * - a slug change and its reservation are **one transaction**, so a writer that + * loses the version race leaves the history exactly as it found it; + * - each locale's history is its own, because `languageId` is part of the key. + * + * Runs only with `DATABASE_TEST_URL` set, and **wipes** the database it points at - + * so the URL has to name one with "test" in it: + * + * ```bash + * DATABASE_TEST_URL=postgres://postgres:postgres@localhost:5432/vitnode_test \ + * pnpm --filter @vitnode/example test + * ``` + */ +const url = process.env.DATABASE_TEST_URL; + +const databaseName = (() => { + if (!url) return ""; + try { + return new URL(url).pathname.replace(/^\//, ""); + } catch { + return ""; + } +})(); + +const here = dirname(fileURLToPath(import.meta.url)); + +const migrationSql = (files: readonly string[]): string => + files + .map(file => + readFileSync( + resolve(here, "../../../../apps/docs/migrations", file), + "utf8", + ), + ) + .join("\n--> statement-breakpoint\n"); + +const CORE_STUBS = ` + CREATE TABLE "core_users" ( + "id" serial PRIMARY KEY NOT NULL, + "name" varchar(255) NOT NULL + ); + CREATE TABLE "core_queue" ( + "id" serial PRIMARY KEY NOT NULL, + "pluginId" varchar(255) NOT NULL, + "name" varchar(100) NOT NULL, + "queue" varchar(100) DEFAULT 'default' NOT NULL, + "status" varchar(20) DEFAULT 'pending' NOT NULL, + "payload" jsonb DEFAULT '{}'::jsonb NOT NULL, + "priority" integer DEFAULT 0 NOT NULL, + "attempts" integer DEFAULT 0 NOT NULL, + "maxAttempts" integer DEFAULT 3 NOT NULL, + "availableAt" timestamp DEFAULT now() NOT NULL, + "reservedAt" timestamp, + "lastError" text, + "createdAt" timestamp DEFAULT now() NOT NULL, + "updatedAt" timestamp DEFAULT now() NOT NULL, + "completedAt" timestamp + ); + CREATE TABLE "core_languages" ( + "id" serial PRIMARY KEY NOT NULL, + "code" varchar(32) NOT NULL, + "name" varchar(255) NOT NULL, + "default" boolean DEFAULT false NOT NULL, + "protected" boolean DEFAULT false NOT NULL, + CONSTRAINT "core_languages_code_unique" UNIQUE("code") + ); +`; + +const ACTOR = { type: "staff" as const, userId: null }; +const PLUGIN = CONFIG_PLUGIN.pluginId; + +let sql: ReturnType; +let db: ReturnType; +let context: Context; +/** A second connection, for the tests that need two writers at once. */ +let rival: ReturnType; +let rivalContext: Context; +let categoryId = 0; + +const emitted: { name: string; payload: Record }[] = []; +const indexed: SearchDocument[] = []; + +const pgErrorCode = async (run: () => Promise) => { + try { + await run(); + } catch (error) { + const cause = (error as { cause?: { code?: string } }).cause; + + return cause?.code ?? (error as { code?: string }).code; + } + + return undefined; +}; + +// --------------------------------------------------------------------------- +// Nonlocalized fixture: `example.article` +// --------------------------------------------------------------------------- + +const editorial = (target: Context = context) => + articleContent.editorialService?.(target, { pluginId: PLUGIN }); + +const delivery = (target: Context = context) => + articleContent.deliveryService?.(target, { pluginId: PLUGIN }); + +/** + * A monotonic counter for the `unique: true` `code` field. + * + * `Date.now()` is not enough: several articles are created inside one millisecond by + * the tests below, and a duplicate `code` would surface as a `23505` from an + * unrelated constraint. + */ +let nextCode = 0; + +const createArticle = async ( + values: Record = {}, +): Promise<{ id: number; version: number }> => { + nextCode += 1; + + const outcome = await editorial()?.create( + { + category: categoryId, + code: `code-${nextCode}`, + title: "Hello world", + ...values, + }, + { actor: ACTOR }, + ); + if (!outcome) throw new Error("create returned nothing"); + + return { id: outcome.row.id, version: outcome.version }; +}; + +/** Creates, publishes, and hands back the version to write against next. */ +const publishArticle = async ( + values: Record = {}, +): Promise<{ id: number; version: number }> => { + const created = await createArticle(values); + const published = await editorial()?.publish(created.id, { actor: ACTOR }); + if (!published) throw new Error("publish returned nothing"); + + return { id: created.id, version: published.version }; +}; + +/** + * One record's addresses, as plain objects. + * + * `postgres.js` hands back a `Result` array subclass whose prototype is not + * `Array.prototype`, which `toStrictEqual` compares - so every raw read in this file + * is normalised rather than asserted directly. + */ +const historyRows = async ( + itemId: number, +): Promise<{ path: string; retired: boolean; slug: string }[]> => { + const rows = await sql< + { path: string; retiredAt: null | string; slug: string }[] + >` + SELECT "slug", "path", "retiredAt" + FROM "core_content_slug_history" + WHERE "contentTypeId" = 'example.article' AND "itemId" = ${itemId} + ORDER BY "id" + `; + + return rows.map(row => ({ + path: row.path, + retired: row.retiredAt !== null, + slug: row.slug, + })); +}; + +// --------------------------------------------------------------------------- +// Localized fixture: `example.advanced-article` +// --------------------------------------------------------------------------- + +const localizedService = () => + advancedArticleContent.localizedService?.(context, { pluginId: PLUGIN }); + +const translationEditorial = (target: Context = context) => + advancedArticleContent.translationEditorialService?.(target, { + pluginId: PLUGIN, + }); + +const advancedEditorial = () => + advancedArticleContent.editorialService?.(context, { pluginId: PLUGIN }); + +const advancedDelivery = () => + advancedArticleContent.deliveryService?.(context, { pluginId: PLUGIN }); + +/** + * A localized article, published in `en` and optionally in `pl`. + * + * Both halves are published on purpose: a translation is only publicly reachable + * when the record is too, which is the subordination the delivery layer reads. + */ +const publishLocalized = async ({ + pl, + title = "Hello world", +}: { pl?: string; title?: string } = {}) => { + const localized = localizedService(); + if (!localized) throw new Error("no localized service"); + + const created = await localized.create({ + shared: {}, + translation: { title }, + }); + + const base = await advancedEditorial()?.publish(created.row.id, { + actor: ACTOR, + }); + if (!base) throw new Error("base publish returned nothing"); + + const en = await translationEditorial()?.publish(created.row.id, "en", { + actor: ACTOR, + }); + if (!en) throw new Error("en publish returned nothing"); + + if (pl !== undefined) { + await translationEditorial()?.create( + created.row.id, + "pl", + { title: pl }, + { actor: ACTOR }, + ); + await translationEditorial()?.publish(created.row.id, "pl", { + actor: ACTOR, + }); + } + + return { enVersion: en.version, id: created.row.id }; +}; + +const localizedHistory = async ( + itemId: number, +): Promise< + { languageId: null | number; path: string; retired: boolean; slug: string }[] +> => { + const rows = await sql< + { + languageId: null | number; + path: string; + retiredAt: null | string; + slug: string; + }[] + >` + SELECT "slug", "path", "retiredAt", "languageId" + FROM "core_content_slug_history" + WHERE "contentTypeId" = 'example.advanced-article' AND "itemId" = ${itemId} + ORDER BY "id" + `; + + return rows.map(row => ({ + languageId: row.languageId, + path: row.path, + retired: row.retiredAt !== null, + slug: row.slug, + })); +}; + +const localeIds: Record = {}; + +describe.skipIf(!url)("Stage 8 content delivery against Postgres", () => { + beforeAll(async () => { + if (!/test/i.test(databaseName)) { + throw new Error( + `DATABASE_TEST_URL points at "${databaseName || url}". This suite wipes the database it runs against, so its name must contain "test".`, + ); + } + + sql = postgres(url ?? "", { max: 1, onnotice: () => undefined }); + + await sql.unsafe(` + DROP SCHEMA IF EXISTS public CASCADE; + CREATE SCHEMA public; + `); + await sql.unsafe(CORE_STUBS); + const languages = await sql<{ code: string; id: number }[]>` + INSERT INTO "core_languages" ("code", "name", "default") VALUES + ('en', 'English', true), + ('pl', 'Polski', false) + RETURNING "id", "code" + `; + for (const language of languages) localeIds[language.code] = language.id; + + for (const statement of migrationSql(EXAMPLE_MIGRATIONS).split( + "--> statement-breakpoint", + )) { + const trimmed = statement.trim(); + if (trimmed) await sql.unsafe(trimmed); + } + + db = drizzle(sql, { casing: "camelCase" }); + rival = postgres(url ?? "", { max: 1, onnotice: () => undefined }); + + const buildContext = (handle: ReturnType) => + ({ + get: (key: string) => { + if (key === "db") return handle; + if (key === "search") { + return { + delete: async () => await Promise.resolve(), + index: async (document: SearchDocument) => { + indexed.push(document); + + return await Promise.resolve(); + }, + }; + } + if (key === "events") { + return { + emit: async (name: string, payload: Record) => { + emitted.push({ name, payload }); + + return await Promise.resolve({ failures: [] }); + }, + }; + } + if (key === "log") + return { error: async () => await Promise.resolve() }; + if (key === "core") { + return { + contentModels: [ + { model: advancedArticleContent, pluginId: PLUGIN }, + { model: articleContent, pluginId: PLUGIN }, + { model: categoryContent, pluginId: PLUGIN }, + ], + i18n: { + locales: [ + { code: "en", name: "English" }, + { code: "pl", name: "Polski" }, + ], + }, + }; + } + + return undefined; + }, + }) as unknown as Context; + + context = buildContext(db); + rivalContext = buildContext(drizzle(rival, { casing: "camelCase" })); + }, 60_000); + + afterAll(async () => { + await sql?.end(); + await rival?.end(); + }); + + beforeEach(async () => { + await sql`DELETE FROM "example_articles"`; + await sql`DELETE FROM "example_advanced_articles"`; + await sql`DELETE FROM "core_content_slug_history"`; + await sql`DELETE FROM "core_content_revisions"`; + await sql`DELETE FROM "example_categories"`; + + const [category] = await sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('News') RETURNING "id" + `; + categoryId = category.id; + emitted.length = 0; + indexed.length = 0; + }); + + // ------------------------------------------------------------------------- + // The table itself + // ------------------------------------------------------------------------- + + describe("the reservation constraints", () => { + it("refuses two shared rows for the same address", async () => { + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "slug", "path") + VALUES (${PLUGIN}, 'example.article', 1, 'hello', '/articles/hello') + `; + + const code = await pgErrorCode( + async () => + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "slug", "path") + VALUES (${PLUGIN}, 'example.article', 2, 'hello', '/articles/hello') + `, + ); + + // The partial unique index over `(contentTypeId, slug) WHERE languageId IS + // NULL` is what makes a retired URL a reservation rather than only a log. + expect(code).toBe("23505"); + }); + + it("allows the same address in two different locales", async () => { + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "languageId", "slug", "path") + VALUES + (${PLUGIN}, 'example.advanced-article', 1, ${localeIds.en}, 'shared', '/en/advanced-articles/shared'), + (${PLUGIN}, 'example.advanced-article', 2, ${localeIds.pl}, 'shared', '/pl/advanced-articles/shared') + `; + + const [row] = await sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "core_content_slug_history" + `; + + // Locale-scoped uniqueness: `/en/x/shared` and `/pl/x/shared` are two URLs. + expect(row.count).toBe(2); + }); + + it("refuses two rows for the same address in one locale", async () => { + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "languageId", "slug", "path") + VALUES (${PLUGIN}, 'example.advanced-article', 1, ${localeIds.en}, 'hello', '/en/advanced-articles/hello') + `; + + const code = await pgErrorCode( + async () => + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "languageId", "slug", "path") + VALUES (${PLUGIN}, 'example.advanced-article', 2, ${localeIds.en}, 'hello', '/en/advanced-articles/hello') + `, + ); + + expect(code).toBe("23505"); + }); + + it("keeps two content types' histories apart", async () => { + await sql` + INSERT INTO "core_content_slug_history" + ("pluginId", "contentTypeId", "itemId", "slug", "path") + VALUES + (${PLUGIN}, 'example.article', 1, 'hello', '/articles/hello'), + (${PLUGIN}, 'other.thing', 1, 'hello', '/things/hello') + `; + + const [row] = await sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "core_content_slug_history" + `; + + expect(row.count).toBe(2); + }); + + it("indexes the resolver's lookup", async () => { + // The redirect lookup is on a public request path for a URL that is very + // often a typo, so it has to be an index hit rather than a scan. + const rows = await sql<{ indexdef: string; indexname: string }[]>` + SELECT indexname, indexdef FROM pg_indexes + WHERE tablename = 'core_content_slug_history' + `; + const names = rows.map(row => row.indexname); + + expect(names).toContain("core_content_slug_history_shared_unique"); + expect(names).toContain("core_content_slug_history_locale_unique"); + expect(names).toContain("core_content_slug_history_item_idx"); + + const shared = rows.find( + row => row.indexname === "core_content_slug_history_shared_unique", + ); + expect(shared?.indexdef).toContain("UNIQUE"); + expect(shared?.indexdef).toMatch(/"?languageId"? IS NULL/); + }); + }); + + // ------------------------------------------------------------------------- + // The redirect lifecycle + // ------------------------------------------------------------------------- + + describe("the redirect lifecycle", () => { + it("records nothing while the record is still a draft", async () => { + const article = await createArticle({ title: "Draft article" }); + + await editorial()?.update( + article.id, + { slug: "corrected" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + // A draft has no public URL, so neither the original nor the corrected slug + // was ever addressable - and neither is reserved. + expect(await historyRows(article.id)).toStrictEqual([]); + }); + + it("reserves the current address on publication", async () => { + const article = await publishArticle({ title: "Hello world" }); + + const rows = await historyRows(article.id); + + expect(rows).toStrictEqual([ + { + path: "/articles/hello-world", + retired: false, + slug: "hello-world", + }, + ]); + }); + + it("resolves the current slug as canonical content", async () => { + const article = await publishArticle({ title: "Hello world" }); + + expect(await delivery()?.resolveSlug("hello-world")).toMatchObject({ + canonicalPath: "/articles/hello-world", + // `example.article` does not expose `id`, so delivery reports none rather + // than publishing a column the public API withheld. + itemId: null, + type: "content", + }); + expect(article.id).toBeGreaterThan(0); + }); + + it("redirects the old address after a slug change", async () => { + const article = await publishArticle({ title: "Hello world" }); + + await editorial()?.update( + article.id, + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + expect(await delivery()?.resolveSlug("hello-world")).toStrictEqual({ + location: "/articles/hello-there", + status: 308, + type: "redirect", + }); + expect(await delivery()?.resolveSlug("hello-there")).toMatchObject({ + canonicalPath: "/articles/hello-there", + type: "content", + }); + }); + + it("collapses a chain: A and B both resolve straight to C", async () => { + const article = await publishArticle({ title: "Slug a" }); + + const toB = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + await editorial()?.update( + article.id, + { slug: "slug-c" }, + { actor: ACTOR, expectedVersion: toB?.version ?? 0 }, + ); + + // One hop each, never A -> B -> C. + for (const retired of ["slug-a", "slug-b"]) { + expect(await delivery()?.resolveSlug(retired)).toStrictEqual({ + location: "/articles/slug-c", + status: 308, + type: "redirect", + }); + } + expect(await delivery()?.resolveSlug("slug-c")).toMatchObject({ + type: "content", + }); + }); + + it("keeps three rows: two retired and one current", async () => { + const article = await publishArticle({ title: "Slug a" }); + const toB = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + await editorial()?.update( + article.id, + { slug: "slug-c" }, + { actor: ACTOR, expectedVersion: toB?.version ?? 0 }, + ); + + const rows = await historyRows(article.id); + + expect(rows.map(row => row.slug)).toStrictEqual([ + "slug-a", + "slug-b", + "slug-c", + ]); + expect(rows.map(row => row.retired)).toStrictEqual([true, true, false]); + // The database keeps the chronology; the resolver is what collapses it. + expect(rows[0].path).toBe("/articles/slug-a"); + }); + + it("stops redirecting while the record is unpublished, and starts again", async () => { + const article = await publishArticle({ title: "Slug a" }); + const moved = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + const unpublished = await editorial()?.unpublish(article.id, { + actor: ACTOR, + }); + + expect(await delivery()?.resolveSlug("slug-a")).toStrictEqual({ + type: "not_found", + }); + expect(await delivery()?.resolveSlug("slug-b")).toStrictEqual({ + type: "not_found", + }); + // The history survives - it is what makes the redirect come back. + expect((await historyRows(article.id)).length).toBe(2); + + await editorial()?.publish(article.id, { + actor: ACTOR, + expectedVersion: unpublished?.version, + }); + + expect(await delivery()?.resolveSlug("slug-a")).toStrictEqual({ + location: "/articles/slug-b", + status: 308, + type: "redirect", + }); + expect(moved?.delivery?.redirectCreated).toBe(true); + }); + + it("keeps the history but resolves nothing after a delete", async () => { + const article = await publishArticle({ title: "Slug a" }); + const moved = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + await editorial()?.delete(article.id, { + actor: ACTOR, + expectedVersion: moved?.version ?? 0, + }); + + // Retained for audit, and never a redirect to content that is gone. + expect((await historyRows(article.id)).length).toBe(2); + expect(await delivery()?.resolveSlug("slug-a")).toStrictEqual({ + type: "not_found", + }); + expect(await delivery()?.resolveSlug("slug-b")).toStrictEqual({ + type: "not_found", + }); + }); + + it("brings a slug back into service when it is restored", async () => { + const article = await publishArticle({ title: "Original name" }); + const [original] = + (await editorial()?.revisions.list(article.id))?.edges ?? []; + + const moved = await editorial()?.update( + article.id, + { slug: "new-name" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + const restored = await editorial()?.restore(article.id, original.id, { + actor: ACTOR, + expectedVersion: moved?.version ?? 0, + }); + + expect(restored?.delivery).toMatchObject({ + canonicalPath: "/articles/original-name", + previousPath: "/articles/new-name", + redirectCreated: true, + slugChanged: true, + }); + + // The two addresses have swapped roles: `new-name` now redirects to the + // restored `original-name`. + expect(await delivery()?.resolveSlug("new-name")).toStrictEqual({ + location: "/articles/original-name", + status: 308, + type: "redirect", + }); + expect(await delivery()?.resolveSlug("original-name")).toMatchObject({ + type: "content", + }); + }); + + it("writes no history for a restore that moves no slug", async () => { + const article = await publishArticle({ title: "Stable" }); + const [first] = + (await editorial()?.revisions.list(article.id))?.edges ?? []; + + const edited = await editorial()?.update( + article.id, + { excerpt: "Changed prose" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + const restored = await editorial()?.restore(article.id, first.id, { + actor: ACTOR, + expectedVersion: edited?.version ?? 0, + }); + + expect(restored?.delivery?.slugChanged).toBe(false); + expect( + (await historyRows(article.id)).map(row => row.slug), + ).toStrictEqual(["stable"]); + }); + }); + + // ------------------------------------------------------------------------- + // Reservations + // ------------------------------------------------------------------------- + + describe("slug reservations", () => { + it("refuses an address another record retired", async () => { + const first = await publishArticle({ title: "Hello" }); + await editorial()?.update( + first.id, + { slug: "hello-world" }, + { actor: ACTOR, expectedVersion: first.version }, + ); + + // `hello` is free on the content table now - the first article moved off it - + // so the reservation is the only thing standing between the second article + // and somebody else's incoming links. + await expect( + createArticle({ slug: "hello", title: "Second" }), + ).rejects.toThrow(ContentDeliverySlugReserved); + }); + + it("names the address in the structured error", async () => { + const first = await publishArticle({ title: "Hello" }); + await editorial()?.update( + first.id, + { slug: "hello-world" }, + { actor: ACTOR, expectedVersion: first.version }, + ); + + const error = await createArticle({ + slug: "hello", + title: "Second", + }).catch((thrown: unknown) => thrown); + + expect(error).toBeInstanceOf(ContentDeliverySlugReserved); + expect(error).toMatchObject({ locale: null, slug: "hello" }); + }); + + it("lets a record take its own retired address back", async () => { + const article = await publishArticle({ title: "Slug a" }); + const toB = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + const back = await editorial()?.update( + article.id, + { slug: "slug-a" }, + { actor: ACTOR, expectedVersion: toB?.version ?? 0 }, + ); + + expect(back?.delivery?.canonicalPath).toBe("/articles/slug-a"); + expect(await delivery()?.resolveSlug("slug-b")).toStrictEqual({ + location: "/articles/slug-a", + status: 308, + type: "redirect", + }); + // Two rows, and `slug-a` is live again rather than duplicated. + const rows = await historyRows(article.id); + expect(rows).toHaveLength(2); + expect(rows.find(row => row.slug === "slug-a")?.retired).toBe(false); + }); + + it("never reserves a draft's address", async () => { + await createArticle({ slug: "wanted", title: "A draft" }); + + // A draft has no public URL, so another record may still publish at that + // address - the content table's own unique index is what stops a *live* + // duplicate, and it fires on the create below rather than the reservation. + const code = await pgErrorCode( + async () => await createArticle({ slug: "wanted", title: "Another" }), + ); + + expect(code).toBe("23505"); + const [rows] = await sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "core_content_slug_history" + `; + expect(rows.count).toBe(0); + }); + }); + + // ------------------------------------------------------------------------- + // Concurrency + // ------------------------------------------------------------------------- + + describe("concurrency", () => { + it("lets one of two racing slug edits win and refuses the other", async () => { + const article = await publishArticle({ title: "Original" }); + + const results = await Promise.allSettled([ + editorial()?.update( + article.id, + { slug: "winner" }, + { actor: ACTOR, expectedVersion: article.version }, + ), + editorial(rivalContext)?.update( + article.id, + { slug: "loser" }, + { actor: ACTOR, expectedVersion: article.version }, + ), + ]); + + const rejected = results.filter(result => result.status === "rejected"); + expect(rejected).toHaveLength(1); + expect(rejected[0]).toMatchObject({ + reason: expect.any(ContentVersionConflict), + }); + + // One winner, so exactly one retirement and one new reservation - the loser's + // transaction rolled back and left the history as it found it. + const rows = await historyRows(article.id); + expect(rows).toHaveLength(2); + expect(rows.filter(row => !row.retired)).toHaveLength(1); + expect(rows.filter(row => row.slug === "loser")).toHaveLength(0); + }); + + it("keeps the history consistent when the write rolls back", async () => { + const article = await publishArticle({ title: "Original" }); + + // A stale expectation: the guarded UPDATE matches nothing, so the reservation + // never runs at all. + await expect( + editorial()?.update( + article.id, + { slug: "never-written" }, + { actor: ACTOR, expectedVersion: article.version + 5 }, + ), + ).rejects.toThrow(ContentVersionConflict); + + expect( + (await historyRows(article.id)).map(row => row.slug), + ).toStrictEqual(["original"]); + }); + + it("serialises two records racing for the same retired address", async () => { + const first = await publishArticle({ title: "Contested" }); + await editorial()?.update( + first.id, + { slug: "moved-on" }, + { actor: ACTOR, expectedVersion: first.version }, + ); + + const second = await createArticle({ slug: "second", title: "Second" }); + const third = await createArticle({ slug: "third", title: "Third" }); + + const results = await Promise.allSettled([ + editorial()?.update( + second.id, + { slug: "contested" }, + { actor: ACTOR, expectedVersion: second.version }, + ), + editorial(rivalContext)?.update( + third.id, + { slug: "contested" }, + { actor: ACTOR, expectedVersion: third.version }, + ), + ]); + + // Both lose: the address belongs to the first article's history, and neither + // of the two may take it. + expect(results.every(result => result.status === "rejected")).toBe(true); + }); + }); + + // ------------------------------------------------------------------------- + // Sitemap + // ------------------------------------------------------------------------- + + describe("sitemap", () => { + it("lists only published records", async () => { + const published = await publishArticle({ title: "Published one" }); + await createArticle({ title: "Still a draft" }); + + const page = await delivery()?.sitemap(); + + expect(page?.entries.map(entry => entry.itemId)).toStrictEqual([ + published.id, + ]); + expect(page?.entries[0]).toMatchObject({ + changeFrequency: "weekly", + path: "/articles/published-one", + priority: 0.7, + }); + }); + + it("omits a record whose publication date is in the future", async () => { + const article = await publishArticle({ title: "Scheduled" }); + await sql` + UPDATE "example_articles" + SET "publishedAt" = now() + interval '1 day' + WHERE "id" = ${article.id} + `; + + expect((await delivery()?.sitemap())?.entries).toStrictEqual([]); + }); + + it("paginates by keyset, without duplicates or gaps", async () => { + const ids: number[] = []; + for (const title of ["One", "Two", "Three", "Four", "Five"]) { + ids.push((await publishArticle({ title })).id); + } + + const seen: number[] = []; + let cursor: null | number | undefined = undefined; + + for (let page = 0; page < 10; page += 1) { + // Annotated, because the optional-call chain through `deliveryService?.()` + // loses the element type in the typed-lint program even though `tsc` + // resolves it - and `itemId` is exactly what this test is about. + const result: ContentDeliverySitemapPage | undefined = + await delivery()?.sitemap({ cursor: cursor ?? undefined, limit: 2 }); + if (!result) break; + + for (const entry of result.entries) seen.push(entry.itemId); + cursor = result.nextCursor; + if (cursor === null) break; + } + + // Every record exactly once, in ascending primary-key order. + expect(seen).toStrictEqual([...ids].sort((a, b) => a - b)); + expect(new Set(seen).size).toBe(seen.length); + expect(cursor).toBeNull(); + }); + + it("moves lastModified on an ordinary edit that keeps the URL", async () => { + const article = await publishArticle({ title: "Timestamped" }); + const first = await delivery()?.sitemap(); + const before = first?.entries[0].lastModified.getTime() ?? 0; + + // A title edit. The slug is never re-derived on update, so the URL is + // unchanged - and the sitemap's `` still has to move, which is the + // whole reason `contentChanged` cannot be "did membership change". + await editorial()?.update( + article.id, + { excerpt: "A new summary." }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + const second = await delivery()?.sitemap(); + + expect(second?.entries[0].path).toBe(first?.entries[0].path); + expect(second?.entries[0].lastModified.getTime()).toBeGreaterThan(before); + }); + + it("reports the edit as a sitemap content change but not an index change", async () => { + const article = await publishArticle({ title: "Timestamped two" }); + + const outcome = await editorial()?.update( + article.id, + { excerpt: "Changed." }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + // The invariant the cache layer reads: the file's bytes moved, the set of files + // did not. A stale sitemap is exactly what the first half prevents. + expect(outcome?.delivery?.sitemap).toStrictEqual({ + contentChanged: true, + indexChanged: false, + }); + }); + + it("reports no sitemap change for a no-op edit", async () => { + const article = await publishArticle({ title: "Untouched" }); + const before = await delivery()?.sitemap(); + + // Re-sending the stored value writes nothing, so `updatedAt` does not move and + // the cached sitemap is still byte-correct. + const outcome = await editorial()?.update( + article.id, + { title: "Untouched" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + + expect(outcome?.changed).toBe(false); + expect(outcome?.delivery).toBeUndefined(); + + const after = await delivery()?.sitemap(); + expect(after?.entries[0].lastModified.getTime()).toBe( + before?.entries[0].lastModified.getTime(), + ); + }); + + it("uses the base row's updatedAt for a nonlocalized entry", async () => { + const article = await publishArticle({ title: "Timestamped" }); + // Read through the same driver as the sitemap, never as `::text`: a + // `timestamp` column is rendered in the session's timezone as text and parsed + // back as an instant, so comparing the two forms compares two clocks. + const row = await articleContent.service(context).findById(article.id); + + const page = await delivery()?.sitemap(); + + expect(page?.entries[0].lastModified.toISOString()).toBe( + row?.updatedAt.toISOString(), + ); + }); + }); + + // ------------------------------------------------------------------------- + // Events + // ------------------------------------------------------------------------- + + describe("delivery events", () => { + it("emits both events after a live URL moves", async () => { + const article = await publishArticle({ title: "Slug a" }); + emitted.length = 0; + + const outcome = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + if (!outcome) throw new Error("update returned nothing"); + + await contentDeliveryEffects( + context, + articleContent.definition, + outcome.delivery, + { pluginId: PLUGIN }, + ); + + expect(emitted.map(entry => entry.name)).toStrictEqual([ + "content.example.article.delivery_slug_changed", + "content.example.article.delivery_redirect_created", + ]); + expect(emitted[0].payload).toMatchObject({ + canonicalPath: "/articles/slug-b", + contentId: article.id, + previousPath: "/articles/slug-a", + previousSlug: "slug-a", + slug: "slug-b", + }); + }); + + it("emits them alongside the ordinary update event, never instead of it", async () => { + const article = await publishArticle({ title: "Slug a" }); + emitted.length = 0; + + const outcome = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + if (!outcome) throw new Error("update returned nothing"); + + await contentEditorialEffects( + context, + articleContent.definition, + outcome, + { + model: articleContent, + pluginId: PLUGIN, + }, + ); + + expect(emitted.map(entry => entry.name)).toStrictEqual([ + "content.example.article.updated", + "content.example.article.delivery_slug_changed", + "content.example.article.delivery_redirect_created", + ]); + }); + + it("emits nothing for a corrected draft", async () => { + const article = await createArticle({ title: "Draft" }); + emitted.length = 0; + + const outcome = await editorial()?.update( + article.id, + { slug: "corrected" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + if (!outcome) throw new Error("update returned nothing"); + + await contentDeliveryEffects( + context, + articleContent.definition, + outcome.delivery, + { pluginId: PLUGIN }, + ); + + // The URL moved, but it had never been live - so a listener that warms a CDN + // or writes an edge redirect table hears about a redirect that does not exist. + expect( + emitted.filter(entry => + entry.name.includes("delivery_redirect_created"), + ), + ).toStrictEqual([]); + }); + }); + + // ------------------------------------------------------------------------- + // Search integration + // ------------------------------------------------------------------------- + + describe("search integration", () => { + it("indexes the current canonical URL and never a historical one", async () => { + const article = await publishArticle({ title: "Slug a" }); + + const outcome = await editorial()?.update( + article.id, + { slug: "slug-b" }, + { actor: ACTOR, expectedVersion: article.version }, + ); + if (!outcome) throw new Error("update returned nothing"); + + indexed.length = 0; + await contentEditorialEffects( + context, + articleContent.definition, + outcome, + { + model: articleContent, + pluginId: PLUGIN, + }, + ); + + // One document, pointing at the new address. A retired URL never becomes a + // second search result competing with the page it redirects to. + expect(indexed).toHaveLength(1); + expect(indexed[0].url).toBe("/articles/slug-b"); + expect( + indexed.filter(document => document.url === "/articles/slug-a"), + ).toStrictEqual([]); + }); + }); + + // ------------------------------------------------------------------------- + // Localization + // ------------------------------------------------------------------------- + + describe("localized delivery", () => { + it("reserves one address per published language", async () => { + const article = await publishLocalized({ pl: "Witaj swiecie" }); + + const rows = await localizedHistory(article.id); + + expect(rows).toHaveLength(2); + expect(rows.map(row => row.path).sort()).toStrictEqual([ + "/en/advanced-articles/hello-world", + "/pl/advanced-articles/witaj-swiecie", + ]); + // Each carries its own language, which is what keeps the two histories apart. + expect(new Set(rows.map(row => row.languageId)).size).toBe(2); + }); + + it("keeps an English slug change out of the Polish history", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + const before = await translationEditorial()?.update( + article.id, + "en", + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + expect(before?.delivery).toMatchObject({ + canonicalPath: "/en/advanced-articles/hello-there", + locale: "en", + previousPath: "/en/advanced-articles/hello-world", + redirectCreated: true, + }); + + const rows = await localizedHistory(article.id); + const polish = rows.filter(row => row.languageId === localeIds.pl); + + // Polish gained nothing and retired nothing. + expect(polish).toHaveLength(1); + expect(polish[0]).toMatchObject({ + path: "/pl/advanced-articles/witaj", + retired: false, + }); + }); + + it("redirects only inside the locale that moved", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + await translationEditorial()?.update( + article.id, + "en", + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + expect( + await advancedDelivery()?.resolvePath( + "/en/advanced-articles/hello-world", + ), + ).toStrictEqual({ + location: "/en/advanced-articles/hello-there", + status: 308, + type: "redirect", + }); + // The Polish URL is untouched and still canonical. + expect( + await advancedDelivery()?.resolvePath("/pl/advanced-articles/witaj"), + ).toMatchObject({ + canonicalPath: "/pl/advanced-articles/witaj", + type: "content", + }); + }); + + it("allows the same historical address in two locales", async () => { + const first = await publishLocalized({ title: "Shared" }); + await translationEditorial()?.update( + first.id, + "en", + { slug: "english-now" }, + { actor: ACTOR, expectedVersion: first.enVersion }, + ); + + const second = await publishLocalized({ title: "Second" }); + await translationEditorial()?.create( + second.id, + "pl", + { title: "Shared" }, + { actor: ACTOR }, + ); + const pl = await translationEditorial()?.publish(second.id, "pl", { + actor: ACTOR, + }); + await translationEditorial()?.update( + second.id, + "pl", + { slug: "polski-teraz" }, + { actor: ACTOR, expectedVersion: pl?.version ?? 0 }, + ); + + // `/en/.../shared` and `/pl/.../shared` are two URLs, so both may be retired + // by two different records. + const [rows] = await sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "core_content_slug_history" + WHERE "slug" = 'shared' + `; + expect(rows.count).toBe(2); + }); + + it("carries the alternates through resolveSlug, not only findById", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + const resolution = await advancedDelivery()?.resolveSlug("hello-world", { + locale: "en", + }); + + // The public resolve route is what a frontend calls, so an empty `hreflang` + // here would be invisible in the AdminCP and wrong on every page. It is only + // possible because the content type exposes `id` - which `delivery` requires + // of a localized content type for exactly this reason. + expect(resolution).toMatchObject({ + itemId: article.id, + type: "content", + }); + expect( + resolution?.type === "content" ? resolution.alternates : [], + ).toStrictEqual([ + { locale: "en", path: "/en/advanced-articles/hello-world" }, + { locale: "pl", path: "/pl/advanced-articles/witaj" }, + ]); + }); + + it("resolves the default locale when the caller names none", async () => { + const article = await publishLocalized(); + await translationEditorial()?.update( + article.id, + "en", + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + // The public read resolves `defaultLocale` internally when no locale is given, + // so the history lookup has to be about the same language - otherwise the live + // branch would search `en` while the redirect branch searched the shared rows + // and found nothing. + expect( + await advancedDelivery()?.resolveSlug("hello-world"), + ).toStrictEqual({ + location: "/en/advanced-articles/hello-there", + status: 308, + type: "redirect", + }); + }); + + it("lists only real published translations as alternates", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + expect(await advancedDelivery()?.alternates(article.id)).toStrictEqual([ + { locale: "en", path: "/en/advanced-articles/hello-world" }, + { locale: "pl", path: "/pl/advanced-articles/witaj" }, + ]); + }); + + it("never fabricates an alternate from a draft translation", async () => { + const article = await publishLocalized(); + // Created but deliberately not published. + await translationEditorial()?.create( + article.id, + "pl", + { title: "Wersja robocza" }, + { actor: ACTOR }, + ); + + expect(await advancedDelivery()?.alternates(article.id)).toStrictEqual([ + { locale: "en", path: "/en/advanced-articles/hello-world" }, + ]); + }); + + it("reports the served locale on a fallback read", async () => { + const article = await publishLocalized(); + + const metadata = await advancedDelivery()?.findById(article.id, { + locale: "pl", + }); + + // The Polish translation does not exist and the content type falls back to + // English, so the canonical URL is the English one - `/pl/...` would be a + // self-declared canonical that answers 404. + expect(metadata).toMatchObject({ + canonicalPath: "/en/advanced-articles/hello-world", + isFallback: true, + locale: "en", + requestedLocale: "pl", + }); + }); + + it("emits an x-default only when the default locale is published", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + const metadata = await advancedDelivery()?.findById(article.id, { + locale: "pl", + }); + + expect(metadata?.hreflang).toStrictEqual({ + languages: { + en: "/en/advanced-articles/hello-world", + pl: "/pl/advanced-articles/witaj", + }, + xDefault: "/en/advanced-articles/hello-world", + }); + }); + + it("stops a locale's redirects when its translation is unpublished", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + const moved = await translationEditorial()?.update( + article.id, + "en", + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + await translationEditorial()?.unpublish(article.id, "en", { + actor: ACTOR, + expectedVersion: moved?.version, + }); + + expect( + await advancedDelivery()?.resolvePath( + "/en/advanced-articles/hello-world", + ), + ).toStrictEqual({ type: "not_found" }); + // Polish is unaffected: one language going dark is not the record going dark. + expect( + await advancedDelivery()?.resolvePath("/pl/advanced-articles/witaj"), + ).toMatchObject({ type: "content" }); + }); + + it("emits the localized delivery event alongside the translation one", async () => { + const article = await publishLocalized(); + emitted.length = 0; + + const outcome = await translationEditorial()?.update( + article.id, + "en", + { slug: "hello-there" }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + if (!outcome) throw new Error("update returned nothing"); + + await contentTranslationEffects( + context, + advancedArticleContent.definition, + outcome, + { model: advancedArticleContent, pluginId: PLUGIN }, + ); + + const names = emitted.map(entry => entry.name); + expect(names).toContain( + "content.example.advanced-article.translation_updated", + ); + expect(names).toContain( + "content.example.advanced-article.delivery_slug_changed", + ); + expect( + emitted.find(entry => entry.name.includes("delivery_slug_changed")) + ?.payload, + ).toMatchObject({ locale: "en" }); + }); + }); + + // ------------------------------------------------------------------------- + // Localized sitemap and SEO + // ------------------------------------------------------------------------- + + describe("localized sitemap", () => { + it("lists one URL per published translation, per locale", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + const en = await advancedDelivery()?.sitemap({ locale: "en" }); + const pl = await advancedDelivery()?.sitemap({ locale: "pl" }); + + expect(en?.entries.map(entry => entry.path)).toStrictEqual([ + "/en/advanced-articles/hello-world", + ]); + expect(pl?.entries.map(entry => entry.path)).toStrictEqual([ + "/pl/advanced-articles/witaj", + ]); + // `example.advanced-article` withholds `id` from its public allowlist too, but + // a sitemap entry is built from the row rather than the projection - so the + // identifier is there, and it is what an `xhtml:link` group is keyed by. + expect(en?.entries[0].itemId).toBe(article.id); + }); + + it("omits a draft translation and never falls back for it", async () => { + const article = await publishLocalized(); + await translationEditorial()?.create( + article.id, + "pl", + { title: "Wersja robocza" }, + { actor: ACTOR }, + ); + + // No Polish entry at all: it has no URL of its own, and listing the English + // one under a Polish path would put the same content in the sitemap twice. + expect( + (await advancedDelivery()?.sitemap({ locale: "pl" }))?.entries, + ).toStrictEqual([]); + expect(article.id).toBeGreaterThan(0); + }); + + it("moves a translation's lastModified on an ordinary edit", async () => { + const article = await publishLocalized(); + const before = await advancedDelivery()?.sitemap({ locale: "en" }); + + const outcome = await translationEditorial()?.update( + article.id, + "en", + { seo: { description: "A new summary." } }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + const after = await advancedDelivery()?.sitemap({ locale: "en" }); + + // Same URL, later timestamp - and the outcome says so, which is what expires + // `sitemap:en` and nothing else. + expect(after?.entries[0].path).toBe(before?.entries[0].path); + expect(after?.entries[0].lastModified.getTime()).toBeGreaterThan( + before?.entries[0].lastModified.getTime() ?? 0, + ); + expect(outcome?.delivery?.sitemap).toStrictEqual({ + contentChanged: true, + indexChanged: false, + }); + }); + + it("reports an index change when a translation is published", async () => { + const localized = localizedService(); + if (!localized) throw new Error("no localized service"); + + const created = await localized.create({ + shared: {}, + translation: { title: "Fresh" }, + }); + await advancedEditorial()?.publish(created.row.id, { actor: ACTOR }); + + const outcome = await translationEditorial()?.publish( + created.row.id, + "en", + { actor: ACTOR }, + ); + + // A language gained a URL, so how many the index counts moved too. + expect(outcome?.delivery?.sitemap).toStrictEqual({ + contentChanged: true, + indexChanged: true, + }); + }); + + it("takes the later of the base and translation timestamps", async () => { + const article = await publishLocalized(); + + // A shared field moving changes what every language's page renders, even + // though no translation row was touched. + await sql` + UPDATE "example_advanced_articles" + SET "updatedAt" = now() + interval '1 hour' + WHERE "id" = ${article.id} + `; + const base = await advancedArticleContent + .service(context) + .findById(article.id); + const translation = await advancedArticleContent + .translationService?.(context) + .findByLocale(article.id, "en"); + + const page = await advancedDelivery()?.sitemap({ locale: "en" }); + + // The base row is now the later of the two, and that is the timestamp the + // sitemap carries - a shared field moving has to look like a change. + expect(base?.updatedAt.getTime()).toBeGreaterThan( + translation?.updatedAt.getTime() ?? 0, + ); + expect(page?.entries[0].lastModified.toISOString()).toBe( + base?.updatedAt.toISOString(), + ); + }); + + it("excludes a record whose noIndex flag is set", async () => { + const article = await publishLocalized(); + + await sql` + UPDATE "example_advanced_articles" + SET "syndicationNoIndex" = true + WHERE "id" = ${article.id} + `; + + expect( + (await advancedDelivery()?.sitemap({ locale: "en" }))?.entries, + ).toStrictEqual([]); + + // And the two agree: a record absent from the sitemap reports `index: false`. + const metadata = await advancedDelivery()?.findById(article.id, { + locale: "en", + }); + expect(metadata?.robots).toStrictEqual({ follow: true, index: false }); + }); + }); + + describe("localized SEO projection", () => { + it("reads each language's own SEO fields", async () => { + const article = await publishLocalized({ pl: "Witaj" }); + + await translationEditorial()?.update( + article.id, + "en", + { + seo: { description: "English summary", title: "English SEO" }, + }, + { actor: ACTOR, expectedVersion: article.enVersion }, + ); + + const en = await advancedDelivery()?.findById(article.id, { + locale: "en", + }); + const pl = await advancedDelivery()?.findById(article.id, { + locale: "pl", + }); + + expect(en?.seo).toStrictEqual({ + description: "English summary", + title: "English SEO", + }); + // Polish set none, so its title falls back to the localized `title` field - + // its own, never English's. + expect(pl?.seo).toStrictEqual({ description: null, title: "Witaj" }); + }); + + it("never leaks a private field into the metadata", async () => { + const article = await publishLocalized(); + + const metadata = await advancedDelivery()?.findById(article.id, { + locale: "en", + }); + + // `syndication.indexable` is a declared field that `publicApi.fields` does not + // expose, so it is not even fetched - the projection cannot reach it. + expect(JSON.stringify(metadata)).not.toContain("indexable"); + }); + }); + + // ------------------------------------------------------------------------- + // Stage 1-7 regression + // ------------------------------------------------------------------------- + + describe("a content type without delivery", () => { + it("has no delivery service and writes no history", async () => { + expect(categoryContent.deliveryService).toBeUndefined(); + expect(categoryContent.definition.delivery.enabled).toBe(false); + + const outcome = await categoryContent + .service(context) + .create({ name: "Guides" }); + + expect(outcome).toBeTruthy(); + const [rows] = await sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "core_content_slug_history" + WHERE "contentTypeId" = 'example.category' + `; + expect(rows.count).toBe(0); + }); + + it("reports no delivery outcome on its mutations", async () => { + const outcome = await categoryContent + .service(context) + .create({ name: "News two" }); + + expect(outcome).not.toHaveProperty("delivery"); + }); + }); + + describe("preview", () => { + it("registers no slug history and appears in no sitemap", async () => { + const article = await createArticle({ title: "Unpublished draft" }); + + // A preview reads a revision; it writes nothing. The record is still a draft, + // so it has no reservation and no sitemap line either. + const revisions = await editorial()?.revisions.list(article.id); + expect(revisions?.edges.length).toBeGreaterThan(0); + + expect(await historyRows(article.id)).toStrictEqual([]); + expect( + (await delivery()?.sitemap())?.entries.filter( + entry => entry.itemId === article.id, + ), + ).toStrictEqual([]); + expect(await delivery()?.resolveSlug("unpublished-draft")).toStrictEqual({ + type: "not_found", + }); + }); + }); +}); diff --git a/plugins/example/src/database/harness.ts b/plugins/example/src/database/harness.ts new file mode 100644 index 000000000..89cfa3e73 --- /dev/null +++ b/plugins/example/src/database/harness.ts @@ -0,0 +1,549 @@ +import type { SearchDocument } from "@vitnode/core/api/models/search"; +import type { Context } from "hono"; + +import { core_queue } from "@vitnode/core/database/queue"; +import { drizzle } from "drizzle-orm/postgres-js"; +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import postgres from "postgres"; + +import { CONFIG_PLUGIN, EXAMPLE_MIGRATIONS } from "@/const"; + +import { advancedArticleContent } from "./advanced-articles"; +import { articleContent } from "./articles"; +import { categoryContent } from "./categories"; +import { localizedArticleContent } from "./localized-articles"; + +/** + * The shared Postgres fixture for the Stage 7 hardening suites. + * + * Extracted rather than copied because there are now several suites that need + * the same thing: a schema built from the committed migrations, the core tables + * the Content Engine writes to, and a request context whose event transport, + * search engine, queue and logger all *record* instead of doing. + * + * Every suite runs against the same database and every one of them drops the + * schema in its `beforeAll`, which is why `vitest.config.ts` sets + * `fileParallelism: false`. That is a deliberate trade: one shared, real + * database beats several mocked ones, and a suite that cannot see the + * constraints is not testing the thing it claims to. + */ + +export const DATABASE_TEST_URL = process.env.DATABASE_TEST_URL; + +const databaseName = (() => { + if (!DATABASE_TEST_URL) return ""; + try { + return new URL(DATABASE_TEST_URL).pathname.replace(/^\//, ""); + } catch { + return ""; + } +})(); + +const here = dirname(fileURLToPath(import.meta.url)); + +const migrationSql = (files: readonly string[]): string => + files + .map(file => + readFileSync( + resolve(here, "../../../../apps/docs/migrations", file), + "utf8", + ), + ) + .join("\n--> statement-breakpoint\n"); + +/** + * The core tables the engine writes to, stubbed to the columns it touches. + * + * Core's own migration history is not replayed: one of its migrations builds a + * full-text column from per-language text-search configurations a stock + * Postgres image does not ship, and none of that has anything to do with the + * Content Engine. Pulling it in would make every unrelated core change a reason + * for these suites to break. + */ +const CORE_STUBS = ` + CREATE TABLE "core_users" ( + "id" serial PRIMARY KEY NOT NULL, + "name" varchar(255) NOT NULL + ); + CREATE TABLE "core_queue" ( + "id" serial PRIMARY KEY NOT NULL, + "pluginId" varchar(255) NOT NULL, + "name" varchar(100) NOT NULL, + "queue" varchar(100) DEFAULT 'default' NOT NULL, + "status" varchar(20) DEFAULT 'pending' NOT NULL, + "payload" jsonb DEFAULT '{}'::jsonb NOT NULL, + "priority" integer DEFAULT 0 NOT NULL, + "attempts" integer DEFAULT 0 NOT NULL, + "maxAttempts" integer DEFAULT 3 NOT NULL, + "availableAt" timestamp DEFAULT now() NOT NULL, + "reservedAt" timestamp, + "lastError" text, + "createdAt" timestamp DEFAULT now() NOT NULL, + "updatedAt" timestamp DEFAULT now() NOT NULL, + "completedAt" timestamp + ); + CREATE TABLE "core_languages" ( + "id" serial PRIMARY KEY NOT NULL, + "code" varchar(32) NOT NULL, + "name" varchar(255) NOT NULL, + "default" boolean DEFAULT false NOT NULL, + "protected" boolean DEFAULT false NOT NULL, + CONSTRAINT "core_languages_code_unique" UNIQUE("code") + ); + CREATE TABLE "core_search_index" ( + "id" serial PRIMARY KEY NOT NULL, + "pluginId" varchar(255) NOT NULL, + "itemType" varchar(100) NOT NULL, + "itemId" integer NOT NULL, + "languageCode" varchar(32) DEFAULT '' NOT NULL, + "authorId" integer, + "title" text NOT NULL, + "content" text NOT NULL, + "containerType" varchar(100), + "containerId" integer, + "url" text, + "isPublic" boolean DEFAULT true NOT NULL, + "metadata" jsonb DEFAULT '{}'::jsonb NOT NULL, + "createdAt" timestamp NOT NULL, + "updatedAt" timestamp, + "indexedAt" timestamp DEFAULT now() NOT NULL, + CONSTRAINT "core_search_index_item_key" + UNIQUE("itemType", "itemId", "languageCode") + ); +`; + +/** Who the editorial suites act as. `userId: null` - see `postgres.test.ts`. */ +export const ACTOR = { type: "staff" as const, userId: null }; + +export interface RecordedSearchDelete { + itemId: number; + itemType: string; + locale?: string; +} + +export interface RecordedEvent { + name: string; + payload: unknown; +} + +/** A listener that did not receive an event, in the shape `emit` reports. */ +export interface RecordedEventFailure { + error: string; + listener: string; + module: string; + pluginId: string; +} + +export interface ContentTestHarness { + /** Injected failures, flipped per test. */ + readonly behaviour: { + /** Listeners `emit` should report as having failed. */ + eventFailures: RecordedEventFailure[]; + /** When set, the provider's `count` throws it. */ + providerCountError: Error | null; + /** + * What the provider's own diagnostics answer. + * + * `"canonical"` is the bundled Postgres provider - its store *is* + * `core_search_index`, so it is verified without a second query. + * `"unsupported"` is a provider with no `count`, which has to be reported as + * unverified rather than healthy. + * + * The object form is a mirroring provider that can be counted: `byLocale` + * answers a filtered count and `total` answers an unfiltered one. They are + * separate on purpose - a ghost document lives in a locale nothing + * enumerates, so the only way to simulate one is a total that exceeds the + * locales anybody thinks to ask about. + */ + providerCounts: + | "canonical" + | "unsupported" + | { byLocale: Map; total: number }; + providerName: string; + /** + * Web origins the revalidation bridge should post to. + * + * Empty by default, which is what an API with no `NEXT_PUBLIC_WEB_URL` sees + * - and what makes `attempted: 0` mean "there was nothing to tell" rather + * than "nobody answered". + */ + revalidateOrigins: string[]; + /** When set, every `search.index`/`search.delete` throws it. */ + searchError: Error | null; + }; + context: Context; + /** + * A third connection that records every statement it issues. + * + * Query *counting* is the only way to state an N+1 guard as an invariant + * rather than as a hope: "one page costs a bounded number of round trips + * whatever the page size" is a fact about the SQL, and the SQL is the only + * place to observe it. Separate from the main handle so an ordinary test pays + * nothing for the instrumentation. + */ + counted: { + context: Context; + db: ReturnType; + /** Every statement since the last `reset`, in order. */ + queries: string[]; + reset: () => void; + }; + db: ReturnType; + /** Every `search.delete` the engine asked for, in order. */ + deleted: RecordedSearchDelete[]; + /** Every event the engine emitted, in order. */ + emitted: RecordedEvent[]; + end: () => Promise; + /** Every document the engine handed the search engine, in order. */ + indexed: SearchDocument[]; + /** Every line written through `c.get("log").error`. */ + logs: string[]; + /** Clears the recorders and the injected failures. */ + reset: () => void; + /** + * A second connection with its own context. + * + * The main client is `max: 1`, which serialises everything through one + * backend - fine for optimistic locking, useless for row locks, because a + * statement waiting on `FOR UPDATE` would be waiting on itself. + */ + rivalContext: Context; + rivalDb: ReturnType; + /** The Postgres major, for the assertions whose SQLSTATE moved in 18. */ + serverMajor: number; + sql: ReturnType; +} + +/** + * Builds the schema and returns everything a suite needs to drive it. + * + * **Wipes the database it points at**, so the URL has to name one with "test" + * in it. That check is not politeness: the suite runs `DROP SCHEMA public + * CASCADE`. + */ +export const createContentTestHarness = + async (): Promise => { + if (!DATABASE_TEST_URL) { + throw new Error("DATABASE_TEST_URL is not set."); + } + if (!/test/i.test(databaseName)) { + throw new Error( + `DATABASE_TEST_URL points at "${databaseName || DATABASE_TEST_URL}". This suite wipes the database it runs against, so its name must contain "test".`, + ); + } + + const sql = postgres(DATABASE_TEST_URL, { + max: 1, + onnotice: () => undefined, + }); + + const [{ version }] = await sql<{ version: number }[]>` + SELECT current_setting('server_version_num')::int AS version + `; + const serverMajor = Math.floor(version / 10_000); + + await sql.unsafe(` + DROP SCHEMA IF EXISTS public CASCADE; + CREATE SCHEMA public; + `); + await sql.unsafe(CORE_STUBS); + await sql` + INSERT INTO "core_languages" ("code", "name", "default") VALUES + ('en', 'English', true), + ('pl', 'Polski', false), + ('de', 'Deutsch', false) + `; + + for (const statement of migrationSql(EXAMPLE_MIGRATIONS).split( + "--> statement-breakpoint", + )) { + const trimmed = statement.trim(); + if (trimmed) await sql.unsafe(trimmed); + } + + const db = drizzle(sql, { casing: "camelCase" }); + const rival = postgres(DATABASE_TEST_URL, { + max: 1, + onnotice: () => undefined, + }); + const rivalDb = drizzle(rival, { casing: "camelCase" }); + + const queries: string[] = []; + const countedSql = postgres(DATABASE_TEST_URL, { + debug: (_connection, query) => { + queries.push(query); + }, + max: 1, + onnotice: () => undefined, + }); + const countedDb = drizzle(countedSql, { casing: "camelCase" }); + + const indexed: SearchDocument[] = []; + const deleted: RecordedSearchDelete[] = []; + const emitted: RecordedEvent[] = []; + const logs: string[] = []; + const behaviour: ContentTestHarness["behaviour"] = { + eventFailures: [], + providerCountError: null, + providerCounts: "canonical", + providerName: "postgres", + revalidateOrigins: [], + searchError: null, + }; + + /** + * Everything the Content Engine reads off a request context. + * + * The queue stands in for `QueueModel.dispatch`, writing the row it would + * and honouring the `tx` it is handed - which is the property the schedule + * tests are about. The search engine and the event transport record rather + * than deliver, and both can be made to fail on demand: that is what makes + * "a committed write survives a downstream outage" testable at all. + */ + const buildContext = (handle: typeof db): Context => + ({ + get: (key: string) => { + if (key === "db") return handle; + if (key === "search") { + return { + countDocuments: async ({ + languageCode, + }: { + itemType: string; + languageCode?: string; + }) => { + if (behaviour.providerCountError) { + throw behaviour.providerCountError; + } + if (behaviour.providerCounts === "unsupported") { + return await Promise.resolve(null); + } + if (behaviour.providerCounts === "canonical") { + return await Promise.resolve(0); + } + + // No language means every language - which is what makes a + // ghost in an unenumerated locale visible at all. + return await Promise.resolve( + languageCode === undefined + ? behaviour.providerCounts.total + : (behaviour.providerCounts.byLocale.get(languageCode) ?? + 0), + ); + }, + isCanonicalStorage: () => + behaviour.providerCounts === "canonical", + name: () => behaviour.providerName, + delete: async ( + itemType: string, + itemId: number, + locale?: string, + ) => { + if (behaviour.searchError) throw behaviour.searchError; + deleted.push({ itemId, itemType, locale }); + + return await Promise.resolve(); + }, + index: async (document: SearchDocument) => { + if (behaviour.searchError) throw behaviour.searchError; + indexed.push(document); + + return await Promise.resolve(); + }, + }; + } + if (key === "events") { + return { + emit: async (name: string, payload: unknown) => { + emitted.push({ name, payload }); + + return await Promise.resolve({ + delivered: behaviour.eventFailures.length === 0 ? 1 : 0, + eventId: `event-${emitted.length}`, + failures: [...behaviour.eventFailures], + status: "delivered" as const, + }); + }, + }; + } + if (key === "log") { + return { + error: async (message: string) => { + logs.push(message); + + return await Promise.resolve(); + }, + }; + } + if (key === "core") { + return { + // What the revalidation bridge posts to, and the secret it signs + // with. Both live on the context in a real install too. + contentRevalidateOrigins: behaviour.revalidateOrigins, + cronSecret: "content-engine-test-secret", + hasCronAdapter: false, + contentModels: [ + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + { model: categoryContent, pluginId: CONFIG_PLUGIN.pluginId }, + { + model: localizedArticleContent, + pluginId: CONFIG_PLUGIN.pluginId, + }, + { + model: advancedArticleContent, + pluginId: CONFIG_PLUGIN.pluginId, + }, + ], + // Which locales this app *serves*. `core_languages` is the + // registry of the ones that exist; a locale listed here with + // `enabled: false` is a deliberate switch-off. + i18n: { + locales: [ + { code: "en", name: "English" }, + { code: "pl", name: "Polski" }, + { code: "de", enabled: false, name: "Deutsch" }, + ], + }, + searchIndexers: [], + }; + } + if (key === "queue") { + return { + dispatch: async ({ + availableAt, + name, + payload, + pluginId, + tx, + }: { + availableAt?: Date; + name: string; + payload?: Record; + pluginId?: string; + tx?: typeof db; + }) => { + const [queued] = await (tx ?? handle) + .insert(core_queue) + .values({ + availableAt: availableAt ?? new Date(), + name, + payload: payload ?? {}, + pluginId: pluginId ?? "@vitnode/core", + }) + .returning({ id: core_queue.id }); + + return queued; + }, + }; + } + + return undefined; + }, + }) as unknown as Context; + + return { + behaviour, + context: buildContext(db), + counted: { + context: buildContext(countedDb), + db: countedDb, + queries, + reset: () => { + queries.length = 0; + }, + }, + db, + deleted, + emitted, + end: async () => { + await sql.end(); + await rival.end(); + await countedSql.end(); + }, + indexed, + logs, + reset: () => { + indexed.length = 0; + deleted.length = 0; + emitted.length = 0; + logs.length = 0; + behaviour.eventFailures = []; + behaviour.providerCountError = null; + behaviour.providerCounts = "canonical"; + behaviour.providerName = "postgres"; + behaviour.revalidateOrigins = []; + behaviour.searchError = null; + }, + rivalContext: buildContext(rivalDb), + rivalDb, + serverMajor, + sql, + }; + }; + +/** + * Empties every table the suites write to, in dependency order. + * + * `DELETE` rather than `TRUNCATE ... CASCADE`: the cascade would silently prove + * nothing about the foreign keys, and several suites are specifically about what + * the database refuses. + */ +export const clearContentTables = async ( + sql: ReturnType, +): Promise => { + await sql`DELETE FROM "core_search_index"`; + await sql`DELETE FROM "core_content_schedules"`; + await sql`DELETE FROM "core_content_revisions"`; + await sql`DELETE FROM "core_queue"`; + await sql`DELETE FROM "example_advanced_articles"`; + await sql`DELETE FROM "example_localized_articles"`; + await sql`DELETE FROM "example_articles"`; + await sql`DELETE FROM "example_categories"`; + await sql`DELETE FROM "core_users"`; +}; + +/** The SQLSTATE a failing call reported, or `undefined` if it succeeded. */ +export const pgErrorCode = async ( + run: () => Promise, +): Promise => { + try { + await run(); + } catch (error) { + const cause = (error as { cause?: { code?: string } }).cause; + + return cause?.code ?? (error as { code?: string }).code; + } + + return undefined; +}; + +/** + * Runs two writers at once and reports what each one did. + * + * `Promise.allSettled` rather than `Promise.all`, because the whole point is + * that one of them is expected to lose - and `all` would reject before the + * winner's result could be inspected. + */ +export const race = async ( + first: () => Promise, + second: () => Promise, +): Promise< + [PromiseSettledResult>, PromiseSettledResult>] +> => { + const results = await Promise.allSettled([first(), second()]); + + return results; +}; + +/** How many of a race's two sides succeeded. */ +export const fulfilledCount = ( + results: readonly PromiseSettledResult[], +): number => results.filter(entry => entry.status === "fulfilled").length; + +/** The reasons the losing sides gave. */ +export const reasons = ( + results: readonly PromiseSettledResult[], +): unknown[] => + results.flatMap(entry => (entry.status === "rejected" ? [entry.reason] : [])); diff --git a/plugins/example/src/database/integrity-postgres.test.ts b/plugins/example/src/database/integrity-postgres.test.ts new file mode 100644 index 000000000..11b1b6a51 --- /dev/null +++ b/plugins/example/src/database/integrity-postgres.test.ts @@ -0,0 +1,733 @@ +import type { Context } from "hono"; + +import { + ContentLanguageError, + ContentRevisionNotRestorable, +} from "@vitnode/core/content"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { CONFIG_PLUGIN } from "@/const"; + +import type { ContentTestHarness } from "./harness"; + +import { advancedArticleContent } from "./advanced-articles"; +import { articleContent } from "./articles"; +import { + ACTOR, + clearContentTables, + createContentTestHarness, + DATABASE_TEST_URL, + pgErrorCode, +} from "./harness"; +import { localizedArticleContent } from "./localized-articles"; + +/** + * What the **database** guarantees, and what the engine does when a definition + * has moved on since a revision was written. + * + * Two halves that look unrelated and are not: both are about a record outliving + * the assumptions it was written under. A delete has to take exactly the rows + * that belong to the record and refuse exactly the ones that belong to somebody + * else; a restore has to apply a snapshot written against an older shape, or + * refuse it whole. + * + * Wherever Postgres can enforce something, the assertion is against Postgres + * rather than against the service - a check in application code is one a direct + * `DELETE` walks straight past. + */ + +let h: ContentTestHarness; +let categoryId = 0; +let seq = 0; + +const editorial = (on: Context) => { + const build = articleContent.editorialService; + if (!build) throw new Error("example.article has no editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const advanced = (on: Context) => { + const build = advancedArticleContent.editorialService; + if (!build) throw new Error("no advanced editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const localizedService = (on: Context) => { + const build = localizedArticleContent.localizedService; + if (!build) throw new Error("no localized service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const translationEditorial = (on: Context) => { + const build = localizedArticleContent.translationEditorialService; + if (!build) throw new Error("no translation editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const advancedTranslations = (on: Context) => { + const build = advancedArticleContent.translationEditorialService; + if (!build) throw new Error("no advanced translation editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const article = async () => { + seq += 1; + const outcome = await editorial(h.context).create( + { + category: categoryId, + code: `integrity-${seq}`, + title: `Integrity subject ${seq}`, + }, + { actor: ACTOR }, + ); + + return { id: outcome.row.id, version: outcome.version }; +}; + +const countOf = async (table: string): Promise => { + const [row] = await h.sql.unsafe( + `SELECT count(*)::int AS count FROM "${table}"`, + ); + + return Number(row.count); +}; + +describe.skipIf(!DATABASE_TEST_URL)("Content Engine integrity", () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + }); + + beforeEach(async () => { + await clearContentTables(h.sql); + h.reset(); + + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Integrity') + RETURNING "id" + `; + categoryId = category.id; + }); + + // ------------------------------------------------------------------------- + // Delete integrity + // ------------------------------------------------------------------------- + + describe("deleting a record", () => { + it("takes its translations with it", async () => { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "Body", title: "Cascade Subject" }, + }, + { actor: ACTOR }, + ); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + expect(await countOf("example_localized_articles_translations")).toBe(2); + + await h.sql` + DELETE FROM "example_localized_articles" WHERE "id" = ${row.id} + `; + + expect(await countOf("example_localized_articles_translations")).toBe(0); + }); + + it("takes its junction and child rows with it", async () => { + const created = await advanced(h.context).create( + { categories: [categoryId] }, + { actor: ACTOR }, + ); + await advanced(h.context).repeatable.faq.set( + created.row.id, + [{ answer: "An answer", question: "A question" }], + { actor: ACTOR, expectedVersion: created.version }, + ); + expect(await countOf("example_advanced_articles_categories")).toBe(1); + expect(await countOf("example_advanced_articles_faq")).toBe(1); + + await h.sql` + DELETE FROM "example_advanced_articles" WHERE "id" = ${created.row.id} + `; + + expect(await countOf("example_advanced_articles_categories")).toBe(0); + expect(await countOf("example_advanced_articles_faq")).toBe(0); + }); + + it("keeps its history, which outlives it deliberately", async () => { + const created = await article(); + + await editorial(h.context).delete(created.id, { + actor: ACTOR, + expectedVersion: created.version, + }); + + const revisions = await h.sql<{ operation: string }[]>` + SELECT "operation" FROM "core_content_revisions" + WHERE "contentTypeId" = 'example.article' AND "itemId" = ${created.id} + ORDER BY "version" + `; + // "Who removed this, and what did it say" is only answerable if the + // history is not a foreign key to the row it describes. + expect(revisions.map(row => row.operation)).toEqual(["create", "delete"]); + }); + + it("refuses to remove a category that content still points at", async () => { + await article(); + + const code = await pgErrorCode( + async () => + await h.sql`DELETE FROM "example_categories" WHERE "id" = ${categoryId}`, + ); + + // `onDelete: "restrict"`, enforced by Postgres rather than by a check in + // service code that a direct `DELETE` would walk past. + expect(code).toBe(h.serverMajor >= 18 ? "23001" : "23503"); + }); + + it("refuses to remove a category a to-many relation still points at", async () => { + const created = await advanced(h.context).create( + { categories: [categoryId] }, + { actor: ACTOR }, + ); + expect(created.row.id).toBeGreaterThan(0); + + const code = await pgErrorCode( + async () => + await h.sql`DELETE FROM "example_categories" WHERE "id" = ${categoryId}`, + ); + + expect(code).toBe(h.serverMajor >= 18 ? "23001" : "23503"); + }); + + it("drops a self-relation's reference when its target goes", async () => { + // `relatedArticles` is `onDelete: "cascade"`: forgetting the reference is + // the honest analogue of nulling a column, because a junction row has no + // column to null. + const source = await advanced(h.context).create({}, { actor: ACTOR }); + const target = await advanced(h.context).create({}, { actor: ACTOR }); + await advanced(h.context).relations.relatedArticles.set( + source.row.id, + [target.row.id], + { actor: ACTOR, expectedVersion: source.version }, + ); + expect(await countOf("example_advanced_articles_related_articles")).toBe( + 1, + ); + + await h.sql` + DELETE FROM "example_advanced_articles" WHERE "id" = ${target.row.id} + `; + + expect(await countOf("example_advanced_articles_related_articles")).toBe( + 0, + ); + // And the source record is still there: a cascade on the reference is not + // a cascade on the record that held it. + const [row] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "example_advanced_articles" WHERE "id" = ${source.row.id} + `; + expect(row).toBeDefined(); + }); + + it("leaves no orphaned junction row behind, in either direction", async () => { + const created = await advanced(h.context).create( + { categories: [categoryId] }, + { actor: ACTOR }, + ); + await advanced(h.context).relations.relatedArticles.set( + created.row.id, + [created.row.id], + { actor: ACTOR, expectedVersion: created.version }, + ); + + await h.sql` + DELETE FROM "example_advanced_articles" WHERE "id" = ${created.row.id} + `; + + const orphans = await h.sql<{ count: number }[]>` + SELECT ( + (SELECT count(*) FROM "example_advanced_articles_categories" j + LEFT JOIN "example_advanced_articles" a ON a."id" = j."itemId" + WHERE a."id" IS NULL) + + + (SELECT count(*) FROM "example_advanced_articles_related_articles" r + LEFT JOIN "example_advanced_articles" a ON a."id" = r."itemId" + WHERE a."id" IS NULL) + )::int AS count + `; + expect(orphans[0].count).toBe(0); + }); + + it("clears a user reference rather than removing the record", async () => { + // `onDelete: "set null"` on a nullable user field: an article does not + // stop existing because its author's account did. + const [user] = await h.sql<{ id: number }[]>` + INSERT INTO "core_users" ("name") VALUES ('Ada') RETURNING "id" + `; + seq += 1; + const created = await editorial(h.context).create( + { + author: user.id, + category: categoryId, + code: `authored-${seq}`, + title: "Authored subject", + }, + { actor: ACTOR }, + ); + + await h.sql`DELETE FROM "core_users" WHERE "id" = ${user.id}`; + + const [row] = await h.sql<{ author: null | number }[]>` + SELECT "author" FROM "example_articles" WHERE "id" = ${created.row.id} + `; + expect(row.author).toBeNull(); + }); + }); + + // ------------------------------------------------------------------------- + // Schema evolution + // ------------------------------------------------------------------------- + + /** + * A revision written under an older definition, applied to today's. + * + * The snapshots are written straight into `core_content_revisions`, which is + * the honest way to model this: a real installation's history is full of rows + * written by code that no longer exists, and there is no way to get one except + * by having been there. + */ + describe("restoring a revision written under an older definition", () => { + /** + * A revision row in the envelope the engine really writes. + * + * `fields` is the whole point: a snapshot is a versioned envelope around + * the declared field values, and `projectRevisionSnapshot` reads that half + * rather than the row it came from. Writing a flat object here would test a + * shape no revision has ever had. + */ + const writeRevision = async ( + itemId: number, + version: number, + fields: Record, + contentTypeId = "example.article", + ) => { + const snapshot = { + contentTypeId, + createdAt: new Date(0).toISOString(), + fields, + id: itemId, + schemaVersion: 1, + updatedAt: new Date(0).toISOString(), + version, + }; + + const [row] = await h.sql<{ id: number }[]>` + INSERT INTO "core_content_revisions" + ("pluginId", "contentTypeId", "itemId", "version", "operation", + "actorType", "snapshot") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${contentTypeId}, ${itemId}, ${version}, + 'update', 'staff', ${JSON.stringify(snapshot)}::jsonb + ) + RETURNING "id" + `; + + return row.id; + }; + + it("ignores a field the content type has since dropped", async () => { + const created = await article(); + const revisionId = await writeRevision(created.id, 900, { + // `subtitle` was a field once. It is not one now, and a restore has to + // drop it rather than hand it to a strict schema that will refuse it. + subtitle: "A field that no longer exists", + title: "Restored from an older shape", + }); + + const outcome = await editorial(h.context).restore( + created.id, + revisionId, + { actor: ACTOR, expectedVersion: created.version }, + ); + + expect(outcome?.changed).toBe(true); + const [row] = await h.sql<{ title: string }[]>` + SELECT "title" FROM "example_articles" WHERE "id" = ${created.id} + `; + expect(row.title).toBe("Restored from an older shape"); + }); + + it("leaves a field added since the snapshot exactly as it stands", async () => { + // The update schema is partial, so a field the snapshot never carried is + // simply not written - which is the only answer that does not invent a + // value nobody chose. + const created = await article(); + await editorial(h.context).update( + created.id, + { excerpt: "Written after the snapshot" }, + { actor: ACTOR, expectedVersion: created.version }, + ); + const revisionId = await writeRevision(created.id, 901, { + title: "Older still", + }); + + await editorial(h.context).restore(created.id, revisionId, { + actor: ACTOR, + expectedVersion: created.version + 1, + }); + + const [row] = await h.sql<{ excerpt: null | string; title: string }[]>` + SELECT "title", "excerpt" FROM "example_articles" WHERE "id" = ${created.id} + `; + expect(row.title).toBe("Older still"); + expect(row.excerpt).toBe("Written after the snapshot"); + }); + + it("refuses a snapshot whose value no longer validates, and writes nothing", async () => { + const created = await article(); + const before = await h.sql<{ title: string; version: number }[]>` + SELECT "title", "version" FROM "example_articles" WHERE "id" = ${created.id} + `; + const revisionId = await writeRevision(created.id, 902, { + // `title` has a three-character minimum today. It did not always. + title: "No", + }); + + await expect( + editorial(h.context).restore(created.id, revisionId, { + actor: ACTOR, + expectedVersion: created.version, + }), + ).rejects.toBeInstanceOf(ContentRevisionNotRestorable); + + // All or nothing: the record is byte-identical to what it was. + const after = await h.sql<{ title: string; version: number }[]>` + SELECT "title", "version" FROM "example_articles" WHERE "id" = ${created.id} + `; + expect(after).toEqual(before); + }); + + it("names the field, and nothing internal, when it refuses", async () => { + const created = await article(); + const revisionId = await writeRevision(created.id, 903, { + title: "No", + }); + + try { + await editorial(h.context).restore(created.id, revisionId, { + actor: ACTOR, + expectedVersion: created.version, + }); + throw new Error("Expected the restore to be refused."); + } catch (error) { + expect(error).toBeInstanceOf(ContentRevisionNotRestorable); + const refusal = error as ContentRevisionNotRestorable; + expect(refusal.fields).toEqual(["title"]); + // Never a Zod issue tree: it names internal paths, and the route's + // OpenAPI schema already describes the contract. + expect(JSON.stringify(refusal.fields)).not.toContain("_zod"); + } + }); + + it("refuses when a relation target in the snapshot is gone", async () => { + const created = await advanced(h.context).create( + { categories: [categoryId] }, + { actor: ACTOR }, + ); + const [spare] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Doomed') RETURNING "id" + `; + const revisionId = await writeRevision( + created.row.id, + 904, + { categories: [spare.id] }, + "example.advanced-article", + ); + await h.sql`DELETE FROM "example_categories" WHERE "id" = ${spare.id}`; + + await expect( + advanced(h.context).restore(created.row.id, revisionId, { + actor: ACTOR, + expectedVersion: created.version, + }), + ).rejects.toBeInstanceOf(ContentRevisionNotRestorable); + + // Nothing partial: the relation it *could* have restored is untouched. + const rows = await h.sql<{ relatedItemId: number }[]>` + SELECT "relatedItemId" FROM "example_advanced_articles_categories" + WHERE "itemId" = ${created.row.id} + `; + expect(rows.map(row => row.relatedItemId)).toEqual([categoryId]); + }); + + it("recreates a repeatable child whose identifier is gone", async () => { + // The other rule, and the reason the two kinds differ: a child's values + // are all in the snapshot, so recreating it loses nothing but its + // identifier. A relation target's values were never there to begin with. + const created = await advanced(h.context).create({}, { actor: ACTOR }); + const seeded = await advanced(h.context).repeatable.faq.set( + created.row.id, + [{ answer: "The answer", question: "The question" }], + { actor: ACTOR, expectedVersion: created.version }, + ); + const [child] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "example_advanced_articles_faq" + WHERE "itemId" = ${created.row.id} + `; + + await advanced(h.context).repeatable.faq.delete( + created.row.id, + child.id, + { actor: ACTOR, expectedVersion: seeded?.version ?? created.version }, + ); + + const revisionId = await writeRevision( + created.row.id, + 905, + { + faq: [ + { answer: "The answer", id: child.id, question: "The question" }, + ], + }, + "example.advanced-article", + ); + + const [current] = await h.sql<{ version: number }[]>` + SELECT "version" FROM "example_advanced_articles" + WHERE "id" = ${created.row.id} + `; + const outcome = await advanced(h.context).restore( + created.row.id, + revisionId, + { actor: ACTOR, expectedVersion: current.version }, + ); + + expect(outcome?.changed).toBe(true); + const rows = await h.sql<{ id: number; question: string }[]>` + SELECT "id", "question" FROM "example_advanced_articles_faq" + WHERE "itemId" = ${created.row.id} + `; + expect(rows).toHaveLength(1); + expect(rows[0].question).toBe("The question"); + // A new identifier, because the old row is gone. The values came back; + // the identity did not, and could not. + expect(rows[0].id).not.toBe(child.id); + }); + + it("keeps the restored-from revision untouched", async () => { + const created = await article(); + const revisionId = await writeRevision(created.id, 906, { + title: "Immutable source", + }); + const [before] = await h.sql<{ snapshot: unknown; version: number }[]>` + SELECT "snapshot", "version" FROM "core_content_revisions" + WHERE "id" = ${revisionId} + `; + + await editorial(h.context).restore(created.id, revisionId, { + actor: ACTOR, + expectedVersion: created.version, + }); + + const [after] = await h.sql<{ snapshot: unknown; version: number }[]>` + SELECT "snapshot", "version" FROM "core_content_revisions" + WHERE "id" = ${revisionId} + `; + expect(after).toEqual(before); + }); + + it("moves the record forward rather than backward", async () => { + const created = await article(); + const revisionId = await writeRevision(created.id, 907, { + title: "Rolled forward", + }); + + const outcome = await editorial(h.context).restore( + created.id, + revisionId, + { actor: ACTOR, expectedVersion: created.version }, + ); + + // A restore is an edit, not a rewind: the version increases and the + // history gains an entry rather than losing one. + expect(outcome?.version).toBe(created.version + 1); + expect(outcome?.restoredFromRevisionId).toBe(revisionId); + }); + }); + + // ------------------------------------------------------------------------- + // Disabled locales + // ------------------------------------------------------------------------- + + /** + * The Stage 5 policy, unchanged and now pinned on both kinds of localized + * content type: + * + * | create | update | restore | publish | unpublish | delete | read | + * | ------ | ------ | ------- | ------- | --------- | ------ | ---- | + * | refuse | refuse | refuse | refuse | allow | allow | allow| + * + * The asymmetry is the point. Switching a language off must stop new content + * going into it, and must **not** trap the content that is already there: + * taking a page down and deleting it are exactly the operations an + * administrator needs after switching the language off. + */ + describe("a locale the installation has switched off", () => { + const DISABLED = "de"; + + /** A record with a `de` translation already written, before the switch-off. */ + const withGermanTranslation = async () => { + const { row } = await localizedService(h.context).create( + { shared: {}, translation: { body: "Body", title: "Locale Policy" } }, + { actor: ACTOR }, + ); + const [german] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "core_languages" WHERE "code" = ${DISABLED} + `; + await h.sql` + INSERT INTO "example_localized_articles_translations" + ("itemId", "languageId", "title", "slug", "body", "version", "status") + VALUES (${row.id}, ${german.id}, 'Deutsch', 'deutsch', 'Körper', 1, 'published') + `; + + return { itemId: row.id, languageId: german.id }; + }; + + const isDisabled = (error: unknown): boolean => + error instanceof ContentLanguageError && error.reason === "disabled"; + + it("refuses a create", async () => { + const { row } = await localizedService(h.context).create( + { shared: {}, translation: { body: "Body", title: "Refused Create" } }, + { actor: ACTOR }, + ); + + await expect( + translationEditorial(h.context).create( + row.id, + DISABLED, + { body: "Körper", title: "Deutsch" }, + { actor: ACTOR }, + ), + ).rejects.toSatisfy(isDisabled); + }); + + it("refuses an update", async () => { + const { itemId } = await withGermanTranslation(); + + await expect( + translationEditorial(h.context).update( + itemId, + DISABLED, + { title: "Deutsch Neu" }, + { actor: ACTOR, expectedVersion: 1 }, + ), + ).rejects.toSatisfy(isDisabled); + }); + + it("refuses a publish", async () => { + const { itemId } = await withGermanTranslation(); + + await expect( + translationEditorial(h.context).publish(itemId, DISABLED, { + actor: ACTOR, + }), + ).rejects.toSatisfy(isDisabled); + }); + + it("refuses a restore", async () => { + const { itemId, languageId } = await withGermanTranslation(); + const [revision] = await h.sql<{ id: number }[]>` + INSERT INTO "core_content_revisions" + ("pluginId", "contentTypeId", "itemId", "languageId", "version", + "operation", "actorType", "snapshot") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, 'example.localized-article', ${itemId}, + ${languageId}, 1, 'create', 'staff', + ${JSON.stringify({ body: "Alt", slug: "alt", title: "Alt", version: 1 })}::jsonb + ) + RETURNING "id" + `; + + await expect( + translationEditorial(h.context).restore(itemId, DISABLED, revision.id, { + actor: ACTOR, + expectedVersion: 1, + }), + ).rejects.toSatisfy(isDisabled); + }); + + it("still allows an unpublish, which is how a page comes down", async () => { + const { itemId } = await withGermanTranslation(); + + const outcome = await translationEditorial(h.context).unpublish( + itemId, + DISABLED, + { actor: ACTOR }, + ); + + expect(outcome?.changed).toBe(true); + }); + + it("still allows a delete", async () => { + const { itemId } = await withGermanTranslation(); + + const outcome = await translationEditorial(h.context).delete( + itemId, + DISABLED, + { actor: ACTOR, expectedVersion: 1 }, + ); + + expect(outcome?.changed).toBe(true); + const rows = await h.sql<{ count: number }[]>` + SELECT count(*)::int AS count + FROM "example_localized_articles_translations" + WHERE "itemId" = ${itemId} + `; + expect(rows[0].count).toBe(1); + }); + + it("still allows a read and a history read", async () => { + const { itemId } = await withGermanTranslation(); + const build = localizedArticleContent.translationService; + if (!build) throw new Error("no translation service"); + + const translation = await build(h.context).findByLocale(itemId, DISABLED); + const history = await translationEditorial(h.context).listRevisions( + itemId, + DISABLED, + ); + + expect(translation?.locale).toBe(DISABLED); + expect(history.edges).toEqual([]); + }); + + it("applies the same policy to an advanced localized content type", async () => { + // The rule is the language resolver's, not the content type's - so a + // content type with groups and repeatables gets exactly the same answers. + const created = await advanced(h.context).create({}, { actor: ACTOR }); + + await expect( + advancedTranslations(h.context).create( + created.row.id, + DISABLED, + { title: "Deutsch" }, + { actor: ACTOR }, + ), + ).rejects.toSatisfy(isDisabled); + }); + }); +}); diff --git a/plugins/example/src/database/migration-postgres.test.ts b/plugins/example/src/database/migration-postgres.test.ts new file mode 100644 index 000000000..9dd4f50bd --- /dev/null +++ b/plugins/example/src/database/migration-postgres.test.ts @@ -0,0 +1,680 @@ +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import type { ContentTestHarness } from "./harness"; + +import { + createContentTestHarness, + DATABASE_TEST_URL, + pgErrorCode, +} from "./harness"; + +/** + * The documented migration patterns, run against real data. + * + * Stage 1-6 prove the schema a *fresh* install gets. What they never proved is + * the thing an existing install actually does: take a table with rows in it and + * move it onto the newer shape. Every pattern below is copied from + * `apps/docs/content/docs/dev/content-engine/` - so this suite is what stops the + * docs describing a migration that quietly loses rows. + * + * The rule the patterns share, and the one every test here checks: **the + * destructive statement is in a different migration from the copy.** A backfill + * that silently dropped three rows and then deleted its source is not something + * anybody can notice afterwards. + * + * The tables are built here rather than taken from the committed migrations, + * because what is under test is the *shape* of the upgrade rather than one + * install's history - and a Stage 1-era table no longer exists anywhere to + * borrow. + */ + +let h: ContentTestHarness; + +/** Runs a script as one statement per `--> statement-breakpoint`. */ +const migrate = async (script: string): Promise => { + for (const statement of script.split("--> statement-breakpoint")) { + const trimmed = statement.trim(); + if (trimmed) await h.sql.unsafe(trimmed); + } +}; + +const countOf = async (table: string): Promise => { + const [row] = await h.sql.unsafe( + `SELECT count(*)::int AS count FROM "${table}"`, + ); + + return Number(row.count); +}; + +const columnsOf = async (table: string) => + await h.sql<{ column_name: string; is_nullable: string }[]>` + SELECT column_name, is_nullable + FROM information_schema.columns + WHERE table_name = ${table} + ORDER BY column_name + `; + +/** A Stage 1-era flat table: no publication, no editorial, no translations. */ +const STAGE_ONE = ` + CREATE TABLE "legacy_articles" ( + "id" serial PRIMARY KEY NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + "updatedAt" timestamp DEFAULT now() NOT NULL, + "title" varchar(200) NOT NULL, + "slug" varchar(160) NOT NULL, + "seoTitle" varchar(200), + "category" integer NOT NULL, + "faqJson" jsonb, + CONSTRAINT "legacy_articles_slug_key" UNIQUE("slug") + ); +`; + +const seedLegacy = async (): Promise => { + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Legacy') RETURNING "id" + `; + await h.sql` + INSERT INTO "legacy_articles" ("title", "slug", "seoTitle", "category", "faqJson") + VALUES + ('First', 'first', 'First SEO', ${category.id}, + '[{"question":"Q1","answer":"A1"},{"question":"Q2","answer":"A2"}]'::jsonb), + ('Second', 'second', NULL, ${category.id}, + '[{"question":"Q3","answer":"A3"}]'::jsonb), + ('Third', 'third', NULL, ${category.id}, NULL) + `; +}; + +describe.skipIf(!DATABASE_TEST_URL)("Content Engine migration patterns", () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + }); + + beforeEach(async () => { + await h.sql`DROP TABLE IF EXISTS "legacy_articles_translations"`; + await h.sql`DROP TABLE IF EXISTS "legacy_articles_categories"`; + await h.sql`DROP TABLE IF EXISTS "legacy_articles_faq"`; + await h.sql`DROP TABLE IF EXISTS "legacy_articles"`; + await h.sql`DELETE FROM "example_categories"`; + await h.sql.unsafe(STAGE_ONE); + await seedLegacy(); + }); + + // ------------------------------------------------------------------------- + // Additive upgrades + // ------------------------------------------------------------------------- + + describe("adding a structured group to a populated table", () => { + it("gives every existing row a null group without touching its other values", async () => { + const before = await countOf("legacy_articles"); + + await migrate(` + ALTER TABLE "legacy_articles" + ADD COLUMN "syndicationIndexable" boolean DEFAULT true NOT NULL, + ADD COLUMN "syndicationPriority" integer DEFAULT 5 NOT NULL; + `); + + expect(await countOf("legacy_articles")).toBe(before); + + const rows = await h.sql< + { syndicationIndexable: boolean; syndicationPriority: number }[] + >` + SELECT "syndicationIndexable", "syndicationPriority" FROM "legacy_articles" + `; + // A defaulted leaf is what makes this additive at all: a `NOT NULL` + // column with no default cannot be added to a table with rows in it. + expect(rows.every(row => row.syndicationPriority === 5)).toBe(true); + expect(rows.every(row => row.syndicationIndexable)).toBe(true); + }); + + it("regroups an existing column with no data migration at all", async () => { + // `seoTitle` as a top-level field and `seo.title` as a group leaf compile + // to the same column, so the upgrade is a definition change and nothing + // else. The test is that the column and its values are still there. + const before = await h.sql<{ id: number; seoTitle: null | string }[]>` + SELECT "id", "seoTitle" FROM "legacy_articles" ORDER BY "id" + `; + + await migrate(` + ALTER TABLE "legacy_articles" ADD COLUMN "seoDescription" text; + `); + + const after = await h.sql<{ id: number; seoTitle: null | string }[]>` + SELECT "id", "seoTitle" FROM "legacy_articles" ORDER BY "id" + `; + expect(after).toEqual(before); + }); + + it("refuses a non-null leaf with no default, rather than inventing values", async () => { + const code = await pgErrorCode( + async () => + await h.sql.unsafe(` + ALTER TABLE "legacy_articles" + ADD COLUMN "syndicationOwner" varchar(100) NOT NULL + `), + ); + + // 23502: not_null_violation. `defineContentType` refuses this shape at + // definition time, and Postgres refuses it here - which is what makes the + // rule a fact rather than a convention. + expect(code).toBe("23502"); + }); + }); + + // ------------------------------------------------------------------------- + // To-one to to-many + // ------------------------------------------------------------------------- + + describe("moving a to-one relation onto a junction table", () => { + const CREATE_JUNCTION = ` + CREATE TABLE "legacy_articles_categories" ( + "itemId" integer NOT NULL, + "relatedItemId" integer NOT NULL, + "position" integer NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + CONSTRAINT "legacy_articles_categories_pk" + PRIMARY KEY("itemId","relatedItemId") + );--> statement-breakpoint + CREATE UNIQUE INDEX "legacy_articles_categories_position_key" + ON "legacy_articles_categories" ("itemId","position");--> statement-breakpoint + INSERT INTO "legacy_articles_categories" ("itemId", "relatedItemId", "position") + SELECT "id", "category", 0 FROM "legacy_articles" WHERE "category" IS NOT NULL; + `; + + it("copies every reference, at position zero, before anything is dropped", async () => { + const before = await countOf("legacy_articles"); + + await migrate(CREATE_JUNCTION); + + expect(await countOf("legacy_articles_categories")).toBe(before); + const rows = await h.sql<{ position: number }[]>` + SELECT "position" FROM "legacy_articles_categories" + `; + expect(rows.every(row => row.position === 0)).toBe(true); + + // The source column is still there. That is the pattern: the destructive + // statement is a *second* migration, run after somebody has looked at the + // counts. + expect( + (await columnsOf("legacy_articles")).map(row => row.column_name), + ).toContain("category"); + }); + + it("keeps the foreign key honest once it is added", async () => { + await migrate(` + ${CREATE_JUNCTION}--> statement-breakpoint + ALTER TABLE "legacy_articles_categories" + ADD CONSTRAINT "legacy_articles_categories_related_fk" + FOREIGN KEY ("relatedItemId") REFERENCES "example_categories"("id") + ON DELETE restrict; + `); + + const code = await pgErrorCode( + async () => + await h.sql` + INSERT INTO "legacy_articles_categories" ("itemId", "relatedItemId", "position") + VALUES (1, 999999, 1) + `, + ); + + expect(code).toBe("23503"); + }); + + it("drops the source column only in the second migration", async () => { + await migrate(CREATE_JUNCTION); + const copied = await countOf("legacy_articles_categories"); + const source = await countOf("legacy_articles"); + + // The pause in the middle, made explicit: the drop is guarded by the very + // comparison the docs tell an operator to make by hand. + expect(copied).toBe(source); + + await migrate(`ALTER TABLE "legacy_articles" DROP COLUMN "category";`); + + expect( + (await columnsOf("legacy_articles")).map(row => row.column_name), + ).not.toContain("category"); + expect(await countOf("legacy_articles_categories")).toBe(copied); + }); + + it("aborts the copy whole when one row cannot be copied", async () => { + // Postgres runs a migration statement in a transaction, so a backfill + // that fails halfway leaves nothing behind - which is what makes the + // "verify before you drop" pattern safe to retry. + await migrate(` + CREATE TABLE "legacy_articles_categories" ( + "itemId" integer NOT NULL, + "relatedItemId" integer NOT NULL, + "position" integer NOT NULL, + CONSTRAINT "legacy_articles_categories_pk" + PRIMARY KEY("itemId","relatedItemId"), + CONSTRAINT "legacy_articles_categories_position_check" + CHECK ("position" >= 0) + ); + `); + + const code = await pgErrorCode( + async () => + await h.sql` + INSERT INTO "legacy_articles_categories" ("itemId", "relatedItemId", "position") + SELECT "id", "category", "id" - 100 FROM "legacy_articles" + `, + ); + + expect(code).toBe("23514"); + expect(await countOf("legacy_articles_categories")).toBe(0); + }); + }); + + // ------------------------------------------------------------------------- + // JSON array to repeatable + // ------------------------------------------------------------------------- + + describe("moving a JSON array onto a repeatable child table", () => { + const CREATE_CHILD = ` + CREATE TABLE "legacy_articles_faq" ( + "id" serial PRIMARY KEY NOT NULL, + "itemId" integer NOT NULL, + "position" integer NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + "updatedAt" timestamp DEFAULT now() NOT NULL, + "question" varchar(200) NOT NULL, + "answer" text NOT NULL + );--> statement-breakpoint + CREATE UNIQUE INDEX "legacy_articles_faq_position_key" + ON "legacy_articles_faq" ("itemId","position");--> statement-breakpoint + INSERT INTO "legacy_articles_faq" ("itemId", "position", "question", "answer") + SELECT + a."id", + entry.ordinality - 1, + entry.value ->> 'question', + entry.value ->> 'answer' + FROM "legacy_articles" a, + jsonb_array_elements(a."faqJson") + WITH ORDINALITY AS entry(value, ordinality) + WHERE a."faqJson" IS NOT NULL; + `; + + it("copies every entry and preserves its order", async () => { + await migrate(CREATE_CHILD); + + const [{ expected }] = await h.sql<{ expected: number }[]>` + SELECT coalesce(sum(jsonb_array_length("faqJson")), 0)::int AS expected + FROM "legacy_articles" + `; + expect(await countOf("legacy_articles_faq")).toBe(expected); + + const rows = await h.sql<{ position: number; question: string }[]>` + SELECT f."position", f."question" FROM "legacy_articles_faq" f + JOIN "legacy_articles" a ON a."id" = f."itemId" + WHERE a."slug" = 'first' + ORDER BY f."position" + `; + // `WITH ORDINALITY` is what carries the order across, and the engine reads + // a repeatable back in `position` order - so getting this wrong reorders + // somebody's FAQ silently. + expect(rows).toEqual([ + { position: 0, question: "Q1" }, + { position: 1, question: "Q2" }, + ]); + }); + + it("starts positions at zero, which is where the engine reads from", async () => { + await migrate(CREATE_CHILD); + + const [row] = await h.sql<{ min: number }[]>` + SELECT min("position")::int AS min FROM "legacy_articles_faq" + `; + expect(row.min).toBe(0); + }); + + it("copies nothing for a record whose array was null", async () => { + await migrate(CREATE_CHILD); + + const rows = await h.sql<{ count: number }[]>` + SELECT count(*)::int AS count FROM "legacy_articles_faq" f + JOIN "legacy_articles" a ON a."id" = f."itemId" + WHERE a."slug" = 'third' + `; + expect(rows[0].count).toBe(0); + }); + + it("leaves the source column in place for the operator to check", async () => { + await migrate(CREATE_CHILD); + + expect( + (await columnsOf("legacy_articles")).map(row => row.column_name), + ).toContain("faqJson"); + }); + + it("refuses two entries in one position once the index is there", async () => { + await migrate(CREATE_CHILD); + + const code = await pgErrorCode( + async () => + await h.sql` + INSERT INTO "legacy_articles_faq" ("itemId", "position", "question", "answer") + SELECT "itemId", "position", 'Dup', 'Dup' FROM "legacy_articles_faq" LIMIT 1 + `, + ); + + expect(code).toBe("23505"); + }); + }); + + // ------------------------------------------------------------------------- + // Non-localized to localized + // ------------------------------------------------------------------------- + + describe("localizing a table that already has rows", () => { + const CREATE_TRANSLATIONS = ` + CREATE TABLE "legacy_articles_translations" ( + "itemId" integer NOT NULL, + "languageId" integer NOT NULL, + "version" integer DEFAULT 1 NOT NULL, + "createdAt" timestamp DEFAULT now() NOT NULL, + "updatedAt" timestamp DEFAULT now() NOT NULL, + "title" varchar(200) NOT NULL, + "slug" varchar(160) NOT NULL + );--> statement-breakpoint + INSERT INTO "legacy_articles_translations" + ("itemId", "languageId", "title", "slug", "createdAt", "updatedAt") + SELECT + a."id", + (SELECT "id" FROM "core_languages" WHERE "code" = 'en'), + a."title", + a."slug", + a."createdAt", + a."updatedAt" + FROM "legacy_articles" a; + `; + + /** Step 4 of the documented six, verbatim in shape. */ + const VERIFY = ` + DO $$ + DECLARE + source_count integer; + copied_count integer; + language_id integer; + BEGIN + SELECT "id" INTO language_id FROM "core_languages" WHERE "code" = 'en'; + IF language_id IS NULL THEN + RAISE EXCEPTION 'No core_languages row for the default locale "en".'; + END IF; + + SELECT count(*) INTO source_count FROM "legacy_articles"; + SELECT count(*) INTO copied_count FROM "legacy_articles_translations"; + + IF source_count <> copied_count THEN + RAISE EXCEPTION 'Copied % of % rows; refusing to drop the source columns.', + copied_count, source_count; + END IF; + END $$; + `; + + const CONSTRAIN = ` + ALTER TABLE "legacy_articles_translations" + ADD CONSTRAINT "legacy_articles_translations_pk" + PRIMARY KEY ("itemId", "languageId");--> statement-breakpoint + ALTER TABLE "legacy_articles_translations" + ADD CONSTRAINT "legacy_articles_translations_item_fk" + FOREIGN KEY ("itemId") REFERENCES "legacy_articles"("id") + ON DELETE cascade ON UPDATE cascade;--> statement-breakpoint + ALTER TABLE "legacy_articles_translations" + ADD CONSTRAINT "legacy_articles_translations_language_fk" + FOREIGN KEY ("languageId") REFERENCES "core_languages"("id") + ON DELETE restrict ON UPDATE cascade;--> statement-breakpoint + CREATE UNIQUE INDEX "legacy_articles_translations_language_id_slug_key" + ON "legacy_articles_translations" ("languageId","slug"); + `; + + it("copies every row into the default language, timestamps included", async () => { + const before = await h.sql< + { createdAt: Date; id: number; slug: string; title: string }[] + >` + SELECT "id", "title", "slug", "createdAt" FROM "legacy_articles" ORDER BY "id" + `; + + await migrate(CREATE_TRANSLATIONS); + + const after = await h.sql< + { createdAt: Date; itemId: number; slug: string; title: string }[] + >` + SELECT "itemId", "title", "slug", "createdAt" + FROM "legacy_articles_translations" ORDER BY "itemId" + `; + + expect(after).toHaveLength(before.length); + expect(after.map(row => [row.itemId, row.title, row.slug])).toEqual( + before.map(row => [row.id, row.title, row.slug]), + ); + // The original timestamps travel with the values. A translation stamped + // `now()` would tell every editor the whole collection was rewritten on + // deployment day. + expect(after.map(row => row.createdAt)).toEqual( + before.map(row => row.createdAt), + ); + }); + + it("resolves the language rather than hardcoding an identifier", async () => { + await migrate(CREATE_TRANSLATIONS); + + const [english] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "core_languages" WHERE "code" = 'en' + `; + const rows = await h.sql<{ languageId: number }[]>` + SELECT DISTINCT "languageId" FROM "legacy_articles_translations" + `; + + // A literal `1` is right on the machine it was written on and wrong on + // every other install. + expect(rows).toEqual([{ languageId: english.id }]); + }); + + it("passes its own verification step and only then drops the columns", async () => { + await migrate(CREATE_TRANSLATIONS); + await migrate(VERIFY); + await migrate(CONSTRAIN); + await migrate(` + ALTER TABLE "legacy_articles" DROP COLUMN "title";--> statement-breakpoint + ALTER TABLE "legacy_articles" DROP COLUMN "slug"; + `); + + const columns = (await columnsOf("legacy_articles")).map( + row => row.column_name, + ); + expect(columns).not.toContain("title"); + expect(columns).not.toContain("slug"); + expect(await countOf("legacy_articles_translations")).toBe(3); + }); + + it("aborts rather than dropping the source when a row did not copy", async () => { + // The failure the verification exists for, produced deliberately: one + // source row that the copy missed. + await migrate(CREATE_TRANSLATIONS); + await h.sql` + DELETE FROM "legacy_articles_translations" + WHERE "itemId" = (SELECT min("itemId") FROM "legacy_articles_translations") + `; + + await expect(migrate(VERIFY)).rejects.toThrow( + /refusing to drop the source columns/, + ); + + // And the source is untouched, which is the whole point. + const columns = (await columnsOf("legacy_articles")).map( + row => row.column_name, + ); + expect(columns).toContain("title"); + expect(columns).toContain("slug"); + }); + + it("moves uniqueness from global to per language", async () => { + await migrate(CREATE_TRANSLATIONS); + await migrate(CONSTRAIN); + + const [polish] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "core_languages" WHERE "code" = 'pl' + `; + + // The same slug in another language is now legal - it was not before, + // when the column carried a global unique index. + await h.sql` + INSERT INTO "legacy_articles_translations" + ("itemId", "languageId", "title", "slug") + VALUES ( + (SELECT min("id") FROM "legacy_articles"), ${polish.id}, 'Pierwszy', 'first' + ) + `; + + const code = await pgErrorCode( + async () => + await h.sql` + INSERT INTO "legacy_articles_translations" + ("itemId", "languageId", "title", "slug") + VALUES ( + (SELECT max("id") FROM "legacy_articles"), ${polish.id}, 'Drugi', 'first' + ) + `, + ); + expect(code).toBe("23505"); + }); + + it("surfaces a pre-existing duplicate as a named failure with the data intact", async () => { + // Step 5 is where a collision shows up, deliberately after the copy: the + // rows are still there to look at, rather than half-migrated. + await migrate(CREATE_TRANSLATIONS); + await h.sql` + UPDATE "legacy_articles_translations" SET "slug" = 'first' + WHERE "itemId" = (SELECT max("itemId") FROM "legacy_articles_translations") + `; + + const code = await pgErrorCode(async () => await migrate(CONSTRAIN)); + + expect(code).toBe("23505"); + expect(await countOf("legacy_articles_translations")).toBe(3); + }); + + it("takes the translations with the record it belongs to", async () => { + await migrate(CREATE_TRANSLATIONS); + await migrate(CONSTRAIN); + + await h.sql` + DELETE FROM "legacy_articles" + WHERE "id" = (SELECT min("id") FROM "legacy_articles") + `; + + expect(await countOf("legacy_articles_translations")).toBe(2); + }); + + it("refuses to remove a language that content is written in", async () => { + await migrate(CREATE_TRANSLATIONS); + await migrate(CONSTRAIN); + + const code = await pgErrorCode( + async () => + await h.sql`DELETE FROM "core_languages" WHERE "code" = 'en'`, + ); + + // `ON DELETE restrict`, which Postgres 18 reports as `23001` and earlier + // majors as `23503`. The version decides which is correct rather than the + // assertion accepting either. + expect(code).toBe(h.serverMajor >= 18 ? "23001" : "23503"); + }); + }); + + // ------------------------------------------------------------------------- + // Transactional behaviour + // ------------------------------------------------------------------------- + + describe("a failed migration leaves nothing half-applied", () => { + it("rolls a multi-statement data migration back whole", async () => { + // The migrator wraps a file in a transaction, so this is what an operator + // gets when statement three of four fails: the schema and the data exactly + // as they were. + await expect( + h.sql.begin(async transaction => { + await transaction.unsafe(` + CREATE TABLE "legacy_articles_faq" ( + "id" serial PRIMARY KEY NOT NULL, + "itemId" integer NOT NULL, + "position" integer NOT NULL, + "question" varchar(200) NOT NULL, + "answer" text NOT NULL + ) + `); + await transaction.unsafe(` + INSERT INTO "legacy_articles_faq" ("itemId", "position", "question", "answer") + SELECT "id", 0, 'Q', 'A' FROM "legacy_articles" + `); + await transaction.unsafe( + `ALTER TABLE "legacy_articles_faq" ADD COLUMN "answer" text`, + ); + }), + ).rejects.toThrow(); + + // DDL is transactional in Postgres, so even the `CREATE TABLE` is gone. + const tables = await h.sql<{ table_name: string }[]>` + SELECT table_name FROM information_schema.tables + WHERE table_name = 'legacy_articles_faq' + `; + expect(tables).toEqual([]); + }); + + it("keeps a verification failure and its copy in one transaction", async () => { + await expect( + h.sql.begin(async transaction => { + await transaction.unsafe(` + CREATE TABLE "legacy_articles_translations" ( + "itemId" integer NOT NULL, + "languageId" integer NOT NULL, + "title" varchar(200) NOT NULL, + "slug" varchar(160) NOT NULL + ) + `); + await transaction.unsafe(` + INSERT INTO "legacy_articles_translations" + ("itemId", "languageId", "title", "slug") + SELECT a."id", + (SELECT "id" FROM "core_languages" WHERE "code" = 'en'), + a."title", a."slug" + FROM "legacy_articles" a LIMIT 1 + `); + await transaction.unsafe(` + DO $$ + DECLARE source_count integer; copied_count integer; + BEGIN + SELECT count(*) INTO source_count FROM "legacy_articles"; + SELECT count(*) INTO copied_count FROM "legacy_articles_translations"; + IF source_count <> copied_count THEN + RAISE EXCEPTION 'Copied % of % rows.', copied_count, source_count; + END IF; + END $$; + `); + }), + ).rejects.toThrow(/Copied 1 of 3 rows/); + + const tables = await h.sql<{ table_name: string }[]>` + SELECT table_name FROM information_schema.tables + WHERE table_name = 'legacy_articles_translations' + `; + expect(tables).toEqual([]); + }); + + it("cannot roll back a CREATE INDEX CONCURRENTLY, which is why none is generated", async () => { + // The documented exception: `CONCURRENTLY` cannot run inside a + // transaction at all, so a migration using it is not atomic. Nothing the + // engine generates does, and this pins that. + await expect( + h.sql.begin(async transaction => { + await transaction.unsafe( + `CREATE INDEX CONCURRENTLY "legacy_articles_title_idx" ON "legacy_articles" ("title")`, + ); + }), + ).rejects.toThrow(); + }); + }); +}); diff --git a/plugins/example/src/database/pagination-postgres.test.ts b/plugins/example/src/database/pagination-postgres.test.ts new file mode 100644 index 000000000..68794fedb --- /dev/null +++ b/plugins/example/src/database/pagination-postgres.test.ts @@ -0,0 +1,1137 @@ +import type { Context } from "hono"; + +import { withPagination } from "@vitnode/core/api/lib/with-pagination"; +import { HTTPException } from "hono/http-exception"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { CONFIG_PLUGIN } from "@/const"; + +import type { ContentTestHarness } from "./harness"; + +import { articleContent, example_articles } from "./articles"; +import { + clearContentTables, + createContentTestHarness, + DATABASE_TEST_URL, +} from "./harness"; + +/** + * Keyset pagination, against a collection built to break an id-only cursor. + * + * The bug this suite exists for: the cursor used to be the row identifier while + * the `ORDER BY` was something else entirely. Those describe two different + * sequences, so a page boundary landing anywhere except a coincidence would + * skip rows - permanently, and silently, because a short page looks exactly + * like the end of a collection. + * + * Every fixture here therefore makes the sort value **disagree** with the + * identifier on purpose. A cursor that is the ordered tuple walks them + * correctly; one that is only an identifier cannot. + * + * The oracle in every case is the same query without pagination: a full walk + * has to produce exactly the rows a single `ORDER BY` produces, in the same + * order. + */ + +let h: ContentTestHarness; +let categoryId = 0; + +const service = (on: Context = h.context) => articleContent.service(on); + +interface Seed { + code: string; + publishedAt?: Date | null; + title: string; + updatedAt: Date; +} + +/** + * Inserts rows in the order given, so identifiers ascend with the array while + * the sort values do whatever the fixture says. + */ +const seed = async (rows: readonly Seed[]): Promise => { + const ids: number[] = []; + for (const [index, row] of rows.entries()) { + const [inserted] = await h.sql<{ id: number }[]>` + INSERT INTO "example_articles" + ("title", "slug", "code", "category", "status", "publishedAt", "updatedAt") + VALUES ( + ${row.title}, + ${`slug-${row.code}`}, + ${row.code}, + ${categoryId}, + ${row.publishedAt === undefined || row.publishedAt === null ? "draft" : "published"}, + ${row.publishedAt?.toISOString() ?? null}::timestamp, + ${row.updatedAt.toISOString()}::timestamp + ) + RETURNING "id" + `; + ids.push(inserted.id); + expect(index).toBeGreaterThanOrEqual(0); + } + + return ids; +}; + +/** The order a single un-paginated query produces - the oracle. */ +const oracle = async ( + column: string, + order: "asc" | "desc", +): Promise => { + const rows = await h.sql.unsafe( + `SELECT "id" FROM "example_articles" + ORDER BY "${column}" ${order.toUpperCase()}, "id" ${order.toUpperCase()}`, + ); + + return rows.map(row => Number(row.id)); +}; + +/** Walks every page forward and returns the identifiers, in order. */ +const walkForward = async ({ + column, + order = "asc", + pageSize, +}: { + column?: string; + order?: "asc" | "desc"; + pageSize: number; +}): Promise => { + const seen: number[] = []; + let cursor: string | undefined; + + for (let page = 0; page < 200; page += 1) { + const result = await service().findMany({ + orderBy: column ? { column: column as never, order } : { order }, + query: { cursor, first: String(pageSize) }, + }); + + seen.push(...result.edges.map(row => row.id)); + if (!result.pageInfo.hasNextPage) break; + + // The invariant the reviewer asked for: a page that claims a neighbour has + // to hand out a cursor that reaches it. + expect(result.pageInfo.endCursor).not.toBeNull(); + cursor = result.pageInfo.endCursor ?? undefined; + } + + return seen; +}; + +const statusOf = (error: unknown): number => + error instanceof HTTPException ? error.status : 0; + +describe.skipIf(!DATABASE_TEST_URL)( + "cursor pagination against Postgres", + () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + }); + + beforeEach(async () => { + await clearContentTables(h.sql); + h.reset(); + + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Pagination') + RETURNING "id" + `; + categoryId = category.id; + }); + + // ------------------------------------------------------------------------- + // The regression + // ------------------------------------------------------------------------- + + /** + * Identifiers ascending, sort values deliberately shuffled. + * + * `id=1` sorts last, `id=2` sorts first, `id=3` sits in the middle. An id-only + * cursor mints `2` after the first page and then asks for `id > 2`, which + * skips `id=1` forever. + */ + const NON_MONOTONIC: Seed[] = [ + { + code: "n1", + title: "Zulu", + updatedAt: new Date("2026-03-01T00:00:00.000Z"), + }, + { + code: "n2", + title: "Alpha", + updatedAt: new Date("2026-01-01T00:00:00.000Z"), + }, + { + code: "n3", + title: "Mike", + updatedAt: new Date("2026-02-01T00:00:00.000Z"), + }, + ]; + + it("skips nothing when the sort value does not follow the identifier", async () => { + const ids = await seed(NON_MONOTONIC); + + const walked = await walkForward({ column: "title", pageSize: 1 }); + + expect(walked).toHaveLength(ids.length); + expect(new Set(walked).size).toBe(ids.length); + // And in the order a single query would have produced. + expect(walked).toEqual(await oracle("title", "asc")); + }); + + it("skips nothing ordering by a timestamp that does not follow the identifier", async () => { + const ids = await seed(NON_MONOTONIC); + + const walked = await walkForward({ column: "updatedAt", pageSize: 1 }); + + expect(walked).toHaveLength(ids.length); + expect(new Set(walked).size).toBe(ids.length); + expect(walked).toEqual(await oracle("updatedAt", "asc")); + }); + + it("skips nothing descending either", async () => { + const ids = await seed(NON_MONOTONIC); + + const walked = await walkForward({ + column: "updatedAt", + order: "desc", + pageSize: 1, + }); + + expect(walked).toHaveLength(ids.length); + expect(walked).toEqual(await oracle("updatedAt", "desc")); + }); + + it("holds over a larger, thoroughly shuffled collection", async () => { + // 40 rows whose sort values are a permutation of their identifiers, walked + // two at a time so every page boundary lands somewhere different. + const rows: Seed[] = Array.from({ length: 40 }, (_row, index) => ({ + code: `bulk-${index}`, + title: `Title ${String((index * 17) % 40).padStart(2, "0")}`, + updatedAt: new Date(2026, 0, 1 + ((index * 23) % 40)), + })); + await seed(rows); + + for (const column of ["title", "updatedAt"] as const) { + for (const order of ["asc", "desc"] as const) { + const walked = await walkForward({ column, order, pageSize: 3 }); + + expect([column, order, walked.length]).toEqual([column, order, 40]); + expect(new Set(walked).size).toBe(40); + expect(walked).toEqual(await oracle(column, order)); + } + } + }); + + // ------------------------------------------------------------------------- + // The cursor is a historical position + // ------------------------------------------------------------------------- + + /** + * A cursor names where the page ended, not which row ended it. + * + * The distinction only shows itself when the boundary row moves. If the next + * page's comparison were built from that row's *current* value, one edit + * would drag the boundary with it and silently skip every row the ordering + * used to have in between - a page of results nobody ever sees, with no + * error and no short page to notice. + * + * The fixture makes the continuation deterministic: five rows ascending by + * `updatedAt`, identifiers ascending with them. + */ + describe("when the boundary row changes after the cursor was issued", () => { + const LADDER: Seed[] = [ + { + code: "l1", + title: "One", + updatedAt: new Date("2026-08-09T10:00:00Z"), + }, + { + code: "l2", + title: "Two", + updatedAt: new Date("2026-08-09T10:01:00Z"), + }, + { + code: "l3", + title: "Three", + updatedAt: new Date("2026-08-09T11:00:00Z"), + }, + { + code: "l4", + title: "Four", + updatedAt: new Date("2026-08-09T12:00:00Z"), + }, + { + code: "l5", + title: "Five", + updatedAt: new Date("2026-08-09T13:00:00Z"), + }, + ]; + + /** Page 1 of one row, plus the cursor it handed back. */ + const firstPage = async (order: "asc" | "desc" = "asc") => { + const page = await service().findMany({ + orderBy: { column: "updatedAt" as never, order }, + query: { first: "1" }, + }); + + return { + boundary: page.edges[0].id, + cursor: page.pageInfo.endCursor ?? undefined, + }; + }; + + const walkFrom = async ( + cursor: string | undefined, + order: "asc" | "desc" = "asc", + ) => { + const seen: number[] = []; + let next = cursor; + for (let page = 0; page < 20; page += 1) { + const result = await service().findMany({ + orderBy: { column: "updatedAt" as never, order }, + query: { cursor: next, first: "2" }, + }); + seen.push(...result.edges.map(row => row.id)); + if (!result.pageInfo.hasNextPage) break; + next = result.pageInfo.endCursor ?? undefined; + } + + return seen; + }; + + const moveTo = async (id: number, when: string) => { + await h.sql` + UPDATE "example_articles" + SET "updatedAt" = ${when}::timestamp + WHERE "id" = ${id} + `; + }; + + it("still reaches every row that was after the cursor when it was issued", async () => { + // The regression. Moving the boundary row to the *end* of the ordering + // would drag a re-read boundary with it, and 10:01, 11:00, 12:00 and + // 13:00 would be skipped for good. + const ids = await seed(LADDER); + const { boundary, cursor } = await firstPage(); + expect(boundary).toBe(ids[0]); + + await moveTo(boundary, "2026-08-09T14:00:00"); + + const seen = await walkFrom(cursor); + + for (const id of ids.slice(1)) expect(seen).toContain(id); + expect(new Set(seen).size).toBe(seen.length); + }); + + it("shows the moved row again, because it moved into unvisited ground", async () => { + // The honest consequence, stated rather than hidden: a keyset walk is + // not a snapshot, so a row that moves from behind the cursor to ahead + // of it is seen a second time. What matters is that nothing else moved. + const ids = await seed(LADDER); + const { boundary, cursor } = await firstPage(); + + await moveTo(boundary, "2026-08-09T14:00:00"); + + const seen = await walkFrom(cursor); + + expect(seen).toContain(boundary); + expect(seen.sort((a, b) => a - b)).toEqual( + [...ids].sort((a, b) => a - b), + ); + }); + + it("does not show it again when it moves further behind the cursor", async () => { + const ids = await seed(LADDER); + const { boundary, cursor } = await firstPage(); + + await moveTo(boundary, "2026-08-09T09:00:00"); + + const seen = await walkFrom(cursor); + + expect(seen).not.toContain(boundary); + expect(seen.sort((a, b) => a - b)).toEqual(ids.slice(1)); + }); + + it("keeps working when the boundary row is deleted outright", async () => { + // Nothing to re-read, and nothing that needs re-reading: the position + // is in the cursor. + const ids = await seed(LADDER); + const { boundary, cursor } = await firstPage(); + + await h.sql`DELETE FROM "example_articles" WHERE "id" = ${boundary}`; + + const seen = await walkFrom(cursor); + + expect(seen.sort((a, b) => a - b)).toEqual(ids.slice(1)); + expect(new Set(seen).size).toBe(seen.length); + }); + + it("survives the whole first page being deleted", async () => { + const ids = await seed(LADDER); + const page = await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { first: "3" }, + }); + const read = page.edges.map(row => row.id); + + for (const id of read) { + await h.sql`DELETE FROM "example_articles" WHERE "id" = ${id}`; + } + + const seen = await walkFrom(page.pageInfo.endCursor ?? undefined); + + expect(seen.sort((a, b) => a - b)).toEqual( + ids.filter(id => !read.includes(id)).sort((a, b) => a - b), + ); + }); + + it("holds the same way descending", async () => { + const ids = await seed(LADDER); + const { boundary, cursor } = await firstPage("desc"); + + await moveTo(boundary, "2026-08-09T00:01:00"); + + const seen = await walkFrom(cursor, "desc"); + + for (const id of ids.slice(0, 4)) expect(seen).toContain(id); + expect(new Set(seen).size).toBe(seen.length); + }); + + it("carries the database's own timestamp text, microseconds included", async () => { + // The reason the cursor can be self-contained at all. A JavaScript + // `Date` holds milliseconds; `now()` writes microseconds. A cursor that + // had been through a `Date` would be strictly smaller than the stored + // value and would exclude the whole millisecond it came from - and here + // every row shares one `now()`, so the walk would stop after page one. + await h.sql` + INSERT INTO "example_articles" + ("title", "slug", "code", "category", "status", "updatedAt") + SELECT 'Micro ' || i, 'micro-' || i, 'micro-' || i, ${categoryId}, + 'draft', now() + FROM generate_series(1, 6::int) AS i + `; + + const page = await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { first: "1" }, + }); + const decoded = JSON.parse( + Buffer.from(page.pageInfo.endCursor ?? "", "base64url").toString( + "utf8", + ), + ) as { value: string }; + + // Byte-identical to what the column holds, rather than "has enough + // digits": trailing zeros are dropped by `::text`, so counting them + // would be a coin flip, and equality is the property that matters. + const [stored] = await h.sql<{ text: string }[]>` + SELECT "updatedAt"::text AS text FROM "example_articles" + WHERE "id" = ${page.edges[0].id} + `; + expect(decoded.value).toBe(stored.text); + + // And the walk completes, which it cannot if the boundary was + // truncated: every row here shares one `now()`. + const seen = await walkFrom(page.pageInfo.endCursor ?? undefined); + expect(seen).toHaveLength(5); + }); + + /** + * The window between choosing the rows and describing where they were. + * + * The cursor value used to be fetched by a *second* statement, after the + * page rows had already come back - which is a time-of-check / + * time-of-use gap wide enough for another writer to walk through. The row + * was chosen at one position and handed back as a cursor pointing at + * another, so the next page started somewhere the reader had never been + * and everything in between was gone. + * + * There is no window now: the value is projected by the page query + * itself. These tests prove that by mutating the boundary row in exactly + * that gap - after the rows are in hand, before the cursor is minted - + * and showing the cursor does not notice. + */ + describe("while the page is being turned into cursors", () => { + const decode = (cursor: null | string | undefined) => + JSON.parse( + Buffer.from(cursor ?? "", "base64url").toString("utf8"), + ) as { id: number; value: null | string }; + + /** + * One page, with a mutation spliced into the mint-time gap. + * + * `withPagination` is driven directly because the gap is inside it: + * the callback is what fetched the rows, so running the mutation on the + * way out of it lands precisely between the page query and the cursor. + */ + const pageWithRace = async ( + mutate: (boundaryId: number) => Promise, + ) => + await withPagination({ + c: h.context, + orderBy: { column: example_articles.updatedAt, order: "asc" }, + params: { query: { first: "1" } }, + primaryCursor: example_articles.id, + query: async ({ cursorSelection, limit, orderBy, where }) => { + const rows = await h.db + .select({ id: example_articles.id, ...cursorSelection }) + .from(example_articles) + .where(where) + .orderBy(orderBy) + .limit(typeof limit === "number" ? limit : 2); + + const boundary = rows[0]; + if (boundary) await mutate(boundary.id); + + return rows; + }, + table: example_articles, + }); + + it("mints the position the row had, not the one it was given meanwhile", async () => { + const ids = await seed(LADDER); + + const page = await pageWithRace( + async id => await moveTo(id, "2026-08-09T14:00:00"), + ); + + expect(page.edges[0].id).toBe(ids[0]); + // The row now says 14:00. The cursor still says 10:00, because 10:00 + // is where the row was when it ended this page. + expect(decode(page.pageInfo.endCursor).value).toBe( + "2026-08-09 10:00:00", + ); + + // And the consequence that matters: 10:01, 11:00, 12:00 and 13:00 are + // all still reachable. A cursor carrying 14:00 would have skipped + // every one of them, permanently and without an error. + const seen = await walkFrom(page.pageInfo.endCursor ?? undefined); + for (const id of ids.slice(1)) expect(seen).toContain(id); + }); + + it("mints a real position when the row is deleted meanwhile", async () => { + // The worse half of the old race. A second lookup found nothing, and + // "nothing" became `null` - which for a nullable ordering is not the + // absence of a position but a real one, inside the null block. The + // walk jumped there and abandoned the rest of the collection. + const ids = await seed(LADDER); + + const page = await pageWithRace(async id => { + await h.sql`DELETE FROM "example_articles" WHERE "id" = ${id}`; + }); + + expect(decode(page.pageInfo.endCursor).value).toBe( + "2026-08-09 10:00:00", + ); + + const seen = await walkFrom(page.pageInfo.endCursor ?? undefined); + expect(seen.sort((a, b) => a - b)).toEqual(ids.slice(1)); + }); + + it("reads its cursor value out of the page query, not a second one", async () => { + // The structural regression assertion. There is nothing to race with + // if there is no second statement, so this is the property to guard + // rather than the symptom. + await seed(LADDER); + + const list = async () => + await service(h.counted.context).findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { first: "2" }, + }); + + // Warmed first: `postgres` prepares a statement the first time it + // sees its shape, so a cold call issues messages a warm one does not. + await list(); + h.counted.reset(); + await list(); + + const selects = h.counted.queries.filter(query => + /^\s*select/i.test(query), + ); + + // Two, and only two: the total count, and the page. A third would be + // the boundary lookup coming back. + expect(selects).toHaveLength(2); + // The page carries the cursor value with it, at the database's own + // precision. + expect(selects.some(query => query.includes("::text"))).toBe(true); + }); + }); + }); + + // ------------------------------------------------------------------------- + // Ties + // ------------------------------------------------------------------------- + + it("returns every row exactly once when the sort values are all equal", async () => { + // The tie case: with no tiebreaker the rows sit wherever Postgres feels + // like putting them, and a page boundary inside the tie loses one. + const stamp = new Date("2026-05-05T00:00:00.000Z"); + const ids = await seed( + Array.from({ length: 12 }, (_row, index) => ({ + code: `tie-${index}`, + title: `Tie ${index}`, + updatedAt: stamp, + })), + ); + + const walked = await walkForward({ column: "updatedAt", pageSize: 5 }); + + expect(walked).toHaveLength(ids.length); + expect(new Set(walked).size).toBe(ids.length); + expect(walked).toEqual(await oracle("updatedAt", "asc")); + }); + + it("returns every row exactly once with ties in a published-at ordering", async () => { + const stamp = new Date("2026-05-05T00:00:00.000Z"); + const ids = await seed( + Array.from({ length: 9 }, (_row, index) => ({ + code: `pub-${index}`, + publishedAt: stamp, + title: `Published ${index}`, + updatedAt: new Date(2026, 0, 1 + index), + })), + ); + + const walked = await walkForward({ + column: "publishedAt", + order: "desc", + pageSize: 4, + }); + + expect(walked).toHaveLength(ids.length); + expect(walked).toEqual(await oracle("publishedAt", "desc")); + }); + + // ------------------------------------------------------------------------- + // Nulls + // ------------------------------------------------------------------------- + + /** + * A nullable order column is where the null block has to be named explicitly: + * Postgres sorts `NULLS LAST` ascending and `NULLS FIRST` descending, and + * `column > NULL` is `NULL` rather than true - so a page boundary landing on + * the block would otherwise end the walk early and silently. + */ + it("walks a nullable order column through its null block, ascending", async () => { + const ids = await seed([ + { + code: "u1", + publishedAt: new Date("2026-02-01"), + title: "One", + updatedAt: new Date("2026-01-01"), + }, + { + code: "u2", + publishedAt: null, + title: "Two", + updatedAt: new Date("2026-01-02"), + }, + { + code: "u3", + publishedAt: new Date("2026-01-01"), + title: "Three", + updatedAt: new Date("2026-01-03"), + }, + { + code: "u4", + publishedAt: null, + title: "Four", + updatedAt: new Date("2026-01-04"), + }, + { + code: "u5", + publishedAt: new Date("2026-03-01"), + title: "Five", + updatedAt: new Date("2026-01-05"), + }, + ]); + + const walked = await walkForward({ column: "publishedAt", pageSize: 2 }); + + expect(walked).toHaveLength(ids.length); + expect(new Set(walked).size).toBe(ids.length); + expect(walked).toEqual(await oracle("publishedAt", "asc")); + }); + + it("walks a nullable order column through its null block, descending", async () => { + const ids = await seed([ + { + code: "d1", + publishedAt: new Date("2026-02-01"), + title: "One", + updatedAt: new Date("2026-01-01"), + }, + { + code: "d2", + publishedAt: null, + title: "Two", + updatedAt: new Date("2026-01-02"), + }, + { + code: "d3", + publishedAt: new Date("2026-01-01"), + title: "Three", + updatedAt: new Date("2026-01-03"), + }, + { + code: "d4", + publishedAt: null, + title: "Four", + updatedAt: new Date("2026-01-04"), + }, + { + code: "d5", + publishedAt: new Date("2026-03-01"), + title: "Five", + updatedAt: new Date("2026-01-05"), + }, + ]); + + const walked = await walkForward({ + column: "publishedAt", + order: "desc", + pageSize: 2, + }); + + expect(walked).toHaveLength(ids.length); + expect(walked).toEqual(await oracle("publishedAt", "desc")); + }); + + // ------------------------------------------------------------------------- + // Ordering by the identifier + // ------------------------------------------------------------------------- + + it.each([["asc" as const], ["desc" as const]])( + "walks the identifier ordering (%s)", + async order => { + const ids = await seed(NON_MONOTONIC); + + const walked = await walkForward({ column: "id", order, pageSize: 2 }); + + expect(walked).toHaveLength(ids.length); + expect(walked).toEqual(await oracle("id", order)); + }, + ); + + it("still accepts a legacy numeric cursor when ordering by the identifier", async () => { + // Old bookmarks keep working exactly where an identifier really is the + // whole ordered tuple - and nowhere else. + const ids = await seed(NON_MONOTONIC); + + const page = await service().findMany({ + orderBy: { column: "id" as never, order: "asc" }, + query: { cursor: String(ids[0]), first: "10" }, + }); + + expect(page.edges.map(row => row.id)).toEqual(ids.slice(1)); + }); + + it("refuses a legacy numeric cursor on any other ordering", async () => { + await seed(NON_MONOTONIC); + + await expect( + service().findMany({ + orderBy: { column: "title" as never, order: "asc" }, + query: { cursor: "1", first: "10" }, + }), + ).rejects.toThrow(/cannot be used with the "title" ordering/); + }); + + // ------------------------------------------------------------------------- + // Backward pagination + // ------------------------------------------------------------------------- + + it("walks backward from the end and reaches the beginning", async () => { + const rows: Seed[] = Array.from({ length: 11 }, (_row, index) => ({ + code: `back-${index}`, + title: `Title ${String((index * 7) % 11).padStart(2, "0")}`, + updatedAt: new Date(2026, 0, 1 + ((index * 5) % 11)), + })); + await seed(rows); + + const expected = await oracle("title", "asc"); + + // Forward to the end, keeping the cursor of the final page's first row. + const forward = await service().findMany({ + orderBy: { column: "title" as never, order: "asc" }, + query: { first: "11" }, + }); + expect(forward.edges.map(row => row.id)).toEqual(expected); + + // Then backward from the last row, four at a time. + const seen: number[] = []; + let cursor = forward.pageInfo.endCursor ?? undefined; + for (let page = 0; page < 20; page += 1) { + const result = await service().findMany({ + orderBy: { column: "title" as never, order: "asc" }, + query: { cursor, last: "4" }, + }); + if (result.edges.length === 0) break; + + seen.unshift(...result.edges.map(row => row.id)); + if (!result.pageInfo.hasPreviousPage) break; + expect(result.pageInfo.startCursor).not.toBeNull(); + cursor = result.pageInfo.startCursor ?? undefined; + } + + // Everything before the row we started from, in the same order. + expect(seen).toEqual(expected.slice(0, expected.length - 1)); + expect(new Set(seen).size).toBe(seen.length); + }); + + it("keeps backward pagination correct with a non-monotonic ordering", async () => { + await seed(NON_MONOTONIC); + const expected = await oracle("updatedAt", "desc"); + + const forward = await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "desc" }, + query: { first: "3" }, + }); + + const back = await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "desc" }, + query: { cursor: forward.pageInfo.endCursor ?? undefined, last: "2" }, + }); + + expect(back.edges.map(row => row.id)).toEqual(expected.slice(0, 2)); + }); + + // ------------------------------------------------------------------------- + // Page info + // ------------------------------------------------------------------------- + + it("never claims a next page it cannot hand out a cursor for", async () => { + const rows: Seed[] = Array.from({ length: 6 }, (_row, index) => ({ + code: `info-${index}`, + title: `Info ${index}`, + updatedAt: new Date(2026, 0, 1 + index), + })); + await seed(rows); + + let cursor: string | undefined; + for (let page = 0; page < 10; page += 1) { + const result = await service().findMany({ + orderBy: { column: "title" as never, order: "asc" }, + query: { cursor, first: "2" }, + }); + + if (result.pageInfo.hasNextPage) { + expect(result.pageInfo.endCursor).toEqual(expect.any(String)); + } + if (result.pageInfo.hasPreviousPage) { + expect(result.pageInfo.startCursor).toEqual(expect.any(String)); + } + if (!result.pageInfo.hasNextPage) break; + cursor = result.pageInfo.endCursor ?? undefined; + } + }); + + it("reports an empty collection with no cursors and no neighbours", async () => { + const result = await service().findMany({ query: { first: "5" } }); + + expect(result.pageInfo).toMatchObject({ + endCursor: null, + hasNextPage: false, + hasPreviousPage: false, + startCursor: null, + totalCount: 0, + }); + }); + + it("hands out an opaque cursor rather than a row identifier", async () => { + const ids = await seed(NON_MONOTONIC); + + const page = await service().findMany({ + orderBy: { column: "title" as never, order: "asc" }, + query: { first: "1" }, + }); + + const cursor = page.pageInfo.endCursor ?? ""; + expect(cursor).not.toBe(String(ids[0])); + expect(Number.isNaN(Number(cursor))).toBe(true); + // It carries the ordered tuple, which is the whole point. + expect( + JSON.parse(Buffer.from(cursor, "base64url").toString("utf8")), + ).toMatchObject({ column: "title" }); + }); + + it("keeps its own projected column out of every row it returns", async () => { + // The page query selects the cursor value so it can be minted from the + // same statement. That column is pagination's business: it is taken back + // before a row reaches a handler, so it cannot reach a response, a + // schema, a search document or a revision snapshot either. + await seed(NON_MONOTONIC); + const publicService = articleContent.publicService; + if (!publicService) throw new Error("no public service"); + + const admin = await service().findMany({ query: { first: "3" } }); + const anonymous = await publicService(h.context).findMany({ + query: { first: "3" }, + }); + + expect(admin.edges.length).toBeGreaterThan(0); + for (const row of [...admin.edges, ...anonymous.edges]) { + expect(Object.keys(row)).not.toContain("__cursorValue"); + } + }); + + // ------------------------------------------------------------------------- + // Validation + // ------------------------------------------------------------------------- + + describe("refuses a request it cannot answer", () => { + const expect400 = async (query: Record) => { + try { + await service().findMany({ query }); + } catch (error) { + expect(statusOf(error)).toBe(400); + + return; + } + + throw new Error(`Expected ${JSON.stringify(query)} to be refused.`); + }; + + /** + * The cursor is opaque but not signed, so every field is hostile input. + * + * These go through the real service, which is where the column is known - + * a value that looks fine in isolation is only wrong relative to the + * column it claims to describe. + */ + const tampered = (value: unknown, column = "updatedAt") => + Buffer.from(JSON.stringify({ column, id: 1, value })).toString( + "base64url", + ); + + it.each([ + ["nonsense", "not-a-date", "updatedAt"], + ["a number", 1_700_000_000, "updatedAt"], + ["a boolean", true, "updatedAt"], + [ + "an injection attempt", + "2026-08-09'; DROP TABLE example_articles; --", + "updatedAt", + ], + ["a number where a string belongs", 12, "title"], + ["a boolean where a string belongs", false, "title"], + ])( + "answers 400 for a cursor holding %s, without reaching Postgres", + async (_why, value, column) => { + try { + await service().findMany({ + orderBy: { column: column as never, order: "asc" }, + query: { cursor: tampered(value, column), first: "5" }, + }); + } catch (error) { + // An `HTTPException`, never a `SyntaxError`, a `RangeError` or a + // Postgres cast failure - each of which would surface as a 500. + expect(error).toBeInstanceOf(HTTPException); + expect(statusOf(error)).toBe(400); + + return; + } + + throw new Error(`Expected ${String(value)} to be refused.`); + }, + ); + + /** + * The values a pattern lets through and Postgres does not. + * + * `2026-02-30` has the shape of a timestamp and is not a day, so a shape + * check passes it straight into `'2026-02-30'::timestamp` - and the + * answer to that is `invalid input syntax`, arriving at a client as a 500 + * from a route whose contract says it does not do that. Each of these is + * refused before anything is bound. + */ + it.each([ + ["month 13", "2026-13-01"], + ["month 0", "2026-00-01"], + ["30 February", "2026-02-30"], + ["29 February in a common year", "2025-02-29"], + ["31 April", "2026-04-31"], + ["day 32", "2026-01-32"], + ["hour 24", "2026-08-09 24:00:00"], + ["minute 60", "2026-08-09 23:60:00"], + ["second 61", "2026-08-09 23:59:61"], + ["an impossible offset", "2026-08-09 10:00:00+25:00"], + ["an offset with 99 minutes", "2026-08-09 10:00:00+12:99"], + ])( + "answers 400 for a cursor holding %s, which Postgres would refuse", + async (_why, value) => { + await seed(NON_MONOTONIC); + + try { + await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { cursor: tampered(value), first: "5" }, + }); + } catch (error) { + expect(error).toBeInstanceOf(HTTPException); + expect(statusOf(error)).toBe(400); + // Specifically not a Postgres error wearing a different hat. + expect((error as Error).message).not.toMatch( + /invalid input syntax/i, + ); + + return; + } + + throw new Error(`Expected ${value} to be refused.`); + }, + ); + + it("still answers a leap day, which is a real one", async () => { + // The other half of the check: refusing impossible values must not + // refuse possible ones. 2024 is a leap year and 2024-02-29 exists. + const ids = await seed([ + { + code: "leap", + title: "Leap", + updatedAt: new Date("2024-02-29T10:00:00Z"), + }, + { + code: "after", + title: "After", + updatedAt: new Date("2024-03-01T10:00:00Z"), + }, + ]); + + const cursor = Buffer.from( + JSON.stringify({ + column: "updatedAt", + id: ids[0], + value: "2024-02-29 10:00:00", + }), + ).toString("base64url"); + + const page = await service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { cursor, first: "5" }, + }); + + expect(page.edges.map(row => row.id)).toEqual([ids[1]]); + }); + + it("leaves the table alone when a cursor tries to inject SQL", async () => { + await seed(NON_MONOTONIC); + + await expect( + service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { + cursor: tampered("2026-08-09'; DROP TABLE example_articles; --"), + first: "5", + }, + }), + ).rejects.toBeInstanceOf(HTTPException); + + await expect(service().findMany()).resolves.toMatchObject({ + pageInfo: { totalCount: 3 }, + }); + }); + + it("refuses a cursor whose identifier is not a positive integer", async () => { + const zeroId = Buffer.from( + JSON.stringify({ column: "updatedAt", id: 0, value: null }), + ).toString("base64url"); + + await expect( + service().findMany({ + orderBy: { column: "updatedAt" as never, order: "asc" }, + query: { cursor: zeroId, first: "5" }, + }), + ).rejects.toBeInstanceOf(HTTPException); + }); + + it.each([ + ["first=0", { first: "0" }], + ["last=0", { last: "0" }], + ["first=-1", { first: "-1" }], + ["last=-1", { last: "-1" }], + ["first=abc", { first: "abc" }], + ["last=abc", { last: "abc" }], + ["first=1.5", { first: "1.5" }], + ["both first and last", { first: "5", last: "5" }], + ["cursor=garbage", { cursor: "!!!not-a-cursor!!!" }], + ])("answers 400 for %s", async (_why, query) => { + await expect400(query); + }); + + it("does not turn first=0 into a one-row page", async () => { + // What it used to do: clamp to a limit of one, return a row, and report + // `hasNextPage: true` for a page nobody asked for. + await seed(NON_MONOTONIC); + + await expect400({ first: "0" }); + }); + + it("caps a page at the maximum rather than trusting the caller", async () => { + const rows: Seed[] = Array.from({ length: 3 }, (_row, index) => ({ + code: `cap-${index}`, + title: `Cap ${index}`, + updatedAt: new Date(2026, 0, 1 + index), + })); + await seed(rows); + + const page = await service().findMany({ query: { first: "100000" } }); + + expect(page.edges).toHaveLength(3); + }); + }); + + // ------------------------------------------------------------------------- + // The public list, which is the anonymous half of the same machinery + // ------------------------------------------------------------------------- + + it("walks the public list with a non-monotonic publication order", async () => { + const build = articleContent.publicService; + if (!build) throw new Error("no public service"); + + await seed([ + { + code: "p1", + publishedAt: new Date("2026-03-01"), + title: "Zulu", + updatedAt: new Date("2026-01-01"), + }, + { + code: "p2", + publishedAt: new Date("2026-01-01"), + title: "Alpha", + updatedAt: new Date("2026-01-02"), + }, + { + code: "p3", + publishedAt: new Date("2026-02-01"), + title: "Mike", + updatedAt: new Date("2026-01-03"), + }, + ]); + + const seen: number[] = []; + let cursor: string | undefined; + for (let page = 0; page < 10; page += 1) { + const result = await build(h.context).findMany({ + orderBy: { column: "publishedAt" as never, order: "desc" }, + query: { cursor, first: "1" }, + }); + seen.push(...result.edges.map(row => Number(row.publishedAt))); + if (!result.pageInfo.hasNextPage) break; + cursor = result.pageInfo.endCursor ?? undefined; + } + + expect(seen).toHaveLength(3); + // Newest first, which is the ordering the route asked for. + expect([...seen].sort((a, b) => b - a)).toEqual(seen); + expect(CONFIG_PLUGIN.pluginId).toBe("@vitnode/example"); + }); + }, +); diff --git a/plugins/example/src/database/performance-postgres.test.ts b/plugins/example/src/database/performance-postgres.test.ts new file mode 100644 index 000000000..39acf3afc --- /dev/null +++ b/plugins/example/src/database/performance-postgres.test.ts @@ -0,0 +1,777 @@ +import type { SearchDocument } from "@vitnode/core/api/models/search"; +import type { Context } from "hono"; + +import { + createContentLocalizedSearchIndexer, + createContentSearchIndexer, +} from "@vitnode/core/content/server"; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from "vitest"; + +import { CONFIG_PLUGIN } from "@/const"; + +import type { ContentTestHarness } from "./harness"; + +import { advancedArticleContent } from "./advanced-articles"; +import { articleContent } from "./articles"; +import { + ACTOR, + clearContentTables, + createContentTestHarness, + DATABASE_TEST_URL, +} from "./harness"; +import { localizedArticleContent } from "./localized-articles"; + +/** + * Pagination, query counts and index use, at a scale that can tell them apart. + * + * Nothing here measures milliseconds. A wall-clock number in CI says more about + * the machine than about the code, and it fails on a busy runner for reasons + * nobody can act on. What is measured instead is **algorithmic**: how many + * round trips one page costs, whether that number moves when the page grows, + * and whether a lookup seeks on an index or reads the whole table. + * + * The dataset is deliberately modest - a few thousand rows rather than the ten + * thousand the plan suggests - because the properties under test are visible at + * any size above "a handful", and a suite nobody waits for is a suite nobody + * runs. Where scale genuinely matters (a sequential scan is cheaper than an + * index on a tiny table, so the planner picks it) the fixture is grown until + * the planner has a real choice to make. + */ + +const PAGE = 25; +/** Enough rows that the planner prefers an index over a sequential scan. */ +const SCALE = 2_000; + +let h: ContentTestHarness; +let categoryId = 0; + +const editorial = (on: Context) => { + const build = articleContent.editorialService; + if (!build) throw new Error("example.article has no editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const localizedService = (on: Context) => { + const build = localizedArticleContent.localizedService; + if (!build) throw new Error("no localized service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const translationEditorial = (on: Context) => { + const build = localizedArticleContent.translationEditorialService; + if (!build) throw new Error("no translation editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +/** + * Bulk-inserts published articles straight through SQL. + * + * The service would write one row per statement and one revision alongside it, + * which at this scale is minutes rather than seconds - and none of these tests + * are about the write path. + */ +let seeded = 0; + +const seedArticles = async (count: number): Promise => { + const from = seeded + 1; + seeded += count; + await h.sql` + INSERT INTO "example_articles" + ("title", "slug", "code", "category", "status", "publishedAt", "version") + SELECT + 'Article ' || i, + 'article-' || i, + 'code-' || i, + ${categoryId}, + 'published', + -- Ascending with the identifier, which is what a real collection looks + -- like: rows are published roughly in the order they were created. The + -- cursor is the identifier, so an order column that moves against it + -- cannot page exactly - see the pagination docs. + now() - ((100000 - i) || ' seconds')::interval, + 1 + FROM generate_series(${from}::int, ${seeded}::int) AS i + `; + await h.sql`ANALYZE "example_articles"`; +}; + +const plan = async (query: string): Promise => { + const rows = await h.sql.unsafe(`EXPLAIN ${query}`); + + return rows.map(row => String(row["QUERY PLAN"])).join("\n"); +}; + +const indexesOn = async (table: string) => + await h.sql<{ indexdef: string; indexname: string }[]>` + SELECT indexname, indexdef FROM pg_indexes WHERE tablename = ${table} + ORDER BY indexname + `; + +/** + * Statements the counted connection issued while `run` was in flight. + * + * The call is made **twice** and only the second is counted. `postgres` + * prepares a statement the first time it sees its shape and reuses it + * afterwards, so a cold call and a warm one legitimately issue different + * numbers of protocol messages - and comparing a cold count against a warm one + * would report a difference that has nothing to do with the query plan. + */ +const countQueries = async (run: () => Promise): Promise => { + await run(); + h.counted.reset(); + await run(); + + return [...h.counted.queries]; +}; + +describe.skipIf(!DATABASE_TEST_URL)("Content Engine at scale", () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + }); + + beforeEach(async () => { + await clearContentTables(h.sql); + h.reset(); + seeded = 0; + + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Scale') RETURNING "id" + `; + categoryId = category.id; + }); + + // ------------------------------------------------------------------------- + // Cursor pagination + // ------------------------------------------------------------------------- + + describe("cursor pagination", () => { + it("answers an empty collection without a cursor", async () => { + const page = await articleContent.service(h.context).findMany(); + + expect(page.edges).toEqual([]); + expect(page.pageInfo).toMatchObject({ + endCursor: null, + hasNextPage: false, + startCursor: null, + totalCount: 0, + }); + }); + + it("answers a single row without offering a next page", async () => { + await seedArticles(1); + + const page = await articleContent.service(h.context).findMany(); + + expect(page.edges).toHaveLength(1); + expect(page.pageInfo.hasNextPage).toBe(false); + }); + + it("stops exactly at the page boundary", async () => { + // The off-by-one that matters: with exactly `first` rows there is no next + // page, and with one more there is. + await seedArticles(PAGE); + const exact = await articleContent + .service(h.context) + .findMany({ query: { first: String(PAGE) } }); + expect(exact.edges).toHaveLength(PAGE); + expect(exact.pageInfo.hasNextPage).toBe(false); + + await seedArticles(1); + const overflowing = await articleContent + .service(h.context) + .findMany({ query: { first: String(PAGE) } }); + expect(overflowing.pageInfo.hasNextPage).toBe(true); + }); + + it("walks every row exactly once across many pages", async () => { + await seedArticles(103); + const seen: number[] = []; + let cursor: string | undefined; + + for (let page = 0; page < 20; page += 1) { + const result = await articleContent.service(h.context).findMany({ + query: { cursor, first: String(PAGE) }, + }); + seen.push(...result.edges.map(row => row.id)); + if (!result.pageInfo.hasNextPage) break; + cursor = result.pageInfo.endCursor ?? undefined; + } + + expect(seen).toHaveLength(103); + expect(new Set(seen).size).toBe(103); + }); + + it("never repeats a row because something was inserted between pages", async () => { + // A cursor is a position in an ordering, not a snapshot. Rows that arrive + // behind the cursor are simply not seen; the guarantee is that nothing + // already returned comes back a second time. + await seedArticles(60); + const first = await articleContent + .service(h.context) + .findMany({ query: { first: String(PAGE) } }); + + await seedArticles(10); + + const second = await articleContent.service(h.context).findMany({ + query: { + cursor: first.pageInfo.endCursor ?? undefined, + first: String(PAGE), + }, + }); + + const overlap = second.edges + .map(row => row.id) + .filter(id => first.edges.some(row => row.id === id)); + expect(overlap).toEqual([]); + }); + + it("does not loop forever when rows are deleted between pages", async () => { + await seedArticles(60); + const first = await articleContent + .service(h.context) + .findMany({ query: { first: String(PAGE) } }); + + // Everything after the first page. The cursor is opaque now, so the + // boundary is the last identifier the page actually handed back - the + // list is newest-first, so "after" means a smaller identifier. + const boundary = first.edges.at(-1)?.id ?? 0; + await h.sql` + DELETE FROM "example_articles" WHERE "id" < ${boundary} + `; + + const second = await articleContent.service(h.context).findMany({ + query: { + cursor: first.pageInfo.endCursor ?? undefined, + first: String(PAGE), + }, + }); + + expect(second.edges).toEqual([]); + expect(second.pageInfo.hasNextPage).toBe(false); + }); + + it("caps a public page however large a caller asks for", async () => { + // An anonymous caller controls `first`, so the ceiling has to be the + // server's rather than theirs. + await seedArticles(200); + const service = articleContent.publicService; + if (!service) throw new Error("no public service"); + + const page = await service(h.context).findMany({ + query: { first: "10000" }, + }); + + expect(page.edges.length).toBeLessThanOrEqual(100); + }); + }); + + // ------------------------------------------------------------------------- + // Query counts + // ------------------------------------------------------------------------- + + describe("query counts stay bounded per page", () => { + /** + * Upper bounds rather than exact numbers. + * + * A planner change, a different Postgres major or a Drizzle release can all + * move the exact count by one without anything being wrong. What must never + * move is the *shape*: a page of 25 and a page of 100 cost the same number + * of round trips, and that is what an N+1 would break. + * + * The bounds came down by one when the cursor value moved into the page + * query's own projection. There is no boundary-row lookup left to pay for - + * and that saving is the same change that closed the window another writer + * could edit the boundary through. + */ + const boundedAcrossPageSizes = async ( + run: (size: number) => Promise, + bound: number, + ) => { + const small = await countQueries(async () => await run(5)); + const large = await countQueries(async () => await run(60)); + + expect(small.length).toBeLessThanOrEqual(bound); + expect(large.length).toBeLessThanOrEqual(bound); + // The invariant an N+1 breaks: twelve times the rows, the same number of + // statements. + expect(large.length).toBe(small.length); + }; + + it("keeps the admin list bounded, labels included", async () => { + await seedArticles(200); + + await boundedAcrossPageSizes( + async size => + await articleContent + .service(h.counted.context) + .findMany({ query: { first: String(size) } }), + 3, + ); + }); + + it("keeps the public list bounded", async () => { + await seedArticles(200); + const service = articleContent.publicService; + if (!service) throw new Error("no public service"); + + await boundedAcrossPageSizes( + async size => + await service(h.counted.context).findMany({ + query: { first: String(size) }, + }), + 3, + ); + }); + + it("keeps a localized public list bounded across languages", async () => { + const service = localizedArticleContent.publicService; + if (!service) throw new Error("no public service"); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + + for (let index = 0; index < 30; index += 1) { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: `Body ${index}`, title: `Localized ${index}` }, + }, + { actor: ACTOR }, + ); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: `Tresc ${index}`, title: `Polski ${index}` }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + await translationEditorial(h.context).publish(row.id, "en", { + actor: ACTOR, + }); + await translationEditorial(h.context).publish(row.id, "pl", { + actor: ACTOR, + }); + } + + await boundedAcrossPageSizes( + async size => + await service(h.counted.context).findMany({ + locale: "pl", + query: { first: String(size) }, + }), + 5, + ); + }); + + it("loads a page of advanced collections in batches, not per row", async () => { + // The whole reason a to-many relation is absent from `ContentSelect`: a + // list that carried one would issue a query per row. + const service = advancedArticleContent.publicService; + if (!service) throw new Error("no public service"); + const base = advancedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + const categories = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") + VALUES ('A'), ('B') RETURNING "id" + `; + const translations = advancedArticleContent.translationEditorialService; + if (!translations) throw new Error("no translation editorial service"); + + for (let index = 0; index < 20; index += 1) { + const created = await base(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).create( + { categories: categories.map(row => row.id) }, + { + actor: ACTOR, + }, + ); + await base(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).repeatable.faq.set( + created.row.id, + [ + { answer: "Answer one", question: `Question one ${index}` }, + { answer: "Answer two", question: `Question two ${index}` }, + ], + { actor: ACTOR, expectedVersion: created.version }, + ); + await translations(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).create( + created.row.id, + "en", + { title: `Advanced ${index}` }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + created.row.id, + { actor: ACTOR }, + ); + await translations(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).publish(created.row.id, "en", { actor: ACTOR }); + } + + const small = await countQueries( + async () => + await service(h.counted.context).findMany({ + locale: "en", + query: { first: "3" }, + }), + ); + const large = await countQueries( + async () => + await service(h.counted.context).findMany({ + locale: "en", + query: { first: "20" }, + }), + ); + + expect(large.length).toBe(small.length); + // Two exposed collections - `categories` and `faq` - so two batch reads + // for the whole page, however many rows are on it. + expect( + large.filter(query => + query.includes("example_advanced_articles_categories"), + ), + ).toHaveLength(1); + expect( + large.filter(query => query.includes("example_advanced_articles_faq")), + ).toHaveLength(1); + }); + + it("fetches no collection the public projection does not expose", async () => { + // `relatedArticles` is private on this content type, so a public read + // must not touch its junction table at all - querying it to discard the + // rows afterwards is work with no answer attached. + const service = advancedArticleContent.publicService; + if (!service) throw new Error("no public service"); + + const queries = await countQueries( + async () => + await service(h.counted.context).findMany({ + locale: "en", + query: { first: "20" }, + }), + ); + + expect( + queries.filter(query => + query.includes("example_advanced_articles_related_articles"), + ), + ).toEqual([]); + }); + + it("keeps a revision history page bounded", async () => { + const created = await editorial(h.context).create( + { category: categoryId, code: "history", title: "History subject" }, + { actor: ACTOR }, + ); + let version = created.version; + for (let index = 0; index < 12; index += 1) { + const outcome = await editorial(h.context).update( + created.row.id, + { title: `History subject ${index}` }, + { actor: ACTOR, expectedVersion: version }, + ); + version = outcome?.version ?? version; + } + + const small = await countQueries( + async () => + await editorial(h.counted.context).revisions.list(created.row.id, { + limit: 2, + }), + ); + const large = await countQueries( + async () => + await editorial(h.counted.context).revisions.list(created.row.id, { + limit: 13, + }), + ); + + expect(large.length).toBe(small.length); + expect(large.length).toBeLessThanOrEqual(2); + }); + }); + + // ------------------------------------------------------------------------- + // Search rebuild + // ------------------------------------------------------------------------- + + describe("the search rebuild reads in batches", () => { + it("keeps a page of the non-localized rebuild to a bounded number of queries", async () => { + await seedArticles(200); + const indexer = createContentSearchIndexer(articleContent, { + pluginId: CONFIG_PLUGIN.pluginId, + }); + + const small = await countQueries( + async () => await indexer.load(h.counted.context, 0, 5), + ); + const large = await countQueries( + async () => await indexer.load(h.counted.context, 0, 100), + ); + + expect(large.length).toBe(small.length); + expect(large.length).toBeLessThanOrEqual(2); + }); + + it("loads a shared repeatable once for a page, not once per locale", async () => { + // The localized rebuild emits one document per published translation, so + // a record with three languages appears three times on a page. Its FAQ is + // shared, and loading it three times would be an N+1 hiding behind a + // correct result. + const base = advancedArticleContent.editorialService; + const translations = advancedArticleContent.translationEditorialService; + if (!base || !translations) throw new Error("no editorial services"); + + for (let index = 0; index < 5; index += 1) { + const created = await base(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).create({}, { actor: ACTOR }); + await base(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).repeatable.faq.set( + created.row.id, + [{ answer: "Answer", question: `Question ${index}` }], + { actor: ACTOR, expectedVersion: created.version }, + ); + for (const locale of ["en", "pl"] as const) { + await translations(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).create( + created.row.id, + locale, + { title: `Advanced ${locale} ${index}` }, + { actor: ACTOR }, + ); + await translations(h.context, { + pluginId: CONFIG_PLUGIN.pluginId, + }).publish(created.row.id, locale, { actor: ACTOR }); + } + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + created.row.id, + { actor: ACTOR }, + ); + } + + const indexer = createContentLocalizedSearchIndexer( + advancedArticleContent, + { pluginId: CONFIG_PLUGIN.pluginId }, + ); + + const queries = await countQueries( + async () => await indexer.load(h.counted.context, 0, 50), + ); + + expect( + queries.filter(query => + query.includes("example_advanced_articles_faq"), + ), + ).toHaveLength(1); + expect(queries.length).toBeLessThanOrEqual(4); + }); + + it("pages the localized rebuild by translation, without repeating one", async () => { + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + + for (let index = 0; index < 6; index += 1) { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: `Body ${index}`, title: `Paged ${index}` }, + }, + { actor: ACTOR }, + ); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: `Tresc ${index}`, title: `Polski ${index}` }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + for (const locale of ["en", "pl"] as const) { + await translationEditorial(h.context).publish(row.id, locale, { + actor: ACTOR, + }); + } + } + + const indexer = createContentLocalizedSearchIndexer( + localizedArticleContent, + { pluginId: CONFIG_PLUGIN.pluginId }, + ); + const documents: SearchDocument[] = []; + for (let offset = 0; ;) { + const page = await indexer.load(h.context, offset, 4); + if (page.itemsRead === 0) break; + documents.push(...page.documents); + offset += page.itemsRead; + } + + const keys = documents.map( + document => `${document.itemId}:${document.languageCode ?? ""}`, + ); + expect(keys).toHaveLength(12); + expect(new Set(keys).size).toBe(12); + expect(await indexer.count?.(h.context)).toBe(12); + }); + }); + + // ------------------------------------------------------------------------- + // Indexes and plans + // ------------------------------------------------------------------------- + + describe("the generated indexes exist and are the ones the queries need", () => { + it("indexes the slug uniquely", async () => { + const indexes = await indexesOn("example_articles"); + const slug = indexes.find(entry => entry.indexname.includes("slug")); + + expect(slug?.indexdef).toContain("CREATE UNIQUE INDEX"); + expect(slug?.indexdef).toContain("slug"); + }); + + it("indexes the publication predicate the public list orders by", async () => { + const indexes = await indexesOn("example_articles"); + + expect( + indexes.some( + entry => + entry.indexdef.includes("status") && + entry.indexdef.includes("publishedAt"), + ), + ).toBe(true); + }); + + it("indexes a revision history by record and version", async () => { + const indexes = await indexesOn("core_content_revisions"); + + expect( + indexes.some( + entry => + entry.indexname === "core_content_revisions_item_version_unique", + ), + ).toBe(true); + }); + + it("indexes a junction from both ends", async () => { + const indexes = await indexesOn("example_advanced_articles_categories"); + + // The primary key covers `(itemId, relatedItemId)`, which is what the + // membership `EXISTS` seeks on; the second index covers the reverse + // lookup, which Postgres does not create for a foreign key on its own. + expect(indexes.some(entry => entry.indexname.endsWith("_pk"))).toBe(true); + expect( + indexes.some(entry => entry.indexname.endsWith("_related_item_id_idx")), + ).toBe(true); + }); + + it("indexes a repeatable's position uniquely per parent", async () => { + const indexes = await indexesOn("example_advanced_articles_faq"); + + const position = indexes.find(entry => + entry.indexname.endsWith("_position_key"), + ); + expect(position?.indexdef).toContain("CREATE UNIQUE INDEX"); + expect(position?.indexdef).toContain("position"); + }); + + it("seeks rather than scans for a slug lookup", async () => { + await seedArticles(SCALE); + + const explained = await plan( + `SELECT "id" FROM "example_articles" WHERE "slug" = 'article-1234'`, + ); + + // A unique index over two thousand rows is not a close call for the + // planner, which is why this one is safe to assert. + expect(explained).toContain("Index"); + expect(explained).not.toContain("Seq Scan"); + }); + + it("seeks rather than scans for a lookup by identifier", async () => { + await seedArticles(SCALE); + const [row] = await h.sql<{ id: number }[]>` + SELECT "id" FROM "example_articles" LIMIT 1 + `; + + const explained = await plan( + `SELECT "id" FROM "example_articles" WHERE "id" = ${row.id}`, + ); + + expect(explained).toContain("Index"); + expect(explained).not.toContain("Seq Scan"); + }); + + it("seeks rather than scans for one record's revision history", async () => { + await h.sql` + INSERT INTO "core_content_revisions" + ("pluginId", "contentTypeId", "itemId", "version", "operation", "snapshot") + SELECT + ${CONFIG_PLUGIN.pluginId}, 'example.article', i / 20 + 1, + i % 20 + 1, 'update', '{}'::jsonb + FROM generate_series(1, ${SCALE}::int) AS i + `; + await h.sql`ANALYZE "core_content_revisions"`; + + const explained = await plan( + `SELECT "id" FROM "core_content_revisions" + WHERE "contentTypeId" = 'example.article' AND "itemId" = 7 + AND "languageId" IS NULL + ORDER BY "version" DESC LIMIT 25`, + ); + + expect(explained).not.toContain("Seq Scan"); + }); + }); + + // ------------------------------------------------------------------------- + // Memory + // ------------------------------------------------------------------------- + + describe("reads stay page-bound", () => { + it("never materialises more rows than the page asked for", async () => { + // The property that keeps a large collection usable: a page is a page + // whatever the table holds behind it. + await seedArticles(SCALE); + + const page = await articleContent + .service(h.context) + .findMany({ query: { first: "25" } }); + + expect(page.edges).toHaveLength(25); + expect(page.pageInfo.totalCount).toBe(SCALE); + }); + + it("counts the whole collection without reading it", async () => { + await seedArticles(SCALE); + + const queries = await countQueries( + async () => + await articleContent + .service(h.counted.context) + .findMany({ query: { first: "5" } }), + ); + + // The count is an aggregate, not a fetch: no statement in the page's set + // asks for every row. + expect(queries.some(query => /count\(/i.test(query))).toBe(true); + expect(queries.length).toBeLessThanOrEqual(4); + }); + }); +}); diff --git a/plugins/example/src/database/resilience-postgres.test.ts b/plugins/example/src/database/resilience-postgres.test.ts new file mode 100644 index 000000000..1c0a1a9ac --- /dev/null +++ b/plugins/example/src/database/resilience-postgres.test.ts @@ -0,0 +1,1695 @@ +import type { SearchDocument } from "@vitnode/core/api/models/search"; +import type { Context } from "hono"; + +import { executeContentSchedule } from "@vitnode/core/api/modules/content/helpers/execute-content-schedule"; +import { + contentEditorialEffects, + contentEngineDiagnostics, + contentSearchDrift, + createContentLocalizedSearchIndexer, + createContentSearchIndexer, + runContentScheduleEffects, + syncContentSearch, +} from "@vitnode/core/content/server"; +import { + afterAll, + beforeAll, + beforeEach, + describe, + expect, + it, + vi, +} from "vitest"; + +import { CONFIG_PLUGIN } from "@/const"; +import { articleContentType } from "@/content/article"; + +import type { ContentTestHarness } from "./harness"; + +import { articleContent } from "./articles"; +import { + ACTOR, + clearContentTables, + createContentTestHarness, + DATABASE_TEST_URL, +} from "./harness"; +import { localizedArticleContent } from "./localized-articles"; + +/** + * What happens to a **committed** mutation when the things it has to tell go + * down. + * + * The rule the whole stage rests on: a database write that committed did commit. + * No event transport, search engine or cache origin may undo it, and none of + * them may make the engine report it as having failed. What they *may* do is + * leave the announcement outstanding - and Stage 7's job is to make that + * outstanding state visible and repairable rather than silent. + * + * The three downstream systems fail in different ways, so they are tested + * separately and then together: + * + * | System | Fails by | Repaired by | + * | -------- | ------------------------------ | ------------------------------ | + * | events | reporting `failures` | nothing - at-least-once | + * | search | throwing from `index`/`delete` | the next write, or a rebuild | + * | cache | an origin refusing the POST | the effects task's own retry | + */ + +let h: ContentTestHarness; +let categoryId = 0; +let seq = 0; + +const editorial = (on: Context) => { + const build = articleContent.editorialService; + if (!build) throw new Error("example.article has no editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const localizedService = (on: Context) => { + const build = localizedArticleContent.localizedService; + if (!build) throw new Error("no localized service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const translationEditorial = (on: Context) => { + const build = localizedArticleContent.translationEditorialService; + if (!build) throw new Error("no translation editorial service"); + + return build(on, { pluginId: CONFIG_PLUGIN.pluginId }); +}; + +const article = async () => { + seq += 1; + const outcome = await editorial(h.context).create( + { + category: categoryId, + code: `resilient-${seq}`, + title: `Resilient subject ${seq}`, + }, + { actor: ACTOR }, + ); + + return { id: outcome.row.id, version: outcome.version }; +}; + +const rowOf = async (id: number) => { + const [row] = await h.sql< + { publishedAt: Date | null; status: string; version: number }[] + >` + SELECT "status", "publishedAt", "version" FROM "example_articles" + WHERE "id" = ${id} + `; + + return row; +}; + +/** A published article, ready for the index. */ +const published = async () => { + const created = await article(); + const outcome = await editorial(h.context).publish(created.id, { + actor: ACTOR, + }); + + return { id: created.id, version: outcome?.version ?? created.version }; +}; + +/** + * Writes the canonical index rows a healthy install would hold. + * + * Shared, because "search is fine" is the baseline several tests need before + * they can say anything about a *different* dimension of health. + */ +const indexPublished = async (): Promise => { + const rows = await h.sql<{ id: number; title: string }[]>` + SELECT "id", "title" FROM "example_articles" + WHERE "status" = 'published' AND "publishedAt" IS NOT NULL + `; + for (const row of rows) { + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${articleContentType.id}, ${row.id}, + '', ${row.title}, ${row.title}, now() + ) + `; + } +}; + +const DEAD_LISTENER = { + error: "Service unavailable", + listener: "send-notification", + module: "notifications", + pluginId: CONFIG_PLUGIN.pluginId, +}; + +describe.skipIf(!DATABASE_TEST_URL)("Content Engine failure resilience", () => { + beforeAll(async () => { + h = await createContentTestHarness(); + }, 60_000); + + afterAll(async () => { + await h?.end(); + vi.unstubAllGlobals(); + }); + + beforeEach(async () => { + await clearContentTables(h.sql); + h.reset(); + // A web origin that accepts everything, by default. `originsFor` falls back + // to `NEXT_PUBLIC_WEB_URL` when none is configured, so without this the + // bridge would try to reach a real host and every scheduled run would fail + // on the cache rather than on the thing under test. + vi.stubGlobal( + "fetch", + vi.fn( + async () => await Promise.resolve(new Response("ok", { status: 200 })), + ), + ); + + const [category] = await h.sql<{ id: number }[]>` + INSERT INTO "example_categories" ("name") VALUES ('Resilience') + RETURNING "id" + `; + categoryId = category.id; + }); + + // ------------------------------------------------------------------------- + // Events + // ------------------------------------------------------------------------- + + describe("a dead event listener", () => { + it("leaves the write committed and reports the failure", async () => { + h.behaviour.eventFailures = [DEAD_LISTENER]; + const { id, version } = await published(); + + const outcome = await editorial(h.context).update( + id, + { title: "Edited despite the outage" }, + { actor: ACTOR, expectedVersion: version }, + ); + if (!outcome) throw new Error("Expected an outcome."); + + const result = await contentEditorialEffects( + h.context, + articleContentType, + outcome, + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + ); + + // Committed, and readable. + expect((await rowOf(id)).version).toBe(version + 1); + // Visible, rather than swallowed. + expect(result.event?.failures).toHaveLength(1); + expect(h.logs.some(line => line.includes("[content-effects]"))).toBe( + true, + ); + expect(h.logs.join("\n")).toContain("send-notification"); + }); + + it("still writes the search document", async () => { + // Two independent systems: one being down is not a reason to skip the + // other, and by the time either runs the row is already committed. + h.behaviour.eventFailures = [DEAD_LISTENER]; + const { id, version } = await published(); + + const outcome = await editorial(h.context).update( + id, + { title: "Still indexed" }, + { actor: ACTOR, expectedVersion: version }, + ); + if (!outcome) throw new Error("Expected an outcome."); + + await contentEditorialEffects(h.context, articleContentType, outcome, { + model: articleContent, + pluginId: CONFIG_PLUGIN.pluginId, + }); + + expect(h.indexed.map(document => document.itemId)).toContain(id); + }); + + it("logs nothing when every listener received it", async () => { + const { id, version } = await published(); + const outcome = await editorial(h.context).update( + id, + { title: "Quiet" }, + { actor: ACTOR, expectedVersion: version }, + ); + if (!outcome) throw new Error("Expected an outcome."); + + await contentEditorialEffects(h.context, articleContentType, outcome, { + model: articleContent, + pluginId: CONFIG_PLUGIN.pluginId, + }); + + expect(h.logs.filter(line => line.includes("[content-effects]"))).toEqual( + [], + ); + }); + }); + + // ------------------------------------------------------------------------- + // Search + // ------------------------------------------------------------------------- + + describe("a search engine that is down", () => { + it("never rolls the database write back", async () => { + const { id, version } = await published(); + h.behaviour.searchError = new Error("elasticsearch unreachable"); + + const outcome = await editorial(h.context).update( + id, + { title: "Written while search was down" }, + { actor: ACTOR, expectedVersion: version }, + ); + + expect(outcome?.changed).toBe(true); + const [row] = await h.sql<{ title: string }[]>` + SELECT "title" FROM "example_articles" WHERE "id" = ${id} + `; + expect(row.title).toBe("Written while search was down"); + }); + + it("reports the failure on the outcome and in the log", async () => { + const { id } = await published(); + h.behaviour.searchError = new Error("elasticsearch unreachable"); + + const result = await syncContentSearch(h.context, articleContentType, { + changedFields: ["title"], + operation: "update", + pluginId: CONFIG_PLUGIN.pluginId, + row: { ...(await rowOf(id)), id, slug: "x", title: "T" }, + }); + + expect(result.error?.message).toBe("elasticsearch unreachable"); + expect(h.logs.some(line => line.includes("[content-search]"))).toBe(true); + }); + + it("is repaired by the next successful write", async () => { + // "Eventually consistent, bounded by the next publish or the next + // rebuild" - the first half of that, shown. + const { id, version } = await published(); + h.behaviour.searchError = new Error("down"); + + const first = await editorial(h.context).update( + id, + { title: "Lost to the outage" }, + { actor: ACTOR, expectedVersion: version }, + ); + await syncContentSearch(h.context, articleContentType, { + changedFields: first?.changedFields, + operation: "update", + pluginId: CONFIG_PLUGIN.pluginId, + row: first?.row ?? {}, + }); + expect(h.indexed).toHaveLength(0); + + h.behaviour.searchError = null; + const second = await editorial(h.context).update( + id, + { title: "Recovered" }, + { actor: ACTOR, expectedVersion: first?.version ?? version }, + ); + await syncContentSearch(h.context, articleContentType, { + changedFields: second?.changedFields, + operation: "update", + pluginId: CONFIG_PLUGIN.pluginId, + row: second?.row ?? {}, + }); + + expect(h.indexed.at(-1)?.title).toBe("Recovered"); + }); + }); + + // ------------------------------------------------------------------------- + // Scheduled effects, where all three meet + // ------------------------------------------------------------------------- + + describe("scheduled effects", () => { + const schedules = (on: Context) => { + const model = editorial(on).schedules; + if (!model) throw new Error("example.article has no scheduling"); + + return model; + }; + + /** Books a publish that is already due, runs it, and returns the payload. */ + const runTransition = async () => { + const created = await article(); + const booked = await schedules(h.context).schedule({ + action: "publish", + actorUserId: null, + itemId: created.id, + scheduledFor: new Date(Date.now() - 1000), + }); + + await executeContentSchedule(h.context, { + generation: booked.generation, + scheduleId: booked.id, + }); + + const [queued] = await h.sql<{ payload: Record }[]>` + SELECT "payload" FROM "core_queue" + WHERE "name" = 'content-schedule-effects' + ORDER BY "id" DESC LIMIT 1 + `; + + return { + id: created.id, + payload: queued.payload as Parameters< + typeof runContentScheduleEffects + >[1], + scheduleId: booked.id, + }; + }; + + const effectsErrorOf = async (scheduleId: number) => { + const [row] = await h.sql<{ effectsError: null | string }[]>` + SELECT "effectsError" FROM "core_content_schedules" + WHERE "id" = ${scheduleId} + `; + + return row.effectsError; + }; + + it("delivers everything on a healthy run and records no error", async () => { + const { payload, scheduleId } = await runTransition(); + + const outcome = await runContentScheduleEffects(h.context, payload); + + expect(outcome.status).toBe("delivered"); + expect(await effectsErrorOf(scheduleId)).toBeNull(); + expect(h.emitted.map(entry => entry.name)).toContain( + "content.example.article.published", + ); + }); + + it("fails the run and records why when the event transport reports a failure", async () => { + const { payload, scheduleId } = await runTransition(); + h.behaviour.eventFailures = [DEAD_LISTENER]; + + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(/committed, but its effects did not/); + + const error = await effectsErrorOf(scheduleId); + expect(error).toContain("event:"); + expect(error).toContain("send-notification"); + }); + + it("fails the run when the search write is refused", async () => { + const { payload, scheduleId } = await runTransition(); + h.behaviour.searchError = new Error("index refused"); + + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + + expect(await effectsErrorOf(scheduleId)).toContain("search:"); + }); + + it("reports every outstanding failure, not just the first", async () => { + // The whole point of combining them: an operator looking at one line has + // to see everything that is still outstanding, or they will fix one + // system, retry, and discover the next. + const { payload, scheduleId } = await runTransition(); + h.behaviour.eventFailures = [DEAD_LISTENER]; + h.behaviour.searchError = new Error("index refused"); + h.behaviour.revalidateOrigins = ["http://web-a.invalid"]; + vi.stubGlobal( + "fetch", + vi.fn( + async () => + await Promise.resolve(new Response("no", { status: 500 })), + ), + ); + + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + + const error = await effectsErrorOf(scheduleId); + expect(error).toContain("event:"); + expect(error).toContain("search:"); + expect(error).toContain("cache:"); + }); + + it("treats a partial cache delivery as a failure, not a success", async () => { + // Two web apps behind one API: one of them accepting an unpublish while + // the other does not leaves the withdrawn page cached and readable. + const { payload, scheduleId } = await runTransition(); + h.behaviour.revalidateOrigins = [ + "http://web-a.invalid", + "http://web-b.invalid", + ]; + vi.stubGlobal( + "fetch", + vi.fn(async (input: string | URL) => { + const url = input instanceof URL ? input.href : input; + + return await Promise.resolve( + url.includes("web-a") + ? new Response("ok", { status: 200 }) + : new Response("no", { status: 500 }), + ); + }), + ); + + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + + expect(await effectsErrorOf(scheduleId)).toContain( + "1/2 web origins accepted", + ); + }); + + it("never re-runs the transition when the effects are retried", async () => { + const { id, payload, scheduleId } = await runTransition(); + const before = await rowOf(id); + + h.behaviour.searchError = new Error("index refused"); + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + + h.behaviour.searchError = null; + const retried = await runContentScheduleEffects(h.context, payload); + + expect(retried.status).toBe("delivered"); + // Same version, same publication timestamp: the retry announced the + // transition again, it did not perform it again. + expect(await rowOf(id)).toEqual(before); + expect(await effectsErrorOf(scheduleId)).toBeNull(); + + const revisions = await h.sql<{ id: number }[]>` + SELECT "id" FROM "core_content_revisions" + WHERE "itemId" = ${id} AND "operation" = 'publish' + `; + expect(revisions).toHaveLength(1); + }); + + it("re-emits the event on a retry, which is why delivery is at-least-once", async () => { + const { payload } = await runTransition(); + + h.behaviour.searchError = new Error("index refused"); + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + + h.behaviour.searchError = null; + await runContentScheduleEffects(h.context, payload); + + const published = h.emitted.filter( + entry => entry.name === "content.example.article.published", + ); + // Twice, with the same `scheduleId` both times - which is the key a + // listener that must act once uses. There is no outbox and no + // exactly-once claim. + expect(published).toHaveLength(2); + expect( + published.map( + entry => (entry.payload as { scheduleId: number }).scheduleId, + ), + ).toEqual([payload.scheduleId, payload.scheduleId]); + }); + + it("indexes the same document on a retry, so the repeat is harmless", async () => { + const { payload } = await runTransition(); + + h.behaviour.eventFailures = [DEAD_LISTENER]; + await expect( + runContentScheduleEffects(h.context, payload), + ).rejects.toThrow(); + const first = h.indexed.at(-1); + + h.behaviour.eventFailures = []; + await runContentScheduleEffects(h.context, payload); + const second = h.indexed.at(-1); + + // An upsert is the same operation however many times it runs, and the + // document it writes is byte-identical. + expect(second).toEqual(first); + }); + + it("gives up rather than retrying forever when the content type is gone", async () => { + const { payload, scheduleId } = await runTransition(); + + const outcome = await runContentScheduleEffects(h.context, { + ...payload, + contentTypeId: "example.removed-by-an-uninstall", + }); + + expect(outcome.status).toBe("unregistered"); + expect(await effectsErrorOf(scheduleId)).toContain( + "no longer registered", + ); + }); + }); + + // ------------------------------------------------------------------------- + // Idempotency + // ------------------------------------------------------------------------- + + describe("idempotency", () => { + it("makes a second publish a no-op with no revision and no event", async () => { + const { id } = await published(); + h.reset(); + + const outcome = await editorial(h.context).publish(id, { actor: ACTOR }); + await contentEditorialEffects( + h.context, + articleContentType, + outcome ?? ({} as never), + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + ); + + expect(outcome?.changed).toBe(false); + expect(outcome?.revisionId).toBeNull(); + expect(h.emitted).toEqual([]); + expect(h.indexed).toEqual([]); + }); + + it("makes a second unpublish a no-op", async () => { + const { id } = await published(); + await editorial(h.context).unpublish(id, { actor: ACTOR }); + const before = await rowOf(id); + h.reset(); + + const outcome = await editorial(h.context).unpublish(id, { + actor: ACTOR, + }); + + expect(outcome?.changed).toBe(false); + expect(await rowOf(id)).toEqual(before); + expect(h.emitted).toEqual([]); + }); + + it("makes a restore to the values already stored a no-op", async () => { + const created = await article(); + const history = await editorial(h.context).revisions.list(created.id); + const only = history.edges[0]; + + const outcome = await editorial(h.context).restore(created.id, only.id, { + actor: ACTOR, + expectedVersion: created.version, + }); + + expect(outcome?.changed).toBe(false); + expect(outcome?.revisionId).toBeNull(); + expect((await rowOf(created.id)).version).toBe(created.version); + }); + + it("bumps no version for a relation add that is already there", async () => { + const created = await article(); + + const outcome = await editorial(h.context).update( + created.id, + { title: `Resilient subject ${seq}` }, + { actor: ACTOR, expectedVersion: created.version }, + ); + + expect(outcome?.changed).toBe(false); + expect((await rowOf(created.id)).version).toBe(created.version); + }); + }); + + // ------------------------------------------------------------------------- + // Search consistency + // ------------------------------------------------------------------------- + + describe("live synchronisation and rebuild agree", () => { + const indexer = () => + createContentSearchIndexer(articleContent, { + pluginId: CONFIG_PLUGIN.pluginId, + }); + + const rebuild = async (limit = 50): Promise => { + const documents: SearchDocument[] = []; + const build = indexer(); + for (let offset = 0; ;) { + const page = await build.load(h.context, offset, limit); + if (page.itemsRead === 0) break; + documents.push(...page.documents); + offset += page.itemsRead; + } + + return documents; + }; + + it("reproduces the live document byte for byte", async () => { + const created = await article(); + const outcome = await editorial(h.context).publish(created.id, { + actor: ACTOR, + }); + // The live path is the effects layer, not the transition: publishing + // writes the row, and the announcement writes the document. + await contentEditorialEffects( + h.context, + articleContentType, + outcome ?? ({} as never), + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + ); + const id = created.id; + const live = h.indexed.find(document => document.itemId === id); + expect(live).toBeDefined(); + + // The live path indexes on publish; the rebuild reads the same row + // through a different query. Equality is the invariant. + const rebuilt = (await rebuild()).find( + document => document.itemId === id, + ); + + expect(rebuilt).toEqual(live); + }); + + it("pages a rebuild without skipping or repeating a record", async () => { + const ids: number[] = []; + for (let index = 0; index < 7; index += 1) { + const { id } = await published(); + ids.push(id); + } + + const documents = await rebuild(2); + + const ascending = (a: number, b: number) => a - b; + expect( + documents.map(document => document.itemId).sort(ascending), + ).toEqual([...ids].sort(ascending)); + expect(new Set(documents.map(document => document.itemId)).size).toBe( + ids.length, + ); + }); + + /** + * The rebuild walks by key, not by offset. + * + * `OFFSET` counts rows in a set that is *moving*: a record unpublished after + * page one shifts everything behind it forward by one, and the next + * `OFFSET 100` steps straight over a row nobody ever indexed. A rebuild that + * silently misses rows is the failure a rebuild exists to fix. + */ + describe("while the collection changes underneath it", () => { + /** Reads one page at a time so the fixture can mutate between them. */ + const pager = () => { + const build = indexer(); + let offset = 0; + + return async (limit: number) => { + const page = await build.load(h.context, offset, limit); + offset += page.itemsRead; + + return page; + }; + }; + + it("visits every remaining row when an already-read one is unpublished", async () => { + // The regression, exactly: page one is read, one of *its* rows goes + // away, and the walk continues. With `OFFSET` the next page would start + // one row too far in and skip an untouched record forever. + const ids: number[] = []; + for (let index = 0; index < 10; index += 1) { + const { id } = await published(); + ids.push(id); + } + + const next = pager(); + const first = await next(5); + expect(first.itemsRead).toBe(5); + + // A row from the page just read is withdrawn. + await h.sql` + UPDATE "example_articles" SET "status" = 'draft' + WHERE "id" = ${ids[2]} + `; + + const seen = first.documents.map(document => document.itemId); + for (let page = 0; page < 10; page += 1) { + const result = await next(5); + if (result.itemsRead === 0) break; + seen.push(...result.documents.map(document => document.itemId)); + } + + // Every row, exactly once. The withdrawn one is in there because page + // one had already read it - the live unpublish is what removes its + // document, and that is a different mechanism. What matters here is + // that nothing *else* moved: with `OFFSET` the shift would have stepped + // over an untouched record and lost it for the whole rebuild. + expect(seen.sort((a, b) => a - b)).toEqual( + [...ids].sort((a, b) => a - b), + ); + expect(new Set(seen).size).toBe(ids.length); + }); + + it("simply never reaches a row unpublished before it got there", async () => { + const ids: number[] = []; + for (let index = 0; index < 10; index += 1) { + const { id } = await published(); + ids.push(id); + } + + const next = pager(); + await next(4); + + // Withdrawn while it is still ahead of the cursor. + await h.sql` + UPDATE "example_articles" SET "status" = 'draft' + WHERE "id" = ${ids[8]} + `; + + const seen: number[] = []; + for (let page = 0; page < 10; page += 1) { + const result = await next(4); + if (result.itemsRead === 0) break; + seen.push(...result.documents.map(document => document.itemId)); + } + + expect(seen).not.toContain(ids[8]); + // And nothing near it was disturbed. + expect(seen).toContain(ids[9]); + expect(seen).toContain(ids[7]); + }); + + /** + * A row published mid-rebuild with a **higher** identifier is picked up by + * the same pass, because the cursor has not reached it yet. One with a + * lower identifier is not - the walk is already past that point. + * + * That is the honest consequence of a keyset walk, and it is stated here + * rather than described as a snapshot: a rebuild is not one. + */ + it("picks up a row published ahead of the cursor, and not one behind it", async () => { + const ids: number[] = []; + for (let index = 0; index < 6; index += 1) { + const { id } = await published(); + ids.push(id); + } + + const next = pager(); + const first = await next(3); + expect(first.itemsRead).toBe(3); + + // One behind the cursor, one ahead of it. + const behind = await article(); + await h.sql` + UPDATE "example_articles" + SET "status" = 'published', "publishedAt" = now(), "id" = ${ids[0] - 1} + WHERE "id" = ${behind.id} + `; + const ahead = await published(); + + const seen: number[] = []; + for (let page = 0; page < 10; page += 1) { + const result = await next(3); + if (result.itemsRead === 0) break; + seen.push(...result.documents.map(document => document.itemId)); + } + + expect(seen).toContain(ahead.id); + expect(seen).not.toContain(ids[0] - 1); + }); + + it("issues no SQL OFFSET at all", async () => { + // The property, asserted against the statements the driver really sent. + await published(); + await published(); + await published(); + + const build = createContentSearchIndexer(articleContent, { + pluginId: CONFIG_PLUGIN.pluginId, + }); + + h.counted.reset(); + let offset = 0; + for (let page = 0; page < 5; page += 1) { + const result = await build.load(h.counted.context, offset, 2); + if (result.itemsRead === 0) break; + offset += result.itemsRead; + } + + expect(h.counted.queries).not.toHaveLength(0); + expect( + h.counted.queries.filter(query => /\boffset\b/i.test(query)), + ).toEqual([]); + // And it does seek by key instead. + expect( + h.counted.queries.some(query => /"id"\s*>\s*\$/.test(query)), + ).toBe(true); + }); + + it("restarts from the beginning when a fresh rebuild begins", async () => { + // `offset === 0` is the contract's only "this is a new pass" signal. + const ids: number[] = []; + for (let index = 0; index < 4; index += 1) { + const { id } = await published(); + ids.push(id); + } + + const build = indexer(); + await build.load(h.context, 0, 2); + await build.load(h.context, 2, 2); + + const restarted = await build.load(h.context, 0, 2); + + expect(restarted.documents.map(document => document.itemId)).toEqual( + ids.slice(0, 2), + ); + }); + }); + + it("counts exactly the records it would index", async () => { + await published(); + await published(); + await article(); // a draft, which is never indexed + + expect(await indexer().count?.(h.context)).toBe(2); + expect(await rebuild()).toHaveLength(2); + }); + }); + + describe("stale documents are cleaned up", () => { + it("removes a record's document when it is unpublished", async () => { + const { id, version } = await published(); + h.reset(); + + const outcome = await editorial(h.context).unpublish(id, { + actor: ACTOR, + expectedVersion: version, + }); + await contentEditorialEffects( + h.context, + articleContentType, + outcome ?? ({} as never), + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + ); + + expect(h.deleted).toContainEqual({ + itemId: id, + itemType: articleContentType.id, + locale: undefined, + }); + }); + + it("removes it when the record is deleted", async () => { + const { id, version } = await published(); + h.reset(); + + const outcome = await editorial(h.context).delete(id, { + actor: ACTOR, + expectedVersion: version, + }); + await contentEditorialEffects( + h.context, + articleContentType, + outcome ?? ({} as never), + { model: articleContent, pluginId: CONFIG_PLUGIN.pluginId }, + ); + + expect(h.deleted.map(entry => entry.itemId)).toContain(id); + }); + + it("removes only the language a translation was taken down in", async () => { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "English body", title: "Stale Cleanup" }, + }, + { actor: ACTOR }, + ); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: "Tresc", title: "Polski" }, + { actor: ACTOR }, + ); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + await translationEditorial(h.context).publish(row.id, "en", { + actor: ACTOR, + }); + await translationEditorial(h.context).publish(row.id, "pl", { + actor: ACTOR, + }); + h.reset(); + + const outcome = await translationEditorial(h.context).unpublish( + row.id, + "pl", + { actor: ACTOR }, + ); + const { contentTranslationEffects } = + await import("@vitnode/core/content/server"); + await contentTranslationEffects( + h.context, + localizedArticleContent.definition, + outcome ?? ({} as never), + { + model: localizedArticleContent, + pluginId: CONFIG_PLUGIN.pluginId, + }, + ); + + // One language out, the other left exactly where it was. + expect(h.deleted).toEqual([ + { + itemId: row.id, + itemType: localizedArticleContent.definition.id, + locale: "pl", + }, + ]); + }); + }); + + // ------------------------------------------------------------------------- + // Drift diagnostics + // ------------------------------------------------------------------------- + + describe("index drift is diagnosable", () => { + it("reports a healthy index as healthy", async () => { + await published(); + await published(); + await indexPublished(); + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift).toMatchObject({ + canonicalHealthy: true, + canonicalIndexedTotal: 2, + contentTypeId: articleContentType.id, + expectedTotal: 2, + healthy: true, + }); + expect(drift.provider.indexedTotal).toBe(2); + expect(drift.locales).toEqual([ + { + canonicalHealthy: true, + canonicalIndexed: 2, + expected: 2, + locale: "", + providerHealthy: true, + providerIndexed: 2, + }, + ]); + }); + + it("reports the bundled provider as verified without counting twice", async () => { + // Its store *is* `core_search_index`, so the canonical counts are its + // counts - asking the same table again would cost a query to learn + // something already known. + await published(); + await indexPublished(); + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.provider).toEqual({ + healthy: true, + // Reused from the canonical count rather than queried again. + indexedTotal: 1, + name: "postgres", + verified: true, + }); + }); + + it("reports a document the index never received", async () => { + await published(); + await published(); + await indexPublished(); + // A live sync that threw, simulated at the row level. + await h.sql`DELETE FROM "core_search_index" WHERE "id" = ( + SELECT MIN("id") FROM "core_search_index" + )`; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.healthy).toBe(false); + expect(drift.canonicalHealthy).toBe(false); + expect(drift.locales[0]).toMatchObject({ + canonicalIndexed: 1, + expected: 2, + }); + }); + + it("reports a document that outlived its record", async () => { + await published(); + await indexPublished(); + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${articleContentType.id}, 999999, + '', 'Ghost', 'Ghost', now() + ) + `; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + // More documents than records is drift in the other direction, and it is + // reported as measured rather than clamped - a stale document is exactly + // what an operator needs to see. + expect(drift.healthy).toBe(false); + expect(drift.locales[0]).toMatchObject({ + canonicalIndexed: 2, + expected: 1, + }); + }); + + /** + * The regression the whole provider split exists for. + * + * `SearchModel.index` writes the canonical row and *then* hands the document + * to the provider. An Elasticsearch that refuses the second half leaves a + * canonical table that is perfectly correct and a search box that is missing + * results - and a diagnostic that only ever looked at the canonical table + * would call that healthy. + */ + it("reports the provider unhealthy when only the provider is missing a document", async () => { + await published(); + await published(); + await indexPublished(); + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { byLocale: new Map([["", 1]]), total: 1 }; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.canonicalHealthy).toBe(true); + expect(drift.locales[0]).toMatchObject({ + canonicalHealthy: true, + canonicalIndexed: 2, + expected: 2, + providerHealthy: false, + providerIndexed: 1, + }); + expect(drift.provider).toMatchObject({ + healthy: false, + name: "elasticsearch", + verified: true, + }); + // The part that used to be wrong: a healthy canonical table is not a + // healthy search. + expect(drift.healthy).toBe(false); + }); + + /** + * The other direction, and the one per-locale counts cannot see. + * + * Deletion runs canonical-first: `SearchModel.delete` removes the row and + * then asks the provider. If the provider's half fails, the document + * survives in a locale that no longer appears in the database *or* the + * canonical table - so the locale list, which is built from those two, never + * thinks to ask about it. Only an unfiltered total can find it. + */ + describe("a document that exists only in the provider", () => { + it("is caught on a content type with nothing in it at all", async () => { + // The empty case matters on its own: a localized content type with no + // published translations enumerates *no* locales, so `[].every(...)` is + // `true` and a ghost would sail straight through on the per-locale + // checks alone. + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { byLocale: new Map(), total: 1 }; + + const drift = await contentSearchDrift(h.context, { + model: localizedArticleContent, + }); + + expect(drift.locales).toEqual([]); + expect(drift.expectedTotal).toBe(0); + expect(drift.canonicalIndexedTotal).toBe(0); + expect(drift.canonicalHealthy).toBe(true); + expect(drift.provider).toMatchObject({ + healthy: false, + indexedTotal: 1, + verified: true, + }); + expect(drift.healthy).toBe(false); + }); + + it("is caught on a non-localized content type with no rows either", async () => { + // Here one locale *is* enumerated - the empty one - and it agrees on + // both sides. The total is still the thing that catches the ghost. + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { byLocale: new Map(), total: 1 }; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.locales).toEqual([ + { + canonicalHealthy: true, + canonicalIndexed: 0, + expected: 0, + locale: "", + providerHealthy: true, + providerIndexed: 0, + }, + ]); + expect(drift.provider).toMatchObject({ + healthy: false, + indexedTotal: 1, + }); + expect(drift.healthy).toBe(false); + }); + + it("is caught when every locale it does enumerate agrees", async () => { + // The proof that the total is doing the work: `""` matches on both + // sides, so per-locale parity is perfect and the total is not. + await published(); + await indexPublished(); + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { + byLocale: new Map([["", 1]]), + total: 2, + }; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.locales).toEqual([ + { + canonicalHealthy: true, + canonicalIndexed: 1, + expected: 1, + locale: "", + providerHealthy: true, + providerIndexed: 1, + }, + ]); + expect(drift.canonicalHealthy).toBe(true); + expect(drift.provider).toMatchObject({ + healthy: false, + indexedTotal: 2, + }); + expect(drift.healthy).toBe(false); + }); + + it("is caught in a locale the content type no longer has", async () => { + // EN is published and agrees everywhere. PL exists only in the + // provider - no translation, no canonical row, no expectation - so it + // is never enumerated, and the total is the only thing that sees it. + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "English body", title: "Ghost Subject" }, + }, + { actor: ACTOR }, + ); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + await translationEditorial(h.context).publish(row.id, "en", { + actor: ACTOR, + }); + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${localizedArticleContent.definition.id}, + ${row.id}, 'en', 'Ghost Subject', 'Ghost Subject', now() + ) + `; + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { + // Only `en` is ever asked for, and it agrees. + byLocale: new Map([["en", 1]]), + total: 2, + }; + + const drift = await contentSearchDrift(h.context, { + model: localizedArticleContent, + }); + + expect(drift.locales.map(entry => entry.locale)).toEqual(["en"]); + expect(drift.locales[0].providerHealthy).toBe(true); + expect(drift.canonicalHealthy).toBe(true); + expect(drift.expectedTotal).toBe(1); + expect(drift.provider.indexedTotal).toBe(2); + expect(drift.provider.healthy).toBe(false); + expect(drift.healthy).toBe(false); + }); + + it("makes the whole engine report unhealthy", async () => { + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { byLocale: new Map(), total: 1 }; + + const report = await contentEngineDiagnostics(h.context); + expect(report.contentTypes).not.toHaveLength(0); + + expect(report.searchHealthy).toBe(false); + expect(report.healthy).toBe(false); + }); + + it("still reports healthy when the total agrees as well", async () => { + // The control: same provider, same enumeration, honest total. + await published(); + await indexPublished(); + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { + byLocale: new Map([["", 1]]), + total: 1, + }; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.provider).toMatchObject({ + healthy: true, + indexedTotal: 1, + verified: true, + }); + expect(drift.healthy).toBe(true); + }); + }); + + it("reports a canonical row in a locale nothing expects", async () => { + // The canonical side has the same failure mode, and the grouped query + // already sees every locale the table holds - so the total closes it too. + await published(); + await indexPublished(); + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${articleContentType.id}, 424242, + 'de', 'Ghost', 'Ghost', now() + ) + `; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.expectedTotal).toBe(1); + expect(drift.canonicalIndexedTotal).toBe(2); + expect(drift.canonicalHealthy).toBe(false); + expect(drift.healthy).toBe(false); + }); + + it("reports a provider that cannot be counted as unverified, not healthy", async () => { + await published(); + await indexPublished(); + + h.behaviour.providerName = "custom-search"; + h.behaviour.providerCounts = "unsupported"; + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + expect(drift.canonicalHealthy).toBe(true); + expect(drift.locales[0].providerHealthy).toBeNull(); + expect(drift.locales[0].providerIndexed).toBeNull(); + expect(drift.provider).toEqual({ + healthy: null, + indexedTotal: null, + name: "custom-search", + verified: false, + }); + // Absence of evidence is not a clean bill of health. + expect(drift.healthy).toBe(false); + }); + + it("stays usable when the provider itself is unavailable", async () => { + await published(); + await indexPublished(); + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { byLocale: new Map([["", 1]]), total: 1 }; + h.behaviour.providerCountError = new Error("connect ECONNREFUSED"); + + const drift = await contentSearchDrift(h.context, { + model: articleContent, + }); + + // It answers rather than throwing: a diagnostic that crashes when the + // thing it diagnoses is broken is a diagnostic nobody can use. + expect(drift.canonicalHealthy).toBe(true); + expect(drift.provider).toMatchObject({ + error: "connect ECONNREFUSED", + healthy: false, + verified: true, + }); + expect(drift.healthy).toBe(false); + expect(h.logs.some(line => line.includes("[content-diagnostics]"))).toBe( + true, + ); + }); + + it("keeps the whole status route answering when the provider is down", async () => { + await published(); + await indexPublished(); + h.behaviour.providerCounts = { byLocale: new Map([["", 1]]), total: 1 }; + h.behaviour.providerCountError = new Error("elasticsearch unavailable"); + + const report = await contentEngineDiagnostics(h.context); + + expect(report.contentTypes).not.toHaveLength(0); + expect(report.searchHealthy).toBe(false); + expect(report.healthy).toBe(false); + }); + + it("counts a localized content type per locale", async () => { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "English body", title: "Drift Subject" }, + }, + { actor: ACTOR }, + ); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: "Tresc", title: "Polski Drift" }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + await translationEditorial(h.context).publish(row.id, "en", { + actor: ACTOR, + }); + await translationEditorial(h.context).publish(row.id, "pl", { + actor: ACTOR, + }); + + // Only English made it into the index. + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${localizedArticleContent.definition.id}, + ${row.id}, 'en', 'Drift Subject', 'Drift Subject', now() + ) + `; + + const drift = await contentSearchDrift(h.context, { + model: localizedArticleContent, + }); + + expect(drift.healthy).toBe(false); + expect(drift.locales).toEqual([ + { + canonicalHealthy: true, + canonicalIndexed: 1, + expected: 1, + locale: "en", + providerHealthy: true, + providerIndexed: 1, + }, + { + canonicalHealthy: false, + canonicalIndexed: 0, + expected: 1, + locale: "pl", + providerHealthy: false, + providerIndexed: 0, + }, + ]); + }); + + it("reports one locale unhealthy when only that locale is missing from the provider", async () => { + // The localized shape of the same regression: English agrees everywhere, + // Polish is in the canonical table and absent from the provider. A single + // total cannot show that; a per-locale provider count can. + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "English body", title: "Locale Drift" }, + }, + { actor: ACTOR }, + ); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: "Tresc", title: "Polski Locale" }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + for (const locale of ["en", "pl"] as const) { + await translationEditorial(h.context).publish(row.id, locale, { + actor: ACTOR, + }); + await h.sql` + INSERT INTO "core_search_index" + ("pluginId", "itemType", "itemId", "languageCode", "title", "content", "createdAt") + VALUES ( + ${CONFIG_PLUGIN.pluginId}, ${localizedArticleContent.definition.id}, + ${row.id}, ${locale}, 'Locale Drift', 'Locale Drift', now() + ) + `; + } + + h.behaviour.providerName = "elasticsearch"; + h.behaviour.providerCounts = { + byLocale: new Map([ + ["en", 1], + ["pl", 0], + ]), + total: 1, + }; + + const drift = await contentSearchDrift(h.context, { + model: localizedArticleContent, + }); + + expect(drift.canonicalHealthy).toBe(true); + expect( + drift.locales.map(entry => [entry.locale, entry.providerHealthy]), + ).toEqual([ + ["en", true], + ["pl", false], + ]); + expect(drift.healthy).toBe(false); + }); + + it("agrees with the localized rebuild about how many documents there should be", async () => { + const { row } = await localizedService(h.context).create( + { + shared: {}, + translation: { body: "English body", title: "Parity Subject" }, + }, + { actor: ACTOR }, + ); + const base = localizedArticleContent.editorialService; + if (!base) throw new Error("no editorial service"); + await translationEditorial(h.context).create( + row.id, + "pl", + { body: "Tresc", title: "Polski Parity" }, + { actor: ACTOR }, + ); + await base(h.context, { pluginId: CONFIG_PLUGIN.pluginId }).publish( + row.id, + { actor: ACTOR }, + ); + await translationEditorial(h.context).publish(row.id, "en", { + actor: ACTOR, + }); + await translationEditorial(h.context).publish(row.id, "pl", { + actor: ACTOR, + }); + + const build = createContentLocalizedSearchIndexer( + localizedArticleContent, + { pluginId: CONFIG_PLUGIN.pluginId }, + ); + const drift = await contentSearchDrift(h.context, { + model: localizedArticleContent, + }); + + // The diagnostic and the indexer have to agree about "published", or the + // health check would be measuring something the rebuild does not produce. + expect( + drift.locales.reduce((sum, entry) => sum + entry.expected, 0), + ).toBe(await build.count?.(h.context)); + }); + + it("summarises every registered content type, with schedule failures", async () => { + const { id } = await published(); + const model = editorial(h.context).schedules; + if (!model) throw new Error("no scheduling"); + const booked = await model.schedule({ + action: "unpublish", + actorUserId: null, + itemId: id, + scheduledFor: new Date(Date.now() + 3_600_000), + }); + await h.sql` + UPDATE "core_content_schedules" + SET "effectsError" = 'search: down' + WHERE "id" = ${booked.id} + `; + + const report = await contentEngineDiagnostics(h.context); + const entry = report.contentTypes.find( + item => item.contentTypeId === articleContentType.id, + ); + + expect(report.contentTypes.map(item => item.contentTypeId)).toEqual([ + "example.advanced-article", + "example.article", + "example.category", + "example.localized-article", + ]); + expect(entry?.features).toMatchObject({ + editorial: true, + localization: false, + publicApi: true, + scheduling: true, + search: true, + }); + expect(entry?.schedules).toEqual({ + failedEffects: 1, + pending: 1, + withErrors: 0, + }); + // A content type with no search indexes nothing, so it has no drift to + // report rather than a drift of zero. + expect( + report.contentTypes.find( + item => item.contentTypeId === "example.category", + )?.search, + ).toBeNull(); + }); + }); + + // ------------------------------------------------------------------------- + // Overall health + // ------------------------------------------------------------------------- + + /** + * `healthy: true` beside `failedEffects: 15` is worse than no answer - it + * tells an operator to stop looking. So the report carries the two dimensions + * separately and derives the headline from them. + */ + describe("overall health", () => { + const bookSchedule = async (itemId: number) => { + const model = editorial(h.context).schedules; + if (!model) throw new Error("no scheduling"); + + return await model.schedule({ + action: "unpublish", + actorUserId: null, + itemId, + scheduledFor: new Date(Date.now() + 3_600_000), + }); + }; + + it("is healthy when search agrees and nothing is outstanding", async () => { + const report = await contentEngineDiagnostics(h.context); + + expect(report).toMatchObject({ + effectsHealthy: true, + healthy: true, + searchHealthy: true, + }); + }); + + it("treats a pending schedule as normal rather than unhealthy", async () => { + // It has not fired yet. Nothing is wrong. + const { id } = await published(); + await indexPublished(); + await bookSchedule(id); + + const report = await contentEngineDiagnostics(h.context); + + expect(report.effectsHealthy).toBe(true); + expect(report.healthy).toBe(true); + expect( + report.contentTypes.find( + item => item.contentTypeId === articleContentType.id, + )?.schedules?.pending, + ).toBe(1); + }); + + it("treats a pending schedule whose last attempt threw as still pending", async () => { + // The transition has not happened and the queue is retrying it, so this + // is visible - `withErrors` - without being a failure of the engine. + const { id } = await published(); + await indexPublished(); + const booked = await bookSchedule(id); + await h.sql` + UPDATE "core_content_schedules" + SET "lastError" = 'connection reset' + WHERE "id" = ${booked.id} + `; + + const report = await contentEngineDiagnostics(h.context); + + expect(report.effectsHealthy).toBe(true); + expect(report.healthy).toBe(true); + expect( + report.contentTypes.find( + item => item.contentTypeId === articleContentType.id, + )?.schedules?.withErrors, + ).toBe(1); + }); + + it("is unhealthy when a committed transition was never announced", async () => { + // The record *is* published and nobody was told. No amount of waiting + // fixes that on its own, so it is the one that moves the headline. + const { id } = await published(); + await indexPublished(); + const booked = await bookSchedule(id); + await h.sql` + UPDATE "core_content_schedules" + SET "effectsError" = 'search: down' + WHERE "id" = ${booked.id} + `; + + const report = await contentEngineDiagnostics(h.context); + + expect(report).toMatchObject({ + effectsHealthy: false, + healthy: false, + // Search is fine; the headline is not, and the two are separable. + searchHealthy: true, + }); + }); + + it("is unhealthy when search drifts even though nothing is outstanding", async () => { + await published(); + // Nothing indexed at all, so the canonical table disagrees. + + const report = await contentEngineDiagnostics(h.context); + + expect(report).toMatchObject({ + effectsHealthy: true, + healthy: false, + searchHealthy: false, + }); + }); + }); +});