Sync fixes: partial pushes were erasing data Chronicle had, and resync stopped at 500

API-CONTRACT.md

@@ -214,9 +214,55 @@ Creates a new entity.
 Returns a single entity with full content.
 
 #### PUT /entities/:entityId
-Updates an entity.
+Updates an entity. **PARTIAL update** — see the contract below.
 
-**Request:** Same shape as POST (partial updates supported).
+##### The partial-update contract (Chronicle sweep R4, 2026-08-07)
+
+Every JSON update endpoint in Chronicle now reads a request body three ways:
+
+| what the body does with a key | what happens to the stored value |
+|---|---|
+| **absent** | **preserved** |
+| present with a **value** | replaced |
+| present and **explicitly `null`** | cleared (nullable columns only) |
+
+The distinction is real on the server, not incidental: the request structs
+bind `patch.Field[T]`, which records presence during JSON decoding, so
+"absent" and `null` are different things. Non-nullable columns have no
+cleared state, so an explicit `null` on one of those preserves rather than
+writing a zero.
+
+This **replaced** the previous behaviour, which had no contract at all — it
+was whatever each Go field's type happened to do. Pointer fields preserved
+on absence; value-typed fields (`string`, `bool`, `int`) wrote their zero.
+Two consequences were live on this module's own traffic:
+
+- **`is_private` was value-typed.** `actor-sync.mjs` pushes `{name}` alone
+  on a rename, which bound `is_private = false` and **published a hidden
+  character entity to every player in the campaign.** The old wording of
+  this document said absent meant public. Nobody designed that; it was the
+  Go zero value being read as an intention.
+- **`parent_id` was not on the request struct at all**, so every update
+  from this module detached the entity from the Chronicle hierarchy.
+
+Both are fixed on the server. Send only the fields you mean to change.
+
+**Request:** any subset of the POST shape, plus `parent_id`:
+
+```json
+{ "name": "Renamed Character" }
+```
+
+```json
+{ "parent_id": null }
+```
+  ↑ explicitly unparents. Omitting `parent_id` leaves the parent alone.
+
+> **Version skew.** A module talking to a Chronicle older than sweep R4 still
+> gets the old whole-replace behaviour. Where a client-side echo already
+> exists for that reason — `ChronicleMarkerConfigDialog.#onSave` spreads the
+> stored marker under the edited fields — it is kept: it is harmless against
+> a merging server and load-bearing against an old one.
 
 #### DELETE /entities/:entityId
 Deletes an entity.
@@ -253,9 +299,13 @@ Updates entity permissions.
 #### POST /entities/:entityId/reveal
 Toggles entity reveal state (NPC reveal to players). Body is exactly
 `{ "is_private": <bool> }` (a `*bool`); an explicit value matching the current
-state is a no-op. This is the ONLY correct way to flip visibility from the
-module — a bare `PUT /entities/:id` with just `{is_private}` 400s because
-UpdateEntity requires a name.
+state is a no-op. This remains the correct way to flip visibility from the
+module.
+
+> Post-sweep-R4, a bare `PUT /entities/:id` with just `{is_private}` no longer
+> 400s — an absent name means "not editing the name" rather than "empty name".
+> Keep using `/reveal` anyway: it is the named, single-purpose route, and it
+> works against every Chronicle version this module supports.
 
 **Request:**
 ```json
@@ -620,7 +670,15 @@ Lists map markers (pins/notes on the map).
 Creates a map marker.
 
 #### PUT /maps/:mapId/markers/:markerId
-Updates a map marker.
+Updates a map marker. **PARTIAL update** — absent preserves, an explicit
+`null` clears, a present value replaces (see "The partial-update contract"
+under `PUT /entities/:entityId`).
+
+`foundry_id` is this module's pairing key. It stays clearable HERE — send
+`{"foundry_id": null}` to unpair — while Chronicle's own web marker form,
+which never sends the key, can no longer NULL it by omission. That omission
+used to unpair every marker a GM edited in the browser, which showed up later
+as duplicate markers on the next sync.
 
 #### DELETE /maps/:mapId/markers/:markerId
 Deletes a map marker.
@@ -852,7 +910,17 @@ Creates a calendar event.
 - `recurrence_max_occurrences` — Maximum number of recurrences
 
 #### PUT /calendar/events/:eventId
-Updates a calendar event. Same fields as POST.
+Updates a calendar event. Same fields as POST, but it is a **PARTIAL update**
+— absent preserves, an explicit `null` clears, a present value replaces (see
+"The partial-update contract" under `PUT /entities/:entityId`).
+
+This is the endpoint `calendar-sync.mjs` pushes note edits to, from three
+paths (`_onCalendariaNoteUpdated`, `_onLocalEventUpdate`,
+`_onSimpleCalendarNoteUpdate`), each with a five-key body. Before sweep R4
+every one of those pushes also wrote `is_recurring = false`, `all_day = false`
+and a cleared `entity_id`, because those were value-typed / clear-on-nil on
+the server. They are preserved now. Keep the bodies narrow: a Foundry note
+edit means the name, the date and the body, and nothing else.
 
 #### DELETE /calendar/events/:eventId
 Deletes a calendar event.

CLAUDE.md

@@ -102,6 +102,33 @@ Integration — Install & Updates".
   hits a `default:` that logs once per type per session. See
   `scripts/_calendar-subresources.mjs`, `tools/test-calendar-subresources.mjs`,
   `tools/test-calendar-subresource-routing.mjs` (FM-SYNC-SUBRESOURCES-P1).
+- **Chronicle update endpoints are PARTIAL: absent preserves, an explicit
+  `null` clears, a present value replaces** (Chronicle sweep R4, 2026-08-07;
+  API-CONTRACT.md → "The partial-update contract"). Chronicle's request
+  structs bind `patch.Field[T]`, which records presence during decoding, so
+  absent and `null` are genuinely different. **Send only the fields you mean
+  to change**, and do NOT "harden" a narrow body by echoing the untouched
+  fields back: an echo re-arms the endpoint for the next writer and goes
+  stale — that is how `ChronicleMarkerConfigDialog` lost the pairing key. The
+  narrow bodies are pinned by `tools/test-partial-put-contract.mjs`
+  (`actor-sync`'s `{name}` rename push; `calendar-sync`'s three note-edit
+  pushes). Before the contract existed, `{name}` alone bound
+  `is_private=false` and **published a hidden character entity to every
+  player**, and the calendar pushes turned `is_recurring` and `all_day` off.
+  The one surviving echo is the marker dialog's spread, kept deliberately:
+  harmless against a merging server, load-bearing against a pre-R4 one.
+- **Never walk a list with a small hard-coded page cap.** The two places that
+  need every entity in the campaign — `JournalSync.resyncAll` and the
+  dashboard's `_buildEntityGroups` — both had `while (hasMore && page <= 5)`
+  inline, a silent 500-entity ceiling: past it entities were never seen, and
+  the GM got a completed resync and a full-looking dashboard anyway. Both now
+  share `scripts/_entity-page-walk.mjs`, whose bound is 200 pages and whose
+  `truncated` flag MUST be surfaced by the caller. A bound is fine; a bound
+  nobody is told about is the defect. See `tools/test-entity-page-walk.mjs`.
+  Chronicle's server-side twin (`POST /sync` capped at 1000 with no cursor)
+  was fixed in Chronicle sweep R4 stage 18 — that endpoint now returns
+  `next_cursor`, which this module does not yet consume because it pulls via
+  `GET /entities`, not `POST /sync`.
 - WebSocket messages are routed by type through `SyncManager`.
 - Chronicle-side serving rules live in `chronicle-package.json` at repo root; CI validates it against `module.json` via `tools/check-package-descriptor.mjs`.
 

scripts/_entity-page-walk.mjs

@@ -0,0 +1,77 @@
+/**
+ * Shared entity-list page walk.
+ *
+ * Both places that need "every entity in the campaign" — JournalSync.resyncAll
+ * and the dashboard's entity groups — had the same loop inline, and both
+ * stopped after five pages:
+ *
+ *     while (hasMore && page <= 5) { … per_page=100 … }
+ *
+ * That is a hard 500-entity ceiling with no signal. Past it, entities were
+ * not synced late or partially; they were never seen at all, and the GM was
+ * shown a completed resync and a full-looking dashboard. Chronicle's own sync
+ * pull carried the matching ceiling on the server side, fixed in sweep R4
+ * stage 18; fixing that half and leaving this one would have left the
+ * operator exactly as stuck.
+ *
+ * This module is pure — no Foundry globals, no api-client import — so it is
+ * unit-testable and both callers share one implementation instead of two that
+ * drift. See tools/test-entity-page-walk.mjs.
+ */
+
+/** Page size used for entity list requests. Matches Chronicle's list default. */
+export const ENTITY_PAGE_SIZE = 100;
+
+/**
+ * Upper bound on pages walked in one pass: 200 pages x 100 = 20,000 entities.
+ *
+ * A bound is still wanted — a broken server that always answers with a full
+ * page would otherwise spin forever — but it is set where no real campaign
+ * reaches it, and unlike the old cap, hitting it is REPORTED rather than
+ * silently swallowed. A ceiling nobody is told about is the actual defect;
+ * the number is secondary.
+ */
+export const MAX_ENTITY_PAGES = 200;
+
+/**
+ * Walk the entity list to exhaustion.
+ *
+ * @param {(page: number, perPage: number) => Promise<unknown>} fetchPage
+ *   Fetches one page. Receives the 1-based page number and the page size.
+ * @param {(raw: unknown) => Array<object>} normalize
+ *   Unwraps the response into an array. Chronicle returns some list endpoints
+ *   bare and some enveloped, so every caller must unwrap defensively; the
+ *   caller passes in whichever unwrapper it already owns.
+ * @param {{pageSize?: number, maxPages?: number}} [opts]
+ * @returns {Promise<{entities: Array<object>, truncated: boolean, pages: number}>}
+ *   `truncated` is true only when the walk stopped at maxPages with a full
+ *   page still coming back — i.e. entities certainly exist that this pass did
+ *   not see. Callers must surface it.
+ */
+export async function walkEntityPages(fetchPage, normalize, opts = {}) {
+  const pageSize = opts.pageSize || ENTITY_PAGE_SIZE;
+  const maxPages = opts.maxPages || MAX_ENTITY_PAGES;
+
+  const all = [];
+  let page = 1;
+  let truncated = false;
+
+  for (;;) {
+    const batch = normalize(await fetchPage(page, pageSize)) || [];
+    if (batch.length === 0) break;
+    all.push(...batch);
+
+    // A short page is the end of the list. Only a FULL page means there may
+    // be more, which is also why an exact multiple costs one extra request
+    // rather than guessing.
+    if (batch.length < pageSize) break;
+
+    page += 1;
+    if (page > maxPages) {
+      truncated = true;
+      break;
+    }
+  }
+
+  return { entities: all, truncated, pages: page };
+}

scripts/actor-sync.mjs

@@ -517,6 +517,21 @@ export class ActorSync {
 
       // Update name separately if changed, with conflict detection.
       if (change.name) {
+        // A rename means ONE thing, so the body carries one field.
+        //
+        // Chronicle's PUT /entities/:id is a partial update: an absent key
+        // preserves, an explicit null clears, a present value replaces
+        // (API-CONTRACT.md → "The partial-update contract"). Before that
+        // contract existed, `is_private` was a value-typed bool on the
+        // server's request struct, so THIS body — {name} alone — bound
+        // is_private=false and published a hidden character entity to every
+        // player in the campaign. The fix is the server's; what belongs here
+        // is the discipline that made it visible.
+        //
+        // Do NOT "fix" this by echoing is_private / type_label / parent_id
+        // back. Echoing re-arms the endpoint for the next writer and would
+        // reintroduce the same break the moment one of the echoed values is
+        // stale. Visibility has its own route: POST /entities/:id/reveal.
         const nameBody = { name: change.name };
         const chronicleUpdatedAt = actor.getFlag(FLAG_SCOPE, 'chronicleUpdatedAt');
         if (chronicleUpdatedAt) {

scripts/calendar-sync.mjs

@@ -1337,6 +1337,13 @@ export class CalendarSync {
     const date = chronicleDateFromCalendariaStartDate(startDate);
     if (!date) return null;
 
+    // Six keys, and no more. This payload is used for BOTH create and update;
+    // on the update path Chronicle's PUT /calendar/events/:id is a partial
+    // update — absent preserves, explicit null clears, a value replaces
+    // (API-CONTRACT.md). Before that contract, the keys missing here also
+    // wrote is_recurring=false, all_day=false and a cleared entity_id onto
+    // the event. Do not widen this to echo them back: an echo re-arms the
+    // endpoint for the next writer and goes stale.
     return {
       name: name || 'Untitled Note',
       year: date.year,
@@ -1460,6 +1467,13 @@ export class CalendarSync {
     }
 
     try {
+      // Narrow body ON PURPOSE: a Foundry note edit means the name, the date
+      // and the body. Chronicle's PUT /calendar/events/:id is a partial
+      // update — absent preserves, explicit null clears, a value replaces
+      // (API-CONTRACT.md). Before that contract, this exact five-key body
+      // also wrote is_recurring=false, all_day=false and a cleared entity_id
+      // onto the event, because those were value-typed / clear-on-nil on the
+      // server. Do not widen this to echo them back.
       await this._api.put(`/calendar/events/${chronicleId}`, {
         name: eventData.name || 'Untitled Event',
         year: eventData.year,
@@ -1557,6 +1571,8 @@ export class CalendarSync {
     }
 
     try {
+      // Narrow body ON PURPOSE — see _onLocalEventUpdate for why this stays
+      // five keys and must not grow an echo of the event's other columns.
       await this._api.put(`/calendar/events/${chronicleId}`, {
         name: scData.name,
         year: scData.year,

scripts/journal-sync.mjs

@@ -16,6 +16,7 @@ import { _sanitizeIncomingHTML } from './_html-sanitizer.mjs';
 import { defaultLevelForVisibility } from './_ownership.mjs';
 import { isCalendarNoteJournal } from './calendar-sync.mjs';
 import { _isAllowedImageHost, _describeRejection } from './_url-validation.mjs';
+import { walkEntityPages } from './_entity-page-walk.mjs';
 
 /**
  * Validate and resolve a Chronicle entity's `image_path` to a safe src string.
@@ -182,24 +183,21 @@ export class JournalSync {
 
     if (verbose) ui.notifications.info('Chronicle: fetching entities for resync…');
 
-    // Paginated fetch — mirrors the dashboard's _buildEntityGroups page loop.
+    // Paginated fetch — shares the walk with the dashboard's
+    // _buildEntityGroups (scripts/_entity-page-walk.mjs). Both used to stop
+    // after five pages, so a campaign past 500 entities resynced only its
+    // first 500 and reported a clean finish.
     let allEntities = [];
+    let truncated = false;
     try {
-      let page = 1;
-      let hasMore = true;
-      while (hasMore && page <= 5) {
-        const result = await this._api.get(`/entities?per_page=100&page=${page}`);
-        const entities = Array.isArray(result) ? result
+      const walked = await walkEntityPages(
+        (page, perPage) => this._api.get(`/entities?per_page=${perPage}&page=${page}`),
+        (result) => (Array.isArray(result) ? result
           : (Array.isArray(result?.entities) ? result.entities
-          : (Array.isArray(result?.data) ? result.data : []));
-        if (entities.length > 0) {
-          allEntities.push(...entities);
-          hasMore = entities.length === 100;
-          page++;
-        } else {
-          hasMore = false;
-        }
-      }
+          : (Array.isArray(result?.data) ? result.data : []))),
+      );
+      allEntities = walked.entities;
+      truncated = walked.truncated;
     } catch (err) {
       const status = err?.status || null;
       console.warn(`Chronicle JournalSync.resyncAll: GET /entities failed (${status || 'network'})`, err);
@@ -211,6 +209,16 @@ export class JournalSync {
 
     console.debug(`Chronicle: resyncAll fetched ${allEntities.length} entity(ies).`);
 
+    // A truncated walk means entities exist that this pass never saw. Say so
+    // — a resync that quietly covers part of the campaign and reports success
+    // is worse than one that refuses.
+    if (truncated) {
+      console.warn(`Chronicle: resyncAll stopped at ${allEntities.length} entities; the campaign has more.`);
+      ui.notifications.warn(
+        `Chronicle: resync covered the first ${allEntities.length} entities only — the campaign has more. Run it again or narrow your sync exclusions.`
+      );
+    }
+
     // Build a fast lookup of already-linked journals by chronicle entity id.
     const journalByEntityId = new Map();
     for (const j of game.journal.contents) {

scripts/map-viewer.mjs

@@ -1170,7 +1170,17 @@ export class ChronicleMarkerConfigDialog extends HandlebarsApplicationMixin(Appl
     const safeCategory = CHRONICLE_MARKER_CATEGORIES.includes(category) ? category : 'note';
     const safeVisibility = VISIBILITY_VALUES.includes(visibility) ? visibility : 'everyone';
 
+    // `PUT /maps/:id/markers/:mid` is a FULL REPLACE: Chronicle binds the body
+    // into a struct with pointer fields and UPDATEs entity_id, visibility_rules
+    // and foundry_id unconditionally, so any of those keys missing from the
+    // body is written back as NULL — clearing the entity link, the per-user
+    // allow/deny list, and the module's own Foundry pairing key. Spread the
+    // stored marker under the edited fields (as PinConfigDialog.#onSave above
+    // does for local pins); the read-only keys the spread carries along
+    // (id, map_id, created_at, entity_name, …) are undeclared on the wire
+    // struct and ignored by the binder.
     const data = {
+      ...this._marker,
       name,
       description,
       x: this._marker.x,