Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions .claude/skills/build-plugin/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Create a TaskCreate task for each phase. The flow deploys early and tests as it
- [ ] **Phase 3** β€” Scaffold files (icon, file structure, `docs/README.md`)
- [ ] **Phase 4** β€” Write `metadata.json`, `ui.json`, `configValidation.json` + its backing stream β€” the deployable **shell** β†’ [metadata.md](references/metadata.md), [ui.md](references/ui.md)
- [ ] **Checkpoint A** β€” Deploy the shell and authenticate (invoke `deploy-plugin`, probe auth) β†’ [checkpoints.md](references/checkpoints.md)
- [ ] **Phase 5** β€” Write import definitions and import streams; test each in parallel sub-agents β†’ [index-defs.md](references/index-defs.md), [test-agent.md](references/test-agent.md)
- [ ] **Phase 5** β€” Write import definitions and import streams; test each in parallel sub-agents. Repeat per dependency level if any step `dependsOn` another β†’ [index-defs.md](references/index-defs.md), [test-agent.md](references/test-agent.md)
- [ ] **Checkpoint B** β€” Redeploy, then trigger + await the import via the CLI so objects exist β†’ [checkpoints.md](references/checkpoints.md)
- [ ] **Phase 6** β€” Build + test data streams in parallel sub-agents β†’ [test-agent.md](references/test-agent.md), [data-streams.md](references/data-streams.md)
- [ ] **Phase 7** β€” Build OOB default content in a sub-agent (it reads [oob-content.md](references/oob-content.md))
Expand Down Expand Up @@ -79,7 +79,7 @@ This phase produces a written plan and a user-approval gate before any files are
### The plan must cover

1. **Object types** β€” Every type that should appear in the SquaredUp graph. These go in `objectTypes` in `metadata.json` and as `sourceType` throughout.
2. **Import steps** β€” Let the API shape dictate: one step returning many types, or separate steps per type.
2. **Import steps** β€” Let the API shape dictate: one step returning many types, or separate steps per type. If an object type is only listable in the context of an already-imported parent (the API has "list X for parent Y" but no "list all X"), plan it as a **dependent step** instead β€” `dependsOn` the parent step and `scope` to its objects; see [index-defs.md](references/index-defs.md#scoped-dependent-steps).
3. **Data streams** β€” For each object type, plan:
- A **summary/current state** stream (`"timeframes": false`, returns current values)
- A **history/metrics** stream (supports timeframes, returns time-series rows)
Expand Down Expand Up @@ -244,9 +244,21 @@ Do not proceed to Phase 5 until auth is confirmed. See [checkpoints.md](referenc

## Phase 5: Import definitions & import streams

Write `indexDefinitions/default.json` and the unscoped list/import streams it calls β€” these are coupled (the index steps reference the stream columns), so author them here in the main agent. Read [index-defs.md](references/index-defs.md) and [data-streams.md](references/data-streams.md).
Write `indexDefinitions/default.json` and its import streams β€” these are coupled (the index steps reference the stream columns), so author them here in the main agent. Read [index-defs.md](references/index-defs.md) and [data-streams.md](references/data-streams.md).

Then **test the import streams in parallel sub-agents** rather than inline β€” the raw paged response bodies are large and the streams are independent. Spawn **one test-mode sub-agent per import stream, all in a single message**, with `model: "sonnet"` (this is run-and-report testing, not deep authoring), passing the `--plugin-id <id> --datasource-id <id>` captured at Checkpoint A. Each sub-agent tests its (already-written) unscoped stream, confirms it returns one flat row per object, and returns a compact report (per [test-agent.md](references/test-agent.md)). Fix any stream a sub-agent flags before Checkpoint B.
**Root steps first** β€” the ones with no `dependsOn`. These call a global/unscoped list endpoint and don't wait on anything. Build them, then **test them in parallel sub-agents** rather than inline β€” the raw paged response bodies are large and these streams are independent of each other. Spawn **one test-mode sub-agent per stream, all in a single message**, with `model: "sonnet"` (this is run-and-report testing, not deep authoring), passing the `--plugin-id <id> --datasource-id <id>` captured at Checkpoint A. Each sub-agent tests its (already-written) unscoped stream, confirms it returns one flat row per object, and returns a compact report (per [test-agent.md](references/test-agent.md)). Fix any stream a sub-agent flags before Checkpoint B.

### Dependent steps (`dependsOn` + `scope`)

Skip this if the Phase 2 plan has no dependent steps β€” go straight to Checkpoint B.

A dependent step's stream is scoped (like a Phase 6 data stream), so it can't be tested until the objects its `scope.query` needs actually exist in the graph. Build one **dependency depth level at a time**:

1. Write that level's step(s) and stream(s) in `indexDefinitions/default.json`.
2. Test each using the same **scoped** procedure [test-agent.md](references/test-agent.md) already uses in Phase 6 (`squaredup objects` to find a real object of the depended-on type, then `squaredup test <stream> --object <id>`) β€” those objects come from the level below, already landed by its own Checkpoint B run.
Comment thread
andrewmumblebee marked this conversation as resolved.
3. Run [Checkpoint B](#checkpoint-b-redeploy--run-the-first-import) again to land this level's objects before building the next.

Repeat per level until every dependent step is built, tested, and imported β€” one dependent step needs a single extra pass; a three-level chain needs two.

### The reconciliation pass

Expand All @@ -256,7 +268,7 @@ Sub-agents run blind to each other, so several can independently rediscover β€”
2. **Propagate every API-level discovery to all sibling streams** that share the endpoint family or scoping β€” timeframe/granularity limits that 404, payload caps that 500, object property/id names, auth quirks. A constraint one sub-agent hit and fixed almost always applies to its siblings too; apply the same edit to each affected stream and **re-test every stream you changed** (re-spawn its sub-agent β€” a propagated edit is unproven until tested).
3. **Resolve conflicting assumptions** before continuing β€” if two reports name the same object property or id differently (e.g. one filters on `projectId`, another on `rawId`), determine the correct one against the real response and fix every stream that used the wrong one, so none ships with a scope filter comparing `undefined === undefined`. Do not proceed with an unresolved contradiction.

Run this pass at the end of both Phase 5 and Phase 6. If reconciliation edits an `indexDefinitions/*.json` mapping or an import stream, that edit only shapes a **future** import. Here in Phase 5 the import hasn't run yet, so the Checkpoint B import picks it up naturally β€” nothing is stale, no extra re-index needed.
Run this pass at the end of each Phase 5 build (the root pass, and each dependent-step level) and at the end of Phase 6. If reconciliation edits an `indexDefinitions/*.json` mapping or an import stream, that edit only shapes a **future** import. Here in Phase 5, reconciliation happens before that pass's own Checkpoint B run, so the next import picks it up naturally β€” nothing is stale, no extra re-index needed.

---

Expand Down
28 changes: 28 additions & 0 deletions .claude/skills/build-plugin/references/index-defs.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,22 @@ Defines what gets imported into the SquaredUp graph.
{ "deviceType": "dbusServiceType" }
]
}
},
{
"name": "deviceAlerts",
"dataStream": { "name": "deviceAlerts" },
"scope": {
"query": "g.V().has(\"sourceType\", \"Router\")"
},
"timeframe": "none",
"objectMapping": {
"id": "uid",
"name": "message",
"type": { "value": "Device Alert" },
"properties": ["severity", "raisedAt", "deviceId"]
},
"optional": true,
"dependsOn": ["devices"]
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}
]
}
Expand All @@ -51,6 +67,18 @@ The stream called by an import step must return one flat row per object with at

---

## Scoped, dependent steps

A step can be scoped to objects a *previous* step already imported, instead of calling a global list endpoint. Use this when an object type is only listable in the context of a parent (the API has "list alerts for device X", not "list all alerts") β€” the `deviceAlerts` step above is one:

- **`scope.query`** β€” a gremlin query run against the tenant's graph. It selects which already-imported objects this step's `dataStream` runs against. The stream must be a scoped stream (like a Phase 6 data stream, not the unscoped kind root steps use) whose `matches` covers the same `sourceType` the query selects β€” use `httpRequestScopedSingle` when the API takes one parent at a time (one request per matched object), or `httpRequestScoped` to receive all matched objects in one call via `{{objects}}`; see [data-streams.md](data-streams.md). The value you filter on must be a `sourceType` the dependency actually produces β€” a fixed `{ "value": ... }` type, or one of the values a dynamic type column resolves to (here, one of the values the `devices` step's `deviceType` type column can return).
- **`dependsOn`** β€” names of steps that must finish (succeed or warn) before this one runs, so the objects `scope.query` needs already exist. Omit it (or leave it `[]`) for a root step, which runs immediately. It can list more than one step, and chains can be more than one level deep (`c` depends on `b`, which depends on `a`) β€” ordering across the whole graph is resolved automatically, so just name whichever step(s) must land first. Names must match real steps and can't form a cycle β€” both are rejected at deploy time.
- **`optional`** β€” when `true`, a failed step is recorded as a `warning` instead of a hard failure, which still counts as a green light for anything depending on it. Set it on a dependent step whose data isn't essential to a successful import, so one failing sub-resource doesn't cancel every step after it.

Building and testing a dependent step needs objects from its dependency to already be in the graph β€” see SKILL.md Phase 5 for the build/test order.

---

## Stream-level requirements

Import streams are ordinary data stream files in `dataStreams/` β€” they need `name`, `displayName`, `description`, and `tags` like any other stream. Mark import-only streams as hidden so they don't clutter the tile editor:
Expand Down
Loading