Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,15 @@ The two are complementary: `deploy-dispatcher-action` stands up the Dispatcher i
your account once; `flare-dispatch-action` runs in each consumer repo's CI to
offload heavy jobs onto it.

```mermaid
flowchart LR
accTitle: Where each action sits
ops["**Operator repo**<br/>deploy-dispatcher-action"] -->|"wrangler deploy"| worker["**Dispatcher Worker**<br/>your Cloudflare account"]
ci["**Consumer repo CI**<br/>flare-dispatch-action"] -->|"signed POST, 202"| worker
worker -->|"check-run verdict"| pr["**Pull request**<br/>flare-dispatch/‹run›"]
class worker accent
```

```yaml
# consumer repo — offload the test suite onto your Dispatcher
- uses: fractalboxdev/flare-dispatch/actions/flare-dispatch-action@<sha>
Expand Down
24 changes: 23 additions & 1 deletion actions/deploy-dispatcher-action/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,29 @@ The action:
3. `pnpm install --frozen-lockfile` in the upstream tree.
4. `wrangler d1 migrations apply <binding> --remote`.
5. `wrangler deploy`.
6. (Optional) Polls `inputs.health-check-url`'s `/health` with backoff.
6. (Optional) Polls `inputs.health-check-url` exactly as given (include the `/health` path) with backoff.

```mermaid
flowchart TB
accTitle: Operator overlay deploy pipeline
subgraph repo["Your repo"]
pin["UPSTREAM_SHA"]
overlay["wrangler.jsonc overlay"]
end
pin -->|"upstream-ref"| checkout["**Checkout upstream**<br/>at the pinned SHA"]
checkout --> apply["**Apply overlay**<br/>replaces upstream wrangler.jsonc"]
overlay -->|"wrangler-config"| apply
apply --> install["pnpm install --frozen-lockfile"]
install --> migrate["wrangler d1 migrations apply --remote"]
migrate --> deploy["wrangler deploy"]
deploy --> hasurl{"health-check-url set?"}
hasurl -->|no| skip["**Done**<br/>worker-url empty"]
hasurl -->|yes| poll{"200 within<br/>health-check-attempts?"}
poll -->|yes| healthy["**Done**<br/>worker-url set"]
poll -->|no| fail["**Step fails**"]
class healthy ok
class fail danger
```

Sibling to [`flare-dispatch-action`](../flare-dispatch-action/) — that one
**dispatches a run** from a consumer repo into a Worker; this one **ships the
Expand Down
37 changes: 36 additions & 1 deletion actions/flare-dispatch-action/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,26 @@ In branch protection, require the **check-run** name (e.g.
`flare-dispatch/offload-test`), not this GHA job — the check-run is the real PR
signal. The step itself succeeds the moment the dispatch is accepted (`202`).

The Dispatcher creates the Workflow instance before it answers `202`; the run
and its check-run happen after the step has already finished.

```mermaid
sequenceDiagram
accTitle: From dispatch to check-run verdict
participant A as flare-dispatch-action
participant D as Dispatcher Worker
participant W as RunWorkflow
participant G as GitHub Checks
A->>D: POST /v1/dispatch/‹run› with signature + Idempotency-Key
D->>D: verify HMAC, look up run, validate inputs
D->>W: create instance, id = executionId
D-->>A: 202 executionId, detailsUrl, logsUrl
Note over A: step green, outputs written
W->>G: open flare-dispatch/‹run› check-run, in_progress
W->>W: execute the run
W->>G: complete check-run with success, failure or neutral
```

## Inputs

| Input | Required | Default | Notes |
Expand Down Expand Up @@ -104,7 +124,22 @@ merged inputs carry signals but no `firedAt`. The machine-readable contract is

Per [`specs/04-gha-integration.md` § Failure handling](../../specs/04-gha-integration.md):

- **Dispatcher unreachable / `429` / `5xx`** — retried up to 3× with backoff
```mermaid
flowchart TB
accTitle: How the step handles each response
post["**POST the signed body**<br/>same bytes every attempt"] --> code{"HTTP status"}
code -->|202| ok["**Step green**<br/>execution-id written"]
code -->|401| drift["**Fail, no retry**<br/>both secret fingerprints printed"]
code -->|"400 / 404"| cfg["**Fail, no retry**<br/>Dispatcher error inlined"]
code -->|"any other, incl. network error"| more{"attempts < 3?"}
more -->|yes| wait["sleep attempt × 5s"]
wait --> post
more -->|no| gaveup["**Fail**<br/>last status inlined"]
class ok ok
class drift,cfg,gaveup danger
```

- **Dispatcher unreachable / `429` / `5xx`** — retried with backoff, 3 attempts in total; every status other than `202`, `400`, `401` and `404` (including a `403` refusal) counts as transient
(`attempt × 5s`, override via `FLARE_RETRY_BACKOFF_MS`), then the step fails.
- **`401`** (HMAC rejected) — config bug; the step fails immediately, no retry.
The job log prints two 8-char fingerprints (`sha256(secret)[:8]`):
Expand Down
93 changes: 93 additions & 0 deletions apps/dispatcher/specs/slack-origin.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,31 @@ Implementation: [`../src/slack-origin.ts`](../src/slack-origin.ts) (policy),
Everything up to § The verdict callback is about runs that **came from** Slack. § The notice covers
the other direction — a scheduled run with something to say and no thread to say it in.

```mermaid
sequenceDiagram
accTitle: A slack-origin dispatch, round trip
participant S as Slack
participant I as Slack ingress
participant D as Dispatcher
participant W as RunWorkflow
participant G as GitHub
S->>I: event
Note over I: classify by event class,<br/>conversational ones stay here
I->>D: POST /v1/dispatch/:run, HMAC, Idempotency-Key, source
alt refused by the envelope
D-->>I: 403 with error, message, run
I->>S: post message in thread
else accepted
D->>W: create instance, source in params
D-->>I: 202 with executionId
I->>S: ack in thread
W->>G: check-run, in_progress
W->>G: complete the check-run
W->>I: POST verdict, signed with k_verdict
I->>S: post verdict in thread, bot token
end
```

## The dispatch body

The ordinary body plus one optional block. Absent, nothing below applies and the GHA-Action path
Expand Down Expand Up @@ -59,6 +84,30 @@ posted verbatim into the thread — one branch for the caller, one sentence for
| `repo_not_pinned` | `github.repo` is not the pinned target |
| `idempotency_key_required` | No `Idempotency-Key` header |

The run half runs after the envelope decodes and before `inputs` decode; the inputs half runs
after:

```mermaid
flowchart TB
accTitle: Where the slack-origin policy sits in the dispatch route
sig{"Signature verifies?"} -->|no| e401["**401**"]
sig -->|"yes, and which key"| body["run lookup, JSON parse,<br/>envelope decode"]
body --> scoped{"Slack-scoped key<br/>without `source`?"}
scoped -->|yes| e403["**403**<br/>error, message, run"]
scoped -->|no| has{"Has `source`?"}
has -->|no| inputs
has -->|yes| runhalf{"**run half**<br/>unconfigured, credential,<br/>approval, allowlist, payload command,<br/>repo pin, Idempotency-Key"}
runhalf -->|first refusal| e403
runhalf -->|admitted| inputs["decode `inputs`<br/>against the run's schema"]
inputs -->|"with `source`"| inhalf{"**inputs half**<br/>credential after defaults"}
inputs -->|"no `source`"| rest
inhalf -->|refused| e403
inhalf -->|"admitted, `secrets` forced empty"| rest["target gate, cooldown, dedup,<br/>create the Workflow"]
class e401 danger
class e403 danger
class rest ok
```

### `secrets: []`, enforced on what executes

Naming a credential is refused against the **raw** payload, before schema decoding — decoding drops
Expand Down Expand Up @@ -158,6 +207,30 @@ A run publishes through the `notice` capability
notice.publish({ useCase, text, dedupeKey, links });
```

```mermaid
sequenceDiagram
accTitle: A scheduled run publishes a notice
participant C as Cron tick
participant R as Run
participant D as Dispatcher
participant N as Notice receiver
participant S as Slack channel
C->>R: start, no origin
R->>D: notice.publish with useCase, text, dedupeKey, links
alt slack-notice.url unset
Note over D: logged no-op
else configured
D->>N: POST /flare-dispatch/notify, signed with k_notice, deliveryId run:dedupeKey
alt useCase unmapped
N-->>D: 403
else mapped
N->>S: post escaped text and links
N-->>D: 2xx, or 409 when already delivered
end
end
Note over R: the run's verdict never depends on the notice
```

**The shape is the security property.** There is no channel, thread, recipient or URL, and no way to
express one. `useCase` is a routing *key* the receiver resolves against a map in its own deploy
config; an unmapped one is refused there. The emit side is the untrusted half — `text` may be
Expand Down Expand Up @@ -202,6 +275,17 @@ to reach the stronger surface.
The `ikm` is still one secret. Two labels off one secret is domain separation; two secrets would be
a second thing to rotate for a separation HKDF already gives.

```mermaid
flowchart TB
accTitle: One secret, two callback keys
ikm["**ikm**<br/>`SLACK_NOTIFY_SECRET`, else `HMAC_SECRET`"] -->|"HKDF, info slack-notify/v1"| kv["**k_verdict**"]
ikm -->|"HKDF, info slack-notice/v1"| kn["**k_notice**"]
kv --> verdict["**verdict callback**<br/>`origin` names channel and thread"]
kn --> notice["**notice**<br/>`useCase` only, no destination"]
class verdict warn
class notice ok
```

> **Deploy the receiver first.** The receiver must derive notices under the exact string
> `flare-dispatch/slack-notice/v1`. A receiver still on `flare-dispatch/slack-notify/v1` rejects
> every notice with a 401 — silently and indefinitely, because a failed notice is correctly never
Expand Down Expand Up @@ -238,6 +322,15 @@ So the id carries **two** states in the receiver's store, and only the second is
| `claimed` | The post was attempted; outcome unknown or failed | **Not** 409 — re-attempt the post, taking over the claim, or answer a 5xx |
| `delivered` | Slack accepted the post | `409` |

```mermaid
stateDiagram-v2
accTitle: A deliveryId in the receiver's store
[*] --> claimed : first POST of the id
claimed --> claimed : a repeat POST re-attempts the post, or answers 5xx
claimed --> delivered : Slack accepts the post
delivered --> delivered : a repeat POST answers 409
```

**`409` is reserved for `delivered`.** A receiver that answers 409 for a `claimed`-but-unposted id
breaks the reading on this side, and does it silently. Implementing the split is the receiver's half
(fractalbot `specs/flare-dispatch-notify.md`); it is written here because it is the assumption the
Expand Down
3 changes: 3 additions & 0 deletions apps/docs/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import { defineConfig } from "astro/config";
import { satteri } from "@astrojs/markdown-satteri";
import starlight from "@astrojs/starlight";
import merlion from "@fractalbox/merlion-astro";
import starlightLinksValidator from "starlight-links-validator";
import { README_PAGES, repoLinks } from "./src/lib/pages.mjs";

Expand Down Expand Up @@ -63,6 +64,8 @@ export default defineConfig({
},
],
}),
// Every ```mermaid block renders to inline, themeable SVG at build time.
merlion({ stylesheet: "src/styles/diagrams.css", width: 720 }),
],
vite: {
server: { allowedHosts: [".ts.net"] },
Expand Down
5 changes: 5 additions & 0 deletions apps/docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@
"@astrojs/check": "0.9.10",
"@astrojs/markdown-satteri": "0.4.1",
"@astrojs/starlight": "0.42.2",
"@fractalbox/merlion-astro": "file:vendor/merlion/fractalbox-merlion-astro-0.1.0.tgz",
"@fractalbox/merlion-rehype": "file:vendor/merlion/fractalbox-merlion-rehype-0.1.0.tgz",
"@fractalbox/merlion-themes": "file:vendor/merlion/fractalbox-merlion-themes-0.1.0.tgz",
"@fractalbox/merlion-view": "file:vendor/merlion/fractalbox-merlion-view-0.1.0.tgz",
"@fractalbox/merlion-wasm": "file:vendor/merlion/fractalbox-merlion-wasm-0.1.0.tgz",
"astro": "7.3.3",
"cookie": "2.0.1",
"starlight-links-validator": "0.26.0",
Expand Down
28 changes: 28 additions & 0 deletions apps/docs/src/styles/diagrams.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
* Merlion stylesheet for the docs site's ```mermaid diagrams. The Astro integration
* compiles it once per build and loads the compiled page CSS after merlion-themes.css;
* this file never reaches a page. The compiler accepts literal values only, so the
* colours repeat brand.css rather than reading Starlight's tokens.
*
* Starlight sets data-theme="light" or "dark" on <html>; `:root` repeats the light values
* for standalone renders.
*/

:root,
[data-theme="light"] {
--paper: #f4f1ea;
--ink: #13131a;
--accent: #d63300;
--merlion-bg: var(--paper);
--merlion-fg: var(--ink);
--merlion-accent: var(--accent);
}

[data-theme="dark"] {
--paper: hsl(224, 10%, 10%);
--ink: #eceef2;
--accent: #ff3d00;
--merlion-bg: var(--paper);
--merlion-fg: var(--ink);
--merlion-accent: var(--accent);
}
12 changes: 12 additions & 0 deletions apps/docs/substrate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@ that owns the account's Containers ceiling, artifacts on R2, and metered model a
it never decides what to run and never renders an outcome. Consumers — the dispatcher in this repo,
fractalbot from its own — reach it through one service-binding facade and nothing else.

```mermaid
flowchart LR
accTitle: Where the substrate sits
disp["**dispatcher**<br/>`DispatcherFacade`"] --> fac["**facade**<br/>one service binding"]
fb["**fractalbot**<br/>`FractalbotFacade`"] --> fac
fac --> adm[("**admission D1**<br/>pool slots")]
fac --> sdo["**Sandbox DO**<br/>ticket gate, exec fence"]
sdo --> box["**container**<br/>deny-all egress"]
sdo --> r2[("**R2**<br/>artifacts, snapshots")]
class fac accent
```

These four documents are what a consumer or an operator needs; none of them requires reading the
substrate's source.

Expand Down
72 changes: 53 additions & 19 deletions apps/docs/substrate/byoc-upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,20 @@ pnpm exec wrangler deploy # then the dispatch
```

In CI the ordering is a job dependency rather than a step order — `.github/workflows/deploy.yml` runs
`ci` → `substrate` → `deploy`, with `needs:` between them. That is deliberate: a step ordering inside
one job is lost the moment someone adds a matrix or reorders for speed, and a substrate deploy that
`ci` → `substrate` → `canary` → `deploy`, with `needs:` between them. That is deliberate: a step
ordering inside one job is lost the moment someone adds a matrix or reorders for speed, and a substrate deploy that
fails must stop the dispatcher's. Half a topology is worse than none of it.

```mermaid
flowchart LR
accTitle: The deploy job graph
ci["**ci**<br/>lint, typecheck, test"] --> sub["**substrate**<br/>D1 migrations, then deploy"]
sub --> can["**canary**<br/>unlisted host must 520,<br/>then `/health`"]
can -->|"target is not substrate-only"| disp["**deploy**<br/>the dispatcher"]
can --> dog["**dogfood**<br/>facade round trip"]
class can accent
```

Container images are rebuilt by `wrangler deploy` on the runner, which is why the deploy job needs
Docker.

Expand All @@ -84,32 +94,56 @@ without the CI product deploys only that worker
## Verify

```sh
curl -fsS https://<your-substrate-host>/health
# {"status":"ok","version":"0.1.0","contractVersion":1,
# "pools":{"lean":6,"browser":3,"agent":3,"task":4},"ceiling":16}
apps/substrate/scripts/verify-deploy.sh https://<your-substrate-host> canary
apps/substrate/scripts/verify-deploy.sh https://<your-substrate-host> health
# /health → {"status":"ok","version":"0.1.0","contractVersion":1,"deployment":"…",
# "pools":{"lean":6,"browser":3,"agent":3,"task":4},"ceiling":16,
# "canary":{"status":"passed","evidence":"…","checkedAt":…,"substrateVersion":"0.1.0"}}
```

`version` is the substrate release an operator is running — the number a security advisory declares a
floor against. `contractVersion` is the facade generation this deployment serves, which is what tells
a consumer maintainer whether their pin still matches
([versioning policy](contract-versioning.md)). A `500` with `"status":"misconfigured"` means the pool
partition does not fit the ceiling and names which; fix the overlay and redeploy.

Two caveats on what health does *not* prove. `SUBSTRATE_VERSION` is a hand-maintained constant kept
in step with `apps/substrate/package.json` by review, so it reports the version someone last wrote,
not the code that is running — a version-floor check is only as good as that discipline. And health
asserts configuration, not enforcement: the deploy-time canary that
[ADR-0011](../../substrate/specs/adr/0011-sdk-pin-as-security-surface.md) specifies — a container
fetch to an unlisted host must die with a 520 before consumer traffic is admitted — is not yet part
of this surface. Until it is, "the substrate is deployed" and "the substrate enforces the floor its
version claims" are two different statements.
The canary is the check
[ADR-0011](../../substrate/specs/adr/0011-sdk-pin-as-security-surface.md) specifies: `POST /canary`
boots a container and asserts that an HTTPS fetch to an unlisted host dies with a 520. It needs no
credential, and a verdict is stored per deployment, so repeat calls reuse it rather than boot another
container — a `passed` or `failed` verdict stands for 24 hours, an `inconclusive` one for 10 minutes.

`/health` answers `200 ok` only while a fresh, passing canary exists for the running deployment;
otherwise it answers `503` with `"status":"unverified"` and the canary block says why (`never-run`,
`failed`, `inconclusive`, or a verdict past its window). A `500` with `"status":"misconfigured"` means the pool
partition does not fit the ceiling and names which; fix the overlay and redeploy. `version` is the
substrate release an operator is running — the number a security advisory declares a floor against.
`contractVersion` is the facade generation this deployment serves, which is what tells a consumer
maintainer whether their pin still matches ([versioning policy](contract-versioning.md)).

One caveat on what health does *not* prove: `SUBSTRATE_VERSION` is a hand-maintained constant kept in
step with `apps/substrate/package.json` by review, so it reports the version someone last wrote, not
the code that is running — a version-floor check is only as good as that discipline. The canary
verdict is keyed to the Worker's version id, so it speaks for the build that is running — provided the
overlay keeps the `version_metadata` binding; without it the key falls back to that same semver.

## Upgrading

A routine upgrade is: move your pin to the new upstream commit, re-apply your overlay, run the two
deploys in order, check health. Migrations under `apps/substrate/migrations` are applied by
`wrangler d1 migrations apply` and are not run by a deploy — a release that adds one says so.

```mermaid
flowchart TB
accTitle: A routine upgrade
pin["**move the pin**<br/>re-apply the overlay"] --> mig["**D1 migrations**<br/>already-applied ones are no-ops"]
mig --> sub["**deploy the substrate**"]
sub --> can["**canary**<br/>`verify-deploy.sh` canary"]
can --> health{"`/health`"}
health -->|500 misconfigured| fix["**fix the overlay**<br/>pool caps exceed the ceiling"]
fix --> sub
health -->|503 unverified| stop["**stop**<br/>floor unproven;<br/>deploy no consumer"]
health -->|200 ok| sec{"substrate-only<br/>security fix?"}
sec -->|yes| done(["done"])
sec -->|no| disp["**deploy the dispatcher**"]
disp --> done
class stop danger
```

A **security release** carries three things a routine one does not: the minimum supported version, a
statement of what the floor loses below it, and whether the fix is substrate-only. Compare the
declared minimum against `/health`'s `version`, and if the fix is substrate-only take only the first
Expand Down
Loading
Loading