diff --git a/README.md b/README.md index 438b8cc..afd1d3f 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,15 @@ and every call is a door. ![Inside main(): the call room, with the if/else lanes ahead behind their tinted door frames](docs/media/hero.jpg) -Drop a zip of any JavaScript or TypeScript repository on the page, or pick a -folder. Nothing is uploaded: the code is parsed in your browser tab. +Paste the URL of any public JavaScript or TypeScript repository on GitHub, or +walk a local folder. The code is parsed in your browser tab, and a folder is +never uploaded. Once a repository is open, the address bar is its share link, +for example +[`?repo=valuecodes/code-backrooms`](https://code-backrooms.valuecodes.fi/?repo=valuecodes/code-backrooms). + +A GitHub repository comes through this site's Cloudflare Worker as one zip +download from GitHub, after a Cloudflare Turnstile check that keeps bots from +using it as a proxy. ## What you are walking through @@ -67,17 +74,17 @@ Then open http://localhost:3000. ## How it works -Everything runs in the browser tab. Code decides which rooms and doors exist. +Parsing and building run in the browser tab. Code decides which rooms and doors exist. A seed taken from a hash of the source decides only sizes and placement, so the same code always gives the same world. ```mermaid flowchart TD subgraph load["Load: apps/web"] - A["Zip, folder, or bundled example"] --> B["Source filter
JS/TS files and manifests,
size and count caps"] + A["GitHub repository, folder,
or bundled example"] --> B["Source filter
JS/TS files and manifests,
size and count caps"] end - subgraph analyse["Analyse: Web Worker for a zip or folder"] + subgraph analyse["Analyse: Web Worker for a repository or folder"] B --> C["@repo/parser
Babel AST per file"] C --> D["CodeGraph
modules, functions, calls,
control-flow trees, imports"] end diff --git a/apps/web/AGENTS.md b/apps/web/AGENTS.md index d49a1e8..0e91fe9 100644 --- a/apps/web/AGENTS.md +++ b/apps/web/AGENTS.md @@ -153,8 +153,8 @@ not be laid out`. The HUD reads `2/7 modules · last 14 ms` on the way; whole world, on `?view=map` too; wider than 600 m the map hides rooms and area labels and draws zone names larger, each within its band. - Milestone 20: repository loading (Phase 7). A bare URL opens a - landing page: drop or pick a `.zip` (GitHub's "Download ZIP" works as - is), pick a folder, or follow Demo (`?code=self`, this repository's own + landing page: drop or pick a `.zip` (removed in Milestone 23), pick a + folder, or follow Demo (`?code=self`, this repository's own sources) or a bundled example. Files are read in the tab and never leave it, and a reload returns to the landing page. `source-filter.ts` keeps sources (`.ts .tsx .mts .cts .js .jsx .mjs .cjs`) and manifests @@ -175,10 +175,10 @@ not be laid out`. The HUD reads `2/7 modules · last 14 ms` on the way; toward its first doorway and back while its gaze sways (`landing-backdrop.tsx`, `driftPose` in `backdrop.ts`). It is skipped without WebGL2 (it runs under `prefers-reduced-motion` too), swallowed by an error - boundary if the canvas fails, and dropped as soon as a read starts. The + boundary if the canvas fails, and darkened while a read runs. The world view (`world-app.tsx`) and the backdrop are lazy chunks, so the landing page paints before three.js loads. -- Milestone 22 (current): a public GitHub repository by URL. The landing +- Milestone 22: a public GitHub repository by URL. The landing page takes `github.com/owner/repo`, `owner/repo` or a `/tree//` URL (`github-url.ts`; refs with `/` are not supported). One load is one request to GitHub: the page calls this site's @@ -191,16 +191,22 @@ not be laid out`. The HUD reads `2/7 modules · last 14 ms` on the way; only once a URL is typed). Each IP is rate-limited to 5 loads a minute. Answers are edge-cached for 10 minutes, and zips over 64 MB are refused or cut off. A loaded repository writes `?repo=` so a reload or a shared link - fills the field in again. The visitor still presses open, because each - Turnstile token is single-use. Without a site key at build time the field + fills the field in again. Without a site key at build time the field is hidden, so production never falls back to the test key. +- Milestone 23 (current): GitHub first. The landing page drops the zip + drop zone and picker: its card is the GitHub field, with a quiet "walk a + local folder" link below (the card's only button when the build has no + Turnstile key). A `?repo=` link opens by itself as soon as its check + passes, once, unless the field is edited first; the token is spent + through a ref so an auto-open and a click cannot send it twice. While a + read runs the backdrop stays and darkens over `FADE_MS`, and no read hands + over sooner, so the page fades into the loading screen instead of + cutting to black. --- ## Workspace Commands -| Task | Command | -| ---------- | ------------------------------ | | Task | Command | | ---------- | ------------------------------ | | Dev | `pnpm --filter web dev` | @@ -224,14 +230,20 @@ Or run repo-wide via root `pnpm typecheck` / `pnpm test` / `pnpm build`. ### Deploying (Cloudflare Workers Builds, repository connected in the dashboard) -- **Build settings:** set the root directory to the repository root. The build - command is `pnpm --filter web build`, and the deploy command is - `pnpm --filter web exec wrangler deploy`. -- **Build variables:** `PNPM_VERSION=12.4.2` and `NODE_VERSION=24.21.0`, since the - build image's pnpm default is older. Also set `VITE_TURNSTILE_SITE_KEY` to the - widget's site key. Vite inlines it at build time, so it is a _build_ variable. -- **Runtime secret:** `TURNSTILE_SECRET` under the Worker's Variables and Secrets. - `wrangler.jsonc` declares no value for it, so deploys keep it. +- **Worker:** `code-backrooms`, the `name` in `wrangler.jsonc`. Workers Builds + expects the connected Worker's name to match, and local `wrangler secret` and + `wrangler tail` target it without `--name`. +- **Build settings:** root directory `apps/web`, build command `pnpm run build`, + deploy command `npx wrangler deploy`. +- **Build variables:** `VITE_TURNSTILE_SITE_KEY`, the widget's site key. Vite + inlines it at build time, so it is a _build_ variable. Also set + `NODE_VERSION=24.21.0` and `PNPM_VERSION=12.4.2`: the version files sit + outside the `apps/web` root, and the build image's defaults are older. +- **Runtime secret:** `TURNSTILE_SECRET` under the Worker's Settings → Variables + and Secrets, type Secret, or `pnpm --filter web exec wrangler secret put +TURNSTILE_SECRET`. Never a build variable: those exist only during the build, + and the Worker then answers every GitHub load with 503 "not set up on this + server". `wrangler.jsonc` declares no value for it, so deploys keep it. - **Turnstile widget:** create it in the Cloudflare dashboard. Its hostnames must include the production domain, plus the `workers.dev` domain if preview builds are on. A preview without its hostname refuses GitHub loads and everything else @@ -252,7 +264,7 @@ src/ app.tsx landing page, loading screen, or the world as a lazy chunk world-app.tsx the world: full-screen , seed and current-room state, pointer lock, HUD - landing.tsx the bare URL's page: zip drop and pickers, GitHub URL, Demo, examples + landing.tsx the bare URL's page: GitHub URL, folder picker, Demo, examples github-url.ts a GitHub URL -> owner, repo, ref, folder; checks shared with the Worker, pure github-files.ts a GitHub repository -> sources: one call to the Worker, then zip-files github-form.tsx the landing page's GitHub URL field, check and open button, markup only @@ -263,10 +275,10 @@ src/ landing-backdrop.tsx the demo world drifting behind the landing page, lazy backdrop.ts when the backdrop runs and where its camera is, pure backdrop-world.ts the demo world for the backdrop, every area built - local-reads.ts one read at a time, zip and folder labels, read errors, pure + local-reads.ts one read at a time, folder labels, read errors, pure source-request.ts what to build: nothing, a URL example, or local files, pure source-filter.ts which zip or folder entries become sources, and the caps, pure - zip-files.ts a zip -> sources (fflate's sync API, two passes) + zip-files.ts a GitHub zip -> sources (fflate's sync API, two passes) folder-files.ts a folder pick -> sources, reading only the kept files use-source-files.ts a request's files: bundled at once, `self` as its own chunk world-setup.ts a request -> everything a page needs to generate its world diff --git a/apps/web/README.md b/apps/web/README.md index 1392a80..0d53cad 100644 --- a/apps/web/README.md +++ b/apps/web/README.md @@ -21,22 +21,24 @@ where the current function was entered, and R to the entrance. ## Loading a repository -A bare URL opens the landing page. Drop or pick a `.zip` (GitHub's -"Download ZIP" works as is) or pick a folder. Files are read in the browser -tab and never uploaded. Only JavaScript and TypeScript sources and manifests -are kept, and dependency, build and test directories are skipped. - -You can also paste a public GitHub repository URL, such as -`github.com/owner/repo` or `github.com/owner/repo/tree/main/src` for one -folder. After a Cloudflare Turnstile check, this site's Worker downloads the -repository's zip from GitHub in one request and the page reads it like a -dropped zip. `?repo=owner/repo` fills the field in. Running this locally -needs `pnpm --filter web dev:worker` beside `dev`. See `AGENTS.md` for that -and for the deploy settings. +A bare URL opens the landing page. Paste a public GitHub repository URL, such +as `github.com/owner/repo`, or `github.com/owner/repo/tree/main/src` for one +folder, and press open once the Cloudflare Turnstile check passes. This +site's Worker downloads the repository's zip from GitHub in one request, and +the page reads it in the tab. Only JavaScript and TypeScript sources and +manifests are kept, and dependency, build and test directories are skipped. + +A loaded repository's address is its share link: `?repo=owner/repo` fills the +field in and opens the repository as soon as the visitor's check passes, +usually with no click. Running this locally needs `pnpm --filter web +dev:worker` beside `dev`. See `AGENTS.md` for that. + +For local or private code, "walk a local folder" picks a folder instead. It is +read in the browser tab and never uploaded. | Limit | Value | | ------------------- | ------ | -| Zip size | 64 MB | +| GitHub zip size | 64 MB | | Kept source files | 400 | | Kept bytes in total | 8 MB | | One file | 256 KB | @@ -96,23 +98,23 @@ at startup (`packages/renderer/src/textures.ts`), so there are no image assets. ## Deploy (Cloudflare) -The app deploys as static assets on a Cloudflare Worker. `wrangler.jsonc` serves -`dist/`, and `public/_headers` (copied into `dist/` by Vite) sets the security and -cache headers. `wrangler` is a pinned devDependency, so the deploy uses the lockfile -version rather than `npx`. +The app deploys as a Cloudflare Worker named `code-backrooms` (`wrangler.jsonc`). +It serves `dist/` as static assets and runs code only for `/api/*`, the GitHub +zip route (`src/api/`). `public/_headers` (copied into `dist/` by Vite) sets the +security and cache headers. There is no CI deploy job. To connect the repo in the Cloudflare dashboard (Workers & Pages → Create → Import a repository): -| Setting | Value | -| -------------- | --------------------------- | -| Root directory | `apps/web` | -| Build command | `pnpm build` | -| Deploy command | `pnpm exec wrangler deploy` | +| Setting | Value | +| -------------- | --------------------- | +| Root directory | `apps/web` | +| Build command | `pnpm run build` | +| Deploy command | `npx wrangler deploy` | -Also set these build variables (Settings → Build → Variables), so Cloudflare installs -dependencies with the repo's toolchain. The install runs before the build command, -so pinning versions there would be too late: +Also set these build variables (Settings → Build → Variables). With `apps/web` +as the root, the repository's `.nvmrc` and `packageManager` sit outside it, so the +build image is not guaranteed to pick them up, and its defaults are older: | Variable | Value | | -------------- | --------- | @@ -121,10 +123,31 @@ so pinning versions there would be too late: Keep them in step with `.nvmrc` and the root `packageManager` field. +### Turnstile + +Loading from GitHub needs a Cloudflare Turnstile widget. Create one in the +dashboard (Turnstile → Add widget) with the production domain as a hostname, +then set its two keys in two different places: + +| Key | Where | +| ------------------------- | ----------------------------------------------------------- | +| `VITE_TURNSTILE_SITE_KEY` | Settings → Build → Variables: Vite inlines it into the page | +| `TURNSTILE_SECRET` | Settings → Variables and Secrets, type Secret | + +The secret must be a runtime secret, not a build variable: build variables exist +only while the site builds, and without the secret the Worker answers every +GitHub load with "loading from GitHub is not set up on this server". It can also +be set with `pnpm --filter web exec wrangler secret put TURNSTILE_SECRET`. +Without the site key the GitHub field is hidden and the folder picker is the +page's only loader. The Worker verifies each token with Turnstile's `siteverify` +before it fetches, and rate-limits each IP to 5 loads a minute. + To deploy from your machine instead, run `pnpm --filter web exec wrangler login` once, then `pnpm --filter web deploy`. -The Content-Security-Policy in `public/_headers` allows only this origin plus -Cloudflare Web Analytics. Loading scripts, fonts, images or APIs from any other -origin needs a matching entry there, or the browser blocks it in production (the -Vite dev server does not apply `_headers`). +The Content-Security-Policy in `public/_headers` allows only this origin, +Cloudflare Web Analytics and Turnstile (`challenges.cloudflare.com` as a script +and a frame). Loading scripts, fonts, images or APIs from any other origin needs a +matching entry there, or the browser blocks it in production (the Vite dev server +does not apply `_headers`). Cloudflare's Bot Fight Mode injects an inline script +this policy blocks, so keep it off for this domain. diff --git a/apps/web/src/github-files.ts b/apps/web/src/github-files.ts index fe3d7af..e2b8b31 100644 --- a/apps/web/src/github-files.ts +++ b/apps/web/src/github-files.ts @@ -6,7 +6,7 @@ import type { LocalFiles } from "~/zip-files"; // A public GitHub repository to the files of a code world: one request to // this site's Worker (`api/github-zip.ts`), which fetches the repository's zip from -// GitHub, then the same zip reader as a dropped file. +// GitHub, then `zip-files.ts` reads it in the tab. type Fetch = (url: string, init: RequestInit) => Promise; diff --git a/apps/web/src/landing.test.tsx b/apps/web/src/landing.test.tsx index c30251a..ffc8bc4 100644 --- a/apps/web/src/landing.test.tsx +++ b/apps/web/src/landing.test.tsx @@ -18,10 +18,7 @@ const render = ( { } }); - it("offers a zip picker and a folder picker", () => { - const html = render(); - expect(html).toContain('accept=".zip,application/zip"'); - expect(html).toMatch(/]*webkitdirectory=""[^>]*>/); - expect(html).not.toContain('disabled=""'); + const GITHUB = { url: "", valid: false, ready: false }; + + it("offers no zip picker, and one folder picker either way", () => { + for (const html of [render(), render({ github: GITHUB })]) { + expect(html).not.toContain(".zip"); + expect(html.match(/]*webkitdirectory=""[^>]*>/g)).toHaveLength( + 1 + ); + expect(html).not.toMatch(/]*disabled=""/); + } + }); + + it("makes the folder the main action without a Turnstile key", () => { + expect(render()).toContain("choose a folder"); + expect(render()).toContain("never uploaded"); + const html = render({ github: GITHUB }); + expect(html).not.toContain("choose a folder"); + expect(html).toContain("walk a local folder"); + expect(html).toContain("public repositories"); }); - it("disables both pickers and drops the links while reading", () => { - const html = render({ reading: "repo-main.zip" }); - expect(html).toContain("reading repo-main.zip…"); - expect(html.match(/]*disabled=""/g)).toHaveLength(2); + it("disables the picker, shows the status and drops the links while reading", () => { + const html = render({ reading: "reading my-repo…" }); + expect(html).toContain("reading my-repo…"); + expect(html.match(/]*disabled=""/g)).toHaveLength(1); expect(html).not.toContain("href="); }); + it("darkens the backdrop while reading instead of dropping it", () => { + const backdrop = ; + expect(render({ backdrop })).toContain("opacity-45"); + const html = render({ backdrop, reading: "reading my-repo…" }); + expect(html).toContain(''); + expect(html).toContain("opacity-90"); + }); + it("draws a backdrop behind the page, hidden from screen readers", () => { expect(render()).not.toContain("aria-hidden"); const html = render({ backdrop: }); @@ -69,10 +88,6 @@ describe("LandingView", () => { expect(html).toContain('href="?code=self"'); }); - it("says that nothing is uploaded", () => { - expect(render()).toContain("nothing is uploaded"); - }); - it("has no GitHub field when the build has no Turnstile key", () => { expect(render()).not.toContain("GitHub repository URL"); }); @@ -101,17 +116,18 @@ describe("LandingView", () => { it("disables the GitHub field and hides the check while reading", () => { const html = render({ - reading: "o/r", + reading: "downloading o/r from GitHub…", github: { url: "o/r", valid: true, ready: true }, }); + expect(html).toContain("downloading o/r from GitHub…"); expect(html).toMatch(/