Skip to content

refactor(ingest): move single-source ingest orchestration into an application service - #1167

Merged
jasonssdev merged 9 commits into
mainfrom
refactor/ingest-application-service
Sep 30, 2026
Merged

jasonssdev merged 9 commits into
mainfrom
refactor/ingest-application-service

Conversation

@jasonssdev

Copy link
Copy Markdown
Owner

Summary

Moves the single-source ingest orchestration out of the CLI adapter into application/ingest_service.py, so MVP 4's folder watch can call an in-process ingest with a root and get an outcome back instead of shelling out.

  • ingest_source(root, src, policy, *, ports, observer, confirm) -> IngestOutcome takes an explicit workspace root. It never reads the cwd, prompts, renders, inspects sys.stdin, or raises typer.Exit.
  • Phase A and B, the drift guard call, the auto-commit and the post-commit embed step all live in the service. The early return that used to end the command is now an IngestUnchanged outcome (vs IngestWritten).
  • Every refusal is a typed IngestRefused subclass carrying the exact user-facing message: SourceNotReadable, NotAWorkspace, RawImmutabilityRefused, InconsistentWorkspace, SymlinkedDestination (bug: ingest writes through a symlinked bundle/<dir>, and query/reindex read symlinked concept files — the #937 guard covers only forget/get #1126, before any write), SourceCheckFailed, PreparationFailed, WriteFailed, DriftDetected, ConfirmationDeclined, ConfirmationUnavailable.
  • cli/main.py::_ingest_single stays as a thin adapter (used by ingest and _ingest_batch): it resolves the cwd, builds the ports from its own seams, asks the prompt when a TTY is present, renders the observer callbacks, and maps refusals to the same exit codes (3 for drift, 1 for the rest, Typer's abort for a declined prompt). Batch semantics, including the non-TTY --auto rule and the cost gate, are untouched and stay in the CLI.
  • The ports (chat_client, autocommit, after_commit, snapshot_read, clock) are the adapter's own functions, so every existing monkeypatch seam (OllamaClient, _autocommit, _embed_client, _snapshot_read, datetime, Console) still intercepts and the layering guard (no cli/typer/rich/vcs in application/) holds.
  • Supporting moves: raw-destination resolution to application/ingest.py, and the drift decision to a pure application/drift.py::describe_drift that _reject_drifted_targets now wraps.

Why a new module rather than growing application/ingest.py: that module is the effect-free staging and composition core (1.6k lines); this one sequences it with effects. No new ADR: ADR-0018 already decided the shape.

No behaviour change and no CHANGELOG entry (nothing user-visible).

Related issue

Closes #1138

Type of change

  • feat — new feature
  • fix — bug fix
  • docs — documentation only
  • refactor — no behavior change
  • test — tests only
  • chore / ci — tooling, build, or CI
  • Breaking change

How was this tested?

  • Characterization first: 14 new byte-for-byte goldens in tests/unit/cli/test_ingest_characterization.py (interactive confirm and decline, non-TTY refusal, regenerate preview, drift on index.md and log.md, missing source, not a workspace, inconsistent workspace, disambiguated destination, malformed config, batch auto, batch non-TTY, batch with a refused file). They were recorded on the tree before the move and pass unchanged after it.
  • New direct tests for the service (tests/unit/application/test_ingest_service.py) run from outside the workspace: outcome variants, every typed refusal, the confirmation matrix, drift via the snapshot port, symlink refusal, commit-then-embed ordering.
  • Absence guards (test_ingest_service_seams.py) for the names that moved off cli.main.
  • Mutation checks: dropping the drift raise, dropping the final unmarked Source rewrite (fix(ingest): make a Phase B crash completable via a pending marker on the Source #1158), and dropping the symlink check each turned the relevant tests red; reverted byte-exactly.
  • Existing test_ingest.py, test_ingest_crash_window.py and the rest of the suite pass unchanged. Gates: ruff check, ruff format --check, mypy ., full pytest.

Checklist

  • My commits follow Conventional Commits.
  • I added or updated tests for the change.
  • I updated docs where behavior, interfaces, or the knowledge model changed (openspec/specs/ingest-application-service/spec.md).
  • Lint, format, type check, and tests pass locally (ruff, mypy, pytest).
  • Output remains OKF-conformant and derived stores stay reconstructible from the bundle + sources.
  • The change is consistent with the project's guiding principles (local-first, provenance, freshness, human-in-the-loop).

Notes for reviewers

  • Commits are ordered to review one move at a time: goldens, raw-destination move, pure drift function, the service, spec. The merge of main brought in fix(bundle): refuse to read or write through a symlinked bundle path #1159; its two symlink checks now sit inside the service at the same points (before extraction, and after staging chose the derived paths), both before any write.
  • Deliberately left in the CLI: the confirmation prompt and its non-TTY refusal wording (TTY is an adapter concern), the batch cost gate and --auto rule, _autocommit, _embed_after_ingest and _refresh_derived_after_write (the service reaches them through ports), the spinner and all output wording for the preview and staged-object notices.
  • design: ingest orchestration still lives in the CLI adapter — MVP 4's daemon has nothing to call but the CLI #1138's proposal lists BackendUnavailable among the refusals; in the code a backend failure degrades to a Source-only run and never refuses, so there is no such refusal to type.
  • Not done here (the issue's items 3 and 4): the same treatment for reindex, findings and the merge/reconcile cores, and splitting test_ingest.py.

@jasonssdev

Copy link
Copy Markdown
Owner Author

Items 3 and 4 of #1138 (the same service shape for reindex, findings and merge/reconcile, and splitting test_ingest.py) are tracked in #1168, so closing #1138 with this PR loses nothing.

@jasonssdev
jasonssdev merged commit 82de2ef into main Sep 30, 2026
9 checks passed
@jasonssdev
jasonssdev deleted the refactor/ingest-application-service branch September 30, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

design: ingest orchestration still lives in the CLI adapter — MVP 4's daemon has nothing to call but the CLI

1 participant