Skip to content

feat(harness): add Python provider preflight#756

Merged
AbirAbbas merged 4 commits into
Agent-Field:mainfrom
jstar0:feat/harness-provider-doctor-python
Jul 15, 2026
Merged

feat(harness): add Python provider preflight#756
AbirAbbas merged 4 commits into
Agent-Field:mainfrom
jstar0:feat/harness-provider-doctor-python

Conversation

@jstar0

@jstar0 jstar0 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Summary

This is the Python-first phase of #685. It adds a reusable harness availability contract and the af harness doctor preflight without starting a paid provider run.

Related issues

Part of #685.

Changes

  • Add Python extras for the Claude, OpenAI CLI, and OpenCode wrappers plus the harness-all bundle.
  • Add structured ProviderHealth reports separating binary/wrapper installation, version, auth signal, usability, install command, and machine-readable issues.
  • Add Agent.harness_doctor() and the af harness doctor [--provider ...] [--json] control-plane command.
  • Make Python CLI providers perform availability checks before subprocess spawn and raise typed HarnessProviderUnavailable errors.
  • Keep static auth checks offline-safe: auth="unknown" represents providers that may use local login state which cannot be proven without a paid liveness probe.
  • Document installation and container/CI verification, including the boundary between Python extras and actual provider binaries.
  • Leave TypeScript and Go SDK parity for follow-up phases so each public change remains reviewable and independently useful.

Verification

(cd sdk/python && pytest -q)
(cd sdk/python && ruff check agentfield tests)
(cd control-plane && go test ./internal/cli ./cmd/af)
git diff --check

Python tests pass with 94% configured coverage. The Go CLI tests pass. The focused formatter output is empty.

Checklist

  • I have read the contribution guide
  • I agree to follow the Code of Conduct
  • Tests added/updated
  • Documentation updated
  • I will sign the CLA

@jstar0
jstar0 requested review from a team and AbirAbbas as code owners July 11, 2026 18:39
@CLAassistant

CLAassistant commented Jul 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
Python 9.5 KB +6% 0.29 µs -17%

✓ No regressions detected

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran a multi-agent review over this (PR-AF on AgentField, findings hand-verified before posting). The Python preflight/doctor implementation itself is solid and well-tested — the issues are all at the edges: docs promising APIs that don't exist yet, and the Go CLI's provider table disagreeing with the Python SDK about what claude-code even is. Three inline comments below.

One structural suggestion, no action required: the provider spec table now lives twice (sdk/python/agentfield/harness/_availability.py and control-plane/internal/cli/harness_doctor.go) with no sync check — a tiny parity test (or generating one from the other) would stop them drifting further, which is exactly how the claude-code divergence below crept in.

Comment thread docs/harness-providers.md Outdated
Comment thread control-plane/internal/cli/harness_doctor.go Outdated
Comment thread sdk/python/agentfield/harness/_availability.py Outdated

@santoshkumarradha santoshkumarradha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pulled this into a clean clone and reran the new harness coverage locally. go test ./internal/cli/... passed, and the Python harness suite passed under uv run --python 3.12 --extra dev pytest for the new doctor and provider tests.

AbirAbbas and others added 2 commits July 15, 2026 10:39
…, KeyError fallback

Address the three unresolved review threads on Agent-Field#756:

- docs/harness-providers.md: drop the TypeScript/Go doctor examples — those
  APIs (agent.harnessDoctor, harness.Doctor) do not exist yet. Document the
  Python SDK + af CLI surface that actually ships and note TS/Go as planned
  follow-ups of Agent-Field#685.

- control-plane/internal/cli/harness_doctor.go: the claude-code row now
  mirrors the Python doctor (_doctor.py::_claude_health). It probes a Python
  interpreter for the claude_agent_sdk pip package (which bundles its own
  CLI) instead of looking for a global `claude` binary, and hints
  `pip install 'agentfield[harness-claude]'` instead of npm. Interpreter
  candidates (python3/python/py) are run-probed so dead launcher stubs are
  skipped; issues are wrapper_not_installed / python_not_found.

- sdk/python/agentfield/harness/_availability.py: provider_unavailable()
  falls back to a generic ProviderSpec via .get() so providers without a
  PROVIDER_SPECS entry (claude-code, or any future provider) raise the
  helpful HarnessProviderUnavailable instead of a bare KeyError. Regression
  test added for both provider_unavailable and ensure_cli_available.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AbirAbbas

Copy link
Copy Markdown
Contributor

Maintainer takeover to get this merge-ready — pushed to this branch (thanks for maintainerCanModify):

  • f86e340 merges origin/main into the branch (it was ~7 commits behind; clean merge, no conflicts).
  • 4ba8484 addresses the three open review threads:
    • docs/harness-providers.md now documents only the shipped Python SDK + af CLI surface; the TypeScript/Go examples referenced APIs that don't exist yet and were replaced with a "planned follow-ups of Harness provider availability: install extras + harness doctor preflight + standardized missing-binary errors #685" note.
    • af harness doctor's claude-code row now mirrors the Python doctor (_doctor.py::_claude_health): it probes a Python interpreter for the claude_agent_sdk pip package (which bundles its own CLI) instead of checking for a global claude binary, and hints pip install 'agentfield[harness-claude]'. New issues vocabulary: wrapper_not_installed / python_not_found.
    • provider_unavailable() falls back to a generic spec via .get(), so providers without a PROVIDER_SPECS entry raise HarnessProviderUnavailable instead of a bare KeyError (regression test added).

Verification:

  • Python SDK: full suite run — all 84 harness tests pass; ruff check and ruff format --check clean. (11 unrelated failures on my Windows box reproduce identically on pristine origin/main — environmental, not from this PR.)
  • Control plane: go build ./... OK; go test ./internal/cli passes fully on Linux (WSL), including 6 harness-doctor tests (3 new).
  • Manual: af harness doctor renders the table and reports claude-code unavailable with the pip hint until claude_agent_sdk is importable, then ready (exit 0); --json output verified for both states.

@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 86.90% 87.40% ↓ -0.50 pp 🟡
sdk-go 92.50% 92.00% ↑ +0.50 pp 🟢
sdk-python 93.76% 93.73% ↑ +0.03 pp 🟢
sdk-typescript 90.42% 90.42% → +0.00 pp 🟢
web-ui 84.75% 84.79% ↓ -0.04 pp 🟡
aggregate 85.54% 85.75% ↓ -0.21 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 131 83.00%
sdk-go 0 ➖ no changes
sdk-python 0 ➖ no changes
sdk-typescript 0 ➖ no changes
web-ui 0 ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

@AbirAbbas

Copy link
Copy Markdown
Contributor

recheck

@AbirAbbas AbirAbbas closed this Jul 15, 2026
auto-merge was automatically disabled July 15, 2026 15:32

Pull request was closed

@AbirAbbas AbirAbbas reopened this Jul 15, 2026
@AbirAbbas
AbirAbbas enabled auto-merge July 15, 2026 17:14
@AbirAbbas
AbirAbbas added this pull request to the merge queue Jul 15, 2026
Merged via the queue into Agent-Field:main with commit ea8aaad Jul 15, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants