[docs/agenteye] Sync AgentEye documentation

docs/agenteye/audits.mdx

@@ -1,116 +1,52 @@
 ---
-title: "Audits: agentic improvement detection"
-description: "AgentEye Audits: agentic improvement detection documentation."
+title: "Audits: your automatic reliability analyst"
+description: "AgentEye Audits: your automatic reliability analyst documentation."
 ---
 
 
-Audits tell you what to fix in your AI agents. An audit is a scheduled job that reads across all your agent sessions, hunts for patterns worth improving (a leaked API key, a silently failing tool, a prompt that is quietly degrading) and writes up concrete recommendations with the evidence behind each one. Use audits to answer "what should I improve?"; use alerts to be paged the moment a threshold you already know about is crossed.
+AgentEye goes looking for the failures you never wrote a rule for and hands you a ranked, evidence-backed to-do list of exactly what to fix. It is like having an analyst comb your logs every night, then leaving the short list on your desk by morning.
 
-Every recommendation links to the exact sessions and queries behind it, and a one-click shortcut drafts a recurrence alert so you catch the issue if it comes back. You will find audits in the dashboard at **`/<org-slug>/audits`** (sidebar → *analyze* → *audits*), gated by the `audits:read` / `audits:write` permissions.
+<div style={{ position: "relative", width: "100%", paddingBottom: "56.25%", height: 0, overflow: "hidden", borderRadius: "12px", margin: "1.5rem 0" }}>
+  <iframe src="https://www.youtube.com/embed/C5nHKGQIqZQ?autoplay=1&mute=1&loop=1&playlist=C5nHKGQIqZQ&rel=0&playsinline=1" title="Watch: Automatically find where your agent is failing (2 min)" allow="autoplay; encrypted-media; picture-in-picture; fullscreen" allowFullScreen style={{ position: "absolute", top: 0, left: 0, width: "100%", height: "100%", border: 0 }}></iframe>
+</div>
 
-[![Watch: Automatically find where your agent is failing (2 min)](/agenteye/images/video-audit.jpg)](https://youtu.be/C5nHKGQIqZQ)
+*A two-minute tour: from a scheduled run to a fix you can act on.*
 
-*A two-minute tour of automatic failure audits: from a scheduled run to a fix you can act on.*
+![The Audits page: recurring jobs that scan your sessions for failure patterns, each with a schedule and sensitivity](/agenteye/images/audits.png)
+*Each audit is a recurring job that mines your sessions and writes up ranked, evidence-backed recommendations.*
 
-![The Audits page: recurring jobs that mine your logs for failure patterns, each with a schedule and analysis mode](/agenteye/images/audits.png)
+## Stop guessing what to fix next
 
----
-
-## How a run works
-
-Each run has two layers: a deterministic floor and an agentic investigation.
-
-### 1. The policy pass (deterministic)
-
-Before any model runs, the audit executes a small catalog of **SQL policy checks** over the window: bounded aggregate queries that flag known-bad patterns and report *how many* events / *which* sessions matched, never the matched text itself. The catalog includes:
-
-- **Secret / credential leakage** in event payloads: AWS access keys, `sk-…` API keys, PEM private keys, JWT / bearer tokens, and `KEY=…` credential assignments.
-- **Prompt-injection markers**: "ignore previous instructions", "reveal your system prompt", and similar.
-- **PII**: SSN-shaped numbers (heuristic).
-- **Tool-permission denials** and **runaway tool-call loops**.
-
-Policy hits are persisted as findings (kind `policy`) that **always surface** (they are never trimmed by the per-run cap), and they are handed to the AI agent as starting leads. Because this layer needs no model, an audit still produces its most important security signals even if the AI agent is unavailable.
-
-### 2. The agentic investigation (AI)
-
-The audit then runs an **autonomous reliability agent**: the same agent service that powers the AI assistant, running an audit-specific prompt. Given the audit's **scope** (selected agents × environments) and **time window**, the agent:
-
-- runs read-only SQL queries against your analytics tables,
-- reads a handful of representative session transcripts,
-- optionally writes and runs short **Python scripts in a locked-down in-pod sandbox** (no network, no filesystem access, secrets scrubbed) for analysis SQL can't express: clustering errors, computing distributions, sweeping payloads it already fetched,
-- and records each well-evidenced **improvement** it finds.
-
-It works through several investigative lines (error clustering, drift vs. a baseline, goal failure in transcripts, tool misuse, quality/cost trade-offs, and coverage gaps) at the audit's **sensitivity** (low / medium / high). Every improvement **must cite evidence**: the session ids the agent actually inspected and/or the SQL it ran. The server validates that cited sessions exist and **discards any improvement with no surviving evidence**, so the agent investigates but never invents.
-
-Each improvement carries:
-
-- a **recommendation** (the concrete change to make: a prompt tweak, a tool-schema fix, a retry policy, a guardrail, more eval coverage),
-- an **expected impact** and an **effort** estimate (low / medium / high),
-- a **magnitude** (how loudly it should surface): `big` (an operator should be paged), `medium` (belongs in the run report), or `small` (dashboard context),
-- a stable **fingerprint**: an identity derived from the issue's category and scope, *not* this run's sessions, so the same issue is tracked run-over-run even as the evidence changes,
-- and, where a simple watcher could catch the issue coming back, a **suggested alert**: a read-only recommendation shown on the run page as a "graduation candidate" for what to watch. (Separately, every finding page has a one-click shortcut that drafts a recurrence alert prefilled with a sensible starting trigger, which you then tune.)
-
-> **Note:** The AI layer is optional but recommended. If no AI agent is configured for the audit pipeline, runs still execute, persist the policy findings, and honestly report "analysis unavailable" for the agentic layer rather than silently passing.
-
-### Failure modes
-
-Improvements classify into your org's durable **failure-mode catalog** (the running list of distinct ways your agents fail) or propose a new mode. Modes give patterns a stable identity across runs and long-horizon recurrence tracking.
-
-## Triage lifecycle
-
-On a finding page (`/audits/<id>/findings/<finding-id>`):
-
-| Action | Effect |
-|---|---|
-| **acknowledge** | Keeps the finding visible but halves its priority. |
-| **resolve** | Marks it fixed. If the pattern genuinely returns later, it re-opens as **new**, so a regression is loud, not silently folded into history. |
-| **mute** / **dismiss** | Durable suppression: the pattern's fingerprint is remembered and never surfaces again, even across runs. Use mute for "known, accepted"; dismiss for "not useful". |
-| **reopen** | Clears suppression / resolution and ranks the pattern again. |
-
-Low-signal noise is controlled per audit with a per-run findings cap (`top_k`) on the agentic improvements. Policy findings bypass the cap (they are security-relevant and always shown). Anything cut by the cap is counted in the run's stats; nothing is silently dropped.
-
-## Scheduling
-
-- **Cadence** (`schedule_interval_secs`): hourly to weekly; **daily is the default**. Audits are deliberately coarser than alerts: an agentic investigation scans whole windows and runs for minutes.
-- **Window**: either a fixed rolling lookback (e.g. "each run scans the last 7 days") or **since-last-run** (the default): each run picks up where the previous successful one ended, with a small overlap so boundary events are never missed.
-- The next run is scheduled a full interval after the previous one **completes**, so a slow run never stacks a second concurrent run of the same audit.
-- **Run now** on the audit page makes it due immediately.
-
-## Model selection
-
-When creating an audit you can pick which model the investigation uses, from the **list of models your operator has configured** for the agent service. With a single model configured, the picker shows it as a caption; with several, you choose. Leaving it unset uses the configured default.
+Alerts catch the problems you already know to watch for. Audits catch the ones you don't. On a schedule you set, an audit reads across all of your agent sessions and hunts for the patterns worth fixing, so you spend your time acting on findings instead of scrolling logs hoping to spot them yourself.
 
-## Notifications
+A single run goes after the failure modes that actually break agents in production:
 
-When a run surfaces **new** findings, the audit notifies your organization's configured channels, the same `alerts.enabled_channels` gate and settings the alerts pipeline uses:
+- **Error clusters**: the same failure repeating under a shared root cause.
+- **Drift versus a baseline**: behaviour quietly sliding away from a known-good window.
+- **Goal failure in transcripts**: runs that technically finished but never did the job.
+- **Tool misuse**: the wrong tool, bad arguments, or loops that burn calls.
+- **Quality and cost trade-offs**: where you are overpaying for output you could get cheaper.
+- **Coverage gaps**: behaviour that no eval or alert is watching.
 
-- **Slack**: a summary of the significant (`big`) new items with a deep link.
-- **Email**: a designed **audit report** listing the new improvements (top severity, per-item recommendations, deep link), sent when the audit has an **email** channel attached and there is at least one new finding.
+You decide how hard it looks with a single **sensitivity** setting (low, medium, or high), so a noisy staging agent and a locked-down production one can each be tuned to the signal you want.
 
-Recurring-but-known findings do not re-notify.
+## Every recommendation comes with receipts
 
-## Configuration reference
+You never have to take a finding on faith. Each recommendation cites the exact sessions it came from and the SQL that surfaced it, so you can open the evidence and confirm the problem in a click instead of reverse-engineering a claim.
 
-Audit definitions are managed in the dashboard (`/audits/new`) or via the API. Per-audit settings include the schedule cadence and window, the scope (`{"environments": [...], "agent_ids": [...]}`), the sensitivity (`low` / `medium` / `high`), the notification channels, the per-run findings cap (`top_k`), and the model (via `llm_budget.model`). Operator-level server settings (timeouts, sandbox, the agent-service URL) are documented in [Deployment](/agenteye/deployment).
+That is also what keeps audits honest. The server checks that every cited session actually exists and **discards any recommendation whose evidence does not hold up**, so the audit investigates but never invents. What lands on your list is real, reproducible, and ranked by how much it matters, with the biggest wins at the top.
 
-## API
+## Turn a fix into a guardrail
 
-All endpoints are org-scoped and follow the standard bearer-key auth (see [API keys](/agenteye/api-keys)).
+Fixing an issue is only half the win. The other half is making sure it cannot quietly come back. Every finding carries a **one-click shortcut that drafts a recurrence alert**, prefilled with a sensible starting trigger you can tune. Close the finding, arm the alert, and the next time that pattern reappears you get paged instead of rediscovering it in a future audit.
 
-| Endpoint | Permission | Purpose |
-|---|---|---|
-| `GET /audits` · `POST /audits` | `audits:read` / `audits:write` | List / create audit definitions. |
-| `GET` / `PUT` / `DELETE /audits/:id` | read / write / write | Inspect, edit, delete an audit. |
-| `POST /audits/:id/run` | `audits:write` | Make the audit due immediately. |
-| `GET /audits/:id/runs` | `audits:read` | Run history (window, status, stats, finding counts). |
-| `GET /audits/findings` | `audits:read` | Org-wide findings, filterable by `audit_id`, `status`; sorted by priority. |
-| `GET /audits/findings/:fid` | `audits:read` | Full finding detail (recommendation, evidence, priority). |
-| `POST /audits/findings/:fid/status` | `audits:write` | Triage: `{"action": "ack" \| "mute" \| "dismiss" \| "resolve" \| "reopen" \| "assign"}`. |
+## Where to find it
 
-For "audit ran but found nothing", "the code sandbox is disabled", and "audit email not delivered", see [Troubleshooting](/agenteye/troubleshooting#audits).
+Audits live in the dashboard at **`/<org-slug>/audits`** (sidebar to *analyze* to *audits*). Viewing runs and findings needs **`audits:read`**; creating, editing, and triaging audits needs **`audits:write`**. Set an audit's scope and cadence, then hit **Run now** whenever you want results immediately instead of waiting for the next scheduled pass.
 
-## Next steps
+## Related
 
-- [Alerts](/agenteye/alerts): get paged the moment a specific threshold you already know about is crossed.
-- [Evaluation suite](/agenteye/evaluation-suite): score every completed run so quality regressions surface on their own.
-- [AI assistant](/agenteye/assistant): ask questions about your agents in natural language and dig into any finding.
+- [Alerts](/agenteye/alerts): get paged the moment a threshold you already know about is crossed.
+- [Evaluations](/agenteye/evaluations): score every run so quality regressions surface on their own.
+- [Error tracking](/agenteye/error-tracking): group and follow the errors your agents throw.
+- [Incidents](/agenteye/incidents): track an issue an audit turns up through to its fix.

docs/agenteye/cli-and-agents.mdx

@@ -0,0 +1,80 @@
+---
+title: "CLI"
+description: "AgentEye CLI documentation."
+---
+
+
+Your entire AgentEye deployment, one command away. Check production, cut an API key, or ack an incident without leaving your terminal, then script any of it into CI, or let a coding agent do it for you in plain English.
+
+```bash
+pipx install agenteye
+agenteye login --email you@example.com      # a 6-digit code lands in your inbox
+agenteye --json sessions --since 24h         # every agent run from the last day, newest first
+```
+
+*The `agenteye` CLI talks to your dashboard. It is a different tool from the collector, which ships events to the server.*
+
+## Your whole deployment, one command away
+
+Stop tab-hopping to answer a quick question. The `agenteye` CLI reads your data and administers your org from a single binary, so a check that used to mean clicking through the dashboard becomes one line you can rerun, alias, or paste into a runbook. You get four surfaces:
+
+- **Read your data:** `sessions`, `events`, `evals`, and `errors`, filtered by time, agent, and environment.
+- **Manage your org:** `keys`, `users`, `settings`, `alerts`, and `incidents`.
+- **Run analytics:** saved SQL plus an ad-hoc `query` runner over your event data.
+- **Ask the assistant:** `agent ask` reaches the same read-only analyst you chat with in the dashboard.
+
+Install it once with `pipx`, sign in with an emailed 6-digit code, and you are ready. The session lasts about a day; rerun `agenteye login` when it expires. Reach for it to spot-check production, provision a key, or triage a firing incident, all without opening a browser:
+
+```bash
+agenteye errors --since 24h --aggregate      # what is breaking, grouped by error type
+agenteye incidents list --state firing       # what is on fire right now
+agenteye keys create ci --add events:add     # a key that can only push events, secret shown once
+```
+
+One habit to know: global options like `--json` go before the command. `agenteye --json sessions` is right; `agenteye sessions --json` is not.
+
+## Script it, wire it into CI
+
+Every command takes `--json`, and that changes everything. Clean JSON goes to stdout while human status and warnings go to stderr, so a `--json` capture pipes straight into `jq` with no stray line to strip. That is what makes the CLI equally good for you at a prompt and for a coding agent parsing output:
+
+```bash
+agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id'
+```
+
+It is built to run unattended. Confirmation prompts auto-skip when no terminal is attached, so nothing hangs in a pipeline, and every command returns a meaningful exit code: `0` success, `4` not logged in, `5` missing a permission (the message names it, for example `alerts:write`), `3` dashboard unreachable. A script can branch on a `4` to reauthenticate or a `5` to tell you exactly what to ask an admin for, instead of failing blind.
+
+## Let a coding agent drive it in plain English
+
+Better yet, you should not have to remember any of these flags at all. The **CLI skill** is a small Agent Skill folder named `agenteye-cli` that teaches a coding agent such as Claude Code or Codex to drive the CLI from plain-English requests. Ask "is anything broken today?" and the agent picks the command, runs it as you, and answers in prose.
+
+For Claude Code, drop the `agenteye-cli` folder into `~/.claude/skills/` and it is auto-discovered. AgentEye provides the folder; there is nothing to deploy and no image to pull, because it only drives the CLI you already installed. Log in yourself first: the skill cannot complete the emailed-code login for you.
+
+Because the agent runs the CLI as you, it can do everything your login permits, reads and writes alike: create keys, change settings, resolve incidents. The CLI's "are you sure?" prompt does not fire for an agent, so the skill is written to state the exact command and wait for your OK before any change. You are the confirmation step.
+
+```text
+you    Why did session run-001 fail?
+
+agent  Running: agenteye --json events --session-id run-001 --all
+       The checkout-agent hit a TimeoutError on its third tool call.
+```
+
+Reads stay instant, and every write pauses for you:
+
+```text
+you    Give CI a key that can only push events.
+
+agent  This creates an API key. I'll run:
+         agenteye keys create ci --add events:add
+       OK to proceed?
+
+you    yes
+
+agent  Done. Key "ci" created with events:add only. The secret is shown once, so store it now.
+```
+
+## Related
+
+- [CLI reference](/agenteye/cli): every command, flag, and JSON shape.
+- [CLI recipes for agents](/agenteye/cli-recipes): copy-paste `jq` patterns and exit-code handling.
+- [CLI agent skill](/agenteye/cli-skill): install and run the `agenteye-cli` skill.
+- [AI assistant](/agenteye/assistant): the in-dashboard analyst that `agent ask` talks to.

docs/agenteye/cli-recipes.mdx

@@ -111,7 +111,7 @@ A multi-org login without `--org` exits non-zero and prints the orgs to choose f
 ## Provision an API key for the SDK/collector
 
 ```bash
-# the secret is printed ONCE — with --json it's the .key field
+# the secret is printed ONCE, with --json it's the .key field
 key=$(agenteye --json keys create ci-bot --add events:read.add | jq -r '.key')
 agenteye keys regenerate ci-bot --yes    # rotate; agenteye keys disable ci-bot --yes to revoke
 ```

docs/agenteye/cli-skill.mdx

@@ -102,7 +102,7 @@ agent ▸ This creates an API key. I'll run:
 you   ▸ yes
 
 agent ▸ Done. Key "ci" created with events:add only.
-        The secret is shown only once — store it now, I can't reprint it.
+        The secret is shown only once, so store it now. I can't reprint it.
 ```
 
 The skill maps each plain-English intent to the right `agenteye` command, discovering valid values first (`list <kind>`, `whoami`) so it doesn't guess, and stating the exact command before any change. More examples: