feat(langfuse): migrate observability integration

.github/dependabot.yml

@@ -18,3 +18,12 @@ updates:
     groups:
       dsh-deps:
         patterns: ["*"]
+
+  # Langfuse replay and fixture verification dependencies
+  - package-ecosystem: pip
+    directory: /langfuse
+    schedule:
+      interval: weekly
+    groups:
+      langfuse-deps:
+        patterns: ["*"]

.github/workflows/langfuse.yml

@@ -0,0 +1,56 @@
+name: Langfuse integration
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "langfuse/**"
+      - ".github/workflows/langfuse.yml"
+  pull_request:
+    paths:
+      - "langfuse/**"
+      - ".github/workflows/langfuse.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: langfuse-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  verify:
+    name: syntax, fixture, and replay contract
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    defaults:
+      run:
+        working-directory: langfuse
+
+    steps:
+      - name: Check out source
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"
+          cache: pip
+          cache-dependency-path: langfuse/requirements.txt
+
+      - name: Install locked dependencies
+        run: python -m pip install --disable-pip-version-check -r requirements.txt
+
+      - name: Compile scripts
+        run: python -m py_compile demo.py replay.py record_trace.py
+
+      - name: Verify CLI entry points
+        run: |
+          python replay.py --help
+          python record_trace.py --help
+
+      - name: Verify recording and replay contract
+        run: python -m unittest discover -s tests -v

README.md

@@ -14,13 +14,16 @@ app.
 | [`hermes/`](./hermes) | [Hermes Agent](https://github.com/NousResearch/hermes-agent) | `hermes plugins install EverMind-AI/plugins/hermes` | 🧪 built — pre-release verification |
 | [`dsh/`](./dsh) | [DeepSeek Harness](https://github.com/deepseek-ai/DeepSeek-Harness) | `dsh plugin --profile web add @evermind-ai/dsh-plugin` | 🧪 built — pre-release verification |
 | [`dify/`](./dify) | [Dify](https://dify.ai) | Package with the Dify CLI, then upload the `.difypkg` in Dify | 🧪 built — Marketplace submission pending |
+| [`langfuse/`](./langfuse) | [Langfuse](https://langfuse.com) | `python replay.py` for the recorded trace, or enable EverOS OpenTelemetry for live traces | ✅ replay and live tracing integration |
 
 ## Integration models
 
 - **Agent hosts** such as OpenClaw, Hermes, and DSH automate the recall → capture →
   seal lifecycle and fail open when EverOS is unavailable.
 - **Workflow platforms** such as Dify expose explicit search and add tools, so
   builders decide exactly where memory runs in a workflow.
+- **Observability backends** such as Langfuse receive EverOS's native
+  OpenTelemetry traces and recall-quality scores without wrapping the memory API.
 
 Each integration's own README documents its lifecycle, setup, security model,
 and troubleshooting.

langfuse/.gitignore

@@ -0,0 +1,2 @@
+__pycache__/
+.venv/

langfuse/README.md

@@ -0,0 +1,146 @@
+# EverOS × Langfuse (native OpenTelemetry)
+
+EverOS emits OpenTelemetry spans for its own memory operations — write, memcell
+boundary + episode extraction (LLM), search with recall-quality scores, and OME
+reflection — and exports them over OTLP to any backend, including
+[Langfuse](https://langfuse.com). There is **no wrapper and no extra
+instrumentation code**: enable it in config and the traces appear.
+
+Two ways to look at it:
+
+| | What it is | What you need |
+| --- | --- | --- |
+| [Replay a recording](#replay-a-recording-no-everos-needed) | A trace a real EverOS server produced, pushed into your Langfuse project | Langfuse keys only |
+| [Trace your own server](#trace-your-own-server) | Your EverOS, your data, live | An EverOS server |
+
+Clone this integrations repository and enter this directory before running the
+commands below:
+
+```bash
+git clone https://github.com/EverMind-AI/plugins.git
+cd plugins/langfuse
+```
+
+Python 3.11 or newer is required.
+
+## Replay a recording (no EverOS needed)
+
+`recorded_trace.json` is a capture of one real `demo.py` run against EverOS
+1.2.1: 237 spans over 60 traces. Eleven conversations are ingested and flushed,
+each with its LLM extraction and OME strategies nested underneath; reflection
+then consolidates two of them and deprecates what they superseded; and five
+questions are asked of the resulting memory, with their recall scores.
+`replay.py` pushes it into your own Langfuse project, so you can see what the
+integration looks like before deploying anything.
+
+```bash
+pip install -r requirements.txt
+export LANGFUSE_PUBLIC_KEY="pk-lf-..."
+export LANGFUSE_SECRET_KEY="sk-lf-..."
+export LANGFUSE_HOST="https://cloud.langfuse.com"   # US: https://us.cloud.langfuse.com
+python replay.py
+```
+
+Then open Langfuse → **Tracing** and filter on the `replay` tag.
+
+Span names, attributes, token usage, parent/child structure and durations are
+EverOS's own output, replayed verbatim. Three things are rewritten: trace and
+span ids are minted fresh so repeated runs do not collide, timestamps are
+shifted so the trace lands at the current time, and root spans carry a `replay`
+tag so a recording is never mistaken for live traffic.
+
+Two things in the trace list are not self-explanatory. The short keyword
+searches beyond the five questions are `demo.py` waiting for each conversation
+to become searchable. And the OME spans outlast the `flush` span they hang
+under, because reflection continues after the request returns and re-attaches
+to the originating trace through its `traceparent`.
+
+Recall scores are per-method scales: read HYBRID against HYBRID, not against
+AGENTIC. Agent cases and skills are not in this recording; the span and score
+contract is the same when they appear.
+
+## Trace your own server
+
+1. Install the optional OpenTelemetry extra:
+
+   ```bash
+   pip install "everos[otel]"
+   ```
+
+2. Add `[observability]` to your `everos.toml`. The Langfuse keys derive the
+   OTLP endpoint and auth automatically:
+
+   ```toml
+   [observability]
+   enabled             = true
+   langfuse_public_key = "pk-lf-..."
+   langfuse_secret_key = "sk-lf-..."
+   langfuse_host       = "https://us.cloud.langfuse.com"   # EU: https://cloud.langfuse.com
+   # capture_content   = true   # opt-in: also record query / extracted memory text
+   ```
+
+   Container/CI equivalent via env vars: `EVEROS_OBSERVABILITY__ENABLED=true`,
+   `EVEROS_OBSERVABILITY__LANGFUSE_PUBLIC_KEY=...`, and so on.
+
+3. Run EverOS normally, then drive one memory lifecycle through it:
+
+   ```bash
+   everos server start
+   python demo.py            # add -> flush -> search against 127.0.0.1:8000
+   ```
+
+   `demo.py` uses only the standard library and contains no instrumentation
+   code; the spans come from the server. It ingests eleven conversations, nudges
+   reflection (a weekly cron otherwise), then asks five questions, so the
+   traces show recall choosing between memories rather than returning the only
+   one there is.
+
+Off by default — with `enabled = false` (or the `otel` extra absent) there is
+zero tracing overhead.
+
+The signal is plain OTLP/HTTP and vendor-neutral, so the same config exports to
+an OpenTelemetry Collector or any other OTLP backend. The `langfuse_*` keys are
+just a shortcut that fills in the endpoint and auth header for you.
+
+## What you get
+
+| EverOS operation | Langfuse observation |
+| --- | --- |
+| `POST /api/v2/memory/add` · `flush` | span `everos.memory.add` / `everos.memory.flush` |
+| memcell boundary detection (LLM) | generation `everos.memcell.boundary` (model + tokens) |
+| episode extraction (LLM) | generation `everos.extract` |
+| markdown persistence | span `everos.persist.markdown` |
+| `POST /api/v2/memory/search` | retriever `everos.memory.search` → `recall` / `rank` |
+| query / recall embedding | embedding `everos.embedding` |
+| OME extraction strategies | agent `everos.ome.<strategy>` (linked to the triggering request's trace) |
+| reflection consolidating a cluster | span `everos.reflect.consolidate` under `everos.ome.reflect_episodes` |
+
+`langfuse.session.id` / `langfuse.user.id` group the traces. Recall quality is
+pushed as Langfuse scores, split by whether the method's score is calibrated:
+`recall_top_score` plus `recall_hit` for HYBRID / AGENTIC (comparable `[0, 1]`),
+and `recall_top_score_raw` for KEYWORD / single-route VECTOR, whose raw BM25 or
+cosine values are on a different scale and must not be averaged in with the
+calibrated ones. Query and memory text are captured only when
+`capture_content = true`.
+
+## Re-recording the fixture
+
+`record_trace.py` is the maintainer-side tool that produced
+`recorded_trace.json`. It stands in for Langfuse's two ingestion endpoints on
+localhost, so a real EverOS server exports its spans *and* its recall scores
+there instead of to Langfuse. Nothing about the recording is synthesized.
+
+```bash
+python record_trace.py     # sink on :4318; writes the fixture on Ctrl-C
+```
+
+Point `[observability].langfuse_host` at `http://127.0.0.1:4318`, start the
+server, run `demo.py`, then stop the sink. Only worth redoing when the span
+contract changes (a span added, renamed, or given new attributes); ordinary
+releases do not invalidate a recording.
+
+## Learn more
+
+- Langfuse OpenTelemetry: https://langfuse.com/integrations/native/opentelemetry
+- EverOS config reference: the
+  [`[observability]` block](https://github.com/EverMind-AI/EverOS/blob/main/src/everos/config/default.toml).