Skip to content
5 changes: 5 additions & 0 deletions .sampo/changesets/mcp-analytics-defaults.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
pypi/posthog: minor

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated comment by QA Swarm — not written by a human

[convergent: router + paul] 🟠 HIGH

Two reviewers landed on this line independently: is minor the right tier?

Verified empirically — with capture_model=True by default, the advertised inputSchema of every tool on the official high-level FastMCP / MCPServer adapters now lists llm_model in required (next to the already-default context). On a routine pip install --upgrade posthog, every existing deployment's wire-visible tool contract changes with zero code change by the user. enable_conversation_id=True adds a handle to eligible tool responses on top of that.

The dispatch path does not enforce the required flag — a call omitting context and llm_model still dispatches fine — so there is no server-side breakage. The risk is client-side: strict-schema MCP clients that validate before sending, and tooling that generates call templates from a cached schema.

Paul's read: "i can talk myself into minor (capture_model only landed in #927 six days ago, so the blast radius is genuinely small, and context already set the precedent of injecting a required arg), and i'm not going to block on the tier. but the thing i'd actually want confirmed: references/public_api_snapshot.txt changes on four lines here, and CONTRIBUTING.md now says that means 'this touches public API, agree the shape on the issue first'. i'm lazily asking rather than digging — was this one agreed somewhere?"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping minor: context shipped the same required-argument precedent as a minor, and dispatch never enforces required. On the CONTRIBUTING.md point: this repo has no "agree the shape on an issue first" rule; AGENTS.md only asks that the snapshot be regenerated (make public_api_snapshot), which it is. posthog-js does have that rule for new or changed option shapes, and this change alters no shape: same options, same types, different defaults. The decision and the rejected alternatives are now recorded in posthog-js ADR-0013, which both SDKs cite.

---

Enable MCP model capture and conversation correlation by default. Advertised tool schemas gain an `llm_model` argument (never enforced at dispatch) and eligible tool results gain a conversation handle; `MCPAnalyticsOptions(capture_model=False, enable_conversation_id=False)` restores the previous shape. Fresh low-level instances now read the self-reported model instead of staying silent.
38 changes: 27 additions & 11 deletions posthog/mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,21 +36,29 @@ Request headers use the same identity and package version, so SDK Health can com
Because `$lib` is a client-level identity, `instrument()` relabels every event sent by the client passed to it.
Use a client dedicated to MCP analytics if the application also captures unrelated events.

## Capture the calling model
## Defaults and opt-outs

Model capture is off by default. Enable it for an instrumented MCP Python SDK 1.x or
2.x server:
Intent, model capture, conversation correlation, and MCP exception capture are on by default.
Comment thread
lucasheriques marked this conversation as resolved.
Missing-capability reporting and feedback collection remain off.

```python
from posthog.mcp import MCPAnalyticsOptions, instrument

analytics = instrument(
server,
posthog,
MCPAnalyticsOptions(capture_model=True),
)
instrument(server, posthog, MCPAnalyticsOptions(capture_model=False, enable_conversation_id=False))
Comment thread
lucasheriques marked this conversation as resolved.
```

Model capture adds an `llm_model` argument to compatible tool schemas, required on the official
high-level adapters and optional elsewhere. Dispatch never enforces it, so servers keep working;
strict-schema clients see the new field. Set `capture_model=False` to leave schemas untouched.
Conversation correlation adds an optional `conversation_id` argument and returns a handle in
eligible tool results. Clients must echo it to group later calls; calls without it mint new handles.
Set `enable_conversation_id=False` to retain transport-based session grouping and unchanged
response content. Custom `PostHogMCP` dispatchers enable model capture by default but still
supply their own session IDs. The reasoning is recorded in posthog-js `docs/adr/0013`.

## Capture the calling model

Model capture is on by default for instrumented MCP Python SDK 1.x and 2.x servers.
The SDK records the best model identifier visible to the server as
`$mcp_llm_model`. Recognized client metadata wins and sets
`$mcp_llm_model_source` to `client_metadata`. The SDK also adds an `llm_model`
Expand All @@ -75,13 +83,21 @@ If a tool already declares `llm_model`, or uses a root `$ref`, `oneOf`, `allOf`,
`anyOf` schema, PostHog leaves the schema and argument untouched. Client metadata
can still be captured in those cases.

For a custom dispatcher, use the same option on `PostHogMCP` and pass request
A raw low-level server learns ownership while serving `tools/list`. A fresh instance that never
served one has no answer, so it records `llm_model` as the self-reported model and strips
nothing (posthog-js ADR-0011: reads fail open, strips fail closed). A tool that declares its own
`llm_model` on such an instance is therefore recorded under `$mcp_llm_model` until a listing says
otherwise; `capture_model=False` or `before_send` are the escapes. High-level adapters and
standalone `fastmcp.FastMCP` read ownership from the registered tool schema, so they are
unaffected and need no prior listing.

For a custom dispatcher, `PostHogMCP` enables the same option by default; pass request
metadata through explicitly:

```python
from posthog.mcp import PostHogMCP

posthog = PostHogMCP("phc_...", capture_model=True)
posthog = PostHogMCP("phc_...")
tools = posthog.prepare_tool_list(server_tools)
original_tool = next(tool for tool in server_tools if tool["name"] == tool_name)
call = posthog.prepare_tool_call(
Expand Down Expand Up @@ -229,7 +245,7 @@ sess = get_mcp_session(request) # sess.session_id, sess.client_name, ...

### Or skip the middleware entirely: conversation ids

`MCPAnalyticsOptions(enable_conversation_id=True)` derives `$session_id` from the
`enable_conversation_id` is on by default and derives `$session_id` from the
agent's conversation handle, deterministically and identically on every pod. That
needs no middleware and no ordering discipline, and it is the only thing that
correlates a session under the 2026-07-28 revision's per-request server instances.
Expand Down
238 changes: 213 additions & 25 deletions posthog/mcp/_instrument_lowlevel.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,14 @@

from __future__ import annotations

import functools
import inspect
import time
from typing import Any, Optional, Tuple
from typing import Any, Dict, List, Optional, Tuple

import mcp.types as mcp_types

from ._context_parameters import schema_has_param
from ._context_parameters import is_context_enabled, schema_has_param
from ._conversation_id import build_prompt_back
from ._event_types import MCPAnalyticsEventType
from ._instrumentation import (
Expand All @@ -42,6 +43,7 @@
)
from ._internal import MCPAnalyticsData
from ._model_parameters import request_meta_from_context
from ._model_parameters import can_inject_model_parameter, is_capture_model_enabled
from ._output_instructions import mirror_instructions_into_structured_content
from .logger import log
from .tools import get_more_tools_result_text, resolve_missing_capability_tool_name
Expand Down Expand Up @@ -182,6 +184,11 @@ def _wrap_call_tool(
async def handler(req: Any) -> Any:
name = req.params.name
arguments = dict(req.params.arguments or {})
strip, model_ours = (
await _standalone_ownership(data, high_level, name, req.params.meta)
if strip_injected
else (set(), data.tool_model_parameter_injected.get(name))
)
client_name, client_version = _client_info(server)
protocol_version = _protocol_version(server)
mcp_session_id = _mcp_session_id(server)
Expand All @@ -195,9 +202,8 @@ async def handler(req: Any) -> Any:
name=name,
arguments=arguments,
request_meta=request_meta_from_context(_request_context(server)),
allow_self_reported_model=data.tool_model_parameter_injected.get(
name, False
),
# Reads fail open on unknown ownership (posthog-js ADR-0011).
allow_self_reported_model=model_ours is not False,
mcp_session_id=mcp_session_id,
token=token,
client_name=client_name,
Expand Down Expand Up @@ -237,14 +243,9 @@ async def handler(req: Any) -> Any:
# but NOT a key the tool declares itself (that's a real argument). Ownership
# is read from the tool's own signature, so it holds with or without a prior
# tools/list and across stateless per-request server instances.
if strip_injected and req.params.arguments:
owned = await _tool_owned_injected_keys(high_level, name)
injected_keys = ["context", "conversation_id"]
if data.tool_model_parameter_injected.get(name, False):
injected_keys.append("llm_model")
for key in injected_keys:
if key not in owned:
req.params.arguments.pop(key, None)
if strip and req.params.arguments:
for key in strip:
req.params.arguments.pop(key, None)

# Settle the shared session before the tool body runs, so an in-tool
# `analytics.capture()` is attributed to this caller and not the last one.
Expand Down Expand Up @@ -325,6 +326,7 @@ def _inject_tool_schemas(
population pass, so the schema the SDK validates against always matches the
one we advertised — see the note in ``handler``.
"""
verdicts: Dict[str, bool] = {}
for tool in tools:
schema = getattr(tool, "inputSchema", None)
mutate_tool_schema(
Expand All @@ -334,6 +336,14 @@ def _inject_tool_schemas(
owns_context=schema_has_param(schema, "context"),
context_required=context_required,
)
verdict = data.tool_model_parameter_injected.get(tool.name)
if verdict is None:
continue
if verdicts.setdefault(tool.name, verdict) != verdict:
# Two advertised tools share this name and disagree (FastMCP 2.x
# lists a middleware tool beside the registered one it shadows).
# Which one dispatches is unknown, so the strip fails closed.
data.tool_model_parameter_injected[tool.name] = False


def _wrap_list_tools(
Expand Down Expand Up @@ -441,20 +451,198 @@ async def _feedback_name_owned_by_real_tool(high_level: Any, name: str) -> bool:
return False


async def _tool_owned_injected_keys(high_level: Any, name: str) -> set:
"""Which of (``context``, ``conversation_id``) the jlowin FastMCP tool declares
itself, read from its function signature. These are real tool arguments we must
not strip. On any lookup failure, return empty (strip both) — same as the prior
unconditional behaviour, so a flaky introspection never leaks an injected key."""
if high_level is None:
return set()
_INJECTED_KEYS = ("context", "conversation_id", "llm_model")


async def _standalone_ownership(
data: MCPAnalyticsData, high_level: Any, name: str, meta: Any
) -> Tuple[set, Optional[bool]]:
"""Ownership of the injected arguments on jlowin's standalone FastMCP: the
keys to strip before it validates the call, and whether ``llm_model`` is
ours (``None`` when nothing can say).

Only keys injected under the current options are candidates. ``context``
and ``conversation_id`` are stripped unless the registered schema (or,
without one, the function signature) declares them; a failed lookup strips
both — the prior behaviour, so a flaky introspection never leaks an injected
key into validation. ``llm_model`` is judged by the effective listing first,
because middleware can provide or shadow the tool the registry knows, then
by the registry; with neither witness it stays and is still read — strips
fail closed, reads fail open (posthog-js ADR-0011).
"""
try:
declared, model_injectable = await _registry_view(high_level, name, meta)
registry_trusted = model_injectable is not None and not _dispatch_can_differ(
high_level
)
except Exception: # noqa: BLE001 - ownership inference must never prevent dispatch
declared, model_injectable, registry_trusted = None, None, False
listed = data.tool_model_parameter_injected.get(name)
if listed is not None:
model_ours: Optional[bool] = listed
elif registry_trusted:
model_ours = model_injectable
else:
model_ours = None
candidates = _injected_keys(data)
strip = {k for k in candidates - {"llm_model"} if k not in (declared or set())}
if "llm_model" in candidates and model_ours:
strip.add("llm_model")
return strip, model_ours


def _injected_keys(data: MCPAnalyticsData) -> set:
"""The analytics arguments the SDK injects under the current options — the
only ones it may strip. A disabled feature injects nothing, so its key is
the application's even when the schema does not declare it."""
keys = set()
if is_context_enabled(data.options.context):
keys.add("context")
if data.options.enable_conversation_id:
keys.add("conversation_id")
if is_capture_model_enabled(data.options.capture_model):
keys.add("llm_model")
return keys


async def _registry_view(
high_level: Any, name: str, meta: Any
) -> Tuple[Optional[set], Optional[bool]]:
"""What the registered tool says about the injected keys: which of
``_INJECTED_KEYS`` it declares itself, and whether a listing would have
injected ``llm_model`` into its schema (the same test the listing applies,
on the schema as the client would see it). Read from the schema (a ``Tool``
subclass may have no function) else the signature. The registry is read
directly, never through middleware, so a cold instance answers without a
listing and rate limiters are not charged. ``(None, None)`` when the
registry has no such tool or cannot be read."""
try:
tool = await high_level.get_tool(name)
fn = getattr(tool, "fn", None)
params = set(inspect.signature(fn).parameters) if fn is not None else set()
return {k for k in ("context", "conversation_id") if k in params}
tool = await _registered_tool(high_level, name, meta)
except Exception: # noqa: BLE001 - introspection is best-effort
return set()
return None, None
if tool is None:
return None, None
schema = getattr(tool, "parameters", None)
if isinstance(schema, dict):
return _schema_view(schema, dereferenced=_server_dereferences(high_level))
fn = getattr(tool, "fn", None)
if fn is None:
return set(), True
try:
declared = {k for k in _INJECTED_KEYS if k in inspect.signature(fn).parameters}
except Exception: # noqa: BLE001 - introspection is best-effort
return set(), True
return declared, "llm_model" not in declared


async def _registered_tool(high_level: Any, name: str, meta: Any) -> Any:
"""The tool version the request pinned in ``_meta.fastmcp.version``, else
the highest one — the same choice FastMCP makes when dispatching."""
version = _requested_tool_version(meta)
if version is None:
return await high_level.get_tool(name)
from fastmcp.utilities.versions import VersionSpec

return await high_level.get_tool(name, version=VersionSpec(eq=version))


def _requested_tool_version(meta: Any) -> Optional[str]:
"""Ownership must follow dispatch. Only a FastMCP that exposes the
``_meta`` version extractor its own dispatch uses honours a pinned
version; every earlier release calls the highest version regardless."""
try:
from fastmcp.server.dependencies import extract_version_spec
except ImportError:
return None
dump = getattr(meta, "model_dump", None)
if callable(dump):
meta = dump(by_alias=True)
return extract_version_spec(meta) if isinstance(meta, dict) else None


def _schema_view(schema: Dict[str, Any], *, dereferenced: bool) -> Tuple[set, bool]:
"""The injected keys a schema declares as its own, and whether a listing
would inject ``llm_model`` into it. With dereferencing on, every node along
a root ``$ref`` chain counts, as the client sees the merged schema; with it
off the client sees the reference itself, which the listing never injects
into. Nothing is injected into a composed or unresolvable schema either, so
all keys count as declared there and nothing is stripped."""
nodes = _reference_chain(schema) if dereferenced else [schema]
if nodes is None or any(
node.get(key) for node in nodes for key in ("oneOf", "allOf", "anyOf")
):
return set(_INJECTED_KEYS), False
declared = {
key
for node in nodes
if isinstance(node.get("properties"), dict)
for key in node["properties"]
if key in _INJECTED_KEYS
}
injectable = can_inject_model_parameter(nodes[-1]) and "llm_model" not in declared
return declared, injectable


def _server_dereferences(server: Any) -> bool:
"""Whether this FastMCP dereferences schemas before advertising them (its
built-in middleware, on by default from 3.x; absent on 2.x)."""
return any(
type(middleware).__module__.endswith(".dereference")
for middleware in getattr(server, "middleware", ())
)


_DISPATCH_HOOKS = ("on_message", "on_request", "on_list_tools", "on_call_tool")


def _dispatch_can_differ(server: Any) -> bool:
"""Whether application middleware can provide, shadow, or reroute a tool,
making the registry an unreliable witness for what actually runs. Any
override of a listing or dispatch hook can; FastMCP's own built-ins are
excluded. Without a trustworthy registry a cold instance keeps ``llm_model``
(strips fail closed), which is exactly what it did before this path."""
try:
from fastmcp.server.middleware import Middleware
except ImportError: # FastMCP before middleware existed: nothing can differ
return False

return any(
type(middleware) not in _builtin_middleware_types() # subclasses are the app's
and any(
getattr(type(middleware), hook) is not getattr(Middleware, hook)
for hook in _DISPATCH_HOOKS
)
for middleware in getattr(server, "middleware", ())
)


@functools.lru_cache(maxsize=1)
def _builtin_middleware_types() -> tuple:
"""The middleware a bare ``FastMCP()`` installs on its own, probed rather
than named so a new built-in in a later release is still recognised."""
from fastmcp import FastMCP

return tuple(type(middleware) for middleware in FastMCP("posthog-probe").middleware)


def _reference_chain(schema: Dict[str, Any]) -> Optional[List[Dict[str, Any]]]:
"""The schema and each local root ``$ref`` target in turn; ``None`` for a
reference that is external, dangling, or cyclic."""
nodes = [schema]
while len(nodes) <= 8:
ref = nodes[-1].get("$ref")
if ref is None:
return nodes
if not isinstance(ref, str) or not ref.startswith("#/"):
return None
target: Any = schema
for part in ref[2:].split("/"):
key = part.replace("~1", "/").replace("~0", "~") # JSON Pointer escapes
target = target.get(key) if isinstance(target, dict) else None
if not isinstance(target, dict) or any(target is node for node in nodes):
return None
nodes.append(target)
return None


def _request_context(server: Any) -> Any:
Expand Down
7 changes: 5 additions & 2 deletions posthog/mcp/_instrument_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -486,15 +486,18 @@ def _wrap_v2_call_tool(server: Any, data: MCPAnalyticsData) -> None:
async def handler(ctx: Any, params: Any) -> Any:
name = params.name
arguments = dict(params.arguments or {})
analytics_owns_model = data.tool_model_parameter_injected.get(name, False)
# A raw instance that never served a listing has no ownership answer and
# reads the self-reported model anyway; only a listing that proved the
# application owns `llm_model` stops it (posthog-js ADR-0011).
analytics_owns_model = data.tool_model_parameter_injected.get(name) is not False
standalone = data.standalone_fastmcp() if data.standalone_fastmcp else None
if standalone is not None:
version = _requested_tool_version(ctx)
injected = await _standalone_injected_parameters(
standalone, data, name, version
)
analytics_owns_model = injected is not None and "llm_model" in injected
if injected is not None:
analytics_owns_model = "llm_model" in injected
call_arguments = {
key: value
for key, value in arguments.items()
Expand Down
Loading