Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ from microsoft_agents_a365.observability.core import (
configure(
service_name="my-agent",
service_namespace="my-namespace",
token_resolver=lambda agent_id, tenant_id: get_auth_token(),
token_resolver=lambda agent_id, tenant_id: get_app_only_obs_token(agent_id, tenant_id),
cluster_category="prod"
)

Expand Down Expand Up @@ -388,8 +388,8 @@ BatchSpanProcessor ← Accumulate spans
▼
Agent365Exporter.export() ← Send to backend
├── Partition by (tenant_id, agent_id)
├── Resolve endpoint via PowerPlatformApiDiscovery
└── POST to /maven/agent365/agents/{agentId}/traces
├── Resolve app-only OBS token via token_resolver(agent_id, tenant_id)
└── POST to /observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces
```

### MCP Tool Discovery Flow
Expand Down Expand Up @@ -434,7 +434,7 @@ from microsoft_agents_a365.observability.core import configure
configure(
service_name="my-agent",
service_namespace="my-namespace",
token_resolver=lambda agent_id, tenant_id: get_token(),
token_resolver=lambda agent_id, tenant_id: get_app_only_obs_token(agent_id, tenant_id),
cluster_category="prod", # or "ppe", "test"
# Advanced options via exporter_options parameter:
# max_queue_size=2048,
Expand Down
13 changes: 10 additions & 3 deletions docs/integrating-with-existing-opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This guide is for developers whose application **already** initializes OpenTelem

## The integration rule

> **Initialize your existing OpenTelemetry stack first, then call Agent 365's `configure()`.** The SDK detects the existing `TracerProvider` and adds its processors to it. Your existing backend receives every span; the Agent 365 backend also receives spans when `ENABLE_A365_OBSERVABILITY_EXPORTER=true` and a `token_resolver` is provided (otherwise `configure()` falls back to `ConsoleSpanExporter`).
> **Initialize your existing OpenTelemetry stack first, then call Agent 365's `configure()`.** The SDK detects the existing `TracerProvider` and adds its processors to it. Your existing backend receives every span; the Agent 365 backend also receives spans when `ENABLE_A365_OBSERVABILITY_EXPORTER=true` and an app-only OBS `token_resolver` is provided (otherwise `configure()` falls back to `ConsoleSpanExporter`). If the Agent 365 exporter is enabled without a resolver, the console fallback is kept and nothing is sent to Agent 365.

The detection happens in [`config.py`](../libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/config.py): if a real (non-no-op) `TracerProvider` is already set (detected via a non-None `resource` attribute), `configure()` adds an `_EnrichingBatchSpanProcessor` (wrapping the configured exporter) and a custom `SpanProcessor` to that provider rather than creating a new one.

Expand All @@ -25,7 +25,7 @@ configure_azure_monitor(connection_string=os.environ["APPLICATIONINSIGHTS_CONNEC
configure(
service_name="my-agent",
service_namespace="my-namespace",
token_resolver=my_token_resolver,
token_resolver=my_app_only_obs_token_resolver,
)
```

Expand Down Expand Up @@ -55,12 +55,19 @@ trace.set_tracer_provider(provider)
configure(
service_name="my-agent",
service_namespace="my-namespace",
token_resolver=my_token_resolver,
token_resolver=my_app_only_obs_token_resolver,
)
```

→ Runnable version: [`observability-with-otlp`](https://github.com/microsoft/Agent365-Samples/tree/main/python/observability-with-otlp) sample (defaults to `ConsoleSpanExporter` for zero setup).

The Agent 365 backend exporter always uses
`/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1`.
The resolver must return an app-only OBS token for the exporting agent identity;
delegated workload/OBO tokens are not read from request context and are rejected
by the S2S service. Cache the resolver result and refresh near expiry because it
is invoked for each export batch/identity group.

## Auto-instrumentation vs. manual instrumentation

The OTel **backend** (where spans go) and the **instrumentation style** (how spans are produced) are independent axes. You can mix them freely.
Expand Down
27 changes: 27 additions & 0 deletions libraries/microsoft-agents-a365-observability-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,33 @@

All notable changes to this package will be documented in this file.

## [Unreleased]

### Breaking Changes

- **OBS exports always use `/observabilityService`** — The `use_s2s_endpoint`
option is deprecated and ignored, even when `False`. Exports no longer select
or fall back to `/observability`. Provide an app-only OBS token independently
of your agent's workload auth; the S2S service rejects delegated `scp` tokens.
- **OBS export requires the configured app-only resolver** — The exporter uses
only `token_resolver(agent_id, tenant_id)` for Agent 365 authentication. If
the Agent 365 exporter is enabled without a resolver, the existing console
fallback is kept and nothing is sent to Agent 365. Empty tokens or acquisition
failures fail export without an HTTP request or delegated fallback. The
exporter invokes the resolver on every export batch/identity group, so
resolvers must cache the acquired token and refresh only near expiry.

### Migration

- Request the OBS resource `/.default` scope
(`api://9b975845-388f-4429-889e-eab1ef63949c/.default`) for the exporting
agent identity. Do not use delegated `Agent365.Observability.OtelWrite` tokens
for OBS export.
- Validate resolver tokens before returning them: reject any `scp` claim and any
`idtyp` other than `app`; when `idtyp` is absent, accept only a non-empty
`roles` array or a non-empty `oid` equal to `sub`; verify `aud` is the OBS
resource and the token is not expired.

## [0.3.0]

### Breaking Changes
Expand Down
30 changes: 29 additions & 1 deletion libraries/microsoft-agents-a365-observability-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,35 @@ pip install microsoft-agents-a365-observability-core

For usage examples and detailed documentation, see the [Observability documentation](https://learn.microsoft.com/microsoft-agent-365/developer/observability?tabs=python) on Microsoft Learn.

### Agent 365 OBS export authentication

When `ENABLE_A365_OBSERVABILITY_EXPORTER` is enabled, exports always use the
S2S OTLP route:

```text
/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1
```

The deprecated `use_s2s_endpoint` option is ignored, even when set to `False`;
domain overrides change only the host. The exporter never falls back to
`/observability` and never reads delegated request-context tokens.

Provide an app-only OBS `token_resolver(agent_id, tenant_id)` for the exporting
agent identity. The resolver is invoked for each export batch and identity
group, so it should cache the acquired token and refresh near expiry. Empty
tokens or resolver failures fail that export batch without sending an HTTP
request or retrying on a delegated route. If the Agent 365 exporter is enabled
without a resolver, the existing `ConsoleSpanExporter` fallback is kept and
nothing is sent to Agent 365.

Resolvers should request the OBS resource `/.default` scope
(`api://9b975845-388f-4429-889e-eab1ef63949c/.default`) and validate the token
before returning it: reject any `scp` claim and any `idtyp` other than `app`;
when `idtyp` is absent, accept only a non-empty `roles` array or a non-empty
`oid` equal to `sub`; also verify `aud` is the OBS resource and the token is not
expired. Workload authentication for MCP, Microsoft Graph, and other OBO calls
is separate and unchanged.

## Support

For issues, questions, or feedback:
Expand All @@ -33,4 +62,3 @@ For issues, questions, or feedback:
Copyright (c) Microsoft Corporation. All rights reserved.

Licensed under the MIT License - see the [LICENSE](../../LICENSE.md) file for details.

Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ from microsoft_agents_a365.observability.core import configure
configure(
service_name="my-agent",
service_namespace="my-namespace",
token_resolver=lambda agent_id, tenant_id: get_token(),
token_resolver=lambda agent_id, tenant_id: get_app_only_obs_token(agent_id, tenant_id),
cluster_category="prod"
)
```
Expand Down Expand Up @@ -219,20 +219,35 @@ This ensures that context values set via `BaggageBuilder` are recorded as span a
**Export flow:**
1. Partition spans by `(tenant_id, agent_id)` tuple
2. For each partition:
- Resolve endpoint via `PowerPlatformApiDiscovery`
- Resolve auth token via `token_resolver(agent_id, tenant_id)`
- Resolve endpoint host via the configured domain override or production endpoint
- Build the S2S OTLP URL `/observabilityService/tenants/{tenant_id}/otlp/agents/{agent_id}/traces?api-version=1`
- Resolve an app-only OBS token via `token_resolver(agent_id, tenant_id)`
- Build OTLP-like JSON payload
- POST to `/maven/agent365/agents/{agentId}/traces`
- POST to the S2S OTLP route
3. Retry transient failures (408, 429, 5xx) up to 3 times with exponential backoff

The exporter always uses the S2S `/observabilityService` route. The
`use_s2s_endpoint` option is deprecated and ignored, including when `False`.
There is no delegated `/observability` fallback for batch export, request export,
401/403/404 responses, missing tokens, or token acquisition failures.

`token_resolver` must return an app-only OBS token for the exporting agent and
tenant. It is invoked on every export batch/identity group, so resolvers should
cache and refresh tokens near expiry. Resolvers should request the OBS resource
`/.default` scope (`api://9b975845-388f-4429-889e-eab1ef63949c/.default`) and
validate the token before returning it: reject any `scp` claim and any `idtyp`
other than `app`; when `idtyp` is absent, accept only a non-empty `roles` array
or a non-empty `oid` equal to `sub`; verify `aud` is the OBS resource and the
token is not expired.

**Configuration via `Agent365ExporterOptions`:**
```python
from microsoft_agents_a365.observability.core.exporters import Agent365ExporterOptions

options = Agent365ExporterOptions(
cluster_category="prod",
token_resolver=my_token_resolver,
use_s2s_endpoint=False,
use_s2s_endpoint=False, # Deprecated and ignored; export still uses S2S.
max_queue_size=2048,
scheduled_delay_ms=5000,
exporter_timeout_ms=30000,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import os
import logging
import threading
from collections.abc import Callable
from typing import Any, Optional

from opentelemetry import trace
Expand All @@ -17,7 +16,7 @@
from opentelemetry.sdk.trace.export import ConsoleSpanExporter

from .exporters.agent365_exporter import _Agent365Exporter
from .exporters.agent365_exporter_options import Agent365ExporterOptions
from .exporters.agent365_exporter_options import Agent365ExporterOptions, TokenResolver
from .exporters.enriching_span_processor import (
_EnrichingBatchSpanProcessor,
)
Expand Down Expand Up @@ -60,7 +59,7 @@ def configure(
service_name: str,
service_namespace: str,
logger_name: str = DEFAULT_LOGGER_NAME,
token_resolver: Callable[[str, str], str | None] | None = None,
token_resolver: TokenResolver | None = None,
cluster_category: str = "prod",
exporter_options: Agent365ExporterOptions | SpectraExporterOptions | None = None,
suppress_invoke_agent_input: bool = False,
Expand All @@ -72,8 +71,8 @@ def configure(
:param service_name: The name of the service.
:param service_namespace: The namespace of the service.
:param logger_name: The name of the logger to collect telemetry from.
:param token_resolver: (Deprecated) Callable that returns an auth token for a given agent + tenant.
Use exporter_options instead.
:param token_resolver: (Deprecated) Callable that returns an app-only OBS token
for a given agent + tenant. Use exporter_options instead.
:param cluster_category: (Deprecated) Environment / cluster category (e.g. "prod").
Use exporter_options instead.
:param exporter_options: Exporter configuration. Pass Agent365ExporterOptions for A365 API
Expand Down Expand Up @@ -103,7 +102,7 @@ def _configure_internal(
service_name: str,
service_namespace: str,
logger_name: str,
token_resolver: Callable[[str, str], str | None] | None = None,
token_resolver: TokenResolver | None = None,
cluster_category: str = "prod",
exporter_options: Agent365ExporterOptions | SpectraExporterOptions | None = None,
suppress_invoke_agent_input: bool = False,
Expand Down Expand Up @@ -270,7 +269,7 @@ def configure(
service_name: str,
service_namespace: str,
logger_name: str = DEFAULT_LOGGER_NAME,
token_resolver: Callable[[str, str], str | None] | None = None,
token_resolver: TokenResolver | None = None,
cluster_category: str = "prod",
exporter_options: Agent365ExporterOptions | SpectraExporterOptions | None = None,
suppress_invoke_agent_input: bool = False,
Expand All @@ -282,8 +281,8 @@ def configure(
:param service_name: The name of the service.
:param service_namespace: The namespace of the service.
:param logger_name: The name of the logger to collect telemetry from.
:param token_resolver: (Deprecated) Callable that returns an auth token for a given agent + tenant.
Use exporter_options instead.
:param token_resolver: (Deprecated) Callable that returns an app-only OBS token for a given
agent + tenant. Use exporter_options instead.
:param cluster_category: (Deprecated) Environment / cluster category (e.g. "prod").
Use exporter_options instead.
:param exporter_options: Exporter configuration. Pass Agent365ExporterOptions for A365 API
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

from .agent365_exporter_options import Agent365ExporterOptions
from .agent365_exporter_options import Agent365ExporterOptions, TokenResolver
from .spectra_exporter_options import SpectraExporterOptions

# Agent365Exporter is not exported intentionally.
# It should only be used internally by the observability core module.
__all__ = ["Agent365ExporterOptions", "SpectraExporterOptions"]
__all__ = ["Agent365ExporterOptions", "SpectraExporterOptions", "TokenResolver"]
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,21 @@

from __future__ import annotations

import asyncio
import inspect
import json
import logging
import threading
import time
from collections.abc import Callable, Sequence
from typing import Any, final
from collections.abc import Awaitable, Sequence
from typing import Any, cast, final

import requests
from opentelemetry.sdk.trace import ReadableSpan
from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult
from opentelemetry.trace import StatusCode

from .agent365_exporter_options import TokenResolver
from .utils import (
DEFAULT_MAX_PAYLOAD_BYTES,
build_export_url,
Expand All @@ -43,20 +46,23 @@
logger = logging.getLogger(__name__)


async def _await_token(awaitable: Awaitable[str | None]) -> str | None:
return await awaitable


@final
class _Agent365Exporter(SpanExporter):
"""
Agent 365 span exporter for Agent 365:
* Partitions spans by (tenantId, agentId)
* Builds OTLP-like JSON: resourceSpans -> scopeSpans -> spans
* POSTs per group to https://{endpoint}/observability/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1
* or, when use_s2s_endpoint is True, https://{endpoint}/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1
* Adds Bearer token via token_resolver(agentId, tenantId)
* POSTs per group to the S2S /observabilityService OTLP route.
* Adds an app-only authorization token via token_resolver(agentId, tenantId)
"""

def __init__(
self,
token_resolver: Callable[[str, str], str | None],
token_resolver: TokenResolver,
cluster_category: str = "prod",
use_s2s_endpoint: bool = False,
max_payload_bytes: int = DEFAULT_MAX_PAYLOAD_BYTES,
Expand Down Expand Up @@ -126,22 +132,30 @@ def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:

headers = {"content-type": "application/json"}
try:
token = self._token_resolver(agent_id, tenant_id)
if token:
token = self._resolve_token(agent_id, tenant_id)
if token is not None and token.strip():
# Warn if sending bearer token over non-HTTPS connection
if not url.lower().startswith("https://"):
logger.warning(
"Bearer token is being sent over a non-HTTPS connection. "
"This may expose credentials in transit."
)
headers["authorization"] = f"Bearer {token}"
logger.debug(f"Token resolved successfully for agent {agent_id}")
logger.debug(
f"App-only OBS token resolved successfully for agent {agent_id}"
)
else:
logger.debug(f"No token returned for agent {agent_id}")
logger.error(
f"No app-only OBS token returned for agent {agent_id}, "
f"tenant {tenant_id}; export request will not be sent"
)
any_failure = True
continue
except Exception as e:
# If token resolution fails, treat as failure for this group
logger.error(
f"Token resolution failed for agent {agent_id}, tenant {tenant_id}: {e}"
f"App-only OBS token resolution failed for agent {agent_id}, "
f"tenant {tenant_id}: {e}"
)
any_failure = True
continue
Expand Down Expand Up @@ -276,6 +290,24 @@ def _post_with_retries(self, url: str, body: str, headers: dict[str, str]) -> bo
return False
return False

def _resolve_token(self, agent_id: str, tenant_id: str) -> str | None:
token = self._token_resolver(agent_id, tenant_id)
if inspect.isawaitable(token):
try:
asyncio.get_running_loop()
except RuntimeError:
return asyncio.run(_await_token(cast(Awaitable[str | None], token)))
if inspect.iscoroutine(token):
token.close()
elif isinstance(token, asyncio.Future):
token.cancel()
raise RuntimeError(
"Agent365Exporter cannot await an async token_resolver while running "
"inside an active event loop; use a synchronous cached resolver or refresh "
"the app-only OBS token before export."
)
return token

# ------------- Payload mapping ------------------

def _map_and_truncate_spans(
Expand Down
Loading
Loading