You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
a365 setup all no longer requests Observability API (Agent365.Observability.OtelWrite) permissions for blueprint agents in any auth mode, so those agents need no Observability admin consent. Registration becomes their only Observability authorization, so a failed or unverifiable registration now fails setup (exit 1).
This follows the 3P Dev Scale scrum decision to make the no-consent flow the main path rather than an opt-in flag. The PR originally added --skip-observability-permissions; that flag is gone.
Why
With microsoft/Agent365-nodejs#290 and microsoft/Agent365-Samples#339, agents export telemetry to the S2S endpoint with an app-only token for their own agent identity. The endpoint authorizes a token without OtelWrite when the agent is registered, which setup all already does for blueprint agents. Requesting OtelWrite (and its admin consent) is therefore unnecessary.
Validated live (dom97), sending through the real @microsoft/opentelemetry S2S exporter:
Agent identity
Token
Result
Registered, no role
idtyp=app, roles=[], no scp
200, delivered to all sinks
Unregistered, no role
same
403insufficient_scope
Unregistered, OtelWrite role
roles=[OtelWrite]
200
Behavior
Blueprint agents, every auth mode (obo, s2s, both) and every cloud: no Observability API in inheritable permissions, app-role grants, or admin consent URLs. The dry run and setup output say so. Registration is then the agent's only Observability authorization, so a registration failure is an error (exit 1).
--authmode s2s|both: still grant any other app-role specs (for example Defender, once Add Defender permissions part of "a365 setup all" #485 lands), but not OtelWrite; the S2S endpoint authorizes registered agents without it in every mode.
AI Teammate setup: unchanged. Instance creation in the admin center couldn't be validated end to end in the test tenant (it fails tenant-wide for unrelated agents too).
Registration: main (Add cloud-aware endpoints and harden GCC blueprint setup #478) already makes --agent-registration-only exit 1 when registration fails. This PR also exits 1 when an existing registration can't be verified, and for full setup of blueprint agents, including when a missing blueprint client secret prevents identity creation.
Re-running setup does not revoke permissions granted earlier.
Upgrade note
Agents on older SDKs that export through the delegated route need OtelWrite. The CHANGELOG upgrade note covers existing blueprints (Entra portal) and new ones (a365 setup permissions custom --resource-app-id <Observability app ID for your cloud> --scopes Agent365.Observability.OtelWrite).
New and updated tests cover the default plan (omits Observability), s2s/both from the flag or config (keeps it), AI Teammate (keeps it), spec and consent-URL wiring, registration severity, and the summary. Each new test fails against a mutation of the line it guards. GCC cases pin the skip filter and consent-URL clear across clouds.
Follow-ups
Decide the AI Teammate default once admin-center instance creation can be validated.
a365 setup permissions bot still configures Observability API.
Track pending S2S app roles per role and after the az rest fallback (deferred; no spec carries more than one app role today).
SDK: make the exporter's 403 log point to registration.
Blueprint agents that export telemetry through the app-only S2S endpoint
(microsoft/Agent365-nodejs#290, microsoft/Agent365-Samples#339) are
authorized by their agent registration, so the Observability API OtelWrite
permission, and the admin consent it needs, is unnecessary for them.
- New opt-in `setup all --skip-observability-permissions` omits Observability
API from the permission specs (inheritable permissions, app role grants,
batch consent) and from the per-resource and combined admin consent URLs.
Defaults are unchanged: the published SDKs still export to the non-S2S
endpoint by default.
- The flag fails fast for AI Teammate agents and with authMode s2s/both,
since OtelWrite is the only app role those modes grant. A contradicting
--authmode flag is rejected before bootstrap signs in.
- With the flag, a failed agent registration is an error (exit 1), because
registration is then the agent's only Observability authorization.
- Fix: `setup all --agent-registration-only` exited 0 when registration failed.
- Dry run plan, setup summary, CHANGELOG, and docs updated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
The reason will be displayed to describe this comment to others. Learn more.
Requesting changes. The flag is well scoped and the incompatible-combination guards look right, but two gaps undercut the contract the docs promise ("setup exits with code 1 if registration fails"). Details inline. Both need a regression test.
Per 3P Dev Scale scrum feedback, the no-consent flow becomes the main
path instead of an opt-in flag.
- Remove --skip-observability-permissions. Blueprint agents in the
default (obo) auth mode no longer request Observability API
permissions; registered agents export telemetry with an app-only
token over the S2S endpoint.
- authMode s2s/both keep requesting OtelWrite, the only app role those
modes grant; `both` also covers agents whose SDK still exports
through the delegated (OBO) route.
- AI Teammate setup is unchanged until instance creation can be
validated end to end.
- Registration failure stays an error on the default path.
- Tests: the default plan omits Observability; s2s/both (flag or
config) keep it; AI Teammate keeps it. Mutation-checked.
Validated live: a roleless app-only token for a registered agent
identity exports 200 on S2S; an unregistered identity gets 403
insufficient_scope.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
Krishnadheeraj (DheerajPannala)
changed the title
Add --skip-observability-permissions to setup all
Stop requesting Observability API permissions for blueprint agents by default
Sep 24, 2026
The generated API documentation is grammatically incomplete here: it renders as “Observability API unless ...” because the new text omits “is included.”
The S2S endpoint authorizes registered agents without OtelWrite whatever
the auth mode, so s2s/both no longer request it either. They still grant
any other app-role specs (e.g. Defender once #485 lands). Agents whose SDK
still exports through the delegated route grant OtelWrite manually, as the
CHANGELOG upgrade note describes. AI Teammate setup is unchanged.
Tests encode the changed requirement for s2s/both (flag or config) and are
mutation-checked.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
The Step 4 comment still says the permission-spec build stamps Observability, but this branch intentionally excludes it when SkipObservabilityPermissions is true. Keeping that description here makes the implementation contract misleading for future changes; state the conditional explicitly.
- When registration is required (--agent-registration-only, or Observability
permissions not requested), an inconclusive registration check now fails
setup instead of passing. The stored ID is kept and no duplicate
registration is created. The optional path still retains the stored ID.
- When Observability is not included, drop an Observability consent entry
saved by an earlier run so the admin is not asked for it.
- Keep Observability for an AI Teammate config retained for a dry run (skip
only for an effective blueprint selection).
- Scope the guided-setup OtelWrite grant steps to AI Teammates and SDKs that
still export through the delegated route; fix two stale doc comments.
Regression tests cover each case and are mutation-checked.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
Make the upgrade note one consumer-facing sentence, and update the Fixed
entry: setup exits 1 when registration fails or cannot be verified for
blueprint agents as well as with --agent-registration-only. Replace
"without the flag" in a registration test, since the flag was removed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
When a prior non-admin run persisted an Observability consent URL, a subsequent run with Observability skipped can leave that stale entry behind whenever TenantWideConsentOutcome is Granted (or the blueprint ID is absent), because this method returns before PopulateAdminConsentUrls performs the removal. The generated config can therefore still advertise an admin-consent URL for a permission this run did not request; move the stale-entry cleanup before this early return (while preserving any intentional record-retention policy).
The upgrade note opened by saying every existing agent needs the
Observability permissions, which contradicted the S2S exception. Scope
the heading and requirement to agents that export through the delegated
(OBO) route.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
…y-permissions
Conflict resolutions:
- GetFixedApiPermissionSpecs / admin-consent URL builders take both the
cloud environment (#478) and includeObservability (#501); the skipped
Observability spec, URL, and combined-URL scope stay omitted in every cloud.
- Portal walkthrough filter and ClearSkippedObservabilityConsentUrl match any
cloud's Observability app ID (ConfigConstants.IsObservabilityApiAppId).
- S2S PowerShell keeps the spec-driven per-target block (resource IDs come from
the cloud-aware specs); delegated Observability block keeps the skip gate and
uses #478's cloud resource ID and Graph base URL.
- Registration failure keeps RecordRegistrationFailure (a superset of #478's
--agent-registration-only error); #478's registration-only test now accepts
the longer message, and #501's duplicate registration-only test is removed.
- CHANGELOG: kept all #478 entries, dropped the "full setup continues to treat
registration as best-effort" clause that #501 changes for blueprint agents,
and pointed Option B at the Observability app ID for the user's cloud.
- Tests: GCC cases pin the cross-cloud skip filter and consent-URL clear.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
After merging #478, sovereign clouds use their own Observability app IDs, so
the README's opt-back-in command no longer hard-codes the commercial ID.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
The new rule says every blueprint agent must use app-only S2S telemetry, but Phase 2 still branches on workload authMode: its OBO branch installs the delegated Hosting/Runtime packages and Python says S2S dependencies are optional. An OBO blueprint therefore reaches Phases 3–5 without the dependencies required by the mandated app-only resolver. Update Phase 2 for non-AI-Teammate blueprints to install the S2S dependency set regardless of workload auth mode.
The guided-setup docs now point blueprint agents to the agent365-skills instrument-observability skill (S2S with an app-only resolver, and registration as the first fix for a 403).
The CHANGELOG line uses your wording. The upgrade note limits Option A to existing blueprints and gives a365 setup permissions custom for new ones.
A missing blueprint client secret now exits 1.
The dry run and the --authmode help say that blueprint agents are granted no S2S app roles by default.
The delegated PowerShell block is gated on the Observability skip.
Copilot's comment and blueprint-SP threads are fixed. The two per-role tracking threads are deferred, as you suggested, with replies.
Your questions
--authmode s2s for blueprint agents: right, it grants nothing today. OtelWrite was the only app role setup requested, and custom permissions carry delegated scopes only. It stays accepted, and the help text, dry run and summary now say so ("blueprint agents grant none by default", "not required (no S2S app roles to grant)"). Whether to deprecate it for blueprint agents can be a follow-up.
Registration permission: the call uses the CLI client app's delegated Graph token and needs Microsoft Graph AgentRegistration.ReadWrite.All consented on that app; setup checks that consent before registering. The failure message now names the permission. I haven't found a documented directory-role requirement beyond it, so the message doesn't name a role.
Opting back in with customBlueprintPermissions:Commands/SetupSubcommands/README.md now has a line for it. It notes that custom permissions need admin-run consent, because they aren't in the non-admin combined consent URL.
Nits: the design.md table (Non-DW Observability is now "—"), the closing-line rule ("granted or not required") and the one-line comment are fixed.
Observability specs, consent URLs and PowerShell now use each cloud's Observability app ID. Blueprint agents still skip Observability in every cloud.
The portal-walkthrough filter and ClearSkippedObservabilityConsentUrl match any cloud's Observability app ID. New GCC test cases pin both and fail when the filters are narrowed to the commercial ID.
CHANGELOG: I kept all Add cloud-aware endpoints and harden GCC blueprint setup #478 entries but dropped its "full setup continues to treat registration as best-effort" clause, since this PR makes registration failure exit 1 for blueprint agents. The upgrade note and README now use the Observability app ID for the user's cloud.
Full suite: 2289 passed, 12 skipped (the same pre-existing skips), 0 failed.
Cross-PR: I've noted the SDK exporter's generic 403 log as a follow-up in the description. agent365-skills#84 now explains that its application-only OtelWrite fallback is for the S2S route, while this CHANGELOG note covers the delegated route.
Separate default Observability omission from effective requested permissions when custom Observability permissions are configured, and update dry-run/help/docs for cloud-aware S2S guidance.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
Pushed 69fb074 for Copilot's re-review after the merge. I replied on each thread. In summary:
A custom Observability opt-back-in (customBlueprintPermissions or a365 setup permissions custom) is no longer treated as skipped. It keeps its saved consent URL, and a registration failure stays a warning for it.
The guided-setup docs no longer hard-code the commercial Observability ID or a tenant-specific service-principal object ID.
Phase 2 now installs the app-only S2S dependencies for every non-AI-Teammate blueprint agent, regardless of workload auth mode. Copilot flagged this as "previously missed".
This branch records the identity failure only in Errors, but DisplaySetupSummary renders every AgentIdentityFailed row as failed — see warnings. For the missing-secret path the summary therefore points users to a warning list that does not contain the failure. Track the failure severity explicitly or update the renderer so this row points to errors.
This issue also appears in the following locations of the same file:
A custom Observability opt-back-in must now name the configured cloud's Observability app ID and request Agent365.Observability.OtelWrite. A wrong-cloud ID or other scope no longer counts.
The setup summary now tracks the severity of agent identity and registration failures explicitly, so each failed row points to the list that actually holds the failure. This was the "previously missed" item: before, the missing-blueprint-secret path said "see warnings" but recorded an error. Registration now follows where the failure was recorded instead of inferring it from the Observability skip.
In the guided-setup docs, the PowerShell app-role alternative is now limited to AI Teammates. Blueprint agents on the delegated route are pointed to a365 setup permissions custom.
NoS2SAppRolesToGrant is initialized only when this method runs, but the caller invokes it only after an agent identity exists. If identity creation fails (including the new missing-secret error path), an s2s/both run with no app-role specs leaves this flag false, so the summary can report PENDING or a tenant-wide delegated grant instead of “no S2S app roles to grant.” Derive this state from specs before the identity-existence gate so failure summaries remain accurate.
EffectiveAuthMode and NoS2SAppRolesToGrant were set only after an agent
identity existed, so when identity creation failed an s2s/both run with no
app roles to grant could be summarized as a pending or delegated grant
instead of "no S2S app roles to grant".
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 5cbf5f6b-cc40-4b7e-a591-65848db73a12
Pushed b21addd for the item Copilot's latest pass flagged as "previously missed". The auth mode and whether any S2S app role is requested are now recorded before the agent identity step. Before, they were set only after an identity existed, so an s2s run whose identity creation failed could be summarized with delegated-consent wording instead of "not required (no S2S app roles to grant)". A new end-to-end test covers this: an s2s run with a missing blueprint secret, checked through DisplaySetupSummary. Full suite: 2302 passed, 12 skipped, 0 failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
documentationImprovements or additions to documentation
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
Bot
added
the
documentation
Rick Brighenti (rbrighenti)
left a comment
Summary
a365 setup allno longer requests Observability API (Agent365.Observability.OtelWrite) permissions for blueprint agents in any auth mode, so those agents need no Observability admin consent. Registration becomes their only Observability authorization, so a failed or unverifiable registration now fails setup (exit 1).This follows the 3P Dev Scale scrum decision to make the no-consent flow the main path rather than an opt-in flag. The PR originally added
--skip-observability-permissions; that flag is gone.Why
With microsoft/Agent365-nodejs#290 and microsoft/Agent365-Samples#339, agents export telemetry to the S2S endpoint with an app-only token for their own agent identity. The endpoint authorizes a token without
OtelWritewhen the agent is registered, whichsetup allalready does for blueprint agents. Requesting OtelWrite (and its admin consent) is therefore unnecessary.Validated live (dom97), sending through the real
@microsoft/opentelemetryS2S exporter:idtyp=app,roles=[], noscpinsufficient_scoperoles=[OtelWrite]Behavior
obo,s2s,both) and every cloud: no Observability API in inheritable permissions, app-role grants, or admin consent URLs. The dry run and setup output say so. Registration is then the agent's only Observability authorization, so a registration failure is an error (exit 1).--authmode s2s|both: still grant any other app-role specs (for example Defender, once Add Defender permissions part of "a365 setup all" #485 lands), but not OtelWrite; the S2S endpoint authorizes registered agents without it in every mode.--agent-registration-onlyexit 1 when registration fails. This PR also exits 1 when an existing registration can't be verified, and for full setup of blueprint agents, including when a missing blueprint client secret prevents identity creation.Upgrade note
Agents on older SDKs that export through the delegated route need OtelWrite. The CHANGELOG upgrade note covers existing blueprints (Entra portal) and new ones (
a365 setup permissions custom --resource-app-id <Observability app ID for your cloud> --scopes Agent365.Observability.OtelWrite).Testing
Follow-ups
a365 setup permissions botstill configures Observability API.az restfallback (deferred; no spec carries more than one app role today).