Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
---
title: How Agent Installation through the AWS Integration Works
description: "Understand how Datadog installs and maintains the Datadog Agent on Amazon EC2 through the AWS integration: the AWS resources created, the installation mechanism, the security model, and the Agent lifecycle."
private: true # TODO(DOCS-14545): remove at v1 rollout to publish, at the same time as the setup guide this page links to; also add the nextlink entry under "AWS guides" in hugo/content/en/integrations/guide/_index.md at that time
further_reading:
- link: "https://docs.datadoghq.com/integrations/guide/aws-agent-installation/"
tag: "Documentation"
text: "Install the Datadog Agent through the AWS Integration"
- link: "https://docs.datadoghq.com/integrations/amazon_web_services/"
tag: "Documentation"
text: "AWS Integration"
- link: "https://docs.datadoghq.com/agent/fleet_automation/"
tag: "Documentation"
text: "Fleet Automation"
---

This page explains how Datadog installs and maintains the Agent on Amazon EC2 through the AWS integration. For setup instructions and the permissions Datadog requires, see [Install the Datadog Agent through the AWS Integration][1].

<div class="alert alert-info">This page covers the Amazon EC2 experience only.</div>

## AWS resources that Datadog creates

The CloudFormation template you launch creates the following resources one time, in a single stack:

| Resource | Name | Purpose |
|---|---|---|
| EventBridge connection | `datadog-agent-resource-update-intake-connection` | Holds your Datadog API and application keys so events can be sent to Datadog |
| EventBridge API destination | `datadog-agent-resource-update-intake-destination` | Sends events to `https://api.<YOUR_DD_SITE>/api/unstable/instrumenter/events` (capped at 10 events per second) |
| EventBridge rule | `datadog-agent-resource-update-rule-ec2` | Notifies Datadog when a covered instance changes |
| IAM role | auto-named | Lets EventBridge send events to the `datadog-agent-resource-update-intake-destination` API destination |
| IAM role | `datadog-eventbridge-cross-region-role` | Lets other regions forward events to your primary region |

Datadog creates the following resources as needed, at install time:

| Resource | Name | Purpose |
|---|---|---|
| Systems Manager document | `datadog-ec2-instrumenter` | The install and uninstall script. One document per account. |
| Secrets Manager secret | `/datadog/ec2-instrumenter/<ACCOUNT_ID>/<INSTANCE_ID>` | Holds the Datadog API key so the instance can fetch it itself. Encrypted with the default AWS-managed key. |
| IAM role and instance profile | `datadog-ssm-<INSTANCE_ID>` and `datadog-ssm-profile-<INSTANCE_ID>` | Created only when the instance has no instance profile, under the IAM path `/datadog-ec2-instrumenter/` so they are identifiable. Receives the AWS-managed `AmazonSSMManagedInstanceCore` policy so Systems Manager can reach the instance. |
| Inline IAM policy | `datadog-ec2-instrumenter-secrets` | Added to the instance's role. Grants read access only to secrets under `/datadog/ec2-instrumenter/`. |
| EventBridge rules in your other regions | Same names as the primary-region resources | Forward change events to your primary region. |

Datadog does not create S3 buckets, event buses, log groups, or SSM parameters, and does not tag your instances.

## How Agent installation works

After you save an installation rule, Datadog resolves the query you defined into a fixed list of covered instances. Datadog then runs the following sequence against each one. For prerequisites, including supported platforms, see [Prerequisites][2] in the setup guide.

1. Datadog checks that each covered instance is running, on a supported platform, and reachable by AWS Systems Manager.
2. When an instance has no IAM instance profile, Datadog creates one so Systems Manager can reach it. When an instance already has one, Datadog adds the SSM policy and the scoped secret-read policy to the existing role.
3. Datadog checks whether an Agent is already present. When an Agent is present that Datadog did not install, Datadog stops and leaves the instance alone.
4. Datadog calls `ssm:SendCommand`, one instance at a time, running the `datadog-ec2-instrumenter` document.
5. On the instance, the document fetches the API key from Secrets Manager using the instance's own IAM role. It then runs Datadog's standard Agent installer (`install_script_agent7.sh` on Linux, or the standard MSI on Windows) with log collection and APM host instrumentation enabled. The command times out after 6 minutes.

Datadog does not reboot or restart your instances. The only service Datadog touches is the Datadog Agent itself, which is started on install and stopped on uninstall. Your applications and other services are untouched.

### Instances that Datadog excludes

Datadog automatically excludes:

- Instances that are not running
- EKS worker nodes
- ECS container instances
- Instances that already have a non-Datadog-installed Agent

## Security, auditing, and change control

### How Datadog gets access

Datadog uses the same cross-account IAM role as the AWS integration, authenticated with an external ID. Datadog receives short-lived, temporary credentials, and each type of work (reading EC2, managing IAM, sending commands) uses a separately scoped credential session rather than one broad session. Datadog does not store any long-lived AWS keys.

### Auditing Datadog's actions

Every action Datadog takes is a standard AWS API call, so all actions appear in AWS CloudTrail. Everything Datadog creates is identifiable by name: resources are prefixed with `datadog-`, secrets are stored under `/datadog/ec2-instrumenter/`, and IAM roles use the immutable path `/datadog-ec2-instrumenter/`. Because an IAM path cannot be edited after creation, the path cannot be silently changed. On-instance command results appear in the Systems Manager Run Command history.

### How the API key is handled

The API key is stored in your own Secrets Manager, encrypted at rest. Only the secret's Amazon Resource Name (ARN) is passed in the SSM command; the key itself never appears in command parameters or in CloudTrail. The instance reads the secret with its own IAM role, restricted to a single path. Datadog stores only a reference to the key internally, not the key itself.

### Who can change installations

- **In AWS**: Access is governed by your own IAM policies. Removing the cross-account permissions stops Datadog immediately.
- **In Datadog**: Viewing installation rules requires the **Hosts Read** permission. Creating, editing, or deleting rules requires the **Agent Install** permission. Rule changes are rate-limited.

### Guardrails

- Datadog never removes an Agent it did not install.
- Datadog tracks which instances it installed an Agent on, so it cleans up only its own work.
- When some regions cannot be listed, Datadog skips cleanup for that pass rather than risk uninstalling in bulk.

## Agent lifecycle and reconciliation

### Rule coverage is fixed at save time

A rule covers the list of instances it resolved to when you saved it, and Datadog does not instrument anything outside that list. Instances launched later are not picked up automatically. To cover them, update the rule, which re-resolves your query against your current fleet.

### How Datadog keeps covered instances in sync

Datadog continuously maintains the state you define on the covered instances:

- A full reconciliation runs hourly per AWS account. Reconciliation reinstalls the Agent if it goes missing, retries anything that failed, and cleans up instances that no longer exist.
- Already-installed instances are re-verified about once per day rather than every hour, to avoid unnecessary activity.
- Change events from the CloudFormation stack let Datadog react to covered instances within minutes, instead of waiting for the hourly pass.

### What happens when you edit a rule

Datadog re-resolves your query and compares it against the previous list. Instances no longer covered have the Agent uninstalled. Newly covered instances have the Agent installed. Deleting a rule uninstalls the Agent from everything the rule covered.

### Terminated or stopped instances

Datadog detects terminated instances on the next hourly pass and cleans up the IAM resources it created for them. Datadog leaves stopped instances alone until they return.

### When an install fails

Datadog retries with an increasing delay (1 hour, then 2 hours, up to once per day) and continues retrying. Missing-permission problems appear as an issue on the **AWS integration tile** and on the Fleet install page.

<div class="alert alert-warning">
When someone manually removes the Agent from a covered instance, the next reconciliation reinstalls it. The rule is the source of truth. To stop coverage, change the rule.
</div>

## Uninstall the Agent

Uninstalling removes the Datadog Agent, the `/etc/datadog-agent` and `/opt/datadog-agent` directories on Linux (or performs an MSI uninstall on Windows), and any IAM role or instance profile Datadog created for that instance. To uninstall, remove instances from a rule, edit the rule's query, or delete the rule.

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: https://docs.datadoghq.com/integrations/guide/aws-agent-installation/
[2]: https://docs.datadoghq.com/integrations/guide/aws-agent-installation/#prerequisites
140 changes: 140 additions & 0 deletions hugo/content/en/integrations/guide/aws-agent-installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
---
title: Install the Datadog Agent through the AWS Integration
description: "Install and manage the Datadog Agent on your Amazon EC2 instances directly from the AWS integration, without connecting to each host or running per-host scripts."
private: true # TODO(DOCS-14545): remove at v1 rollout to publish; also add the nextlink entry under "AWS guides" in hugo/content/en/integrations/guide/_index.md at that time
further_reading:
- link: "https://docs.datadoghq.com/integrations/guide/aws-agent-installation-technical-reference/"
tag: "Documentation"
text: "How Agent installation through the AWS integration works"
- link: "https://docs.datadoghq.com/integrations/amazon_web_services/"
tag: "Documentation"
text: "AWS Integration"
- link: "https://docs.datadoghq.com/integrations/guide/aws-manual-setup/"
tag: "Documentation"
text: "AWS Manual Setup Guide"
- link: "https://docs.datadoghq.com/agent/guide/why-should-i-install-the-agent-on-my-cloud-instances/"
tag: "Documentation"
text: "Why install the Datadog Agent on your cloud instances?"
- link: "https://docs.datadoghq.com/agent/fleet_automation/"
tag: "Documentation"
text: "Fleet Automation"
- link: "https://docs.datadoghq.com/agent/configuration/"
tag: "Documentation"
text: "Agent Configuration"
---

## Overview

The [AWS integration][1] collects metrics, events, and logs from Amazon CloudWatch without installing anything on your hosts. Installing the Datadog Agent adds telemetry from inside your AWS workloads that CloudWatch alone can't provide, including host-level metrics, distributed traces (APM), live processes, and detailed logs.

You can deploy the Datadog Agent to your Amazon EC2 instances directly from Datadog, without connecting to each host or running per-host scripts. Enable Agent installation while you set up the AWS integration, or at any time afterward.

Amazon EKS is not supported.

## Prerequisites

Before you begin, confirm the following:

- **CloudFormation access**: You can approve a CloudFormation stack in the target AWS account. Installation deploys a stack in your account, so you (or a teammate) need permission to review and create it. For the required permissions and why they're needed, see the [Required AWS permissions](#required-aws-permissions) section.
- **SSM Agent**: The [AWS Systems Manager (SSM) Agent][2] must already be present on the target instances. Datadog installs the Agent through SSM and can't install the SSM Agent for you, so instances built from custom AMIs without the SSM Agent are not eligible. Datadog flags these instances so you can address them.
- **Supported platforms**: Linux (x86_64 and arm64) and Windows (x86_64). macOS and Windows on arm64 are not supported.

## Required AWS permissions

{{% aws-agent-installation %}}

Datadog uses each of these permissions for a specific task:

| Permission | Why Datadog needs it |
|---|---|
| `ec2:DescribeInstances` | Find your instances and check which ones match your rule (state, tags, OS, architecture) |
| `ssm:DescribeInstanceInformation` | Confirm the SSM Agent is running before Datadog attempts anything |
| `ssm:GetDocument`, `ssm:CreateDocument`, `ssm:UpdateDocument`, `ssm:UpdateDocumentDefaultVersion` | Publish the install script in your account and keep it up to date |
| `ssm:SendCommand`, `ssm:ListCommandInvocations` | Run the install and confirm when it finishes |
| `secretsmanager:DescribeSecret`, `secretsmanager:CreateSecret` | Store the API key so it is never passed in a command |
| `iam:CreateRole`, `iam:CreateInstanceProfile`, `iam:AddRoleToInstanceProfile`, `iam:AttachRolePolicy`, `iam:PutRolePolicy`, `iam:PassRole`, `ec2:AssociateIamInstanceProfile`, and the matching `Get` and `List` reads | Give an instance the minimum access it needs in case it does not have an IAM role: reachable by Systems Manager, and able to read its own API key secret |
| `iam:Detach*`, `iam:Delete*`, `iam:RemoveRoleFromInstanceProfile`, `ec2:Disassociate*`, `ec2:DescribeIamInstanceProfileAssociations` | Cleanly undo the resources above when you uninstall |
| `ecs:ListClusters`, `ecs:ListContainerInstances` | Recognize Amazon Elastic Container Service (ECS) container instances so Datadog skips them (they are handled at the cluster level) |
| `events:PutRule`, `events:PutTargets`, `events:RemoveTargets`, `events:DeleteRule` | Set up the change notifications that let Datadog react to instance changes |

`iam:CreateRole` and `iam:PassRole` are the most sensitive grants. `iam:CreateRole` is restricted to role names matching `datadog-ec2-instrumenter/datadog-ssm-*` in your account, and `iam:PassRole` is further restricted to the Amazon EC2 service.

## How it works

Agent installation is based on an **installation rule**: an AWS account paired with a query that describes which EC2 instances to cover. Saving a rule resolves the query into a fixed list of instances. Datadog then installs the Agent on each one, inside your own account:

1. You select the EC2 instances to cover, or opt in to all eligible instances.
1. Datadog resolves your selection into a list of covered instances and records it.
1. Datadog installs the Agent on each covered instance through AWS Systems Manager, adding any missing IAM configuration automatically.
1. Datadog keeps the covered instances instrumented. Instances launched later aren't added until you update the rule.

You approve one CloudFormation stack, one time, during initial setup. After that, installations run automatically from Datadog, with no new CloudFormation template to launch for each installation.

For the full technical and security details, including the AWS resources Datadog creates, the installation mechanism, and the reconciliation model, see [How Agent installation through the AWS integration works][6].

{{< img src="integrations/amazon_web_services/aws-agent-installation-how-it-works.png" alt="Flowchart of the AWS Agent installation process, showing which steps happen in Datadog and which run inside your AWS account." style="width:70%;" >}}

## Install the Agent

You can start Agent installation from two entry points, depending on how much control you want over which instances are instrumented:

- **AWS integration setup (install on all eligible instances)**: When you [set up the AWS integration][5], enable the Agent installation toggle on the [AWS integration page][7], shown alongside log and resource collection. The Agent installs on all eligible EC2 instances.
- **Fleet Automation (install on specific instances)**: Open the [AWS Install Agents page][8] at any time to select the specific EC2 instances you want.

<!-- TODO(DOCS-14545): per AWS team, surfacing the Agent install flow in the main AWS setup flow for non-first-time users is still rolling out; confirm it's live before publish. -->

The Agent installation toggle appears during setup:

{{< img src="integrations/amazon_web_services/aws-agent-installation-setup-toggle.png" alt="The Install the Datadog Agent step in AWS setup, with the install toggle enabled and the Hosts (EC2) workload toggle turned on." style="width:80%;" >}}

To install from the AWS Install Agents page:

1. Opt in to all eligible instances, or select specific EC2 instances from the resource list.
1. Review the generated CloudFormation stack, then continue to AWS and create it. Datadog prompts you for this only once.
1. Return to Datadog. The installation proceeds automatically, and Datadog reports progress as Agents come online.

<!-- TODO(DOCS-14545): add resource-selection / Manage Agents page screenshot (AWS Install Agents page) — setup-toggle screenshot added. -->

## Verify the installation

After the installation completes:

- The newly installed Agents appear in the [Infrastructure List][3] and on the host map.
- Fleet Automation lists the same Agents in the Fleet View.

<!-- TODO(DOCS-14545): add expected time-to-data once confirmed. -->

## Manage installed Agents

Use the [AWS Install Agents page][8] in Fleet Automation to manage the Agents you've installed through the AWS integration.

From this page, you can:

- View the installed Agents and their status.
- Install the Agent on new instances in your AWS environment.
- Uninstall Agents from instances you no longer want to monitor.

To stop coverage, update the rule. If you manually remove the Agent from a covered instance, Datadog reinstalls it on the next reconciliation. Manage Agent configuration and version upgrades through [Fleet Automation][4].

## Troubleshooting

### The SSM Agent is not present on an EC2 instance

Agent installation on EC2 relies on the AWS Systems Manager (SSM) Agent, which Datadog can't install for you. Datadog flags any instance that lacks it as ineligible, including those built from custom AMIs. Install the SSM Agent on the instance, then retry. See [Working with SSM Agent][2] in the AWS documentation.

### A permission or IAM error occurs

If installation can't complete because of missing permissions, Datadog shows a notification linking to the CloudFormation resource that needs the new permission. Update your existing stack to grant the [required permissions](#required-aws-permissions). You don't need to create a new stack.

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: https://docs.datadoghq.com/integrations/amazon_web_services/
[2]: https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html
[3]: https://app.datadoghq.com/infrastructure
[4]: https://docs.datadoghq.com/agent/fleet_automation/
[5]: https://docs.datadoghq.com/getting_started/integrations/aws/
[6]: https://docs.datadoghq.com/integrations/guide/aws-agent-installation-technical-reference/
[7]: https://app.datadoghq.com/integrations/amazon-web-services
[8]: https://app.datadoghq.com/fleet/install-agent/latest?platform=aws
10 changes: 10 additions & 0 deletions hugo/layouts/shortcodes/aws-agent-installation.en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
Agent installation requires permissions beyond the base [AWS integration IAM policy](https://docs.datadoghq.com/integrations/amazon_web_services/#aws-iam-permissions). These permissions let Datadog:

- Store your Datadog API key (AWS Secrets Manager).
- Create and attach the IAM role used to install the Agent (IAM).
- Install the Agent on your EC2 instances (AWS Systems Manager).
- Maintain the Agent's presence (Amazon EventBridge).

<div class="alert alert-info">All write actions run inside your own AWS account through the role created by the CloudFormation stack. Datadog does not hold persistent write credentials to your account.</div>

The CloudFormation stack you approve during setup adds these permissions to your AWS integration IAM role. There is no policy to apply manually.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading