Skip to content

Consolidate AWS billing alerting into the organization account #175

Description

@ale210

Overview

We need to consolidate all AWS billing alerting into the organization (payer) account 504695638931 and retire the alerts that fire without carrying information, so that a real spend problem is noticed instead of being lost in routine noise. Research and findings are in hackforla/incubator#118.

Action Items

  • Confirm the amounts below with Bonnie before creating anything. They assume the incubator RDS Extended Support charge (~$149/month) is no longer being billed; if it still is, they need revisiting.
  • Delete the CloudWatch alarm Billing Anomaly Detection (us-east-1, account 504695638931), along with its underlying anomaly detector on AWS/Billing / EstimatedCharges. It fired 10 times in the 29 days to 2026-08-09 and cannot be usefully retuned: it applies an anomaly-detection band to a cumulative month-to-date counter that resets on the 1st, which is a metric shape those bands are not built for.
  • Create a consolidated AWS Budget in account 504695638931 at the agreed limit, with actual and forecast notifications, publishing to the existing SNS topic Monthly_Org_Billing_Alarm. Budgets handle the monthly reset natively, which the EstimatedCharges alarm does not.
  • Create per-member-account budgets in the org account using a LinkedAccount cost filter: 035866691871 (incubator) at $350/month, 470363915259 (foodoasis) at $120/month. This keeps per-project visibility without putting anything in a member account. Note that at $350 the 85% notification will fire in months with a domain renewal — roughly once or twice a year, and informative rather than noise.
  • Delete the CloudWatch alarm Monthly Billing Alarm once the budgets above are live and have been confirmed to notify. Do this last so there is no window without any billing alert. It is a static >= $210 threshold on a cumulative metric, so it trips every month by design and then sits in ALARM for three weeks.
  • Delete the AWS Budget Monthly Cost Budget from the incubator account 035866691871. Billing alerting belongs solely in the org account. Note its only subscriber is an individual's personal Gmail address rather than an org address — removing it also removes that dependency on one person's inbox.
  • Confirm hackforla@hackforla.org is still the right destination for billing alerts, and that the SNS subscription is confirmed and delivering.
  • After the changes are in place, verify: aws cloudwatch describe-alarms --region us-east-1 returns no billing alarms in the org account; aws budgets describe-budgets --account-id 504695638931 returns the new budgets; and aws budgets describe-budgets --account-id 035866691871 returns none. Paste the output into this issue.
  • Watch for one full billing cycle and report back here how many alert emails were sent. The point of the work is fewer, meaningful alerts — that is only observable after a month.

Note: there is no Terraform for the org account, so this is delivered through the console or CLI rather than as a PR. That means the usual "verify after the PR merges" step does not apply — verify directly, as described above.

Resources/Instructions

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    New Issue Review

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions