Skip to content

Add AKS Container Insights tests - #762

Open
Paamicky wants to merge 6 commits into
mainfrom
AKS-ContainerInsights
Open

Paamicky wants to merge 6 commits into
mainfrom
AKS-ContainerInsights

Conversation

@Paamicky

@Paamicky Paamicky commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Description of the issue

The CloudWatch Agent now translates OTEL Container Insights configs itself, including the AKS path. EKS CI exercises this, but nothing validates that the agent translates and delivers Container Insights on AKS. (aks/azure resource detection, apiserver TLS server_name, azure.vm.* suppression) was untested end-to-end.

Description of changes

Adds an AKS Container Insights integration test that deploys the agent from a mounted JSON config (agent does the translation, not a pre-rendered otelConfig) and validates delivery to CloudWatch:

  • New test test/azure/aks/containerinsights asserts node metrics, cluster metrics (apiserver + KSM), keda/karpenter solution metrics, and node applicationlogs, scoped to the run's cluster and cloud.platform=azure_aks.
  • Node config (logs enabled) + cluster-scraper config, plus keda/karpenter stub emitters.
  • Reuses the existing terraform/azure/aks stack via a new test_mode toggle. containerinsights mode deploys the node DaemonSet, cluster-scraper Deployment, kube-state-metrics, node-exporter, and stubs for keda and karpenter scrapping

Tests

  • go vet -tags integration ./test/azure/aks/containerinsights/ passes; configs are valid JSON.
  • End-to-end test for AKS Container Insights tests: AKS-container-insights
  • AKS metics verified in CloudWatch:
    • Node metrics: cadvisor, kubeletstats, node-exporter
    • Cluster metrics: kube-state-metrics, apiserver, control plane metrics (apiserver)
    • Node logs: application and host container logs
    • Solution metrics: Keda and Karpenter
Screenshot 2026-09-16 at 11 58 06 Screenshot 2026-09-16 at 12 01 01

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Paamicky
Paamicky requested a review from a team as a code owner September 14, 2026 20:26
@Paamicky
Paamicky force-pushed the AKS-ContainerInsights branch from 6c38eea to 2dfaa24 Compare September 16, 2026 01:58
@olowosulu

Copy link
Copy Markdown
Contributor

One blocker and three majors:

  • [BLOCKER] test/azure/aks/containerinsights/aks_containerinsights_test.go:68 has a stray .:
    func TestAKSContainerInsights(t *testing.T) {.
    The package fails gofmt and go vet -tags integration and never builds. The compile CI job misses it because it doesn't set -tags integration. Delete the ..
  • [MAJOR] PR body says go vet -tags integration ... passes — it exits 1 until the above is fixed; please re-run and correct.
  • [MAJOR] test_mode / test_dir are independent knobs. A caller that sets one but not the other silently runs the wrong suite against the wrong topology. agent#2284 sets both, but please couple them (a locals map from test_modetest_dir, or a validation block) so other callers can't misfire.
  • [MAJOR] Duplicates test/e2e/containerinsights (metric lists plus ci_node.json / ci_cluster.json / keda_karpenter.yaml). Two copies will drift — please share the slices/resource files.

@Paamicky
Paamicky force-pushed the AKS-ContainerInsights branch from 98482ee to c0908a8 Compare September 17, 2026 16:34
@Paamicky

Copy link
Copy Markdown
Contributor Author

One blocker and three majors:

  • [BLOCKER] test/azure/aks/containerinsights/aks_containerinsights_test.go:68 has a stray .:
    func TestAKSContainerInsights(t *testing.T) {.
    The package fails gofmt and go vet -tags integration and never builds. The compile CI job misses it because it doesn't set -tags integration. Delete the ..
  • [MAJOR] PR body says go vet -tags integration ... passes — it exits 1 until the above is fixed; please re-run and correct.
  • [MAJOR] test_mode / test_dir are independent knobs. A caller that sets one but not the other silently runs the wrong suite against the wrong topology. agent#2284 sets both, but please couple them (a locals map from test_modetest_dir, or a validation block) so other callers can't misfire.
  • [MAJOR] Duplicates test/e2e/containerinsights (metric lists plus ci_node.json / ci_cluster.json / keda_karpenter.yaml). Two copies will drift — please share the slices/resource files.
  1. Fixed the stray .
  2. Passes now due to the above fix
  3. Coupled them so callers don't misfire
  4. different test and K8s env so kept the same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants