Skip to content

fix: deduplicate mirrored Linux RAPL counters - #1276

Merged
benoit-cty merged 6 commits into
mlco2:masterfrom
lntutor:fix-linux-rapl-mirrored-counters-1274
Sep 3, 2026
Merged

fix: deduplicate mirrored Linux RAPL counters#1276
benoit-cty merged 6 commits into
mlco2:masterfrom
lntutor:fix-linux-rapl-mirrored-counters-1274

Conversation

@lntutor

@lntutor lntutor commented Jul 26, 2026

Copy link
Copy Markdown

Description

Drop mirrored nonzero Linux RAPL package counters at monitoring startup while preserving distinct package counters, zero counters, and DRAM domains. The documentation explains the multi-die mirrored-counter behavior.

Related Issue

Fixes #1274

Motivation and Context

Some Linux multi-die systems expose mirrored package-energy counters. Counting each mirror separately overstates CPU energy consumption, while indiscriminate deduplication could incorrectly remove distinct or zero-valued counters.

How Has This Been Tested?

  • uv run pytest tests/test_rapl_mmio_scanning.py -q
  • uv run pytest tests/test_cpu.py tests/test_rapl_mmio_scanning.py tests/test_rapl_parameters.py -q
  • uv run ruff check codecarbon/core/cpu.py tests/test_rapl_mmio_scanning.py
  • uv run ruff format --check codecarbon/core/cpu.py tests/test_rapl_mmio_scanning.py
  • git diff --check

Screenshots (if appropriate):

Not applicable.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

  • 🟥 AI-vibecoded: You cannot explain the logic. Car analogy : the car drive by itself, you are outside it and just tell it where to go.
  • 🟠 AI-generated: Car analogy : the car drive by itself, you are inside and give instructions.
  • ⭐ AI-assisted. Car analogy : you drive the car, AI help you find your way.
  • ♻️ No AI used. Car analogy : you drive the car.

OpenAI Codex assisted with implementation, tests, and documentation preparation. I personally reviewed and tested every submitted line, understand the implementation, and can explain and maintain it.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the docs/how-to/contributing.md document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@lntutor
lntutor requested a review from a team as a code owner July 26, 2026 12:29
@SaboniAmine

Copy link
Copy Markdown
Member

Thanks for your contribution, that seems great!
Letting @benoit-cty validate the dedup mechanism, as he worked on the windows side.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.68%. Comparing base (d1fb78a) to head (77db9bb).

Files with missing lines Patch % Lines
codecarbon/core/cpu.py 96.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1276      +/-   ##
==========================================
+ Coverage   91.58%   91.68%   +0.09%     
==========================================
  Files          49       49              
  Lines        5100     5136      +36     
==========================================
+ Hits         4671     4709      +38     
+ Misses        429      427       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@benoit-cty

Copy link
Copy Markdown
Contributor

Hello, you have created more than 20 repository today, I suspect you are an OpenClaw agent. I'm not against AI contribution, but a human must be accountable and willing to contribute to our project.

Please respect the PR submission template.

We will merge EMI PR in a few days then look back at this PR.

@lntutor

lntutor commented Jul 31, 2026

Copy link
Copy Markdown
Author

Updated the PR description to the complete repository template, including the required 🟠 AI-generated disclosure. I personally reviewed and tested every submitted line, understand the implementation, and can explain and support it through review and after merge.

benoit-cty pushed a commit to lntutor/codecarbon that referenced this pull request Sep 2, 2026
The Windows EMI backend (mlco2#1263) and the Linux RAPL backend (mlco2#1276) both
need to detect channels that mirror the same hardware energy counter.
Move the tolerance constant and the matching/detection helpers from
windows_emi.py to rapl.py, next to RAPLFile, so both backends share one
definition instead of duplicating the 1e-6 magic number. No behavior
change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added size/L and removed size/M labels Sep 2, 2026
Loi Nguyen and others added 6 commits September 2, 2026 22:10
The Windows EMI backend (mlco2#1263) and the Linux RAPL backend (mlco2#1276) both
need to detect channels that mirror the same hardware energy counter.
Move the tolerance constant and the matching/detection helpers from
windows_emi.py to rapl.py, next to RAPLFile, so both backends share one
definition instead of duplicating the 1e-6 magic number. No behavior
change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
start() used to permanently remove suspected mirrored counters from
self._rapl_files based on a single instantaneous comparison. A file
dropped by mistake (two independent packages coincidentally holding the
same counter value) was gone for the life of the tracker, and each new
start() of a task re-ran the destructive detection on whatever was left.

Flag suspected mirrors in a candidate map instead, exactly like the EMI
backend does: the files stay monitored, candidates are only left out of
the reported details, and every start() re-evaluates the detection from
the full set of files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ropping

Port the second stage of the EMI mirrored-channel detection (mlco2#1263) that
the Linux side was missing: a counter flagged at start() is only dropped
for good once it has also accumulated the very same energy as its
reference over a measurement interval. A candidate whose delta diverges
is a real meter and is restored into the measurement; a candidate that
has not accumulated anything stays pending until an interval is
conclusive. This prevents a genuine second package from being silently
discarded for the life of the run because it coincidentally held the
same counter value at start.

Unlike EMI, which snapshots a device's channels atomically, the powercap
sysfs files are read one after the other, so two views of the same
counter differ by the energy accrued between the reads. A relative
tolerance alone misses true mirrors when the counters are small (right
after a wrap-around, which happens every few minutes on packages with a
small max_energy_range_uj). Matching therefore also allows an absolute
1 J difference, both when flagging candidates and when comparing deltas;
a wrongly flagged independent package is restored by the confirmation
stage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@benoit-cty
benoit-cty force-pushed the fix-linux-rapl-mirrored-counters-1274 branch from 503f7b8 to 77db9bb Compare September 2, 2026 20:10
@benoit-cty

Copy link
Copy Markdown
Contributor

Hello, sorry for the delay.
I tested on my Threadripper on Ubuntu 26.04 and there is no more duplication in RAPL.
But it is a good security for older kernel, thanks !
I made some changes to the PR.

@benoit-cty
benoit-cty merged commit c43563f into mlco2:master Sep 3, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux RAPL over-counts CPU power on multi-die CPUs (port the Windows EMI fix)

3 participants