Skip to content

Add opt-in interval CSV measurement export - #1270

Open
sankalpsthakur wants to merge 4 commits into
mlco2:masterfrom
sankalpsthakur:fix/467-interval-csv-export
Open

Add opt-in interval CSV measurement export#1270
sankalpsthakur wants to merge 4 commits into
mlco2:masterfrom
sankalpsthakur:fix/467-interval-csv-export

Conversation

@sankalpsthakur

@sankalpsthakur sankalpsthakur commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #467 by writing CSV measurement rows on the same live cadence as API/Prometheus (api_call_interval * measure_power_secs).

  • Opt-in csv_run_name (filename, or "auto" / ""emissions_<run_id>.csv). Default emissions.csv stays final-only unless csv_run_name targets the same file.
  • FileOutput.live_out behind enable_live_out — reuses the existing measurement loop instead of ad-hoc flush threads.

Why this matters

Long-running jobs previously had no structured way to inspect intermediate power/emissions without enabling API or Prometheus export. Interval CSV gives operators a file-based audit trail (notebooks, log pipelines, offline analysis) with predictable row cadence and without changing the default single-row emissions.csv contract.

Related (not in scope here): #559 asks whether power columns belong in CSV — this PR follows the existing measurement schema; #448 covers periodic export for other output modes beyond CSV.

Usage

from codecarbon import EmissionsTracker

tracker = EmissionsTracker(
    project_name="Test",
    tracking_mode="machine",
    csv_run_name="emissions_test54.csv",
)
tracker.start()
# ... workload ...
tracker.stop()

Test plan

  • tests/output_methods/test_file.py live_out coverage
  • tests/test_emissions_tracker.py named / auto / empty / same-as-output_file / default final-only / without CSV method
  • Docs: docs/reference/output.md interval CSV note

AI/LLM disclosure

  • AI coding tools (including Grok and/or Codex agent-assisted editing) were used to help draft or modify code and this PR description.
  • I reviewed the complete change, understand the reasoning, and ran the reported local tests before submitting.
  • This submission is original work of authorship under the project CLA / contributor terms; AI output was not pasted unreviewed.

@sankalpsthakur
sankalpsthakur requested a review from a team as a code owner July 22, 2026 11:56
@sankalpsthakur
sankalpsthakur force-pushed the fix/467-interval-csv-export branch 2 times, most recently from 1c57416 to ba687ca Compare July 22, 2026 11:59
@sankalpsthakur

sankalpsthakur commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

@benoit-cty @mlco2/core-maintainers — implements the #467 approach: opt-in csv_run_name writes measurement rows on the API/Prometheus live cadence. Default emissions.csv stays final-only.

CI is waiting on first-time contributor approval — could a maintainer run the workflows?

@sankalpsthakur
sankalpsthakur force-pushed the fix/467-interval-csv-export branch 5 times, most recently from 8658551 to 8b243de Compare July 28, 2026 10:09
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.73%. Comparing base (336da33) to head (8b243de).
⚠️ Report is 43 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1270      +/-   ##
==========================================
+ Coverage   89.70%   89.73%   +0.03%     
==========================================
  Files          48       48              
  Lines        4778     4804      +26     
==========================================
+ Hits         4286     4311      +25     
- Misses        492      493       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Write measurement rows on the API/Prometheus live cadence when
csv_run_name is set, avoiding ad-hoc flush threads (mlco2#467).
Cover empty-string auto naming and default final-only CSV behavior,
and fix spacing so pre-commit style checks stay clean.
Cover the opt-in live CSV path in output docs and add a regression test
for interval export without OutputMethod.CSV.
@sankalpsthakur
sankalpsthakur force-pushed the fix/467-interval-csv-export branch from 8b243de to cd1f681 Compare September 4, 2026 03:09
@github-actions github-actions Bot added the size/L label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Capture measurements in CSV based on time intervals

1 participant