The current release gate can tell whether committed artifacts match regenerated output, but the failure mode is mostly binary: either the artifacts match or they do not. A future standalone technical track could add a structured artifact-contract diff report so a maintainer can quickly see what changed before deciding whether to accept regenerated outputs.
This is not urgent for the current v1.x contract, but it is a clean independent project because it touches comparison logic, schemas, fixtures, and tests without adding another demo.
Potential scope:
- Compare two artifact trees, for example committed artifacts vs a regenerated output directory.
- Report missing files, extra files, changed files, schema-version changes, and run-manifest digest changes.
- For JSON/JSONL artifacts, produce lightweight structural summaries such as record counts, top-level keys, and schema IDs/versions where available.
- Emit both a human-readable summary and a machine-readable JSON report.
- Keep text/Markdown report diffs shallow; the goal is contract triage, not a full prose diff engine.
Possible CLI shape:
python scripts/artifact_contract_diff.py --expected . --actual /tmp/telemetry-lab-regenerated --json-out artifact_diff.json
Acceptance criteria for a future PR:
- Add fixture-based tests for unchanged artifacts, missing/extra files, changed JSON records, changed schema versions, and changed run-manifest digests.
- Keep comparison deterministic across platforms, including line-ending normalization for text artifacts.
- Do not replace
python scripts/regenerate_artifacts.py --check; this should complement the existing gate by explaining mismatches.
- Document the output fields in a short technical note only if the JSON report becomes reviewer-facing.
Boundaries:
- No new demo.
- No live telemetry or account integration.
- No dashboard, alert routing, case management, autonomous response, or incident verdict.
- No production SIEM claim.
The current release gate can tell whether committed artifacts match regenerated output, but the failure mode is mostly binary: either the artifacts match or they do not. A future standalone technical track could add a structured artifact-contract diff report so a maintainer can quickly see what changed before deciding whether to accept regenerated outputs.
This is not urgent for the current v1.x contract, but it is a clean independent project because it touches comparison logic, schemas, fixtures, and tests without adding another demo.
Potential scope:
Possible CLI shape:
python scripts/artifact_contract_diff.py --expected . --actual /tmp/telemetry-lab-regenerated --json-out artifact_diff.jsonAcceptance criteria for a future PR:
python scripts/regenerate_artifacts.py --check; this should complement the existing gate by explaining mismatches.Boundaries: