Finalize noisy sweep and continual comparison results with a figure renderer

docs/comparisons/bridge-balloons-integrity-results.json

@@ -0,0 +1,135 @@
+{
+  "generated_by": "/home/ycliang/predicators/logs/bridge_balloons_integrity_20260912/report.py",
+  "updated_at": "2026-09-13T09:07:21.054343+00:00",
+  "source_commit": "cad1000f92c5e5e94714ae613426b89761823ee0",
+  "rows": [
+    {
+      "domain": "Bridge",
+      "approach": "agent_continual",
+      "arm": "MB + uncertainty",
+      "seed": 0,
+      "job": "22646173_0",
+      "job_state": "COMPLETED",
+      "status": "Finished",
+      "scorecard": "/home/ycliang/predicators/logs/agent_continual/bridge-agent_continual_span_transfer_r1/seed0/run_20260912_165804/scorecard.json",
+      "finished": true,
+      "success": true,
+      "wins": 2,
+      "levels": 2,
+      "steps": 2939,
+      "resets": 0,
+      "end_reason": "all_levels_won",
+      "quota_message": null
+    },
+    {
+      "domain": "Bridge",
+      "approach": "agent_continual_model_free",
+      "arm": "MF direct coding agent",
+      "seed": 0,
+      "job": "22646174_0",
+      "job_state": "COMPLETED",
+      "status": "Finished",
+      "scorecard": "/home/ycliang/predicators/logs/agent_continual_model_free/bridge-agent_continual_model_free_span_transfer_r1/seed0/run_20260912_165804/scorecard.json",
+      "finished": true,
+      "success": true,
+      "wins": 2,
+      "levels": 2,
+      "steps": 2634,
+      "resets": 0,
+      "end_reason": "all_levels_won",
+      "quota_message": null
+    },
+    {
+      "domain": "Balloons",
+      "approach": "agent_continual_model_free",
+      "arm": "MF direct coding agent",
+      "seed": 0,
+      "job": "22649185_0",
+      "job_state": "COMPLETED",
+      "status": "Finished",
+      "scorecard": "/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed0/run_20260912_180112/scorecard.json",
+      "finished": true,
+      "success": true,
+      "wins": 3,
+      "levels": 3,
+      "steps": 771,
+      "resets": 1,
+      "end_reason": "all_levels_won",
+      "quota_message": null
+    },
+    {
+      "domain": "Balloons",
+      "approach": "agent_continual_model_free",
+      "arm": "MF direct coding agent",
+      "seed": 1,
+      "job": "22649185_1",
+      "job_state": "COMPLETED",
+      "status": "Finished",
+      "scorecard": "/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed1/run_20260912_180112/scorecard.json",
+      "finished": true,
+      "success": true,
+      "wins": 3,
+      "levels": 3,
+      "steps": 554,
+      "resets": 1,
+      "end_reason": "all_levels_won",
+      "quota_message": null
+    },
+    {
+      "domain": "Balloons",
+      "approach": "agent_continual_model_free",
+      "arm": "MF direct coding agent",
+      "seed": 2,
+      "job": "22649185_2",
+      "job_state": "COMPLETED",
+      "status": "Finished",
+      "scorecard": "/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed2/run_20260912_180113/scorecard.json",
+      "finished": true,
+      "success": true,
+      "wins": 3,
+      "levels": 3,
+      "steps": 829,
+      "resets": 2,
+      "end_reason": "all_levels_won",
+      "quota_message": null
+    }
+  ],
+  "aggregate": [
+    {
+      "domain": "Bridge",
+      "arm": "MB + uncertainty",
+      "finished": 1,
+      "expected": 1,
+      "successes": 1,
+      "solve_rate": 1.0,
+      "level_solve_rate": 1.0,
+      "steps_mean_success": 2939,
+      "steps_n": 1,
+      "resets_mean_finished": 0
+    },
+    {
+      "domain": "Bridge",
+      "arm": "MF direct coding agent",
+      "finished": 1,
+      "expected": 1,
+      "successes": 1,
+      "solve_rate": 1.0,
+      "level_solve_rate": 1.0,
+      "steps_mean_success": 2634,
+      "steps_n": 1,
+      "resets_mean_finished": 0
+    },
+    {
+      "domain": "Balloons",
+      "arm": "MF direct coding agent",
+      "finished": 3,
+      "expected": 3,
+      "successes": 3,
+      "solve_rate": 1.0,
+      "level_solve_rate": 1.0,
+      "steps_mean_success": 718,
+      "steps_n": 3,
+      "resets_mean_finished": 1.3333333333333333
+    }
+  ]
+}

docs/comparisons/bridge-balloons-integrity-results.md

@@ -0,0 +1,25 @@
+<!-- Generated by logs/bridge_balloons_integrity_20260912/report.py; do not edit. -->
+# Bridge transfer and sustained-hover Balloons pilots
+
+Updated 2026-09-13T09:07:21.054343+00:00.
+
+Bridge: one seed per MB/MF arm, three-block train span and four-block test span. Balloons: three MF seeds, original non-hatch task distribution, 25-step sustained hovering. Historical cohorts and other comparisons remain separate.
+Whole-run success means winning every training and test level.
+Solve rates and mean resets use finished agent seeds only.
+Mean steps uses only whole-run successes, with the qualifying count shown.
+Unfinished, setup, and infrastructure outcomes are excluded from averages.
+These tables are refreshed from scorecards and Slurm accounting.
+
+| Domain | Arm | Finished | Whole-run solve | Level solve | Mean successful steps (n) | Mean resets |
+|---|---|---:|---:|---:|---:|---:|
+| Bridge | MB + uncertainty | 1/1 | 100.0% (1/1) | 100.0% | 2,939 (n=1) | 0 |
+| Bridge | MF direct coding agent | 1/1 | 100.0% (1/1) | 100.0% | 2,634 (n=1) | 0 |
+| Balloons | MF direct coding agent | 3/3 | 100.0% (3/3) | 100.0% | 718 (n=3) | 1.3 |
+
+| Domain | Arm | Seed | Status | Wins | Steps | Resets | Job | Scorecard |
+|---|---|---:|---|---:|---:|---:|---|---|
+| Bridge | MB + uncertainty | 0 | Finished | 2/2 | 2,939 | 0 | 22646173_0 | [scorecard](/home/ycliang/predicators/logs/agent_continual/bridge-agent_continual_span_transfer_r1/seed0/run_20260912_165804/scorecard.json) |
+| Bridge | MF direct coding agent | 0 | Finished | 2/2 | 2,634 | 0 | 22646174_0 | [scorecard](/home/ycliang/predicators/logs/agent_continual_model_free/bridge-agent_continual_model_free_span_transfer_r1/seed0/run_20260912_165804/scorecard.json) |
+| Balloons | MF direct coding agent | 0 | Finished | 3/3 | 771 | 1 | 22649185_0 | [scorecard](/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed0/run_20260912_180112/scorecard.json) |
+| Balloons | MF direct coding agent | 1 | Finished | 3/3 | 554 | 1 | 22649185_1 | [scorecard](/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed1/run_20260912_180112/scorecard.json) |
+| Balloons | MF direct coding agent | 2 | Finished | 3/3 | 829 | 2 | 22649185_2 | [scorecard](/home/ycliang/predicators/logs/agent_continual_model_free/balloons-agent_continual_model_free_dwell_r1/seed2/run_20260912_180113/scorecard.json) |