Skip to content

fix(tools): shell job retention, output deltas, and child process lifetimes - #6759

Merged
Hmbown merged 16 commits into
mainfrom
fix/bh2-tools-shell-process-lifecycle
Sep 30, 2026
Merged

Hmbown merged 16 commits into
mainfrom
fix/bh2-tools-shell-process-lifecycle

Conversation

@Hmbown

@Hmbown Hmbown commented Sep 29, 2026 •

Copy link
Copy Markdown
Owner

No-Issue: verified bug-hunt findings

Long-running shell jobs now remain available after they finish, output polling returns new bytes without repeating retained text, non-interactive commands receive EOF, and cancellation/timeout owns child-process cleanup. Original commits 5cf67ed, d314fa4 and 48267b5 and their authorship are retained; main was merged without rewriting the PR.

  • Measure finished-job retention and eviction order from finish time, including undelivered completions. Retain a useful terminal output tail and count omitted bytes correctly. Bounded shell deltas identify skipped bytes and their spill file.
  • Close stdin for foreground pipe commands and synchronous commands with no input. TTY/background interaction keeps its existing contract; a foreground command moved to jobs keeps its closed stdin.
  • Reuse one contained process owner for JavaScript, Python, plugin tools, gate commands, run_tests, and the equivalent Runtime Git and git_history paths. Plugin input is fed while output drains in the same future. Stop/drop kills owned descendants; timed stops retain partial output. Clean exits can leave deliberately detached, redirected daemons running. Unix signal-exit cleanup includes active groups.
  • Require process-tree attachment to succeed. Attachment errors are now returned immediately while direct-child kill-on-drop remains active. A deterministic failure test launches a real child, injects PermissionDenied, and checks both the original error and child cleanup.
  • Preserve both sets of tests in the Git merge conflict: process cleanup and main's literal-path/effective-remote validation. Import the shared configuration/plugin fixture corrections and limit concurrent extension-host fixtures in nextest; production deadlines and retries are unchanged.

Recovered-branch local evidence:

  • Governed focused Rust tests: 32 passed / 0 failed. Includes process-tree lifetime/output behavior, actual Python/plugin/Git cancellation, GitHistory, real in-flight run_tests timeout, and the merged Git validation cases.
  • The attachment negative control compiled and failed at runtime, 0 passed / 1 failed: the old swallowed error ran until the enclosing timeout. Child cleanup passed before that assertion. Fixed source was restored byte-for-byte; the final governed check passed both the attachment regression and imported typed-host fixture (2/0), plus all four changed configuration fixtures (4/0).
  • Full npm test: 740 passed / 0 failed (68 wrapper + 16 SDK + 54 extension-host + 602 web). check:web passed separately. Scoped formatting and diff checks passed.
  • Earlier author receipts remain tied to their original heads: 48267b5 reported 108 focused passes and four failing call-site reversions; d314fa4 reported 233 touched-module passes plus 1501 broader filtered passes. These were not repeated or presented as fresh recovery results.

Limits: process groups and Windows jobs are lifetime controls, not a security sandbox. Tokio children are assigned after spawn on Windows, so descendants may start before assignment. Parent SIGKILL/crash is not covered. The synchronous hardened Git precondition reader remains uncancellable. run_tests has its existing 30-minute ceiling; the existing pre-cancel test is not in-flight token-cancellation proof. Concurrent waiters still share one job cursor. Local macOS results do not replace hosted Windows/current-head CI or release qualification.

PTY fixture follow-up:

The Ubuntu run at 507686df6c450824ea639037509f21c7a87ec7b2 finished with 17,558 passed / 1 failed. The automation editor fixture waited for the substring deleted, which was already present in the open review text Nothing was deleted. It could therefore assert before the queued confirmation completed. The fixture now waits within its existing deadline for file absence, the deletion receipt, and closure of the armed review. All confirmation, cancellation, and final file-absence assertions remain. Production code is unchanged by this follow-up.

The corrected real-terminal test passed 1/0 in 78.44 seconds, covering all five sizes: 40×12, 60×16, 80×24, 100×32, and 140×40. This run used combined candidate 9791f0ffc139b83da228b700ceca8e5ba9bd7cac, not a fresh build of this PR branch. Its TUI binary SHA-256 is 1bd9f79e76351dc3b9fd221d8d012c7b29658b113cf7c6ac25131ce2b502a4a8. The test, automation manager, routing, panel, views/editor, automation tool, harness, and English automation strings were compared and match exactly. This is scoped PTY evidence; updated-head whole-branch CI remains required. The earlier npm 740/0 and web pass receipts above remain tied to the prior recovery source; no additional npm or Cargo run was performed for this test-only follow-up.

Native macOS child-reaper follow-up (Codex, 2026-09-30):

Original CI36674931118 attempt1 macOS finished 17565 passed / 1 failed / 27 skipped. Only in_flight_contained_run_is_listed_for_signal_exit failed: its synchronous 5s cleanup probe starved the same current-thread Tokio runtime that reaps the dropped child. The probe now uses the existing neighboring spawn_blocking convention. Production ownership, cancellation and deadlines are unchanged.

Frozen source63dd0340d8fb5f89f2a42d4c253136eeba1068c1/treef22a19e6c99dd5ae21a4f74ab8b1cba062f1250a: native process-tree focused7/0 in.73s; original-probe negative0/1 FAILED in5.14s; exact fixed bytes restored, regression1/0 in.03s; CI-policy all-target/all-feature locked TUI clippy passed2m19s. Fresh npm740/0 and check:web passed at the same source. This native regression/control proof is separate from older branch and PTY receipts above. Exact new-head three-OS hosted tests/doctests and all required checks remain required before merge; no native Windows/release/provider claim.

…etimes

- Shell job retention ages finished jobs from their finish time and never
  ages out a completion that has not been delivered, so a job that ran over
  an hour still reports its result.
- terminal/run output pruning keeps the real tail (errors, completion
  marker) and reports the true omitted byte count.
- Backgrounded lowercase `bash` jobs return only output after the read
  cursor, so `wait` no longer repeats the retained tail on every poll.
- Foreground pipe commands get EOF on stdin, matching the synchronous path,
  instead of blocking until the timeout.
- js_execution, tasks gate_run and run_tests run through a new
  process_tree::contained_output helper: a timeout or a dropped call kills
  the whole process tree. run_tests is now async, honors cancellation and
  has a 30 minute ceiling.

Tests: 10 new/updated regression tests pass; 8 of them fail with the fixes
reverted. Touched modules (tools::shell, terminal_session, js_execution,
tasks, test_runner, run_tool, process_tree): 225 passed, 0 failed,
1 ignored. Broader shell/bash/jobs/stdin/terminal/gate filter: 1105 passed,
0 failed. cargo clippy -p codewhale-tui --lib --tests --all-features with
CI flags: clean. cargo fmt --check: clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
Copilot AI balanced review requested due to automatic review settings September 29, 2026 13:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Hmbown and others added 10 commits September 29, 2026 07:00
… processes

- Shell job age: undelivered completions age out again, counted from
  finish. Jobs launched over the Runtime API, or owned by an inactive
  session, are never drained, so the earlier exemption kept them until the
  count cap.
- Count/byte eviction orders finished jobs by finish time, the same clock
  as the age rule, so a long job that just finished is no longer evicted
  first.
- Synchronous shell path: stdin is closed (not inherited) when no input
  is given, so `cat` or a prompt gets EOF instead of reading Codewhale's
  own stdin until the timeout.
- process_tree: contained_output_until returns partial output when a stop
  future fires; run_tests and tasks gate_run use it, so a timed-out run
  still reports what it wrote. A command that exits on its own is released
  instead of having its process group killed, as with a bare output().
  In-flight contained groups are killed on the signal-exit path, which
  runs no destructors.
- Omitted-bytes notice in bash deltas names the spill file.
- Tests wait for fixture pid files instead of fixed sleeps, treat zombies
  as exited, and the delta test gates its second chunk on a file.

Tests: 233 passed / 0 failed / 1 ignored across process_tree, tools::shell,
js_execution, tasks, test_runner, terminal_session, run_tool; broader
filter (runtime_api jobs shell bash stdin gate terminal test_runner)
1501 passed / 0 failed. 8 new regression tests fail with the fixes reverted.
clippy (CI flags) and fmt clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
Review finding on d314fa4: code_execution, plugin tools and runtime-API
Git still ran children with a bare output()/wait_with_output(), so a
timeout or a cancelled call left the interpreter, a plugin's children, or
git's hooks/filters/transport running. Each now runs through the existing
process_tree containment (process group / Job Object, killed on drop):

- code_execution (core/engine/tool_catalog.rs): contained_output.
- plugin tools (tools/plugin.rs): new contained_output_with_input, which
  feeds the JSON request on stdin while the output pipes drain. Plugins
  previously had no kill_on_drop at all.
- runtime_api git_read/git_write: contained_output (hooks run on writes).
- git_history run_git_command_bounded: contained_output under the
  deadline, replacing its hand-rolled process-group kill; cancellation is
  now covered, not only the deadline.

Tests (new; all four fail with the call sites reverted to output()):
dropped_code_execution_kills_the_interpreter_tree,
dropped_plugin_call_kills_the_plugin_process_tree,
dropped_git_write_kills_what_git_started,
cancelled_bounded_git_run_kills_what_git_started; plus
contained_output_with_input_feeds_then_closes_stdin.
Focused run (process_tree, tools::plugin, tools::git_history,
runtime_api::git, code_execution, js_execution, tools::tasks,
test_runner): 108 passed, 0 failed. clippy -p codewhale-tui --lib --tests
with CI -D warnings: clean. rustfmt --check: clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014ZwqatxgVFxHvovngywnks
Original head48267b5629 retained before bounded recovery. Declared corrective paths: process_tree.rs and the actual runtime_api/git.rs merge conflict. No source changes or test claims.

Signed-off-by: Hunter B <hmbown@gmail.com>
…tests

Retain both adjacent regression additions in runtime_api/git.rs: contained write cancellation and main literal path/effective remote validation. Preserve original PR ancestry. Focused post-merge checks pending.

Signed-off-by: Hunter B <hmbown@gmail.com>
Propagate process-tree attachment errors rather than accepting a run with only direct-child cleanup. Keep kill-on-drop in place for this failure path and preserve normal completion, timeout, and cancellation behavior. Add a deterministic attachment-failure seam in the existing run owner and a real child cleanup regression. Document the actual post-spawn Windows containment limit.

Source and scoped formatting/diff checks complete; focused governed tests and compiled negative control pending. Original PR and main histories retained.

Signed-off-by: Hunter B <hmbown@gmail.com>
…tracts

Portable imports reject machine-bound authority settings. Keep closed-choice
acceptance coverage on portable verbosity and preserve unknown existing fields.
Run ordinary typed plugin activation under normal supervision rather than the
600 ms fault watchdog, and use the clippy-approved search error assertion.

Validation on combined source 69824d3 plus recorded patch: CLI bundle tests
61 passed, 0 failed; typed plugin activation 1 passed; custom search fallback
1 passed. Combined all-feature CLI/TUI/workflow/workflow-js test build passed.
Unrelated Calm glyph assertion remains open (TUI combined 97 passed, 1 failed).
Full npm test: 740 passed, 0 failed (68 wrapper + 16 SDK + 54 extension + 602 web);
check:web passed. Windows handshake scheduling remains separate, unverified.

Signed-off-by: Hunter B <hmbown@gmail.com>
Six typed-save fixtures used quiet even though the validated enum accepts normal, concise and verbose. Use concise for these writes; preserve the separate raw legacy-load coverage. Six focused config tests passed, zero failed. Shared source gate: npm test674 passed/0 failed; check:web passed. No production configuration behavior changed.

Signed-off-by: Hunter B <hmbown@gmail.com>
Windows full-suite runs on PRs6776 and6780 reported normal extension hosts missing the production five-second handshake deadline. Serialize the host fixture family across nextest processes without changing production deadlines, hang-detection tests, or retries. Nextest profile ci resolves all39 tests (35 extension-host module and4 engine callers) to max-threads1; governed focused run39/39 passed in28.901s on macOS. Exact config SHA256456e681709fe5838e404fbf537f6a934d700d0577ec15bf6a18eeae17aecde53. Full npm740/0 and check:web passed. Windows resolution still requires new hosted runs.

Signed-off-by: Hunter B <hmbown@gmail.com>
Use a direct Result pattern for the failure assertion without requiring a Debug implementation on production output. Governed TUI focused tests32 passed/0 failed, including contained child ownership, equivalent consumers, and merged git validation. Full npm740/0 (68 wrapper+16 SDK+54 extension-host+602 web); check:web passed. Source negative control remains pending; no current-head hosted CI or Windows runtime claim.

Signed-off-by: Hunter B <hmbown@gmail.com>
Governed focused Rust tests: 32 passed, 0 failed. Negative control restoring
swallowed attachment errors compiled and failed at runtime: 0 passed, 1 failed.
Child cleanup passed before the expected timeout assertion. Fixed source
restored byte-for-byte; final governed checks: TUI attachment/typed-host 2/0,
changed configuration fixtures 4/0.

Full npm test: 740 passed, 0 failed (68 wrapper, 16 SDK, 54 extension-host,
602 web). check:web passed separately. Scoped formatting and diff checks pass.
Parent source review found no further blocker. Original PR and main ancestry
preserved; Windows post-spawn assignment race remains documented. No native
Windows or current-head hosted CI/release claim.

Signed-off-by: Hunter B <hmbown@gmail.com>
@gitguardian

gitguardian Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

The review already contains "Nothing was deleted", so waiting for the
substring "deleted" can return before queued confirmation completes.
Wait within the existing deadline for durable file absence, the receipt,
and closure of the armed review. Preserve the explicit final absence
assertion, input sequence, confirmation checks and diagnostic output.

Evidence: inspected exact Linux failure at 507686d (17558 passed,
1 failed, 28 skipped), including earlier 40x12/60x16 successes and the
next 80x24 failure. rustfmt and git diff --check pass. The five-size real
PTY rerun is pending the coordinated combined candidate build; no local
Rust or npm test run is claimed for this one-file fixture checkpoint.

Refs #6759
Corrected automation editor acceptance passed 1/0 in 78.44 seconds across 40x12, 60x16, 80x24, 100x32 and 140x40. Combined candidate 9791f0ffc139b83da228b700ceca8e5ba9bd7cac supplied the tested TUI, SHA-256 1bd9f79e76351dc3b9fd221d8d012c7b29658b113cf7c6ac25131ce2b502a4a8. Exact fixture, automation manager/routing/panel/views/editor/tool, harness and English automation string equivalence was verified against this branch. This is scoped combined-candidate PTY proof, not a fresh whole-branch Rust build.

Previous recovery gate: npm test 740 passed / 0 failed and check:web passed on 507686d. The follow-up changes only the PTY completion wait; no additional Cargo/npm run was performed. Formatting and diff checks pass. Updated-head Linux, Windows and macOS CI remains required.

Signed-off-by: Hunter B <hmbown@gmail.com>
Hmbown added a commit that referenced this pull request Sep 30, 2026
The review already contains "Nothing was deleted", so waiting for the
substring "deleted" can return before queued confirmation completes.
Wait within the existing deadline for durable file absence, the receipt,
and closure of the armed review. Preserve the explicit final absence
assertion, input sequence, confirmation checks and diagnostic output.

Evidence: inspected exact Linux failure at 507686d (17558 passed,
1 failed, 28 skipped), including earlier 40x12/60x16 successes and the
next 80x24 failure. rustfmt and git diff --check pass. The five-size real
PTY rerun is pending the coordinated combined candidate build; no local
Rust or npm test run is claimed for this one-file fixture checkpoint.

Refs #6759

Signed-off-by: Hunter B <hmbown@gmail.com>
Hmbown added a commit that referenced this pull request Sep 30, 2026
Combined source9791f0ffc built3m47 after an interrupted attempt. CLI103/0,configuration10/0,protocol6/0,localization52/0 and real five-size automation-editor PTY1/0 passed. TUI239 passed/1 failed/2 ignored; the only failure compared Chinese text across a visible layout rail. Test-only follow-up1b9702085 corrected that comparison; nextest41/0 includes the repaired case, real Node22.20 extension-host runtime, deferred first-use, cleared to-do and stopship receipt cases. Total452 focused passes across those batches;2 explicit measurement tests ignored. Production source is unchanged after9791; source treee06381c5238120124f9d24666982fbe221653c82 is clean.

Canonicalstream41+7 positives and3 deliberate negative failures; undo14 restored positives,8 ordered races and2 independent guard/checkpoint negative failures. All temporary mutations restored byte-identically. Combinednpm774/0 andcheck:web pass; VSCode42/0+compile; finalnotes18/0,credits8/8,35feature refs andmirror pass. No full local Rust suite. Optimized candidate build is running with explicit SHA1b9702085; this evidence-only commit leaves its source tree unchanged. Final CPU/native/install/provider/hosted qualification is not claimed. Refs #6094 #6788 #6700 #6759 #6775.

Signed-off-by: Hunter B <hmbown@gmail.com>
CodeWhale Bot added 3 commits September 30, 2026 05:43
Signed-off-by: CodeWhale Bot <bot@codewhale.net>
After dropping the contained run, use the existing spawn_blocking cleanup probe convention so the current-thread Tokio worker can poll its orphan reaper. Keep the real leader-exit deadline and contained-group listing/unlisting assertions. Production process ownership is unchanged. Focused native test/control, npm/web gate and CI-policy lint qualification are pending before the original branch push.

Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Original PR #6759 authorship and history are preserved. After contained_run is dropped, let the existing current-thread runtime poll its Tokio child reaper while the kill(pid, 0) cleanup probe runs through the neighboring existing spawn_blocking convention. Production lifetime, cancellation, deadline and containment semantics are unchanged.

Native macOS proof at 63dd034, tree f22a19e:
- test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 14140 filtered out; finished in 0.73s
- Original synchronous-wait negative: test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 14146 filtered out; finished in 5.14s
- Exact-byte restored regression: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 14146 filtered out; finished in 0.03s
- CI-policy TUI clippy --all-targets --all-features --locked -D warnings with the three standing style exceptions passed in 2m19s.
- npm test 740 passed / 0 failed; check:web passed at the same frozen source tree. Scoped formatting and diff check passed.

Hosted original macOS failure was 17565 passed / 1 failed / 27 skipped. This local proof does not replace new-head Linux/macOS/Windows CI or release acceptance. Process-tree lifetime controls are not a security sandbox; original broader branch receipts remain dated evidence. No provider spend.

Signed-off-by: CodeWhale Bot <bot@codewhale.net>
@Hmbown
Hmbown merged commit 636c700 into main Sep 30, 2026
35 checks passed
@Hmbown
Hmbown deleted the fix/bh2-tools-shell-process-lifecycle branch September 30, 2026 16:42
Hmbown pushed a commit that referenced this pull request Sep 30, 2026
#6759 already runs js_execution through process_tree::contained_output, so
the PR's hand-rolled spawn, pipe drains and process-group kill resolve to
that shared primitive (which also gives Windows its Job Object). Kept from
the contributor: the 600 s budget (5 s under test) and both regression
tests — the timed-out Node child is killed, not orphaned, and the timeout
returns promptly while a grandchild holds the pipes.

The contributor's commit 6372cbc is an unmodified parent, so #6743 lands
as itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Hmbown pushed a commit that referenced this pull request Sep 30, 2026
Brings in main through #6759 and 9507c4c, where #6743's unmodified head
6372cbc resolves onto main's contained js_execution with the
contributor's 600 s budget and both regression tests. With this, #6743
lands as itself alongside #6736/#6737/#6738/#6740/#6742; @asto18089 is
already on the unreleased credit line of this branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Hmbown pushed a commit that referenced this pull request Sep 30, 2026
Signed-off-by: CodeWhale Bot <bot@codewhale.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants