Skip to content

fix(generators): execute task generators atomically - #933

Merged
tobiasraabe merged 10 commits into
mainfrom
codex/fix-task-generator-execution
Sep 20, 2026
Merged

tobiasraabe merged 10 commits into
mainfrom
codex/fix-task-generator-execution

Conversation

@tobiasraabe

@tobiasraabe tobiasraabe commented Aug 2, 2026 •

Copy link
Copy Markdown
Member

Summary

  • stop task generators after the generator hook has handled them so their bodies execute once
  • collect every generated child report and surface all child collection failures
  • commit generated tasks, the rebuilt DAG, and the rebuilt scheduler as one in-memory batch
  • process a failed generator like a normal task failure so descendants skip and unrelated tasks continue
  • preserve the identity of the session task list for plugin compatibility

Root cause

pytask_execute_task is a first-result hook, but the generator implementation returned None after executing a generator. Pluggy therefore continued to the default executor and called the generator body a second time. The implementation also committed successful generated siblings before checking failed collection reports and only retained the final report.

Impact

A generator now executes once. Generated child collection is atomic: all reports are retained for diagnostics, but no generated child or rebuilt DAG is committed when any child fails. The generator receives one ordinary failed execution report, its descendants are skipped, and unrelated tasks can continue.

This PR changes only core pytask. pytask-parallel requires its separately coordinated controller-side generator handling before it can support this contract.

@tobiasraabe tobiasraabe changed the title Fix task generator execution fix(generators): execute task generators atomically Aug 2, 2026
@tobiasraabe
tobiasraabe marked this pull request as ready for review August 2, 2026 18:45
@tobiasraabe
tobiasraabe added this pull request to the merge queue Sep 20, 2026
Merged via the queue into main with commit aef28f6 Sep 20, 2026
23 checks passed
@tobiasraabe
tobiasraabe deleted the codex/fix-task-generator-execution branch September 20, 2026 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant