Skip to content

refactor(config): RunConfig hygiene pass - #941

Open
chethanuk wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
chethanuk:refactor/runconfig-hygiene
Open

chethanuk wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
chethanuk:refactor/runconfig-hygiene

Conversation

@chethanuk

Copy link
Copy Markdown

📋 Summary

With disable_early_shutdown=True, an after-validator overwrote shutdown_error_rate with 1.0, so the value didn't round-trip. A data-designer create --run-config YAML that only set disable_early_shutdown: true also replaced the baseline rate. This PR replaces the mutation with an effective_shutdown_error_rate property and does the rest of the #807 cleanup. Field names, types and defaults don't change.

🔗 Related Issue

Closes #807

🔄 Changes

  • The builder and ValidationColumnGenerator read effective_shutdown_error_rate, so enforcement gets the same value as before. It's a plain @property because a computed_field gets dumped and then rejected by extra="forbid" on model_validate.
  • ThrottleConfig and _THROTTLE_DEPRECATION_MESSAGE move to run_config_deprecated.py and are imported back into run_config.py, so both import paths and old pickles still resolve.
  • Field(description=...) replaces the Attributes: block. 12 fields get a description, and section comments group the fields without reordering them.
  • STYLEGUIDE.md still uses normalize_shutdown_settings as its after-validator example. This PR doesn't touch it.

🧪 Testing

  • make test passes

  • Unit tests added/updated

  • E2E tests added/updated (if applicable)

  • RunConfig(disable_early_shutdown=True, shutdown_error_rate=0.2) reads back 0.2 with an effective rate of 1.0 and survives model_dump/model_validate and the JSON round-trip. RunConfig.model_json_schema() with descriptions stripped is identical to main.

  • DataDesigner.create with a custom column that fails on about half the rows and shutdown_error_rate=0.2: with early shutdown on it stops and salvages 4 of 40 records, with it off it finishes with the 20 good ones.

  • The 6 touched test files pass (256), including a legacy-pickle test for the ThrottleConfig move.

✅ Checklist

  • Follows commit message conventions
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

With disable_early_shutdown=True, an after-validator overwrote
shutdown_error_rate with 1.0 and marked it as set. The value did not
round-trip, and a run-config YAML that only disabled early shutdown
replaced the baseline rate during the CLI overlay.

Replace the validator with a read-only effective_shutdown_error_rate
property and point the two engine readers at it, so enforcement gets
the same value as before while shutdown_error_rate keeps the user's
input.

Make Field(description=...) the single place field docs live: the 12
undescribed fields take their text from the Attributes block, which is
removed. Group fields with section comments without reordering them.

Move the deprecated ThrottleConfig and its message to
run_config_deprecated.py and import them back, so every existing import
path and old pickles still resolve.

No public field names, types, defaults or bounds change.

Signed-off-by: ChethanUK <chethanuk@outlook.com>
The run-config overlay was only exercised with load_run_config mocked, so
the YAML-file hop that produces the reported symptom was never covered
through the command a user actually runs.

Invoke the real create command with a run-config YAML on disk and assert
the baseline shutdown_error_rate survives the overlay, with the derived
effective rate still 1.0 when early shutdown is disabled.

Signed-off-by: ChethanUK <chethanuk@outlook.com>
The move to run_config_deprecated changed the module ThrottleConfig
reports. The compatibility claim that pickles written before the move
still load rested on the module-level re-import in run_config, with
nothing covering it.

Rebuild a pre-move payload by pickling with __module__ set to the old
path and assert it loads. Dropping the re-import makes this fail with
AttributeError, so the test guards that binding instead of restating it.

Signed-off-by: ChethanUK <chethanuk@outlook.com>
@chethanuk
chethanuk requested a review from a team as a code owner September 16, 2026 20:10
@github-actions

Copy link
Copy Markdown
Contributor

Linked Issue Check

Issue #807 has not been triaged yet. A maintainer needs to review
the issue and add the triaged label for this check to pass.

You can continue working on the PR in the meantime. The check will
re-run automatically once the issue is triaged.

@greptile-apps

greptile-apps Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with runtime consumers, configuration overlays, and deprecated compatibility paths aligned with the new non-mutating semantics.

Summary

This PR separates the configured shutdown error rate from the rate used during execution, allowing RunConfig to round-trip without mutation while preserving existing early-shutdown behavior.

  • Adds effective_shutdown_error_rate and routes both dataset generation and parallel validation through it.
  • Moves deprecated throttle compatibility code into a dedicated module while preserving public imports and legacy pickle resolution.
  • Adds field-level schema descriptions and expands configuration, CLI-overlay, engine, and compatibility tests.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[RunConfig input] --> B[Preserved shutdown_error_rate]
    A --> C{disable_early_shutdown?}
    C -- Yes --> D[effective rate 1.0]
    C -- No --> B
    B --> E[effective configured rate]
    D --> F[Dataset builder and validation executor]
    E --> F
Loading

Reviews (1) · Last reviewed commit: "test(config): cover legacy ThrottleConfi..."

@github-actions

Copy link
Copy Markdown
Contributor

Stale PR reminder

This PR has had failing checks for 7 days without activity.

Failing checks: check

Please push an update or leave a comment if you're still working on this.
Otherwise, this PR will be automatically closed in 7 days.

To prevent auto-close, add the keep-open label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(config): non-breaking hygiene pass on RunConfig

1 participant