Stop pretending include-checks works, and point it at .oasdiff.yaml - #197
Merged
Conversation
Closes #196. oasdiff retired the optional-checks mechanism include-checks drove and now ignores the flag. The actions still advertised it as functional, so the documented way to enforce a check silently did nothing: the user believes a check is enforced, no error appears, and it never fires. That is the worst of the available failure modes, and it is the bug here. The action no longer forwards the flag and emits a warning annotation instead. Forwarding it was pointless, since oasdiff ignores it, and would break outright if the flag is ever removed rather than ignored. The input stays so workflows that set it today keep working. The replacement, severity-levels, is deliberately NOT added as an input. .oasdiff.yaml is the configuration surface these actions converge on, and the inputs are a deliberate subset: 20 of oasdiff's 30 breaking flags have no input, including auto-upgrade, stability-level, match-path and format. Adding one more would grow a surface we intend to stop growing, and every input is another positional argument in a scheme where a mis-ordered index corrupts a run silently. The deprecation message and the README therefore teach the config file, which already worked and needed no code change. Tests: one job configures severity-levels through .oasdiff.yaml and asserts the downgrade takes effect, alongside the existing yaml-config jobs for fail-on and err-ignore. Another passes a check name that does not exist to include-checks and asserts the run is unaffected, which fails if the action ever forwards it again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
reuvenharrison
force-pushed
the
fix/severity-levels-input
branch
from
August 9, 2026 22:33
4ee5d59 to
4c04ab4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #196.
The bug
oasdiff retired the optional-checks mechanism
include-checksdrove, and now ignores the flag:The actions still advertised it as functional. So the documented way to enforce a check did nothing at all: the user believes a check is enforced, no error appears, and it never fires. Silence is the worst of the available failure modes, and it is the whole of the bug.
The fix
The action no longer forwards the flag, and emits a warning annotation instead:
The input itself stays, so workflows that pass it today keep working rather than failing on an unknown input. Forwarding it was pointless (oasdiff ignores it) and would break outright if the flag is ever removed rather than merely ignored.
Why there is no
severity-levelsinput#196 proposed adding one. It is deliberately not here.
.oasdiff.yamlis the configuration surface these actions converge on, and the inputs are a deliberate subset, not an aspiration to cover the flag set. Countingbreaking:fail-on,composed,err-ignoreauto-upgrade,stability-level,match-path,prefix-base,format,lang,templateSo not having an input is the norm.
severity-levelsalready worked through the config file before this PR, needing no code at all, and adding an input would grow a surface we intend to stop growing. It would also add another positional argument, in a scheme where a mis-ordered index corrupts a run silently rather than erroring.The deprecation message and the README therefore teach the config file:
That is the recipe #154 needed.
Tests
severity-levelsvia.oasdiff.yaml— downgradesapi-removed-without-deprecation(this repo's only breaking finding on the test specs) toinfo, and assertsfail-on: ERRstops failing. Sits alongside the existing yaml-config jobs forfail-onanderr-ignore, so the config path is pinned rather than assumed.include-checks— passesnot-a-real-checkand asserts the run is unaffected. This fails if the action ever forwards the input again, since oasdiff would reject the name.Also verified by running the entrypoint against a stubbed
oasdiffthat logs its argv: neither--include-checksnor--severity-levelsreaches the CLI, and the annotation is emitted.