Summary
The actions advertise include-checks, which current oasdiff ignores, and do not expose severity-levels, which replaced it. So the documented way to enable an optional check silently does nothing, and the working way is not reachable from an action input.
include-checks is ignored
oasdiff deprecated the flag and now ignores it, pointing at --severity-levels instead:
"include-checks": "it is now ignored; use --severity-levels to enforce a check as breaking"
The actions still advertise it as a supported input. From the README:
| input |
default |
description |
include-checks |
'' |
Include optional breaking change checks |
It is also still passed through in action.yml and the entrypoints. A user who sets it gets no error and no effect, which is the worst of the three possible outcomes: they believe a check is enabled when it is not, and a check they wanted enforced never fires.
This came up while closing #154. The version-bump checks there are exactly the sort of thing someone would reach for include-checks to enable, and they would silently get nothing.
severity-levels is only reachable through a config file
The replacement works from an action today, but only via .oasdiff.yaml:
# .oasdiff.yaml
severity-levels: oasdiff-levels.txt
fail-on: ERR
That is fine as a mechanism, and worth keeping. But every other commonly used oasdiff flag has a first-class input, so the one flag that controls which changes fail the build being config-file-only is a surprising gap, and it is undiscoverable from the action docs.
Proposal
- Add a
severity-levels input to the actions that take one (breaking, changelog, diff, pr-comment), passed through to --severity-levels, documented in the README alongside fail-on.
- Deal with
include-checks. Either drop the input, or keep it and emit a deprecation annotation naming severity-levels, so an existing workflow that sets it is told rather than silently ignored. A warning annotation is probably the kinder first step, since removing the input outright breaks workflows that currently pass it harmlessly.
- Update the README, which currently documents
include-checks as functional.
Note
Whatever is decided for the actions, the underlying flag's behaviour is oasdiff's, not the actions'. This issue is only about the actions advertising a flag that no longer does anything and not surfacing the one that replaced it.
Summary
The actions advertise
include-checks, which current oasdiff ignores, and do not exposeseverity-levels, which replaced it. So the documented way to enable an optional check silently does nothing, and the working way is not reachable from an action input.include-checksis ignoredoasdiff deprecated the flag and now ignores it, pointing at
--severity-levelsinstead:The actions still advertise it as a supported input. From the README:
include-checks''It is also still passed through in
action.ymland the entrypoints. A user who sets it gets no error and no effect, which is the worst of the three possible outcomes: they believe a check is enabled when it is not, and a check they wanted enforced never fires.This came up while closing #154. The version-bump checks there are exactly the sort of thing someone would reach for
include-checksto enable, and they would silently get nothing.severity-levelsis only reachable through a config fileThe replacement works from an action today, but only via
.oasdiff.yaml:That is fine as a mechanism, and worth keeping. But every other commonly used oasdiff flag has a first-class input, so the one flag that controls which changes fail the build being config-file-only is a surprising gap, and it is undiscoverable from the action docs.
Proposal
severity-levelsinput to the actions that take one (breaking,changelog,diff,pr-comment), passed through to--severity-levels, documented in the README alongsidefail-on.include-checks. Either drop the input, or keep it and emit a deprecation annotation namingseverity-levels, so an existing workflow that sets it is told rather than silently ignored. A warning annotation is probably the kinder first step, since removing the input outright breaks workflows that currently pass it harmlessly.include-checksas functional.Note
Whatever is decided for the actions, the underlying flag's behaviour is oasdiff's, not the actions'. This issue is only about the actions advertising a flag that no longer does anything and not surfacing the one that replaced it.