code that overrides `process_value` receives `Sentinel` object

[ggainey]

I am running our release process 27 minutes after click/8.3.0 release, and failing with the stacktrace seen here: https://github.com/pulp/pulpcore/actions/runs/17837218351/job/50717339446#step:7:29

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.13/x64/bin/bump-my-version", line 7, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/rich_click/rich_command.py", line 395, in __call__
    return super().__call__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/rich_click/rich_command.py", line 209, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/click/core.py", line 1848, in invoke
    sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/click/core.py", line 1206, in make_context
    self.parse_args(ctx, args)
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/click/core.py", line 1217, in parse_args
    _, args = param.handle_parse_result(ctx, opts, args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/click/core.py", line 2516, in handle_parse_result
    value = self.process_value(ctx, value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/bumpversion/click_config.py", line 71, in process_value
    return resolve_conf_location(value) if value else None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/bumpversion/click_config.py", line 117, in resolve_conf_location
    parsed_url = urlparse(url_or_path)
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/urllib/parse.py", line 394, in urlparse
    url, scheme, _coerce_result = _coerce_args(url, scheme)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/urllib/parse.py", line 133, in _coerce_args
    return _decode_args(args) + (_encode_result,)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/urllib/parse.py", line 117, in _decode_args
    return tuple(x.decode(encoding, errors) if x else '' for x in args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/urllib/parse.py", line 117, in <genexpr>
    return tuple(x.decode(encoding, errors) if x else '' for x in args)
                 ^^^^^^^^
AttributeError: 'Sentinel' object has no attribute 'decode'
Error: Process completed with exit code 1.

The only difference between this failure, and a previous run in a different repo about 30 minutes prior, is the click-update:

diff --git a/core.sorted b/rpm.sorted
index 121f62a..d193b72 100644
--- a/core.sorted
+++ b/rpm.sorted
@@ -1,9 +1,10 @@
+
 annotated-types-0.7.0
 anyio-4.10.0
 bracex-2.6
 bump-my-version-1.2.2
 certifi-2025.8.3
-click-8.3.0
+click-8.2.1
 h11-0.16.0
 httpcore-1.0.9
 httpx-0.28.1

You can see the release-script here: https://github.com/pulp/pulpcore/blob/main/.github/workflows/scripts/release.sh

I don't have an isolated reproducer yet - thought I'd get a notification up ASAP.

Environment:

• Python version: 3.11
• Click version: 8.3.0

[sirosen]

8.3 is known to be a breaking release.
The pallets projects declare their versioning here: https://palletsprojects.com/versions

I have a few things broken by this too, but the resolution is to pin to <8.3 right now, as you might with a major release of a semver-versioned library.

[davidism]

This appears to be from bumpversion:

  File "/opt/hostedtoolcache/Python/3.11.13/x64/lib/python3.11/site-packages/bumpversion/click_config.py", line 71, in process_value
    return resolve_conf_location(value) if value else None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The processing pipeline internals have changed a bit, in order to more consistently handle the various ways defaults and flags interact in options. A new internal sentinel is passed around to indicate that a value was not manually set to None.

bumpversion is overriding process_value and using the value if it's true. The sentinel value is true. We did consider whether to make it false, but decided we wanted to be sure code that overrode very internal things like this was correctly handling the new combinations.

[davidism]

It would be great to see some other reproducers, to know how existing code is running into the new sentinel. Standalone examples would also be helpful so we don't have to dig through other project's code.

[ggainey]

8.3 is known to be a breaking release. The pallets projects declare their versioning here: https://palletsprojects.com/versions

I have a few things broken by this too, but the resolution is to pin to <8.3 right now, as you might with a major release of a semver-versioned library.

Yeah - in our case, we don't (or didn't) specifically require click in the infrastructure/CI paths and it's being pulled in by black and bump-my-version unrestrained. So the transitive dependency bit us. We're explicitly pinning in a few places to resolve.

(Fortunately, the actual app is explicit about click's LB/UB, for exactly this reason, so at least end-users are safe)

[sirosen]

Ah, I didn't realize that this is about downstream projects from click, but upstream from you!

Also, I think this release will impact pip-tools -- I just checked and I see that the version is set to >=8. I'll be testing that when I have time this evening. ($WORK gets priority right now.)

[ggainey]

Ah, I didn't realize that this is about downstream projects from click, but upstream from you!

Yeah - it's often hard to tell whether the problem is an unexpected-bug, or that Somebody Somewhere is Doing It Worng :) I opened here, because a) it might be an actual unexpected behavior for click/8.3, but also b) this is prob the first place ppl will go if/when/as they hit the problem.

[adrianlyjak]

Seems like this is breaking a fair number of mature/highly depended on packages that don't have a "minor" version restriction (hatch, black). This seems messy

[ryanbordo]

I ran into a similar issue when using deepcopy on a click.Command but only in 8.3.0. Is this no longer supported?

ValueError: <object object at 0x760957b6fb90> is not a valid Sentinel

[davidism]

@ryanbordo the sentinel is a Python enum.Enum. If that's not deep-copyable, you'll need to report that to Python. I don't think it's possible to assume any object is deep-copyable, in which case the code could have failed with other option values as well.

[davidism]

@adrianlyjak it would be more helpful to say how these packages are affected, as requested above. We are totally willing to work on this, but we need to be shown what's going on in the code, not just told "go search for it".

[ggainey]

Well, in the case of my specific bump-my-version stacktrace : https://github.com/callowayproject/bump-my-version/blob/master/bumpversion/click_config.py#L68

BMV creates a ConfigOption, which is-A click.Option, and they're overriding process_value(ctx, value) and "assuming" the value is a str. That gets passed into urllib, and eventually breaks deep inside.

(Which is as far as I care to go digging around the insides of Someone Else's Codebase :) )

[epmog]

Hatch appears to be relying on an unspecified value as being None, which now seems to be Sentinel which breaks truthiness checks.

https://github.com/pypa/hatch/blob/master/src/hatch/cli/env/run.py#L41

@click.option('--filter', '-f', 'filter_json', help='The JSON data used to select environments')
...
    filter_json: str | None,
...

if filter_json:
    json.loads(filter_json)

So when you do hatch run lint, you get the following crash:

│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/hatch/cli/env/ru │
│ n.py:123 in run                                                                                  │
│                                                                                                  │
│   120 │   if filter_json:                                                                        │
│   121 │   │   import json                                                                        │
│   122 │   │                                                                                      │
│ ❱ 123 │   │   filter_data = json.loads(filter_json)                                              │
│   124 │   │   if not isinstance(filter_data, dict):                                              │
│   125 │   │   │   app.abort('The --filter/-f option must be a JSON mapping')                     │
│   126                                                                                            │
│                                                                                                  │
│ /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py:339 in loads  │
│                                                                                                  │
│   336 │   │   │   │   │   │   │   │     s, 0)                                                    │
│   337 │   else:                                                                                  │
│   338 │   │   if not isinstance(s, (bytes, bytearray)):                                          │
│ ❱ 339 │   │   │   raise TypeError(f'the JSON object must be str, bytes or bytearray, '           │
│   340 │   │   │   │   │   │   │   f'not {s.__class__.__name__}')                                 │
│   341 │   │   s = s.decode(detect_encoding(s), 'surrogatepass')                                  │
│   342                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: the JSON object must be str, bytes or bytearray, not Sentinel

[davidism]

@epmog I can't reproduce that with the information provided. The sentinel value is not passed to commands, it's only used during processing. So the following standalone example works as expected still:

import click

@click.command()
@click.option("-f", "--filter-json", "filter_json")
def cli(filter_json) -> None:
    click.echo(str(filter_json))

cli()

$ python example.py
None
$ python example.py -f test
test

[epmog]

Ah, they have a bit more logic where they have another wrapper command and perform a ctx.invoke. I'll see if I can't get something like that isolated.

[davidism]

process_value is not present in the documentation or API docs, and therefore is not considered a public API. Private code may change at any time. If projects need to override private code (or code that is documented but marked internal), they either accept the risk, or should open an feature request asking for a public API for what they need to do.

We still might fix this case, but I just want to emphasize: process_value was not a public API and did not come with any sort of guarantees.