Skip to content

fix: bound remaining mcp_servers.json reads and close fifth unlocked writer - #802

Open
thomwebb wants to merge 1 commit into
mpfaffenberger:mainfrom
thomwebb:fix/mcp-servers-json-remaining-gaps
Open

fix: bound remaining mcp_servers.json reads and close fifth unlocked writer#802
thomwebb wants to merge 1 commit into
mpfaffenberger:mainfrom
thomwebb:fix/mcp-servers-json-remaining-gaps

Conversation

@thomwebb

Copy link
Copy Markdown
Collaborator

Jira: PUP-610

Second follow-up to PUP-605, per Andrew Tilson's post-merge review of #761

After #761 merged (extending bounded/atomic/locked I/O to mcp_servers.json, extra_models.json, spinners.json, and agent JSON files), Andrew left a further comment identifying two remaining gaps specific to mcp_servers.json -- the exact file this whole line of fixes exists to protect.

HIGH: config.py's load_mcp_server_configs() was still an unbounded read, and it's on the startup path

Reached at every CLI startup via MCPManager.__init__ -> sync_from_config(), and again from agents/_builder.py. This is the original PUP-605 MemoryError crash shape, unfixed, on a startup path, for the very file mcp_servers_store.py's own docstring already warned could "balloon json.load the same way configparser ballooned in the original PUP-605 crash." The sibling JSONAgent startup read was correctly bounded in #761 -- this one looked like a straightforward oversight.

Fixed via atomic_io.read_bounded_bytes, matching every other call site.

MEDIUM-HIGH: a fifth, previously unidentified mcp_servers.json writer

mcp_/config_wizard.py::run_add_wizard() (public API, exported from mcp_/__init__.py) was still doing an unlocked, unbounded read-modify-write with a fixed (collision-prone) temp filename and no fsync. #761 correctly collapsed the four writers flagged in #757's original review into one locked mcp_servers_store.py module -- but this fifth writer was missed. A lock only buys mutual exclusion if every writer participates; this one unlocked writer could silently void the safety guarantee the other four now have, which is arguably worse than before the fix, since the other four now look safe.

Fixed by migrating to the same shared mcp_servers_store.upsert_mcp_server() the other four use.

Also from the same review comment

  • Bounded two lower-priority project-level mcp_servers.json reads (mcp_/project_config.py, command_line/mcp/trust_command.py) -- project config is repo-supplied input (whatever repo you cd into), so these were flagged as next-most-wanted after the two items above.
  • atomic_io.atomic_write_bytes now masks stat mode bits via stat.S_IMODE before chmod, rather than passing os.stat()'s raw st_mode (which includes file-type bits) straight through. Harmless today since chmod masks it anyway, but worth doing properly now that it's a shared primitive.
  • path_lock's docstring now documents that it is not reentrant (flock is scoped per open-file-description, not per-process/thread) -- a sentence to keep it that way rather than a real bug found today.

One nit from the same comment -- documenting or dropping aws_bedrock/azure_foundry's now-dead save_extra_models() -- turned out to be moot: those plugins were relocated to the separate code_puppy_core_plugins package in the interim (27efe8f5) and no longer exist in this repo.

Testing

  • Added regression tests pinning both bounded-read fixes (tests/test_config_full_coverage.py, tests/mcp/test_project_config.py).
  • Rewrote the wizard-save test in tests/mcp/test_config_wizard.py to use a real tmp_path file instead of mocking open()/os.path.exists()/json.dump internals that no longer exist post-refactor -- that stale mocking had been silently letting a real file write leak to /tmp/mcp_servers.json on every test run (cleaned up).
  • Full suite: 7461 passed, 28 skipped, 1 xpassed (rebased onto the current main tip, including the code_puppy_core_plugins extraction and the session-persistence migration).
  • ruff check and ruff format --check both clean.

cc @AndrewTilson -- both flagged gaps should be closed now.

…writer

Second follow-up to PUP-605, prompted by Andrew Tilson's post-merge review
comment on mpfaffenberger#761. Two gaps remained in mcp_servers.json handling specifically:

- HIGH: config.py's load_mcp_server_configs() -- reached at every CLI
  startup via MCPManager.__init__ -> sync_from_config(), and again from
  agents/_builder.py -- still did an unbounded open()/f.read(). This is the
  exact PUP-605 MemoryError crash shape, unfixed, on a startup path, for the
  file mcp_servers_store.py's own docstring already warned could balloon
  json.load the same way configparser ballooned in the original crash. The
  sibling JSONAgent startup read was correctly bounded in mpfaffenberger#761; this one was
  an oversight. Fixed via atomic_io.read_bounded_bytes, matching every other
  call site.

- MEDIUM-HIGH: a fifth mcp_servers.json writer in
  mcp_/config_wizard.py::run_add_wizard() (public API, exported from
  mcp_/__init__.py) was still doing an unlocked, unbounded read-modify-write
  with a fixed (collision-prone) temp filename and no fsync. mpfaffenberger#761 correctly
  collapsed the four writers flagged in mpfaffenberger#757's review into one locked
  mcp_servers_store.py, but this fifth writer was missed -- and a lock only
  provides mutual exclusion if every writer participates. Migrated to the
  same shared mcp_servers_store.upsert_mcp_server() the other four use.

Also addressed, from the same review comment:

- Bounded two lower-priority project-level mcp_servers.json reads
  (mcp_/project_config.py, command_line/mcp/trust_command.py) -- project
  config is repo-supplied input (whatever repo you cd into), so these were
  flagged as next-most-wanted after the two items above.
- atomic_io.atomic_write_bytes now masks stat mode bits via stat.S_IMODE
  before chmod, rather than passing os.stat()'s raw st_mode (which includes
  file-type bits) straight through -- harmless today since chmod masks it
  anyway, but worth doing properly now that it is a shared primitive.
- path_lock's docstring now documents that it is not reentrant (flock is
  scoped per open-file-description, not per-process/thread).

Note: this review also flagged that aws_bedrock/azure_foundry's now-dead
save_extra_models() should be documented or dropped -- moot here, since
those plugins were relocated to the separate code_puppy_core_plugins
package in the interim (27efe8f) and no longer exist in this repo.

Added regression tests pinning both bounded-read fixes, and rewrote the
wizard-save test to use a real tmp_path file instead of mocking
open()/os.path.exists()/json.dump internals that no longer exist post-
refactor -- that stale mocking had been silently leaking a real file write
to /tmp/mcp_servers.json on every test run.

Full suite (minus one unrelated, pre-existing untracked/broken test file):
7461 passed, 28 skipped, 1 xpassed. ruff check + format clean.
@thomwebb thomwebb self-assigned this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant