Skip to content

1.100 beta crater regression: test failures in 64+ crates with "No such file or directory" #164098

Open

Description

@theemathas

This was discovered in the 1.100 beta crater run.

64+ crates in the crater run had test failures with messages like:

called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }

The most notable affected crates, from a quick glance, are ripgrep and once_cell:

A larger list of affected crates can be found at https://crater-reports.s3.amazonaws.com/beta-1.100-4/index.html. (Click on regressed: root results and then scroll down to test failed (unknown) (156). Not all of the 156 crates listed are affected by this. Only 64 of them had the "No such file or directory" error.)

(Links above for ripgrep and once_cell are from an earlier crater run, which had more than 64 crates found, but potentially has more false positives due to spurious failures.)


I ran a bisection on once_cell-1.21.4 (obtained via cargo download), with the command cargo bisect-rustc --start=1.98.0 --end=beta -- test -- reentrant_init, and got the following output.

searched toolchains 96eb6d756bb98a82ae056c384575ff059bf80e8f through fd98695850065878fa7de48808dca4f0901cefbb


********************************************************************************
Regression in 83709eedfff36840b0749ca429ac43f5479b268f
********************************************************************************

Attempting to search unrolled perf builds
ERROR: couldn't find perf build comment

The regression happened in #159857. I personally suspect that the cause is the build-dir v2 change in cargo.

Note that running the bisection with --start=1.99.0 instead gave the error: ERROR: the commit at the start of the range (d453bdd8f092d099bc336f0bda4163f809ad18e0) includes the regression. This is consistent with my suspicion above, since cargo build-dir v2 is one of the changes that were initially in 1.99.0, and then had a beta backport to delay the change.

Activity

  1. added
    T-cargoRelevant to the cargo team, which will review and decide on the PR/issue.
    C-bugCategory: This is a bug.
    E-needs-mcveCall for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example
    I-cargo-nominatedNominated for discussion during a cargo team meeting.
    on Oct 10, 2026
  2. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Oct 10, 2026
  3. theemathas commented on Oct 10, 2026

    @theemathas
    ContributorAuthor

    If we assume that all the failures with "No such file or directory" in the beta-1.100-3 run are real, then the number of crates affected would be 257+. The real number might be even higher.

  4. epage commented on Oct 10, 2026

    @epage
    Contributor

    For once_cell, we filed an issue back in March, see matklad/once_cell#297.

    For all of the issues and their status see our previous crater run: #149852

    These crates are relying on internals of cargo and this is a change that unblocks a lot of high-value work so we were moving forward at a faster pace.

  5. theemathas commented on Oct 11, 2026

    @theemathas
    ContributorAuthor

    There are 53+ other crates that are failing with

    [INFO] [stdout] called `Result::unwrap()` on an `Err` value: CargoError { cause: Some(NotFoundError { path: "/opt/rustwide/target/debug/build/ash/9573f013a5037aaa/out/ash" }) }
    

    These appear to be due to the tests using the assert_cmd crate, which appears to be maintained by @epage.

  6. theemathas commented on Oct 11, 2026

    @theemathas
    ContributorAuthor

    Also, cc @matklad, maintainer of once_cell.

  7. theemathas commented on Oct 11, 2026

    @theemathas
    ContributorAuthor

    Are all of these crates also failing due to the same cause, and therefore accepted regressions?

    "CARGO_BIN_EXE_crate-name-here is unset":

    Others:

  8. removed
    E-needs-mcveCall for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Oct 11, 2026
  9. theemathas commented on Oct 11, 2026

    @theemathas
    ContributorAuthor

    Other crater regressions that might be due to the cargo build dir v2 change: #164082, #164085, #164160, #164136

  10. ranger-ross commented on Oct 11, 2026

    @ranger-ross
    Member

    for ripgrep, should already be resolved by BurntSushi/ripgrep#3501 but a new version has not been published.

    These appear to be due to the tests using the assert_cmd crate

    assert_cmd was updated to handle the new build-dir, its just a matter of users bumping to the newer versions. (see assert-rs/assert_cmd#234 and assert-rs/assert_cmd#282)

  11. weihanglo commented on Oct 11, 2026

    @weihanglo
    Member

    I used LLM to analyze #164098 (comment), and verified the analysis. So most of the failures above are expected.

    LLM analysis

    Issue/PR statuses were checked on 2026-10-11.

    Category Crate / code permalink Layout dependency Previous report / fix
    Unit-test CLI lookup 13262233692/api-migrate-186 assert_cmd path fallback None found
    Unit-test CLI lookup berquerant/semv assert_cmd path fallback Old run; epage #25, open
    Unit-test CLI lookup onplt/conflic assert_cmd path fallback None found
    Unit-test CLI lookup remix7531/silktouch assert_cmd path fallback None found
    Unit-test CLI lookup rickprice/csvargs assert_cmd path fallback Old run; epage #1, open
    Cross-package binary lookup Boluwatife-AJB/nest-forge assert_cmd path fallback None found
    Cross-package binary lookup brevity1swos/sift assert_cmd path fallback None found
    Cross-package binary lookup David610/singbox-vpn Inferred deps/ layout; assert_cmd path fallback None found
    Cross-package binary lookup Ajay-26/rivet Inferred deps/ layout None found
    Cross-package binary lookup muhnschein/cairn Inferred deps/ layout None found
    Cross-package binary lookup nickderobertis/allowlister-remote Inferred deps/ layout None found
    Manual binary lookup 777genius/lintai Inferred deps/ layout None found
    Manual binary lookup ax-lab/tux Inferred deps/ layout Old run; epage #10, open
    Manual binary lookup D-A-G-O-A-T/dagoat Inferred deps/ layout None found
    Manual binary lookup narnaud/git-loom Test harness identified by deps/ None found
    Manual binary lookup ybzhang-nxu/egg-simplify Inferred deps/ layout Old run; epage #1, open
    Manual binary lookup feedlynx 0.4.0 Inferred deps/ layout Old run; epage #23, open; ranger-ross PR #28, open
    Outdated test helper Bubbler-4/esobox test_bin 0.4 path lookup Old run; epage #1, open; ranger-ross PR #2, merged
    Outdated test helper lipen_grrs 0.1.0 assert_cmd 1.x path lookup Old run; epage's triage; no linked upstream issue found
    Library / intermediate lookup AltVolts/AltVolts-ya-rust-sprint-4-project Inferred deps/ layout None found
    Library / intermediate lookup ethever/souffle-rs Assumed artifacts share deps/ None found
    Library / intermediate lookup Boshen/cargo-bsize Assumed artifacts reside in deps/ ranger-ross PR #6, open
    Directory / profile inference reovim-testing 0.14.4 Profile directory inferred from executable path None found
    Directory / profile inference rustline 0.1.0 Profile inferred from executable path None found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: This is a bug.I-cargo-nominatedNominated for discussion during a cargo team meeting.I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}T-cargoRelevant to the cargo team, which will review and decide on the PR/issue.regression-from-stable-to-betaPerformance or correctness regression from stable to beta.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions