Repository navigation
Blessing clippy with x.py does not work the same way as other tools #102093
Description
I can this command:
./x.py test src/tools/clippy --stage 1 --bless
I expected this to bless the test files. Instead I got output like this:
-error: aborting due to 8 previous errors
+error[E0080]: evaluation of constant value failed
+ --> $DIR/indexing_slicing_index.rs:10:24
+ |
+LL | const REF_ERR: &i32 = &ARR[idx4()]; // Ok, let rustc handle const contexts.
+ | ^^^^^^^^^^^ index out of bounds: the length is 2 but the index is 4
+
+error: aborting due to 9 previous errors
For more information about this error, try `rustc --explain E0080`.
The actual stderr differed from the expected stderr.
Actual stderr saved to /home/r/src/rust/rustc.3/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/test/ui/indexing_slicing_index.stage-id.stderr
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args indexing_slicing_index.rs`
Ironically, the output tells me to add --bless, which I already did!
Cc @rust-lang/clippy
Activity
matthiaskrgr commentedon Sep 21, 2022
does it work if you cd into the clippy dir root and run cargo dev bless ?
flip1995 commentedon Sep 21, 2022
The
--blesscommand with Clippy functions a little different from the--blesscommand that is usually used.For Clippy it first runs all UI tests, which will give you the above output, then compile
clippy_devand runclippy_dev --bless. IIRC this then copies the newly generatedstderrfile out of thebuilddir into Clippy. If that doesn't work then maybe some paths changed, so it cannot be copied?RalfJung commentedon Sep 21, 2022
Or maybe I misinterpreted the output? I got a clippy test failure, scrolled up across a screen worth of noise in the output, and then found that error.
Details
Maybe part of the problem is that I first ran it without
--bless, which failed, and then I re-ran it with--blesswhich should fix the issue but did not?flip1995 commentedon Sep 21, 2022
This part of the output shows that
clippy_devwas indeed compiled and run.And this line:
Shows that the file was also updated by the
--blesscommand. So--blessdefinitely did something. Maybe it copied a wrong/outdated generated reference file? But that's unlikely...RalfJung commentedon Sep 21, 2022
The odd thing is that I did see the file changed in vscode. (I didn't realize that that was the case but now I think that is what happened.) But the tests still failed until I manually edited the file...
jyn514 commentedon Sep 22, 2022
That sounds like you accidentally overwrote the blessed file at first.
I agree the UI here is confusing but I'm not yet sure that bless is actually broken, just that it's hard to use.
RalfJung commentedon Sep 23, 2022
How can anything I do in an editor lead to
x.py --blessfailing when the only error is a wrong output file? That's not possible, so there's definitely a bug here. I definitely never did Ctrl-S in that file.RalfJung commentedon Sep 23, 2022
Or maybe the UI is so confusing that I still don't know how to use it. 🤷 I would assume that
x.pytakes care of all the weirdness, so all I have to do is callx.py --blesslike for all other test suites.RalfJung commentedon Sep 23, 2022
Oh are you saying that with
--blessthe test will still fail but then when I run it a second time it should succeed because it updated the output files?I don't know if I tried that. I was working on rustc so I did not read any clippy-specific documentation (and I don't think I should have to), but rustc CI fails when clippy tests don't pass so I was forced to try and get them to pass again, which I expected to work like it does for rustc and Miri tests. (It's already bad enough that clippy does not work with
--stage 0which means a lot of time and disk space is consumed each time a PR makes clippy fail.)flip1995 commentedon Sep 28, 2022
Yes, this is how the flag works for clippy
RalfJung commentedon Sep 28, 2022
Okay, that would probably explain what I saw.
So this issue is then tracking to make
x.py --blesson clippy work the same way it works for all other parts of the rustc repo.[-]Cannot bless clippy with x.py[/-][+]Blessing clippy with x.py does not work the same way as other tools[/+]jyn514 commentedon Feb 19, 2023
This isn't something bootstrap controls. It would need to be changed in clippy's version of compiletest (or I guess we could try and guess which output on stderr will be hidden on the next run, but that seems very error prone).
Compiletest for clippy is kind of in limbo right now ... @flip1995 @matthiaskrgr @oli-obk do you know the current state of it? are you planning to use oli's rewrite or try and sync it with the rust-lang/rust compiletest?
Manishearth commentedon Feb 19, 2023
I don't think we have active plans to use oli's rewrite but we're open to it if it supports everything we need.
We have proposed multiple times that rustc compiletest be published out of tree (perhaps with the plugin API exposed so rustc can have add modes without doing a dance) and it's been shot down by compiler team members because they want it to be easy to tweak the test suite.
oli-obk commentedon Feb 19, 2023
I still need to implement rustfix support. After that I just need to rebase my clippy PR and open it for the clippy team to discuss.
Manishearth commentedon Feb 19, 2023
Great!
flip1995 commentedon Feb 27, 2023
I think this is a good reason for Clippy to also switch to oli's implementation. Trying to maintain a compiletest copy out of tree is probably more annoying than something build for out of tree tools.
oli-obk commentedon Feb 28, 2023
I got it working with clippy, so now there's a PR up at rust-lang/rust-clippy#10426
Auto merge of rust-lang#113260 - flip1995:clippyup, r=Manishearth