Repository navigation
Linker hangs with no output 9/10 times. #88704
Description
Activity
Could you obtain stacktraces for the hanged process? I though we fixed all hangs like that in LLD 13 but maybe Rust's ancient GCC is even more broken than more recent versions.
The easiest way is to attach debugger to the process (this sometimes fixes the hang so you cannot get the trace) or create dump by right clicking on the process in task manager and load it into the debugger.
Anyway I'm back home so I've obtained the trace myself:[0x0] ntdll!NtWaitForSingleObject + 0x14 [0x1] KERNELBASE!WaitForSingleObjectEx + 0x8e [0x2] libwinpthread_1!pthread_cond_init + 0x22c [0x3] libwinpthread_1!pthread_cond_init + 0x36e [0x4] libwinpthread_1!pthread_cond_signal + 0xcf [0x5] rust_lld + 0x282fcd9 [0x6] rust_lld + 0x1f6f14c [0x7] rust_lld + 0xdda465 [0x8] rust_lld + 0xcc9958 [0x9] rust_lld + 0xdb497c [0xa] rust_lld + 0xdc70ab [0xb] rust_lld + 0xdc059a [0xc] rust_lld + 0xe32940 [0xd] rust_lld + 0x3cc319 [0xe] rust_lld + 0x2b22910 [0xf] rust_lld + 0x13f8 [0x10] rust_lld + 0x151b [0x11] kernel32!BaseThreadInitThunk + 0x14 [0x12] ntdll!RtlUserThreadStart + 0x21Indeed it looks like bug in old GCC or mingw-w64 used by Rust to build this target.
Unfortunately there is no easy fix, last time I tried to upgrade the tools it couldn't make it on the CI: #51989
Mingw-builds haven't provided any new build since then, I'll try to provide more up to date toolchain but cannot give any ETA.I don't know how
rust-lldis called here but as a workaround you could try usingld.lldfrom official LLVM builds or MSYS2.rust-lldis called viacargo->rustc->rust-lldIIRC.Can you reach out the mingw developers to see if they can provide an updated build?
How would I go about using
ld.lld? overwrite therust-lld.exewith a different.exeor?I found the documentation for changing the linker:
https://doc.rust-lang.org/rustc/codegen-options/index.html#linker-flavor
I will give that a try as time permits and report back.
This is still an issue, I changed my linker to the gnu-ld linker for other reasons (map file generation, printing memory usage stats, etc) and forgot about this issue. Today when working on a new project I encountered the error again. I'm using a much newer version of rust since I opened this issue.
rustup showreports the following:Default host: x86_64-pc-windows-gnu rustup home: D:\Users\Hydra\.rustup installed toolchains -------------------- stable-x86_64-pc-windows-gnu nightly-2019-07-01-x86_64-pc-windows-gnu nightly-2021-08-23-x86_64-pc-windows-gnu nightly-2021-09-07-x86_64-pc-windows-gnu nightly-2022-03-27-x86_64-pc-windows-gnu (default) nightly-2022-04-24-x86_64-pc-windows-gnu nightly-x86_64-pc-windows-gnu installed targets for active toolchain -------------------------------------- thumbv7em-none-eabihf x86_64-pc-windows-gnu active toolchain ---------------- nightly-2022-03-27-x86_64-pc-windows-gnu (default) rustc 1.61.0-nightly (1d9c262ee 2022-03-26)Has there been any update to the tools? what's the current status of this? are there any other workarounds?
Any update on this? I find it a bit crazy that such a fundamental issue, i.e. it randomly fails 9/10 times, hasn't been prioritized over other changes.
This still happens on a very recent nightly:
Default host: x86_64-pc-windows-gnu rustup home: D:\Users\Hydra\.rustup installed toolchains -------------------- nightly-2019-07-01-x86_64-pc-windows-gnu nightly-2021-08-23-x86_64-pc-windows-gnu nightly-2021-09-07-x86_64-pc-windows-gnu nightly-2022-03-27-x86_64-pc-windows-gnu nightly-2022-04-24-x86_64-pc-windows-gnu nightly-2022-05-31-x86_64-pc-windows-gnu nightly-2022-06-17-x86_64-pc-windows-gnu nightly-2022-07-13-x86_64-pc-windows-gnu (default) installed targets for active toolchain -------------------------------------- thumbv7em-none-eabihf thumbv7m-none-eabi x86_64-pc-windows-gnu active toolchain ---------------- nightly-2022-07-13-x86_64-pc-windows-gnu (default) rustc 1.64.0-nightly (1c7b36d4d 2022-07-12)example, frustrating, user experience:
D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor>cargo build Compiling stm32f4_pwm_input v0.1.0 (D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor) ^C Building [=========================> ] 97/98: stm32f4_pwm_input(bin) D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor>cargo build Compiling stm32f4_pwm_input v0.1.0 (D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor) ^C Building [=========================> ] 97/98: stm32f4_pwm_input(bin) D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor>cargo build Compiling stm32f4_pwm_input v0.1.0 (D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor) ^C Building [=========================> ] 97/98: stm32f4_pwm_input(bin) D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor>cargo build Compiling stm32f4_pwm_input v0.1.0 (D:\Users\Hydra\Documents\dev\playground\rust\rtic\rtic-examples\rtic_v6\stm32f4_pwm_monitor) Finished dev [unoptimized + debuginfo] target(s) in 0.45sThe MinGW version had been updated, does this issue still happen with recent nightly?
I'll check with the latest nightly and report back ASAP.
The MinGW version had been updated, does this issue still happen with recent nightly?
Hi, Sorry for not getting back to you sooner.
I confirm this doesn't happen with the nightly from 2023-06-17, installed via rustup. To verify I ran
rustup default 1.54.0and the problem occurred again. I again switched back to the new nightly withrustup default nightly-2023-06-17-x86_64-pc-windows-gnuand the problem doesn't occur again.Happy days!
Do we know exactly which nightly and/or or stable version this issue issue was fixed in so that others coming back to this issue can ensure they are on known-working fixed version?
ok, closing since it seems to be fixed now.
When building various example projects, I find that the rust-lld.exe linker will just stall instead of creating the .elf file. When it works, it takes about 0.4seconds, most of the time it fails and you have to ctrl+c the command and retry, it's obviously ridiculously frustrating, especially during a pair programming session.
To make sure there was nothing odd on my PC that might be affecting it, I did a complete fresh install of the OS, rust and the tools, yet the same thing still happens.
I was able to record detailed logs using ProcessMonitor of the failure. Attached to this issue are 3 files:
good.CSV, a log from when it works.good.CSV
The output was this:
bad.CSVand 3)bad-after-killing.CSV. A trace was started and saved asbad.CSVwhen rust-lld.exe hanged, then a new trace was started, ctrl+c was pressed and the new trace was saved asbad-after-killing.CSVso that you can see what happens before and after ctrl+c was pressed.bad.CSV
bad-after-killing.CSV
The output, when it fails is this:
I'm compiling for an STM32F446RE CPU.
I tried with
stableand withnightlyMeta
Project source: rtic-examples.zip
Nightly:
Stable:
Hardware:
OS:
If you need more information/logs/etc, just tell me what commands you want me to run and what output/files to provide.