Skip to content

tidy compiles very slowly #123002

Closed
#123339
Closed
tidy compiles very slowly#123002
#123339
Assignees
onur-ozkan
Labels
A-testsuiteArea: The testsuite used to check the correctness of rustcT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Description

@matthiaskrgr
matthiaskrgr
opened on Mar 24, 2024
  1. modify some code of tidy
  2. run ./x.py test
    =>
    for the first ~6 seconds, tidy is being built in parallel, but for the remaining ~1 minute and 9 seconds llvm is doing some sequential work.

I'm not sure what takes so long, there must be one epic cgu that it is processing or something?

Activity

  1. matthiaskrgr
    added
    C-bugCategory: This is a bug.
    on Mar 24, 2024
  2. rustbot
    added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Mar 24, 2024
  3. matthiaskrgr
    added
    T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
    and removed
    C-bugCategory: This is a bug.
    on Mar 24, 2024

  4. onur-ozkan commented on Apr 1, 2024

    @onur-ozkan
    onur-ozkan
    on Apr 1, 2024
    Contributor

    I wanted to check the problem but couldn't reproduce it locally.

  5. matthiaskrgr commented on Apr 1, 2024

    @matthiaskrgr
    matthiaskrgr
    on Apr 1, 2024
    MemberAuthor
       Compiling tidy v0.1.0 (/home/matthias/vcs/github/rust/src/tools/tidy)
time:   0.000; rss:   44MB ->   45MB (   +1MB)	parse_crate
time:   0.000; rss:   45MB ->   46MB (   +1MB)	setup_global_ctxt
time:   0.000; rss:   47MB ->   47MB (   +0MB)	crate_injection
time:   0.019; rss:   48MB ->   84MB (  +37MB)	expand_crate
time:   0.019; rss:   48MB ->   84MB (  +37MB)	macro_expand_crate
time:   0.000; rss:   84MB ->   84MB (   +0MB)	AST_validation
time:   0.000; rss:   85MB ->   85MB (   +0MB)	finalize_macro_resolutions
time:   0.004; rss:   85MB ->   88MB (   +4MB)	late_resolve_crate
time:   0.000; rss:   88MB ->   89MB (   +0MB)	resolve_check_unused
time:   0.000; rss:   89MB ->   89MB (   +0MB)	resolve_postprocess
time:   0.005; rss:   84MB ->   89MB (   +4MB)	resolve_crate
time:   0.001; rss:   96MB ->   96MB (   +0MB)	drop_ast
time:   0.011; rss:   89MB ->   96MB (   +7MB)	looking_for_derive_registrar
time:   0.013; rss:   89MB ->   97MB (   +9MB)	misc_checking_1
time:   0.023; rss:   97MB ->  129MB (  +32MB)	coherence_checking
time:   0.139; rss:   97MB ->  158MB (  +61MB)	type_check_crate
time:   0.143; rss:  158MB ->  201MB (  +43MB)	MIR_borrow_checking
time:   0.026; rss:  201MB ->  201MB (   +0MB)	MIR_effect_checking
time:   0.011; rss:  202MB ->  202MB (   +1MB)	module_lints
time:   0.011; rss:  202MB ->  202MB (   +1MB)	lint_checking
time:   0.004; rss:  202MB ->  202MB (   +0MB)	privacy_checking_modules
time:   0.016; rss:  201MB ->  202MB (   +1MB)	misc_checking_3
time:   0.091; rss:  202MB ->  214MB (  +12MB)	generate_crate_metadata
time:   0.084; rss:  214MB ->  220MB (   +5MB)	monomorphization_collector_graph_walk
time:   0.021; rss:  220MB ->  225MB (   +5MB)	partition_and_assert_distinct_symbols
time:   0.972; rss:  225MB ->  409MB ( +184MB)	codegen_to_LLVM_IR
time:   1.080; rss:  214MB ->  409MB ( +194MB)	codegen_crate
time:   0.016; rss:  409MB ->  355MB (  -54MB)	free_global_ctxt
time:  52.535; rss:  249MB ->  532MB ( +283MB)	LLVM_passes
time:   0.011; rss:  523MB ->  216MB ( -307MB)	join_worker_thread
time:  51.617; rss:  355MB ->  216MB ( -139MB)	finish_ongoing_codegen
time:   0.270; rss:  216MB ->  217MB (   +1MB)	link_rlib
time:   0.074; rss:  217MB ->  217MB (   +0MB)	link_binary_remove_temps
time:   0.344; rss:  216MB ->  217MB (   +1MB)	link_binary
time:   0.344; rss:  216MB ->  216MB (   +0MB)	link_crate
time:  51.961; rss:  355MB ->  216MB ( -139MB)	link
time:  53.517; rss:   32MB ->  186MB ( +154MB)	total
time:   0.000; rss:   44MB ->   45MB (   +1MB)	parse_crate
time:   0.000; rss:   46MB ->   46MB (   +1MB)	setup_global_ctxt
time:   0.000; rss:   47MB ->   48MB (   +0MB)	crate_injection
time:   0.009; rss:   48MB ->   76MB (  +28MB)	expand_crate
time:   0.000; rss:   76MB ->   76MB (   +0MB)	check_unused_macros
time:   0.009; rss:   48MB ->   76MB (  +28MB)	macro_expand_crate
time:   0.000; rss:   76MB ->   77MB (   +0MB)	AST_validation
time:   0.000; rss:   77MB ->   77MB (   +0MB)	finalize_imports
time:   0.002; rss:   77MB ->   79MB (   +2MB)	late_resolve_crate
time:   0.000; rss:   79MB ->   79MB (   +0MB)	resolve_check_unused
time:   0.000; rss:   79MB ->   80MB (   +0MB)	resolve_postprocess
time:   0.002; rss:   77MB ->   80MB (   +3MB)	resolve_crate
time:   0.000; rss:   80MB ->   80MB (   +0MB)	complete_gated_feature_checking
time:   0.001; rss:   80MB ->   82MB (   +2MB)	looking_for_entry_point
time:   0.002; rss:   80MB ->   83MB (   +3MB)	misc_checking_1
time:   0.001; rss:   83MB ->   87MB (   +4MB)	coherence_checking
time:   0.030; rss:   83MB ->  111MB (  +29MB)	type_check_crate
time:   0.023; rss:  111MB ->  123MB (  +12MB)	MIR_borrow_checking
time:   0.004; rss:  123MB ->  124MB (   +1MB)	MIR_effect_checking
time:   0.002; rss:  124MB ->  126MB (   +2MB)	module_lints
time:   0.003; rss:  124MB ->  126MB (   +2MB)	lint_checking
time:   0.003; rss:  124MB ->  126MB (   +2MB)	misc_checking_3
time:   0.061; rss:  126MB ->  153MB (  +27MB)	monomorphization_collector_graph_walk
time:   0.010; rss:  153MB ->  155MB (   +2MB)	partition_and_assert_distinct_symbols
time:   0.000; rss:  156MB ->  159MB (   +3MB)	write_allocator_module
time:   0.437; rss:  165MB ->  288MB ( +122MB)	codegen_to_LLVM_IR
time:   0.513; rss:  126MB ->  288MB ( +161MB)	codegen_crate
time:   0.012; rss:  288MB ->  238MB (  -50MB)	free_global_ctxt
time:   1.769; rss:  189MB ->  273MB (  +85MB)	LLVM_passes
time:   0.001; rss:  228MB ->  225MB (   -2MB)	join_worker_thread
time:   1.367; rss:  238MB ->  226MB (  -12MB)	finish_ongoing_codegen
time:   0.879; rss:  226MB ->  226MB (   +0MB)	run_linker
time:   0.884; rss:  226MB ->  226MB (   +1MB)	link_binary
time:   0.884; rss:  226MB ->  226MB (   +1MB)	link_crate
time:   2.251; rss:  238MB ->  226MB (  -12MB)	link
time:   2.859; rss:   32MB ->  152MB ( +120MB)	total
    Finished `release` profile [optimized + debuginfo] target(s) in 56.66s

  6. matthiaskrgr commented on Apr 1, 2024

    @matthiaskrgr
    matthiaskrgr
    on Apr 1, 2024
    MemberAuthor

    perf
    🤔

  7. matthiaskrgr commented on Apr 1, 2024

    @matthiaskrgr
    matthiaskrgr
    on Apr 1, 2024
    MemberAuthor

    aha lol!
    it boils down to issues.txt, an array of ~4K static &strs which is being inlined into the code it seems.
    When I cut the file down to just a handful of items, compilation time reduces to 7 seconds 🙃

    rust/src/tools/tidy/src/ui_tests.rs

    Line 108 in 3d5528c

    let issues_list =

    cc @workingjubilee I guess?

  8. onur-ozkan
    added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    and removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Apr 1, 2024

  9. onur-ozkan commented on Apr 1, 2024

    @onur-ozkan
    onur-ozkan
    on Apr 1, 2024
    Contributor

    Can you check if #123339 reduces this slowness?

  10. matthiaskrgr commented on Apr 1, 2024

    @matthiaskrgr
    matthiaskrgr
    on Apr 1, 2024
    MemberAuthor

    It does indeed.
    Build completed successfully in 0:00:06

  11. onur-ozkan
    self-assigned this
    on Apr 1, 2024

  12. workingjubilee commented on Apr 1, 2024

    @workingjubilee
    workingjubilee
    on Apr 1, 2024
    Member

    Ah, but if people give the tests meaningful names, it will reduce the build time. 😌

  13. matthiaskrgr
    added 3 commits that reference this issue on Apr 6, 2024

    Rollup merge of rust-lang#123339 - onur-ozkan:optimize-tidy-check, r=…

    534f20a

    Rollup merge of rust-lang#123339 - onur-ozkan:optimize-tidy-check, r=…

    620d1bc

    Rollup merge of rust-lang#123339 - onur-ozkan:optimize-tidy-check, r=…

    e27f5c4
  14. bors
    added a commit that references this issue on Apr 6, 2024

    Auto merge of rust-lang#123339 - onur-ozkan:optimize-tidy-check, r=Ma…

    773fb88
  15. bors
    closed this as completedin 773fb88on Apr 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions