Skip to content

Tracking issue: convert library-testing ui tests to unit tests #104676

Description

@RalfJung

We generally prefer library features to be tested as unit tests, not ui tests: this clearly separates things testing the compiler from things testing the library, and also ui tests are easier and faster to run and they are formatted, checked, and linted together with the rest of the library more easily. Also our unit tests are actually run by Miri, but ui tests are not (and it's much harder to do so).

AFAIK some existing tests have already been converted, but I don't know if this was ever done systematically.

  • Identify which parts of the ui test suite are really just testing the library
  • Convert them to regular unit tests if possible
    • it might not be possible for deliberately failing tests -- some of them might become compile_fail doc tests, but this does not always make sense
  • Properly document this policy and find some way to reduce the chance of new such tests being added

Cc @rust-lang/libs -- AFAIK this has already been policy for a while? @thomcc said there is no tracking issue though so here we go. :D

Activity

  1. added
    C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFC
    T-libsRelevant to the library team, which will review and decide on the PR/issue.
    on Nov 21, 2022
  2. the8472 commented on Nov 21, 2022

    @the8472
    Member

    it might not be possible for deliberately failing tests -- some of them might become compile_fail doc tests, but this does not always make sense

    Some tests also modify global state or spawn child processes. Since libtest uses threads we need some helper binaries for that that the test-suite could spawn. There was a seconded compiler MCP to add a test attribute for that but I don't think it was ever implemented

  3. cuviper commented on Nov 21, 2022

    @cuviper
    Member

    Some tests also modify global state or spawn child processes.

    Isn't that what library/*/tests/ is for?

  4. the8472 commented on Nov 21, 2022

    @the8472
    Member

    To spawn a child you still need a helper binary that can be spawned.

  5. thomcc commented on Nov 21, 2022

    @thomcc
    Member

    To be clear, converting these to be integration tests would be fine. The important thing is mostly that they use the libtest test runner. It's also fine if the conversion can't be done in all cases.

    At the moment, we have some of these as uitests because tier3 targets (even the ones that support libstd) without support in getrandom@0.1 cannot run the stdlib test suite. We probably have some for other reasons, some of which are better. The first limitation will be lifted as of #104658, so we should use the opportunity to go through and take a look at these, and move them where we can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-tracking-issueCategory: An issue tracking the progress of sth. like the implementation of an RFCT-libsRelevant to the library team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions