Skip to content

Add ainfer tests for a checker that declares @RelevantJavaTypes - #8176

Merged
smillst merged 3 commits into
typetools:masterfrom
mernst:ainfer-relevance-tests
Sep 16, 2026
Merged

smillst merged 3 commits into
typetools:masterfrom
mernst:ainfer-relevance-tests

Conversation

@mernst

@mernst mernst commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

Merge before #8177.

No existing whole-program-inference test uses a checker that declares @RelevantJavaTypes, so nothing tests the interaction of relevance with inference.

This PR adds that test infrastructure, with no change to main source code:

  • AinferRelevanceTestChecker uses the same type system and visitor as AinferTestChecker, but lists only CharSequence, int, and Map.Entry as relevant. So String is relevant through subtyping, and arrays are irrelevant because Object[].class is not listed.
  • The ainferRelevanceAjavaTest Gradle task, and the generation/validation test pair that it runs.
  • Test inputs covering relevant types, irrelevant types, and the several ways a program can name a type: a nested type, a type variable that shadows a class, a type in the same package that shadows one in java.lang, a lexically enclosing type, and a fully-qualified name.

The generation pass compares its output to .ajava.goal files. That comparison is what makes these tests useful: the validation pass alone cannot detect an annotation that inference writes on an irrelevant type, because such an annotation does not change the result of type-checking. Conversely, the validation pass is what detects an annotation that inference wrongly discards, since the error that the annotation suppresses reappears.

The goal files here record current behavior -- inference writes annotations even on irrelevant types. #8177 changes that behavior, and its diff to these goal files shows exactly which annotations stop being written.

🤖 Generated with Claude Code

No existing whole-program-inference test uses a checker that restricts the
Java types on which its qualifiers may be written, so nothing tests the
interaction of `@RelevantJavaTypes` with inference.

Add `AinferRelevanceTestChecker`, which uses the same type system and visitor
as `AinferTestChecker` but lists only `CharSequence`, `int`, and `Map.Entry`
as relevant.  Add the `ainferRelevanceAjavaTest` Gradle task and test inputs
that cover relevant types, irrelevant types, and the several ways a program
can refer to a type (a nested type, a type variable that shadows a class, a
type in the same package that shadows one in `java.lang`, and so on).

The generation pass compares its output to `.ajava.goal` files.  The
validation pass alone cannot detect an annotation that inference writes on an
irrelevant type, because such an annotation does not change the result of
type-checking.

The goal files record current behavior:  inference writes annotations even on
types that the checker treats as irrelevant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 17cf62d1-15f7-4595-a289-79cf17ec5895

📥 Commits

Reviewing files that changed from the base of the PR and between 1a4909e and 3e84900.

📒 Files selected for processing (28)
  • .gitignore
  • checker/build.gradle
  • checker/src/test/java/org/checkerframework/checker/test/junit/ainferrunners/AinferRelevanceAjavaGenerationTest.java
  • checker/src/test/java/org/checkerframework/checker/test/junit/ainferrunners/AinferRelevanceAjavaValidationTest.java
  • checker/src/test/java/org/checkerframework/checker/testchecker/ainfer/AinferRelevanceTestChecker.java
  • checker/src/test/java/org/checkerframework/checker/testchecker/ainfer/AinferTestAnnotatedTypeFactory.java
  • checker/tests/ainfer-relevance/IShouldBeSibling1.ajava.goal
  • checker/tests/ainfer-relevance/IrrelevantTypes.ajava.goal
  • checker/tests/ainfer-relevance/LexicallyNestedType.ajava.goal
  • checker/tests/ainfer-relevance/README
  • checker/tests/ainfer-relevance/RelevantNestedType.ajava.goal
  • checker/tests/ainfer-relevance/RelevantSubtype.ajava.goal
  • checker/tests/ainfer-relevance/RelevantTypeVariable.ajava.goal
  • checker/tests/ainfer-relevance/RelevantTypes.ajava.goal
  • checker/tests/ainfer-relevance/SamePackageShadows.ajava.goal
  • checker/tests/ainfer-relevance/TypeVariableShadowsClass.ajava.goal
  • checker/tests/ainfer-relevance/non-annotated/IShouldBeSibling1.java
  • checker/tests/ainfer-relevance/non-annotated/IrrelevantTypes.java
  • checker/tests/ainfer-relevance/non-annotated/LexicallyNestedType.java
  • checker/tests/ainfer-relevance/non-annotated/List.java
  • checker/tests/ainfer-relevance/non-annotated/Number.java
  • checker/tests/ainfer-relevance/non-annotated/RelevantNestedType.java
  • checker/tests/ainfer-relevance/non-annotated/RelevantSubtype.java
  • checker/tests/ainfer-relevance/non-annotated/RelevantTypeVariable.java
  • checker/tests/ainfer-relevance/non-annotated/RelevantTypes.java
  • checker/tests/ainfer-relevance/non-annotated/Runnable.java
  • checker/tests/ainfer-relevance/non-annotated/SamePackageShadows.java
  • checker/tests/ainfer-relevance/non-annotated/TypeVariableShadowsClass.java

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

Added a relevance inference test checker and fixtures for relevant types, irrelevant types, declaration annotations, nested types, and shadowed names. Added generation and validation JUnit runners that compare generated .ajava files with goals and validate annotated output. Added Gradle tasks, cleanup, aggregate test integration, and ignore rules for generated directories.

Priority: ⬇️ Low

Change: Other

Merge Risk: ⚪ Minimal · up to 3e849

The new relevance inference test pipeline is wired into the aggregate test task, with no actionable current-head risk identified.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.69% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 58 functions across 16 files. (12 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 20.69% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 58 functions across 16 files. (12 skipped: 12 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

mernst and others added 2 commits September 15, 2026 17:12
Goal files existed for only 2 of the 9 ajava files that inference generates
for the ainfer-relevance tests.  In particular, nothing observed the
annotations that inference writes on the irrelevant types in
`RelevantSubtype` (`@AinferTop char charAt(...)`) and in `RelevantTypes`
(the varargs `String @AinferTop ... args` that `RelevantTypes.java` claims
to test):  such an annotation does not change the result of type-checking,
so the validation pass cannot detect it either.

Add the 7 missing goal files, and require a goal file for every generated
ajava file and vice versa, so that no inference result goes unexamined.
Comment the two test inputs whose goal files now record an annotation on an
irrelevant type.

Also:

 * When the goal file or the ajava file is absent, do not advise copying a
   file that does not exist.
 * Find the generated ajava files by walking the output directory, and fail
   with an explanation if one is in a package subdirectory, where the flat
   goal-file naming scheme could never match it.  (Latent today:  every test
   input is in the unnamed package.)
 * Drop `build/ainfer-relevance` from the directories that
   `ainferRelevanceGenerateAjava` deletes; nothing creates it.
 * Say in `AinferRelevanceTestChecker`'s Javadoc that, unlike
   `AinferTestChecker`, it does not use the Value Checker as a subchecker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@smillst
smillst merged commit 60608a5 into typetools:master Sep 16, 2026
27 checks passed
@smillst
smillst deleted the ainfer-relevance-tests branch September 16, 2026 15:52
mernst added a commit to mernst/checker-framework that referenced this pull request Sep 16, 2026
…-insert-irrelevant-3

Upstream merged the ainfer-relevance tests (typetools#8172, typetools#8176) in their
pre-fix state, which records inference writing annotations on irrelevant
types.  Keep upstream's test infrastructure and take this branch's goal
files and comments, which record the fixed behavior.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants