[ https://issues.apache.org/jira/browse/KAFKA-14682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Egerton reassigned KAFKA-14682: ------------------------------------- Assignee: Chris Egerton (was: Christo Lolov) > Unused stubbings are not reported by Mockito during CI builds > ------------------------------------------------------------- > > Key: KAFKA-14682 > URL: https://issues.apache.org/jira/browse/KAFKA-14682 > Project: Kafka > Issue Type: Test > Components: unit tests > Reporter: Chris Egerton > Assignee: Chris Egerton > Priority: Major > > We've started using [strict > stubbing|https://javadoc.io/static/org.mockito/mockito-core/4.6.1/org/mockito/junit/MockitoJUnitRunner.StrictStubs.html] > for unit tests written with Mockito, which is supposed to automatically fail > tests when they set up mock expectations that go unused. > However, these failures are not reported during Jenkins builds, even if they > are reported when building/testing locally. > In at least one case, this difference appears to be because our [Jenkins > build|https://github.com/apache/kafka/blob/6d11261d5deaca300e273bebe309f9e4f814f815/Jenkinsfile#L32-L35] > uses the custom {{unitTest}} and {{integrationTest}} tasks defined in the > project's [Gradle build > file|https://github.com/apache/kafka/blob/6d11261d5deaca300e273bebe309f9e4f814f815/build.gradle#L452-L543], > instead of the {{test}} task. Some IDEs (such as IntelliJ) may use the > latter instead of the former, which can cause tests to fail due to > unnecessary stubbings when being run in that IDE but not when being built on > Jenkins. > It's possible that, because the custom test tasks filter out some tests from > running, Mockito does not check for unnecessary stubbings in order to avoid > incorrectly failing tests that set up mocks in, e.g., a {{@BeforeEach}} > method. > > This exact behavior has been reported elsewhere as a [Gradle > issue|https://github.com/gradle/gradle/issues/10694]; based on [discussion on > that > thread|https://github.com/gradle/gradle/issues/10694#issuecomment-1374911274], > it appears this is a known and somewhat-intentional limitation of Mockito: > {quote}I spent some time trying to solve this and eventually I stumbled upon > this piece in Mockito's JUnit runner: > [https://github.com/mockito/mockito/blob/main/src/main/java/org/mockito/internal/runners/StrictRunner.java#L47-L53] > {{ }} > {{}} > {{ // only report when: > // 1. if all tests from given test have ran (filter requested is > false) > // Otherwise we would report unnecessary stubs even if the user > runs just single test > // from the class > // 2. tests are successful (we don't want to add an extra failure > on top of any existing > // failure, to avoid confusion)}} > > (1) suggests that skipping unused stub validation is the intended behavior > when the user filters a single test from the class. However, this behavior > applies to any type of filter. > And Gradle indeed applies a {{CategoryFilter}} if categories are configured: > [https://github.com/rieske/gradle/blob/e82029abb559d620fdfecb4708a95c6313af625c/subprojects/testing-jvm-infrastructure/src/main/java/org/gradle/api/internal/tasks/testing/junit/JUnitTestClassExecutor.java#L70-L96] > Which then causes Mockito to not validate unused stubs. > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010)