Hey all, MirrorConnectorsIntegration*Test#testOffsetTranslationBehindReplicationFlow() flakiness should be addressed by https://github.com/apache/kafka/pull/14156 which is currently in review. There is other flakiness in the suite which this PR does not address and needs further investigation.
The MirrorConnectorsIntegration*Test suites certainly need some optimization. I've noticed that even in local test environments the tear-down can be quite lengthy, and we haven't figured out the cause yet. Each suite has >=11 tests, and there's 6 suites each contributing 15-20m to the test execution time. Thanks, Greg On Tue, Aug 8, 2023 at 3:23 PM Kirk True <k...@kirktrue.pro> wrote: > > Here are some useful links for filtering failing tests by branches: > > All branches: > https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.relativeStartTime=P28D > > trunk-only: > https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.relativeStartTime=P28D&search.names=Git%20branch&search.values=trunk > > All branches except trunk: > https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.relativeStartTime=P28D&search.names=not:Git%20branch&search.values=trunk > > Thanks, > Kirk > > > On Aug 8, 2023, at 3:18 PM, Justine Olshan <jols...@confluent.io.INVALID> > > wrote: > > > > Thanks Kirk! > > I will try to go through in the next day or so and see if there is any > > tests I can fix. > > > > On Tue, Aug 8, 2023 at 3:13 PM Kirk True <k...@kirktrue.pro> wrote: > > > >> Hi Justine, > >> > >>> On Aug 1, 2023, at 4:50 PM, Justine Olshan <jols...@confluent.io.INVALID> > >> wrote: > >>> > >>> Is that right that the first one on the list ( > >>> > >> org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest) > >>> takes > >>> 20 minutes?! That's quite a test. > >>> I wonder if the length corresponds to whether it passes, but we should > >> fix > >>> it and maybe move it out of our PR builds. > >> > >> It certainly does seems like it is taking that long PLUS any retries since > >> it’s flaky. > >> > >>> I was also wondering if we could distinguish PR builds from trunk builds. > >>> That might give us a better signal since PR builds could be before tests > >>> are fixed. Not sure which one is being reported here. > >> > >> It includes both trunk and PR branch test runs. I’ll see how easy it is to > >> filter them out. > >> > >> Thanks! > >> > >>> > >>> Thanks for sharing though! This is a useful tool that we've needed for a > >>> while. > >>> > >>> Justine > >>> > >>> On Tue, Aug 1, 2023 at 4:38 PM Kirk True <k...@kirktrue.pro> wrote: > >>> > >>>> Hi! > >>>> > >>>> According to the Gradle Enterprise statistics on our recent Kafka > >> builds, > >>>> over 90% have flaky tests [1]. > >>>> > >>>> We also have 106 open Jiras with the “flaky-test” label across several > >>>> functional areas of the project [2]. > >>>> > >>>> Can I ask that those familiar with those different functional areas > >> take a > >>>> look at the list of flaky tests and triage them? > >>>> > >>>> Thanks, > >>>> Kirk > >>>> > >>>> [1] > >>>> > >> https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=kafka > >>>> [2] > >>>> > >> https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20labels%20%3D%20flaky-test > >> > >> >
