Thanks for jumping on this guys, and nice find Haruki I agree with Luke that we should disable the offending tests ASAP to unblock other things, and file a 3.8 blocker to investigate this further. Thanks for the PR Haruki and thanks for filing the ticket Luke -- I marked it as a blocker for 3.8.0 so this doesn't slip.
By the way -- someone else was looking into this on the side and mentioned another test they were suspicious of as well. It's SaslClientsWithInvalidCredentialsTest.testKafkaAdminClientWithAuthenticationFailure Does one of you have time to look into this test as well? Don't want to have to wait another 8 hours for your PR to hit the timeout and abort again. If not, I can spend a minute looking at this other test a bit later tonight On Fri, Jun 7, 2024 at 6:05 PM Haruki Okada <ocadar...@gmail.com> wrote: > Opened the PR h(ttps://github.com/apache/kafka/pull/16248) > Let's see if CI runs properly. > > Thanks, > > 2024年6月8日(土) 10:01 Luke Chen <show...@gmail.com>: > > > > Let's disable for now to unblock builds, and revert later if we can't > > solve > > it until code freeze? > > > > That's exactly what I meant to do. > > I've opened KAFKA-16916 < > https://issues.apache.org/jira/browse/KAFKA-16916 > > > > > for this issue and assigned to you. > > Welcome to unassign yourselves if you don't have time to fix the > > adminClient behavior change issue. > > But, let's disable it first. > > > > Thanks. > > Luke > > > > > > On Sat, Jun 8, 2024 at 8:55 AM Haruki Okada <ocadar...@gmail.com> wrote: > > > > > Hi Luke, > > > > > > I see, but since this is likely due to AdminClient's behavior change, > we > > > need to fix it anyways not only disabling test before 3.8 release. > > > Let's disable for now to unblock builds, and revert later if we can't > > solve > > > it until code freeze? > > > > > > 2024年6月8日(土) 9:31 Luke Chen <show...@gmail.com>: > > > > > > > Hi Haruki, > > > > > > > > Thanks for identifying this blocking test. > > > > Could you help quickly open a PR to disable this test to unblock the > CI > > > > build? > > > > > > > > Thanks. > > > > Luke > > > > > > > > On Sat, Jun 8, 2024 at 8:20 AM Haruki Okada <ocadar...@gmail.com> > > wrote: > > > > > > > > > Hi > > > > > > > > > > I found that the hanging can be reproduced locally. > > > > > The blocking test is > > > > > > > > > > > > > > > > > > > > "org.apache.kafka.common.security.authenticator.ClientAuthenticationFailureTest.testAdminClientWithInvalidCredentials". > > > > > It started to block after this commit ( > > > > > > > > > > > > > > > > > > > > https://github.com/apache/kafka/commit/c01279b92acefd9135089588319910bac79bfd4c > > > > > ) > > > > > > > > > > Thanks, > > > > > > > > > > 2024年6月8日(土) 8:30 Sophie Blee-Goldman <sop...@responsive.dev>: > > > > > > > > > > > Seems like the build is currently broken -- specifically, a test > is > > > > > hanging > > > > > > and causing it to abort after 7+ hours. There are many examples > in > > > the > > > > > > current PRs, such as > > > > > > > > > > > > Timed out after almost 8 hours: > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-16238/1/pipeline/ > > > > > > 2. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-16201/15/pipeline > > > > > > > > > > > > Still running after 6+ hours: > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-16236/3/pipeline/ > > > > > > > > > > > > It's pretty difficult to tell which test is hanging but it seems > > like > > > > one > > > > > > of the commits in the last 1-2 days is the likely culprit. If > > anyone > > > > has > > > > > an > > > > > > idea of what may have caused this or is actively investigating, > > > please > > > > > let > > > > > > everyone know. > > > > > > > > > > > > Needless to say, this is rather urgent given the upcoming 3.8 > code > > > > > freeze. > > > > > > > > > > > > Thanks, > > > > > > Sophie > > > > > > > > > > > > > > > > > > > > > -- > > > > > ======================== > > > > > Okada Haruki > > > > > ocadar...@gmail.com > > > > > ======================== > > > > > > > > > > > > > > > > > > -- > > > ======================== > > > Okada Haruki > > > ocadar...@gmail.com > > > ======================== > > > > > > > > -- > ======================== > Okada Haruki > ocadar...@gmail.com > ======================== >
