Hey Stan, Just wanted to clarify -- KAFKA-14616 is not particularly related to JBOD but to ZK -> KRaft migration.
There were some other related migration bugs like https://issues.apache.org/jira/browse/KAFKA-16180. This may or may not influence decisions, but wanted to paint the full picture of the blocker bugs and their causes. Justine On Tue, Jan 23, 2024 at 12:17 PM Stanislav Kozlovski <stanis...@confluent.io.invalid> wrote: > Hey all, I figured I'd give an update about what known blockers we have > right now: > > - KAFKA-16101: KRaft migration rollback documentation is incorrect - > https://github.com/apache/kafka/pull/15193; This need not block RC > creation, but we need the docs updated so that people can test properly > - KAFKA-14616: Topic recreation with offline broker causes permanent URPs - > https://github.com/apache/kafka/pull/15230 ; I am of the understanding > that > this is blocking JBOD for 3.7 > - KAFKA-16162: New created topics are unavailable after upgrading to 3.7 - > a strict blocker with an open PR > https://github.com/apache/kafka/pull/15232 > - although I understand Proveen is out of office > - KAFKA-16082: JBOD: Possible dataloss when moving leader partition - I am > hearing mixed opinions on whether this is a blocker ( > https://github.com/apache/kafka/pull/15136) > > Given that there are 3 JBOD blocker bugs, and I am not confident they will > all be merged this week - I am on the edge of voting to revert JBOD from > this release, or mark it early access. > > By all accounts, it seems that if we keep with JBOD the release will have > to spill into February, which is a month extra from the time-based release > plan we had of start of January. > > Can I ask others for an opinion? > > Best, > Stan > > On Thu, Jan 18, 2024 at 1:21 PM Luke Chen <show...@gmail.com> wrote: > > > Hi all, > > > > I think I've found another blocker issue: KAFKA-16162 > > <https://issues.apache.org/jira/browse/KAFKA-16162> . > > The impact is after upgrading to 3.7.0, any new created topics/partitions > > will be unavailable. > > I've put my findings in the JIRA. > > > > Thanks. > > Luke > > > > On Thu, Jan 18, 2024 at 9:50 AM Matthias J. Sax <mj...@apache.org> > wrote: > > > > > Stan, thanks for driving this all forward! Excellent job. > > > > > > About > > > > > > > StreamsStandbyTask - > https://issues.apache.org/jira/browse/KAFKA-16141 > > > > StreamsUpgradeTest - > https://issues.apache.org/jira/browse/KAFKA-16139 > > > > > > For `StreamsUpgradeTest` it was a test setup issue and should be fixed > > > now in trunk and 3.7 (and actually also in 3.6...) > > > > > > For `StreamsStandbyTask` the failing test exposes a regression bug, so > > > it's a blocker. I updated the ticket accordingly. We already have an > > > open PR that reverts the code introducing the regression. > > > > > > > > > -Matthias > > > > > > On 1/17/24 9:44 AM, Proven Provenzano wrote: > > > > We have another blocking issue for the RC : > > > > https://issues.apache.org/jira/browse/KAFKA-16157. This bug is > similar > > > to > > > > https://issues.apache.org/jira/browse/KAFKA-14616. The new issue > > however > > > > can lead to the new topic having partitions that a producer cannot > > write > > > to. > > > > > > > > --Proven > > > > > > > > On Tue, Jan 16, 2024 at 12:04 PM Proven Provenzano < > > > pprovenz...@confluent.io> > > > > wrote: > > > > > > > >> > > > >> I have a PR https://github.com/apache/kafka/pull/15197 for > > > >> https://issues.apache.org/jira/browse/KAFKA-16131 that is building > > now. > > > >> --Proven > > > >> > > > >> On Mon, Jan 15, 2024 at 5:03 AM Jakub Scholz <ja...@scholz.cz> > wrote: > > > >> > > > >>> *> Hi Jakub,> > Thanks for trying the RC. I think what you found > is a > > > >>> blocker bug because it * > > > >>> *> will generate huge amount of logspam. I guess we didn't find it > in > > > >>> junit > > > >>> tests * > > > >>> *> since logspam doesn't fail the automated tests. But certainly > it's > > > not > > > >>> suitable * > > > >>> *> for production. Did you file a JIRA yet?* > > > >>> > > > >>> Hi Colin, > > > >>> > > > >>> I opened https://issues.apache.org/jira/browse/KAFKA-16131. > > > >>> > > > >>> Thanks & Regards > > > >>> Jakub > > > >>> > > > >>> On Mon, Jan 15, 2024 at 8:57 AM Colin McCabe <cmcc...@apache.org> > > > wrote: > > > >>> > > > >>>> Hi Stanislav, > > > >>>> > > > >>>> Thanks for making the first RC. The fact that it's titled RC2 is > > > messing > > > >>>> with my mind a bit. I hope this doesn't make people think that > we're > > > >>>> farther along than we are, heh. > > > >>>> > > > >>>> On Sun, Jan 14, 2024, at 13:54, Jakub Scholz wrote: > > > >>>>> *> Nice catch! It does seem like we should have gated this behind > > the > > > >>>>> metadata> version as KIP-858 implies. Is the cluster configured > > with > > > >>>>> multiple log> dirs? What is the impact of the error messages?* > > > >>>>> > > > >>>>> I did not observe any obvious impact. I was able to send and > > receive > > > >>>>> messages as normally. But to be honest, I have no idea what else > > > >>>>> this might impact, so I did not try anything special. > > > >>>>> > > > >>>>> I think everyone upgrading an existing KRaft cluster will go > > through > > > >>> this > > > >>>>> stage (running Kafka 3.7 with an older metadata version for at > > least > > > a > > > >>>>> while). So even if it is just a logged exception without any > other > > > >>>> impact I > > > >>>>> wonder if it might scare users from upgrading. But I leave it to > > > >>> others > > > >>>> to > > > >>>>> decide if this is a blocker or not. > > > >>>>> > > > >>>> > > > >>>> Hi Jakub, > > > >>>> > > > >>>> Thanks for trying the RC. I think what you found is a blocker bug > > > >>> because > > > >>>> it will generate huge amount of logspam. I guess we didn't find it > > in > > > >>> junit > > > >>>> tests since logspam doesn't fail the automated tests. But > certainly > > > it's > > > >>>> not suitable for production. Did you file a JIRA yet? > > > >>>> > > > >>>>> On Sun, Jan 14, 2024 at 10:17 PM Stanislav Kozlovski > > > >>>>> <stanis...@confluent.io.invalid> wrote: > > > >>>>> > > > >>>>>> Hey Luke, > > > >>>>>> > > > >>>>>> This is an interesting problem. Given the fact that the KIP for > > > >>> having a > > > >>>>>> 3.8 release passed, I think it weights the scale towards not > > calling > > > >>>> this a > > > >>>>>> blocker and expecting it to be solved in 3.7.1. > > > >>>>>> > > > >>>>>> It is unfortunate that it would not seem safe to migrate to > KRaft > > in > > > >>>> 3.7.0 > > > >>>>>> (given the inability to rollback safely), but if that's true - > the > > > >>> same > > > >>>>>> case would apply for 3.6.0. So in any case users w\ould be > > expected > > > >>> to > > > >>>> use a > > > >>>>>> patch release for this. > > > >>>> > > > >>>> Hi Luke, > > > >>>> > > > >>>> Thanks for testing rollback. I think this is a case where the > > > >>>> documentation is wrong. The intention was to for the steps to > > > basically > > > >>> be: > > > >>>> > > > >>>> 1. roll all the brokers into zk mode, but with migration enabled > > > >>>> 2. take down the kraft quorum > > > >>>> 3. rmr /controller, allowing a hybrid broker to take over. > > > >>>> 4. roll all the brokers into zk mode without migration enabled (if > > > >>> desired) > > > >>>> > > > >>>> With these steps, there isn't really unavailability since a ZK > > > >>> controller > > > >>>> can be elected quickly after the kraft quorum is gone. > > > >>>> > > > >>>>>> Further, since we will have a 3.8 release - it is > > > >>>>>> likely we will ultimately recommend users upgrade from that > > version > > > >>>> given > > > >>>>>> its aim is to have strategic KRaft feature parity with ZK. > > > >>>>>> That being said, I am not 100% on this. Let me know whether you > > > think > > > >>>> this > > > >>>>>> should block the release, Luke. I am also tagging Colin and > David > > to > > > >>>> weigh > > > >>>>>> in with their opinions, as they worked on the migration logic. > > > >>>> > > > >>>> The rollback docs are new in 3.7 so the fact that they're wrong > is a > > > >>> clear > > > >>>> blocker, I think. But easy to fix, I believe. I will create a PR. > > > >>>> > > > >>>> best, > > > >>>> Colin > > > >>>> > > > >>>>>> > > > >>>>>> Hey Kirk and Chris, > > > >>>>>> > > > >>>>>> Unless I'm missing something - KAFKALESS-16029 is simply a bad > log > > > >>> due > > > >>>> to > > > >>>>>> improper closing. And the PR description implies this has been > > > >>> present > > > >>>>>> since 3.5. While annoying, I don't see a strong reason for this > to > > > >>> block > > > >>>>>> the release. > > > >>>>>> > > > >>>>>> Hey Jakub, > > > >>>>>> > > > >>>>>> Nice catch! It does seem like we should have gated this behind > the > > > >>>> metadata > > > >>>>>> version as KIP-858 implies. Is the cluster configured with > > multiple > > > >>> log > > > >>>>>> dirs? What is the impact of the error messages? > > > >>>>>> > > > >>>>>> Tagging Igor (the author of the KIP) to weigh in. > > > >>>>>> > > > >>>>>> Best, > > > >>>>>> Stanislav > > > >>>>>> > > > >>>>>> On Sat, Jan 13, 2024 at 7:22 PM Jakub Scholz <ja...@scholz.cz> > > > >>> wrote: > > > >>>>>> > > > >>>>>>> Hi, > > > >>>>>>> > > > >>>>>>> I was trying the RC2 and run into the following issue ... when > I > > > >>> run > > > >>>>>>> 3.7.0-RC2 KRaft cluster with metadata version set to 3.6-IV2 > > > >>> metadata > > > >>>>>>> version, I seem to be getting repeated errors like this in the > > > >>>> controller > > > >>>>>>> logs: > > > >>>>>>> > > > >>>>>>> 2024-01-13 16:58:01,197 INFO [QuorumController id=0] > > > >>>>>> assignReplicasToDirs: > > > >>>>>>> event failed with UnsupportedVersionException in 15 > microseconds. > > > >>>>>>> (org.apache.kafka.controller.QuorumController) > > > >>>>>>> [quorum-controller-0-event-handler] > > > >>>>>>> 2024-01-13 16:58:01,197 ERROR [ControllerApis nodeId=0] > > Unexpected > > > >>>> error > > > >>>>>>> handling request RequestHeader(apiKey=ASSIGN_REPLICAS_TO_DIRS, > > > >>>>>>> apiVersion=0, clientId=1000, correlationId=14, headerVersion=2) > > -- > > > >>>>>>> AssignReplicasToDirsRequestData(brokerId=1000, brokerEpoch=5, > > > >>>>>>> directories=[DirectoryData(id=w_uxN7pwQ6eXSMrOKceYIQ, > > > >>>>>>> topics=[TopicData(topicId=bvAKLSwmR7iJoKv2yZgygQ, > > > >>>>>>> partitions=[PartitionData(partitionIndex=2), > > > >>>>>>> PartitionData(partitionIndex=1)]), > > > >>>>>>> TopicData(topicId=uNe7f5VrQgO0zST6yH1jDQ, > > > >>>>>>> partitions=[PartitionData(partitionIndex=0)])])]) with context > > > >>>>>>> > > RequestContext(header=RequestHeader(apiKey=ASSIGN_REPLICAS_TO_DIRS, > > > >>>>>>> apiVersion=0, clientId=1000, correlationId=14, > headerVersion=2), > > > >>>>>>> connectionId='172.16.14.219:9090-172.16.14.217:53590-7', > > > >>>> clientAddress=/ > > > >>>>>>> 172.16.14.217, principal=User:CN=my-cluster-kafka,O=io.strimzi, > > > >>>>>>> listenerName=ListenerName(CONTROLPLANE-9090), > > securityProtocol=SSL, > > > >>>>>>> > > clientInformation=ClientInformation(softwareName=apache-kafka-java, > > > >>>>>>> softwareVersion=3.7.0), fromPrivilegedListener=false, > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > principalSerde=Optional[org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder@71004ad2 > > > >>>>>>> ]) > > > >>>>>>> (kafka.server.ControllerApis) > [quorum-controller-0-event-handler] > > > >>>>>>> java.util.concurrent.CompletionException: > > > >>>>>>> org.apache.kafka.common.errors.UnsupportedVersionException: > > > >>> Directory > > > >>>>>>> assignment is not supported yet. > > > >>>>>>> > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:636) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.controller.QuorumController$ControllerWriteEvent.complete(QuorumController.java:880) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.controller.QuorumController$ControllerWriteEvent.handleException(QuorumController.java:871) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.queue.KafkaEventQueue$EventContext.completeWithException(KafkaEventQueue.java:148) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:137) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:210) > > > >>>>>>> at > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:181) > > > >>>>>>> at java.base/java.lang.Thread.run(Thread.java:840) > > > >>>>>>> > > > >>>>>>> Caused by: > > > >>> org.apache.kafka.common.errors.UnsupportedVersionException: > > > >>>>>>> Directory assignment is not supported yet. > > > >>>>>>> > > > >>>>>>> Is that expected? I guess with the metadata version set to > > > >>> 3.6-IV2, it > > > >>>>>>> makes sense that the request is not supported. But shouldn't > then > > > >>> the > > > >>>>>>> request not be sent at all by the brokers? (I did not opened a > > JIRA > > > >>>> for > > > >>>>>> it, > > > >>>>>>> but I can open one if you agree this is not expected) > > > >>>>>>> > > > >>>>>>> Thanks & Regards > > > >>>>>>> Jakub > > > >>>>>>> > > > >>>>>>> On Sat, Jan 13, 2024 at 8:03 AM Luke Chen <show...@gmail.com> > > > >>> wrote: > > > >>>>>>> > > > >>>>>>>> Hi Stanislav, > > > >>>>>>>> > > > >>>>>>>> I commented in the "Apache Kafka 3.7.0 Release" thread, but > > maybe > > > >>>> you > > > >>>>>>>> missed it. > > > >>>>>>>> cross-posting here: > > > >>>>>>>> > > > >>>>>>>> There is a bug KAFKA-16101 > > > >>>>>>>> <https://issues.apache.org/jira/browse/KAFKA-16101> reporting > > > >>> that > > > >>>>>>> "Kafka > > > >>>>>>>> cluster will be unavailable during KRaft migration rollback". > > > >>>>>>>> The impact for this issue is that if brokers try to rollback > to > > > >>> ZK > > > >>>> mode > > > >>>>>>>> during KRaft migration process, there will be a period of time > > > >>> the > > > >>>>>>> cluster > > > >>>>>>>> is unavailable. > > > >>>>>>>> Since ZK migrating to KRaft feature is a production ready > > > >>> feature, I > > > >>>>>>> think > > > >>>>>>>> this should be addressed soon. > > > >>>>>>>> Do you think this is a blocker for v3.7.0? > > > >>>>>>>> > > > >>>>>>>> Thanks. > > > >>>>>>>> Luke > > > >>>>>>>> > > > >>>>>>>> On Sat, Jan 13, 2024 at 8:36 AM Chris Egerton < > > > >>>> fearthecel...@gmail.com > > > >>>>>>> > > > >>>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> Thanks, Kirk! > > > >>>>>>>>> > > > >>>>>>>>> @Stanislav--do you believe that this warrants a new RC? > > > >>>>>>>>> > > > >>>>>>>>> On Fri, Jan 12, 2024, 19:08 Kirk True <k...@kirktrue.pro> > > > >>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> Hi Chris/Stanislav, > > > >>>>>>>>>> > > > >>>>>>>>>> I'm working on the 'Unable to find FetchSessionHandler' log > > > >>>> problem > > > >>>>>>>>>> (KAFKA-16029) and have put out a draft PR ( > > > >>>>>>>>>> https://github.com/apache/kafka/pull/15186). I will use the > > > >>>>>>> quickstart > > > >>>>>>>>>> approach as a second means to reproduce/verify while I wait > > > >>> for > > > >>>> the > > > >>>>>>>> PR's > > > >>>>>>>>>> Jenkins job to finish. > > > >>>>>>>>>> > > > >>>>>>>>>> Thanks, > > > >>>>>>>>>> Kirk > > > >>>>>>>>>> > > > >>>>>>>>>> On Fri, Jan 12, 2024, at 11:31 AM, Chris Egerton wrote: > > > >>>>>>>>>>> Hi Stanislav, > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thanks for running this release! > > > >>>>>>>>>>> > > > >>>>>>>>>>> To verify, I: > > > >>>>>>>>>>> - Built from source using Java 11 with both: > > > >>>>>>>>>>> - - the 3.7.0-rc2 tag on GitHub > > > >>>>>>>>>>> - - the kafka-3.7.0-src.tgz artifact from > > > >>>>>>>>>>> > > > >>> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/ > > > >>>>>>>>>>> - Checked signatures and checksums > > > >>>>>>>>>>> - Ran the quickstart using both: > > > >>>>>>>>>>> - - The kafka_2.13-3.7.0.tgz artifact from > > > >>>>>>>>>>> > > > >>> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/ > > > >>>>>> with > > > >>>>>>>> Java > > > >>>>>>>>>> 11 > > > >>>>>>>>>>> and Scala 13 in KRaft mode > > > >>>>>>>>>>> - - Our shiny new broker Docker image, > > > >>> apache/kafka:3.7.0-rc2 > > > >>>>>>>>>>> - Ran all unit tests > > > >>>>>>>>>>> - Ran all integration tests for Connect and MM2 > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> I found two minor areas for concern: > > > >>>>>>>>>>> > > > >>>>>>>>>>> 1. (Possibly a blocker) > > > >>>>>>>>>>> When running the quickstart, I noticed this ERROR-level log > > > >>>>>> message > > > >>>>>>>>> being > > > >>>>>>>>>>> emitted frequently (not not every time) when I killed my > > > >>>> console > > > >>>>>>>>> consumer > > > >>>>>>>>>>> via ctrl-C: > > > >>>>>>>>>>> > > > >>>>>>>>>>>> [2024-01-12 11:00:31,088] ERROR [Consumer > > > >>>>>>>> clientId=console-consumer, > > > >>>>>>>>>>> groupId=console-consumer-74388] Unable to find > > > >>>>>> FetchSessionHandler > > > >>>>>>>> for > > > >>>>>>>>>> node > > > >>>>>>>>>>> 1. Ignoring fetch response > > > >>>>>>>>>>> (org.apache.kafka.clients.consumer.internals.AbstractFetch) > > > >>>>>>>>>>> > > > >>>>>>>>>>> I see that this error message is already reported in > > > >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-16029. I > > > >>> think we > > > >>>>>>> should > > > >>>>>>>>>>> prioritize fixing it for this release. I know it's probably > > > >>>>>> benign > > > >>>>>>>> but > > > >>>>>>>>>> it's > > > >>>>>>>>>>> really not a good look for us when basic operations log > > > >>> error > > > >>>>>>>> messages, > > > >>>>>>>>>> and > > > >>>>>>>>>>> it may give new users some headaches. > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> 2. (Probably not a blocker) > > > >>>>>>>>>>> The following unit tests failed the first time around, and > > > >>>> all of > > > >>>>>>>> them > > > >>>>>>>>>>> passed the second time I ran them: > > > >>>>>>>>>>> > > > >>>>>>>>>>> - (clients) > > > >>>>>>>>>> > > > >>> ClientUtilsTest.testParseAndValidateAddressesWithReverseLookup() > > > >>>>>>>>>>> - (clients) SelectorTest.testConnectionsByClientMetric() > > > >>>>>>>>>>> - (clients) > > > >>> Tls13SelectorTest.testConnectionsByClientMetric() > > > >>>>>>>>>>> - (connect) > > > >>>>>>>> TopicAdminTest.retryEndOffsetsShouldRetryWhenTopicNotFound > > > >>>>>>>>> (I > > > >>>>>>>>>>> thought I fixed this one! 🤬🤬) > > > >>>>>>>>>>> - (core) > > > >>>> ProducerIdManagerTest.testUnrecoverableErrors(Errors)[2] > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> Thanks again for your work on this release, and > > > >>>> congratulations > > > >>>>>> to > > > >>>>>>>>> Kafka > > > >>>>>>>>>>> Streams for having zero flaky unit tests during my > > > >>>>>>>> highly-experimental > > > >>>>>>>>>>> single laptop run! > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> Cheers, > > > >>>>>>>>>>> > > > >>>>>>>>>>> Chris > > > >>>>>>>>>>> > > > >>>>>>>>>>> On Thu, Jan 11, 2024 at 1:33 PM Stanislav Kozlovski > > > >>>>>>>>>>> <stanis...@confluent.io.invalid> wrote: > > > >>>>>>>>>>> > > > >>>>>>>>>>>> Hello Kafka users, developers, and client-developers, > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> This is the first candidate for release of Apache Kafka > > > >>>> 3.7.0. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Note it's named "RC2" because I had a few "failed" RCs > > > >>> that > > > >>>> I > > > >>>>>> had > > > >>>>>>>>>>>> cut/uploaded but ultimately had to scrap prior to > > > >>> announcing > > > >>>>>> due > > > >>>>>>> to > > > >>>>>>>>> new > > > >>>>>>>>>>>> blockers arriving before I could even announce them. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Further - I haven't yet been able to set up the system > > > >>> tests > > > >>>>>>>>>> successfully. > > > >>>>>>>>>>>> And the integration/unit tests do have a few failures > > > >>> that I > > > >>>>>> have > > > >>>>>>>> to > > > >>>>>>>>>> spend > > > >>>>>>>>>>>> time triaging. I would appreciate any help in case anyone > > > >>>>>> notices > > > >>>>>>>> any > > > >>>>>>>>>> tests > > > >>>>>>>>>>>> failing that they're subject matters experts in. Expect > > > >>> me > > > >>>> to > > > >>>>>>>> follow > > > >>>>>>>>>> up in > > > >>>>>>>>>>>> a day or two with more detailed analysis. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Major changes include: > > > >>>>>>>>>>>> - Early Access to KIP-848 - the next generation of the > > > >>>> consumer > > > >>>>>>>>>> rebalance > > > >>>>>>>>>>>> protocol > > > >>>>>>>>>>>> - KIP-858: Adding JBOD support to KRaft > > > >>>>>>>>>>>> - KIP-714: Observability into Client metrics via a > > > >>>> standardized > > > >>>>>>>>>> interface > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Check more information in the WIP blog post: > > > >>>>>>>>>>>> https://github.com/apache/kafka-site/pull/578 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Release notes for the 3.7.0 release: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/RELEASE_NOTES.html > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> *** Please download, test and vote by Thursday, January > > > >>> 18, > > > >>>> 9am > > > >>>>>>> PT > > > >>>>>>>>> *** > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Usually these deadlines tend to be 2-3 days, but due to > > > >>> this > > > >>>>>>> being > > > >>>>>>>>> the > > > >>>>>>>>>>>> first RC and the tests not having ran yet, I am giving > > > >>> it a > > > >>>> bit > > > >>>>>>>> more > > > >>>>>>>>>> time. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Kafka's KEYS file containing PGP keys we use to sign the > > > >>>>>> release: > > > >>>>>>>>>>>> https://kafka.apache.org/KEYS > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Release artifacts to be voted upon (source and binary): > > > >>>>>>>>>>>> > > > >>>> https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/ > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Docker release artifact to be voted upon: > > > >>>>>>>>>>>> apache/kafka:3.7.0-rc2 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Maven artifacts to be voted upon: > > > >>>>>>>>>>>> > > > >>>>>>>>> > > > >>>>>> > > > >>> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Javadoc: > > > >>>>>>>>>>>> > > > >>>>>>>> > > > >>>> > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/javadoc/ > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag: > > > >>>>>>>>>>>> https://github.com/apache/kafka/releases/tag/3.7.0-rc2 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Documentation: > > > >>>>>>>>>>>> https://kafka.apache.org/37/documentation.html > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Protocol: > > > >>>>>>>>>>>> https://kafka.apache.org/37/protocol.html > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Successful Jenkins builds for the 3.7 branch: > > > >>>>>>>>>>>> Unit/integration tests: > > > >>>>>>>>>>>> > > > >>>> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.7/58/ > > > >>>>>>>>>>>> There are failing tests here. I have to follow up with > > > >>>> triaging > > > >>>>>>>> some > > > >>>>>>>>> of > > > >>>>>>>>>>>> the failures and figuring out if they're actual problems > > > >>> or > > > >>>>>>> simply > > > >>>>>>>>>> flakes. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> System tests: > > > >>>>>>>>>> https://jenkins.confluent.io/job/system-test-kafka/job/3.7/ > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> No successful system test runs yet. I am working on > > > >>> getting > > > >>>> the > > > >>>>>>> job > > > >>>>>>>>> to > > > >>>>>>>>>> run. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> * Successful Docker Image Github Actions Pipeline for 3.7 > > > >>>>>> branch: > > > >>>>>>>>>>>> Attached are the scan_report and report_jvm output files > > > >>>> from > > > >>>>>> the > > > >>>>>>>>>> Docker > > > >>>>>>>>>>>> Build run: > > > >>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>> > > > >>>>>> > > > >>> > > > > https://github.com/apache/kafka/actions/runs/7486094960/job/20375761673 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> And the final docker image build job - Docker Build Test > > > >>>>>>> Pipeline: > > > >>>>>>>>>>>> https://github.com/apache/kafka/actions/runs/7486178277 > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> The image is apache/kafka:3.7.0-rc2 - > > > >>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > > > > https://hub.docker.com/layers/apache/kafka/3.7.0-rc2/images/sha256-5b4707c08170d39549fbb6e2a3dbb83936a50f987c0c097f23cb26b4c210c226?context=explore > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> /************************************** > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> Thanks, > > > >>>>>>>>>>>> Stanislav Kozlovski > > > >>>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> -- > > > >>>>>> Best, > > > >>>>>> Stanislav > > > >>>>>> > > > >>>> > > > >>> > > > >> > > > > > > > > > > > > -- > Best, > Stanislav >
