Re: Kafka Streams reaching ERROR state during rolling upgrade / restart of brokers

2023-10-02 Thread Matthias J. Sax
I did mean client side... If KS goes into ERROR state, it should log the reason. If the logs are indeed empty, try to register an uncaught-exception-handler via KafkaStreamssetUncaughtExceptionHandler(...) -Matthias On 10/2/23 12:11 PM, Debraj Manna wrote: Are you suggesting to check

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Proven Provenzano
Hi, To verify the release of release 3.6.0 RC2 I did the following: - Downloaded the source, built and ran the tests. - Validated SCRAM with KRaft including creating credentials with kafka-storage. - Validated Delegation Tokens with KRaft +1 (non-binding) --Proven On Mon, Oct 2,

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Bill Bejeck
Hi Satish, Thanks for running the release. I performed the following steps: - Validated all the checksums, signatures, and keys - Built the release from source - Ran all unit tests - Quick start validations - ZK and Kraft - Connect - Kafka Streams - Spot checked

Re: Kafka Streams reaching ERROR state during rolling upgrade / restart of brokers

2023-10-02 Thread Debraj Manna
Are you suggesting to check the Kafka broker logs? I do not see any other errors logs on the client / application side. On Fri, 29 Sep, 2023, 22:01 Matthias J. Sax, wrote: > In general, Kafka Streams should keep running. > > Can you inspect the logs to figure out why it's going into ERROR state

Re: mirror-maker

2023-10-02 Thread Richard Bosch
Hi Denny, If you use mirrormaker to replicate data between clusters then it needs access to all brokers. There is also a clear reason for this, Mirrormaker acts as a normal Kafka client when connecting to Kafka clusters. Mirrormaker uses the normal produce and consume protocol, which means that

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Josep Prat
Hi Satish, +1 (non-binding) from me as well! Thanks for running the release I run the following validations steps: - Built from source with Java 11 and Scala 2.13 - Verified Signatures and hashes of the artifacts generated - Navigated through Javadoc including links to JDK classes - Run the unit

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Kamal Chandraprakash
+1 (non-binding) 1. Built the source from 3.6 branch in scala 2.12 and 2.13 2. Ran all the unit and integration tests. 3. Ran quickstart and verified the produce-consume on a 3 node cluster. 4. Verified the tiered storage functionality with local-tiered storage. Thanks, Kamal On Mon, Oct 2,

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Divij Vaidya
+ 1 (non-binding) Verifications: 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd compression using an open messaging benchmark and found 3.6 to be better than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently 6-7% lower CPU utilization, lesser spikes on

Re: mirror-maker

2023-10-02 Thread Denny Fuchs
hi, it seems to be pretty clear .. TCPDump says, it tries to connect to the other hosts too, so I need to allow it. cu denny

mirror-maker

2023-10-02 Thread Denny Fuchs
hi, I try to use mirror maker to replicate the data, and I have one trivial question: Do I need access to / add  **all** Kafka Brokers ? I have a 3.1 Cluster with 7 brokers and a new 3.5 with 5 brokers and I want to migrate from the 3.1 (Zoo) to 3.5 (KRaft). Both clusters are in different

**SOLVED** Re: Offline migration from Zookeeper cluster (3.1) to Kraft enabled Cluster 3.5

2023-10-02 Thread Denny Fuchs
hi, Am 01.10.23 um 22:49 schrieb Denny Fuchs: What is the way, to do this ? it seems, that mirror-maker is the right tool cu denny

Re: Kafka Protocol : Compact Array or Array ?

2023-10-02 Thread Neeraj Vaidya
Thanks Luke. That helps. Where can I find information about schemas of all requests ? Regards, Neeraj Sent from my iPhone > On 2 Oct 2023, at 6:03 pm, Luke Chen wrote: > > Hi Neeraj, > > Yes, for MetadataRequest, version 0 ~ 8, the topic is ARRAY type. After > version 9, it'll be

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Federico Valeri
Hi Satish, I did the following to verify the release: - Built from source with Java 17 and Scala 2.13 - Ran all unit and integration tests - Spot checked documentation - Ran custom client applications using staging artifacts on a 3-nodes cluster - Tested tiered storage with one of the available

Re: Kafka Protocol : Compact Array or Array ?

2023-10-02 Thread Luke Chen
Hi Neeraj, Yes, for MetadataRequest, version 0 ~ 8, the topic is ARRAY type. After version 9, it'll be COMPACT_ARRAY. It's because of this definition: "flexibleVersions": "9+". You can check KIP-482 for more information: https://cwiki.apache.org/confluence/display/KAFKA/KIP-482 Thanks. Luke On

Re: [VOTE] 3.6.0 RC2

2023-10-02 Thread Luke Chen
Hi Satish, I verified with: 1. Ran quick start in KRaft for scala 2.12 artifact 2. Making sure the checksum are correct 3. Browsing release notes, documents, javadocs, protocols. 4. Verified the tiered storage feature works well. +1 (binding). Thanks. Luke On Mon, Oct 2, 2023 at 5:23 AM