Re: [VOTE] 2.7.1 RC2
Excuse me if this is the wrong thread to ask, but I was curious as to if there was an updated timeline for the release of 2.7.1? Thanks in advance, Upesh Upesh Desai Senior Software Developer ude...@itrsgroup.com www.itrsgroup.com Internet communications are not secure and therefore the ITRS Group does not accept legal responsibility for the contents of this message. Any view or opinions presented are solely those of the author and do not necessarily represent those of the ITRS Group unless otherwise specifically stated. [itrs.email.signature] From: Mickael Maison Date: Thursday, April 8, 2021 at 12:36 PM To: dev , Users , kafka-clients Subject: [VOTE] 2.7.1 RC2 Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 2.7.1. Since 2.7.1 RC1, the following JIRAs have been fixed: KAFKA-12593, KAFKA-12474, KAFKA-12602. Release notes for the 2.7.1 release: https://home.apache.org/~mimaison/kafka-2.7.1-rc2/RELEASE_NOTES.html *** Please download, test and vote by Friday, April 16, 5pm BST Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~mimaison/kafka-2.7.1-rc2/ * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://home.apache.org/~mimaison/kafka-2.7.1-rc2/javadoc/ * Tag to be voted upon (off 2.7 branch) is the 2.7.1 tag: https://github.com/apache/kafka/releases/tag/2.7.1-rc2 * Documentation: https://kafka.apache.org/27/documentation.html * Protocol: https://kafka.apache.org/27/protocol.html * Successful Jenkins builds for the 2.7 branch: The build is still running, I'll update the thread once it's complete /** Thanks, Mickael Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
Re: does kafka support reducing topic partitions on the fly?
Thanks... came across KIP however it is still under discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-694%3A+Support+Reducing+Partitions+for+Topics On Wed, May 19, 2021 at 8:30 PM Andrew Grant wrote: > Hi Pushkar, > I don't think so. Per https://kafka.apache.org/documentation/, "Kafka does > not currently support reducing the number of partitions for a topic.". > Andrew > > On Wed, May 19, 2021 at 10:17 AM Pushkar Deole > wrote: > > > Hi All, > > > > I have a question around reducing topic partitions on kafka. Currently, > all > > topics are 6 partitions. I want to reduce it to 3 for some of the topics. > > Is this supported by kafka without deleting the topic? > > > > > -- > Andrew Grant > 8054482621 >
Re: does kafka support reducing topic partitions on the fly?
Hi Pushkar, I don't think so. Per https://kafka.apache.org/documentation/, "Kafka does not currently support reducing the number of partitions for a topic.". Andrew On Wed, May 19, 2021 at 10:17 AM Pushkar Deole wrote: > Hi All, > > I have a question around reducing topic partitions on kafka. Currently, all > topics are 6 partitions. I want to reduce it to 3 for some of the topics. > Is this supported by kafka without deleting the topic? > -- Andrew Grant 8054482621
does kafka support reducing topic partitions on the fly?
Hi All, I have a question around reducing topic partitions on kafka. Currently, all topics are 6 partitions. I want to reduce it to 3 for some of the topics. Is this supported by kafka without deleting the topic?
Possible race in handling IP address change
Hey,
We've experienced few incidents where suddenly Kafka producers weren't able
to send messages and were simply timing out. I've a repro now after some
investigation. We're running the cluster of 3 brokers on K8s and it's
enough to kill the underlying AWS VM using AWS Console which causes also IP
change (in our case 10.128.132.196 → 10.128.135.90). It needs to be VM
hosting broker which is a leader of partition (node 1).
Slightly before the kill there were 2 events:
About to close the idle connection from 10001 due to being idle for 38545
> millis
> About to close the idle connection from 10002 due to being idle for 34946
> millis
And then client picks the node to connect to:
Removing node 10.128.140.202:9092 (id: 10002 rack: us-east-1c) from least
> loaded node selection since it is neither ready for sending or connecting
> Found least loaded connecting node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a)
> About to close the idle connection from 1 due to being idle for 30529
> millis
> Node 1 disconnected.
> Initiating connection to node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a) using address /10.128.132.196
and then it repeats:
Found least loaded connecting node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a)
> About to close the idle connection from 1 due to being idle for 30027
> millis
> Node 1 disconnected.
> Initiating connection to node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a) using address /10.128.132.196
Found least loaded connecting node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a)
> About to close the idle connection from 1 due to being idle for 30027
> millis
> Node 1 disconnected.
> Initiating connection to node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a) using address /10.128.132.196
Found least loaded connecting node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a)
> About to close the idle connection from 1 due to being idle for 30027
> millis
> Node 1 disconnected.
> Initiating connection to node 10.128.132.196:9092 (id: 1 rack:
> us-east-1a) using address /10.128.132.196
Affected clients won't ever request METADATA from other brokers (10001 or
10002) to discover new IP. It doesn't happen for all clients as some handle
the process gracefully and those log with:
log.info("Hostname for node {} changed from {} to {}.", id,
> connectionState.host(), host);
--
BR,
Michał Łowicki
Consumer Node Metrics - request-latency-avg
Hi, I'm trying to gather metrics for my consumers and I'd like to get more visibility on the latency for all types of requests coming from my consumers (not only fetch requests). However when looking at JMX metrics I see that the metrics request-latency-avg and request-latency-max in consumer-node-metrics are NaN. For reference I'm creating my consumers using java with the kafka-clients library in its 2.4.1 version. What could be the reason for the request-latency-max and request-latency-avg being reported as Nan? Thanks in advance
