why includeMetadataInTimeout could be set only in deprecated api

2022-05-26 Thread Xiangyuan LI
Hi kafka team After check the kafka 2.8.0 code, seems org.apache.kafka.clients.consumer.KafkaConsumer#poll(org.apache.kafka.common.utils.Timer, boolean) allow user to keep poll records during consumer-rebalance when use CooperativeStickyAssignor, and allow this behaivor must set includeMetadataI

Re: Kafka upgrade from 0.10 to 2.3.x

2020-05-18 Thread Xiangyuan LI
hi praveen: we had done upgrade from 0.10.0.1 to 2.3.0 lasy year, it's feasible to do a roll upgrade, just follow official upgrade document. although we have met some serious problem during the process(it's such a big change that all clients and related packages such as spring,springboot should

Re: Kakfa Broker scaling tips for high number of consumers for a single topic

2019-11-14 Thread Xiangyuan LI
hi I want to know what's the kafka server & client version you're using? And I want to know ,you said you have 2000 consumer,is that mean 2000 consumer groups or 2000 consumer in one consumer group? aravind s 于2019年11月15日周五 上午3:12写道: > Hi, > > We have a use-case where there are close to 40 prod

produce very slow after upgrade to 2.3.0

2019-11-13 Thread Xiangyuan LI
hi: recently we tried to upgrade our kafka cluster 0.10.0.1 to 2.3.0. we know kafka upgrade should: 1.use 2.3.0.jar and restart all brokers 2.unset inter.broker.protocol.version=0.10.0.1 and restart all brokers 3.unset log.message.format.version=0.10.0.1 and restart all bro

whether kafka can list producers

2019-09-03 Thread Xiangyuan LI
Hi, I want to know whether kafka can list producers that send messages to specified topic? i need know this because recently my consumer consume some messages that producer haven't send to topic,so we now don't know who send this message. thx...

Re: A new way migrate replica from old broker to new one

2019-08-21 Thread Xiangyuan LI
> On Fri, Aug 16, 2019 at 10:03 AM Xiangyuan LI > wrote: > > > Hi, > > Now our kafka cluster has 5 brokers that broker.id is 1,2,3,4,5. > > > > We want to decommission broker 1 and add new broker 6,we expect that all > > replica in broker 1 could migrate

A new way migrate replica from old broker to new one

2019-08-16 Thread Xiangyuan LI
Hi, Now our kafka cluster has 5 brokers that broker.id is 1,2,3,4,5. We want to decommission broker 1 and add new broker 6,we expect that all replica in broker 1 could migrate to broker 6,and nothing change for 2~5. Now we have an idea.our servers are in aws,we plan to remove disk from broker 1,

what's the sequence when consumer subscribe one topic's multi partition execute poll method

2019-07-14 Thread Xiangyuan LI
Hi, When our consumer poll messages from a topic every partition has very large delay,we found that some partitions decrease delay fast but others are even still increase. I think the reason is every consumer subscribes sevral partitions and broker will always response messages with the same p

whether inter.broker.protocol.version could downgrade

2019-06-20 Thread Xiangyuan LI
Hi, recently our team had try to upgrade kafka 0.10.0.1 to 2.2.0.we do so by upgrade guide http://kafka.apache.org/documentation.html#upgrade, after we set all brokers inter.broker.protocol.version to 2.2.0,we found that producer produce rate become very slow and many messages take more than 50