Re: Re: Servers Getting disconnected

17:13 To: users Subject: Re: Servers Getting disconnected Please take a look at KAFKA-4477 On Mon, Oct 30, 2017 at 2:06 AM, Yuanjia <yuanjia8...@163.com> wrote: > Hi, > My cluster version is 0.10.0.0, 10 nodes. One server has many error > log, like that > java.io.IOExceptio

Servers Getting disconnected

problem? Regards Yuanjia Li

how to find the producer for a given topic

Hi all, We can use kafka-consumer-groups.sh to find group owner. Is there any tools can find the producer for a given topic? Thanks. YuanjiaLi

Re: Re: when will the messsages be sent to broker

. Yuanjia Li From: Matthias J. Sax Date: 2017-03-02 13:11 To: users Subject: Re: when will the messsages be sent to broker That's weird. If batch.size=0 and linger.ms=0 records should get send out immediately. buffer.memory is an upper bound for the overall memory. If you write to multiple partitions

Re: Re: when will the messsages be sent to broker

? Thanks. Yuanjia Li From: Matthias J. Sax Date: 2017-03-02 01:42 To: users Subject: Re: when will the messsages be sent to broker There is also linger.ms parameter that is an upper bound how long a (not yet filled) buffer is hold before sending it even if it's not full. Furthermore, you can do sync

when will the messsages be sent to broker

Hi all, When will the messsages be sent in kafka0.10.0?If I use KafkaProducer.send to send one message, the messsages isn't sent immediately except invoke flush or close. Thanks. Yuanjia Li

Re: Storm kafka integration

Hi, You can use storm-kafka-client, which support storing consumer offsets in kafka cluster. Yuanjia Li From: pradeep s Date: 2017-02-20 03:49 To: users Subject: Storm kafka integration Hi, I am using Storm 1.0.2 and Kafka 0.10.1.1 and have query on Spout code to integrate with Kafka. As per

Re: Kafka exception

ownCommand in the queue. Is there any exception information when calling the method ConsumerIterator.makeNext()? Yuanjia Li From: 揣立武 Date: 2017-02-16 14:29 To: dev CC: users; 陈希; 涂扬 Subject: Kafka exception Hi,all! Our program uses the high level consumer api(the version is 0.8.x). Sometimes t

Re: Re: How to choose one broker as Group Coordinator

My question is the selection procedure. Thanks. From: 陈江枫 Date: 2017-02-15 18:01 To: users Subject: Re: How to choose one broker as Group Coordinator when a consumer join a group, selection will be triggered, and then rebalance. 2017-02-15 17:59 GMT+08:00 Yuanjia <yuanjia8...@163.

How to choose one broker as Group Coordinator

Hi all, Group Coordinator can be different for different consumer groups,When a consumer wants to join a group,how to choose the Group Coordinator? Thanks, Yuanjia Li

kafka-log-cleaner-thread exits

(LogCleaner.scala:237) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:215) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) [2016-12-02 11:33:52,744] INFO [kafka-log-cleaner-thread-0], Stopped (kafka.log.LogCleaner) Thanks. Yuanjia Li

How to get consumer creating time and last consuming time

Hi all, In kafka0.8, I can get these infomations from zk.But how to get them in kafka0.10? Thanks Yuanjia Li

kafka balance partition data across directory locations after partition created

move some partition data to different location. Does kafka has balancer to do that automatically, or some utils can help do that? Thanks, Regards, Yuanjia Li

Re: Re: Manually update consumer offset stored in Kafka

Hi Yifan， You can try this procedure with kafka0.10, stop consumer group before do that. consumer.subscribe(Arrays.asList(topic));//consumer "enable.auto.commit" set false consumer.poll(1000); consumer.commitSync(offsets);//the offsets is to be updated consumer offset

回复: Kafka broker failed to work when migrating the broker to another machine

Zhongjing， can you show some server.log ? ie:broker 4 or the leader of topic log_error. 发件人： 俞忠静 发送时间： 2015-03-02 18:37 收件人： users@kafka.apache.org 抄送： 吴红芳; 邵宏亮; 方超超 主题： Kafka broker failed to work when migrating the broker to another machine Hi, I set up a Kafka cluster with 5 brokers, with

delete log file

Regards YuanJia Li

Re: Re: delete log file

. Thanks Regards YuanJia Li From: Yang Fang Date: 2015-01-23 12:15 To: users Subject: Re: delete log file YuanJia, I want to know why your broker's disk is almost full . Maybe your issue is the same with mine. Is it one broker out of service for a long time, then it fails back, at first it looks good

how to order message between different partition

Hi all, I have a topic with 3 partitions, and each partion has its sequency in kafka. How to order message between different partion? Anyone has experiences about that, thanks. Thanks Regards YuanJia Li

Re: kafka monitoring system

Hi Sa Li, You can try to use jmxtrans+opentsdb to monitor kafka. Jmxtrans is collecting data with JMX and sending to opentsdb. Opentsdb is graphing and alerting. YuanJia Li From: Sa Li Date: 2014-12-23 08:41 To: users Subject: kafka monitoring system Hi, all I am thinking to make

Re: Reuse ConsumerConnector

hi Chen, Maybe you can take some tips from kafka.tools.KafkaMigrationTool.java, it's also a multi-threads using case. Reusing the same ConsumerConnector every time is ok. If you create ConsumerConnector repeatedly with the same consumer.id, the conflict will happen in ZK. Yuanjia Li From

Pagecache cause OffsetOutOfRangeException

to kernel's pagecache and not flush to the file yet,while I'm consuming new messages from the file? How to fix it? Thanks, Yuanjia Li

Re: Re: How to push metrics to graphite - jmxtrans does not work

Jmxtrans should connect to the jmxremote port. Try to run ps -aux |grep kafka, and find the process contain -Dcom.sun.management.jmxremote.port or not. If not, try to edit kafka-server-start.sh, add export JMX_PORT=.

Re: Re: Pagecache cause OffsetOutOfRangeException

offset got by SimpleConsumer.getOffsetsBefore API. I don't quite understand the kernel pagecache, so I wonder the delay between writting and reading. Thanks, Yuanjia From: Guozhang Wang Date: 2014-12-03 03:23 To: users@kafka.apache.org Subject: Re: Pagecache cause