Re:Good java consumer example with new 0.9 consumer api

2015-12-06 Thread jinxing
You can refer: http://people.apache.org/~nehanarkhede/kafka-0.9-consumer-javadoc/doc/ At 2015-12-06 08:41:24, "Dhyan Muralidharan" wrote: >Hi All, > > Can some one point me good java Consumer that uses 0.9 new consumer API . I > searched in github and couldn’t find one . > >—Dhyan

kafka 0.8.2 build problem

2015-11-15 Thread jinxing
Hi all, I am newbie for kafka, and I try to build kafka 0.8.2 from source code as a start; but I'm faced a lot of problem; here is the timeline of what I did: 1. I cloned kafka:git clone https://github.com/apache/kafka.git; 2. I created branch 0.8.2 and checkout that branch and reset

Re:Re: Re: Re: Re: Kafka lost data issue

2015-11-12 Thread jinxing
u >requesting from Kafka? > >On Thu, Nov 12, 2015 at 2:03 AM, jinxing wrote: > >> in kafka_0.8.3.0: >> kafkaProducer = new KafkaProducer<>(properties, new ByteArraySerializer(), >> new ByteArraySerializer()); >> kafkaProducer.flush()

Re:Re: Re: Re: Kafka lost data issue

2015-11-12 Thread jinxing
> >The messages are "Thread1_kafka_1" and "Thread2_kafka_1". Something like >that. > >For GetOffsetShell report below: > >[kafka@dn-01 bin]$ ./kafka-run-class.sh kafka.tools.GetOffsetShell >--broker-list dn-01:9092 --time -1 --topic kafka-test >kafka-test:0:12

Re:Re: Re: Kafka lost data issue

2015-11-12 Thread jinxing
there is a flush api of the producer, you can call this to prevent messages lost; maybe it can help; At 2015-11-12 16:43:54, "Hawin Jiang" wrote: >Hi Jinxing > >I don't think we can resolve this issue by increasing producers. if I >increased more pro

Re:Re: Re: Kafka lost data issue

2015-11-12 Thread jinxing
yes, increase producer will trigger more messages lost; that's what I wanted to say; At 2015-11-12 16:43:54, "Hawin Jiang" wrote: >Hi Jinxing > >I don't think we can resolve this issue by increasing producers. if I >increased more producers, it should lost m

Re:Re: Re: Kafka lost data issue

2015-11-12 Thread jinxing
pic ? > >Regards, >Prabhjot > >On Thu, Nov 12, 2015 at 2:13 PM, Hawin Jiang wrote: > >> Hi Jinxing >> >> I don't think we can resolve this issue by increasing producers. if I >> increased more producers, it should lost more messages. >> >&g

Re:Re: Kafka lost data issue

2015-11-12 Thread jinxing
age at the same time : ) At 2015-11-12 16:16:23, "Hawin Jiang" wrote: >Hi Jinxing > >I am using kafka_2.10-0.9.0.0-SNAPSHOT. I have downloaded source code and >installed it last week. > >I saw 97446 messages have been sent to kafka successfully. > >

Re:Kafka lost data issue

2015-11-12 Thread jinxing
Hi, what version are you using ? i am using 0.8.2.0, and I encountered this problem before; it seems that if the message rate of the producer side is to high, some of the messages will lost; also, i found that the callback method of the producer 'send' API is not reliable; only successful

async producer callback not reliable

2015-09-08 Thread jinxing
if i wanna send the message syncronously i can do as below: future=producer.send(producerRecord, callback); future.get(); but the throughput decrease dramatically; is there a method to send the messages by batch but synchronously ?

kafka callback issuekk

2015-09-08 Thread jinxing
hi, all im testing kafka-0.8.2.0 using ASYNC producer; I have two processes, each process have 20 threads to send messages, the testing time is 10 seconds; the message is random UUID; the result is as below: I called "producer.send(producerRecord, callback)" 2200191 times; the "callback"