maybe there some changes in 0.9.0.0;
but still you can try increase producer sending rate, and see if there are
message lost but no exception;
note that, to increase the producer sending rate, you must have enough producer
'power';
in my case, I have 50 producer sending message at the same time : )
At 2015-11-12 16:16:23, "Hawin Jiang" <hawin.ji...@gmail.com> wrote:
>Hi Jinxing
>
>I am using kafka_2.10-0.9.0.0-SNAPSHOT. I have downloaded source code and
>installed it last week.
>
>I saw 97446 messages have been sent to kafka successfully.
>
>So far, I have not found any failed messages.
>
>
>
>Best regards
>Hawin
>
>On Thu, Nov 12, 2015 at 12:07 AM, jinxing <jinxing6...@126.com> wrote:
>
>> Hi, what version are you using ?
>>
>>
>> i am using 0.8.2.0, and I encountered this problem before;
>>
>>
>> it seems that if the message rate of the producer side is to high, some of
>> the messages will lost;
>>
>>
>> also, i found that the callback method of the producer 'send' API is not
>> reliable;
>>
>>
>> only successful sent message will trigger the callback, but the failed
>> ones don't;
>>
>>
>> you saw this?
>>
>>
>>
>>
>>
>>
>>
>>
>> At 2015-11-12 16:01:17, "Hawin Jiang" <hawin.ji...@gmail.com> wrote:
>> >Hi All
>> >
>> >I have sent messages to Kafka for one minute. I found 97446 messages in
>> >producer side and 96896 messages in consumer side for Case 1.
>> >I also tried case 2. I have faced the same issues. The number is not
>> match
>> >between producer and consumer.
>> >Can someone take a look at this issue?
>> >Thanks.
>> >
>> >
>> >Case 1:
>> >
>> >long startTime = System.currentTimeMillis();
>> >long maxDurationInMilliseconds = 1 * 60 * 1000;
>> >int messageNo = 0;
>> >while (true) {
>> >if (System.currentTimeMillis() <= startTime
>> >+ maxDurationInMilliseconds) {
>> >messageNo++;
>> >String messageStr = "KAFKA_"+messageNo;
>> >System.out.println("Message: "+messageNo);
>> >producer.send(new KeyedMessage<Integer, String>(topic,messageStr));
>> >} else {
>> >producer.close();
>> >System.out.println("Total kafka Message: "+messageNo);
>> >break;
>> >}
>> >}
>> >
>> >
>> >Case 2:
>> >
>> >for (int i=1;i<=12000;i++)
>> >String messageStr = "KAFKA_"+i;
>> >producer.send(new KeyedMessage<Integer, String>(topic,messageStr));
>> >
>> >
>> >
>> >Best regards
>> >Hawin
>>
