Hi Frank,
Here are the explicitly set properties:
Consumer:
Properties props = new Properties();
props.put("zk.connect", KafkaProperties.zkConnect);
props.put("groupid", KafkaProperties.groupId);
props.put("zk.sessiontimeout.ms", "400");
props.put("zk.synctime.ms", "200");
props.put("autocommit.interval.ms", "1000");
Producer:
props.put("serializer.class", "kafka.serializer.StringEncoder");
props.put("zk.connect", "localhost:2181");
Everything else is out of box except for log.default.flush.interval.ms=1 in
server.properties which is set according to Stone's suggestion.
Regards,
Kishore
On Fri, May 17, 2013 at 11:48 AM, Francis Dallaire <
[email protected]> wrote:
> What are the settings you have set on the producer ... is there any sync
> ack ? If so which type ?
>
> In any case, it would be useful to know the parameters you set on the
> consumer and producer.
>
> Thanks
>
> Frank
>
>
> Sent from Samsung Mobile
>
>
>
> -------- Original message --------
> From: "Kishore V. Kopalle" <[email protected]>
> Date: 05-17-2013 2:11 AM (GMT-05:00)
> To: [email protected],Francis Dallaire <[email protected]>
> Subject: Re: Time difference between message fetch and message send is
> very high (~900-1200ms)
>
>
> Hi Frank,
>
> Yes, they are on the same machine. Zoo Keeper, Kafka server and the
> KafkaConsumerProducerDemo (with one producer and one consumer) are running
> on the same computer.
>
> Regards,
> Kishore
>
> On Fri, May 17, 2013 at 11:38 AM, Francis Dallaire <
> [email protected]<mailto:[email protected]>> wrote:
> Quick question. Are the producer and consumer on the same machine ? If
> not, are you accounting for any difference in the time setting between
> producer and consumer ?
>
> Frank
>
> Sent from Samsung Mobile
>
>
>
> -------- Original message --------
> From: Kishore Kopalle <[email protected]<mailto:[email protected]>>
> Date: 05-17-2013 1:41 AM (GMT-05:00)
> To: [email protected]<mailto:[email protected]>
> Subject: Time difference between message fetch and message send is very
> high (~900-1200ms)
>
>
> Hello All,
>
> In my earlier mail, I am sorry I did not give lot of details on my setup
> where I am measuring performance between producer and consumer. Here are
> the details:
>
> - I am using Kafka version 0.7 downloaded from Apache web site.
> - I am using a modified version of KafkaConsumerProducerDemo example that
> comes with the distribution. I have made the following modifications:
>
> - In Producer java file:
>
> int messageNo = 1;
> SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
> HH:mm:ss.SSS") ;
>
> while(true)
> {
> String messageStr = new String("Message_" + messageNo);
> producer.send(new ProducerData<Integer, String>(topic,
> "Message produced at " + dateFormat.format(new Date())
> + ": " + messageStr));
> messageNo++;
> }
>
>
> - In Consumer.java file:
>
> SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
> HH:mm:ss.SSS") ;
>
> while(it.hasNext())
> System.out.println("Message received at " + dateFormat.format(new
> Date()) + ": " + ExampleUtils.getMessage(it.next().message()));
>
>
> Based on the messages on the console from consumer, I am getting values
> ranging from 900ms to 1200ms per message for the time difference between
> received message time and the timestamp in the message itself.
>
> Kindly let me know where I am going wrong and are there any configuration
> steps involved for getting optimal performance for the time taken for a
> specific message to be fetched and the time when it was sent by producer.
>
> Regards,
> Kishore
>
>
