Hi, I'm assuming you're using consumer groups? E.g., group.id=X
Cheers, Liam On Mon, 13 Dec 2021, 6:30 pm Jigar Shah, <jigar.shah1...@gmail.com> wrote: > Hello, > I am trying to test the latency between message production and message > consumption using Java Kafka-Client*(2.7.2)* library. > The configuration of cluster is 3 KafkaBrokers*(2.7.2, Scala 2.13)*, 3 > Zookeeper*(3.5.9)* > Here is a pattern what I have observed > Reference: > ConsumerReadTimeStamp: Timestamp when record received in Kafka Consumer > ProducerTimeStamp: Timestamp added before producer.send record > RecordTimeStamp: CreateTimeStamp inside the record obtained at consumer > > [image: kafka1.png] > > *For 100 Messages* > > *ConsumerReadTimeStamp-ProducerTimeStamp(ms)* > > *ConsumerReadTimeStamp-RecordTimeStamp(ms)* > > *Average* > > *252.56* > > *238.85* > > *Max* > > *2723* > > *2016* > > *Min* > > *125* > > *125* > > > On the consumer side it takes too much time for initial few messages but > later on it is quite consistent. > I have executed the above same test for large number of messages : > 100,1000,10000,etc. and the pattern seems to be same > Here are the configurations, mostly using default properties. > Topic: > partitions=16 > min.insync.replica=2 > replication.factor=3 > > > Consumer: > security.protocol=PLAINTEXT > enable.auto.commit=true > > > Producer: > security.protocol=PLAINTEXT > compression.type=gzip > acks=all > > > Is there any reason why there is huge latency at the beginning when a > consumer is created please? > Also please suggest some way to optimise configurations to have some > better consistent results ? > > Thank you in advance for your feedback. >
