> On May 29, 2014, 5:20 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/server/RequestPurgatory.scala, line 64 > > <https://reviews.apache.org/r/21878/diff/2/?file=598165#file598165line64> > > > > Why we want to reduce the default purgatory size?
This interval is in terms of requests. Consider a case where you have 1000 partitions on your broker and set your fetch timeout to be 10ms (either for consumers or for replication), then in a period of inactivity you will have 10,000,000 obsolete entries in purgatory and purge them all at once. During this time no other purgatory expiration happens which leads to large latency spikes. > On May 29, 2014, 5:20 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/tools/TestEndToEndLatency.scala, line 25 > > <https://reviews.apache.org/r/21878/diff/2/?file=598166#file598166line25> > > > > Can we add some comments here indicating the end-to-end latency is for > > async producer mode? Well I'm not sure how relevant the acknowledgement mode is as the consumer can't consume until full acknowledgement. Since the consumer fetch and the producer ack happen in parallel I would expect the difference to be negligible. - Jay ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21878/#review44266 ----------------------------------------------------------- On May 28, 2014, 11:15 p.m., Jay Kreps wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/21878/ > ----------------------------------------------------------- > > (Updated May 28, 2014, 11:15 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1468 > https://issues.apache.org/jira/browse/KAFKA-1468 > > > Repository: kafka > > > Description > ------- > > KAFKA-1468 Misc. improvements from benchmarking. > > > Diffs > ----- > > clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java > f1def508153b7361b8720dbcc939e06b0e7e45d3 > clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java > bc4074ec841650eff48dd8aafc1e6c712d8eae7d > > clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java > 3e83ae0fefb7578cccbd8dc0cd9b28f22e0951b2 > > clients/src/main/java/org/apache/kafka/clients/tools/ProducerPerformance.java > eb18739d181662e2dee5dd6a23e0181f4c740139 > clients/src/test/java/org/apache/kafka/clients/producer/SenderTest.java > a2b77226f8c58caf632a0f4665bd4e4cd93e643d > core/src/main/scala/kafka/server/KafkaConfig.scala > c7508d5568ec6b94c47d1839aa8695e2dbb76b97 > core/src/main/scala/kafka/server/RequestPurgatory.scala > c064c5c4cf1191335572da8a2caf5f95dce902c1 > core/src/main/scala/kafka/tools/TestEndToEndLatency.scala > 37a9ec27abfd0acd6c01f11d987dd719bc9138a5 > > Diff: https://reviews.apache.org/r/21878/diff/ > > > Testing > ------- > > > Thanks, > > Jay Kreps > >
