> On April 20, 2015, 5:30 p.m., Jay Kreps wrote: > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java, > > line 157 > > <https://reviews.apache.org/r/31850/diff/9/?file=931821#file931821line157> > > > > Read locks are very expensive. I am pretty worried about this. If we > > want to do this we need to do a pretty detailed examination of the perf > > impact. > > Jiangjie Qin wrote: > Hi Jay, I looked into the ReentrantReaderWriterLock implementation and it > seems under the hood it uses CompareAndSet which should provide similar > performance as atomic integer. But I agree this definitely largely depends on > implementation. > I modified o.a.k.clients.tools.ProducerPerformance a little bit to make > it multiple threaded. The performance in following tests settings are very > similar which are all ~1M messages/second when target is 10M message/sec. > 1. 10 thread with latest trunk > 2. 10 threads using atomic integer AtomicInteger > 3. 10 threads using ReaderWriterLock > When I increase the thread number to 50. It drops to about 0.82M > messages/second in all cases. > It seems reader lock did not introduce performance issue.
Hey Jay, do you have any other performance tests in mind that we should run? - Jiangjie ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31850/#review80753 ----------------------------------------------------------- On April 21, 2015, 12:38 a.m., Jiangjie Qin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/31850/ > ----------------------------------------------------------- > > (Updated April 21, 2015, 12:38 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1660 > https://issues.apache.org/jira/browse/KAFKA-1660 > > > Repository: kafka > > > Description > ------- > > A minor fix. > > > Incorporated Guozhang's comments. > > > Modify according to the latest conclusion. > > > Patch for the finally passed KIP-15git status > > > Addressed Joel and Guozhang's comments. > > > rebased on trunk > > > Rebase on trunk > > > Addressed Joel's comments. > > > Addressed Joel's comments > > > Addressed Jay's comments > > > Changed javadoc per Jay's suggestion > > > Change java doc as Jay suggested. > > > Diffs > ----- > > clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java > b70e1a3d406338d4b9ddd6188d2820e87545a9b6 > clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java > 6913090af03a455452b0b5c3df78f266126b3854 > clients/src/main/java/org/apache/kafka/clients/producer/Producer.java > 5b3e75ed83a940bc922f9eca10d4008d67aa37c9 > > clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java > 0e7ab29a07301309aa8ca5b75b32b8e05ddc3a94 > > clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java > 70954cadb5c7e9a4c326afcf9d9a07db230e7db2 > > clients/src/main/java/org/apache/kafka/common/errors/InterruptException.java > fee322fa0dd9704374db4a6964246a7d2918d3e4 > clients/src/main/java/org/apache/kafka/common/serialization/Serializer.java > c2fdc23239bd2196cd912c3d121b591f21393eab > > clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java > 05e2929c0a9fc8cf2a8ebe0776ca62d5f3962d5c > core/src/test/scala/integration/kafka/api/ProducerSendTest.scala > 9811a2b2b1e9bf1beb301138f7626e12d275a8db > > Diff: https://reviews.apache.org/r/31850/diff/ > > > Testing > ------- > > Unit tests passed. > > > Thanks, > > Jiangjie Qin > >
