All writes to a single partition are synchronized on the broker. So, the more partitions the more parallelism (and better throughput) you have.
Thanks, Jun On Mon, Dec 22, 2014 at 1:24 PM, Sa Li <sal...@gmail.com> wrote: > Hi, All > > I've run bin/kafka-producer-perf-test.sh on our kafka-production cluster, I > found the number of partitions really have huge impacts on the producer > performance, see: > > start.time, end.time, compression, message.size, batch.size, > total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec > 2014-12-22 19:53:27:392, 2014-12-22 19:54:25:581, 1, 3000, 200, 2861.02, > 49.1678, 1000000, 17185.3787 > 2014-12-22 19:55:27:048, 2014-12-22 19:56:23:318, 1, 3000, 200, 2861.02, > 50.8446, 1000000, 17771.4590 > 2014-12-22 19:58:09:466, 2014-12-22 19:59:05:068, 1, 3000, 200, 2861.02, > 51.4554, 1000000, 17984.9646 > 2014-12-22 19:59:40:389, 2014-12-22 20:00:28:646, 1, 3000, 200, 2861.02, > 59.2872, 1000000, 20722.3822 > 2014-12-22 20:02:41:993, 2014-12-22 20:03:22:481, 1, 3000, 200, 2861.02, > 70.6635, 1000000, 24698.6762 > 2014-12-22 20:03:47:594, 2014-12-22 20:04:26:238, 1, 3000, 200, 2861.02, > 74.0354, 1000000, 25877.2384 > 2014-12-22 20:11:49:492, 2014-12-22 20:12:25:843, 1, 3000, 200, 2861.02, > 78.7055, 1000000, 27509.5596 > 2014-12-22 20:12:53:290, 2014-12-22 20:13:29:746, 1, 3000, 200, 2861.02, > 78.4788, 1000000, 27430.3270 > 2014-12-22 20:13:53:194, 2014-12-22 20:14:29:470, 1, 3000, 200, 2861.02, > 78.8682, 1000000, 27566.4351 > 2014-12-22 20:14:51:491, 2014-12-22 20:15:25:451, 1, 3000, 200, 2861.02, > 84.2468, 1000000, 29446.4075 > 2014-12-22 20:16:51:369, 2014-12-22 20:17:27:452, 1, 3000, 200, 2861.02, > 79.2901, 1000000, 27713.8819 > 2014-12-22 20:17:57:882, 2014-12-22 20:18:33:957, 1, 3000, 200, 2861.02, > 79.3076, 1000000, 27720.0277 > > > The number of partitions above are from 1 to 12, I wonder why it has such > big difference? > > thanks > > -- > > Alec Li >
