In addition to that, currently, the more partitions a topic has, the more IO throughput a topic can get. This is because IOs across partitions can be parallelized.
Thanks, Jun On Sat, Nov 12, 2011 at 7:53 PM, <[email protected]> wrote: > When I was evaluating it, couple of things came up - > > 1. If brokers are of different config, you could potentially configure > same topic with different #partitons depending on how much IO a node is > doing. > 2. General rule of thumb could be to over partition a topic than > #consumers consuming from it as mentioned in kafka white paper. > 3. More partitions could also help in a way that if some brokers are down > still your consumers are getting messages from other ones instead they > getting stalled. > > Inder > > > ------Original Message------ > From: Mark > To: [email protected] > ReplyTo: [email protected] > Subject: Partitions > Sent: Nov 13, 2011 6:02 AM > > I know its a case by case decision but how do you go about determining > how many partitions to use? Thanks > > > Sent from BlackBerry® on Airtel
