Messages in partitions are logically the same. Topic is a logical group of partitions for messages that are of same type and/or from same source e.g. if you have two web-apps producing data, you will like to tell data from one source that from another. These are candidates for a topic each. If you see increase in data over time, you may want to consider increasing partitions to achieve better throughputs. As Dave said, partitions help in distributing / dividing the load from a source into multiple parallel streams, for faster throughputs. This means each partition deals with less (part of) data and maintains its replica state on different nodes.
Same thing on the consumer end, you can start as many parallel processes (or threads) as there are partitions of a topic, to achieve better throughputs. -Zakee > On Nov 16, 2016, at 9:03 AM, Tauzell, Dave <dave.tauz...@surescripts.com> > wrote: > > Partitions are used to distribute the messages in a topic between several > different broker instances. This provides higher throughput. Partitions > can also be replicate which allows for high availability. > > -Dave > > From: Doyle, Keith [mailto:keith.do...@greenwayhealth.com] > Sent: Wednesday, November 16, 2016 11:01 AM > To: users@kafka.apache.org > Subject: Topics vs Partitions > > > Trying to wrap my head around some of the concepts... > > > > For semantic partitioning, is there any rationale for choosing to divide a = > single topic into multiple partitions as opposed to just creating multiple = > topics each with a single partition? > > > > > -- > > [Greenway_Health_PNG_large_NO_tag] > > Keith Doyle | Senior Software Engineer > Greenway Health | 4301 W. Boy Scout Blvd., Suite 800, Tampa, FL 33607 > (702) 256-9911 office | GreenwayHealth.com<http://www.greenwayhealth.com/> > [cid:image003.png@01D04086.868CBCB0]<https://www.linkedin.com/company/greenway-health>[cid:image004.png@01D04086.868CBCB0]<https://www.facebook.com/GreenwayHealthLLC>[cid:image005.png@01D04086.868CBCB0]<https://twitter.com/greenway> > > NOTICE: This e-mail message and all attachments transmitted with it may > contain legally privileged and confidential information intended solely for > the use of the addressee. If the reader of this message is not the intended > recipient, you are hereby notified that any reading, dissemination, > distribution, copying, or other use of this message or its attachments is > strictly prohibited. If you have received this message in error, please > notify the sender immediately by electronic mail and delete this message and > all copies and backups thereof. Thank you. Greenway Health. > This e-mail and any files transmitted with it are confidential, may contain > sensitive information, and are intended solely for the use of the individual > or entity to whom they are addressed. If you have received this e-mail in > error, please notify the sender by reply e-mail immediately and destroy all > copies of the e-mail and any attachments. ____________________________________________________________ 3 Signs You May Have a Fatty Liver [Watch] livecellresearch.com http://thirdpartyoffers.netzero.net/TGL3231/582cb5ecce48a35ec2d52st03vuc
