Yes but you have more than 1 POS terminal per location so you still don't need 20,000 partitions. Just one per location. How many locations do you have?
In doesn’t matter anyway since you can build a Kafka cluster with up to 200,000 partitions if you use the latest versions of Kafka. https://blogs.apache.org/kafka/entry/apache-kafka-supports-more-partitions “As a rule of thumb, we recommend each broker to have up to 4,000 partitions and each cluster to have up to 200,000 partitions” -hans > On Apr 1, 2019, at 2:02 AM, Alexander Kuterin <akute...@gmail.com> wrote: > > Thanks, Hans! > We use location specific SKU pricing and send specific price lists to the > specific POS terminal. > > пн, 1 апр. 2019 г., 3:01 Hans Jespersen <h...@confluent.io>: > >> Doesn’t every one of the 20,000 POS terminals want to get the same price >> list messages? If so then there is no need for 20,000 partitions. >> >> -hans >> >>> On Mar 31, 2019, at 7:24 PM, <akute...@gmail.com> <akute...@gmail.com> >> wrote: >>> >>> Hello! >>> >>> >>> >>> I ask for your help in connection with the my recent task: >>> >>> - Price lists are delivered to 20,000 points of sale with a frequency of >> <10 >>> price lists per day. >>> >>> - The order in which the price lists follow is important. It is also >>> important that the price lists are delivered to the point of sale online. >>> >>> - At each point of sale, an agent application is deployed, which >> processes >>> the received price lists. >>> >>> >>> >>> This task is not particularly difficult. Help in solving the task is not >>> required. >>> >>> >>> >>> The difficulty is that Kafka in our company is a new "silver bullet", and >>> the project manager requires me to implement the following technical >>> decision: >>> >>> deploy 20,000 Kafka consumer instances (one instance for each point of >> sale) >>> for one topic partitioned into 20,000 partitions - one partition per >>> consumer. >>> >>> Technical problems obtained in experiments with this technical decision >> do >>> not convince him. >>> >>> >>> >>> Please give me references to the books/documents/blogposts. which clearly >>> shows that Kafka not intended for this way to use (references to other >>> anti-patterns/pitfalls will be useful). >>> >>> My own attempts to find such references were unsuccessful. >>> >>> >>> >>> Thank you! >>> >>> >>> >>
