Siva, For Kafka brokers as long as the total #.partitions is not too much I think it should be fine (we have been hosting 600+ partitions on a single node). You have to pay some attention on your ZK nodes though since with #.topics increasing its metadata storage will take more and more space.
Guozhang On Tue, Apr 14, 2015 at 2:37 PM, Sivananda Reddy <sivananda2...@gmail.com> wrote: > Hi, > > # I looked the documents of kafka and I see that there is no way a > consume instance can > read specific messages from partition. > # I have an use case where I need to spawn a topic(single partition) > for each user, > so there would be 10k online users at a time, there would be very > less data per topic. > My questions is, what are limitations of having multiple topics(with > 1 partition), I think > this situation would cause heavy memory consumption and are they any > other limitations?. > Basically the problem boils down to what are the scalability > limitations of having > multiple topics(hardware/software)? > > Thanks a lot in advance. > > Regards, > Siva. > -- -- Guozhang
