I wanted to understand whether broker will be unstable with large number of consumers or will consume face some issue like lag will increase?
On Mon, Oct 21, 2019 at 6:55 PM Shyam P <[email protected]> wrote: > What are you trying to do here ? whats your objective ? > > On Sat, Oct 19, 2019 at 8:45 PM Hrishikesh Mishra <[email protected]> > wrote: > > > Can anyone please help me with this? > > > > On Fri, 18 Oct 2019 at 2:58 PM, Hrishikesh Mishra <[email protected]> > > wrote: > > > > > Hi all, > > > > > > I wanted to understand, having large numbers of consumers on > > > producer latency and brokers. I have around 7K independent consumers. > > Each > > > consumer is consuming all partitions of a topic. I have manually > assigned > > > partitions of a topic to a consumer, not using consumer groups. Each > > > consumer is consuming msg only one topic. Message size is small less > > than a > > > KB. Produce throughput is also low. But when produce spike come then > > > produce latency (here ack=1). Brokers resource is also very low. I want > > to > > > understand the impact on having large number of consumers on Kafka. > > > > > > > > > *Cluster Details: * > > > > > > Broker#: 13 (1 Broker : 14 cores & 36GB memory ) > > > > > > Kafka cluster version: 2.0.0 > > > Kafka Java client version: 2.0.0 > > > Number topics: ~15. > > > Number of consumers: 7K (all independent and manually assigned all > > > partitions of a topic to a consumers. One consumer is consuming all > > > partitions from a topic only) > > > > > > > > > > > > Regards, > > > Hrishikesh > > > > > > > > >
