Can anyone please help me with this? On Fri, 18 Oct 2019 at 2:58 PM, Hrishikesh Mishra <[email protected]> wrote:
> Hi all, > > I wanted to understand, having large numbers of consumers on > producer latency and brokers. I have around 7K independent consumers. Each > consumer is consuming all partitions of a topic. I have manually assigned > partitions of a topic to a consumer, not using consumer groups. Each > consumer is consuming msg only one topic. Message size is small less than a > KB. Produce throughput is also low. But when produce spike come then > produce latency (here ack=1). Brokers resource is also very low. I want to > understand the impact on having large number of consumers on Kafka. > > > *Cluster Details: * > > Broker#: 13 (1 Broker : 14 cores & 36GB memory ) > > Kafka cluster version: 2.0.0 > Kafka Java client version: 2.0.0 > Number topics: ~15. > Number of consumers: 7K (all independent and manually assigned all > partitions of a topic to a consumers. One consumer is consuming all > partitions from a topic only) > > > > Regards, > Hrishikesh > >
