afka.apache.org"
Subject: Re: Query related to Kafka Consumer Limit
Satarupa,
Glad I could help, and thanks for the additional context. It doesn't sound like
this use-case requires real-time notification. Why not just poll a web service
periodically, say every 5 minutes?
If you need
Satarupa,
Glad I could help, and thanks for the additional context. It doesn't sound
like this use-case requires real-time notification. Why not just poll a web
service periodically, say every 5 minutes?
If you need something more real-time, I'd suggest using a more traditional
publish-subscribe
Hi Ryanne,
Thank you so much for detailed explanation.
Here is couple of more asks -
1) Here Consumers are not short lived but we want to listen to the message and
become idle. Is there a way to notify the Kafka server that message is reached
to Consumer? So that post which server does not pr
Satarupa, it sounds like you are conflating some concepts here. Some
clarifying points:
- Only one consumer in a consumer group receives any given record from a
topic. So in your scenario of 1 million consumers, they could not be
members of the same group. You'd need 1 million consumer "groups" to
Satarupa,
In my experience kafka has a 10k partition limit per topic. I don't think
you are going to be able to get 1 million partitions to work on a single
topic. A consumer would need to subscribe to listen to a kafka topic. You
will probably need to have multiple consumer groups or multiple top
Hi,
I am from Intuit. We want to use Kafka as message bus where Single Producer
produces message and 1 Million Consumer listens it.
Requirement –
1. Single producer and 1 Million Consumer and one particular Topic with
message.
2. When Pushed Message thru producer, should be received by
