I don't think you'll know for sure until you try. You will have to properly
protect your consumer, as it's not thread safe. (I asked recently in a
situation where all I wanted to do from a separate thread was commit().)
Seems like an easy experiment to run. You have a producer which posts to
two t
Exactly. This is what I want to achieve.
Is it good practice to keep on updating subscription list of same kafka
consumer?
Any concerns in multi threaded environment?
Regards,
Chintan
On 07-Jan-2018 6:32 PM, "Skip Montanaro" wrote:
Got it. Does it work to maintain your currently subscribed to
Got it. Does it work to maintain your currently subscribed topic list, then
spend to it and subscribe to the extended list when you receive an update?
On Jan 6, 2018 9:37 PM, "chintan mavawala" wrote:
> Discovery of new topic is not a concern. Clients who creates a new topic
> sends my process t
Discovery of new topic is not a concern. Clients who creates a new topic
sends my process the topic name. In simpler words, my process performs some
operations which any client might be wanting. So they create a Kafka topic
and send me the name of the topic for me to subscribe.
My process needs st
I'm a bit confused. How do you discover that new topics have been created?
If you discover a new topic somehow, how do you know it's one you are
interested in?
I suppose you could have a consumer which subscribes to all topics using
pattern=".", then have it just publish new topic names to a fixed
In theory pattern "." would work. But that would mean subscribing on all
topics which I don't want.
I share same pain in using pattern which is why I would like to know if it
is good practice to subscribe to topics after kafka consumer has started.
Regards,
Chintan
On 06-Jan-2018 5:47 PM, "Skip
In theory, wouldn't
consumer.subscribe(pattern=".")
work? I say "in theory" because my experience with subscribing by pattern
hasn't been great. I suspect my mental model of how it's implemented isn't
a close approximation of reality.
Skip
On Jan 6, 2018 4:07 AM, "chintan mavawala" wrote:
I w
I want to subscribe to all topics as they are created but topic names does
not have any pattern. Consumer starts with subscription to let's say 3
topics and add more topics to exisiting kafka consumer as they are created.
Regards,
Chintan
On 04-Jan-2018 11:02 PM, "Jordan Pilat" wrote:
> Did you
Did you want to subscribe to all topics as they are created, or to exclude some
of them?
If the latter, is there a pattern in the names of the topics you wish to
exclude?
- Jordan Pilat
On 2018-01-03 23:39, chintan mavawala wrote:
> Hi,
>
> We have a use case where we need to start kafka con
Hi,
We have a use case where we need to start kafka consumer with a fixed list
of kafka topics and add more topics on the fly. Since there is no pattern
in names of topics, using pattern for dynamic subscriptions of topics is
not feasible.
Is it a good practice to subscribe to topics on given kaf
10 matches
Mail list logo
