Hello,
I was hoping, that I could get some input from the experts.
We are currently discussing an architecture, where we are using raspberry
pi sized embedded computers connected via LTE (Southbound) to collect data
and need to forward this data from our system to different customer systems
Don't I lose consumer group coordination with assign?
On Mon, Mar 23, 2020 at 11:49 PM Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:
> Hi Ryan,
>
> The maxPollInterval waits for at-most the given time duration and returns
> ASAP even if a single record is available.
> If you want
Hi all,
I'm trying to aggregate a stream of messages and return a stream of
aggregated results using kafka streams.
At some point, depending on the incoming message, the old aggregate needs
to be closed and a new aggregate needs to be created, just like a session
that is closed due to some close
Hey,
so I turned on debug logging and now I see the MM2 does a lot, so it is
hard to tell exactly what :D
So, what I see the most during that "not mirroring messages" period is logs
about:
- [AdminClient clientId=adminclient-31] Queueing
Call(callName=findCoordinator, deadlineMs=1584976266682)
I bumped tasks.max to 10, but it does not help if I only have two
instances: all MirrorSourceConnector tasks are still in one instance.
But if I start more instances in case I need more bandwidth, it will help
to spread the load, so this is ok for me so far.
Also KAFKA-9352 will help to get
Thanks Sophie. Much appreciated! It's good to know this will be improved in a
later release.
On 2020/03/23 22:52:27, Sophie Blee-Goldman wrote:
> I don't think it has anything to do with your specific topology, but it
> might be
> that the "stickiness" is overriding the "data parallelism
Hi Peter,
Not sure this is what you're looking for -
https://issues.apache.org/jira/browse/KAFKA-9352
On Mon, Mar 23, 2020 at 11:37 PM Ryanne Dolan wrote:
> Thanks Peter for running this experiment. That looks sorta normal. It looks
> like Connect is deciding to use 10 total tasks and doesn't
Hi Ryan,
The maxPollInterval waits for at-most the given time duration and returns
ASAP even if a single record is available.
If you want to collect data once 30-45 minutes, better to use the Consumer
with `assign` mode and poll for records
once in 30 minutes.
If you're using the consumer with
