Re: Latency requirements for Kafka connect workers

t workers communicate > with each other over the rest interface. Is there any specific latency > requirements i.e. there should be a maximum latency of 'x' ms to allow > effective functioning of the Kafka connect distributed worker cluster. > > Regards > Ashish >

Latency requirements for Kafka connect workers

Hi Team, As per my understanding, the distributed Kafka connect workers communicate with each other over the rest interface. Is there any specific latency requirements i.e. there should be a maximum latency of 'x' ms to allow effective functioning of the Kafka connect distributed work

Re: Requirements

Fri, Sep 13, 2019 at 8:26 AM Peter Menapace < peter.menap...@vanderlande.com> wrote: > Hi all, > > I have a small small question. In my company we would like to use Apache > Kafka with KSQL. > > And my small question is: which hardware requirements do you have to run > Kafk

Requirements

Hi all, I have a small small question. In my company we would like to use Apache Kafka with KSQL. And my small question is: which hardware requirements do you have to run Kafka and KSQL in small and big environments? Best regards, Peter Mit freundlichen Grüßen, Peter Menapace Senior IT

RE: Kafka hardware requirements

Thank you Kurt for your insight. Jean-Marc. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Kurt Rudolph Sent: Monday, May 6, 2019 9:22:24 AM To: users@kafka.apache.org Subject: Re: Kafka hardware requirements

Re: Kafka hardware requirements

I am currently running a few clusters on virtual machines connected to a VSAN. I have not had any major ssues. My clusters are small; 3 virtual machines (4 cpu and 8 GB memory each). I am routinely processing 20K msg/sec with surges of over 35K msg/sec. Based on my performance testing, I don't

Kafka hardware requirements

Hello, I would like to know the impact of running kafka in production on virtual machines connected to a SAN (Storage Area Network)? (I mean the impact on kafka performance, what would be a max limit in terms of number of messages per second, number of producers and consumers, kafka can deal wi

Consumer ID requirements and usage clarification

Can someone clarify the usage and requirements of consumer ID when used/specified? I came across this topic and brought that up in a comment in the thread, someone else suggested it being worthy of another post https://stackoverflow.com/questions/34550873/difference-between-groupid-and

Re: Discussion on requirements for Data Encryption functionality in Kafka (KIP-317)

ore people to join the discussion once some more details are known. Other than that, I am also very happy to gather feedback on the mailing list from people who won't be able to make it to the Summit. So anybody who can come up with some thoughts or requirements around encryption function

Re: Discussion on requirements for Data Encryption functionality in Kafka (KIP-317)

sure that the initial implementation provides the necessary > flexibility I'd like some feedback from the community on what requirements > they would have around data encryption and key management. > > The following questions should serve as a starting point for the > discussion, pleas

Discussion on requirements for Data Encryption functionality in Kafka (KIP-317)

flexibility I'd like some feedback from the community on what requirements they would have around data encryption and key management. The following questions should serve as a starting point for the discussion, please feel free to address anything that comes to mind which I have not mentioned

Re: Windowed aggregations memory requirements

This is a timely question and we've updated the documentation here on capacity planning and sizing for Kafka Streams jobs: http://docs.confluent.io/current/streams/sizing.html . Any feedback welcome. It has scenarios with windowed stores too

Re: Windowed aggregations memory requirements

That depends on if your using event, processing or ingestion time. My understanding is that if you play a record through that is T-6, the only way that TimeWindows.of(TimeUnit.MINUTES.toMillis(1)).until(TimeUnit.MINUTES.toMillis(5)) would actually process that record in your window is if your usi

Windowed aggregations memory requirements

The base question I'm trying to answer is "how much memory does my instance need". Considering a use case where I want to keep a rolling average on a tumbling window of 1 minute size allowing for late arrivals up to 5 minutes (lower bound) we would have something like this: TimeWindows.of(TimeUni