Re: High latency for Kafka transactions

2020-12-02 Thread John Coleman
484f03184208d88822d5e0%7C817e531d191b4cf58812f0061d89b53d%7C0%7C0%7C637409030504161229%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=t6XTDwN4QQs4kL5pMZzCSpcbjwMX95QrA8dzpDdrjyo%3D&reserved=0 ________ From: John Coleman Sen

Re: High latency for Kafka transactions

2020-12-02 Thread John Coleman
ucing traffic to the Kafka brokers, e.g. how frequently do you send a `whole group of messages`? If it is every 10ms e.g. then your traffic would be 100K per sec, which is pretty large enough with transactions. Guozhang On Fri, Nov 13, 2020 at 1:09 AM John Colem

High latency for Kafka transactions

2020-11-13 Thread John Coleman
Hi, We have very high latency ~1 second when we send a small number of small messages to Kafka. The whole group of messages is less than 1k. We are thinking better to just send the messages together as 1 message containing a collection of all the messages. Is there something that we can tweak