Hi Patricia, Flink will create one Kafka consumer per parallelism, however, you'll need some testing to measure the capability of a single task. Usu, one consumer can consume at a much higher rate than 1 record per second.
Best, Zhanghao Chen ________________________________ From: patricia lee <plee3...@gmail.com> Sent: Thursday, February 1, 2024 15:26 To: user@flink.apache.org <user@flink.apache.org> Subject: Parallelism and Target TPS Hi, I have a flink job that consumes from kafka and sinks it to an API. I need to ensure that my flink job can send within the rate limit 200 tps, we are planning to increase the parallelism, but I do not know the right number to set. 1 parallelism does equal to 1 consumer? So if 200, should we set it to 200 parallelism too? I only created a simple retry that will send it again if 421 http error is received. Any advise, appreciated. Thanks
