Hi, Do you think there can be any issue with Flinks performance, with 400Kb
up to 1 MB payload record sizes ? my Spark streaming seems to be doing
better. Are there any recommended configurations or increasing parallelism
to improve Flink streaming using flink kafka connect?
Regards,
Vijay
On
Hi Robert,
Thanks for information. payloads so far are 400KB (each record).
To achieve high parallelism at the downstream operator do I rebalance the
kafka stream ? Could you give me an example please.
Regards,
Vijay
On Fri, Aug 14, 2020 at 12:50 PM Robert Metzger wrote:
> Hi,
>
> Also, can
Hi,
Also, can we increase parallel processing, beyond the number of
> kafka partitions that we have, without causing any overhead ?
Yes, the Kafka sources produce a tiny bit of overhead, but the potential
benefit of having downstream operators at a high parallelism might be much
bigger.
How
Hi Team,
I am trying to increase throughput of my flink stream job streaming from
kafka source and sink to s3. Currently it is running fine for small events
records. But records with large payloads are running extremely slow like at
rate 2 TPS.
Could you provide some best practices to tune?
