My job runs fine when running without kafka transactions. The source and
sink are kafka in my job with a couple of RocksDB based stateful operators
taking 100GB each.
When I enable kafka transactions, things go well initially and we can see
high throughput as well. However, after a few hours, the
Hi fellow flink users,
I'd like to seek advice on how to find the performance bottleneck of a
stateful functions pipeline. The throughput is too low. Ideally we could
push it to 2000 messages/s, but I don't get it above 100/s. The pipeline
quickly gets under backpressure.
Some facts:
*
