Performance Issues in Source Operator while migrating to Flink-1.14 from 1.9

Hey team, We are migrating our Flink codebase and a bunch of jobs from Flink-1.9 to Flink-1.14. To ensure uniformity in performance we ran a bunch of jobs for a week both in 1.9 and 1.14 simultaneously with the same resources and configurations and monitored them. Though most of the jobs are runn

Re: Performance Issues in Source Operator while migrating to Flink-1.14 from 1.9

Hi Arujit, I'm also looping in some contributors from the connector and runtime perspective in this thread. Did you also test the upgrade first by only upgrading to Flink 1.14 and keeping the FlinkKafkaConsumer? That would offer a better way to determine if a regression is caused by the upgrade of

Re: Performance Issues in Source Operator while migrating to Flink-1.14 from 1.9

Hey Martijn, Thanks a lot for getting back to us. To give you a little bit more context, we do maintain an open-source project around flink dagger which is a wrapper for proto processing. As part of the upgrade to the latest version, we did some refactoring and move

Re: Performance Issues in Source Operator while migrating to Flink-1.14 from 1.9

Hi, Unfortunately the new KafkaSource was contributed without good benchmarks, and so far you are the first one that noticed and reported this issue. Without more direct comparison (as Martijn suggested), it's hard for us to help right away. It would be a tremendous help for us if you could for ex