Hi, I am having a few issues with the Flink (v1.8.1) backpressure default settings, which lead to poor throughput in a comparison I am doing between Storm, Spark and Flink.
I have a setup that simulates a progressively worse straggling task that Storm and Spark cope with the relatively well. Flink not so much. Code can be found here - https://github.com/owenrh/flink-variance. See this throughput chart for the an idea of how badly - https://owenrh.me.uk/assets/images/blog/smackdown/flink-constant-straggler.png I do not have any production experience with Flink, but I have had a look at the Flink docs and there is nothing in there that jumps out at me to explain or address this. I presume I am missing something, as I cannot believe Flink is this weak in the face of stragglers. It must be configuration right? Would appreciate any help on this. I've got a draft blog post that I will publish in a day or two, and don't want to criticise the Flink backpressure implementation for what seems most likely some default configuration issue. Thanks in advance, Owen -- Owen Rees-Hayward 07912 876046 twitter.com/owen4d
