Re: [Flink Unit Tests] Unit test for Flink streaming codes

Hi Vijay, Any unit test of Flink operators is actually an IT case as it involves a large portion of the stack. A real unit test, would be over a factored out logic class. Similar to Niels, I'd recommend to use simple sources (env.fromElements) and sinks to inject the data and retrieve the data an

Re: sporadic "Insufficient no of network buffers" issue

After debugging more, it seems like this issue is caused by the scheduling strategy. Depending on the tasks assigned to the task manager, probably the amount of memory configured for network buffers is running out. Through these references: FLINK-12122

Re: [Flink Unit Tests] Unit test for Flink streaming codes

Vijay, There's a section of the docs that describes some strategies for writing tests of various types, and it includes some Scala examples [1]. There are also some nice examples from Konstantin Knauf in [2], though they are mostly in Java. [1] https://ci.apache.org/projects/flink/flink-docs-sta

Re: [DISCUSS] FLIP-133: Rework PyFlink Documentation

Hi jincheng, Thanks for the discussion. +1 for the FLIP. A well-organized documentation will greatly improve the efficiency and experience for developers. Best, Shuiqiang Hequn Cheng 于2020年8月1日周六 上午8:42写道： > Hi Jincheng, > > Thanks a lot for raising the discussion. +1 for the FLIP. > > I thin

Re: Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

Hi Vikash, Sorry for the late reply. Is your version of Flink kafka *connector* 1.10.1 too? Actually it's a bug in the connector, so I think you need to upgrade the connector to 1.10.1 too, not just Flink itself. I tried Flink 1.10.0/1.10.1 + flink-kafka-connector 1.10.0 and indeed reproduced the

Re: [DISCUSS] FLIP-133: Rework PyFlink Documentation

Would be great if you could join the contribution of PyFlink documentation @Marta ! Thanks for all of the positive feedback. I will start a formal vote then later... Best, Jincheng Shuiqiang Chen 于2020年8月3日周一 上午9:56写道： > Hi jincheng, > > Thanks for the discussion. +1 for the FLIP. > > A well-o