Dear community, happy to share a brief community update for the past week with configurable memory sharing between Flink and its Python "side car", stateful Python UDFs, an introduction of our GSoD participants and a little bit more.
Flink Development ============== * [datastream api] Dawid has started a vote to remove DataStream#fold and DataStream#split (both already deprecated) in Flink 1.12. [1] * [runtime] Xintong has started a discussion thread for "Intra-Slot Management Memory Sharing", which lets users configure the fraction of managed memory that should be used by Flink internally (RocksDB or Batch algorithms) on the one side and the Python process on the other side. [2] * [python] Wei Zhon has started discussion for FLIP-139, which aims to add support for *stateful *Python UDFs for the Table API/SQL. So far, only stateless functions are supported. [3] * [documentation] Kartik Khare and Mohammad Haseeb Asif will work with the Apache Flink Community to improve the documentation of Flink SQL as part of their participation in Google Season of Docs 2020. [4] [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Remove-deprecated-DataStream-fold-and-DataStream-split-in-1-12-tp44229.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-141-Intra-Slot-Managed-Memory-Sharing-tp44146.html [3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-139-General-Python-User-Defined-Aggregate-Function-on-Table-API-tp44139.html [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Introducing-the-GSoD-2020-Participants-tp44144.html flink-packages.org ============== * fabricalab has published a DynamoDB streaming source on flink-packages.org. [5] [5] https://flink-packages.org/packages/streaming-flink-dynamodb-connector Notable Bugs ========== * [FLINK-18934] [1.11.1] Flink's mechanism to deal with idle sources/partitions introduced in Flink 1.11 [6] does not currently work with co-functions, union or joins. [7] [6] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/event_timestamps_watermarks.html#dealing-with-idle-sources [7] https://issues.apache.org/jira/browse/FLINK-18934 Events, Blog Posts, Misc =================== * Dian Fu is now part of the Apache Flink PMC. Congratulations! [8] [8] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-New-PMC-member-Dian-Fu-tp44170p44240.html Cheers, Konstantin -- Konstantin Knauf https://twitter.com/snntrable https://github.com/knaufk
