Thanks for compiling the list of 1.11 efforts Zhijiang and Piotr! This helps a lot to better understand what the community is currently working on. Looking forward to another successful release.
Best Regards, Yu On Wed, 11 Mar 2020 at 11:17, Zhijiang <wangzhijiang...@aliyun.com.invalid> wrote: > Hi community, > > > Not more than one month ago we have released Flink 1.10. We are now > heading for the Flink 1.11 release and we, as release managers, would like > to share with you what are the features that the community is currently > working on and we are hoping that will be part of the Flink 1.11 release. > Currently we are aiming with the feature freeze to happen in late April. > > As for now, some of the features are in the very early stages of the > development or even brainstorming. Because of that, some of them do not > have associated JIRA tickets or FLIP documents. For the next progress > announcement we are hoping that this will be no longer the case. > > Please also note that because we are still relatively at the beginning of > the release cycle, some of the FLIPs haven’t yet been voted. > > - SQL / Table > - FLIP-42: Restructure documentation [1] > - FLIP-65: New type inference for Table API UDFs [2] > - FLIP-84: Improve TableEnv’s interface [3] > - FLIP-91 Introduce SQL client gateway and provide JDBC driver [4] > - FLIP-93: Introduce JDBC catalog and Postgres catalog [5] > - FLIP-105: Support to interpret and emit changelog in Flink SQL [6] > - FLIP-107: Reading table columns from different parts of source records > [7] > - [FLINK-14807] Add Table#collect API for fetching data [8] > - Support query and table hints > - ML / Connectors > - FLIP-27: New source API [9] > - [FLINK-15670] Wrap a source/sink pair to persist intermediate result for > subgraph failure recovery [10] > - Pulsar source / sink / catalog > - Update ML Pipeline API interface to better support Flink ML lib > algorithms > - PyFlink > - FLIP-58: Debugging and monitoring of Python UDF [11] > - FLIP-106: Expand the usage scope of Python UDF [12] > - Integration with most popular Python libraries (Pandas) > - Performance improvements of Python UDF > - Support running python UDF in docker workers > - Add Python ML API > - Fully support all kinds of Python UDF > - Web UI > - FLIP-98: Better back pressure detection [13] > - FLIP-99: Make max exception configurable [14] > - FLIP-100: Add attempt information [15] > - FLIP-102: Add more metrics to TaskManager [16] > - FLIP-103: Better TM/JM log display [17] > - [FLINK-14816] Add thread dump feature for TaskManager [18] > - Runtime > - FLIP-56: Support for dynamic slots on the TaskExecutor [19] > - FLIP-67: Support for cluster partitions [20] > - FLIP-76: Unaligned checkpoints [21] > - FLIP-83: Flink e2e performance testing framework [22] > - FLIP-85: Support cluster deploy mode [23] > - FLIP-92: Add N-Ary input stream operator in Flink [24] > - FLIP-108: Add GPU to the resource management (specifically for UDTF & > UDF) [25] > - FLIP-111: Consolidate docker images [26] > - Unified memory configuration for JobManager > - Specify upper bound for number of allocated TaskManagers > - [FLINK-9407] ORC format for StreamingFileSink [27] > - [FLINK-10742] Let Netty use Flink's buffers on downstream side [28] > - [FLINK-10934] Support per-job mode for Kubernetes integration [29] > - [FLINK-11395] Avro writer for StreamingFileSink [30] > - [FLINK-11427] Protobuf parquet writer for StreamingFileSink [31] > - [FLINK-11499] Extend StreamingFileSink BulkFormats to support arbitrary > roll policies [32] > - [FLINK-14106] Make SlotManager pluggable [33] > - [FLINK-15672] Switch to Log4j2 by default [34] > - [FLINK-15674] Consolidate Java and Scala type extraction stack [35] > - [FLINK-15679] Improve Flink’s ID system [36] > - [FLINK-15786] Use the separated classloader to load connectors’ jar [37] > - [FLINK-15788] Various Kubernetes improvements [38] > - [FLINK-15911][FLINK-15154] Support Flink work over NAT [39] > - [FLINK-16408] Bind user code class loader to lifetime of a slot [40] > - [FLINK-16428] Network memory management for backpressure [41] > - [FLINK-16430] Pipelined region scheduling [42] > - Calculate required shuffle memory before allocating slots > - State Backend: > - [FLINK-5763] Make savepoint self-contained / relocatable [43] > - [FLINK-8871] Complete checkpoint cancellation messages [44] > - [FLINK-12692] Support disk spilling in HeapKeyedStateBackend [45] > - [FLINK-15012] Cleanup of leftover files in HDFS/OSS/S3 [46] > - Enable local recovery by default > - Support getCustomizedState in KeyedStateStore [47] > - [FLINK-15532] Enable strict capacity limit for memory usage for RocksDB > [48] > Your release managers, > Zhijiang & Piotr Nowojski > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-42%3A+Rework+Flink+Documentation > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-65%3A+New+type+inference+for+Table+API+UDFs > [3] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=134745878 > [4] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway > [5] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-93%3A+JDBC+catalog+and+Postgres+catalog > [6] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-105%3A+Support+to+Interpret+and+Emit+Changelog+in+Flink+SQL > [7] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-107%3A+Reading+table+columns+from+different+parts+of+source+records > [8] https://issues.apache.org/jira/browse/FLINK-14807 > [9] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface > [10] https://issues.apache.org/jira/browse/FLINK-15670 > [11] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table > [12] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-106%3A+Support+Python+UDF+in+SQL+Function+DDL > [13] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-98%3A+Better+Back+Pressure+Detection > [14] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-99%3A+Make+Max+Exception+Configurable > [15] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information > [16] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-102%3A+Add+More+Metrics+to+TaskManager > [17] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=147427143 > [18] https://issues.apache.org/jira/browse/FLINK-14816 > [19] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation > [20] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-67%3A+Cluster+partitions+lifecycle > [21] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints > [22] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-83%3A+Flink+End-to-end+Performance+Testing+Framework > [23] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-85+Flink+Application+Mode > [24] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-92%3A+Add+N-Ary+Stream+Operator+in+Flink > [25] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-108%3A+Add+GPU+support+in+Flink > [26] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-111%3A+Docker+image+unification > [27] https://issues.apache.org/jira/browse/FLINK-9407 > [28] https://issues.apache.org/jira/browse/FLINK-10742 > [29] https://issues.apache.org/jira/browse/FLINK-10934 > [30] https://issues.apache.org/jira/browse/FLINK-11395 > [31] https://issues.apache.org/jira/browse/FLINK-11427 > [32] https://issues.apache.org/jira/browse/FLINK-11499 > [33] https://issues.apache.org/jira/browse/FLINK-14106 > [34] https://issues.apache.org/jira/browse/FLINK-15672 > [35] https://issues.apache.org/jira/browse/FLINK-15674 > [36] https://issues.apache.org/jira/browse/FLINK-15679 > [37] https://issues.apache.org/jira/browse/FLINK-15786 > [38] https://issues.apache.org/jira/browse/FLINK-15788 > [39] https://issues.apache.org/jira/browse/FLINK-15911 > [39] https://issues.apache.org/jira/browse/FLINK-15154 > [40] https://issues.apache.org/jira/browse/FLINK-16408 > [41] https://issues.apache.org/jira/browse/FLINK-16428 > [42] https://issues.apache.org/jira/browse/FLINK-16430 > [43] https://issues.apache.org/jira/browse/FLINK-5763 > [44] https://issues.apache.org/jira/browse/FLINK-8871 > [45] https://issues.apache.org/jira/browse/FLINK-12692 > [46] https://issues.apache.org/jira/browse/FLINK-15012 > [47] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-customize-state-in-customized-KeyedStateBackend-td32771.html > [48] https://issues.apache.org/jira/browse/FLINK-15532
