XComp commented on code in PR #603: URL: https://github.com/apache/flink-web/pull/603#discussion_r1084002403
########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> Review Comment: ```suggestion ``` test instabilities ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29479'>FLINK-29479</a>] - Support whether using system PythonPath for PyFlink jobs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29539'>FLINK-29539</a>] - dnsPolicy in FlinkPod is not overridable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29615'>FLINK-29615</a>] - MetricStore does not remove metrics of nonexistent subtasks when adaptive scheduler lowers job parallelism +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29627'>FLINK-29627</a>] - Sink - Duplicate key exception during recover more than 1 committable. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29677'>FLINK-29677</a>] - Prevent dropping the current catalog +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29728'>FLINK-29728</a>] - TablePlanner prevents Flink from starting is working directory is a symbolic link +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29749'>FLINK-29749</a>] - flink info command support dynamic properties +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29781'>FLINK-29781</a>] - ChangelogNormalize uses wrong keys after transformation by WatermarkAssignerChangelogNormalizeTransposeRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29788'>FLINK-29788</a>] - StatefulJobWBroadcastStateMigrationITCase failed in native savepoints +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29803'>FLINK-29803</a>] - Table API Scala APIs lack proper source jars +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29817'>FLINK-29817</a>] - Published metadata for apache-flink in pypi are inconsistent and causes poetry to fail +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29827'>FLINK-29827</a>] - [Connector][AsyncSinkWriter] Checkpointed states block writer from sending records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29830'>FLINK-29830</a>] - PulsarSinkITCase$DeliveryGuaranteeTest.writeRecordsToPulsar failed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29839'>FLINK-29839</a>] - HiveServer2 endpoint doesn't support TGetInfoType value 'CLI_ODBC_KEYWORDS' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29849'>FLINK-29849</a>] - Event time temporal join on an upsert source may produce incorrect execution plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29857'>FLINK-29857</a>] - Fix failure to connect to 'HiveServer2Endpoint' when using hive3 beeline +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29899'>FLINK-29899</a>] - Stacktrace printing in DefaultExecutionGraphCacheTest is confusing maven test log output +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29923'>FLINK-29923</a>] - Hybrid Shuffle may face deadlock when running a task need to execute big size data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29927'>FLINK-29927</a>] - AkkaUtils#getAddress may cause memory leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30030'>FLINK-30030</a>] - Unexpected behavior for overwrite in Hive dialect +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30133'>FLINK-30133</a>] - HadoopModuleFactory creates error if the security module cannot be loaded +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30138'>FLINK-30138</a>] - BlobServerCleanupTest utilizes failsWithin with a quite low timeout which might result in test instabilities +</li> Review Comment: ```suggestion ``` also test-related ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29479'>FLINK-29479</a>] - Support whether using system PythonPath for PyFlink jobs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29539'>FLINK-29539</a>] - dnsPolicy in FlinkPod is not overridable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29615'>FLINK-29615</a>] - MetricStore does not remove metrics of nonexistent subtasks when adaptive scheduler lowers job parallelism +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29627'>FLINK-29627</a>] - Sink - Duplicate key exception during recover more than 1 committable. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29677'>FLINK-29677</a>] - Prevent dropping the current catalog +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29728'>FLINK-29728</a>] - TablePlanner prevents Flink from starting is working directory is a symbolic link +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29749'>FLINK-29749</a>] - flink info command support dynamic properties +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29781'>FLINK-29781</a>] - ChangelogNormalize uses wrong keys after transformation by WatermarkAssignerChangelogNormalizeTransposeRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29788'>FLINK-29788</a>] - StatefulJobWBroadcastStateMigrationITCase failed in native savepoints +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29803'>FLINK-29803</a>] - Table API Scala APIs lack proper source jars +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29817'>FLINK-29817</a>] - Published metadata for apache-flink in pypi are inconsistent and causes poetry to fail +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29827'>FLINK-29827</a>] - [Connector][AsyncSinkWriter] Checkpointed states block writer from sending records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29830'>FLINK-29830</a>] - PulsarSinkITCase$DeliveryGuaranteeTest.writeRecordsToPulsar failed +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29479'>FLINK-29479</a>] - Support whether using system PythonPath for PyFlink jobs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29539'>FLINK-29539</a>] - dnsPolicy in FlinkPod is not overridable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29615'>FLINK-29615</a>] - MetricStore does not remove metrics of nonexistent subtasks when adaptive scheduler lowers job parallelism +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29627'>FLINK-29627</a>] - Sink - Duplicate key exception during recover more than 1 committable. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29677'>FLINK-29677</a>] - Prevent dropping the current catalog +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29728'>FLINK-29728</a>] - TablePlanner prevents Flink from starting is working directory is a symbolic link +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29749'>FLINK-29749</a>] - flink info command support dynamic properties +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29781'>FLINK-29781</a>] - ChangelogNormalize uses wrong keys after transformation by WatermarkAssignerChangelogNormalizeTransposeRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29788'>FLINK-29788</a>] - StatefulJobWBroadcastStateMigrationITCase failed in native savepoints +</li> Review Comment: ```suggestion ``` test stability ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29479'>FLINK-29479</a>] - Support whether using system PythonPath for PyFlink jobs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29539'>FLINK-29539</a>] - dnsPolicy in FlinkPod is not overridable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29615'>FLINK-29615</a>] - MetricStore does not remove metrics of nonexistent subtasks when adaptive scheduler lowers job parallelism +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29627'>FLINK-29627</a>] - Sink - Duplicate key exception during recover more than 1 committable. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29677'>FLINK-29677</a>] - Prevent dropping the current catalog +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29728'>FLINK-29728</a>] - TablePlanner prevents Flink from starting is working directory is a symbolic link +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29749'>FLINK-29749</a>] - flink info command support dynamic properties +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29781'>FLINK-29781</a>] - ChangelogNormalize uses wrong keys after transformation by WatermarkAssignerChangelogNormalizeTransposeRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29788'>FLINK-29788</a>] - StatefulJobWBroadcastStateMigrationITCase failed in native savepoints +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29803'>FLINK-29803</a>] - Table API Scala APIs lack proper source jars +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29817'>FLINK-29817</a>] - Published metadata for apache-flink in pypi are inconsistent and causes poetry to fail +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29827'>FLINK-29827</a>] - [Connector][AsyncSinkWriter] Checkpointed states block writer from sending records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29830'>FLINK-29830</a>] - PulsarSinkITCase$DeliveryGuaranteeTest.writeRecordsToPulsar failed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29839'>FLINK-29839</a>] - HiveServer2 endpoint doesn't support TGetInfoType value 'CLI_ODBC_KEYWORDS' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29849'>FLINK-29849</a>] - Event time temporal join on an upsert source may produce incorrect execution plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29857'>FLINK-29857</a>] - Fix failure to connect to 'HiveServer2Endpoint' when using hive3 beeline +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29899'>FLINK-29899</a>] - Stacktrace printing in DefaultExecutionGraphCacheTest is confusing maven test log output +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29923'>FLINK-29923</a>] - Hybrid Shuffle may face deadlock when running a task need to execute big size data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29927'>FLINK-29927</a>] - AkkaUtils#getAddress may cause memory leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30030'>FLINK-30030</a>] - Unexpected behavior for overwrite in Hive dialect +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30133'>FLINK-30133</a>] - HadoopModuleFactory creates error if the security module cannot be loaded +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30138'>FLINK-30138</a>] - BlobServerCleanupTest utilizes failsWithin with a quite low timeout which might result in test instabilities +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30168'>FLINK-30168</a>] - PyFlink Deserialization Error with Object Array +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30189'>FLINK-30189</a>] - HsSubpartitionFileReader may load data that has been consumed from memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30239'>FLINK-30239</a>] - The flame graph doesn't work due to groupExecutionsByLocation has bug +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30304'>FLINK-30304</a>] - Possible Deadlock in Kinesis/Firehose/DynamoDB Connector +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30308'>FLINK-30308</a>] - ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.Map is showing in the logging when the job shutdown +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30334'>FLINK-30334</a>] - SourceCoordinator error splitRequest check cause HybridSource loss of data and hang +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30359'>FLINK-30359</a>] - Encountered NoClassDefFoundError when using flink-sql-connector-elasticsearch6 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30366'>FLINK-30366</a>] - Python Group Agg failed in cleaning the idle state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30525'>FLINK-30525</a>] - Cannot open jobmanager configuration web page +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30558'>FLINK-30558</a>] - The metric 'numRestarts' reported in SchedulerBase will be overridden by metric 'fullRestarts' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30637'>FLINK-30637</a>] - In linux-aarch64 environment, using “is” judgment to match the window type of overwindow have returned incorrect matching results +</li> +</ul> + +<h2> Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27327'>FLINK-27327</a>] - Add description about changing max parallelism explicitly leads to state incompatibility +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29134'>FLINK-29134</a>] - fetch metrics may cause oom(ThreadPool task pile up) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29155'>FLINK-29155</a>] - Improve default config of grpcServer in Process Mode +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29198'>FLINK-29198</a>] - RetryExtension doesn't make the test fail if the retries are exhausted +</li> Review Comment: ```suggestion ``` That's also more like an internal thing that only affects test stability. ########## q/gradle-quickstart.sh: ########## @@ -116,7 +116,7 @@ ext { scalaBinaryVersion = '${scalaBinaryVersion}' slf4jVersion = '1.7.32' Review Comment: ```suggestion slf4jVersion = '1.7.36' ``` if we want to fix these in general. ########## _posts/2023-01-26-release-1.16.1.md: ########## @@ -0,0 +1,258 @@ +--- +layout: post +title: "Apache Flink 1.16.1 Release Announcement" +date: 2023-01-23T22:00:00.000Z +categories: news +authors: +- MartijnVisser: + name: "Martijn Visser" + twitter: "martijnvisser82" + +excerpt: The Apache Flink Community is pleased to announce a bug fix release for Flink 1.16. + +--- + +The Apache Flink Community is pleased to announce the first bug fix release of the Flink 1.16 series. + +This release includes 84 bug fixes, vulnerability fixes, and minor improvements for Flink 1.16. +Below you will find a list of all bugfixes and improvements (excluding improvements to the build infrastructure and build stability). For a complete list of all changes see: +[JIRA](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12352344). + +We highly recommend all users upgrade to Flink 1.16.1. + +# Release Artifacts + +## Maven Dependencies + +```xml +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-streaming-java</artifactId> + <version>1.16.1</version> +</dependency> +<dependency> + <groupId>org.apache.flink</groupId> + <artifactId>flink-clients</artifactId> + <version>1.16.1</version> +</dependency> +``` + +## Binaries + +You can find the binaries on the updated [Downloads page]({{ site.baseurl }}/downloads.html). + +## Docker Images + +* [library/flink](https://hub.docker.com/_/flink?tab=tags&page=1&name=1.16.1) (official images) +* [apache/flink](https://hub.docker.com/r/apache/flink/tags?page=1&name=1.16.1) (ASF repository) + +## PyPi + +* [apache-flink==1.16.1](https://pypi.org/project/apache-flink/1.16.1/) + +# Upgrade Notes + +[FLINK-28988 - Incorrect result for filter after temporal join](https://issues.apache.org/jira/browse/FLINK-28988) +The filter will not be pushed down into both inputs of the event time temporal join. +This may cause incompatible plan changes compared to Flink 1.16.0, e.g., when the left input is an upsert source +(like upsert-kafka connector), the query plan will remove the ChangelogNormalize node in Flink 1.16.1, while it +did appear in 1.16.0. + +[FLINK-29849 - Event time temporal join on an upsert source may produce incorrect execution plan](https://issues.apache.org/jira/browse/FLINK-29849) +This resolves the correctness issue when do event time temporal join with a versioned table backed by an upsert source. +When the right input of the join is an upsert source, it no longer generates a ChangelogNormalize node for it. +This is an incompatible plan change compare to 1.16.0 + +[FLINK-30383 - UseLogicalIdentifier makes datadog consider metric as custom](https://issues.apache.org/jira/browse/FLINK-30383) +The Datadog reporter now adds a "flink." prefix to metric identifiers if "useLogicalIdentifier" is enabled. +This is required for these metrics to be recognized as Flink metrics, not custom ones. + +# Release Notes + +<h2> Bug +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-16582'>FLINK-16582</a>] - NettyBufferPoolTest may have warns on NettyBuffer leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-24119'>FLINK-24119</a>] - KafkaITCase.testTimestamps fails due to "Topic xxx already exist" +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26037'>FLINK-26037</a>] - TaskManagerRunner may crash during shutdown sequence +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-26890'>FLINK-26890</a>] - DynamoDB consumer error consuming partitions close to retention +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27341'>FLINK-27341</a>] - TaskManager running together with JobManager are bind to 127.0.0.1 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27691'>FLINK-27691</a>] - RankHarnessTest. testUpdateRankWithRowNumberSortKeyDropsToNotLast test failed with result mismatch +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27944'>FLINK-27944</a>] - IO metrics collision happens if a task has union inputs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28102'>FLINK-28102</a>] - Flink AkkaRpcSystemLoader fails when temporary directory is a symlink +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28526'>FLINK-28526</a>] - Fail to lateral join with UDTF from Table with timstamp column +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28695'>FLINK-28695</a>] - Fail to send partition request to restarted taskmanager +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28742'>FLINK-28742</a>] - Table.to_pandas fails with lit("xxx") +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28766'>FLINK-28766</a>] - UnalignedCheckpointStressITCase.runStressTest failed with NoSuchFileException +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28786'>FLINK-28786</a>] - Cannot run PyFlink 1.16 on MacOS with M1 chip +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28863'>FLINK-28863</a>] - Snapshot result of RocksDB native savepoint should have empty shared-state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28960'>FLINK-28960</a>] - Pulsar throws java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlElement +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-28988'>FLINK-28988</a>] - Incorrect result for filter after temporal join +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29092'>FLINK-29092</a>] - HadoopPathBasedPartFileWriterTest.testWriteFile failed with AssertionError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29217'>FLINK-29217</a>] - CoordinatorEventsToStreamOperatorRecipientExactlyOnceITCase.testConcurrentCheckpoint failed with AssertionFailedError +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29231'>FLINK-29231</a>] - PyFlink UDAF produces different results in the same sliding window +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29234'>FLINK-29234</a>] - Dead lock in DefaultLeaderElectionService +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29298'>FLINK-29298</a>] - LocalBufferPool request buffer from NetworkBufferPool hanging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29419'>FLINK-29419</a>] - HybridShuffleITCase.testHybridFullExchangesRestart hangs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29461'>FLINK-29461</a>] - ProcessDataStreamStreamingTests.test_process_function unstable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29479'>FLINK-29479</a>] - Support whether using system PythonPath for PyFlink jobs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29539'>FLINK-29539</a>] - dnsPolicy in FlinkPod is not overridable +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29615'>FLINK-29615</a>] - MetricStore does not remove metrics of nonexistent subtasks when adaptive scheduler lowers job parallelism +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29627'>FLINK-29627</a>] - Sink - Duplicate key exception during recover more than 1 committable. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29677'>FLINK-29677</a>] - Prevent dropping the current catalog +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29728'>FLINK-29728</a>] - TablePlanner prevents Flink from starting is working directory is a symbolic link +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29749'>FLINK-29749</a>] - flink info command support dynamic properties +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29781'>FLINK-29781</a>] - ChangelogNormalize uses wrong keys after transformation by WatermarkAssignerChangelogNormalizeTransposeRule +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29788'>FLINK-29788</a>] - StatefulJobWBroadcastStateMigrationITCase failed in native savepoints +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29803'>FLINK-29803</a>] - Table API Scala APIs lack proper source jars +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29817'>FLINK-29817</a>] - Published metadata for apache-flink in pypi are inconsistent and causes poetry to fail +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29827'>FLINK-29827</a>] - [Connector][AsyncSinkWriter] Checkpointed states block writer from sending records +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29830'>FLINK-29830</a>] - PulsarSinkITCase$DeliveryGuaranteeTest.writeRecordsToPulsar failed +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29839'>FLINK-29839</a>] - HiveServer2 endpoint doesn't support TGetInfoType value 'CLI_ODBC_KEYWORDS' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29849'>FLINK-29849</a>] - Event time temporal join on an upsert source may produce incorrect execution plan +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29857'>FLINK-29857</a>] - Fix failure to connect to 'HiveServer2Endpoint' when using hive3 beeline +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29899'>FLINK-29899</a>] - Stacktrace printing in DefaultExecutionGraphCacheTest is confusing maven test log output +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29923'>FLINK-29923</a>] - Hybrid Shuffle may face deadlock when running a task need to execute big size data +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29927'>FLINK-29927</a>] - AkkaUtils#getAddress may cause memory leak +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30030'>FLINK-30030</a>] - Unexpected behavior for overwrite in Hive dialect +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30133'>FLINK-30133</a>] - HadoopModuleFactory creates error if the security module cannot be loaded +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30138'>FLINK-30138</a>] - BlobServerCleanupTest utilizes failsWithin with a quite low timeout which might result in test instabilities +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30168'>FLINK-30168</a>] - PyFlink Deserialization Error with Object Array +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30189'>FLINK-30189</a>] - HsSubpartitionFileReader may load data that has been consumed from memory +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30239'>FLINK-30239</a>] - The flame graph doesn't work due to groupExecutionsByLocation has bug +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30304'>FLINK-30304</a>] - Possible Deadlock in Kinesis/Firehose/DynamoDB Connector +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30308'>FLINK-30308</a>] - ClassCastException: class java.io.ObjectStreamClass$Caches$1 cannot be cast to class java.util.Map is showing in the logging when the job shutdown +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30334'>FLINK-30334</a>] - SourceCoordinator error splitRequest check cause HybridSource loss of data and hang +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30359'>FLINK-30359</a>] - Encountered NoClassDefFoundError when using flink-sql-connector-elasticsearch6 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30366'>FLINK-30366</a>] - Python Group Agg failed in cleaning the idle state +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30525'>FLINK-30525</a>] - Cannot open jobmanager configuration web page +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30558'>FLINK-30558</a>] - The metric 'numRestarts' reported in SchedulerBase will be overridden by metric 'fullRestarts' +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30637'>FLINK-30637</a>] - In linux-aarch64 environment, using “is” judgment to match the window type of overwindow have returned incorrect matching results +</li> +</ul> + +<h2> Improvement +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27327'>FLINK-27327</a>] - Add description about changing max parallelism explicitly leads to state incompatibility +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29134'>FLINK-29134</a>] - fetch metrics may cause oom(ThreadPool task pile up) +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29155'>FLINK-29155</a>] - Improve default config of grpcServer in Process Mode +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29198'>FLINK-29198</a>] - RetryExtension doesn't make the test fail if the retries are exhausted +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29244'>FLINK-29244</a>] - Add metric lastMaterializationDuration to ChangelogMaterializationMetricGroup +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29458'>FLINK-29458</a>] - When two tables have the same field, do not specify the table name,Exception will be thrown：SqlValidatorException :Column 'currency' is ambiguous +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29639'>FLINK-29639</a>] - Add ResourceId in TransportException for debugging +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29693'>FLINK-29693</a>] - MiniClusterExtension should respect DEFAULT_PARALLELISM if set +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29834'>FLINK-29834</a>] - Clear static Jackson TypeFactory cache on CL release +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29966'>FLINK-29966</a>] - Replace and redesign the Python api documentation base +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30016'>FLINK-30016</a>] - Update Flink 1.16 release notes about updated oshi-core +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30116'>FLINK-30116</a>] - Don't Show Env Vars in Web UI +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30183'>FLINK-30183</a>] - We should add a proper error message in case the deprecated reflection-based instantiation of a reporter is triggered +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30357'>FLINK-30357</a>] - Wrong link in connector/jdbc doc. +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30436'>FLINK-30436</a>] - Integrate Opensearch connector docs into Flink docs v1.16 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30592'>FLINK-30592</a>] - The unsupported hive version is not deleted on the hive overview document +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30633'>FLINK-30633</a>] - Update AWS SDKv2 to v2.19.14 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30724'>FLINK-30724</a>] - Update doc of kafka per-partition watermark to FLIP-27 source +</li> +</ul> + +<h2> Technical Debt +</h2> +<ul> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-27731'>FLINK-27731</a>] - Remove Hugo Modules integration +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29157'>FLINK-29157</a>] - Clarify the contract between CompletedCheckpointStore and SharedStateRegistry +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29957'>FLINK-29957</a>] - Rework connector docs integration +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29958'>FLINK-29958</a>] - Add new connector_artifact shortcode +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29972'>FLINK-29972</a>] - Pin Flink docs to Elasticsearch Connector 3.0.0 +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-29973'>FLINK-29973</a>] - connector_artifact should append Flink minor version +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30291'>FLINK-30291</a>] - Integrate flink-connector-aws into Flink docs +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30382'>FLINK-30382</a>] - Flink 1.16 to integrate KDS/KDF docs from flink-connector-aws +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30383'>FLINK-30383</a>] - UseLogicalIdentifier makes datadog consider metric as custom +</li> +<li>[<a href='https://issues.apache.org/jira/browse/FLINK-30678'>FLINK-30678</a>] - NettyConnectionManagerTest.testManualConfiguration appears to be unstable +</li> Review Comment: ```suggestion ``` test stability ########## q/gradle-quickstart.sh: ########## @@ -116,7 +116,7 @@ ext { scalaBinaryVersion = '${scalaBinaryVersion}' slf4jVersion = '1.7.32' log4jVersion = '2.17.1' - flinkVersionNew = flinkVersion.toString().replace("-SNAPSHOT", "") >= "1.15" + flinkVersionNew = flinkVersion.toString().replace("-SNAPSHOT", "") >= "1.16" Review Comment: I created FLINK-30768 to improve the version updates in this script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
