[flink-web] 02/03: [FLINK-25748] tidy up source code section on community page
This is an automated email from the ASF dual-hosted git repository. knaufk pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit d63695a8efaaed7df85b7b969c1b7fb8679ff1f8 Author: Konstantin Knauf AuthorDate: Fri Jan 21 14:33:30 2022 +0100 [FLINK-25748] tidy up source code section on community page --- community.md| 48 ++-- community.zh.md | 40 2 files changed, 38 insertions(+), 50 deletions(-) diff --git a/community.md b/community.md index e7b6303..af5af0a 100644 --- a/community.md +++ b/community.md @@ -167,41 +167,31 @@ There are plenty of meetups on [meetup.com](http://www.meetup.com/topics/apache- ## Source Code -### Flink core repository +### Main Repositories -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink.git](https://gitbox.apache.org/repos/asf/flink.git) -- **GitHub mirror**: [https://github.com/apache/flink.git](https://github.com/apache/flink.git) +* **Flink Core Repository** +* ASF repository: [https://gitbox.apache.org/repos/asf/flink.git](https://gitbox.apache.org/repos/asf/flink.git) +* GitHub mirror: [https://github.com/apache/flink.git](https://github.com/apache/flink.git) -### Flink docker repository +* **Flink Docker Repository** +* ASF repository: [https://gitbox.apache.org/repos/asf/flink-docker.git](https://gitbox.apache.org/repos/asf/flink-docker.git) +* GitHub mirror: [https://github.com/apache/flink-docker.git](https://github.com/apache/flink-docker.git) -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink-docker.git](https://gitbox.apache.org/repos/asf/flink-docker.git) -- **GitHub mirror**: [https://github.com/apache/flink-docker.git](https://github.com/apache/flink-docker.git) +* **Flink Stateful Functions Repository** + * ASF repository: [https://gitbox.apache.org/repos/asf/flink-statefun.git](https://gitbox.apache.org/repos/asf/flink-statefun.git) + * GitHub mirror: [https://github.com/apache/flink-statefun](https://github.com/apache/flink-statefun) -### Flink Stateful Functions repository +* **Flink Stateful Functions Docker Repository** + * ASF repository: [https://gitbox.apache.org/repos/asf/flink-statefun-docker.git](https://gitbox.apache.org/repos/asf/flink-statefun-docker.git) + * GitHub mirror: [https://github.com/apache/flink-statefun-docker](https://github.com/apache/flink-statefun-docker) -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink-statefun.git](https://gitbox.apache.org/repos/asf/flink-statefun.git) -- **GitHub mirror**: [https://github.com/apache/flink-statefun](https://github.com/apache/flink-statefun) +* **Flink Website Repository** + * ASF repository: [https://gitbox.apache.org/repos/asf/flink-web.git](https://gitbox.apache.org/repos/asf/flink-web.git) + * GitHub mirror: [https://github.com/apache/flink-web.git](https://github.com/apache/flink-web.git) -### Flink Stateful Functions Docker repository - -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink-statefun-docker.git](https://gitbox.apache.org/repos/asf/flink-statefun-docker.git) -- **GitHub mirror**: [https://github.com/apache/flink-statefun-docker](https://github.com/apache/flink-statefun-docker) - -### Flink-shaded repositories (shaded dependency libraries) - -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink-shaded.git](https://gitbox.apache.org/repos/asf/flink-shaded.git) -- **GitHub mirror**: [https://github.com/apache/flink-shaded.git](https://github.com/apache/flink-shaded.git) - -### Flink Website repositories - -- **ASF repository**: [https://gitbox.apache.org/repos/asf/flink-web.git](https://gitbox.apache.org/repos/asf/flink-web.git) -- **GitHub mirror**: [https://github.com/apache/flink-web.git](https://github.com/apache/flink-web.git) - -### Bahir Flink repositories (additional connectors) - -- **ASF repository**: [https://git-wip-us.apache.org/repos/asf/bahir-flink.git](https://git-wip-us.apache.org/repos/asf/bahir-flink.git) -- **GitHub mirror**: [https://github.com/apache/bahir-flink.git](https://github.com/apache/bahir-flink.git) +### Complete List of Repositories +The complete list of repositories of Apache Flink can be found under https://gitbox.apache.org/repos/asf#flink. ## Training @@ -215,8 +205,6 @@ The Apache Flink https://cwiki.apache.org/confluence/display/FLINK/Apac Flink Forward is a conference happening yearly in different locations around the world. Up to date information about the conference is available on https://www.flink-forward.org/";>Flink-Forward.org. - - # People diff --git a/community.zh.md b/community.zh.md index 1e10ee1..e563079 100644 --- a/community.zh.md +++ b/community.zh.md @@ -165,35 +165,35 @@ Committer 们会关注 [Stack Overflow](http://stackoverflow.com/questions/tagge ## 源代码 -### 主仓库 +### Main Repositories -- **ASF 仓库**: [https://
[flink-web] 01/03: [hotfix] add Konstantin Knauf as PMC member
This is an automated email from the ASF dual-hosted git repository. knaufk pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 8567da8e6ef3610832d57644392299e3f31d24e6 Author: Konstantin Knauf AuthorDate: Fri Jan 21 14:10:46 2022 +0100 [hotfix] add Konstantin Knauf as PMC member --- community.md| 2 +- community.zh.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/community.md b/community.md index 1d1ff2c..e7b6303 100644 --- a/community.md +++ b/community.md @@ -337,7 +337,7 @@ Flink Forward is a conference happening yearly in different locations around the https://avatars0.githubusercontent.com/u/11538663?v=3&s=50"; class="committer-avatar"> Konstantin Knauf -Committer +PMC, Committer knaufk diff --git a/community.zh.md b/community.zh.md index 020cd9b..1e10ee1 100644 --- a/community.zh.md +++ b/community.zh.md @@ -322,7 +322,7 @@ Flink Forward 大会每年都会在世界的不同地方举办。关于大会最 https://avatars0.githubusercontent.com/u/11538663?v=3&s=50"; class="committer-avatar"> Konstantin Knauf -Committer +PMC, Committer knaufk
[flink-web] branch asf-site updated (104f831 -> 37b6c84)
This is an automated email from the ASF dual-hosted git repository. knaufk pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git. from 104f831 Rebuild website new 8567da8 [hotfix] add Konstantin Knauf as PMC member new d63695a [FLINK-25748] tidy up source code section on community page new 37b6c84 rebuild website The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: community.md | 50 +++ community.zh.md | 42 +++ content/community.html| 87 --- content/zh/community.html | 81 +-- 4 files changed, 117 insertions(+), 143 deletions(-)
[flink-web] 03/03: rebuild website
This is an automated email from the ASF dual-hosted git repository. knaufk pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 37b6c8401d21d729be6c1dfb13f6fe68ff21eb23 Author: Konstantin Knauf AuthorDate: Fri Jan 21 17:30:43 2022 +0100 rebuild website --- content/community.html| 87 --- content/zh/community.html | 81 +-- 2 files changed, 77 insertions(+), 91 deletions(-) diff --git a/content/community.html b/content/community.html index 1e73d36..96ce03e 100644 --- a/content/community.html +++ b/content/community.html @@ -210,13 +210,8 @@ Reporting Security Issues Meetups Source Code - Flink core repository - Flink docker repository - Flink Stateful Functions repository - Flink Stateful Functions Docker repository - Flink-shaded repositories (shaded dependency libraries) - Flink Website repositories - Bahir Flink repositories (additional connectors) + Main Repositories + Complete List of Repositories Training @@ -393,54 +388,44 @@ Source Code -Flink core repository +Main Repositories - ASF repository: https://gitbox.apache.org/repos/asf/flink.git";>https://gitbox.apache.org/repos/asf/flink.git - GitHub mirror: https://github.com/apache/flink.git";>https://github.com/apache/flink.git - - -Flink docker repository - - - ASF repository: https://gitbox.apache.org/repos/asf/flink-docker.git";>https://gitbox.apache.org/repos/asf/flink-docker.git - GitHub mirror: https://github.com/apache/flink-docker.git";>https://github.com/apache/flink-docker.git - - -Flink Stateful Functions repository - - - ASF repository: https://gitbox.apache.org/repos/asf/flink-statefun.git";>https://gitbox.apache.org/repos/asf/flink-statefun.git - GitHub mirror: https://github.com/apache/flink-statefun";>https://github.com/apache/flink-statefun - - -Flink Stateful Functions Docker repository - - - ASF repository: https://gitbox.apache.org/repos/asf/flink-statefun-docker.git";>https://gitbox.apache.org/repos/asf/flink-statefun-docker.git - GitHub mirror: https://github.com/apache/flink-statefun-docker";>https://github.com/apache/flink-statefun-docker - - -Flink-shaded repositories (shaded dependency libraries) - - - ASF repository: https://gitbox.apache.org/repos/asf/flink-shaded.git";>https://gitbox.apache.org/repos/asf/flink-shaded.git - GitHub mirror: https://github.com/apache/flink-shaded.git";>https://github.com/apache/flink-shaded.git - - -Flink Website repositories - - - ASF repository: https://gitbox.apache.org/repos/asf/flink-web.git";>https://gitbox.apache.org/repos/asf/flink-web.git - GitHub mirror: https://github.com/apache/flink-web.git";>https://github.com/apache/flink-web.git + Flink Core Repository + + ASF repository: https://gitbox.apache.org/repos/asf/flink.git";>https://gitbox.apache.org/repos/asf/flink.git + GitHub mirror: https://github.com/apache/flink.git";>https://github.com/apache/flink.git + + + Flink Docker Repository + + ASF repository: https://gitbox.apache.org/repos/asf/flink-docker.git";>https://gitbox.apache.org/repos/asf/flink-docker.git + GitHub mirror: https://github.com/apache/flink-docker.git";>https://github.com/apache/flink-docker.git + + + Flink Stateful Functions Repository + + ASF repository: https://gitbox.apache.org/repos/asf/flink-statefun.git";>https://gitbox.apache.org/repos/asf/flink-statefun.git + GitHub mirror: https://github.com/apache/flink-statefun";>https://github.com/apache/flink-statefun + + + Flink Stateful Functions Docker Repository + + ASF repository: https://gitbox.apache.org/repos/asf/flink-statefun-docker.git";>https://gitbox.apache.org/repos/asf/flink-statefun-docker.git + GitHub mirror: https://github.com/apache/flink-statefun-docker";>https://github.com/apache/flink-statefun-docker + + + Flink Website Repository + + ASF repository: https://gitbox.apache.org/repos/asf/flink-web.git";>https://gitbox.apache.org/repos/asf/flink-web.git + GitHub mirror: https://github.com/apache/flink-web.git";>https://github.com/apache/flink-web.git + + -Bahir Flink repositories (additional connectors) +Complete List of Repositories - - ASF repository: https://git-wip-us.apache.org/repos/asf/bahir-flink.git";>https://git-wip-us.apache.org/repos/asf/bahir-flink.git - GitHub mirror: https://github.com/apache/bahir-flink.git";>https://github.com/apache/bahir-flink.git - +The complete list of repositories of Apache Flink can be found under https://gitbox.apache.org/repos/asf#flink. Training @@ -574,7 +559,7 @@ https://avatars0.githubusercontent.com/u/11538663?v=3&s=50"; class="committer-avatar" /> Konstantin Knauf -Committer +PMC, Committer knaufk diff --git a/content/zh/community.html b/conten
[flink-benchmarks] branch master updated: [FLINK-25246] Parametrized executable for exec plugin
This is an automated email from the ASF dual-hosted git repository.
pnowojski pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-benchmarks.git
The following commit(s) were added to refs/heads/master by this push:
new 8f88957 [FLINK-25246] Parametrized executable for exec plugin
8f88957 is described below
commit 8f88957718e03535c77b63602d032836fbc5e26e
Author: Anton Kalashnikov
AuthorDate: Thu Jan 20 12:17:49 2022 +0100
[FLINK-25246] Parametrized executable for exec plugin
---
pom.xml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/pom.xml b/pom.xml
index 76e94a9..ecc9af1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -66,6 +66,7 @@ under the License.
dynamic
org.apache.flink.benchmark.full.*,org.apache.flink.state.benchmark.*,org.apache.flink.scheduler.benchmark.*
.*
+ java
@@ -275,7 +276,7 @@ under the License.
${skipTests}
test
-
java
+
${executableJava}
-Xmx6g
-classpath
@@ -325,7 +326,7 @@ under the License.
test
-
java
+
${executableJava}
-classpath
[flink] branch master updated (9aa879f -> 2e98b3f)
This is an automated email from the ASF dual-hosted git repository. pnowojski pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 9aa879f [FLINK-25683][table] Pass MAX_WATERMARK in InputConversionOperator add de5a699 [hotfix][pulsar] Use correct super types for reader and fetcher manager. add 2e98b3f [hotfix][source] Use only one watermark class in SourceOperatorEventTimeTest. No new revisions were added by this update. Summary of changes: .../pulsar/source/reader/fetcher/PulsarFetcherManagerBase.java | 4 ++-- .../pulsar/source/reader/source/PulsarSourceReaderBase.java| 4 ++-- .../api/operators/source/SourceOperatorEventTimeTest.java | 10 +++--- 3 files changed, 7 insertions(+), 11 deletions(-)
svn commit: r52209 - /release/flink/flink-shaded-13.0/
Author: chesnay Date: Fri Jan 21 14:38:19 2022 New Revision: 52209 Log: Remove flink-shaded 13.0 Removed: release/flink/flink-shaded-13.0/
svn commit: r52208 - /release/flink/flink-shaded-12.0/
Author: chesnay Date: Fri Jan 21 14:38:11 2022 New Revision: 52208 Log: Remove flink-shaded 12.0 Removed: release/flink/flink-shaded-12.0/
[flink-shaded] annotated tag release-15.0 updated (584b4ea -> 8d861cc)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to annotated tag release-15.0 in repository https://gitbox.apache.org/repos/asf/flink-shaded.git. *** WARNING: tag release-15.0 was modified! *** from 584b4ea (commit) to 8d861cc (tag) tagging afe714bacc61f3dcf7d85ae89ed7055928ec2c31 (tag) length 994 bytes by Chesnay Schepler on Fri Jan 21 15:35:21 2022 +0100 - Log - release-15.0 -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEGfIZXhtIFtdlosMkwu7XsRHUZLoFAmHqxKkACgkQwu7XsRHU ZLqTJRAAkU+wvSunPsbp31JMVwOFVqMBC/06dKEAxYqoqPfIchoxmqVkwyOQBgd/ /fZ3Bk2FBiuTRefZ1UZDIY/hSQasDaVFGvGStd7lQmnxhCUI0rDVi6h7ByufWt2y S9OoBPAyi1GuUZhxSL0IY+CXk+JOt8zOPQNx14WvS/py2NhIdjxwDhuNX0u/RAK7 hKQJJ1b+FAxgz0X10J8vuPHhMDfTXTriDy/Ok4tSpKrQvVgT4gZgm4n1i69M2ivL suay4QXZReZx8JJPApHxOgdpK+mUoRvZO9OSjCDSNF1mWoThwsGGDOrhzWIPaTZh alMqjuIPSrLYKm3/706TtojDJYdxx+kTUAJH5rkyEkub6WZ7MUa5+t/2nBBcuhY5 KGqMvD/eBjB+TtGN4rlSsSp9LnYyb0zBGNNt+IoPGL48v9Kmz6G9XABiX7nsIGDj RV7nSbNk+2+UWH9E9OftP/AdXCwB48QiVdqEkUOGVkgVA4nQEWcex1ievRTgFV1g 7W8j/GIreI+lLy2/cJZqAXgSJRx6Ok/e1+PzY2qaL4kyvDpNxruQ++GkvnfzodTH M8760giXl4YqEalnD3XRaZxbt2eF1t9x/1iy2p1eHe/7M2q1PG3aDYw/GDq4pbnM 1aVivE/HhH92/XnpMzJPsWEo2ZdgIiqRl4TASiErPYldFpVCoMw= =3xP0 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r52207 - /dev/flink/flink-shaded-15.0-rc2/ /release/flink/flink-shaded-15.0/
Author: chesnay Date: Fri Jan 21 14:34:08 2022 New Revision: 52207 Log: Release Flink-shaded 15.0 Added: release/flink/flink-shaded-15.0/ - copied from r52206, dev/flink/flink-shaded-15.0-rc2/ Removed: dev/flink/flink-shaded-15.0-rc2/
[flink] branch release-1.14 updated (303e306 -> 142a5ee)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch release-1.14 in repository https://gitbox.apache.org/repos/asf/flink.git. from 303e306 [FLINK-20188][Connectors][Docs][FileSystem] Added documentation for File Source add a033436 [FLINK-25674][connectors][cassandra][tests] Add drop tables to be idempotent in case of retrials and the related test add 142a5ee [FLINK-25674][connectors][cassandra][tests] use constants instead of string literals to avoid copy/paste No new revisions were added by this update. Summary of changes: .../cassandra/CassandraConnectorITCase.java| 96 +- 1 file changed, 77 insertions(+), 19 deletions(-)
[flink] branch master updated: [FLINK-25683][table] Pass MAX_WATERMARK in InputConversionOperator
This is an automated email from the ASF dual-hosted git repository.
dwysakowicz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 9aa879f [FLINK-25683][table] Pass MAX_WATERMARK in
InputConversionOperator
9aa879f is described below
commit 9aa879f50c32de862831e82613f1cf1bc4d760f9
Author: Yao Zhang
AuthorDate: Thu Jan 20 18:28:33 2022 +0800
[FLINK-25683][table] Pass MAX_WATERMARK in InputConversionOperator
MAX_WATERMARK emitted automatically has a special handling e.g. in BATCH
runtime mode. It flushes remaining records at the end of processing.
Therefore we should not discard those when converting to a Table
pipeline.
This closes #18405
---
.../runtime/stream/sql/DataStreamJavaITCase.java | 36 ++
.../operators/source/InputConversionOperator.java | 2 +-
.../source/InputConversionOperatorTest.java| 14 +
3 files changed, 51 insertions(+), 1 deletion(-)
diff --git
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/runtime/stream/sql/DataStreamJavaITCase.java
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/runtime/stream/sql/DataStreamJavaITCase.java
index aef4ebc..e8cbdf5 100644
---
a/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/runtime/stream/sql/DataStreamJavaITCase.java
+++
b/flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/runtime/stream/sql/DataStreamJavaITCase.java
@@ -22,6 +22,7 @@ import org.apache.flink.api.common.RuntimeExecutionMode;
import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.state.ValueState;
import org.apache.flink.api.common.state.ValueStateDescriptor;
+import org.apache.flink.api.common.typeinfo.TypeHint;
import org.apache.flink.api.common.typeinfo.TypeInformation;
import org.apache.flink.api.common.typeinfo.Types;
import org.apache.flink.api.java.tuple.Tuple2;
@@ -31,6 +32,8 @@ import org.apache.flink.api.java.typeutils.GenericTypeInfo;
import org.apache.flink.api.java.typeutils.TupleTypeInfo;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.datastream.DataStreamSource;
+import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.KeyedProcessFunction;
import org.apache.flink.streaming.api.functions.ProcessFunction;
@@ -577,6 +580,39 @@ public class DataStreamJavaITCase extends AbstractTestBase
{
}
@Test
+public void testTableStreamConversionBatch() throws Exception {
+env.setRuntimeMode(RuntimeExecutionMode.BATCH);
+
+DataStreamSource streamSource =
+env.fromElements(
+Row.of("Alice"),
+Row.of("alice"),
+Row.of("lily"),
+Row.of("Bob"),
+Row.of("lily"),
+Row.of("lily"));
+StreamTableEnvironment tableEnvironment =
StreamTableEnvironment.create(env);
+Table sourceTable =
tableEnvironment.fromDataStream(streamSource).as("word");
+tableEnvironment.createTemporaryView("tmp_table", sourceTable);
+Table resultTable = tableEnvironment.sqlQuery("select UPPER(word) as
word from tmp_table");
+SingleOutputStreamOperator> resultStream =
+tableEnvironment
+.toDataStream(resultTable)
+.map(row -> (String) row.getField("word"))
+.returns(TypeInformation.of(String.class))
+.map(s -> new Tuple2<>(s, 1))
+.returns(TypeInformation.of(new
TypeHint>() {}))
+.keyBy(tuple -> tuple.f0)
+.sum(1);
+
+testResult(
+resultStream,
+new Tuple2<>("ALICE", 2),
+new Tuple2<>("BOB", 1),
+new Tuple2<>("LILY", 3));
+}
+
+@Test
public void testComplexUnifiedPipelineStreaming() {
final Table resultTable = getComplexUnifiedPipeline(env);
diff --git
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/source/InputConversionOperator.java
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/source/InputConversionOperator.java
index 41bca4f..94b7fea 100644
---
a/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/source/InputConversionOperator.java
+++
b/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/source/InputConversionOperator.java
@@ -78,7 +78,
[flink] branch master updated (6c84a5b -> b72a5e0)
This is an automated email from the ASF dual-hosted git repository. roman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 6c84a5b [FLINK-25674][connectors][cassandra][tests] use constants instead of string literals to avoid copy/paste add b72a5e0 [FLINK-25678][runtime] Make TaskExecutorStateChangelogStoragesManager.shutdown thread-safe No new revisions were added by this update. Summary of changes: .../TaskExecutorStateChangelogStoragesManager.java | 106 - 1 file changed, 61 insertions(+), 45 deletions(-)
[flink] branch release-1.13 updated: [hotfix] Fix spotless violation
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.13 by this push:
new ee93467 [hotfix] Fix spotless violation
ee93467 is described below
commit ee934678be5cb8dba9e65875056c16c971205c05
Author: Chesnay Schepler
AuthorDate: Fri Jan 21 12:35:09 2022 +0100
[hotfix] Fix spotless violation
---
.../src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java
b/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java
index d2ecac1..06a12be 100644
---
a/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java
+++
b/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java
@@ -678,7 +678,7 @@ public class StreamGraph implements Pipeline {
typeNumber,
partitioner,
outputTag,
-shuffleMode,
+shuffleMode,
uniqueId);
getStreamNode(edge.getSourceId()).addOutEdge(edge);
[flink] branch master updated (3dbb497 -> 6c84a5b)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 3dbb497 [FLINK-20188][Connectors][Docs][FileSystem] Added documentation for File Source add 153bb9b [FLINK-25674][connectors][cassandra][tests] Add drop tables to be idempotent in case of retrials and the related test add 6c84a5b [FLINK-25674][connectors][cassandra][tests] use constants instead of string literals to avoid copy/paste No new revisions were added by this update. Summary of changes: .../cassandra/CassandraConnectorITCase.java| 96 +- 1 file changed, 77 insertions(+), 19 deletions(-)
[flink] branch master updated (0885ddc -> 3dbb497)
This is an automated email from the ASF dual-hosted git repository.
fpaul pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 0885ddc [FLINK-23944][test][pulsar] 1. change the Matcher to validate
both size and data 2. pulsar IT test generate deterministic data
add 3dbb497 [FLINK-20188][Connectors][Docs][FileSystem] Added
documentation for File Source
No new revisions were added by this update.
Summary of changes:
.../docs/connectors/datastream/file_sink.md| 748
.../docs/connectors/datastream/filesystem.md} | 357 --
.../docs/connectors/datastream/formats/_index.md | 0
.../docs/connectors/datastream/formats/avro.md | 0
.../datastream/formats/azure_table_storage.md | 4 +-
.../docs/connectors/datastream/formats/hadoop.md | 0
.../docs/connectors/datastream/formats}/mongodb.md | 4 +-
.../connectors/datastream/formats/overview.md} | 26 +-
.../docs/connectors/datastream/formats/parquet.md | 14 +-
.../connectors/datastream/formats/text_files.md| 8 +-
.../docs/connectors/datastream/overview.md | 2 +-
.../docs/connectors/datastream/streamfile_sink.md | 741
.../content.zh/docs/connectors/table/filesystem.md | 347 +
docs/content.zh/docs/deployment/filesystems/s3.md | 4 +-
.../docs/dev/datastream/execution_mode.md | 2 +-
.../datastream/{file_sink.md => filesystem.md} | 258 ++-
.../datastream/formats/azure_table_storage.md | 4 +-
.../datastream/formats/overview.md}| 26 +-
.../docs/connectors/datastream/formats/parquet.md | 14 +-
.../connectors/datastream/formats/text_files.md| 8 +-
.../content/docs/connectors/datastream/overview.md | 4 +-
.../docs/connectors/datastream/streamfile_sink.md | 776 -
docs/content/docs/connectors/table/filesystem.md | 4 +-
docs/content/docs/deployment/filesystems/s3.md | 7 +-
docs/content/docs/dev/datastream/execution_mode.md | 4 +-
25 files changed, 818 insertions(+), 2544 deletions(-)
delete mode 100644 docs/content.zh/docs/connectors/datastream/file_sink.md
copy docs/{content/docs/connectors/datastream/file_sink.md =>
content.zh/docs/connectors/datastream/filesystem.md} (68%)
copy docs/{content => content.zh}/docs/connectors/datastream/formats/_index.md
(100%)
copy docs/{content => content.zh}/docs/connectors/datastream/formats/avro.md
(100%)
copy docs/{content =>
content.zh}/docs/connectors/datastream/formats/azure_table_storage.md (98%)
copy docs/{content => content.zh}/docs/connectors/datastream/formats/hadoop.md
(100%)
copy docs/{content/docs/connectors/datastream =>
content.zh/docs/connectors/datastream/formats}/mongodb.md (91%)
copy docs/{content/docs/dev/table/sql/queries/limit.md =>
content.zh/docs/connectors/datastream/formats/overview.md} (60%)
copy docs/{content =>
content.zh}/docs/connectors/datastream/formats/parquet.md (92%)
copy docs/{content =>
content.zh}/docs/connectors/datastream/formats/text_files.md (91%)
delete mode 100644
docs/content.zh/docs/connectors/datastream/streamfile_sink.md
rename docs/content/docs/connectors/datastream/{file_sink.md => filesystem.md}
(75%)
copy docs/content/docs/{dev/table/sql/queries/limit.md =>
connectors/datastream/formats/overview.md} (60%)
delete mode 100644 docs/content/docs/connectors/datastream/streamfile_sink.md
[flink] branch master updated (172db40 -> 0885ddc)
This is an automated email from the ASF dual-hosted git repository. fpaul pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 172db40 [FLINK-25731][connectors/kinesis] Deprecated FlinkKinesisProducer add 0885ddc [FLINK-23944][test][pulsar] 1. change the Matcher to validate both size and data 2. pulsar IT test generate deterministic data No new revisions were added by this update. Summary of changes: .../pulsar/testutils/PulsarTestContext.java| 12 ++ .../test/common/utils/TestDataMatchers.java| 27 ++ 2 files changed, 24 insertions(+), 15 deletions(-)
[flink] branch release-1.13 updated: [FLINK-25199][network] Make sure StreamEdges are unique
This is an automated email from the ASF dual-hosted git repository.
pnowojski pushed a commit to branch release-1.13
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.13 by this push:
new 9c1bf6c [FLINK-25199][network] Make sure StreamEdges are unique
9c1bf6c is described below
commit 9c1bf6cdb3a2ea9d1b7f2b7d8e687a003ce9d919
Author: Piotr Nowojski
AuthorDate: Thu Jan 20 09:31:24 2022 +0100
[FLINK-25199][network] Make sure StreamEdges are unique
Previously, if there was a node that was self-unioned with itself,
it was creating a situation with two identical StreamEdges. Both
with the same partitioning, from the same source node to the same
target node.
This was causing issues when constructing output collectors and
picking the correct RecordWriters, as StreamTask was not able to
uniquely identify given StreamEdge and was assigning the same
RecordWriter to both of the edges. As a result all stream elements
were sent twice through the same RecordWriter. It was actually pretty
harmless apart of calculating the combined watermark downstream,
since all watermarks were always comming just from one single
edge/inputgate, and the unused edges were always stuck with
min watermark.
As a solution we are making sure that StreamEdges are unique
by introducing a uniqueId field, incremented for every pair
of StreamEdges connecting the same node.
---
.../flink/streaming/api/graph/StreamEdge.java | 34 ++-
.../flink/streaming/api/graph/StreamGraph.java | 103 +
.../flink/streaming/api/graph/StreamNode.java | 11 +++
.../test/streaming/runtime/TimestampITCase.java| 19
.../test/savepoint-921b4e-06351b8d4134/_metadata | Bin 0 -> 198 bytes
5 files changed, 123 insertions(+), 44 deletions(-)
diff --git
a/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamEdge.java
b/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamEdge.java
index f76edd5..2ea5671 100644
---
a/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamEdge.java
+++
b/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamEdge.java
@@ -44,6 +44,15 @@ public class StreamEdge implements Serializable {
private final int sourceId;
private final int targetId;
+/**
+ * Note that this field doesn't have to be unique among all {@link
StreamEdge}s. It's enough if
+ * this field ensures that all logical instances of {@link StreamEdge} are
unique, and {@link
+ * #hashCode()} are different and {@link #equals(Object)} returns false,
for every possible pair
+ * of {@link StreamEdge}. Especially among two different {@link
StreamEdge}s that are connecting
+ * the same pair of nodes.
+ */
+private final int uniqueId;
+
/** The type number of the input for co-tasks. */
private final int typeNumber;
/** The side-output tag (if any) of this {@link StreamEdge}. */
@@ -78,7 +87,8 @@ public class StreamEdge implements Serializable {
ALWAYS_FLUSH_BUFFER_TIMEOUT,
outputPartitioner,
outputTag,
-ShuffleMode.UNDEFINED);
+ShuffleMode.UNDEFINED,
+0);
}
public StreamEdge(
@@ -87,7 +97,8 @@ public class StreamEdge implements Serializable {
int typeNumber,
StreamPartitioner outputPartitioner,
OutputTag outputTag,
-ShuffleMode shuffleMode) {
+ShuffleMode shuffleMode,
+int uniqueId) {
this(
sourceVertex,
@@ -96,7 +107,8 @@ public class StreamEdge implements Serializable {
sourceVertex.getBufferTimeout(),
outputPartitioner,
outputTag,
-shuffleMode);
+shuffleMode,
+uniqueId);
}
public StreamEdge(
@@ -106,10 +118,12 @@ public class StreamEdge implements Serializable {
long bufferTimeout,
StreamPartitioner outputPartitioner,
OutputTag outputTag,
-ShuffleMode shuffleMode) {
+ShuffleMode shuffleMode,
+int uniqueId) {
this.sourceId = sourceVertex.getId();
this.targetId = targetVertex.getId();
+this.uniqueId = uniqueId;
this.typeNumber = typeNumber;
this.bufferTimeout = bufferTimeout;
this.outputPartitioner = outputPartitioner;
@@ -118,7 +132,15 @@ public class StreamEdge implements Serializable {
this.targetOperatorName = targetVertex.getOperatorName();
this.shuffleMode = checkNotNull(shuffleMode);
this.edgeId =
-sourceVertex + "_" + targetVertex + "_" + typeNumber + "_" +
outputPartitioner;
+sourceVertex
+
[flink] branch master updated (5192fd7 -> 172db40)
This is an automated email from the ASF dual-hosted git repository. dannycranmer pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git. from 5192fd7 [FLINK-25199][network] Make sure StreamEdges are unique add 172db40 [FLINK-25731][connectors/kinesis] Deprecated FlinkKinesisProducer No new revisions were added by this update. Summary of changes: .../flink/streaming/connectors/kinesis/FlinkKinesisProducer.java| 6 ++ 1 file changed, 6 insertions(+)
[flink] branch release-1.14 updated (bd1cd34 -> 444641c)
This is an automated email from the ASF dual-hosted git repository. pnowojski pushed a change to branch release-1.14 in repository https://gitbox.apache.org/repos/asf/flink.git. from bd1cd34 Update japicmp configuration for 1.14.3 add 444641c [FLINK-25199][network] Make sure StreamEdges are unique No new revisions were added by this update. Summary of changes: .../flink/streaming/api/graph/StreamEdge.java | 34 ++- .../flink/streaming/api/graph/StreamGraph.java | 103 + .../flink/streaming/api/graph/StreamNode.java | 11 +++ .../test/streaming/runtime/TimestampITCase.java| 19 4 files changed, 123 insertions(+), 44 deletions(-)
