(flink) branch master updated (012704d9884 -> d18a4bfe596)
This is an automated email from the ASF dual-hosted git repository. guoyangze pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 012704d9884 [FLINK-32922][docs] Update compatibility matrix for release 1.18 add 8ef71ba1f19 [FLINK-33532][rpc] Support configured akka remote dispatcher thread pool size add d18a4bfe596 [FLINK-33532][network] Move the serialization of ShuffleDescriptorGroup out of the RPC main thread] No new revisions were added by this update. Summary of changes: .../shortcodes/generated/akka_configuration.html | 18 +++ .../apache/flink/configuration/AkkaOptions.java| 32 .../rpc/pekko/ActorSystemBootstrapTools.java | 13 + .../apache/flink/runtime/rpc/pekko/PekkoUtils.java | 24 + .../flink/runtime/rpc/pekko/PekkoUtilsTest.java| 9 .../org/apache/flink/runtime/blob/BlobWriter.java | 11 ++-- .../deployment/CachedShuffleDescriptors.java | 2 +- .../deployment/InputGateDeploymentDescriptor.java | 41 +-- .../deployment/TaskDeploymentDescriptor.java | 19 +++ .../TaskDeploymentDescriptorFactory.java | 58 ++ .../deployment/CachedShuffleDescriptorsTest.java | 30 +-- .../TaskDeploymentDescriptorTestUtils.java | 9 ++-- .../partition/consumer/SingleInputGateTest.java| 6 +-- 13 files changed, 189 insertions(+), 83 deletions(-)
(flink) branch release-1.18 updated: [FLINK-32922][docs] Update compatibility matrix for release 1.18
This is an automated email from the ASF dual-hosted git repository. snuyanzin pushed a commit to branch release-1.18 in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/release-1.18 by this push: new f0c7c97ead7 [FLINK-32922][docs] Update compatibility matrix for release 1.18 f0c7c97ead7 is described below commit f0c7c97ead78efd2d5d48fd81c937c7488a787a9 Author: Sergey Nuyanzin AuthorDate: Thu Nov 16 00:26:32 2023 +0100 [FLINK-32922][docs] Update compatibility matrix for release 1.18 --- docs/content.zh/docs/ops/upgrading.md | 40 +++ docs/content/docs/ops/upgrading.md| 40 +++ 2 files changed, 80 insertions(+) diff --git a/docs/content.zh/docs/ops/upgrading.md b/docs/content.zh/docs/ops/upgrading.md index 6e6540bb649..322c00a566d 100644 --- a/docs/content.zh/docs/ops/upgrading.md +++ b/docs/content.zh/docs/ops/upgrading.md @@ -263,6 +263,7 @@ $ bin/flink run -s :savepointPath [:runArgs] 1.15.x 1.16.x 1.17.x + 1.18.x 限制 @@ -286,6 +287,7 @@ $ bin/flink run -s :savepointPath [:runArgs] + 从 Flink 1.1.x 迁移到 1.2.x+ 的作业的最大并行度目前固定为作业的并行度。 这意味着迁移后无法增加并行度。 在未来的错误修复版本中可能会删除此限制。 @@ -309,6 +311,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + 从 Flink 1.2.x 迁移到 Flink 1.3.x+ 时，不支持同时更改并行度。 迁移到 Flink 1.3.x+ 后， @@ -338,6 +341,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + M如果 Savepoint 包含 Scala 案例类，则从 Flink 1.3.0 迁移到 Flink 1.4.[0,1] 将失败。用户必须直接迁移到 1.4.2+。 @@ -359,6 +363,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -380,6 +385,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + 在 1.6.x 到 1.6.2 和 1.7.0 版本中恢复使用 1.5.x 创建的广播状态存在一个已知问题：https://issues.apache.org/jira/browse/FLINK-11087;>FLINK-11087. 升级到 1.6.x 或 1.7.x 系列的用户需要 直接迁移到次要版本分别高于 1.6.2 和 1.7.0。 @@ -403,6 +409,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -424,6 +431,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -445,6 +453,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -466,6 +475,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -487,6 +497,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -508,6 +519,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -529,6 +541,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -550,6 +563,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O Don't upgrade from 1.12.x to 1.13.x with an unaligned checkpoint. Please use a savepoint for migrating. @@ -571,6 +585,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -592,6 +607,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O For Table API: 1.15.0 and 1.15.1 generated non-deterministic UIDs for operators that make it difficult/impossible to restore state or upgrade to next patch version. A new @@ -621,6 +637,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + O @@ -642,6 +659,29 @@ $ bin/flink run -s :savepointPath [:runArgs] O + O + + + + 1.18.x + + + + + + + + + + + + + + + + + + O diff --git a/docs/content/docs/ops/upgrading.md b/docs/content/docs/ops/upgrading.md index c79bf808125..deb25bb876b 100644 --- a/docs/content/docs/ops/upgrading.md +++ b/docs/content/docs/ops/upgrading.md @@ -265,6 +265,7 @@ Savepoints are compatible across Flink versions as indicated by the table below: 1.15.x 1.16.x 1.17.x + 1.18.x Limitations @@ -288,6 +289,7 @@ Savepoints are compatible across Flink versions as indicated by the table below:
(flink) branch master updated: [FLINK-32922][docs] Update compatibility matrix for release 1.18
This is an automated email from the ASF dual-hosted git repository. snuyanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/master by this push: new 012704d9884 [FLINK-32922][docs] Update compatibility matrix for release 1.18 012704d9884 is described below commit 012704d9884f92274495fbf6fdb7234373944212 Author: Sergey Nuyanzin AuthorDate: Thu Nov 16 00:25:34 2023 +0100 [FLINK-32922][docs] Update compatibility matrix for release 1.18 --- docs/content.zh/docs/ops/upgrading.md | 40 +++ docs/content/docs/ops/upgrading.md| 40 +++ 2 files changed, 80 insertions(+) diff --git a/docs/content.zh/docs/ops/upgrading.md b/docs/content.zh/docs/ops/upgrading.md index 6e6540bb649..322c00a566d 100644 --- a/docs/content.zh/docs/ops/upgrading.md +++ b/docs/content.zh/docs/ops/upgrading.md @@ -263,6 +263,7 @@ $ bin/flink run -s :savepointPath [:runArgs] 1.15.x 1.16.x 1.17.x + 1.18.x 限制 @@ -286,6 +287,7 @@ $ bin/flink run -s :savepointPath [:runArgs] + 从 Flink 1.1.x 迁移到 1.2.x+ 的作业的最大并行度目前固定为作业的并行度。 这意味着迁移后无法增加并行度。 在未来的错误修复版本中可能会删除此限制。 @@ -309,6 +311,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + 从 Flink 1.2.x 迁移到 Flink 1.3.x+ 时，不支持同时更改并行度。 迁移到 Flink 1.3.x+ 后， @@ -338,6 +341,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + M如果 Savepoint 包含 Scala 案例类，则从 Flink 1.3.0 迁移到 Flink 1.4.[0,1] 将失败。用户必须直接迁移到 1.4.2+。 @@ -359,6 +363,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -380,6 +385,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + 在 1.6.x 到 1.6.2 和 1.7.0 版本中恢复使用 1.5.x 创建的广播状态存在一个已知问题：https://issues.apache.org/jira/browse/FLINK-11087;>FLINK-11087. 升级到 1.6.x 或 1.7.x 系列的用户需要 直接迁移到次要版本分别高于 1.6.2 和 1.7.0。 @@ -403,6 +409,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -424,6 +431,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + @@ -445,6 +453,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -466,6 +475,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -487,6 +497,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -508,6 +519,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -529,6 +541,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -550,6 +563,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O Don't upgrade from 1.12.x to 1.13.x with an unaligned checkpoint. Please use a savepoint for migrating. @@ -571,6 +585,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O @@ -592,6 +607,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O O + O For Table API: 1.15.0 and 1.15.1 generated non-deterministic UIDs for operators that make it difficult/impossible to restore state or upgrade to next patch version. A new @@ -621,6 +637,7 @@ $ bin/flink run -s :savepointPath [:runArgs] O O + O @@ -642,6 +659,29 @@ $ bin/flink run -s :savepointPath [:runArgs] O + O + + + + 1.18.x + + + + + + + + + + + + + + + + + + O diff --git a/docs/content/docs/ops/upgrading.md b/docs/content/docs/ops/upgrading.md index c79bf808125..deb25bb876b 100644 --- a/docs/content/docs/ops/upgrading.md +++ b/docs/content/docs/ops/upgrading.md @@ -265,6 +265,7 @@ Savepoints are compatible across Flink versions as indicated by the table below: 1.15.x 1.16.x 1.17.x + 1.18.x Limitations @@ -288,6 +289,7 @@ Savepoints are compatible across Flink versions as indicated by the table below:
(flink) branch release-1.18 updated: [FLINK-32913][tests] Updates base version for japicmp check for 1.18.0
This is an automated email from the ASF dual-hosted git repository. snuyanzin pushed a commit to branch release-1.18 in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/release-1.18 by this push: new 07067f3fe7f [FLINK-32913][tests] Updates base version for japicmp check for 1.18.0 07067f3fe7f is described below commit 07067f3fe7fc2353ceecaab0ce001c004d6fa247 Author: Sergey Nuyanzin AuthorDate: Wed Nov 15 22:58:37 2023 +0100 [FLINK-32913][tests] Updates base version for japicmp check for 1.18.0 --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 38a78c5d829..603e4532f54 100644 --- a/pom.xml +++ b/pom.xml @@ -171,7 +171,7 @@ under the License. 3.2.4 2.3.9 1.5.6 - 1.17.0 + 1.18.0 tools/japicmp-output 2.27.1 3.4.3 @@ -2234,7 +2234,7 @@ under the License. @org.apache.flink.annotation.Public - + @org.apache.flink.annotation.PublicEvolving @org.apache.flink.annotation.Experimental
(flink) branch master updated: [FLINK-32913][tests] Updates base version for japicmp check
This is an automated email from the ASF dual-hosted git repository. snuyanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/master by this push: new 5b8d7dfc7cf [FLINK-32913][tests] Updates base version for japicmp check 5b8d7dfc7cf is described below commit 5b8d7dfc7cfb31fcaa62cb5760e46047cab75fbd Author: Sergey Nuyanzin AuthorDate: Wed Nov 15 22:50:25 2023 +0100 [FLINK-32913][tests] Updates base version for japicmp check --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index b34958d8329..5d17a023358 100644 --- a/pom.xml +++ b/pom.xml @@ -173,7 +173,7 @@ under the License. 3.2.4 2.3.9 1.5.6 - 1.17.0 + 1.18.0 tools/japicmp-output 2.27.1 3.4.3
(flink) branch release-1.18 updated: [FLINK-33276][ci] Merges connect_1 and connect_2 stages into a single one
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch release-1.18
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.18 by this push:
new e53acef4374 [FLINK-33276][ci] Merges connect_1 and connect_2 stages
into a single one
e53acef4374 is described below
commit e53acef4374f97aeb2ac5ac491bd58c15347606c
Author: Matthias Pohl
AuthorDate: Tue Nov 14 16:31:29 2023 +0100
[FLINK-33276][ci] Merges connect_1 and connect_2 stages into a single one
---
tools/azure-pipelines/jobs-template.yml | 6 ++
tools/ci/stage.sh | 30 +-
2 files changed, 11 insertions(+), 25 deletions(-)
diff --git a/tools/azure-pipelines/jobs-template.yml
b/tools/azure-pipelines/jobs-template.yml
index 908f91f7e91..0fa043f9d10 100644
--- a/tools/azure-pipelines/jobs-template.yml
+++ b/tools/azure-pipelines/jobs-template.yml
@@ -95,10 +95,8 @@ jobs:
module: python
table:
module: table
- connect_1:
-module: connect_1
- connect_2:
-module: connect_2
+ connect:
+module: connect
tests:
module: tests
misc:
diff --git a/tools/ci/stage.sh b/tools/ci/stage.sh
index 60dbc98e2de..3bd1caac13d 100755
--- a/tools/ci/stage.sh
+++ b/tools/ci/stage.sh
@@ -21,8 +21,7 @@ STAGE_COMPILE="compile"
STAGE_CORE="core"
STAGE_PYTHON="python"
STAGE_TABLE="table"
-STAGE_CONNECTORS_1="connect_1"
-STAGE_CONNECTORS_2="connect_2"
+STAGE_CONNECTORS="connect"
STAGE_TESTS="tests"
STAGE_MISC="misc"
STAGE_CLEANUP="cleanup"
@@ -84,7 +83,7 @@ flink-table/flink-table-code-splitter,\
flink-table/flink-table-test-utils,\
"
-MODULES_CONNECTORS_1="\
+MODULES_CONNECTORS="\
flink-contrib/flink-connector-wikiedits,\
flink-filesystems,\
flink-filesystems/flink-azure-fs-hadoop,\
@@ -124,9 +123,6 @@ flink-metrics/flink-metrics-prometheus,\
flink-metrics/flink-metrics-statsd,\
flink-metrics/flink-metrics-datadog,\
flink-metrics/flink-metrics-slf4j,\
-"
-
-MODULES_CONNECTORS_2="\
flink-connectors/flink-connector-base,\
"
@@ -144,11 +140,8 @@ function get_compile_modules_for_stage() {
(${STAGE_TABLE})
echo "-pl $MODULES_TABLE -am"
;;
-(${STAGE_CONNECTORS_1})
-echo "-pl $MODULES_CONNECTORS_1 -am"
-;;
-(${STAGE_CONNECTORS_2})
-echo "-pl $MODULES_CONNECTORS_2 -am"
+(${STAGE_CONNECTORS})
+echo "-pl $MODULES_CONNECTORS -am"
;;
(${STAGE_TESTS})
echo "-pl $MODULES_TESTS -am"
@@ -170,15 +163,13 @@ function get_test_modules_for_stage() {
local modules_core=$MODULES_CORE
local modules_table=$MODULES_TABLE
-local modules_connectors_2=$MODULES_CONNECTORS_2
-local modules_connectors_1=$MODULES_CONNECTORS_1
+local modules_connectors=$MODULES_CONNECTORS
local modules_tests=$MODULES_TESTS
local negated_core=\!${MODULES_CORE//,/,\!}
local negated_table=\!${MODULES_TABLE//,/,\!}
-local negated_connectors_2=\!${MODULES_CONNECTORS_2//,/,\!}
-local negated_connectors_1=\!${MODULES_CONNECTORS_1//,/,\!}
+local negated_connectors=\!${MODULES_CONNECTORS//,/,\!}
local negated_tests=\!${MODULES_TESTS//,/,\!}
-local
modules_misc="$negated_core,$negated_table,$negated_connectors_1,$negated_connectors_2,$negated_tests"
+local
modules_misc="$negated_core,$negated_table,$negated_connectors,$negated_tests"
case ${stage} in
(${STAGE_CORE})
@@ -187,11 +178,8 @@ function get_test_modules_for_stage() {
(${STAGE_TABLE})
echo "-pl $modules_table"
;;
-(${STAGE_CONNECTORS_1})
-echo "-pl $modules_connectors_1"
-;;
-(${STAGE_CONNECTORS_2})
-echo "-pl $modules_connectors_2"
+(${STAGE_CONNECTORS})
+echo "-pl $modules_connectors"
;;
(${STAGE_TESTS})
echo "-pl $modules_tests"
svn commit: r65311 - in /dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2: ./ flink-connector-gcp-pubsub-3.0.2-src.tgz flink-connector-gcp-pubsub-3.0.2-src.tgz.asc flink-connector-gcp-pubsub-3.0.2-src.t
Author: dannycranmer Date: Wed Nov 15 15:40:35 2023 New Revision: 65311 Log: Add flink-connector-gcp-pubsub-3.0.2-rc2 Added: dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/ dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz (with props) dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.asc dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.sha512 Added: dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz == Binary file - no diff available. Propchange: dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz -- svn:mime-type = application/octet-stream Added: dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.asc == --- dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.asc (added) +++ dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.asc Wed Nov 15 15:40:35 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEED3nyr7I1G8KWeFRFkfnB7BJf2NsFAmVU5mwACgkQkfnB7BJf +2NutDxAAmNSLzl9IZcs31IjnzGQuEu5+Utc6xVfXQ/4e6zpR5DUxb1ju+1GBsfWi +X7o8HUsuOmQD1f4cMW/RU7XTUP+E0kgxa7jEjFJsqgHgEPai1+i5FnjVQWl/vGjD +g2817ONz0XU6ofL1AtOw3e7k5pIfG51yjfJ/2iuClMXuVLSTdiB/Ch0AghC5HHLa +mUfPMC3YhJ1YS16erVEDKMAIn1FalnknXoyzd8845U5LDpMSrq1P7YiY2BU7BfOy +9GCkFnDFIEIrYbJTVH3/CDLJN1NKXN6Vq5u9WTSDDk7mMXmwyBvVw+ghk0OVNGwz +J0/5Xok08berNfCcWmRQ59cI9AgOF4GRTqq/Z31ZoqMvtQt5tER3PgIDc28SyzvB +QEJ0moRpUvHHRfncEwfN1ySim4979pwXwxNNi2hJ9rTUkkny4f35TUs/dCEpDjRP +H2w9oQdsAA5OCV8U8q28lG+kkR9TA+uyPa5ygNwozoBO+pe8zh3UodjQAtRWkCq0 +nzQcANFdUtsk1xC8VaMKCX+zhk/Ezkq09O2aHlcDIFQrOW8XtkmBsl+qn8xotZ29 +SQGTVoY7aqK2Cl79MB++CvGeoFl6OMNT+ghXuxuqontJWCi7vTIc2z+rJSh4bRbH +jYMGbpxBgz5NfsE9P1riAFTcJ086f5lFfVoCslawd3ZL4cy1YdE= +=qFmB +-END PGP SIGNATURE- Added: dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.sha512 == --- dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.sha512 (added) +++ dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2/flink-connector-gcp-pubsub-3.0.2-src.tgz.sha512 Wed Nov 15 15:40:35 2023 @@ -0,0 +1 @@ +99a48285cd6adc7d4449f0370be25d5578c53c4108c6e4f4420a44cd089d932228f9210ceef7ce0b027526e9a6f667d55fa590342b382c066703312c3407531a flink-connector-gcp-pubsub-3.0.2-src.tgz
(flink-connector-gcp-pubsub) annotated tag v3.0.2-rc2 updated (3209c9c -> 9177b86)
This is an automated email from the ASF dual-hosted git repository. dannycranmer pushed a change to annotated tag v3.0.2-rc2 in repository https://gitbox.apache.org/repos/asf/flink-connector-gcp-pubsub.git *** WARNING: tag v3.0.2-rc2 was modified! *** from 3209c9c (commit) to 9177b86 (tag) tagging 3209c9cbba3d0450736254500e65cbd89df21dd8 (commit) by Danny Cranmer on Wed Nov 15 15:39:29 2023 + - Log - v3.0.2-rc2 -BEGIN PGP SIGNATURE- iQIzBAABCAAdFiEED3nyr7I1G8KWeFRFkfnB7BJf2NsFAmVU5jIACgkQkfnB7BJf 2NtHEQ/8C7ObFYEiD5kuOTuqGTznFD3tyOEcQLV8HISmVOUkTLw0TCQDIZ2U4bfb /9AM2d0LqJh0iLoMTbgv004WtxBIxjCSjFRWzDRzp2kYHMX7njIqoX1JRiFyfUCs WOYwPFa1cABa/RkC9MBd+vYeEn9Pect/al1HlPRGP2XRKn/uLAP7tJKHcC44fWvT UHwspCoupUPlMBOPdrlVcgjEB9cQI00RACERRzHa8MdXf7fi9RwysO/mkJ+b3thV xes9y2ToYvCcthTk1KwXHC9Sy7ho/uPUQBBtarWAUO4tGzpRBOwrRDBG/8se02lB iSZybPXIdjHHIGXhCkpASjfQnyZSecbHb7CHRVX3K42FFgkoR37abm6j0gCbzE5D KIJEkntJPR78Gmi+ntr4QZOpWcRqIEmAMjwK3nKaWElICvhqjmOet8eVViNQY4Jn P5ki3Q+KOpeZiUGUJYx26888/H0ceeJXCzm5Xd3A8l09nahs0SqQn29OtZrGOEIb +jrZ/rsJPkSqIWbt6ir0OOQ5KfAIAp5RExcSODMA6u4Ah2+2MDQoNYk81jKvmziG IM5CTY1OoBrLpWPfxVnXzUHxQpBO2iCT+DxVZzFKR7tchfI9bgL3VmiSPzS6aDUg uj3KGZH9QdF85C9arBwt1h3IUgJjSOeaQJIptB9eayToIVdtZeQ= =MUvV -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
(flink) branch master updated: [hotfix] Remove `bundled` from our connector documentation
This is an automated email from the ASF dual-hosted git repository.
martijnvisser pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new b3b4b8ef5f3 [hotfix] Remove `bundled` from our connector documentation
b3b4b8ef5f3 is described below
commit b3b4b8ef5f331de50dcb6f4c92ba21e2ef7fce44
Author: Martijn Visser
AuthorDate: Wed Nov 15 16:14:05 2023 +0100
[hotfix] Remove `bundled` from our connector documentation
The clarification necessary is that this talks about connectors as part of
the Apache Flink project, because they aren't necessarily bundled with the
Flink "main" project itself anymore
---
docs/content.zh/docs/connectors/datastream/overview.md | 5 +++--
docs/content/docs/connectors/datastream/overview.md| 7 +++
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/docs/content.zh/docs/connectors/datastream/overview.md
b/docs/content.zh/docs/connectors/datastream/overview.md
index 26694e174f3..be2bfbec5a7 100644
--- a/docs/content.zh/docs/connectors/datastream/overview.md
+++ b/docs/content.zh/docs/connectors/datastream/overview.md
@@ -33,9 +33,10 @@ under the License.
[预定义 data sources]({{< ref "docs/dev/datastream/overview" >}}#data-sources)
支持从文件、目录、socket，以及 collections 和 iterators 中读取数据。
[预定义 data sinks]({{< ref "docs/dev/datastream/overview" >}}#data-sinks)
支持把数据写入文件、标准输出（stdout）、标准错误输出（stderr）和 socket。
-## 附带的连接器
+## 附带的连接器Flink Project Connectors
-连接器可以和多种多样的第三方系统进行交互。目前支持以下系统:
+连接器可以和多种多样的第三方系统进行交互。目前支持以下系统。
+Currently these systems are supported as part of the Apache Flink project:
* [Apache Kafka]({{< ref "docs/connectors/datastream/kafka" >}}) (source/sink)
* [Apache Cassandra]({{< ref "docs/connectors/datastream/cassandra" >}})
(source/sink)
diff --git a/docs/content/docs/connectors/datastream/overview.md
b/docs/content/docs/connectors/datastream/overview.md
index a1b2cb9ccb5..a9afa3f5f90 100644
--- a/docs/content/docs/connectors/datastream/overview.md
+++ b/docs/content/docs/connectors/datastream/overview.md
@@ -34,9 +34,10 @@ The [predefined data sources]({{< ref
"docs/dev/datastream/overview" >}}#data-so
ingesting data from collections and iterators.
The [predefined data sinks]({{< ref "docs/dev/datastream/overview"
>}}#data-sinks) support writing to files, to stdout and stderr, and to sockets.
-## Bundled Connectors
+## Flink Project Connectors
-Connectors provide code for interfacing with various third-party systems.
Currently these systems are supported:
+Connectors provide code for interfacing with various third-party systems.
+Currently these systems are supported as part of the Apache Flink project:
* [Apache Kafka]({{< ref "docs/connectors/datastream/kafka" >}}) (source/sink)
* [Apache Cassandra]({{< ref "docs/connectors/datastream/cassandra" >}})
(source/sink)
@@ -60,8 +61,6 @@ Note also that while the streaming connectors listed in this
section are part of
Flink project and are included in source releases, they are not included in
the binary distributions.
Further instructions can be found in the corresponding subsections.
-Filesystem source formats are gradually replaced with new Flink Source API
starting with Flink 1.14.0.
-
## Connectors in Apache Bahir
Additional streaming connectors for Flink are being released through [Apache
Bahir](https://bahir.apache.org/), including:
(flink-kubernetes-operator) annotated tag release-1.7.0-rc1 updated (ccb10b83 -> 9b3b2f87)
This is an automated email from the ASF dual-hosted git repository. gyfora pushed a change to annotated tag release-1.7.0-rc1 in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git *** WARNING: tag release-1.7.0-rc1 was modified! *** from ccb10b83 (commit) to 9b3b2f87 (tag) tagging ccb10b8306cfa971ede61d73e1c47af4c60a (commit) by Gyula Fora on Wed Nov 15 15:44:23 2023 +0100 - Log - release-1.7.0-rc1 -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEC0o0rd/6K7VOtyCyIfBjA7h9r/EFAmVU2UcACgkQIfBjA7h9 r/HQ3Q/8CA5nhbiZzVvcwkp6+tGz2Xdp0fZATdVGsVKqdh/d69MUdEDVNTUQAFZe XFepp5afdJl9sZjE98/6PXTjRQXyA4/roXnmusvw2LiIaHIBMy9i5Af5AsHxsEY2 9+BmKH2pt6TFVjtVtNO3AkdE4c3L+w8FdqGlOavE/FoQXX5ziT3DQTk2IqKP3rx5 qrDxaC+/deoNqInxjsudyPl3R70fTOFCE2qenK4IPKqUsm5+lce/1skfZ+UmgoVv 1nFi32TYwASnEfrXkpcrsWs9P6sILgiSOgS/W7oJqOQqrrMd8QII9mrDeGmvdAwu XFne6/hR2sCh5eVcG0g17Cb1A0xGT1DhVgGqfpM5YKgbNZZJ43PonSDcgikM73Aw CqAd7iKYkFUA0Kc+71zNLpfc0y3M25JfaogRoBUPP5uuUyVBAM7SzfjGwlwWv6Rn 1T7RQVPRV4yZlvURCXOcpsiy6lSvW2PeymhLyUzgUFk9hxnG9uV9TqWg6V4mHqFe dWsn3lQ0qBEAcXn/003DxNhwraBG6t7MD7MqZkpbMECaDZT3iG9lMkj4nRaZDxAz P0RB7nPRDNuC8XDNLzU81e0Kpxi/5pv7NF7EDbVrnTufVgvbL/oyiOqwzl7wIkEq ljiYlhPlPpxcDxdUTchZoVNHg6KCsh5LdQO6rJ6kXbqOWqfWsts= =0QW1 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r65308 - /dev/flink/flink-kubernetes-operator-1.7.0-rc1/
Author: gyfora Date: Wed Nov 15 14:46:03 2023 New Revision: 65308 Log: Apache Flink Kubernetes Operator, version 1.7.0, release candidate 1 Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/ dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz (with props) dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.asc dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.sha512 dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz (with props) dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz.asc dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz.sha512 dev/flink/flink-kubernetes-operator-1.7.0-rc1/index.yaml dev/flink/flink-kubernetes-operator-1.7.0-rc1/index.yaml.asc dev/flink/flink-kubernetes-operator-1.7.0-rc1/index.yaml.sha512 Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz == Binary file - no diff available. Propchange: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz -- svn:mime-type = application/octet-stream Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.asc == --- dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.asc (added) +++ dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.asc Wed Nov 15 14:46:03 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEC0o0rd/6K7VOtyCyIfBjA7h9r/EFAmVU2VgACgkQIfBjA7h9 +r/G6BA//esYtOSf0MNyjFpFj1pBzY/pbiBaTChWJtOaaxemVdYIvtKBwKkmvOlS0 +0e5yvSKXN1/X16GMa1Qhy7pUWtx8qpR/pQdAzXXtSqM5tjWaZIXEXVZ6dKr4oBUO +YBbW1XQl9ktuQx/C9eQjYY8ehZjtKPrqm/Y3HbmiQwniBje6V5PdhekcRUH9VW+m +XV044DOwlCWvR4BOJkKMeiWhXwIyZy6Czd0riFV+AVQG566aB8xjlT9xMnbbAJD6 +Vd2KJZie2fjTGlcH0mQh1FIIcXgNWeZ2GJl86MRcU1NxkKvvhgA7zXA/Ce9Xz7iu +Hwl6b81iUX/WB3QNJiGj7eYDrC1yllKyWD6Aq54xLJVkgqRsfoprvQaJnUL+eDo6 +0dDHo2THARi6z9il4wgjDuJYYzjrSrtaNhKQiJjD48O4oRGOW5ZMBzd7HL4JdM6I +ldoN7ddJ9CWcP1mLILpL0gL1o0ZFHhy1oKB/KGjQIWvk/h3JLoLSPkmEVO9RN+W8 +WX+YkpWEnIEJHQsad9cWdBITzFiGRELh8BNKYYCjWo4wb/6gwQpoA7NtwZrTb2XX +EYbCkzjWM1d8knGPkSEtV46371nS5Yux3BMI3+bfrOsKDV6Mh8xI84pQDvT9fZaF +G/mYUD1J8ZpKqw4lmCzzbYRUI+8CdD02Z9C9pY0rtUkbhu4pmNs= +=6A0Y +-END PGP SIGNATURE- Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.sha512 == --- dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.sha512 (added) +++ dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-helm.tgz.sha512 Wed Nov 15 14:46:03 2023 @@ -0,0 +1 @@ +09e4a6c6fb311f3a52994c24c73987e0efedcf7fca9864ba4f30847fc6a8fd6056f772cc18e6b9667209b91ee1b5c1a608f431725026f5661e7076ea16248e7d flink-kubernetes-operator-1.7.0-helm.tgz Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz == Binary file - no diff available. Propchange: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz -- svn:mime-type = application/octet-stream Added: dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz.asc == --- dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz.asc (added) +++ dev/flink/flink-kubernetes-operator-1.7.0-rc1/flink-kubernetes-operator-1.7.0-src.tgz.asc Wed Nov 15 14:46:03 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEC0o0rd/6K7VOtyCyIfBjA7h9r/EFAmVU2VkACgkQIfBjA7h9 +r/ExNA/7BctkNt223o2WvzDhjP3AXVoQicrP85jEp2fgBEmUQAMQBevPQxWY7F8l +vBCYCO6YScphoBmiPj/92M00Z2itNRXvjZ5E3L1NjlpaLkTPEwU/S9PskR6Oig8K +Yfkg5eQ9LJHVW87Ptqb2/lAkUbM+1dhUFgMCerLQUT/ycMc2Y0YVpBphv1bB6rsB +HABh2LooKl6OIZC+YrzbM8B+Unuu0mTTAMhI1KAdBZAHElHjLsUQc+FOU03BKoNf +cPMLUgcxiCZZ4RxY6UzA8Tp4rMd1Quf8mX9Q4pKrk9qG9qRolYRWPrAvJIuWBH2q +yjHkF9NzaPLojBBmGKH5LBPOTIsXsAQaNS4tVuYOCRpRVqe9bnvB9MIJQ/bGo8dp +uqhGVVALT+EkUldj/TuZLkWYMqfks+Kxx6jqg80TQp6HNiZAW4604dDZVljj+rvL +C+IJjHTqvt0tc/4C7Dihzd1B9vyz5GrtasfNAaT0L195VMUFR42+vAMbTES5rfit +RekDsl4ZBJ6qcejGj5C7ZR97FDNp8Wt6IcWd5fVRzFv9HfMhic51BiW8UEYcttki +oVJ2KmUw6iQd8lYG44Jm/XPxVW0v8VbL0PdaCRUvSMzGUTAPvGoMRsDXl0sNOxaL +X1H1mp07hOcsm48WdoazWTtBHFLo3eV1u83EvMSY6ASayyrz16k= +=NZ0o +-END PGP
(flink-kubernetes-operator) branch main updated: [release] Update version to 1.8-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository.
gyfora pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 2c7c078d [release] Update version to 1.8-SNAPSHOT
2c7c078d is described below
commit 2c7c078dd029c24005471cf2a30725afd739d0db
Author: Gyula Fora
AuthorDate: Wed Nov 15 13:22:21 2023 +0100
[release] Update version to 1.8-SNAPSHOT
---
.asf.yaml | 1 +
.github/workflows/docs.yaml | 6 +++---
Dockerfile | 2 +-
docs/config.toml| 8
examples/autoscaling/pom.xml| 2 +-
examples/flink-beam-example/pom.xml | 2 +-
examples/flink-sql-runner-example/pom.xml | 2 +-
examples/kubernetes-client-examples/pom.xml | 4 ++--
flink-autoscaler-standalone/pom.xml | 2 +-
flink-autoscaler/pom.xml| 2 +-
flink-kubernetes-docs/pom.xml | 2 +-
flink-kubernetes-operator-api/pom.xml | 2 +-
flink-kubernetes-operator/pom.xml | 2 +-
flink-kubernetes-standalone/pom.xml | 2 +-
flink-kubernetes-webhook/pom.xml| 2 +-
helm/flink-kubernetes-operator/Chart.yaml | 4 ++--
pom.xml | 2 +-
17 files changed, 24 insertions(+), 23 deletions(-)
diff --git a/.asf.yaml b/.asf.yaml
index f6104dcb..7d48cb75 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -17,6 +17,7 @@ github:
release-1.4: {}
release-1.5: {}
release-1.6: {}
+release-1.7: {}
notifications:
diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
index 1df300fa..4068df96 100644
--- a/.github/workflows/docs.yaml
+++ b/.github/workflows/docs.yaml
@@ -28,8 +28,8 @@ jobs:
matrix:
branch:
- main
- - release-1.5
- release-1.6
+ - release-1.7
steps:
- uses: actions/checkout@v3
with:
@@ -41,8 +41,8 @@ jobs:
echo "flink_branch=${currentBranch}"
echo "flink_branch=${currentBranch}" >> ${GITHUB_ENV}
if [ "${currentBranch}" = "main" ]; then
-echo "flink_alias=release-1.7" >> ${GITHUB_ENV}
- elif [ "${currentBranch}" = "release-1.6" ]; then
+echo "flink_alias=release-1.8" >> ${GITHUB_ENV}
+ elif [ "${currentBranch}" = "release-1.7" ]; then
echo "flink_alias=stable" >> ${GITHUB_ENV}
else
echo "flink_alias=${currentBranch}" >> ${GITHUB_ENV}
diff --git a/Dockerfile b/Dockerfile
index 0d98ec1f..3ee36da2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -36,7 +36,7 @@ RUN cd /app/tools/license; mkdir jars; cd jars; \
FROM eclipse-temurin:11-jre-jammy
ENV FLINK_HOME=/opt/flink
ENV FLINK_PLUGINS_DIR=$FLINK_HOME/plugins
-ENV OPERATOR_VERSION=1.7-SNAPSHOT
+ENV OPERATOR_VERSION=1.8-SNAPSHOT
ENV OPERATOR_JAR=flink-kubernetes-operator-$OPERATOR_VERSION-shaded.jar
ENV WEBHOOK_JAR=flink-kubernetes-webhook-$OPERATOR_VERSION-shaded.jar
ENV KUBERNETES_STANDALONE_JAR=flink-kubernetes-standalone-$OPERATOR_VERSION.jar
diff --git a/docs/config.toml b/docs/config.toml
index 92faf507..4b5317ed 100644
--- a/docs/config.toml
+++ b/docs/config.toml
@@ -34,11 +34,11 @@ pygmentsUseClasses = true
# we change the version for the complete docs when forking of a release
branch
# etc.
# The full version string as referenced in Maven (e.g. 1.2.1)
- Version = "1.7-SNAPSHOT"
+ Version = "1.8-SNAPSHOT"
# For stable releases, leave the bugfix version out (e.g. 1.2). For snapshot
# release this should be the same as the regular version
- VersionTitle = "1.7-SNAPSHOT"
+ VersionTitle = "1.8-SNAPSHOT"
# The branch for this version of the Apache Flink Kubernetes Operator
Branch = "main"
@@ -63,8 +63,8 @@ pygmentsUseClasses = true
]
PreviousDocs = [
-["1.6",
"https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.6;],
-["1.5",
"https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.5;]
+["1.7",
"https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.7;],
+["1.6",
"https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.6;]
]
[markup]
diff --git a/examples/autoscaling/pom.xml b/examples/autoscaling/pom.xml
index f4bea0ac..9626cf2a 100644
--- a/examples/autoscaling/pom.xml
+++ b/examples/autoscaling/pom.xml
@@ -23,7 +23,7 @@ under the License.
org.apache.flink
flink-kubernetes-operator-parent
-1.7-SNAPSHOT
+1.8-SNAPSHOT
../..
diff --git a/examples/flink-beam-example/pom.xml
b/examples/flink-beam-example/pom.xml
index b215f2dc..8a69a2a2 100644
--- a/examples/flink-beam-example/pom.xml
+++ b/examples/flink-beam-example/pom.xml
@@ -23,7 +23,7 @@ under the License.
org.apache.flink
(flink-kubernetes-operator) 01/01: [release] Update docs config for release
This is an automated email from the ASF dual-hosted git repository. gyfora pushed a commit to branch release-1.7 in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git commit b081b75b72ddde643710e869b95b214912882363 Author: Gyula Fora AuthorDate: Wed Nov 15 13:25:35 2023 +0100 [release] Update docs config for release --- docs/config.toml | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/config.toml b/docs/config.toml index 92faf507..7d46ead0 100644 --- a/docs/config.toml +++ b/docs/config.toml @@ -14,7 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -baseURL = '//nightlies.apache.org/flink/flink-kubernetes-operator-docs-main' +baseURL = '//nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.7' languageCode = 'en-us' title = 'Apache Flink Kubernetes Operator' enableGitInfo = false @@ -24,7 +24,7 @@ pygmentsUseClasses = true [params] # Flag whether this is a stable version or not. # Used for the quickstart page. - IsStable = false + IsStable = true # Flag to indicate whether an outdated warning should be shown. ShowOutDatedWarning = false @@ -34,14 +34,14 @@ pygmentsUseClasses = true # we change the version for the complete docs when forking of a release branch # etc. # The full version string as referenced in Maven (e.g. 1.2.1) - Version = "1.7-SNAPSHOT" + Version = "1.7.0" # For stable releases, leave the bugfix version out (e.g. 1.2). For snapshot # release this should be the same as the regular version - VersionTitle = "1.7-SNAPSHOT" + VersionTitle = "1.7" # The branch for this version of the Apache Flink Kubernetes Operator - Branch = "main" + Branch = "release-1.7" # The github repository for the Apache Flink Kubernetes Operator Repo = "//github.com/apache/flink-kubernetes-operator" @@ -59,7 +59,7 @@ pygmentsUseClasses = true # of the menu MenuLinks = [ ["Project Homepage", "//flink.apache.org"], -["JavaDocs", "//nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/api/java/"] +["JavaDocs", "//nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.7/api/java/"] ] PreviousDocs = [
(flink-kubernetes-operator) branch release-1.7 created (now b081b75b)
This is an automated email from the ASF dual-hosted git repository. gyfora pushed a change to branch release-1.7 in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git at b081b75b [release] Update docs config for release This branch includes the following new commits: new b081b75b [release] Update docs config for release The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(flink-kubernetes-operator) branch main updated: [FLINK-33527] Simplify state store cleanup logic (#710)
This is an automated email from the ASF dual-hosted git repository.
gyfora pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 02840b96 [FLINK-33527] Simplify state store cleanup logic (#710)
02840b96 is described below
commit 02840b96ef3116ea95a440af4f945398900d89df
Author: Maximilian Michels
AuthorDate: Wed Nov 15 15:16:41 2023 +0100
[FLINK-33527] Simplify state store cleanup logic (#710)
---
.../apache/flink/autoscaler/JobAutoScalerImpl.java | 38 +--
.../autoscaler/state/AutoScalerStateStore.java | 9 +-
.../state/InMemoryAutoScalerStateStore.java| 7 ++
.../flink/autoscaler/BacklogBasedScalingTest.java | 9 +-
.../flink/autoscaler/JobAutoScalerImplTest.java| 71 +++-
.../autoscaler/RecommendedParallelismTest.java | 9 +-
.../state/TestingAutoscalerStateStore.java | 37 --
.../operator/autoscaler/AutoscalerFactory.java | 2 +
.../autoscaler/{ => state}/ConfigMapStore.java | 99 +---
.../operator/autoscaler/state/ConfigMapView.java | 117 +++
.../KubernetesAutoScalerStateStore.java| 8 +-
.../KubernetesAutoScalerEventHandlerTest.java | 2 +
.../autoscaler/{ => state}/ConfigMapStoreTest.java | 126 ++---
.../KubernetesAutoScalerStateStoreTest.java| 43 ++-
14 files changed, 351 insertions(+), 226 deletions(-)
diff --git
a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java
b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java
index 0e2afb55..1cfdba62 100644
---
a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java
+++
b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/JobAutoScalerImpl.java
@@ -42,7 +42,6 @@ import java.util.concurrent.ConcurrentHashMap;
import static
org.apache.flink.autoscaler.config.AutoScalerOptions.AUTOSCALER_ENABLED;
import static
org.apache.flink.autoscaler.metrics.AutoscalerFlinkMetrics.initRecommendedParallelism;
import static
org.apache.flink.autoscaler.metrics.AutoscalerFlinkMetrics.resetRecommendedParallelism;
-import static
org.apache.flink.autoscaler.metrics.ScalingHistoryUtils.trimScalingHistory;
/** The default implementation of {@link JobAutoScaler}. */
public class JobAutoScalerImpl>
@@ -90,7 +89,8 @@ public class JobAutoScalerImpl>
try {
if (!ctx.getConfiguration().getBoolean(AUTOSCALER_ENABLED)) {
LOG.debug("Autoscaler is disabled");
-clearStatesAfterAutoscalerDisabled(ctx);
+stateStore.clearAll(ctx);
+stateStore.flush(ctx);
return;
}
@@ -120,40 +120,6 @@ public class JobAutoScalerImpl>
stateStore.removeInfoFromCache(jobKey);
}
-private void clearStatesAfterAutoscalerDisabled(Context ctx) throws
Exception {
-var needFlush = false;
-var parallelismOverrides = stateStore.getParallelismOverrides(ctx);
-if (!parallelismOverrides.isEmpty()) {
-needFlush = true;
-stateStore.removeParallelismOverrides(ctx);
-}
-
-var collectedMetrics = stateStore.getCollectedMetrics(ctx);
-if (!collectedMetrics.isEmpty()) {
-needFlush = true;
-stateStore.removeCollectedMetrics(ctx);
-}
-
-var scalingHistory = stateStore.getScalingHistory(ctx);
-if (!scalingHistory.isEmpty()) {
-var trimmedScalingHistory =
-trimScalingHistory(clock.instant(),
ctx.getConfiguration(), scalingHistory);
-if (trimmedScalingHistory.isEmpty()) {
-// All scaling histories are trimmed.
-needFlush = true;
-stateStore.removeScalingHistory(ctx);
-} else if (!scalingHistory.equals(trimmedScalingHistory)) {
-// Some scaling histories are trimmed.
-needFlush = true;
-stateStore.storeScalingHistory(ctx, trimmedScalingHistory);
-}
-}
-
-if (needFlush) {
-stateStore.flush(ctx);
-}
-}
-
@VisibleForTesting
protected Map getParallelismOverrides(Context ctx) throws
Exception {
return stateStore.getParallelismOverrides(ctx);
diff --git
a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/state/AutoScalerStateStore.java
b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/state/AutoScalerStateStore.java
index bb3329a4..6f9e54e2 100644
---
a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/state/AutoScalerStateStore.java
+++
b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/state/AutoScalerStateStore.java
@@ -64,10 +64,13 @@ public interface AutoScalerStateStore context;
private TestingEventCollector>
(flink) branch master updated: [hotfix] Add TestTemplate to fix build
This is an automated email from the ASF dual-hosted git repository.
snuyanzin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 1cf7ec41d62 [hotfix] Add TestTemplate to fix build
1cf7ec41d62 is described below
commit 1cf7ec41d62d59e500bf0cefd35315070abb2d73
Author: Sergey Nuyanzin
AuthorDate: Wed Nov 15 14:39:07 2023 +0100
[hotfix] Add TestTemplate to fix build
---
.../flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
b/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
index 6feab3b9f6f..9850e1afe9c 100644
---
a/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
+++
b/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
@@ -217,7 +217,7 @@ class DistinctAggregateTest(
util.verifyRelPlan(sqlQuery, ExplainDetail.CHANGELOG_MODE)
}
- @Test
+ @TestTemplate
def testListAggWithDistinctMultiArgs(): Unit = {
util.verifyExecPlan("SELECT a, LISTAGG(DISTINCT c, '#') FROM MyTable GROUP
BY a")
}
(flink) branch master updated: [FLINK-18356] Update CI image
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/master by this push: new 499e56f138f [FLINK-18356] Update CI image 499e56f138f is described below commit 499e56f138fb4e81cbb8810385cfb393d16ea454 Author: Chesnay Schepler AuthorDate: Wed Nov 15 10:48:56 2023 +0100 [FLINK-18356] Update CI image --- .github/workflows/docs.yml | 2 +- tools/azure-pipelines/build-apache-repo.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 26754d2c1e6..ed71b759ead 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -48,7 +48,7 @@ jobs: fi - name: Build documentation run: | - docker run --rm --volume "$PWD:/root/flink" chesnay/flink-ci:java_8_11_17_maven_386_v2 bash -c "cd /root/flink && ./.github/workflows/docs.sh" + docker run --rm --volume "$PWD:/root/flink" chesnay/flink-ci:java_8_11_17_maven_386_v3 bash -c "cd /root/flink && ./.github/workflows/docs.sh" - name: Upload documentation uses: burnett01/rsync-deployments@5.2 with: diff --git a/tools/azure-pipelines/build-apache-repo.yml b/tools/azure-pipelines/build-apache-repo.yml index 892d2a519dd..15551710402 100644 --- a/tools/azure-pipelines/build-apache-repo.yml +++ b/tools/azure-pipelines/build-apache-repo.yml @@ -39,7 +39,7 @@ resources: # Container with Maven 3.8.6, SSL to have the same environment everywhere. # see https://github.com/apache/flink-connector-shared-utils/tree/ci_utils - container: flink-build-container -image: chesnay/flink-ci:java_8_11_17_maven_386_v2 +image: chesnay/flink-ci:java_8_11_17_maven_386_v3 variables: MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
(flink) branch master updated: [FLINK-33276][ci] Merges connect_1 and connect_2 stages into a single one
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new e06267f293c [FLINK-33276][ci] Merges connect_1 and connect_2 stages
into a single one
e06267f293c is described below
commit e06267f293cfa7f708fa8a45b914b275017b2110
Author: Matthias Pohl
AuthorDate: Tue Nov 14 16:31:29 2023 +0100
[FLINK-33276][ci] Merges connect_1 and connect_2 stages into a single one
---
tools/azure-pipelines/jobs-template.yml | 6 ++
tools/ci/stage.sh | 30 +-
2 files changed, 11 insertions(+), 25 deletions(-)
diff --git a/tools/azure-pipelines/jobs-template.yml
b/tools/azure-pipelines/jobs-template.yml
index 908f91f7e91..0fa043f9d10 100644
--- a/tools/azure-pipelines/jobs-template.yml
+++ b/tools/azure-pipelines/jobs-template.yml
@@ -95,10 +95,8 @@ jobs:
module: python
table:
module: table
- connect_1:
-module: connect_1
- connect_2:
-module: connect_2
+ connect:
+module: connect
tests:
module: tests
misc:
diff --git a/tools/ci/stage.sh b/tools/ci/stage.sh
index 60dbc98e2de..3bd1caac13d 100755
--- a/tools/ci/stage.sh
+++ b/tools/ci/stage.sh
@@ -21,8 +21,7 @@ STAGE_COMPILE="compile"
STAGE_CORE="core"
STAGE_PYTHON="python"
STAGE_TABLE="table"
-STAGE_CONNECTORS_1="connect_1"
-STAGE_CONNECTORS_2="connect_2"
+STAGE_CONNECTORS="connect"
STAGE_TESTS="tests"
STAGE_MISC="misc"
STAGE_CLEANUP="cleanup"
@@ -84,7 +83,7 @@ flink-table/flink-table-code-splitter,\
flink-table/flink-table-test-utils,\
"
-MODULES_CONNECTORS_1="\
+MODULES_CONNECTORS="\
flink-contrib/flink-connector-wikiedits,\
flink-filesystems,\
flink-filesystems/flink-azure-fs-hadoop,\
@@ -124,9 +123,6 @@ flink-metrics/flink-metrics-prometheus,\
flink-metrics/flink-metrics-statsd,\
flink-metrics/flink-metrics-datadog,\
flink-metrics/flink-metrics-slf4j,\
-"
-
-MODULES_CONNECTORS_2="\
flink-connectors/flink-connector-base,\
"
@@ -144,11 +140,8 @@ function get_compile_modules_for_stage() {
(${STAGE_TABLE})
echo "-pl $MODULES_TABLE -am"
;;
-(${STAGE_CONNECTORS_1})
-echo "-pl $MODULES_CONNECTORS_1 -am"
-;;
-(${STAGE_CONNECTORS_2})
-echo "-pl $MODULES_CONNECTORS_2 -am"
+(${STAGE_CONNECTORS})
+echo "-pl $MODULES_CONNECTORS -am"
;;
(${STAGE_TESTS})
echo "-pl $MODULES_TESTS -am"
@@ -170,15 +163,13 @@ function get_test_modules_for_stage() {
local modules_core=$MODULES_CORE
local modules_table=$MODULES_TABLE
-local modules_connectors_2=$MODULES_CONNECTORS_2
-local modules_connectors_1=$MODULES_CONNECTORS_1
+local modules_connectors=$MODULES_CONNECTORS
local modules_tests=$MODULES_TESTS
local negated_core=\!${MODULES_CORE//,/,\!}
local negated_table=\!${MODULES_TABLE//,/,\!}
-local negated_connectors_2=\!${MODULES_CONNECTORS_2//,/,\!}
-local negated_connectors_1=\!${MODULES_CONNECTORS_1//,/,\!}
+local negated_connectors=\!${MODULES_CONNECTORS//,/,\!}
local negated_tests=\!${MODULES_TESTS//,/,\!}
-local
modules_misc="$negated_core,$negated_table,$negated_connectors_1,$negated_connectors_2,$negated_tests"
+local
modules_misc="$negated_core,$negated_table,$negated_connectors,$negated_tests"
case ${stage} in
(${STAGE_CORE})
@@ -187,11 +178,8 @@ function get_test_modules_for_stage() {
(${STAGE_TABLE})
echo "-pl $modules_table"
;;
-(${STAGE_CONNECTORS_1})
-echo "-pl $modules_connectors_1"
-;;
-(${STAGE_CONNECTORS_2})
-echo "-pl $modules_connectors_2"
+(${STAGE_CONNECTORS})
+echo "-pl $modules_connectors"
;;
(${STAGE_TESTS})
echo "-pl $modules_tests"
(flink-connector-shared-utils) branch parent_pom updated: [FLINK-32563] add additionalExcludes property to add exclusions to surefire tests
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch parent_pom
in repository
https://gitbox.apache.org/repos/asf/flink-connector-shared-utils.git
The following commit(s) were added to refs/heads/parent_pom by this push:
new 808184a [FLINK-32563] add additionalExcludes property to add
exclusions to surefire tests
808184a is described below
commit 808184ab723d678a4966ae4c45096c4bd6d3e756
Author: Etienne Chauchot
AuthorDate: Tue Nov 14 10:30:05 2023 +0100
[FLINK-32563] add additionalExcludes property to add exclusions to surefire
tests
---
pom.xml | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 00c9a9b..0d88817 100644
--- a/pom.xml
+++ b/pom.xml
@@ -27,7 +27,7 @@ under the License.
org.apache.flink
flink-connector-parent
-1.0.0
+1.0.1
pom
https://flink.apache.org
@@ -80,6 +80,10 @@ under the License.
+
+
+
@@ -570,6 +574,9 @@ under the License.
${test.unit.pattern}
+
+${additionalExcludes}
+
${flink.forkCountUnitTest}
${flink.surefire.baseArgLine}
-Xmx${flink.XmxUnitTest}
@@ -587,6 +594,7 @@ under the License.
${test.unit.pattern}
+${additionalExcludes}
${flink.forkCountITCase}
${flink.surefire.baseArgLine}
-Xmx${flink.XmxITCase}
(flink) branch master updated: [FLINK-33489][table-planner] forbid generating partial-final agg with LISTAGG to avoid wrong result
This is an automated email from the ASF dual-hosted git repository.
shengkai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 7e30e5e9fcd [FLINK-33489][table-planner] forbid generating
partial-final agg with LISTAGG to avoid wrong result
7e30e5e9fcd is described below
commit 7e30e5e9fcd51382f48d48c9848bb8df14293e22
Author: xuyang
AuthorDate: Thu Nov 9 12:28:14 2023 +0800
[FLINK-33489][table-planner] forbid generating partial-final agg with
LISTAGG to avoid wrong result
---
.../table/planner/plan/utils/AggregateUtil.scala | 5 +-
.../plan/stream/sql/agg/DistinctAggregateTest.xml | 86 ++
.../stream/sql/agg/IncrementalAggregateTest.xml| 22 ++
.../stream/sql/agg/DistinctAggregateTest.scala | 5 ++
.../runtime/stream/sql/SplitAggregateITCase.scala | 31
5 files changed, 147 insertions(+), 2 deletions(-)
diff --git
a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/AggregateUtil.scala
b/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/AggregateUtil.scala
index ef25844477f..73c5f2a09da 100644
---
a/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/AggregateUtil.scala
+++
b/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/AggregateUtil.scala
@@ -953,9 +953,10 @@ object AggregateUtil extends Enumeration {
aggCall.getAggregation match {
case _: SqlCountAggFunction | _: SqlAvgAggFunction | _:
SqlMinMaxAggFunction |
_: SqlSumAggFunction | _: SqlSumEmptyIsZeroAggFunction |
- _: SqlSingleValueAggFunction | _: SqlListAggFunction =>
+ _: SqlSingleValueAggFunction =>
true
- case _: SqlFirstLastValueAggFunction => aggCall.getArgList.size() ==
1
+ case _: SqlFirstLastValueAggFunction | _: SqlListAggFunction =>
+aggCall.getArgList.size() == 1
case _ => false
}
}
diff --git
a/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.xml
b/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.xml
index cf1f3660adb..6ea406b774f 100644
---
a/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.xml
+++
b/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.xml
@@ -287,6 +287,92 @@ GroupAggregate(groupBy=[b], partialFinalType=[FINAL],
select=[b, FIRST_VALUE_RET
+- Calc(select=[a, b, MOD(HASH_CODE(b), 1024) AS
$f2], changelogMode=[I])
+- MiniBatchAssigner(interval=[1000ms],
mode=[ProcTime], changelogMode=[I])
+-
LegacyTableSourceScan(table=[[default_catalog, default_database, MyTable,
source: [TestTableSource(a, b, c)]]], fields=[a, b, c], changelogMode=[I])
+]]>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git
a/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/IncrementalAggregateTest.xml
b/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/IncrementalAggregateTest.xml
index 3962d19ab57..8585541ae0c 100644
---
a/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/IncrementalAggregateTest.xml
+++
b/flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/stream/sql/agg/IncrementalAggregateTest.xml
@@ -85,6 +85,28 @@ GroupAggregate(groupBy=[b], partialFinalType=[FINAL],
select=[b, FIRST_VALUE_RET
+- Calc(select=[a, b, MOD(HASH_CODE(b), 1024) AS
$f2], changelogMode=[I])
+- MiniBatchAssigner(interval=[1000ms],
mode=[ProcTime], changelogMode=[I])
+-
LegacyTableSourceScan(table=[[default_catalog, default_database, MyTable,
source: [TestTableSource(a, b, c)]]], fields=[a, b, c], changelogMode=[I])
+]]>
+
+
+
+
+
+
+
+
+
+
+
diff --git
a/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
b/flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/stream/sql/agg/DistinctAggregateTest.scala
index
(flink) branch master updated: [FLINK-33543][core] Moves Java 17-specific test classes into their own module to make it easier unload these classes in the IDE.
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new a1aed4f8770 [FLINK-33543][core] Moves Java 17-specific test classes
into their own module to make it easier unload these classes in the IDE.
a1aed4f8770 is described below
commit a1aed4f877099328d4833f8a2781d2edbaaddc70
Author: Matthias Pohl
AuthorDate: Tue Nov 14 12:37:07 2023 +0100
[FLINK-33543][core] Moves Java 17-specific test classes into their own
module to make it easier unload these classes in the IDE.
---
flink-tests-java17/pom.xml | 83 +
.../runtime/PojoRecordSerializerTest.java | 3 +-
.../runtime/PojoRecordSerializerUpgradeTest.java | 16 ++--
...oRecordSerializerUpgradeTestSpecifications.java | 2 +-
.../runtime/RecordBuilderFactoryTest.java | 2 +-
.../serializer-snapshot| Bin
.../test-data | Bin
.../serializer-snapshot| Bin
.../pojo-serializer-to-record-1.19/test-data | Bin
pom.xml| 11 +--
tools/maven/suppressions-core.xml | 2 -
tools/maven/suppressions-tests-java17.xml | 28 +++
12 files changed, 124 insertions(+), 23 deletions(-)
diff --git a/flink-tests-java17/pom.xml b/flink-tests-java17/pom.xml
new file mode 100644
index 000..bf2a3cf133f
--- /dev/null
+++ b/flink-tests-java17/pom.xml
@@ -0,0 +1,83 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+ 4.0.0
+
+
+ org.apache.flink
+ flink-parent
+ 1.19-SNAPSHOT
+
+
+ flink-tests-java17
+ Flink : Java 17 Tests
+ Module that collects Java 17-specific test
code.
+
+ jar
+
+
+
+
+ org.apache.flink
+ flink-core
+ ${project.version}
+ test
+
+
+
+
+ org.apache.flink
+ flink-core
+ ${project.version}
+ test
+ test-jar
+
+
+
+
+ org.apache.flink
+ fink-migration-test-utils
+ ${project.version}
+ test
+
+
+
+
+ org.apache.flink
+ flink-test-utils-junit
+ ${project.version}
+ test
+
+
+
+
+
+
+
+ maven-checkstyle-plugin
+
+
/tools/maven/suppressions-tests-java17.xml
+
+
+
+
+
+
diff --git
a/flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/Java17PojoRecordSerializerTest.java
b/flink-tests-java17/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoRecordSerializerTest.java
similarity index 98%
rename from
flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/Java17PojoRecordSerializerTest.java
rename to
flink-tests-java17/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoRecordSerializerTest.java
index 58e2c2a8535..250e4b30bb4 100644
---
a/flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/Java17PojoRecordSerializerTest.java
+++
b/flink-tests-java17/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoRecordSerializerTest.java
@@ -43,8 +43,7 @@ import java.util.Random;
import static org.assertj.core.api.Assertions.assertThat;
/** A test for the {@link PojoSerializer} with Java Records. */
-class Java17PojoRecordSerializerTest
-extends
SerializerTestBase {
+class PojoRecordSerializerTest extends
SerializerTestBase {
private final TypeInformation type =
TypeExtractor.getForClass(TestUserClass.class);
diff --git
a/flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/Java17PojoRecordSerializerUpgradeTest.java
b/flink-tests-java17/src/test/java/org/apache/flink/api/java/typeutils/runtime/PojoRecordSerializerUpgradeTest.java
similarity index 75%
rename from
flink-core/src/test/java/org/apache/flink/api/java/typeutils/runtime/Java17PojoRecordSerializerUpgradeTest.java
rename to
