[flink-connector-elasticsearch] branch main updated: [hotfix] Deduplicate TestUtils
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
The following commit(s) were added to refs/heads/main by this push:
new 94ab432 [hotfix] Deduplicate TestUtils
94ab432 is described below
commit 94ab432a550f4c9bb1188800e8b3e2a7a16a8f59
Author: Chesnay Schepler
AuthorDate: Thu Sep 15 10:22:43 2022 +0200
[hotfix] Deduplicate TestUtils
---
.../org/apache/flink/tests/util/TestUtils.java | 85 --
.../streaming/tests/Elasticsearch6SinkE2ECase.java | 10 ++-
.../streaming/tests/Elasticsearch7SinkE2ECase.java | 10 ++-
3 files changed, 12 insertions(+), 93 deletions(-)
diff --git
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/tests/util/TestUtils.java
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/tests/util/TestUtils.java
deleted file mode 100644
index 980aaa9..000
---
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/tests/util/TestUtils.java
+++ /dev/null
@@ -1,85 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.flink.tests.util;
-
-import org.apache.flink.test.parameters.ParameterProperty;
-
-import java.io.FileNotFoundException;
-import java.io.IOException;
-import java.nio.file.Files;
-import java.nio.file.Path;
-import java.nio.file.Paths;
-import java.util.List;
-import java.util.regex.Pattern;
-import java.util.stream.Collectors;
-import java.util.stream.Stream;
-
-/** General test utilities. */
-public enum TestUtils {
-;
-
-private static final ParameterProperty MODULE_DIRECTORY =
-new ParameterProperty<>("moduleDir", Paths::get);
-
-/**
- * Searches for a resource file matching the given regex in the given
directory. This method is
- * primarily intended to be used for the initialization of static {@link
Path} fields for
- * resource file(i.e. jar, config file) that reside in the modules {@code
target} directory.
- *
- * @param resourceNameRegex regex pattern to match against
- * @return Path pointing to the matching jar
- * @throws RuntimeException if none or multiple resource files could be
found
- */
-public static Path getResource(final String resourceNameRegex) {
-// if the property is not set then we are most likely running in the
IDE, where the working
-// directory is the
-// module of the test that is currently running, which is exactly what
we want
-Path moduleDirectory =
MODULE_DIRECTORY.get(Paths.get("").toAbsolutePath());
-
-try (Stream dependencyResources = Files.walk(moduleDirectory)) {
-final List matchingResources =
-dependencyResources
-.filter(
-jar ->
-Pattern.compile(resourceNameRegex)
-
.matcher(jar.toAbsolutePath().toString())
-.find())
-.collect(Collectors.toList());
-switch (matchingResources.size()) {
-case 0:
-throw new RuntimeException(
-new FileNotFoundException(
-String.format(
-"No resource file could be found
that matches the pattern %s. "
-+ "This could mean that
the test module must be rebuilt via maven.",
-resourceNameRegex)));
-case 1:
-return matchingResources.get(0);
-default:
-throw new RuntimeException(
-new IOException(
-String.format(
-
[flink] branch master updated (33afc3c8924 -> 68ec91bdc3b)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 33afc3c8924 [FLINK-29305][tests] UpsertTestSinkWriter creates parent directories add 68ec91bdc3b [hotfix][tests] Update job name No new revisions were added by this update. Summary of changes: .../scala/org/apache/flink/quickstarts/test/QuickstartExample.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[flink-connector-elasticsearch] branch main updated: [FLINK-29304][ci] Set timeout
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
The following commit(s) were added to refs/heads/main by this push:
new 7ccd8c2 [FLINK-29304][ci] Set timeout
7ccd8c2 is described below
commit 7ccd8c2b372baa23169d2d56dc404d6ccf51e802
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:21:27 2022 +0200
[FLINK-29304][ci] Set timeout
---
.github/workflows/ci.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index f4affdd..c4d44cf 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -24,6 +24,7 @@ jobs:
strategy:
matrix:
jdk: [8, 11]
+timeout-minutes: 30
env:
MVN_CONNECTION_OPTIONS: -Dhttp.keepAlive=false
-Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
FLINK_URL:
https://s3.amazonaws.com/flink-nightly/flink-1.16-SNAPSHOT-bin-scala_2.12.tgz
@@ -46,6 +47,7 @@ jobs:
maven-version: 3.8.6
- name: Compile and test flink-connector-elasticsearch
+timeout-minutes: 20
run: |
pushd .. \
&& wget -q -c ${{ env.FLINK_URL }} -O - | tar -xz \
[flink] branch master updated: [FLINK-29305][tests] UpsertTestSinkWriter creates parent directories
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 33afc3c8924 [FLINK-29305][tests] UpsertTestSinkWriter creates parent
directories
33afc3c8924 is described below
commit 33afc3c8924861025094ae94291805edad7afcd6
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:59:47 2022 +0200
[FLINK-29305][tests] UpsertTestSinkWriter creates parent directories
---
.../flink/connector/upserttest/sink/UpsertTestSinkWriter.java | 6 ++
.../flink/connector/upserttest/sink/UpsertTestSinkWriterITCase.java | 3 ++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git
a/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
b/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
index c6c8889e951..a91ff3bdfbf 100644
---
a/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
+++
b/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
@@ -27,6 +27,7 @@ import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
+import java.nio.file.Files;
import java.util.HashMap;
import java.util.Map;
@@ -53,6 +54,11 @@ class UpsertTestSinkWriter implements SinkWriter {
this.keySerializationSchema = checkNotNull(keySerializationSchema);
this.valueSerializationSchema = checkNotNull(valueSerializationSchema);
checkNotNull(outputFile);
+try {
+Files.createDirectories(outputFile.toPath().getParent());
+} catch (IOException e) {
+throw new FlinkRuntimeException("Could not parent directories for
path: " + outputFile);
+}
try {
this.bufferedOutputStream =
new BufferedOutputStream(new FileOutputStream(outputFile,
true));
diff --git
a/flink-tests/src/test/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriterITCase.java
b/flink-tests/src/test/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriterITCase.java
index 3493a4dec53..c3430ca461a 100644
---
a/flink-tests/src/test/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriterITCase.java
+++
b/flink-tests/src/test/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriterITCase.java
@@ -35,6 +35,7 @@ import org.junit.jupiter.api.io.TempDir;
import java.io.File;
import java.io.IOException;
+import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@@ -60,7 +61,7 @@ class UpsertTestSinkWriterITCase {
@BeforeEach
void setup() {
-outputFile = new File(tempDir, "records.out");
+outputFile = tempDir.toPath().resolve(Paths.get("dir",
"records.out")).toFile();
writer = createSinkWriter(outputFile);
expectedRecords = writeTestData(writer);
}
[flink-connector-elasticsearch] branch main updated: [FLINK-29303][tests] Add DockerImageVersions
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
The following commit(s) were added to refs/heads/main by this push:
new 861b6fc [FLINK-29303][tests] Add DockerImageVersions
861b6fc is described below
commit 861b6fc53079a407714a5815cdc45e083226393e
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 14:59:50 2022 +0200
[FLINK-29303][tests] Add DockerImageVersions
---
.../sink/ElasticsearchWriterITCase.java| 2 +-
.../elasticsearch/test/DockerImageVersions.java| 32 ++
.../pom.xml| 13 +
.../streaming/tests/Elasticsearch6SinkE2ECase.java | 2 +-
.../pom.xml| 13 +
.../streaming/tests/Elasticsearch7SinkE2ECase.java | 2 +-
.../sink/Elasticsearch6SinkITCase.java | 2 +-
.../table/Elasticsearch6DynamicSinkITCase.java | 2 +-
.../table/Elasticsearch6DynamicSinkITCase.java | 2 +-
.../elasticsearch6/ElasticsearchSinkITCase.java| 2 +-
.../sink/Elasticsearch7SinkITCase.java | 2 +-
.../table/Elasticsearch7DynamicSinkITCase.java | 2 +-
.../table/Elasticsearch7DynamicSinkITCase.java | 2 +-
.../elasticsearch7/ElasticsearchSinkITCase.java| 2 +-
14 files changed, 69 insertions(+), 11 deletions(-)
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
index c4e5255..bd020fb 100644
---
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
@@ -21,6 +21,7 @@ import org.apache.flink.api.common.operators.MailboxExecutor;
import org.apache.flink.api.connector.sink2.SinkWriter;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.connector.elasticsearch.ElasticsearchUtil;
+import org.apache.flink.connector.elasticsearch.test.DockerImageVersions;
import org.apache.flink.metrics.Counter;
import org.apache.flink.metrics.Gauge;
import org.apache.flink.metrics.groups.OperatorIOMetricGroup;
@@ -31,7 +32,6 @@ import
org.apache.flink.runtime.metrics.groups.InternalSinkWriterMetricGroup;
import org.apache.flink.runtime.metrics.groups.UnregisteredMetricGroups;
import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
import org.apache.flink.test.junit5.MiniClusterExtension;
-import org.apache.flink.util.DockerImageVersions;
import org.apache.flink.util.FlinkRuntimeException;
import org.apache.flink.util.TestLoggerExtension;
import org.apache.flink.util.function.ThrowingRunnable;
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/test/DockerImageVersions.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/test/DockerImageVersions.java
new file mode 100644
index 000..16582a7
--- /dev/null
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/test/DockerImageVersions.java
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.connector.elasticsearch.test;
+
+/**
+ * Utility class for defining the image names and versions of Docker
containers used during the Java
+ * tests. The names/versions are centralised here in order to make testing
version updates easier,
+ * as well as to provide a central file to use as a key when caching testing
Docker files.
+ */
+public class DockerImageVersions {
+
+public static final String ELASTICSEARCH_6 =
+"docker.elastic.co/elasticsearch/elasticsearch:6.8.20";
+
+public static final String ELASTICSEARCH_7 =
+"docker.elastic.co/elasticsearch/elasticsearch:7.10.2";
+}
diff --git
a/flink
[flink] branch release-1.16 updated: [FLINK-29301] Publish flink-ci-tools
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/release-1.16 by this push: new 227dad0e449 [FLINK-29301] Publish flink-ci-tools 227dad0e449 is described below commit 227dad0e449fb6e4fb882ae64d79335f95a72f3d Author: Chesnay Schepler AuthorDate: Wed Sep 14 14:17:57 2022 +0200 [FLINK-29301] Publish flink-ci-tools --- tools/ci/flink-ci-tools/pom.xml | 13 - 1 file changed, 13 deletions(-) diff --git a/tools/ci/flink-ci-tools/pom.xml b/tools/ci/flink-ci-tools/pom.xml index 34508a52e67..5f79de45de4 100644 --- a/tools/ci/flink-ci-tools/pom.xml +++ b/tools/ci/flink-ci-tools/pom.xml @@ -69,17 +69,4 @@ under the License. compile - - - - - org.apache.maven.plugins - maven-deploy-plugin - - true - - - - -
[flink] 03/04: [FLINK-29301] Automatically determine modules skipping deployment
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 7ed8022da8e782d82a33b230bd6ae204f1526ea3
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 13:35:45 2022 +0200
[FLINK-29301] Automatically determine modules skipping deployment
In order to work with external modules we can't rely on a static list for
modules that skip deployment (== where the NOTICE contents are irrelevant). We
can determine that easily from the build output.
---
.../tools/ci/licensecheck/NoticeFileChecker.java | 63 +-
.../modules-skipping-deployment.modulelist | 43 ---
2 files changed, 50 insertions(+), 56 deletions(-)
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
index 8ea2cec2945..8ba7f95c449 100644
---
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
+++
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
@@ -37,6 +37,7 @@ import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
@@ -51,9 +52,6 @@ public class NoticeFileChecker {
private static final Logger LOG =
LoggerFactory.getLogger(NoticeFileChecker.class);
-private static final List MODULES_SKIPPING_DEPLOYMENT =
-loadFromResources("modules-skipping-deployment.modulelist");
-
private static final List MODULES_DEFINING_EXCESS_DEPENDENCIES =
loadFromResources("modules-defining-excess-dependencies.modulelist");
@@ -64,6 +62,19 @@ public class NoticeFileChecker {
private static final Pattern SHADE_INCLUDE_MODULE_PATTERN =
Pattern.compile(".*Including ([^:]+):([^:]+):jar:([^ ]+) in the
shaded jar");
+// Examples:
+//
+// Deployment on CI with alternative repo
+// [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @
flink-parent ---
+// [INFO] Using alternate deployment
repository.../tmp/flink-validation-deployment
+//
+// Skipped deployment:
+// [INFO] --- maven-deploy-plugin:2.8.2:deploy (default-deploy) @
flink-parent ---
+// [INFO] Skipping artifact deployment
+private static final Pattern DEPLOY_MODULE_PATTERN =
+Pattern.compile(
+".maven-deploy-plugin:.*:deploy .* @ (?[^
_]+)(_[0-9.]+)? --.*");
+
// Examples:
// "- org.apache.htrace:htrace-core:3.1.0-incubating"
// or
@@ -80,13 +91,22 @@ public class NoticeFileChecker {
parseModulesFromBuildResult(buildResult),
DependencyParser.parseDependencyCopyOutput(buildResult.toPath()));
+final Set deployedModules =
parseDeployedModulesFromBuildResult(buildResult);
+
LOG.info(
"Extracted "
++ deployedModules.size()
++ " modules that were deployed of which "
+ modulesWithBundledDependencies.keySet().size()
-+ " modules with a total of "
++ " bundle dependencies with a total of "
+ modulesWithBundledDependencies.values().size()
+ " dependencies");
+final HashSet moduleSkippingDeployment =
+new HashSet<>(modulesWithBundledDependencies.keySet());
+moduleSkippingDeployment.removeAll(deployedModules);
+
moduleSkippingDeployment.forEach(modulesWithBundledDependencies::removeAll);
+
// find modules producing a shaded-jar
List noticeFiles = findNoticeFiles(root);
LOG.info("Found {} NOTICE files to check", noticeFiles.size());
@@ -127,15 +147,13 @@ public class NoticeFileChecker {
.map(NoticeFileChecker::getModuleFromNoticeFile)
.collect(Collectors.toList()));
for (String moduleWithoutNotice : shadingModules) {
-if (!MODULES_SKIPPING_DEPLOYMENT.contains(moduleWithoutNotice)) {
-LOG.error(
-"Module {} is missing a NOTICE file. It has shaded
dependencies: {}",
-moduleWithoutNotice,
-
modulesWithShadedDependencies.get(moduleWithoutNotice).stream()
-.map(Dependency::toString)
-.collect(Collectors.joining("\n\t", "\n\t",
"")));
-severeIssueCount++;
-}
[flink] 02/04: [FLINK-29301] loadFromResources works as jar
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
commit e9e072c41ad340ec0bffd54d7eb8a8d190263faa
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 13:32:38 2022 +0200
[FLINK-29301] loadFromResources works as jar
loadFromResource implicitly relied on being called in the context of the
java-ci-tools module, directly accessing the resources directly from the source.
When packaged as a jar this approach doesn't work, because then you'd load
files from a jar filesystem that you have to create explicitly.
Instead we now load the resource as an InputStream which works in all cases.
---
.../tools/ci/licensecheck/NoticeFileChecker.java | 26 +++---
1 file changed, 18 insertions(+), 8 deletions(-)
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
index 719c9c6198e..8ea2cec2945 100644
---
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
+++
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
@@ -26,11 +26,12 @@ import com.google.common.collect.Multimap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import java.io.BufferedReader;
import java.io.File;
import java.io.IOException;
+import java.io.InputStreamReader;
import java.nio.file.Files;
import java.nio.file.Path;
-import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
@@ -38,6 +39,7 @@ import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@@ -317,13 +319,21 @@ public class NoticeFileChecker {
private static List loadFromResources(String fileName) {
try {
-Path resource = Paths.get(NoticeFileChecker.class.getResource("/"
+ fileName).toURI());
-List result =
-Files.readAllLines(resource).stream()
-.filter(line -> !line.startsWith("#") &&
!line.isEmpty())
-.collect(Collectors.toList());
-LOG.debug("Loaded {} items from resource {}", result.size(),
fileName);
-return result;
+try (BufferedReader bufferedReader =
+new BufferedReader(
+new InputStreamReader(
+Objects.requireNonNull(
+
NoticeFileChecker.class.getResourceAsStream(
+"/" + fileName) {
+
+List result =
+bufferedReader
+.lines()
+.filter(line -> !line.startsWith("#") &&
!line.isEmpty())
+.collect(Collectors.toList());
+LOG.debug("Loaded {} items from resource {}", result.size(),
fileName);
+return result;
+}
} catch (Throwable e) {
// wrap anything in a RuntimeException to be callable from the
static initializer
throw new RuntimeException("Error while loading resource", e);
[flink] 04/04: [FLINK-29301] Rename java-ci-tools to flink-ci-tools
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 994ef6bf23bc647843bd6f0dae7a5e43bbbfb7b4
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 14:17:24 2022 +0200
[FLINK-29301] Rename java-ci-tools to flink-ci-tools
---
pom.xml | 2 +-
tools/ci/{java-ci-tools => flink-ci-tools}/pom.xml | 2 +-
.../java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java | 0
.../java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java | 0
.../java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java | 0
.../java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java | 0
.../org/apache/flink/tools/ci/utils/dependency/DependencyParser.java| 0
.../main/java/org/apache/flink/tools/ci/utils/shared/Dependency.java| 0
.../src/main/resources/log4j2.properties| 0
.../src/main/resources/modules-defining-excess-dependencies.modulelist | 0
.../java/org/apache/flink/tools/ci/licensecheck/JarFileCheckerTest.java | 0
.../flink/tools/ci/utils/dependency/DependencyParserCopyTest.java | 0
.../flink/tools/ci/utils/dependency/DependencyParserTreeTest.java | 0
.../META-INF/services/org.junit.jupiter.api.extension.Extension | 0
tools/ci/license_check.sh | 2 +-
tools/ci/verify_scala_suffixes.sh | 2 +-
16 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/pom.xml b/pom.xml
index fde290bb61f..7d3718a4884 100644
--- a/pom.xml
+++ b/pom.xml
@@ -101,7 +101,7 @@ under the License.
flink-walkthroughs
flink-kubernetes
flink-external-resources
- tools/ci/java-ci-tools
+ tools/ci/flink-ci-tools
diff --git a/tools/ci/java-ci-tools/pom.xml b/tools/ci/flink-ci-tools/pom.xml
similarity index 98%
rename from tools/ci/java-ci-tools/pom.xml
rename to tools/ci/flink-ci-tools/pom.xml
index 5875860e5a4..34508a52e67 100644
--- a/tools/ci/java-ci-tools/pom.xml
+++ b/tools/ci/flink-ci-tools/pom.xml
@@ -29,7 +29,7 @@ under the License.
../../..
- java-ci-tools
+ flink-ci-tools
1.16-SNAPSHOT
Flink : Tools : CI : Java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
similarity index 100%
rename from
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
rename to
tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
similarity index 100%
rename from
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
rename to
tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
similarity index 100%
rename from
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
rename to
tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
similarity index 100%
rename from
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
rename to
tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
similarity index 100%
rename from
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
rename to
tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/shared/Dependency.java
b/tools/ci/flink-ci-tools/src/main/java/org/apache/flink/tools/ci/utils/shared/Dependency.java
similarity index 100%
ren
[flink] 01/04: [hotfix][ci] Improve output when NOTICE is missing
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 185c713e4b5b499adea3b58992ee18da34cf64d3
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 13:15:11 2022 +0200
[hotfix][ci] Improve output when NOTICE is missing
---
.../org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
index e2f2316bd08..719c9c6198e 100644
---
a/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
+++
b/tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
@@ -129,7 +129,9 @@ public class NoticeFileChecker {
LOG.error(
"Module {} is missing a NOTICE file. It has shaded
dependencies: {}",
moduleWithoutNotice,
-
modulesWithShadedDependencies.get(moduleWithoutNotice));
+
modulesWithShadedDependencies.get(moduleWithoutNotice).stream()
+.map(Dependency::toString)
+.collect(Collectors.joining("\n\t", "\n\t",
"")));
severeIssueCount++;
}
}
[flink] branch release-1.16 updated (b1a2a5c2d2a -> 994ef6bf23b)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
from b1a2a5c2d2a [FLINK-29285][tests] Move TestUtils#getResource
new 185c713e4b5 [hotfix][ci] Improve output when NOTICE is missing
new e9e072c41ad [FLINK-29301] loadFromResources works as jar
new 7ed8022da8e [FLINK-29301] Automatically determine modules skipping
deployment
new 994ef6bf23b [FLINK-29301] Rename java-ci-tools to flink-ci-tools
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
pom.xml| 2 +-
tools/ci/{java-ci-tools => flink-ci-tools}/pom.xml | 2 +-
.../tools/ci/licensecheck/JarFileChecker.java | 0
.../tools/ci/licensecheck/LicenseChecker.java | 0
.../tools/ci/licensecheck/NoticeFileChecker.java | 87 +-
.../tools/ci/suffixcheck/ScalaSuffixChecker.java | 0
.../ci/utils/dependency/DependencyParser.java | 0
.../flink/tools/ci/utils/shared/Dependency.java| 0
.../src/main/resources/log4j2.properties | 0
...modules-defining-excess-dependencies.modulelist | 0
.../tools/ci/licensecheck/JarFileCheckerTest.java | 0
.../utils/dependency/DependencyParserCopyTest.java | 0
.../utils/dependency/DependencyParserTreeTest.java | 0
.../org.junit.jupiter.api.extension.Extension | 0
.../modules-skipping-deployment.modulelist | 43 ---
tools/ci/license_check.sh | 2 +-
tools/ci/verify_scala_suffixes.sh | 2 +-
17 files changed, 72 insertions(+), 66 deletions(-)
rename tools/ci/{java-ci-tools => flink-ci-tools}/pom.xml (98%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
(80%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/utils/shared/Dependency.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/resources/log4j2.properties (100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/resources/modules-defining-excess-dependencies.modulelist
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/licensecheck/JarFileCheckerTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/utils/dependency/DependencyParserCopyTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/utils/dependency/DependencyParserTreeTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/resources/META-INF/services/org.junit.jupiter.api.extension.Extension
(100%)
delete mode 100644
tools/ci/java-ci-tools/src/main/resources/modules-skipping-deployment.modulelist
[flink] branch master updated (db98322472c -> bb23cf7b2f6)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
from db98322472c [FLINK-29282][tests] Decouple Quickstart E2E test from
Elasticsearch
add a37161f0ffb [hotfix][ci] Improve output when NOTICE is missing
add 2d6fa8876a6 [FLINK-29301] loadFromResources works as jar
add d91f7dc5a01 [FLINK-29301] Automatically determine modules skipping
deployment
add 3ae5313d473 [FLINK-29301] Rename java-ci-tools to flink-ci-tools
add bb23cf7b2f6 [FLINK-29301] Publish flink-ci-tools
No new revisions were added by this update.
Summary of changes:
pom.xml| 2 +-
tools/ci/{java-ci-tools => flink-ci-tools}/pom.xml | 15 +---
.../tools/ci/licensecheck/JarFileChecker.java | 0
.../tools/ci/licensecheck/LicenseChecker.java | 0
.../tools/ci/licensecheck/NoticeFileChecker.java | 87 +-
.../tools/ci/suffixcheck/ScalaSuffixChecker.java | 0
.../ci/utils/dependency/DependencyParser.java | 0
.../flink/tools/ci/utils/shared/Dependency.java| 0
.../src/main/resources/log4j2.properties | 0
...modules-defining-excess-dependencies.modulelist | 0
.../tools/ci/licensecheck/JarFileCheckerTest.java | 0
.../utils/dependency/DependencyParserCopyTest.java | 0
.../utils/dependency/DependencyParserTreeTest.java | 0
.../org.junit.jupiter.api.extension.Extension | 0
.../modules-skipping-deployment.modulelist | 43 ---
tools/ci/license_check.sh | 2 +-
tools/ci/verify_scala_suffixes.sh | 2 +-
17 files changed, 72 insertions(+), 79 deletions(-)
rename tools/ci/{java-ci-tools => flink-ci-tools}/pom.xml (89%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/LicenseChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/licensecheck/NoticeFileChecker.java
(80%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/suffixcheck/ScalaSuffixChecker.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/utils/dependency/DependencyParser.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/java/org/apache/flink/tools/ci/utils/shared/Dependency.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/resources/log4j2.properties (100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/main/resources/modules-defining-excess-dependencies.modulelist
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/licensecheck/JarFileCheckerTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/utils/dependency/DependencyParserCopyTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/java/org/apache/flink/tools/ci/utils/dependency/DependencyParserTreeTest.java
(100%)
rename tools/ci/{java-ci-tools =>
flink-ci-tools}/src/test/resources/META-INF/services/org.junit.jupiter.api.extension.Extension
(100%)
delete mode 100644
tools/ci/java-ci-tools/src/main/resources/modules-skipping-deployment.modulelist
[flink] 08/16: Fix hyphens
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 5eb5220a466a6cecb07bb362399e77fb65e59cb7
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:08:06 2022 +0200
Fix hyphens
Likely copy & paste error from bash scripts.
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index b04898f2753..b288d80298a 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -224,16 +224,16 @@ public class SqlClientITCase {
"INSERT INTO AppendSinkTable",
" SELECT 1 as user_id, T.userName as user_name,
cast(1 as BIGINT) as user_count",
" FROM (",
-"SELECT \\`user\\`, \\`rowtime\\`",
+"SELECT `user`, `rowtime`",
"FROM JsonSourceTable",
-"WHERE \\`user\\` IS NOT NULL)",
+"WHERE `user` IS NOT NULL)",
" MATCH_RECOGNIZE (",
"ORDER BY rowtime",
"MEASURES",
-"\\`user\\` as userName",
+"`user` as userName",
"PATTERN (A)",
"DEFINE",
-"A as \\`user\\` = 'Alice'",
+"A as `user` = 'Alice'",
" ) T");
executeSql(sqlLines);
[flink] 07/16: Combine testMatchRecognize statements
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 13012d2a9717b2de6142983f8171667b51a6
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:05:38 2022 +0200
Combine testMatchRecognize statements
Needs to be one statement such that everything runs in the same client, and
not 2 entirely separate ones.
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index d9b4b5514d8..b04898f2753 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -219,11 +219,8 @@ public class SqlClientITCase {
"'key.format' = 'json',",
"'value.format' = 'json',",
"'output-filepath' = '" + outputFilepath + "'",
-" );");
-executeSql(sqlLines);
-
-sqlLines =
-Arrays.asList(
+" );",
+"",
"INSERT INTO AppendSinkTable",
" SELECT 1 as user_id, T.userName as user_name,
cast(1 as BIGINT) as user_count",
" FROM (",
[flink] 01/16: WIP
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 0767370306c0966dc6fe217cda642219b472315e
Author: Alexander Preuß <11444089+alp...@users.noreply.github.com>
AuthorDate: Tue Jun 7 10:24:45 2022 +0200
WIP
---
.../flink-sql-client-test/pom.xml | 12 ++
.../src/test/java/SqlClientITCase.java | 129 +
.../src/test/resources/log4j2-test.properties | 34 ++
.../testframe/container/FlinkContainers.java | 5 +
4 files changed, 180 insertions(+)
diff --git a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
index 8ed8004ce65..06d20839527 100644
--- a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
+++ b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
@@ -38,6 +38,12 @@ under the License.
provided
+
+ org.apache.flink
+ flink-end-to-end-tests-common
+ ${project.version}
+
+
[flink] 04/16: Add kafka dependency
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch pull/19923/head in repository https://gitbox.apache.org/repos/asf/flink.git commit a2fb23f30300f0be82a2ec65b22fc97d04f8fbf7 Author: Chesnay Schepler AuthorDate: Wed Sep 14 15:35:31 2022 +0200 Add kafka dependency --- flink-end-to-end-tests/flink-sql-client-test/pom.xml | 6 ++ 1 file changed, 6 insertions(+) diff --git a/flink-end-to-end-tests/flink-sql-client-test/pom.xml b/flink-end-to-end-tests/flink-sql-client-test/pom.xml index 51bc916b4bb..72a1c874b9f 100644 --- a/flink-end-to-end-tests/flink-sql-client-test/pom.xml +++ b/flink-end-to-end-tests/flink-sql-client-test/pom.xml @@ -86,6 +86,12 @@ under the License. provided + + org.apache.kafka + kafka-clients + 3.2.1 + test + org.testcontainers kafka
[flink] branch pull/19923/head created (now fd543cb8079)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch pull/19923/head in repository https://gitbox.apache.org/repos/asf/flink.git at fd543cb8079 remove bash test logic (but keep packaging tests) This branch includes the following new commits: new 0767370306c WIP new 5afa35733a2 WIP Kafka new df9c887445a rebase new a2fb23f3030 Add kafka dependency new 66cf373b1ff Fix assertions new 7757da8590a Remove batch mode setting from testMatchRecognize new 13012d2a971 Combine testMatchRecognize statements new 5eb5220a466 Fix hyphens new cc61f35337f Fix statement new 451aee6081f Add missing Testcontainers annotation new 316ae61d434 Use KafkaContainerClient new b193c84d83f Explicitly create topic new 7733da986c5 UpsertTestSinkWriter should create parent directories new 857be37b805 Fix inter-container connectivity new dcdfa56beaf Enable checkpointing new fd543cb8079 remove bash test logic (but keep packaging tests) The 16 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[flink] 14/16: Fix inter-container connectivity
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 857be37b805ccad075423d83a04d61259238d9de
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 18:00:43 2022 +0200
Fix inter-container connectivity
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 212c54e8f2e..125d395e4ec 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -54,6 +54,8 @@ public class SqlClientITCase {
private static final Logger LOG =
LoggerFactory.getLogger(SqlClientITCase.class);
+private static final String INTER_CONTAINER_KAFKA_ALIAS = "kafka";
+
private static final Slf4jLogConsumer LOG_CONSUMER = new
Slf4jLogConsumer(LOG);
private static final Path sqlToolBoxJar =
TestUtils.getResource(".*SqlToolbox.jar");
@@ -68,7 +70,7 @@ public class SqlClientITCase {
public static final KafkaContainer KAFKA =
new
KafkaContainer(DockerImageName.parse(DockerImageVersions.KAFKA))
.withNetwork(NETWORK)
-.withNetworkAliases("kafka")
+.withNetworkAliases(INTER_CONTAINER_KAFKA_ALIAS)
.withLogConsumer(LOG_CONSUMER);
public final FlinkContainers flink =
@@ -199,8 +201,8 @@ public class SqlClientITCase {
"'connector' = 'kafka',",
"'topic' = 'test-json',",
"'properties.bootstrap.servers' = '"
-+ KAFKA.getBootstrapServers()
-+ "',",
++ INTER_CONTAINER_KAFKA_ALIAS
++ ":9092',",
"'scan.startup.mode' = 'earliest-offset',",
"'format' = 'json',",
"'json.timestamp-format.standard' = 'ISO-8601'",
[flink] 12/16: Explicitly create topic
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit b193c84d83fd8d4bb35b0c7af4aee64382fbd766
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:59:25 2022 +0200
Explicitly create topic
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index bd7ce195699..212c54e8f2e 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -238,6 +238,7 @@ public class SqlClientITCase {
public void sendMessages(String topic, String... messages) {
KafkaContainerClient kafkaClient = new KafkaContainerClient(KAFKA);
+kafkaClient.createTopic(1, 1, topic);
kafkaClient.sendMessages(topic, new StringSerializer(), messages);
}
[flink] 13/16: UpsertTestSinkWriter should create parent directories
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 7733da986c5f7cea8a55e2b69185810184d0ecce
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:59:47 2022 +0200
UpsertTestSinkWriter should create parent directories
---
.../flink/connector/upserttest/sink/UpsertTestSinkWriter.java | 6 ++
1 file changed, 6 insertions(+)
diff --git
a/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
b/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
index c6c8889e951..a91ff3bdfbf 100644
---
a/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
+++
b/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/connector/upserttest/sink/UpsertTestSinkWriter.java
@@ -27,6 +27,7 @@ import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
+import java.nio.file.Files;
import java.util.HashMap;
import java.util.Map;
@@ -53,6 +54,11 @@ class UpsertTestSinkWriter implements SinkWriter {
this.keySerializationSchema = checkNotNull(keySerializationSchema);
this.valueSerializationSchema = checkNotNull(valueSerializationSchema);
checkNotNull(outputFile);
+try {
+Files.createDirectories(outputFile.toPath().getParent());
+} catch (IOException e) {
+throw new FlinkRuntimeException("Could not parent directories for
path: " + outputFile);
+}
try {
this.bufferedOutputStream =
new BufferedOutputStream(new FileOutputStream(outputFile,
true));
[flink] 10/16: Add missing Testcontainers annotation
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 451aee6081f29bbc57d1016d95745ce1e8001a1d
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:58:12 2022 +0200
Add missing Testcontainers annotation
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java| 2 ++
1 file changed, 2 insertions(+)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 184ba25b2d0..fdd1eb09329 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -42,6 +42,7 @@ import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.output.Slf4jLogConsumer;
import org.testcontainers.junit.jupiter.Container;
+import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;
import java.io.File;
@@ -54,6 +55,7 @@ import java.util.Properties;
import static org.assertj.core.api.Assertions.assertThat;
/** E2E Test for SqlClient. */
+@Testcontainers
public class SqlClientITCase {
private static final Logger LOG =
LoggerFactory.getLogger(SqlClientITCase.class);
[flink] 11/16: Use KafkaContainerClient
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 316ae61d434438545fd84de30b588f88c9dfc447
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:59:16 2022 +0200
Use KafkaContainerClient
---
flink-end-to-end-tests/flink-sql-client-test/pom.xml | 7 +++
.../src/test/java/SqlClientITCase.java | 20 +++-
2 files changed, 10 insertions(+), 17 deletions(-)
diff --git a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
index 72a1c874b9f..0a4b4af2ab7 100644
--- a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
+++ b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
@@ -44,6 +44,13 @@ under the License.
${project.version}
+
+ org.apache.flink
+
flink-end-to-end-tests-common-kafka
+ ${project.version}
+ test
+
+
[flink] 06/16: Remove batch mode setting from testMatchRecognize
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 7757da8590af3be32ee03be1fb4301e1d293b7ac
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:04:53 2022 +0200
Remove batch mode setting from testMatchRecognize
JsonSourceTable is unbounded and throws an error in batch mode.
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java| 2 --
1 file changed, 2 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 7b6d91e6fd3..d9b4b5514d8 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -191,8 +191,6 @@ public class SqlClientITCase {
List sqlLines =
Arrays.asList(
-"SET 'execution.runtime-mode' = 'batch';",
-"",
"CREATE FUNCTION RegReplace AS
'org.apache.flink.table.toolbox.StringRegexReplaceFunction';",
"",
"CREATE TABLE JsonSourceTable (",
[flink] 05/16: Fix assertions
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 66cf373b1ffa9ab6cde1f9eee6520da94b016990
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:04:06 2022 +0200
Fix assertions
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 894c581fded..7b6d91e6fd3 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -169,7 +169,7 @@ public class SqlClientITCase {
"AS UserCountTable(user_id, user_name,
user_count);");
executeSql(sqlLines);
-verifyNumberOfResultRecords(outputFilepath, 6);
+verifyNumberOfResultRecords(outputFilepath, 3);
}
@Test
@@ -242,7 +242,7 @@ public class SqlClientITCase {
" ) T");
executeSql(sqlLines);
-verifyNumberOfResultRecords(outputFilepath, 3);
+verifyNumberOfResultRecords(outputFilepath, 1);
}
public void sendMessages(String topic, String... messages) {
[flink] 02/16: WIP Kafka
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 5afa35733a29df5a3a3c843f61a57aefd744e602
Author: Alexander Preuß <11444089+alp...@users.noreply.github.com>
AuthorDate: Tue Jun 7 14:36:36 2022 +0200
WIP Kafka
---
.../flink-sql-client-test/pom.xml | 6 +
.../src/test/java/SqlClientITCase.java | 160 +++--
2 files changed, 158 insertions(+), 8 deletions(-)
diff --git a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
index 06d20839527..51bc916b4bb 100644
--- a/flink-end-to-end-tests/flink-sql-client-test/pom.xml
+++ b/flink-end-to-end-tests/flink-sql-client-test/pom.xml
@@ -85,6 +85,12 @@ under the License.
${project.version}
provided
+
+
+ org.testcontainers
+ kafka
+ test
+
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 5e7e19683b3..1f8dd6dd8c1 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -20,7 +20,15 @@ import
org.apache.flink.connector.upserttest.sink.UpsertTestFileUtil;
import org.apache.flink.tests.util.TestUtils;
import org.apache.flink.tests.util.flink.SQLJobSubmission;
import org.apache.flink.tests.util.flink.container.FlinkContainers;
+import org.apache.flink.util.DockerImageVersions;
+import org.apache.kafka.clients.producer.KafkaProducer;
+import org.apache.kafka.clients.producer.Producer;
+import org.apache.kafka.clients.producer.ProducerConfig;
+import org.apache.kafka.clients.producer.ProducerRecord;
+import org.apache.kafka.common.serialization.BytesSerializer;
+import org.apache.kafka.common.serialization.StringSerializer;
+import org.apache.kafka.common.utils.Bytes;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -28,13 +36,18 @@ import org.junit.jupiter.api.io.TempDir;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.Network;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.junit.jupiter.Container;
+import org.testcontainers.utility.DockerImageName;
import java.io.File;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.List;
+import java.util.Properties;
import static org.assertj.core.api.Assertions.assertThat;
@@ -43,35 +56,46 @@ public class SqlClientITCase {
private static final Logger LOG =
LoggerFactory.getLogger(SqlClientITCase.class);
+private static final Slf4jLogConsumer LOG_CONSUMER = new
Slf4jLogConsumer(LOG);
private static final Path sqlToolBoxJar =
TestUtils.getResource(".*SqlToolbox.jar");
+private final Path sqlConnectorKafkaJar =
TestUtils.getResource(".*kafka.*.jar");
+
private final Path sqlConnectorUpsertTestJar =
TestUtils.getResource(".*flink-test-utils.*.jar");
public static final Network NETWORK = Network.newNetwork();
+@Container
+public static final KafkaContainer KAFKA =
+new
KafkaContainer(DockerImageName.parse(DockerImageVersions.KAFKA))
+.withNetwork(NETWORK)
+.withNetworkAliases("kafka")
+.withLogConsumer(LOG_CONSUMER);
+
public final FlinkContainers flink =
FlinkContainers.builder()
.setNumTaskManagers(1)
.setNetwork(NETWORK)
-.setLogger(LOG) /*.dependsOn(KAFKA)*/
+.setLogger(LOG)
+.dependsOn(KAFKA)
.build();
@TempDir private File tempDir;
@BeforeEach
-public void setup() throws Exception {
+void setup() throws Exception {
flink.start();
}
@AfterEach
-public void tearDown() {
+void tearDown() {
flink.stop();
}
@Test
-public void testUpsert() throws Exception {
-String outputFilepath = "/flink/records.out";
+void testUpsert() throws Exception {
+String outputFilepath = "/flink/records-upsert.out";
List sqlLines =
Arrays.asList(
@@ -104,16 +128,136 @@ public class SqlClientITCase {
| 42|Kim| 3 |
*/
-
[flink] 09/16: Fix statement
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit cc61f35337ffe50961faa2300604a879aa149e91
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:11:29 2022 +0200
Fix statement
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java| 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index b288d80298a..184ba25b2d0 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -234,7 +234,7 @@ public class SqlClientITCase {
"PATTERN (A)",
"DEFINE",
"A as `user` = 'Alice'",
-" ) T");
+" ) as T;");
executeSql(sqlLines);
verifyNumberOfResultRecords(outputFilepath, 1);
[flink] 16/16: remove bash test logic (but keep packaging tests)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit fd543cb80793e8f347faf73ea8db8406e6700dbc
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 18:28:40 2022 +0200
remove bash test logic (but keep packaging tests)
---
.../test-scripts/test_sql_client.sh| 203 -
1 file changed, 203 deletions(-)
diff --git a/flink-end-to-end-tests/test-scripts/test_sql_client.sh
b/flink-end-to-end-tests/test-scripts/test_sql_client.sh
index ea6a343589b..a5ca0511e80 100755
--- a/flink-end-to-end-tests/test-scripts/test_sql_client.sh
+++ b/flink-end-to-end-tests/test-scripts/test_sql_client.sh
@@ -19,25 +19,6 @@
set -Eeuo pipefail
-KAFKA_VERSION="3.2.1"
-CONFLUENT_VERSION="6.2.2"
-CONFLUENT_MAJOR_VERSION="6.2"
-# Check the Confluent Platform <> Apache Kafka compatibility matrix when
updating KAFKA_VERSION
-KAFKA_SQL_VERSION="universal"
-ELASTICSEARCH_VERSION=7
-# we use the smallest version possible
-ELASTICSEARCH_MAC_DOWNLOAD_URL='https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-darwin-x86_64.tar.gz'
-ELASTICSEARCH_LINUX_DOWNLOAD_URL='https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-linux-x86_64.tar.gz'
-
-source "$(dirname "$0")"/common.sh
-source "$(dirname "$0")"/kafka_sql_common.sh \
- $KAFKA_VERSION \
- $CONFLUENT_VERSION \
- $CONFLUENT_MAJOR_VERSION \
- $KAFKA_SQL_VERSION
-source "$(dirname "$0")"/elasticsearch-common.sh
-
-SQL_TOOLBOX_JAR=$END_TO_END_DIR/flink-sql-client-test/target/SqlToolbox.jar
SQL_JARS_DIR=$END_TO_END_DIR/flink-sql-client-test/target/sql-jars
@@ -80,187 +61,3 @@ for SQL_JAR in $SQL_JARS_DIR/*.jar; do
done
rm -r $EXTRACTED_JAR
-
-
-# Prepare connectors
-
-
-ELASTICSEARCH_INDEX='my_users'
-
-function sql_cleanup() {
- stop_kafka_cluster
- shutdown_elasticsearch_cluster "$ELASTICSEARCH_INDEX"
-}
-on_exit sql_cleanup
-
-function prepare_elasticsearch {
- echo "Preparing Elasticsearch (version=$ELASTICSEARCH_VERSION)..."
- # elastcisearch offers different release binary file for corresponding
system since version 7.
- case "$(uname -s)" in
- Linux*) OS_TYPE=linux;;
- Darwin*)OS_TYPE=mac;;
- *) OS_TYPE="UNKNOWN:${unameOut}"
- esac
-
- if [[ "$OS_TYPE" == "mac" ]]; then
-DOWNLOAD_URL=$ELASTICSEARCH_MAC_DOWNLOAD_URL
- elif [[ "$OS_TYPE" == "linux" ]]; then
-DOWNLOAD_URL=$ELASTICSEARCH_LINUX_DOWNLOAD_URL
- else
-echo "[ERROR] Unsupported OS for Elasticsearch: $OS_TYPE"
-exit 1
- fi
-
- setup_elasticsearch $DOWNLOAD_URL $ELASTICSEARCH_VERSION
- wait_elasticsearch_working
-}
-
-# prepare Kafka
-echo "Preparing Kafka..."
-
-setup_kafka_dist
-
-start_kafka_cluster
-
-create_kafka_json_source test-json
-create_kafka_topic 1 1 test-avro
-
-# prepare Elasticsearch
-prepare_elasticsearch
-
-
-# Prepare Flink
-
-
-echo "Preparing Flink..."
-
-start_cluster
-start_taskmanagers 2
-
-
-# Run SQL statements
-
-
-echo "Testing SQL statements..."
-
-KAFKA_SQL_JAR=$(find "$SQL_JARS_DIR" | grep "kafka" )
-ELASTICSEARCH_SQL_JAR=$(find "$SQL_JARS_DIR" | grep
"elasticsearch$ELASTICSEARCH_VERSION" )
-
-# create session environment file
-RESULT=$TEST_DATA_DIR/result
-INIT_SQL=$TEST_DATA_DIR/sql-client-init.sql
-
-get_kafka_json_source_schema test-json JsonSourceTable >> $INIT_SQL
-
-cat >> $INIT_SQL << EOF
-
- CREATE TABLE ElasticsearchUpsertSinkTable (
-user_id INT,
-user_name STRING,
-user_count BIGINT,
-PRIMARY KEY (user_id) NOT ENFORCED
- ) WITH (
-'connector' = 'elasticsearch-$ELASTICSEARCH_VERSION',
-'hosts' = 'http://localhost:9200',
-'index' = '$ELASTICSEARCH_INDEX',
-'sink.bulk-flush.max-actions' = '1',
-'format' = 'json'
- );
-
- CREATE TABLE ElasticsearchAppendSinkTable (
-user_id INT,
-user_name STRING,
-user_count BIGINT
- ) WITH (
-'connector' = 'elasticsearch-$ELASTICSEARCH_VERSION',
-'hosts' = 'http://localhost:9200',
-'index' = '$ELASTICSEARCH_INDEX',
-'sink.bulk-flush.max-actions' = '1',
-'format' = 'json'
[flink] 15/16: Enable checkpointing
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit dcdfa56beaf0abbeedfa0335769c5de63af7d7ef
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 18:28:07 2022 +0200
Enable checkpointing
---
.../flink-sql-client-test/src/test/java/SqlClientITCase.java | 10 +-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 125d395e4ec..f5381fe1fd2 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -20,6 +20,7 @@ import
org.apache.flink.connector.testframe.container.FlinkContainers;
import org.apache.flink.connector.testframe.container.FlinkContainersSettings;
import org.apache.flink.connector.testframe.container.TestcontainersSettings;
import org.apache.flink.connector.upserttest.sink.UpsertTestFileUtil;
+import
org.apache.flink.streaming.api.environment.ExecutionCheckpointingOptions;
import org.apache.flink.test.util.SQLJobSubmission;
import org.apache.flink.tests.util.TestUtils;
import org.apache.flink.tests.util.kafka.KafkaContainerClient;
@@ -43,6 +44,7 @@ import org.testcontainers.utility.DockerImageName;
import java.io.File;
import java.io.IOException;
import java.nio.file.Path;
+import java.time.Duration;
import java.util.Arrays;
import java.util.List;
@@ -76,7 +78,13 @@ public class SqlClientITCase {
public final FlinkContainers flink =
FlinkContainers.builder()
.withFlinkContainersSettings(
-
FlinkContainersSettings.builder().numTaskManagers(1).build())
+FlinkContainersSettings.builder()
+.numTaskManagers(1)
+// enable checkpointing for the
UpsertTestSink to write anything
+.setConfigOption(
+
ExecutionCheckpointingOptions.CHECKPOINTING_INTERVAL,
+Duration.ofMillis(500))
+.build())
.withTestcontainersSettings(
TestcontainersSettings.builder()
.network(NETWORK)
[flink] 03/16: rebase
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch pull/19923/head
in repository https://gitbox.apache.org/repos/asf/flink.git
commit df9c887445a1e22550d3216ebe37d8d73c953e98
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 15:34:39 2022 +0200
rebase
---
.../src/test/java/SqlClientITCase.java | 18 --
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
index 1f8dd6dd8c1..894c581fded 100644
---
a/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
+++
b/flink-end-to-end-tests/flink-sql-client-test/src/test/java/SqlClientITCase.java
@@ -16,10 +16,12 @@
* limitations under the License.
*/
+import org.apache.flink.connector.testframe.container.FlinkContainers;
+import org.apache.flink.connector.testframe.container.FlinkContainersSettings;
+import org.apache.flink.connector.testframe.container.TestcontainersSettings;
import org.apache.flink.connector.upserttest.sink.UpsertTestFileUtil;
+import org.apache.flink.test.util.SQLJobSubmission;
import org.apache.flink.tests.util.TestUtils;
-import org.apache.flink.tests.util.flink.SQLJobSubmission;
-import org.apache.flink.tests.util.flink.container.FlinkContainers;
import org.apache.flink.util.DockerImageVersions;
import org.apache.kafka.clients.producer.KafkaProducer;
@@ -75,10 +77,14 @@ public class SqlClientITCase {
public final FlinkContainers flink =
FlinkContainers.builder()
-.setNumTaskManagers(1)
-.setNetwork(NETWORK)
-.setLogger(LOG)
-.dependsOn(KAFKA)
+.withFlinkContainersSettings(
+
FlinkContainersSettings.builder().numTaskManagers(1).build())
+.withTestcontainersSettings(
+TestcontainersSettings.builder()
+.network(NETWORK)
+.logger(LOG)
+.dependsOn(KAFKA)
+.build())
.build();
@TempDir private File tempDir;
[flink] branch release-1.16 updated: [FLINK-29285][tests] Move TestUtils#getResource
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.16 by this push:
new b1a2a5c2d2a [FLINK-29285][tests] Move TestUtils#getResource
b1a2a5c2d2a is described below
commit b1a2a5c2d2a9d8e2ce4088b970fc806990f6440f
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 17:12:10 2022 +0200
[FLINK-29285][tests] Move TestUtils#getResource
---
.../table/test/KinesisFirehoseTableITTest.java | 4 +-
.../table/test/KinesisStreamsTableApiIT.java | 5 +-
.../flink/tests/util/kafka/KafkaSinkE2ECase.java | 8 +--
.../flink/tests/util/kafka/KafkaSourceE2ECase.java | 10 +--
.../tests/util/kafka/SQLClientKafkaITCase.java | 8 +--
.../util/kafka/SQLClientSchemaRegistryITCase.java | 11 ++--
.../flink/tests/util/kafka/SmokeKafkaITCase.java | 4 +-
.../org/apache/flink/tests/util/TestUtils.java | 56 -
.../streaming/tests/Elasticsearch6SinkE2ECase.java | 10 +--
.../streaming/tests/Elasticsearch7SinkE2ECase.java | 10 +--
.../tests/util/hbase/SQLClientHBaseITCase.java | 9 +--
.../FlinkContainerWithPulsarEnvironment.java | 4 +-
.../apache/flink/tests/scala/ScalaFreeITCase.java | 7 ++-
.../flink/table/sql/codegen/SqlITCaseBase.java | 5 +-
.../table/sql/codegen/UsingRemoteJarITCase.java| 5 +-
.../flink/table/gateway/SqlGatewayE2ECase.java | 7 ++-
.../kinesis/test/KinesisTableApiITCase.java| 4 +-
.../flink/test/resources/ResourceTestUtils.java| 73 +-
18 files changed, 64 insertions(+), 176 deletions(-)
diff --git
a/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-firehose/src/test/java/org/apache/flink/connector/firehose/table/test/KinesisFirehoseTableITTest.java
b/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-firehose/src/test/java/org/apache/flink/connector/firehose/table/test/KinesisFirehoseTableITTest.java
index ce67a707aec..0741ea51bab 100644
---
a/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-firehose/src/test/java/org/apache/flink/connector/firehose/table/test/KinesisFirehoseTableITTest.java
+++
b/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-firehose/src/test/java/org/apache/flink/connector/firehose/table/test/KinesisFirehoseTableITTest.java
@@ -22,8 +22,8 @@ import org.apache.flink.api.common.time.Deadline;
import org.apache.flink.connector.aws.testutils.LocalstackContainer;
import org.apache.flink.connector.testframe.container.FlinkContainers;
import org.apache.flink.connector.testframe.container.TestcontainersSettings;
+import org.apache.flink.test.resources.ResourceTestUtils;
import org.apache.flink.test.util.SQLJobSubmission;
-import org.apache.flink.tests.util.TestUtils;
import org.apache.flink.util.DockerImageVersions;
import org.apache.flink.util.TestLogger;
import org.apache.flink.util.jackson.JacksonMapperFactory;
@@ -84,7 +84,7 @@ public class KinesisFirehoseTableITTest extends TestLogger {
private static final ObjectMapper OBJECT_MAPPER =
JacksonMapperFactory.createObjectMapper();
-private final Path sqlConnectorFirehoseJar =
TestUtils.getResource(".*firehose.jar");
+private final Path sqlConnectorFirehoseJar =
ResourceTestUtils.getResource(".*firehose.jar");
private SdkHttpClient httpClient;
private S3Client s3Client;
diff --git
a/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-streams/src/test/java/org/apache/flink/connector/kinesis/table/test/KinesisStreamsTableApiIT.java
b/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-streams/src/test/java/org/apache/flink/connector/kinesis/table/test/KinesisStreamsTableApiIT.java
index d52ca2cb83d..6c0a944d524 100644
---
a/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-streams/src/test/java/org/apache/flink/connector/kinesis/table/test/KinesisStreamsTableApiIT.java
+++
b/flink-end-to-end-tests/flink-end-to-end-tests-aws-kinesis-streams/src/test/java/org/apache/flink/connector/kinesis/table/test/KinesisStreamsTableApiIT.java
@@ -24,8 +24,8 @@ import org.apache.flink.connector.aws.util.AWSGeneralUtil;
import org.apache.flink.connector.testframe.container.FlinkContainers;
import org.apache.flink.connector.testframe.container.TestcontainersSettings;
import org.apache.flink.connectors.kinesis.testutils.KinesaliteContainer;
+import org.apache.flink.test.resources.ResourceTestUtils;
import org.apache.flink.test.util.SQLJobSubmission;
-import org.apache.flink.tests.util.TestUtils;
import org.apache.flink.util.DockerImageVersions;
import org.apache.flink.util.jackson.JacksonMapperFactory;
@@ -84,7 +84,8 @@ public class KinesisStreamsTableApiIT {
private SdkHttpClient httpClient;
private KinesisClient kinesisClient;
-private final Path sqlConnectorKinesisJar =
TestUtils.getResource(".*kinesis-streams.jar&q
[flink] branch master updated (469049a4359 -> db98322472c)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
from 469049a4359 [FLINK-29285][tests] Move TestUtils#getResource
add db98322472c [FLINK-29282][tests] Decouple Quickstart E2E test from
Elasticsearch
No new revisions were added by this update.
Summary of changes:
.../pom.xml| 16 +
.../apache/flink/quickstarts/test/utils/Utils.java | 14 ++--
.../flink-quickstart-test/pom.xml | 2 +-
.../test/Elasticsearch7SinkExample.java| 79 --
.../flink/quickstarts/test/QuickstartExample.java | 65 ++
.../test/Elasticsearch7SinkExample.scala | 75
.../flink/quickstarts/test/QuickstartExample.scala | 58
flink-end-to-end-tests/pom.xml | 1 +
.../test-scripts/test_quickstarts.sh | 45 +++-
9 files changed, 144 insertions(+), 211 deletions(-)
copy flink-end-to-end-tests/{flink-streaming-kafka-test-base =>
flink-quickstart-test-dummy-dependency}/pom.xml (80%)
copy
flink-runtime-web/web-dashboard/src/app/pages/task-manager/list/task-manager-list.component.less
=>
flink-end-to-end-tests/flink-quickstart-test-dummy-dependency/src/main/java/org/apache/flink/quickstarts/test/utils/Utils.java
(78%)
delete mode 100644
flink-end-to-end-tests/flink-quickstart-test/src/main/java/org/apache/flink/quickstarts/test/Elasticsearch7SinkExample.java
create mode 100644
flink-end-to-end-tests/flink-quickstart-test/src/main/java/org/apache/flink/quickstarts/test/QuickstartExample.java
delete mode 100644
flink-end-to-end-tests/flink-quickstart-test/src/main/scala/org/apache/flink/quickstarts/test/Elasticsearch7SinkExample.scala
create mode 100644
flink-end-to-end-tests/flink-quickstart-test/src/main/scala/org/apache/flink/quickstarts/test/QuickstartExample.scala
[flink] branch master updated (c0165a8a7e3 -> 469049a4359)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from c0165a8a7e3 [FLINK-29101] ignore non canBePipelined consumedPartitionGroup in getAllSchedulableRegions. add 469049a4359 [FLINK-29285][tests] Move TestUtils#getResource No new revisions were added by this update. Summary of changes: .../table/test/KinesisFirehoseTableITTest.java | 4 +- .../table/test/KinesisStreamsTableApiIT.java | 5 +- .../flink/tests/util/kafka/KafkaSinkE2ECase.java | 8 +-- .../flink/tests/util/kafka/KafkaSourceE2ECase.java | 10 +-- .../tests/util/kafka/SQLClientKafkaITCase.java | 8 +-- .../util/kafka/SQLClientSchemaRegistryITCase.java | 11 ++-- .../flink/tests/util/kafka/SmokeKafkaITCase.java | 4 +- .../org/apache/flink/tests/util/TestUtils.java | 56 - .../streaming/tests/Elasticsearch6SinkE2ECase.java | 10 +-- .../streaming/tests/Elasticsearch7SinkE2ECase.java | 10 +-- .../tests/util/hbase/SQLClientHBaseITCase.java | 9 +-- .../FlinkContainerWithPulsarEnvironment.java | 4 +- .../apache/flink/tests/scala/ScalaFreeITCase.java | 7 ++- .../flink/table/sql/codegen/SqlITCaseBase.java | 5 +- .../table/sql/codegen/UsingRemoteJarITCase.java| 5 +- .../flink/table/gateway/SqlGatewayE2ECase.java | 7 ++- .../kinesis/test/KinesisTableApiITCase.java| 4 +- .../flink/test/resources/ResourceTestUtils.java| 73 +- 18 files changed, 64 insertions(+), 176 deletions(-) copy flink-end-to-end-tests/flink-end-to-end-tests-common/src/main/java/org/apache/flink/tests/util/TestUtils.java => flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/resources/ResourceTestUtils.java (57%)
[flink-connector-elasticsearch] 01/03: [hotfix] Wipe checkstyle suppressions
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit f2be867c51739d264454c2ed1a4ad6cd1048f0b1 Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:27:33 2022 +0200 [hotfix] Wipe checkstyle suppressions --- tools/maven/suppressions.xml | 59 1 file changed, 59 deletions(-) diff --git a/tools/maven/suppressions.xml b/tools/maven/suppressions.xml index 129a0c7..4f80ec2 100644 --- a/tools/maven/suppressions.xml +++ b/tools/maven/suppressions.xml @@ -23,63 +23,4 @@ under the License. "http://www.puppycrawl.com/dtds/suppressions_1_1.dtd;> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[flink-connector-elasticsearch] 03/03: [hotfix] Sync NOTICE files
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit d280eb719ccfed5e31ad1b007614b6081f148af7 Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:44:00 2022 +0200 [hotfix] Sync NOTICE files --- .../src/main/resources/META-INF/NOTICE| 15 +-- .../src/main/resources/META-INF/NOTICE| 15 +++ 2 files changed, 16 insertions(+), 14 deletions(-) diff --git a/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE b/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE index e5ece15..b4ef717 100644 --- a/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE +++ b/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE @@ -6,15 +6,18 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the Apache Software License 2.0. (http://www.apache.org/licenses/LICENSE-2.0.txt) -- com.fasterxml.jackson.core:jackson-core:2.13.0 -- com.fasterxml.jackson.core:jackson-databind:2.13.0 -- com.fasterxml.jackson.core:jackson-annotations:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0 +- com.fasterxml.jackson.core:jackson-core:2.13.2 +- com.fasterxml.jackson.core:jackson-databind:2.13.2.2 +- com.fasterxml.jackson.core:jackson-annotations:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.2 - commons-codec:commons-codec:1.15 +- commons-logging:commons-logging:1.1.3 - org.apache.httpcomponents:httpasyncclient:4.1.2 - org.apache.httpcomponents:httpclient:4.5.13 +- org.apache.httpcomponents:httpcore:4.4.14 +- org.apache.httpcomponents:httpcore-nio:4.4.5 - org.apache.lucene:lucene-analyzers-common:7.7.3 - org.apache.lucene:lucene-backward-codecs:7.7.3 - org.apache.lucene:lucene-core:7.7.3 diff --git a/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE b/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE index 66f4e92..1e7df85 100644 --- a/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE +++ b/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE @@ -7,16 +7,17 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the Apache Software License 2.0. (http://www.apache.org/licenses/LICENSE-2.0.txt) - com.carrotsearch:hppc:0.8.1 -- com.fasterxml.jackson.core:jackson-core:2.13.0 -- com.fasterxml.jackson.core:jackson-databind:2.13.0 -- com.fasterxml.jackson.core:jackson-annotations:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.0 -- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0 +- com.fasterxml.jackson.core:jackson-core:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.13.2 +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.2 - com.github.spullara.mustache.java:compiler:0.9.6 - commons-codec:commons-codec:1.15 +- commons-logging:commons-logging:1.1.3 - org.apache.httpcomponents:httpasyncclient:4.1.4 - org.apache.httpcomponents:httpclient:4.5.13 +- org.apache.httpcomponents:httpcore:4.4.14 +- org.apache.httpcomponents:httpcore-nio:4.4.12 - org.apache.lucene:lucene-analyzers-common:8.7.0 - org.apache.lucene:lucene-backward-codecs:8.7.0 - org.apache.lucene:lucene-core:8.7.0 @@ -28,7 +29,6 @@ This project bundles the following dependencies under the Apache Software Licens - org.apache.lucene:lucene-queries:8.7.0 - org.apache.lucene:lucene-queryparser:8.7.0 - org.apache.lucene:lucene-sandbox:8.7.0 -- org.apache.lucene:lucene-spatial:8.7.0 - org.apache.lucene:lucene-spatial-extras:8.7.0 - org.apache.lucene:lucene-spatial3d:8.7.0 - org.apache.lucene:lucene-suggest:8.7.0 @@ -38,7 +38,6 @@ This project bundles the following dependencies under the Apache Software Licens - org.elasticsearch:elasticsearch-geo:7.10.2 - org.elasticsearch:elasticsearch-secure-sm:7.10.2 - org.elasticsearch:elasticsearch-x-content:7.10.2 -- org.elasticsearch:elasticsearch-plugin-classloader:7.10.2 - org.elasticsearch.client:elasticsearch-rest-high-level-client:7.10.2 - org.elasticsearch.client:elasticsearch-rest-client:7.10.2 - org.elasticsearch.plugin:aggs-matrix-stats-client:7.10.2
[flink-connector-elasticsearch] 02/03: [hotfix] Sync dependency management
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 83758b13a5a0fcb9608d6d03104ea57cf52568fe Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:43:15 2022 +0200 [hotfix] Sync dependency management --- pom.xml | 18 ++ 1 file changed, 18 insertions(+) diff --git a/pom.xml b/pom.xml index e0336fb..4e1feed 100644 --- a/pom.xml +++ b/pom.xml @@ -264,6 +264,24 @@ under the License. 1.3.9 + + commons-codec + commons-codec + 1.15 + + + + org.apache.httpcomponents + httpcore + 4.4.14 + + + + org.apache.httpcomponents + httpclient + 4.5.13 + + org.slf4j slf4j-api
[flink-connector-elasticsearch] branch main updated (66210e9 -> d280eb7)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git from 66210e9 [hotfix] Add a proper README new f2be867 [hotfix] Wipe checkstyle suppressions new 83758b1 [hotfix] Sync dependency management new d280eb7 [hotfix] Sync NOTICE files The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/main/resources/META-INF/NOTICE | 15 +++--- .../src/main/resources/META-INF/NOTICE | 15 +++--- pom.xml| 18 +++ tools/maven/suppressions.xml | 59 -- 4 files changed, 34 insertions(+), 73 deletions(-)
[flink-connector-elasticsearch] branch main updated (31ae746 -> 66210e9)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git from 31ae746 [hotfix][tests] Deduplicate ParameterProperty new d1b8cd8 [hotfix][intellij] Remove /docs/themes/book directory mapping new 678ef2d [hotfix][legal] Update NOTICE files new 66210e9 [hotfix] Add a proper README The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .idea/vcs.xml | 1 - NOTICE | 4 +- README.md | 70 +- .../src/main/resources/META-INF/NOTICE | 2 +- .../src/main/resources/META-INF/NOTICE | 2 +- 5 files changed, 73 insertions(+), 6 deletions(-)
[flink-connector-elasticsearch] 03/03: [hotfix] Add a proper README
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 66210e9db21875a1edad93d785b4e95d7690293e Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:23:46 2022 +0200 [hotfix] Add a proper README --- README.md | 70 ++- 1 file changed, 69 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 84c5808..8550aca 100644 --- a/README.md +++ b/README.md @@ -1 +1,69 @@ -# flink-connector-elasticsearch +# Apache Flink Elasticsearch Connector + +This repository contains the official Apache Flink Elasticsearch connector. + +## Apache Flink + +Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities. + +Learn more about Flink at [https://flink.apache.org/](https://flink.apache.org/) + +## Building the Apache Flink Elasticsearch Connector from Source + +Prerequisites: + +* Unix-like environment (we use Linux, Mac OS X) +* Git +* Maven (we recommend version 3.8.6) +* Java 11 + +``` +git clone https://github.com/apache/flink-connector-elasticsearch.git +cd flink-connector-elasticsearch +./mvn clean package -DskipTests +``` + +The resulting jars can be found in the `target` directory of the respective module. + +## Developing Flink + +The Flink committers use IntelliJ IDEA to develop the Flink codebase. +We recommend IntelliJ IDEA for developing projects that involve Scala code. + +Minimal requirements for an IDE are: +* Support for Java and Scala (also mixed projects) +* Support for Maven with Java and Scala + +### IntelliJ IDEA + +The IntelliJ IDE supports Maven out of the box and offers a plugin for Scala development. + +* IntelliJ download: [https://www.jetbrains.com/idea/](https://www.jetbrains.com/idea/) +* IntelliJ Scala Plugin: [https://plugins.jetbrains.com/plugin/?id=1347](https://plugins.jetbrains.com/plugin/?id=1347) + +Check out our [Setting up IntelliJ](https://nightlies.apache.org/flink/flink-docs-master/flinkDev/ide_setup.html#intellij-idea) guide for details. + +## Support + +Don’t hesitate to ask! + +Contact the developers and community on the [mailing lists](https://flink.apache.org/community.html#mailing-lists) if you need any help. + +[Open an issue](https://issues.apache.org/jira/browse/FLINK) if you found a bug in Flink. + +## Documentation + +The documentation of Apache Flink is located on the website: [https://flink.apache.org](https://flink.apache.org) +or in the `docs/` directory of the source code. + +## Fork and Contribute + +This is an active open-source project. We are always open to people who want to use the system or contribute to it. +Contact us if you are looking for implementation tasks that fit your skills. +This article describes [how to contribute to Apache Flink](https://flink.apache.org/contributing/how-to-contribute.html). + +## About + +Apache Flink is an open source project of The Apache Software Foundation (ASF). +The Apache Flink project originated from the [Stratosphere](http://stratosphere.eu) research project. +
[flink-connector-elasticsearch] 01/03: [hotfix][intellij] Remove /docs/themes/book directory mapping
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit d1b8cd8ae8e528eea750869d9382be17269d4b58 Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:18:09 2022 +0200 [hotfix][intellij] Remove /docs/themes/book directory mapping --- .idea/vcs.xml | 1 - 1 file changed, 1 deletion(-) diff --git a/.idea/vcs.xml b/.idea/vcs.xml index 309fd12..fdb7539 100644 --- a/.idea/vcs.xml +++ b/.idea/vcs.xml @@ -20,6 +20,5 @@ -
[flink-connector-elasticsearch] 02/03: [hotfix][legal] Update NOTICE files
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 678ef2d9d1338c649b34ec234936e22a3d47d758 Author: Chesnay Schepler AuthorDate: Wed Sep 14 11:18:23 2022 +0200 [hotfix][legal] Update NOTICE files --- NOTICE| 4 ++-- flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE | 2 +- flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/NOTICE b/NOTICE index 3036712..3745571 100644 --- a/NOTICE +++ b/NOTICE @@ -1,5 +1,5 @@ -Apache Flink -Copyright 2014-2021 The Apache Software Foundation +Apache Flink Elasticsearch Connector +Copyright 2014-2022 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). diff --git a/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE b/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE index 636bc82..e5ece15 100644 --- a/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE +++ b/flink-sql-connector-elasticsearch6/src/main/resources/META-INF/NOTICE @@ -1,5 +1,5 @@ flink-sql-connector-elasticsearch6 -Copyright 2014-2021 The Apache Software Foundation +Copyright 2014-2022 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). diff --git a/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE b/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE index a01e3cb..66f4e92 100644 --- a/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE +++ b/flink-sql-connector-elasticsearch7/src/main/resources/META-INF/NOTICE @@ -1,5 +1,5 @@ flink-sql-connector-elasticsearch7 -Copyright 2014-2021 The Apache Software Foundation +Copyright 2014-2022 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/).
[flink-connector-elasticsearch] branch main updated: [hotfix][tests] Deduplicate ParameterProperty
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
The following commit(s) were added to refs/heads/main by this push:
new 31ae746 [hotfix][tests] Deduplicate ParameterProperty
31ae746 is described below
commit 31ae7467ba846289c4e56118d96a3ae47d81bc68
Author: Chesnay Schepler
AuthorDate: Wed Sep 14 11:13:42 2022 +0200
[hotfix][tests] Deduplicate ParameterProperty
The class is already available part of the flink-test-utils.
---
.../pom.xml| 6 +++
.../flink/test/parameters/ParameterProperty.java | 58 --
2 files changed, 6 insertions(+), 58 deletions(-)
diff --git
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
index b7b6e1d..deca450 100644
---
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
+++
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
@@ -46,6 +46,12 @@ under the License.
${flink.version}
compile
+
+ org.apache.flink
+ flink-test-utils
+ ${flink.version}
+ compile
+
org.apache.flink
flink-runtime
diff --git
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/test/parameters/ParameterProperty.java
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/test/parameters/ParameterProperty.java
deleted file mode 100644
index a2bcfdf..000
---
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/test/parameters/ParameterProperty.java
+++ /dev/null
@@ -1,58 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.flink.test.parameters;
-
-import java.util.Optional;
-import java.util.function.Function;
-
-/** System-property based parameters for tests and resources. */
-public class ParameterProperty {
-
-private final String propertyName;
-private final Function converter;
-
-public ParameterProperty(final String propertyName, final Function converter) {
-this.propertyName = propertyName;
-this.converter = converter;
-}
-
-public String getPropertyName() {
-return propertyName;
-}
-
-/**
- * Retrieves the value of this property.
- *
- * @return Optional containing the value of this property
- */
-public Optional get() {
-final String value = System.getProperty(propertyName);
-return value == null ? Optional.empty() :
Optional.of(converter.apply(value));
-}
-
-/**
- * Retrieves the value of this property, or the given default if no value
was set.
- *
- * @return the value of this property, or the given default if no value
was set
- */
-public V get(final V defaultValue) {
-final String value = System.getProperty(propertyName);
-return value == null ? defaultValue : converter.apply(value);
-}
-}
[flink] branch release-1.15 updated: [FLINK-29262][docs] Document API compatibility guarantees
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.15
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.15 by this push:
new 1afcba0c61c [FLINK-29262][docs] Document API compatibility guarantees
1afcba0c61c is described below
commit 1afcba0c61cdc654554b91ca1f47646c540850bf
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 20:51:24 2022 +0200
[FLINK-29262][docs] Document API compatibility guarantees
---
docs/content.zh/docs/ops/upgrading.md | 30 ++
docs/content/docs/ops/upgrading.md| 30 ++
docs/layouts/shortcodes/check.html| 21 +
docs/layouts/shortcodes/xmark.html| 21 +
4 files changed, 102 insertions(+)
diff --git a/docs/content.zh/docs/ops/upgrading.md
b/docs/content.zh/docs/ops/upgrading.md
index ba8aa3bc142..0fca75d0e8d 100644
--- a/docs/content.zh/docs/ops/upgrading.md
+++ b/docs/content.zh/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream programs are typically designed to run for
long periods of time
This document describes how to update a Flink streaming application and how to
migrate a running streaming application to a different Flink cluster.
+## API compatibility guarantees
+
+The classes & methods of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for each
annotation when upgrading to a particular release:
+
+|Annotation| Major release(Source / Binary) | Minor
release(Source / Binary) | Patch release(Source / Binary) |
+|::|:--:|:--:|:--:|
+| `Public` |{{< xmark >}}/{{< xmark >}} |{{< check >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `PublicEvolving` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `Experimental` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< xmark >}}/{{< xmark >}} |
+
+{{< hint info >}}
+{{< label Example >}}
+Code written against a `PublicEvolving` API in 1.15.2 will continue to run in
1.15.3, without having to recompile the code.
+That same code would have to be recompiled when upgrading to 1.16.0 though.
+{{< /hint >}}
+
## Restarting Streaming Applications
The line of action for upgrading a streaming application or migrating an
application to a different cluster is based on Flink's [Savepoint]({{< ref
"docs/ops/state/savepoints" >}}) feature. A savepoint is a consistent snapshot
of the state of an application at a specific point in time.
diff --git a/docs/content/docs/ops/upgrading.md
b/docs/content/docs/ops/upgrading.md
index 4dd76928b30..d14a03b0d94 100644
--- a/docs/content/docs/ops/upgrading.md
+++ b/docs/content/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream programs are typically designed to run for
long periods of time
This document describes how to update a Flink streaming application and how to
migrate a running streaming application to a different Flink cluster.
+## API compatibility guarantees
+
+The classes & members of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for eac
[flink] branch release-1.16 updated: [FLINK-29262][docs] Document API compatibility guarantees
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.16 by this push:
new a325214cd0b [FLINK-29262][docs] Document API compatibility guarantees
a325214cd0b is described below
commit a325214cd0b47b286dad40b788488533799a6da5
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 20:51:24 2022 +0200
[FLINK-29262][docs] Document API compatibility guarantees
---
docs/content.zh/docs/ops/upgrading.md | 30 ++
docs/content/docs/ops/upgrading.md| 30 ++
docs/layouts/shortcodes/check.html| 21 +
docs/layouts/shortcodes/xmark.html| 21 +
4 files changed, 102 insertions(+)
diff --git a/docs/content.zh/docs/ops/upgrading.md
b/docs/content.zh/docs/ops/upgrading.md
index c9dff9e9d30..2fcca451d89 100644
--- a/docs/content.zh/docs/ops/upgrading.md
+++ b/docs/content.zh/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream 程序通常设计为长时间运行，例如数周、数月甚
本文档介绍了如何更新 Flink 流式应用程序以及如何将正在运行的流式应用程序迁移到不同的 Flink 集群。
+## API compatibility guarantees
+
+The classes & methods of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for each
annotation when upgrading to a particular release:
+
+|Annotation| Major release(Source / Binary) | Minor
release(Source / Binary) | Patch release(Source / Binary) |
+|::|:--:|:--:|:--:|
+| `Public` |{{< xmark >}}/{{< xmark >}} |{{< check >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `PublicEvolving` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `Experimental` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< xmark >}}/{{< xmark >}} |
+
+{{< hint info >}}
+{{< label Example >}}
+Code written against a `PublicEvolving` API in 1.15.2 will continue to run in
1.15.3, without having to recompile the code.
+That same code would have to be recompiled when upgrading to 1.16.0 though.
+{{< /hint >}}
+
## 重启流式应用程序
升级流式应用程序或将应用程序迁移到不同集群的操作线基于 Flink 的 [Savepoint]({{< ref
"docs/ops/state/savepoints" >}}) 功能。Savepoint 是应用程序在特定时间点的状态的一致快照。
diff --git a/docs/content/docs/ops/upgrading.md
b/docs/content/docs/ops/upgrading.md
index 4dd76928b30..d14a03b0d94 100644
--- a/docs/content/docs/ops/upgrading.md
+++ b/docs/content/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream programs are typically designed to run for
long periods of time
This document describes how to update a Flink streaming application and how to
migrate a running streaming application to a different Flink cluster.
+## API compatibility guarantees
+
+The classes & members of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for each
annotation when upgrading to a particular release:
+
+|Annotation| Major release(Source / Binary) | Minor
release(Source / Binary) | Patch release(Source / Binary) |
+|::|:--:|:--:|:--:|
+| `Public` |{{< xmark >}}/{{< xmark >}} |{{< check >}}/{{<
xmark >
[flink] branch master updated: [FLINK-29262][docs] Document API compatibility guarantees
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 030baed8d6e [FLINK-29262][docs] Document API compatibility guarantees
030baed8d6e is described below
commit 030baed8d6e406b83018671d76b817193c258ded
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 20:51:24 2022 +0200
[FLINK-29262][docs] Document API compatibility guarantees
---
docs/content.zh/docs/ops/upgrading.md | 30 ++
docs/content/docs/ops/upgrading.md| 30 ++
docs/layouts/shortcodes/check.html| 21 +
docs/layouts/shortcodes/xmark.html| 21 +
4 files changed, 102 insertions(+)
diff --git a/docs/content.zh/docs/ops/upgrading.md
b/docs/content.zh/docs/ops/upgrading.md
index c9dff9e9d30..2fcca451d89 100644
--- a/docs/content.zh/docs/ops/upgrading.md
+++ b/docs/content.zh/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream 程序通常设计为长时间运行，例如数周、数月甚
本文档介绍了如何更新 Flink 流式应用程序以及如何将正在运行的流式应用程序迁移到不同的 Flink 集群。
+## API compatibility guarantees
+
+The classes & methods of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for each
annotation when upgrading to a particular release:
+
+|Annotation| Major release(Source / Binary) | Minor
release(Source / Binary) | Patch release(Source / Binary) |
+|::|:--:|:--:|:--:|
+| `Public` |{{< xmark >}}/{{< xmark >}} |{{< check >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `PublicEvolving` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< check >}}/{{< check >}} |
+| `Experimental` |{{< xmark >}}/{{< xmark >}} |{{< xmark >}}/{{<
xmark >}} |{{< xmark >}}/{{< xmark >}} |
+
+{{< hint info >}}
+{{< label Example >}}
+Code written against a `PublicEvolving` API in 1.15.2 will continue to run in
1.15.3, without having to recompile the code.
+That same code would have to be recompiled when upgrading to 1.16.0 though.
+{{< /hint >}}
+
## 重启流式应用程序
升级流式应用程序或将应用程序迁移到不同集群的操作线基于 Flink 的 [Savepoint]({{< ref
"docs/ops/state/savepoints" >}}) 功能。Savepoint 是应用程序在特定时间点的状态的一致快照。
diff --git a/docs/content/docs/ops/upgrading.md
b/docs/content/docs/ops/upgrading.md
index 4dd76928b30..d14a03b0d94 100644
--- a/docs/content/docs/ops/upgrading.md
+++ b/docs/content/docs/ops/upgrading.md
@@ -30,6 +30,36 @@ Flink DataStream programs are typically designed to run for
long periods of time
This document describes how to update a Flink streaming application and how to
migrate a running streaming application to a different Flink cluster.
+## API compatibility guarantees
+
+The classes & members of the Java/Scala APIs that are intended for users are
annotated with the following stability annotations:
+* `Public`
+* `PublicEvolving`
+* `Experimental`
+
+{{< hint info>}}
+Annotations on a class also apply to all members of that class, unless
otherwise annotated.
+{{< /hint >}}
+
+Any API without such an annotation is considered internal to Flink, with no
guarantees being provided.
+
+An API that is `source` compatible means that code **written** against the API
will continue to **compile** against a later version.
+An API that is `binary` compatible means that code **compiled** against the
API will continue to **run** against a later version.
+
+This table lists the `source` / `binary` compatibility guarantees for each
annotation when upgrading to a particular release:
+
+|Annotation| Major release(Source / Binary) | Minor
release(Source / Binary) | Patch release(Source / Binary) |
+|::|:--:|:--:|:--:|
+| `Public` |{{< xmark >}}/{{< xmark >}} |{{< check >}}/{{<
xmark >}} |{{&l
[flink] branch master updated: [FLINK-29249][rpc] Drop RpcService#execute/scheduleRunnable
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 0bbee3047ed [FLINK-29249][rpc] Drop RpcService#execute/scheduleRunnable
0bbee3047ed is described below
commit 0bbee3047ed81d39d25fd736946c527e627a2bad
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 20:42:58 2022 +0200
[FLINK-29249][rpc] Drop RpcService#execute/scheduleRunnable
---
.../flink/runtime/rpc/akka/AkkaRpcService.java | 24 -
.../flink/runtime/rpc/akka/AkkaRpcServiceTest.java | 29 +-
.../rpc/akka/ContextClassLoadingSettingTest.java | 103 ++---
.../org/apache/flink/runtime/rpc/RpcService.java | 40
.../runtime/registration/RetryingRegistration.java | 19 ++--
.../runtime/metrics/util/MetricUtilsTest.java | 7 +-
.../registration/RetryingRegistrationTest.java | 10 --
.../flink/runtime/rpc/TestingRpcService.java | 18
.../OperatorEventSendingCheckpointITCase.java | 17
9 files changed, 91 insertions(+), 176 deletions(-)
diff --git
a/flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcService.java
b/flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcService.java
index 0b5a4819164..40bb509 100644
---
a/flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcService.java
+++
b/flink-rpc/flink-rpc-akka/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcService.java
@@ -38,7 +38,6 @@ import org.apache.flink.util.ExecutorUtils;
import org.apache.flink.util.concurrent.ExecutorThreadFactory;
import org.apache.flink.util.concurrent.FutureUtils;
import org.apache.flink.util.concurrent.ScheduledExecutor;
-import org.apache.flink.util.function.FunctionUtils;
import akka.actor.AbstractActor;
import akka.actor.ActorRef;
@@ -64,12 +63,10 @@ import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
-import java.util.concurrent.Callable;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
-import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
@@ -79,7 +76,6 @@ import scala.reflect.ClassTag$;
import static
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.guardCompletionWithContextClassLoader;
import static
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader;
import static
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.withContextClassLoader;
-import static org.apache.flink.util.Preconditions.checkArgument;
import static org.apache.flink.util.Preconditions.checkNotNull;
import static org.apache.flink.util.Preconditions.checkState;
@@ -464,26 +460,6 @@ public class AkkaRpcService implements RpcService {
return internalScheduledExecutor;
}
-@Override
-public ScheduledFuture scheduleRunnable(Runnable runnable, long delay,
TimeUnit unit) {
-checkNotNull(runnable, "runnable");
-checkNotNull(unit, "unit");
-checkArgument(delay >= 0L, "delay must be zero or larger");
-
-return internalScheduledExecutor.schedule(runnable, delay, unit);
-}
-
-@Override
-public void execute(Runnable runnable) {
-getScheduledExecutor().execute(runnable);
-}
-
-@Override
-public CompletableFuture execute(Callable callable) {
-return CompletableFuture.supplyAsync(
-FunctionUtils.uncheckedSupplier(callable::call),
getScheduledExecutor());
-}
-
//
---
// Private helper methods
//
---
diff --git
a/flink-rpc/flink-rpc-akka/src/test/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceTest.java
b/flink-rpc/flink-rpc-akka/src/test/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceTest.java
index f4512f81cd1..1db75ee82bb 100644
---
a/flink-rpc/flink-rpc-akka/src/test/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceTest.java
+++
b/flink-rpc/flink-rpc-akka/src/test/java/org/apache/flink/runtime/rpc/akka/AkkaRpcServiceTest.java
@@ -85,7 +85,6 @@ class AkkaRpcServiceTest {
//
// tests
//
-
@Test
void testScheduleRunnable() throws Exception {
final OneShotLatch latch = new OneShotLatch();
@@ -93,7 +92,9 @@ class AkkaRpcServiceTest {
final long start = System.nanoTime();
[flink-connector-elasticsearch] 03/06: [hotfix][build] Fix flink-annotations version
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit afd378460a58d4064b6ae12a3fc17459bcb0923f
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 12:42:14 2022 +0200
[hotfix][build] Fix flink-annotations version
---
pom.xml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 0a840e6..01efb89 100644
--- a/pom.xml
+++ b/pom.xml
@@ -55,6 +55,7 @@ under the License.
flink-connector-elasticsearch-base
flink-connector-elasticsearch6
flink-connector-elasticsearch7
+ flink-connector-elasticsearch-e2e-tests
@@ -1314,7 +1315,7 @@ under the License.
org.apache.flink
flink-annotations
-
${project.version}
+
${flink.version}
[flink-connector-elasticsearch] 01/06: [hotfix][build] Rat ignores japicmp output
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 86decf0f5326559b236e391aa245adf95ae334b4 Author: Chesnay Schepler AuthorDate: Tue Sep 13 12:09:57 2022 +0200 [hotfix][build] Rat ignores japicmp output --- pom.xml | 1 + 1 file changed, 1 insertion(+) diff --git a/pom.xml b/pom.xml index 49acc98..0a840e6 100644 --- a/pom.xml +++ b/pom.xml @@ -830,6 +830,7 @@ under the License. **/*.iml + tools/japicmp-output/** out/** **/target/** docs/layouts/shortcodes/generated/**
[flink-connector-elasticsearch] branch main updated (19dfe58 -> 06e38d1)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git from 19dfe58 [hotfix] Remove usage of deprecated RestHighLevelClient#bulkAsync new 86decf0 [hotfix][build] Rat ignores japicmp output new 6823b86 [hotfix][ci] Upgrade Maven to 3.8.6 new afd3784 [hotfix][build] Fix flink-annotations version new a9328df [FLINK-28410][tests] Sync E2E tests new b793850 [hotfix][build] Remove unnecessary surefire config new 06e38d1 [hotfix][ci] Hide Maven download transfer The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/ci.yml | 17 +- .../pom.xml| 77 + .../flink/streaming/tests/ElasticsearchClient.java | 59 +++ .../streaming/tests/ElasticsearchDataReader.java | 57 +++ .../tests/ElasticsearchSinkE2ECaseBase.java| 101 .../ElasticsearchSinkExternalContextBase.java | 123 +++ ...lasticsearchSinkExternalContextFactoryBase.java | 59 +++ .../streaming/tests/ElasticsearchTestEmitter.java | 51 ++ .../org/apache/flink/streaming/tests/KeyValue.java | 92 +++ .../apache/flink/streaming/tests/QueryParams.java | 174 + .../streaming/tests/UpdateRequestFactory.java | 43 + .../flink/test/parameters/ParameterProperty.java | 58 +++ .../org/apache/flink/tests/util/TestUtils.java | 85 ++ .../pom.xml| 120 ++ .../streaming/tests/Elasticsearch6Client.java | 149 ++ .../streaming/tests/UpdateRequest6Factory.java | 48 ++ .../streaming/tests/Elasticsearch6SinkE2ECase.java | 60 +++ .../tests/Elasticsearch6SinkExternalContext.java | 68 .../Elasticsearch6SinkExternalContextFactory.java | 48 ++ .../src/test/resources/log4j2-test.properties | 24 +-- .../pom.xml| 121 ++ .../streaming/tests/Elasticsearch7Client.java | 147 + .../streaming/tests/UpdateRequest7Factory.java | 46 ++ .../streaming/tests/Elasticsearch7SinkE2ECase.java | 60 +++ .../tests/Elasticsearch7SinkExternalContext.java | 68 .../Elasticsearch7SinkExternalContextFactory.java | 48 ++ .../src/test/resources/log4j2-test.properties | 23 ++- flink-connector-elasticsearch-e2e-tests/pom.xml| 128 +++ pom.xml| 11 +- 29 files changed, 2137 insertions(+), 28 deletions(-) create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchClient.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchDataReader.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchSinkE2ECaseBase.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchSinkExternalContextBase.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchSinkExternalContextFactoryBase.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/ElasticsearchTestEmitter.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/KeyValue.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/QueryParams.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/streaming/tests/UpdateRequestFactory.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/src/main/java/org/apache/flink/test/parameters/ParameterProperty.java create mode 100644 flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-
[flink-connector-elasticsearch] 06/06: [hotfix][ci] Hide Maven download transfer
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 06e38d1e2d8d998e06065c011098f85d0235860c
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 14:34:23 2022 +0200
[hotfix][ci] Hide Maven download transfer
---
.github/workflows/ci.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9aff109..f4affdd 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -51,7 +51,7 @@ jobs:
&& wget -q -c ${{ env.FLINK_URL }} -O - | tar -xz \
&& popd
- mvn clean install -U -B \
+ mvn clean install -U -B --no-transfer-progress \
-Dscala-2.12 \
-Prun-end-to-end-tests -DdistDir=$(pwd)/../flink-1.16-SNAPSHOT \
-Dflink.convergence.phase=install -Pcheck-convergence \
[flink-connector-elasticsearch] 05/06: [hotfix][build] Remove unnecessary surefire config
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit b793850a2c7d3717dfc25c72bd0b5a59ff856442 Author: Chesnay Schepler AuthorDate: Tue Sep 13 13:05:04 2022 +0200 [hotfix][build] Remove unnecessary surefire config --- pom.xml | 7 --- 1 file changed, 7 deletions(-) diff --git a/pom.xml b/pom.xml index 01efb89..e0336fb 100644 --- a/pom.xml +++ b/pom.xml @@ -427,13 +427,6 @@ under the License. - - org.apache.maven.plugins - maven-surefire-plugin - - org.apache.flink.testutils.junit.FailsOnJava11 - - org.apache.maven.plugins maven-javadoc-plugin
[flink-connector-elasticsearch] 02/06: [hotfix][ci] Upgrade Maven to 3.8.6
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 6823b8656c896dffa6dc8b132762733c5ac44b64
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 12:32:43 2022 +0200
[hotfix][ci] Upgrade Maven to 3.8.6
---
.github/workflows/ci.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3700352..c23d82c 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -39,10 +39,10 @@ jobs:
distribution: 'temurin'
cache: 'maven'
- - name: Set Maven 3.8.5
+ - name: Set Maven 3.8.6
uses: stCarolas/setup-maven@v4.2
with:
- maven-version: 3.8.5
+ maven-version: 3.8.6
- name: Compile and test flink-connector-elasticsearch
run: mvn clean install -Dscala-2.12 -Dflink.convergence.phase=install
-Pcheck-convergence -U -B ${{ env.MVN_CONNECTION_OPTIONS }}
-Dlog4j.configurationFile=file://$(pwd)/tools/ci/log4j.properties
[flink-connector-elasticsearch] 04/06: [FLINK-28410][tests] Sync E2E tests
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit a9328dfcd7793b73cddc428d4c4a0aa827d220ca
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 12:42:47 2022 +0200
[FLINK-28410][tests] Sync E2E tests
---
.github/workflows/ci.yml | 13 +-
.../pom.xml| 77 +
.../flink/streaming/tests/ElasticsearchClient.java | 59 +++
.../streaming/tests/ElasticsearchDataReader.java | 57 +++
.../tests/ElasticsearchSinkE2ECaseBase.java| 101
.../ElasticsearchSinkExternalContextBase.java | 123 +++
...lasticsearchSinkExternalContextFactoryBase.java | 59 +++
.../streaming/tests/ElasticsearchTestEmitter.java | 51 ++
.../org/apache/flink/streaming/tests/KeyValue.java | 92 +++
.../apache/flink/streaming/tests/QueryParams.java | 174 +
.../streaming/tests/UpdateRequestFactory.java | 43 +
.../flink/test/parameters/ParameterProperty.java | 58 +++
.../org/apache/flink/tests/util/TestUtils.java | 85 ++
.../pom.xml| 120 ++
.../streaming/tests/Elasticsearch6Client.java | 149 ++
.../streaming/tests/UpdateRequest6Factory.java | 48 ++
.../streaming/tests/Elasticsearch6SinkE2ECase.java | 60 +++
.../tests/Elasticsearch6SinkExternalContext.java | 68
.../Elasticsearch6SinkExternalContextFactory.java | 48 ++
.../src/test/resources/log4j2-test.properties | 34
.../pom.xml| 121 ++
.../streaming/tests/Elasticsearch7Client.java | 147 +
.../streaming/tests/UpdateRequest7Factory.java | 46 ++
.../streaming/tests/Elasticsearch7SinkE2ECase.java | 60 +++
.../tests/Elasticsearch7SinkExternalContext.java | 68
.../Elasticsearch7SinkExternalContextFactory.java | 48 ++
.../src/test/resources/log4j2-test.properties | 35 +
flink-connector-elasticsearch-e2e-tests/pom.xml| 128 +++
28 files changed, 2171 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c23d82c..9aff109 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -26,6 +26,7 @@ jobs:
jdk: [8, 11]
env:
MVN_CONNECTION_OPTIONS: -Dhttp.keepAlive=false
-Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
+ FLINK_URL:
https://s3.amazonaws.com/flink-nightly/flink-1.16-SNAPSHOT-bin-scala_2.12.tgz
steps:
- run: echo "Running CI pipeline for JDK version ${{ matrix.jdk }}"
@@ -45,4 +46,14 @@ jobs:
maven-version: 3.8.6
- name: Compile and test flink-connector-elasticsearch
-run: mvn clean install -Dscala-2.12 -Dflink.convergence.phase=install
-Pcheck-convergence -U -B ${{ env.MVN_CONNECTION_OPTIONS }}
-Dlog4j.configurationFile=file://$(pwd)/tools/ci/log4j.properties
+run: |
+ pushd .. \
+&& wget -q -c ${{ env.FLINK_URL }} -O - | tar -xz \
+&& popd
+
+ mvn clean install -U -B \
+-Dscala-2.12 \
+-Prun-end-to-end-tests -DdistDir=$(pwd)/../flink-1.16-SNAPSHOT \
+-Dflink.convergence.phase=install -Pcheck-convergence \
+${{ env.MVN_CONNECTION_OPTIONS }} \
+-Dlog4j.configurationFile=file://$(pwd)/tools/ci/log4j.properties
diff --git
a/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
new file mode 100644
index 000..b7b6e1d
--- /dev/null
+++
b/flink-connector-elasticsearch-e2e-tests/flink-connector-elasticsearch-e2e-tests-common/pom.xml
@@ -0,0 +1,77 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+ 4.0.0
+
+
+ org.apache.flink
+ flink-connector-elasticsearch-e2e-tests
+ 3.0-SNAPSHOT
+
+
+ flink-connector-elasticsearch-e2e-tests-common
+ Flink : Connectors : Elasticsearch : E2E tests : Common
+ jar
+
+
+
+ org.apache.flink
+ flink-streaming-java
+ ${flink.version}
+ provided
+
+
+ org.apache.flink
+ flink-connector-test-utils
+ ${flink.v
[flink] 01/02: [FLINK-29260][release] Wipe exclusion list when updating reference version
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.15
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 771000c511031b4c24917addd4cea494ac4bb48e
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 14:19:50 2022 +0200
[FLINK-29260][release] Wipe exclusion list when updating reference version
---
pom.xml | 2 ++
tools/releasing/update_japicmp_configuration.sh | 11 +++
2 files changed, 13 insertions(+)
diff --git a/pom.xml b/pom.xml
index 5791fc4f90b..1475afe30bc 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2140,6 +2140,7 @@ under the License.
@org.apache.flink.annotation.Experimental
@org.apache.flink.annotation.Internal
+
org.apache.flink.streaming.api.datastream.DataStream#DataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,org.apache.flink.streaming.api.transformations.StreamTransformation)
org.apache.flink.streaming.api.environment.LegacyLocalStreamEnvironment
org.apache.flink.streaming.api.functions.sink.RichSinkFunction#invoke(java.lang.Object)
@@ -2152,6 +2153,7 @@ under the License.
org.apache.flink.api.scala.hadoop.mapred.HadoopOutputFormat
org.apache.flink.api.scala.hadoop.mapreduce.HadoopInputFormat
org.apache.flink.api.scala.hadoop.mapreduce.HadoopOutputFormat
+
public
false
diff --git a/tools/releasing/update_japicmp_configuration.sh
b/tools/releasing/update_japicmp_configuration.sh
index e8678a7c765..45a4661c994 100755
--- a/tools/releasing/update_japicmp_configuration.sh
+++ b/tools/releasing/update_japicmp_configuration.sh
@@ -59,6 +59,15 @@ function set_japicmp_reference_version() {
perl -pi -e
's#().*()#${1}'${version}'${2}#'
${POM}
}
+function clear_exclusions() {
+ exclusion_start=$(($(sed -n '/
[flink] branch release-1.15 updated (776ccfa27d2 -> 4e8247cb855)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git from 776ccfa27d2 [FLINK-29253][runtime] Removes synchronous close call from DefaultJobManagerRunnerRegistry#localCleanupAsync new 771000c5110 [FLINK-29260][release] Wipe exclusion list when updating reference version new 4e8247cb855 [hotfix][build] Wipe legacy japicmp exclusions The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: pom.xml | 14 ++ tools/releasing/update_japicmp_configuration.sh | 11 +++ 2 files changed, 13 insertions(+), 12 deletions(-)
[flink] 02/02: [hotfix][build] Wipe legacy japicmp exclusions
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git commit 4e8247cb8559ce39166fb30c96855fffbc12a73b Author: Chesnay Schepler AuthorDate: Tue Sep 13 14:23:25 2022 +0200 [hotfix][build] Wipe legacy japicmp exclusions --- pom.xml | 12 1 file changed, 12 deletions(-) diff --git a/pom.xml b/pom.xml index 1475afe30bc..458376b558b 100644 --- a/pom.xml +++ b/pom.xml @@ -2141,18 +2141,6 @@ under the License. @org.apache.flink.annotation.Experimental @org.apache.flink.annotation.Internal - org.apache.flink.streaming.api.datastream.DataStream#DataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,org.apache.flink.streaming.api.transformations.StreamTransformation) - org.apache.flink.streaming.api.environment.LegacyLocalStreamEnvironment - org.apache.flink.streaming.api.functions.sink.RichSinkFunction#invoke(java.lang.Object) - org.apache.flink.streaming.api.functions.sink.SinkFunction - org.apache.flink.api.java.hadoop.mapred.HadoopInputFormat - org.apache.flink.api.java.hadoop.mapred.HadoopOutputFormat - org.apache.flink.api.java.hadoop.mapreduce.HadoopInputFormat - org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat - org.apache.flink.api.scala.hadoop.mapred.HadoopInputFormat - org.apache.flink.api.scala.hadoop.mapred.HadoopOutputFormat - org.apache.flink.api.scala.hadoop.mapreduce.HadoopInputFormat - org.apache.flink.api.scala.hadoop.mapreduce.HadoopOutputFormat public
[flink] branch release-1.16 updated: [FLINK-29260][release] Wipe exclusion list when updating reference version
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.16 by this push:
new ebd8d8e0341 [FLINK-29260][release] Wipe exclusion list when updating
reference version
ebd8d8e0341 is described below
commit ebd8d8e0341ccd096d5621f4806a01a94d2b52e7
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 14:19:50 2022 +0200
[FLINK-29260][release] Wipe exclusion list when updating reference version
---
pom.xml | 2 ++
tools/releasing/update_japicmp_configuration.sh | 11 +++
2 files changed, 13 insertions(+)
diff --git a/pom.xml b/pom.xml
index 1e0a18270a4..fde290bb61f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2160,6 +2160,7 @@ under the License.
@org.apache.flink.annotation.Experimental
@org.apache.flink.annotation.PublicEvolving
@org.apache.flink.annotation.Internal
+
org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator#getSideOutput(org.apache.flink.util.OutputTag)
org.apache.flink.streaming.api.datastream.DataStream#DataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment,org.apache.flink.streaming.api.transformations.StreamTransformation)
org.apache.flink.streaming.api.environment.LegacyLocalStreamEnvironment
@@ -2180,6 +2181,7 @@ under the License.
org.apache.flink.metrics.Histogram#getMetricType()
org.apache.flink.api.connector.source.SplitEnumeratorContext#sendEventToSourceReader(int,int,org.apache.flink.api.connector.source.SourceEvent)
org.apache.flink.api.connector.source.SplitEnumeratorContext#registeredReadersOfAttempts()
+
public
false
diff --git a/tools/releasing/update_japicmp_configuration.sh
b/tools/releasing/update_japicmp_configuration.sh
index e8678a7c765..45a4661c994 100755
--- a/tools/releasing/update_japicmp_configuration.sh
+++ b/tools/releasing/update_japicmp_configuration.sh
@@ -59,6 +59,15 @@ function set_japicmp_reference_version() {
perl -pi -e
's#().*()#${1}'${version}'${2}#'
${POM}
}
+function clear_exclusions() {
+ exclusion_start=$(($(sed -n '/
[flink] branch master updated (3ff0ef8474f -> 19ddba04a4b)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 3ff0ef8474f [FLINK-27660][table] Improve Javadoc of TableEnvironment#createFunction add 19ddba04a4b [FLINK-29260][release] Wipe exclusion list when updating reference version No new revisions were added by this update. Summary of changes: pom.xml | 2 ++ tools/releasing/update_japicmp_configuration.sh | 11 +++ 2 files changed, 13 insertions(+)
[flink-connector-elasticsearch] branch main updated: [hotfix] Remove usage of deprecated RestHighLevelClient#bulkAsync
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
The following commit(s) were added to refs/heads/main by this push:
new 19dfe58 [hotfix] Remove usage of deprecated
RestHighLevelClient#bulkAsync
19dfe58 is described below
commit 19dfe58c6d786d6069a6375cd92a5a7755166a5c
Author: Sergey Nuyanzin
AuthorDate: Tue Sep 13 12:44:17 2022 +0200
[hotfix] Remove usage of deprecated RestHighLevelClient#bulkAsync
---
.../connector/elasticsearch/sink/Elasticsearch6SinkBuilder.java | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git
a/flink-connector-elasticsearch6/src/main/java/org/apache/flink/connector/elasticsearch/sink/Elasticsearch6SinkBuilder.java
b/flink-connector-elasticsearch6/src/main/java/org/apache/flink/connector/elasticsearch/sink/Elasticsearch6SinkBuilder.java
index c90ccac..f9f1ae8 100644
---
a/flink-connector-elasticsearch6/src/main/java/org/apache/flink/connector/elasticsearch/sink/Elasticsearch6SinkBuilder.java
+++
b/flink-connector-elasticsearch6/src/main/java/org/apache/flink/connector/elasticsearch/sink/Elasticsearch6SinkBuilder.java
@@ -25,6 +25,7 @@ import org.elasticsearch.action.bulk.BackoffPolicy;
import org.elasticsearch.action.bulk.BulkProcessor;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
+import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.unit.ByteSizeUnit;
import org.elasticsearch.common.unit.ByteSizeValue;
@@ -83,7 +84,10 @@ public class Elasticsearch6SinkBuilder
BulkRequest bulkRequest,
ActionListener
bulkResponseActionListener) {
-client.bulkAsync(bulkRequest,
bulkResponseActionListener);
+client.bulkAsync(
+bulkRequest,
+RequestOptions.DEFAULT,
+bulkResponseActionListener);
}
},
listener);
[flink] branch release-1.16 updated (438eddefbf1 -> 30d6e6122a5)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git from 438eddefbf1 [FLINK-29238] Wrong index information will be obtained after the downstream failover in hybrid full mode new 414d3898d4e [FLINK-25868][build] Limit japicmp to jar modules new 84d9e211457 [FLINK-25868][build] Disable japicmp in packaging/e2e modules new 72da6187355 [FLINK-25868][build] Use custom japicmp plugin version new 30d6e6122a5 [FLINK-25868][build] Enable japicmp for all modules The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: flink-connectors/flink-hadoop-compatibility/pom.xml | 5 - .../flink-sql-connector-aws-kinesis-firehose/pom.xml| 4 .../flink-sql-connector-aws-kinesis-streams/pom.xml | 4 .../flink-sql-connector-elasticsearch6/pom.xml | 4 .../flink-sql-connector-elasticsearch7/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-1.4/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-2.2/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.3.9/pom.xml | 4 flink-connectors/flink-sql-connector-hive-3.1.2/pom.xml | 4 flink-connectors/flink-sql-connector-kafka/pom.xml | 4 flink-connectors/flink-sql-connector-kinesis/pom.xml| 4 flink-connectors/flink-sql-connector-pulsar/pom.xml | 4 flink-connectors/flink-sql-connector-rabbitmq/pom.xml | 4 flink-core/pom.xml | 6 -- flink-dist-scala/pom.xml| 4 flink-dist/pom.xml | 1 + flink-docs/pom.xml | 1 + flink-end-to-end-tests/pom.xml | 1 + flink-examples/pom.xml | 4 flink-filesystems/flink-fs-hadoop-shaded/pom.xml| 4 flink-filesystems/flink-s3-fs-base/pom.xml | 1 + flink-formats/flink-sql-avro-confluent-registry/pom.xml | 4 flink-formats/flink-sql-avro/pom.xml| 4 flink-formats/flink-sql-csv/pom.xml | 4 flink-formats/flink-sql-json/pom.xml| 4 flink-formats/flink-sql-orc/pom.xml | 4 flink-formats/flink-sql-parquet/pom.xml | 4 flink-formats/flink-sql-protobuf/pom.xml| 4 flink-java/pom.xml | 6 -- flink-metrics/flink-metrics-core/pom.xml| 5 - flink-scala/pom.xml | 6 -- flink-streaming-java/pom.xml| 7 --- flink-streaming-scala/pom.xml | 11 +-- flink-table/flink-table-planner-loader-bundle/pom.xml | 4 flink-yarn-tests/pom.xml| 4 pom.xml | 17 + tools/ci/java-ci-tools/pom.xml | 4 37 files changed, 118 insertions(+), 49 deletions(-)
[flink] 04/04: [FLINK-25868][build] Enable japicmp for all modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git commit 30d6e6122a50c6353bc2fb17cd7414e1724ab3df Author: Chesnay Schepler AuthorDate: Tue Feb 1 15:10:37 2022 +0100 [FLINK-25868][build] Enable japicmp for all modules --- flink-connectors/flink-hadoop-compatibility/pom.xml | 5 - flink-core/pom.xml | 6 -- flink-java/pom.xml | 6 -- flink-metrics/flink-metrics-core/pom.xml| 5 - flink-scala/pom.xml | 6 -- flink-streaming-java/pom.xml| 7 --- flink-streaming-scala/pom.xml | 9 - pom.xml | 6 ++ 8 files changed, 6 insertions(+), 44 deletions(-) diff --git a/flink-connectors/flink-hadoop-compatibility/pom.xml b/flink-connectors/flink-hadoop-compatibility/pom.xml index cd8e0b7d468..414ce38d6ed 100644 --- a/flink-connectors/flink-hadoop-compatibility/pom.xml +++ b/flink-connectors/flink-hadoop-compatibility/pom.xml @@ -115,11 +115,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - net.alchim31.maven diff --git a/flink-core/pom.xml b/flink-core/pom.xml index c877e49b65f..95d33baf116 100644 --- a/flink-core/pom.xml +++ b/flink-core/pom.xml @@ -139,12 +139,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins diff --git a/flink-java/pom.xml b/flink-java/pom.xml index 3ae4cc520ed..7f72eba43bf 100644 --- a/flink-java/pom.xml +++ b/flink-java/pom.xml @@ -69,12 +69,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins diff --git a/flink-metrics/flink-metrics-core/pom.xml b/flink-metrics/flink-metrics-core/pom.xml index 73db2963f44..baf32d77c44 100644 --- a/flink-metrics/flink-metrics-core/pom.xml +++ b/flink-metrics/flink-metrics-core/pom.xml @@ -48,11 +48,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - org.apache.maven.plugins maven-jar-plugin diff --git a/flink-scala/pom.xml b/flink-scala/pom.xml index ac8b96642d7..7afbdbf6fb6 100644 --- a/flink-scala/pom.xml +++ b/flink-scala/pom.xml @@ -155,12 +155,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins maven-surefire-plugin diff --git a/flink-streaming-java/pom.xml b/flink-streaming-java/pom.xml index 18d1330ed2c..3b1351d6ec7 100644 --- a/flink-streaming-java/pom.xml +++ b/flink-streaming-java/pom.xml @@ -98,13 +98,6 @@ under the License. - - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins maven-jar-plugin diff --git a/flink-streaming-scala/pom.xml b/flink-streaming-scala/pom.xml index 319f3830819..e008f13ed44 100644 --- a/flink-streaming-scala/pom.xml +++ b/flink-streaming-scala/pom.xml @@ -254,7 +254,6 @@ under the License. - io.github.zentol.japicmp japicmp-maven-plugin @@ -272,14 +271,6 @@ under the License
[flink] 02/04: [FLINK-25868][build] Disable japicmp in packaging/e2e modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git commit 84d9e21145702eb8432ee2024b209d083c2e27e4 Author: Chesnay Schepler AuthorDate: Tue Feb 1 15:09:19 2022 +0100 [FLINK-25868][build] Disable japicmp in packaging/e2e modules Optimize build times and reduce noise (warnings would be printed as these modules are not being deployed, so the previous versions can not be found). --- flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml | 4 flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml | 4 flink-connectors/flink-sql-connector-elasticsearch6/pom.xml | 4 flink-connectors/flink-sql-connector-elasticsearch7/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-1.4/pom.xml| 4 flink-connectors/flink-sql-connector-hbase-2.2/pom.xml| 4 flink-connectors/flink-sql-connector-hive-2.3.9/pom.xml | 4 flink-connectors/flink-sql-connector-hive-3.1.2/pom.xml | 4 flink-connectors/flink-sql-connector-kafka/pom.xml| 4 flink-connectors/flink-sql-connector-kinesis/pom.xml | 4 flink-connectors/flink-sql-connector-pulsar/pom.xml | 4 flink-connectors/flink-sql-connector-rabbitmq/pom.xml | 4 flink-dist-scala/pom.xml | 4 flink-dist/pom.xml| 1 + flink-docs/pom.xml| 1 + flink-end-to-end-tests/pom.xml| 1 + flink-examples/pom.xml| 4 flink-filesystems/flink-fs-hadoop-shaded/pom.xml | 4 flink-filesystems/flink-s3-fs-base/pom.xml| 1 + flink-formats/flink-sql-avro-confluent-registry/pom.xml | 4 flink-formats/flink-sql-avro/pom.xml | 4 flink-formats/flink-sql-csv/pom.xml | 4 flink-formats/flink-sql-json/pom.xml | 4 flink-formats/flink-sql-orc/pom.xml | 4 flink-formats/flink-sql-parquet/pom.xml | 4 flink-formats/flink-sql-protobuf/pom.xml | 4 flink-table/flink-table-planner-loader-bundle/pom.xml | 4 flink-yarn-tests/pom.xml | 4 tools/ci/java-ci-tools/pom.xml| 4 29 files changed, 104 insertions(+) diff --git a/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml b/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml index 2775b732599..883f84c1987 100644 --- a/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml +++ b/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml @@ -30,6 +30,10 @@ flink-sql-connector-aws-kinesis-firehose Flink : Connectors : SQL : Amazon Kinesis Data Firehose + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml b/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml index 69e34b6965b..a76b413a7f5 100644 --- a/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml +++ b/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml @@ -30,6 +30,10 @@ flink-sql-connector-aws-kinesis-streams Flink : Connectors : SQL : Amazon Kinesis Data Streams + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml b/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml index a3f8651b4f7..0e2204c67df 100644 --- a/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml +++ b/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml @@ -34,6 +34,10 @@ under the License. jar + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml b/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml index 15292f9b68d..72552d0febd 100644 --- a/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml +++ b/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml @@ -34,6 +34,10 @@ under the License. jar + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-hbase-1.4/pom.xml b/flink-connectors/flink-sql-connector-hbase-1.4/pom.xml index ceb947f1be5..0ffd518ba6b 100644
[flink] 03/04: [FLINK-25868][build] Use custom japicmp plugin version
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git commit 72da6187355af28273078e583a2d644a8218cd84 Author: Chesnay Schepler AuthorDate: Wed Aug 31 16:27:17 2022 +0200 [FLINK-25868][build] Use custom japicmp plugin version --- flink-connectors/flink-hadoop-compatibility/pom.xml | 2 +- flink-core/pom.xml | 2 +- flink-java/pom.xml | 2 +- flink-metrics/flink-metrics-core/pom.xml| 2 +- flink-scala/pom.xml | 2 +- flink-streaming-java/pom.xml| 2 +- flink-streaming-scala/pom.xml | 2 +- pom.xml | 8 8 files changed, 11 insertions(+), 11 deletions(-) diff --git a/flink-connectors/flink-hadoop-compatibility/pom.xml b/flink-connectors/flink-hadoop-compatibility/pom.xml index f0a12b3b6da..cd8e0b7d468 100644 --- a/flink-connectors/flink-hadoop-compatibility/pom.xml +++ b/flink-connectors/flink-hadoop-compatibility/pom.xml @@ -117,7 +117,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-core/pom.xml b/flink-core/pom.xml index 45be24efc05..c877e49b65f 100644 --- a/flink-core/pom.xml +++ b/flink-core/pom.xml @@ -141,7 +141,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-java/pom.xml b/flink-java/pom.xml index aa3eb516775..3ae4cc520ed 100644 --- a/flink-java/pom.xml +++ b/flink-java/pom.xml @@ -71,7 +71,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-metrics/flink-metrics-core/pom.xml b/flink-metrics/flink-metrics-core/pom.xml index 4c3a29eea25..73db2963f44 100644 --- a/flink-metrics/flink-metrics-core/pom.xml +++ b/flink-metrics/flink-metrics-core/pom.xml @@ -50,7 +50,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-scala/pom.xml b/flink-scala/pom.xml index ec9be622cb0..ac8b96642d7 100644 --- a/flink-scala/pom.xml +++ b/flink-scala/pom.xml @@ -157,7 +157,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-streaming-java/pom.xml b/flink-streaming-java/pom.xml index b75691b1456..18d1330ed2c 100644 --- a/flink-streaming-java/pom.xml +++ b/flink-streaming-java/pom.xml @@ -101,7 +101,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-streaming-scala/pom.xml b/flink-streaming-scala/pom.xml index 3304aa5d057..319f3830819 100644 --- a/flink-streaming-scala/pom.xml +++ b/flink-streaming-scala/pom.xml @@ -256,7 +256,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/pom.xml b/pom.xml index 3f65da6f12d..3aafd7cc5a2 100644 --- a/pom.xml +++ b/pom.xml @@ -964,7 +964,7 @@ under the License. 3.2.4 - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin
[flink] 01/04: [FLINK-25868][build] Limit japicmp to jar modules
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.16
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 414d3898d4e966a437b723fa39b74c1d3647f94f
Author: Chesnay Schepler
AuthorDate: Tue Feb 1 15:08:05 2022 +0100
[FLINK-25868][build] Limit japicmp to jar modules
Skips quickstart and pom modules.
---
pom.xml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/pom.xml b/pom.xml
index 1e95cb98ae2..3f65da6f12d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2185,6 +2185,9 @@ under the License.
true
true
+
+
jar
+
${rootDir}/${japicmp.outputDir}/${project.artifactId}
[flink-connector-elasticsearch] 05/06: [FLINK-25868][build] Use custom japicmp plugin version
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 637f564767ba628bb9d600edbee623cda0aaa4a7 Author: Chesnay Schepler AuthorDate: Tue Sep 13 11:05:44 2022 +0200 [FLINK-25868][build] Use custom japicmp plugin version --- pom.xml | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pom.xml b/pom.xml index 47ecde4..5030069 100644 --- a/pom.xml +++ b/pom.xml @@ -401,7 +401,7 @@ under the License. 3.2.4 - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin @@ -568,7 +568,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin true @@ -1261,9 +1261,9 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin - 0.11.0 + 0.16.0_m325
[flink-connector-elasticsearch] 01/06: [FLINK-29264] Given parent module a proper artifact ID
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 787e772f5335f7b1c53defd3ad753dffbc9de3d9 Author: Chesnay Schepler AuthorDate: Mon Sep 12 16:39:08 2022 +0200 [FLINK-29264] Given parent module a proper artifact ID --- flink-connector-elasticsearch-base/pom.xml | 2 +- flink-connector-elasticsearch6/pom.xml | 2 +- flink-connector-elasticsearch7/pom.xml | 2 +- flink-sql-connector-elasticsearch6/pom.xml | 2 +- flink-sql-connector-elasticsearch7/pom.xml | 2 +- pom.xml| 6 +++--- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/flink-connector-elasticsearch-base/pom.xml b/flink-connector-elasticsearch-base/pom.xml index 51430cb..6a749c8 100644 --- a/flink-connector-elasticsearch-base/pom.xml +++ b/flink-connector-elasticsearch-base/pom.xml @@ -25,7 +25,7 @@ under the License. org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT diff --git a/flink-connector-elasticsearch6/pom.xml b/flink-connector-elasticsearch6/pom.xml index de6c864..8f4be8e 100644 --- a/flink-connector-elasticsearch6/pom.xml +++ b/flink-connector-elasticsearch6/pom.xml @@ -25,7 +25,7 @@ under the License. org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT diff --git a/flink-connector-elasticsearch7/pom.xml b/flink-connector-elasticsearch7/pom.xml index 1ae39e2..3b7a413 100644 --- a/flink-connector-elasticsearch7/pom.xml +++ b/flink-connector-elasticsearch7/pom.xml @@ -25,7 +25,7 @@ under the License. org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT diff --git a/flink-sql-connector-elasticsearch6/pom.xml b/flink-sql-connector-elasticsearch6/pom.xml index a3f8651..e70515c 100644 --- a/flink-sql-connector-elasticsearch6/pom.xml +++ b/flink-sql-connector-elasticsearch6/pom.xml @@ -25,7 +25,7 @@ under the License. org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT diff --git a/flink-sql-connector-elasticsearch7/pom.xml b/flink-sql-connector-elasticsearch7/pom.xml index 15292f9..4b7bdeb 100644 --- a/flink-sql-connector-elasticsearch7/pom.xml +++ b/flink-sql-connector-elasticsearch7/pom.xml @@ -25,7 +25,7 @@ under the License. org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT diff --git a/pom.xml b/pom.xml index cf49ee0..b776204 100644 --- a/pom.xml +++ b/pom.xml @@ -28,9 +28,9 @@ under the License. 4.0.0 org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent 1.16-SNAPSHOT - Flink : Connectors : + Flink : Connectors : Elasticsearch : Parent pom https://flink.apache.org 2022 @@ -1165,7 +1165,7 @@ under the License. rootDir org.apache.flink - flink-connectors + flink-connector-elasticsearch-parent
[flink-connector-elasticsearch] 03/06: [FLINK-29266] Cleanup root pom
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 965a651ce010f3b0f50e9da3e0b36952231e264d
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 16:46:47 2022 +0200
[FLINK-29266] Cleanup root pom
---
pom.xml | 86 +++--
1 file changed, 4 insertions(+), 82 deletions(-)
diff --git a/pom.xml b/pom.xml
index 83e86d5..47ecde4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -63,7 +63,6 @@ under the License.
1.16-SNAPSHOT
15.0
-
2.12.4
-
- org.apache.flink
- flink-shaded-asm-9
- 9.2-${flink.shaded.version}
-
-
-
- org.apache.flink
- flink-shaded-guava
-
30.1.1-jre-${flink.shaded.version}
-
-
-
- org.apache.flink
- flink-shaded-jackson
-
${flink.shaded.jackson.version}-${flink.shaded.version}
-
-
-
- org.apache.flink
-
flink-shaded-jackson-module-jsonSchema
-
${flink.shaded.jackson.version}-${flink.shaded.version}
-
-
-
- org.apache.flink
- flink-shaded-netty
-
4.1.70.Final-${flink.shaded.version}
-
-
-
- org.apache.flink
-
flink-shaded-netty-tcnative-dynamic
-
2.0.44.Final-${flink.shaded.version}
- test
-
-
@@ -336,12 +294,6 @@ under the License.
${log4j.version}
-
- org.apache.commons
- commons-lang3
- 3.3.2
-
-
com.fasterxml.jackson
@@ -373,13 +325,7 @@ under the License.
test
-
- org.xerial.snappy
- snappy-java
- 1.1.8.3
-
-
-
+
com.esotericsoftware.kryo
kryo
@@ -393,18 +339,6 @@ under the License.
2.1
-
- org.scala-lang
- scala-compiler
- ${scala.version}
-
-
-
- org.scala-lang
- scala-library
- ${scala.version}
-
-
org.testcontainers
testcontainers-bom
@@ -1026,10 +960,10 @@ under the License.
-
org.apache.flink:flink-table-planner_${scala.binary.version}
+
org.apache.flink:flink-table-planner_*
-
org.apache.flink:flink-table-planner_${scala.binary.version}:*:*:test
+
org.apache.flink:flink-table-planner_*:*:*:test
Direct dependencies on flink-table-planner are not allowed.
@@ -1356,18 +1290,6 @@ under the License
[flink-connector-elasticsearch] branch main updated (f3ce7e1 -> c0e388b)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git from f3ce7e1 [FLINK-26961][build] Update Jackson to 2.13.2 new 787e772 [FLINK-29264] Given parent module a proper artifact ID new 5e7dae4 [FLINK-29265] Update version to 3.0-SNAPSHOT new 965a651 [FLINK-29266] Cleanup root pom new 366950e [FLINK-25868][build] Disable japicmp in packaging/e2e modules new 637f564 [FLINK-25868][build] Use custom japicmp plugin version new c0e388b [FLINK-25868][build] Enable japicmp for all modules The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: flink-connector-elasticsearch-base/pom.xml | 4 +- flink-connector-elasticsearch6/pom.xml | 4 +- flink-connector-elasticsearch7/pom.xml | 4 +- flink-sql-connector-elasticsearch6/pom.xml | 8 ++- flink-sql-connector-elasticsearch7/pom.xml | 8 ++- pom.xml| 108 + 6 files changed, 36 insertions(+), 100 deletions(-)
[flink-connector-elasticsearch] 02/06: [FLINK-29265] Update version to 3.0-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 5e7dae453fdd4687c14677be4632f54927240b17 Author: Chesnay Schepler AuthorDate: Mon Sep 12 16:40:36 2022 +0200 [FLINK-29265] Update version to 3.0-SNAPSHOT --- flink-connector-elasticsearch-base/pom.xml | 2 +- flink-connector-elasticsearch6/pom.xml | 2 +- flink-connector-elasticsearch7/pom.xml | 2 +- flink-sql-connector-elasticsearch6/pom.xml | 2 +- flink-sql-connector-elasticsearch7/pom.xml | 2 +- pom.xml| 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/flink-connector-elasticsearch-base/pom.xml b/flink-connector-elasticsearch-base/pom.xml index 6a749c8..2854a54 100644 --- a/flink-connector-elasticsearch-base/pom.xml +++ b/flink-connector-elasticsearch-base/pom.xml @@ -26,7 +26,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT flink-connector-elasticsearch-base diff --git a/flink-connector-elasticsearch6/pom.xml b/flink-connector-elasticsearch6/pom.xml index 8f4be8e..b5335ca 100644 --- a/flink-connector-elasticsearch6/pom.xml +++ b/flink-connector-elasticsearch6/pom.xml @@ -26,7 +26,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT flink-connector-elasticsearch6 diff --git a/flink-connector-elasticsearch7/pom.xml b/flink-connector-elasticsearch7/pom.xml index 3b7a413..3511bf3 100644 --- a/flink-connector-elasticsearch7/pom.xml +++ b/flink-connector-elasticsearch7/pom.xml @@ -26,7 +26,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT flink-connector-elasticsearch7 diff --git a/flink-sql-connector-elasticsearch6/pom.xml b/flink-sql-connector-elasticsearch6/pom.xml index e70515c..767afa4 100644 --- a/flink-sql-connector-elasticsearch6/pom.xml +++ b/flink-sql-connector-elasticsearch6/pom.xml @@ -26,7 +26,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT flink-sql-connector-elasticsearch6 diff --git a/flink-sql-connector-elasticsearch7/pom.xml b/flink-sql-connector-elasticsearch7/pom.xml index 4b7bdeb..8f62364 100644 --- a/flink-sql-connector-elasticsearch7/pom.xml +++ b/flink-sql-connector-elasticsearch7/pom.xml @@ -26,7 +26,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT flink-sql-connector-elasticsearch7 diff --git a/pom.xml b/pom.xml index b776204..83e86d5 100644 --- a/pom.xml +++ b/pom.xml @@ -29,7 +29,7 @@ under the License. org.apache.flink flink-connector-elasticsearch-parent - 1.16-SNAPSHOT + 3.0-SNAPSHOT Flink : Connectors : Elasticsearch : Parent pom https://flink.apache.org
[flink-connector-elasticsearch] 06/06: [FLINK-25868][build] Enable japicmp for all modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit c0e388b1009544243614a7efe22110907277884a Author: Chesnay Schepler AuthorDate: Tue Sep 13 11:06:54 2022 +0200 [FLINK-25868][build] Enable japicmp for all modules --- pom.xml | 6 ++ 1 file changed, 6 insertions(+) diff --git a/pom.xml b/pom.xml index 5030069..49acc98 100644 --- a/pom.xml +++ b/pom.xml @@ -757,6 +757,12 @@ under the License. + + + io.github.zentol.japicmp + japicmp-maven-plugin + + org.apache.rat apache-rat-plugin
[flink-connector-elasticsearch] 04/06: [FLINK-25868][build] Disable japicmp in packaging/e2e modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 366950e7df3b2c834fe816a67505ceefa5841d4e Author: Chesnay Schepler AuthorDate: Tue Sep 13 11:04:04 2022 +0200 [FLINK-25868][build] Disable japicmp in packaging/e2e modules Optimize build times and reduce noise (warnings would be printed as these modules are not being deployed, so the previous versions can not be found). --- flink-sql-connector-elasticsearch6/pom.xml | 4 flink-sql-connector-elasticsearch7/pom.xml | 4 2 files changed, 8 insertions(+) diff --git a/flink-sql-connector-elasticsearch6/pom.xml b/flink-sql-connector-elasticsearch6/pom.xml index 767afa4..14f35d6 100644 --- a/flink-sql-connector-elasticsearch6/pom.xml +++ b/flink-sql-connector-elasticsearch6/pom.xml @@ -34,6 +34,10 @@ under the License. jar + + true + + org.apache.flink diff --git a/flink-sql-connector-elasticsearch7/pom.xml b/flink-sql-connector-elasticsearch7/pom.xml index 8f62364..43a42dc 100644 --- a/flink-sql-connector-elasticsearch7/pom.xml +++ b/flink-sql-connector-elasticsearch7/pom.xml @@ -34,6 +34,10 @@ under the License. jar + + true + + org.apache.flink
[flink-connector-elasticsearch] 09/13: [hotfix][build] Introduce property for common surefire arg line
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit f11b92a9df0cbedfc9be3c2f9ab82406f50cb29b
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 16:54:53 2022 +0200
[hotfix][build] Introduce property for common surefire arg line
---
pom.xml | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/pom.xml b/pom.xml
index 859a2d6..fb7becb 100644
--- a/pom.xml
+++ b/pom.xml
@@ -99,6 +99,7 @@ under the License.
to the jvm (-Xmx) and the available memory on the
machine running the test -->
2
4
+ -XX:+UseG1GC
-Xms256m
@@ -953,7 +954,7 @@ under the License.
${test.unit.pattern}
${flink.forkCountUnitTest}
- -Xms256m
-Xmx${flink.XmxUnitTest} -XX:+UseG1GC
+
${flink.surefire.baseArgLine} -Xmx${flink.XmxUnitTest}
@@ -974,7 +975,7 @@ under the License.
**/*$*
${flink.forkCountITCase}
- -Xms256m
-Xmx${flink.XmxITCase} -XX:+UseG1GC
+
${flink.surefire.baseArgLine} -Xmx${flink.XmxITCase}
false
[flink-connector-elasticsearch] 05/13: [FLINK-28807] Honor schema lifecycle
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit c9ec08a7bafa343c4a74e2579d13b0a00b6317b5
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 15:50:07 2022 +0200
[FLINK-28807] Honor schema lifecycle
---
.../table/RowElasticsearchEmitter.java | 22 ++
.../elasticsearch/ElasticsearchSinkBase.java | 2 +-
.../elasticsearch/ElasticsearchSinkFunction.java | 10 +-
.../table/RowElasticsearchSinkFunction.java| 5 -
.../elasticsearch/ElasticsearchSinkBaseTest.java | 2 ++
5 files changed, 38 insertions(+), 3 deletions(-)
diff --git
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java
index e90e0bc..bddc6cb 100644
---
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java
+++
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/RowElasticsearchEmitter.java
@@ -22,8 +22,13 @@ import
org.apache.flink.api.common.serialization.SerializationSchema;
import org.apache.flink.api.connector.sink2.SinkWriter;
import org.apache.flink.connector.elasticsearch.sink.ElasticsearchEmitter;
import org.apache.flink.connector.elasticsearch.sink.RequestIndexer;
+import org.apache.flink.metrics.MetricGroup;
+import org.apache.flink.metrics.groups.UnregisteredMetricsGroup;
import org.apache.flink.table.api.TableException;
import org.apache.flink.table.data.RowData;
+import org.apache.flink.util.FlinkRuntimeException;
+import org.apache.flink.util.SimpleUserCodeClassLoader;
+import org.apache.flink.util.UserCodeClassLoader;
import org.elasticsearch.action.ActionRequest;
import org.elasticsearch.action.delete.DeleteRequest;
@@ -61,6 +66,23 @@ class RowElasticsearchEmitter implements
ElasticsearchEmitter {
@Override
public void open() {
+try {
+serializationSchema.open(
+new SerializationSchema.InitializationContext() {
+@Override
+public MetricGroup getMetricGroup() {
+return new UnregisteredMetricsGroup();
+}
+
+@Override
+public UserCodeClassLoader getUserCodeClassLoader() {
+return SimpleUserCodeClassLoader.create(
+
RowElasticsearchEmitter.class.getClassLoader());
+}
+});
+} catch (Exception e) {
+throw new FlinkRuntimeException("Failed to initialize
serialization schema.", e);
+}
indexGenerator.open();
}
diff --git
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java
index 9ed8309..c0fad6f 100644
---
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java
+++
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBase.java
@@ -320,7 +320,7 @@ public abstract class ElasticsearchSinkBase extends
callBridge.createBulkProcessorIndexer(
bulkProcessor, flushOnCheckpoint, numPendingRequests);
failureRequestIndexer = new BufferingNoOpRequestIndexer();
-elasticsearchSinkFunction.open();
+elasticsearchSinkFunction.open(getRuntimeContext());
}
@Override
diff --git
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkFunction.java
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkFunction.java
index 0cf8523..682d2f4 100644
---
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkFunction.java
+++
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkFunction.java
@@ -64,10 +64,18 @@ public interface ElasticsearchSinkFunction extends
Serializable, Function {
/**
* Initialization method for the function. It is called once before the
actual working process
- * methods.
+ * methods, if {@link #open(RuntimeContext)} is not overridden.
*/
default void open() throws Exception {}
+/**
+ * Initialization method for th
[flink-connector-elasticsearch] 08/13: [FLINK-27209][build] Half the memory and double forkCount for running unit tests
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit dd50bf075b16954b843c9f3f611ea7c7ff535c66
Author: Piotr Nowojski
AuthorDate: Mon Sep 12 16:53:03 2022 +0200
[FLINK-27209][build] Half the memory and double forkCount for running unit
tests
---
pom.xml | 11 +++
1 file changed, 11 insertions(+)
diff --git a/pom.xml b/pom.xml
index 4a4a19c..859a2d6 100644
--- a/pom.xml
+++ b/pom.xml
@@ -92,6 +92,13 @@ under the License.
validate
**/*Test.*
+
+ 2048m
+ 1024m
+
+ 2
+ 4
@@ -945,6 +952,8 @@ under the License.
${test.unit.pattern}
+
${flink.forkCountUnitTest}
+ -Xms256m
-Xmx${flink.XmxUnitTest} -XX:+UseG1GC
@@ -964,6 +973,8 @@ under the License.
e.g.,
'org.apache.flink.api.scala.typeutils.Foo$Bar$Foobar'. -->
**/*$*
+
${flink.forkCountITCase}
+ -Xms256m
-Xmx${flink.XmxITCase} -XX:+UseG1GC
false
[flink-connector-elasticsearch] 06/13: [FLINK-26810] Use local timezone for TIMESTAMP_WITH_LOCAL_TIMEZONE fields in dynamic index
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 6e61d797872f1e49d1840c3339464bb55c3de2f6
Author: Alexander Preuß <11444089+alp...@users.noreply.github.com>
AuthorDate: Mon Sep 12 16:09:44 2022 +0200
[FLINK-26810] Use local timezone for TIMESTAMP_WITH_LOCAL_TIMEZONE fields
in dynamic index
---
.../elasticsearch/table/IndexGeneratorFactory.java | 10 +++--
.../elasticsearch/table/IndexGeneratorFactory.java | 10 +++--
.../elasticsearch/table/IndexGeneratorTest.java| 43 ++
.../table/IndexGeneratorFactoryTest.java | 31 ++--
4 files changed, 82 insertions(+), 12 deletions(-)
diff --git
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/IndexGeneratorFactory.java
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/IndexGeneratorFactory.java
index ec2a006..92886f4 100644
---
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/IndexGeneratorFactory.java
+++
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/connector/elasticsearch/table/IndexGeneratorFactory.java
@@ -33,7 +33,6 @@ import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.ZoneId;
-import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
@@ -133,7 +132,8 @@ final class IndexGeneratorFactory {
final String dateTimeFormat =
indexHelper.extractDateFormat(index,
indexFieldLogicalTypeRoot);
DynamicFormatter formatFunction =
-createFormatFunction(indexFieldType,
indexFieldLogicalTypeRoot);
+createFormatFunction(
+indexFieldType, indexFieldLogicalTypeRoot,
localTimeZoneId);
return new AbstractTimeIndexGenerator(index, dateTimeFormat) {
@Override
@@ -163,7 +163,9 @@ final class IndexGeneratorFactory {
}
private static DynamicFormatter createFormatFunction(
-LogicalType indexFieldType, LogicalTypeRoot
indexFieldLogicalTypeRoot) {
+LogicalType indexFieldType,
+LogicalTypeRoot indexFieldLogicalTypeRoot,
+ZoneId localTimeZoneId) {
switch (indexFieldLogicalTypeRoot) {
case DATE:
return (value, dateTimeFormatter) -> {
@@ -186,7 +188,7 @@ final class IndexGeneratorFactory {
case TIMESTAMP_WITH_LOCAL_TIME_ZONE:
return (value, dateTimeFormatter) -> {
TimestampData indexField = (TimestampData) value;
-return
indexField.toInstant().atZone(ZoneOffset.UTC).format(dateTimeFormatter);
+return
indexField.toInstant().atZone(localTimeZoneId).format(dateTimeFormatter);
};
default:
throw new TableException(
diff --git
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/IndexGeneratorFactory.java
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/IndexGeneratorFactory.java
index 48f0107..8347a47 100644
---
a/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/IndexGeneratorFactory.java
+++
b/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/IndexGeneratorFactory.java
@@ -34,7 +34,6 @@ import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.LocalTime;
import java.time.ZoneId;
-import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
@@ -130,7 +129,8 @@ final class IndexGeneratorFactory {
final String dateTimeFormat =
indexHelper.extractDateFormat(index,
indexFieldLogicalTypeRoot);
DynamicFormatter formatFunction =
-createFormatFunction(indexFieldType,
indexFieldLogicalTypeRoot);
+createFormatFunction(
+indexFieldType, indexFieldLogicalTypeRoot,
localTimeZoneId);
return new AbstractTimeIndexGenerator(index, dateTimeFormat) {
@Override
@@ -160,7 +160,9 @@ final class IndexGeneratorFactory {
}
private static DynamicFormatter createFormatFunction(
-LogicalType indexFieldType, LogicalTypeRoot
indexFieldLogicalTypeRoot) {
+LogicalType indexFieldType,
+LogicalTypeRoot indexFieldLogicalTypeRoot,
+ZoneId localTimeZoneId) {
switch (indexFieldLogicalTypeRoot) {
[flink-connector-elasticsearch] 03/13: [FLINK-28177] Fix unstable test
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 13c029f82c96b6600d29596d67d6bbe7b2061d88
Author: kurt
AuthorDate: Mon Sep 12 15:45:00 2022 +0200
[FLINK-28177] Fix unstable test
---
.../elasticsearch/table/Elasticsearch6DynamicSinkITCase.java | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git
a/flink-connector-elasticsearch6/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch6DynamicSinkITCase.java
b/flink-connector-elasticsearch6/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch6DynamicSinkITCase.java
index fd31844..8c31eb2 100644
---
a/flink-connector-elasticsearch6/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch6DynamicSinkITCase.java
+++
b/flink-connector-elasticsearch6/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/table/Elasticsearch6DynamicSinkITCase.java
@@ -230,8 +230,11 @@ public class Elasticsearch6DynamicSinkITCase extends
TestLogger {
@Test
public void testWritingDocumentsNoPrimaryKey() throws Exception {
-TableEnvironment tableEnvironment =
-TableEnvironment.create(EnvironmentSettings.inStreamingMode());
+EnvironmentSettings settings = EnvironmentSettings.inStreamingMode();
+settings.getConfiguration().setString("restart-strategy",
"fixed-delay");
+
settings.getConfiguration().setInteger("restart-strategy.fixed-delay.attempts",
3);
+// default fixed delay is 1 seconds
+TableEnvironment tableEnvironment = TableEnvironment.create(settings);
String index = "no-primary-key";
String myType = "MyType";
[flink-connector-elasticsearch] 11/13: [FLINK-26553][build] Upgrade spotless to 2.13.0
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit 352167b0afe780268d2725b1fbc0f53d8ee50c6b Author: slinkydeveloper AuthorDate: Mon Sep 12 16:59:38 2022 +0200 [FLINK-26553][build] Upgrade spotless to 2.13.0 --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index bf54438..644e796 100644 --- a/pom.xml +++ b/pom.xml @@ -80,7 +80,7 @@ under the License. 0.22.0 1.16.2 2.21.0 - 2.4.2 + 2.13.0 false 1.14.0
[flink-connector-elasticsearch] 04/13: [hotfix][tests] Remove unnecessary mock
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit 132f0e347a384b771347e320cd3587b6d42605b7
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 15:47:59 2022 +0200
[hotfix][tests] Remove unnecessary mock
---
.../streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java
index 33675d1..30417a5 100644
---
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java
@@ -425,7 +425,7 @@ public class ElasticsearchSinkBaseTest {
new DummyElasticsearchSink<>(
new HashMap<>(), sinkFunction, new
DummyRetryFailureHandler());
-sink.open(mock(Configuration.class));
+sink.open(new Configuration());
sink.close();
assertThat(sinkFunction.openCalled).isTrue();
[flink-connector-elasticsearch] 12/13: [FLINK-27735][build][tests] Update testcontainers dependency to v1.17.2
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit cc4903997e350abc2610368e5413ac1b2f668234 Author: Sergey Nuyanzin AuthorDate: Mon Sep 12 17:00:36 2022 +0200 [FLINK-27735][build][tests] Update testcontainers dependency to v1.17.2 --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 644e796..75f9670 100644 --- a/pom.xml +++ b/pom.xml @@ -78,7 +78,7 @@ under the License. 5.8.1 3.21.0 0.22.0 - 1.16.2 + 1.17.2 2.21.0 2.13.0
[flink-connector-elasticsearch] 13/13: [FLINK-26961][build] Update Jackson to 2.13.2
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit f3ce7e1afda22c430fc31ba7a9d3105ee09d7cfb Author: Sergey Nuyanzin AuthorDate: Mon Sep 12 21:20:03 2022 +0200 [FLINK-26961][build] Update Jackson to 2.13.2 --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 75f9670..cf49ee0 100644 --- a/pom.xml +++ b/pom.xml @@ -348,7 +348,7 @@ under the License. jackson-bom pom import - 2.13.0 + 2.13.2.20220328
[flink-connector-elasticsearch] 02/13: [FLINK-27185] Convert connector-elasticsearch modules to assertj
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit d226fc3e8f36870e82fe860a8e3e46f9d82e7b8c
Author: Alexander Preuß <11444089+alp...@users.noreply.github.com>
AuthorDate: Mon Sep 12 15:43:52 2022 +0200
[FLINK-27185] Convert connector-elasticsearch modules to assertj
---
.../sink/ElasticsearchSinkBaseITCase.java | 2 +-
.../sink/ElasticsearchSinkBuilderBaseTest.java | 10 +-
.../sink/ElasticsearchWriterITCase.java| 4 +-
.../elasticsearch/sink/TestClientBase.java | 4 +-
.../ElasticsearchDynamicSinkFactoryBaseTest.java | 278 ++---
.../elasticsearch/ElasticsearchSinkBaseTest.java | 108 ++--
.../elasticsearch/ElasticsearchSinkTestBase.java | 51 ++--
.../table/IndexGeneratorFactoryTest.java | 95 ---
.../testutils/SourceSinkDataTestKit.java | 5 +-
.../Elasticsearch6DynamicSinkFactoryTest.java | 22 +-
.../Elasticsearch7DynamicSinkFactoryTest.java | 20 +-
11 files changed, 256 insertions(+), 343 deletions(-)
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBaseITCase.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBaseITCase.java
index 65c2af3..bd574d8 100644
---
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBaseITCase.java
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBaseITCase.java
@@ -123,7 +123,7 @@ abstract class ElasticsearchSinkBaseITCase {
runTest(index, false, TestEmitter::jsonEmitter, deliveryGuarantee,
null);
} catch (IllegalStateException e) {
failure = true;
-
assertThat(deliveryGuarantee).isEqualTo(DeliveryGuarantee.EXACTLY_ONCE);
+
assertThat(deliveryGuarantee).isSameAs(DeliveryGuarantee.EXACTLY_ONCE);
} finally {
assertThat(failure).isEqualTo(deliveryGuarantee ==
DeliveryGuarantee.EXACTLY_ONCE);
}
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBuilderBaseTest.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBuilderBaseTest.java
index 8e7d096..3fc2abc 100644
---
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBuilderBaseTest.java
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchSinkBuilderBaseTest.java
@@ -30,7 +30,7 @@ import org.junit.jupiter.api.extension.ExtendWith;
import java.util.stream.Stream;
import static org.assertj.core.api.Assertions.assertThat;
-import static org.assertj.core.api.Assertions.assertThatNoException;
+import static org.assertj.core.api.Assertions.assertThatCode;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
/** Tests for {@link ElasticsearchSinkBuilderBase}. */
@@ -54,7 +54,7 @@ abstract class ElasticsearchSinkBuilderBaseTest assertThatNoException().isThrownBy(builder::build));
+builder ->
assertThatCode(builder::build).doesNotThrowAnyException());
}
@Test
@@ -91,11 +91,11 @@ abstract class ElasticsearchSinkBuilderBaseTest
createEmptyBuilder().setConnectionRequestTimeout(-1))
+assertThatThrownBy(() ->
createEmptyBuilder().setConnectionRequestTimeout(-1).build())
.isInstanceOf(IllegalStateException.class);
-assertThatThrownBy(() -> createEmptyBuilder().setConnectionTimeout(-1))
+assertThatThrownBy(() ->
createEmptyBuilder().setConnectionTimeout(-1).build())
.isInstanceOf(IllegalStateException.class);
-assertThatThrownBy(() -> createEmptyBuilder().setSocketTimeout(-1))
+assertThatThrownBy(() ->
createEmptyBuilder().setSocketTimeout(-1).build())
.isInstanceOf(IllegalStateException.class);
}
diff --git
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
index 0f88d57..c4e5255 100644
---
a/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
+++
b/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java
@@ -238,7 +238,7 @@ class ElasticsearchWriterITCase {
writer.blockingFlushAllActions();
-assertTh
[flink-connector-elasticsearch] 01/13: [FLINK-29269][ci] Setup logging
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit ffc0721badd761c88279bdd28b1e56c1360ead0c
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 21:29:57 2022 +0200
[FLINK-29269][ci] Setup logging
---
.github/workflows/ci.yml | 2 +-
tools/ci/log4j.properties | 43 +++
2 files changed, 44 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 780b792..3700352 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -45,4 +45,4 @@ jobs:
maven-version: 3.8.5
- name: Compile and test flink-connector-elasticsearch
-run: mvn clean install -Dscala-2.12 -Dflink.convergence.phase=install
-Pcheck-convergence -U -B ${{ env.MVN_CONNECTION_OPTIONS }}
+run: mvn clean install -Dscala-2.12 -Dflink.convergence.phase=install
-Pcheck-convergence -U -B ${{ env.MVN_CONNECTION_OPTIONS }}
-Dlog4j.configurationFile=file://$(pwd)/tools/ci/log4j.properties
diff --git a/tools/ci/log4j.properties b/tools/ci/log4j.properties
new file mode 100644
index 000..86b02cc
--- /dev/null
+++ b/tools/ci/log4j.properties
@@ -0,0 +1,43 @@
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+rootLogger.level = INFO
+rootLogger.appenderRef.out.ref = FileAppender
+
+# -
+# Console (use 'console')
+# -
+
+appender.console.name = ConsoleAppender
+appender.console.type = CONSOLE
+appender.console.layout.type = PatternLayout
+appender.console.layout.pattern = %d{HH:mm:ss,SSS} [%20t] %-5p %-60c %x - %m%n
+
+# -
+# File (use 'file')
+# -
+appender.file.name = FileAppender
+appender.file.type = FILE
+appender.file.fileName = ${sys:log.dir}/mvn-${sys:mvn.forkNumber:-output}.log
+appender.file.layout.type = PatternLayout
+appender.file.layout.pattern = %d{HH:mm:ss,SSS} [%20t] %-5p %-60c %x - %m%n
+appender.file.createOnDemand = true
+
+# suppress the irrelevant (wrong) warnings from the netty channel handler
+logger.netty.name = org.jboss.netty.channel.DefaultChannelPipeline
+logger.netty.level = ERROR
[flink-connector-elasticsearch] 10/13: [FLINK-27026][build] Upgrade checkstyle plugin
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git commit b046b9599258f8c1965bace348937f1a02d85d72 Author: Chesnay Schepler AuthorDate: Mon Sep 12 16:56:29 2022 +0200 [FLINK-27026][build] Upgrade checkstyle plugin --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index fb7becb..bf54438 100644 --- a/pom.xml +++ b/pom.xml @@ -1194,7 +1194,7 @@ under the License. org.apache.maven.plugins maven-checkstyle-plugin - 2.17 + 3.1.2 com.puppycrawl.tools
[flink-connector-elasticsearch] 07/13: [FLINK-27024][build] Cleanup surefire configuration
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git
commit aa98d57044622aebf58300a2d2a51fe3637ffe4e
Author: Chesnay Schepler
AuthorDate: Mon Sep 12 16:50:48 2022 +0200
[FLINK-27024][build] Cleanup surefire configuration
---
pom.xml | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/pom.xml b/pom.xml
index 9cda7e3..4a4a19c 100644
--- a/pom.xml
+++ b/pom.xml
@@ -913,8 +913,13 @@ under the License.
false
0${surefire.forkNumber}
+
+
$${surefire.forkNumber}
true
true
+ US
+
en
[flink-connector-elasticsearch] branch main updated (9bc0c89 -> f3ce7e1)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-elasticsearch.git from 9bc0c89 [FLINK-28904][python][docs] Add fat jar link for ES (#27) new ffc0721 [FLINK-29269][ci] Setup logging new d226fc3 [FLINK-27185] Convert connector-elasticsearch modules to assertj new 13c029f [FLINK-28177] Fix unstable test new 132f0e3 [hotfix][tests] Remove unnecessary mock new c9ec08a [FLINK-28807] Honor schema lifecycle new 6e61d79 [FLINK-26810] Use local timezone for TIMESTAMP_WITH_LOCAL_TIMEZONE fields in dynamic index new aa98d57 [FLINK-27024][build] Cleanup surefire configuration new dd50bf0 [FLINK-27209][build] Half the memory and double forkCount for running unit tests new f11b92a [hotfix][build] Introduce property for common surefire arg line new b046b95 [FLINK-27026][build] Upgrade checkstyle plugin new 352167b [FLINK-26553][build] Upgrade spotless to 2.13.0 new cc49039 [FLINK-27735][build][tests] Update testcontainers dependency to v1.17.2 new f3ce7e1 [FLINK-26961][build] Update Jackson to 2.13.2 The 13 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/ci.yml | 2 +- .../elasticsearch/table/IndexGeneratorFactory.java | 10 +- .../table/RowElasticsearchEmitter.java | 22 ++ .../elasticsearch/ElasticsearchSinkBase.java | 2 +- .../elasticsearch/ElasticsearchSinkFunction.java | 10 +- .../elasticsearch/table/IndexGeneratorFactory.java | 10 +- .../table/RowElasticsearchSinkFunction.java| 5 +- .../sink/ElasticsearchSinkBaseITCase.java | 2 +- .../sink/ElasticsearchSinkBuilderBaseTest.java | 10 +- .../sink/ElasticsearchWriterITCase.java| 4 +- .../elasticsearch/sink/TestClientBase.java | 4 +- .../ElasticsearchDynamicSinkFactoryBaseTest.java | 278 ++--- .../elasticsearch/table/IndexGeneratorTest.java| 43 .../elasticsearch/ElasticsearchSinkBaseTest.java | 112 +++-- .../elasticsearch/ElasticsearchSinkTestBase.java | 51 ++-- .../table/IndexGeneratorFactoryTest.java | 126 +- .../testutils/SourceSinkDataTestKit.java | 5 +- .../Elasticsearch6DynamicSinkFactoryTest.java | 22 +- .../table/Elasticsearch6DynamicSinkITCase.java | 7 +- .../Elasticsearch7DynamicSinkFactoryTest.java | 20 +- pom.xml| 27 +- tools/ci/log4j.properties | 43 22 files changed, 448 insertions(+), 367 deletions(-) create mode 100644 tools/ci/log4j.properties
[flink] branch master updated: [hotfix][release] Mark 1.15 as latest stable
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 5ddf4f2568e [hotfix][release] Mark 1.15 as latest stable
5ddf4f2568e is described below
commit 5ddf4f2568e49cea41e501b8d845f37dd51c1252
Author: Chesnay Schepler
AuthorDate: Tue Sep 13 10:45:23 2022 +0200
[hotfix][release] Mark 1.15 as latest stable
---
.github/workflows/docs.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
index 0cf9857bb69..f9b35772aee 100644
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -43,7 +43,7 @@ jobs:
if [ "${currentBranch}" = "master" ]; then
echo "flink_alias=release-1.17" >> ${GITHUB_ENV}
- elif [ "${currentBranch}" = "release-1.16" ]; then
+ elif [ "${currentBranch}" = "release-1.15" ]; then
echo "flink_alias=stable" >> ${GITHUB_ENV}
fi
- name: Build documentation
[flink] branch master updated (813e0d11123 -> ccdf270fc91)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 813e0d11123 [hotfix][javadoc] Move javadoc before @Test add 35c6ac6891a [FLINK-25868][build] Limit japicmp to jar modules add ddd034b1528 [FLINK-25868][build] Disable japicmp in packaging/e2e modules add 8fb3da763d9 [FLINK-25868][build] Use custom japicmp plugin version add ccdf270fc91 [FLINK-25868][build] Enable japicmp for all modules No new revisions were added by this update. Summary of changes: flink-connectors/flink-hadoop-compatibility/pom.xml | 5 - .../flink-sql-connector-aws-kinesis-firehose/pom.xml| 4 .../flink-sql-connector-aws-kinesis-streams/pom.xml | 4 .../flink-sql-connector-elasticsearch6/pom.xml | 4 .../flink-sql-connector-elasticsearch7/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-1.4/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-2.2/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.3.9/pom.xml | 4 flink-connectors/flink-sql-connector-hive-3.1.2/pom.xml | 4 flink-connectors/flink-sql-connector-kafka/pom.xml | 4 flink-connectors/flink-sql-connector-kinesis/pom.xml| 4 flink-connectors/flink-sql-connector-pulsar/pom.xml | 4 flink-connectors/flink-sql-connector-rabbitmq/pom.xml | 4 flink-core/pom.xml | 6 -- flink-dist-scala/pom.xml| 4 flink-dist/pom.xml | 1 + flink-docs/pom.xml | 1 + flink-end-to-end-tests/pom.xml | 1 + flink-examples/pom.xml | 4 flink-filesystems/flink-fs-hadoop-shaded/pom.xml| 4 flink-filesystems/flink-s3-fs-base/pom.xml | 1 + flink-formats/flink-sql-avro-confluent-registry/pom.xml | 4 flink-formats/flink-sql-avro/pom.xml| 4 flink-formats/flink-sql-csv/pom.xml | 4 flink-formats/flink-sql-json/pom.xml| 4 flink-formats/flink-sql-orc/pom.xml | 4 flink-formats/flink-sql-parquet/pom.xml | 4 flink-formats/flink-sql-protobuf/pom.xml| 4 flink-java/pom.xml | 6 -- flink-metrics/flink-metrics-core/pom.xml| 5 - flink-scala/pom.xml | 6 -- flink-streaming-java/pom.xml| 7 --- flink-streaming-scala/pom.xml | 11 +-- flink-table/flink-table-planner-loader-bundle/pom.xml | 4 flink-yarn-tests/pom.xml| 4 pom.xml | 17 + tools/ci/java-ci-tools/pom.xml | 4 37 files changed, 118 insertions(+), 49 deletions(-)
[flink] 02/04: [FLINK-25868][build] Disable japicmp in packaging/e2e modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git commit c8a6233c8138f9747e06663491b0c662f38e826f Author: Chesnay Schepler AuthorDate: Tue Feb 1 15:09:19 2022 +0100 [FLINK-25868][build] Disable japicmp in packaging/e2e modules Optimize build times and reduce noise (warnings would be printed as these modules are not being deployed, so the previous versions can not be found). --- flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml | 4 flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml | 4 flink-connectors/flink-sql-connector-elasticsearch6/pom.xml | 4 flink-connectors/flink-sql-connector-elasticsearch7/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-1.4/pom.xml| 4 flink-connectors/flink-sql-connector-hbase-2.2/pom.xml| 4 flink-connectors/flink-sql-connector-hive-1.2.2/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.2.0/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.3.6/pom.xml | 4 flink-connectors/flink-sql-connector-hive-3.1.2/pom.xml | 4 flink-connectors/flink-sql-connector-kafka/pom.xml| 4 flink-connectors/flink-sql-connector-kinesis/pom.xml | 4 flink-connectors/flink-sql-connector-pulsar/pom.xml | 4 flink-connectors/flink-sql-connector-rabbitmq/pom.xml | 4 flink-dist-scala/pom.xml | 4 flink-dist/pom.xml| 1 + flink-docs/pom.xml| 1 + flink-end-to-end-tests/pom.xml| 1 + flink-examples/pom.xml| 4 flink-filesystems/flink-fs-hadoop-shaded/pom.xml | 4 flink-filesystems/flink-s3-fs-base/pom.xml| 1 + flink-formats/flink-sql-avro-confluent-registry/pom.xml | 4 flink-formats/flink-sql-avro/pom.xml | 4 flink-formats/flink-sql-orc/pom.xml | 4 flink-formats/flink-sql-parquet/pom.xml | 4 flink-table/flink-table-planner-loader-bundle/pom.xml | 4 flink-yarn-tests/pom.xml | 4 tools/ci/java-ci-tools/pom.xml| 4 28 files changed, 100 insertions(+) diff --git a/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml b/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml index 8b4a2df0d4d..8d85dd4c4fd 100644 --- a/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml +++ b/flink-connectors/flink-sql-connector-aws-kinesis-firehose/pom.xml @@ -31,6 +31,10 @@ flink-sql-connector-aws-kinesis-firehose Flink : Connectors : SQL : Amazon Kinesis Data Firehose + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml b/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml index 0804a3a48f0..3a7c8f7ebc9 100644 --- a/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml +++ b/flink-connectors/flink-sql-connector-aws-kinesis-streams/pom.xml @@ -31,6 +31,10 @@ flink-sql-connector-aws-kinesis-streams Flink : Connectors : SQL : Amazon Kinesis Data Streams + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml b/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml index 6aaaf4acba7..5205e444618 100644 --- a/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml +++ b/flink-connectors/flink-sql-connector-elasticsearch6/pom.xml @@ -35,6 +35,10 @@ under the License. jar + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml b/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml index 489e9ea3855..830ed10da92 100644 --- a/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml +++ b/flink-connectors/flink-sql-connector-elasticsearch7/pom.xml @@ -35,6 +35,10 @@ under the License. jar + + true + + org.apache.flink diff --git a/flink-connectors/flink-sql-connector-hbase-1.4/pom.xml b/flink-connectors/flink-sql-connector-hbase-1.4/pom.xml index 78c1e5f32bb..75ccc436b76 100644 --- a/flink-connectors/flink-sql-connector-hbase-1.4/pom.xml +++ b/flink
[flink] branch release-1.15 updated (b552505c513 -> cef796fe237)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git from b552505c513 [FLINK-29223][coordination] Add missing output info for jobs already reached terminal state new 5448f568221 [FLINK-25868][build] Limit japicmp to jar modules new c8a6233c813 [FLINK-25868][build] Disable japicmp in packaging/e2e modules new 7e961c3bac3 [FLINK-25868][build] Use custom japicmp plugin version new cef796fe237 [FLINK-25868][build] Enable japicmp for all modules The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: flink-connectors/flink-hadoop-compatibility/pom.xml | 5 - .../flink-sql-connector-aws-kinesis-firehose/pom.xml| 4 .../flink-sql-connector-aws-kinesis-streams/pom.xml | 4 .../flink-sql-connector-elasticsearch6/pom.xml | 4 .../flink-sql-connector-elasticsearch7/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-1.4/pom.xml | 4 flink-connectors/flink-sql-connector-hbase-2.2/pom.xml | 4 flink-connectors/flink-sql-connector-hive-1.2.2/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.2.0/pom.xml | 4 flink-connectors/flink-sql-connector-hive-2.3.6/pom.xml | 4 flink-connectors/flink-sql-connector-hive-3.1.2/pom.xml | 4 flink-connectors/flink-sql-connector-kafka/pom.xml | 4 flink-connectors/flink-sql-connector-kinesis/pom.xml| 4 flink-connectors/flink-sql-connector-pulsar/pom.xml | 4 flink-connectors/flink-sql-connector-rabbitmq/pom.xml | 4 flink-core/pom.xml | 6 -- flink-dist-scala/pom.xml| 4 flink-dist/pom.xml | 1 + flink-docs/pom.xml | 1 + flink-end-to-end-tests/pom.xml | 1 + flink-examples/pom.xml | 4 flink-filesystems/flink-fs-hadoop-shaded/pom.xml| 4 flink-filesystems/flink-s3-fs-base/pom.xml | 1 + flink-formats/flink-sql-avro-confluent-registry/pom.xml | 4 flink-formats/flink-sql-avro/pom.xml| 4 flink-formats/flink-sql-orc/pom.xml | 4 flink-formats/flink-sql-parquet/pom.xml | 4 flink-java/pom.xml | 6 -- flink-metrics/flink-metrics-core/pom.xml| 5 - flink-scala/pom.xml | 6 -- flink-streaming-java/pom.xml| 3 +-- flink-streaming-scala/pom.xml | 11 +-- flink-table/flink-table-planner-loader-bundle/pom.xml | 4 flink-yarn-tests/pom.xml| 4 pom.xml | 17 + tools/ci/java-ci-tools/pom.xml | 4 36 files changed, 115 insertions(+), 44 deletions(-)
[flink] 03/04: [FLINK-25868][build] Use custom japicmp plugin version
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git commit 7e961c3bac344e67149523aca86d8d5ab11a6aee Author: Chesnay Schepler AuthorDate: Wed Aug 31 16:27:17 2022 +0200 [FLINK-25868][build] Use custom japicmp plugin version --- flink-connectors/flink-hadoop-compatibility/pom.xml | 2 +- flink-core/pom.xml | 2 +- flink-java/pom.xml | 2 +- flink-metrics/flink-metrics-core/pom.xml| 2 +- flink-scala/pom.xml | 2 +- flink-streaming-java/pom.xml| 2 +- flink-streaming-scala/pom.xml | 2 +- pom.xml | 8 8 files changed, 11 insertions(+), 11 deletions(-) diff --git a/flink-connectors/flink-hadoop-compatibility/pom.xml b/flink-connectors/flink-hadoop-compatibility/pom.xml index 976d7b20211..1f2eac45d80 100644 --- a/flink-connectors/flink-hadoop-compatibility/pom.xml +++ b/flink-connectors/flink-hadoop-compatibility/pom.xml @@ -118,7 +118,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-core/pom.xml b/flink-core/pom.xml index 8d255adb479..25d9c9585bf 100644 --- a/flink-core/pom.xml +++ b/flink-core/pom.xml @@ -143,7 +143,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-java/pom.xml b/flink-java/pom.xml index ff789cc8593..95b6ee97911 100644 --- a/flink-java/pom.xml +++ b/flink-java/pom.xml @@ -72,7 +72,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-metrics/flink-metrics-core/pom.xml b/flink-metrics/flink-metrics-core/pom.xml index 8895ef36290..eabcf75dc6e 100644 --- a/flink-metrics/flink-metrics-core/pom.xml +++ b/flink-metrics/flink-metrics-core/pom.xml @@ -51,7 +51,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-scala/pom.xml b/flink-scala/pom.xml index 9412855ef6e..534d8848baa 100644 --- a/flink-scala/pom.xml +++ b/flink-scala/pom.xml @@ -158,7 +158,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-streaming-java/pom.xml b/flink-streaming-java/pom.xml index 952ce286d6d..71ec429dc72 100644 --- a/flink-streaming-java/pom.xml +++ b/flink-streaming-java/pom.xml @@ -102,7 +102,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-streaming-scala/pom.xml b/flink-streaming-scala/pom.xml index eec2636a3e4..bfaa943bd5b 100644 --- a/flink-streaming-scala/pom.xml +++ b/flink-streaming-scala/pom.xml @@ -257,7 +257,7 @@ under the License. - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin diff --git a/pom.xml b/pom.xml index d2493f4b316..5e9f97e726d 100644 --- a/pom.xml +++ b/pom.xml @@ -957,7 +957,7 @@ under the License. 3.2.4 - com.github.siom79.japicmp + io.github.zentol.japicmp japicmp-maven-plugin
[flink] 04/04: [FLINK-25868][build] Enable japicmp for all modules
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git commit cef796fe2377e1a47734acc3d99db3deb5712459 Author: Chesnay Schepler AuthorDate: Tue Feb 1 15:10:37 2022 +0100 [FLINK-25868][build] Enable japicmp for all modules --- flink-connectors/flink-hadoop-compatibility/pom.xml | 5 - flink-core/pom.xml | 6 -- flink-java/pom.xml | 6 -- flink-metrics/flink-metrics-core/pom.xml| 5 - flink-scala/pom.xml | 6 -- flink-streaming-java/pom.xml| 1 - flink-streaming-scala/pom.xml | 9 - pom.xml | 6 ++ 8 files changed, 6 insertions(+), 38 deletions(-) diff --git a/flink-connectors/flink-hadoop-compatibility/pom.xml b/flink-connectors/flink-hadoop-compatibility/pom.xml index 1f2eac45d80..b2b6218a522 100644 --- a/flink-connectors/flink-hadoop-compatibility/pom.xml +++ b/flink-connectors/flink-hadoop-compatibility/pom.xml @@ -116,11 +116,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - net.alchim31.maven diff --git a/flink-core/pom.xml b/flink-core/pom.xml index 25d9c9585bf..f3e9d8e157e 100644 --- a/flink-core/pom.xml +++ b/flink-core/pom.xml @@ -141,12 +141,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins diff --git a/flink-java/pom.xml b/flink-java/pom.xml index 95b6ee97911..681d0b2dc3d 100644 --- a/flink-java/pom.xml +++ b/flink-java/pom.xml @@ -70,12 +70,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins diff --git a/flink-metrics/flink-metrics-core/pom.xml b/flink-metrics/flink-metrics-core/pom.xml index eabcf75dc6e..94b999713d2 100644 --- a/flink-metrics/flink-metrics-core/pom.xml +++ b/flink-metrics/flink-metrics-core/pom.xml @@ -49,11 +49,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - org.apache.maven.plugins maven-jar-plugin diff --git a/flink-scala/pom.xml b/flink-scala/pom.xml index 534d8848baa..1e01b762199 100644 --- a/flink-scala/pom.xml +++ b/flink-scala/pom.xml @@ -156,12 +156,6 @@ under the License. - - - io.github.zentol.japicmp - japicmp-maven-plugin - - org.apache.maven.plugins maven-surefire-plugin diff --git a/flink-streaming-java/pom.xml b/flink-streaming-java/pom.xml index 71ec429dc72..524c477b364 100644 --- a/flink-streaming-java/pom.xml +++ b/flink-streaming-java/pom.xml @@ -100,7 +100,6 @@ under the License. - io.github.zentol.japicmp japicmp-maven-plugin diff --git a/flink-streaming-scala/pom.xml b/flink-streaming-scala/pom.xml index bfaa943bd5b..77e15ce9ebe 100644 --- a/flink-streaming-scala/pom.xml +++ b/flink-streaming-scala/pom.xml @@ -255,7 +255,6 @@ under the License. - io.github.zentol.japicmp japicmp-maven-plugin @@ -273,14 +272,6 @@ under the License. - - - verify
[flink] 01/04: [FLINK-25868][build] Limit japicmp to jar modules
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch release-1.15
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 5448f5682214b3a3e344b4fd6dca68b948626358
Author: Chesnay Schepler
AuthorDate: Tue Feb 1 15:08:05 2022 +0100
[FLINK-25868][build] Limit japicmp to jar modules
Skips quickstart and pom modules.
---
pom.xml | 3 +++
1 file changed, 3 insertions(+)
diff --git a/pom.xml b/pom.xml
index 3c77559f71f..d2493f4b316 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2157,6 +2157,9 @@ under the License.
true
true
+
+
jar
+
${rootDir}/${japicmp.outputDir}/${project.artifactId}
[flink-connector-opensearch] 01/01: Initialize project
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-opensearch.git commit 55dd8d5f85291a614231aa5e95a63620e5056d5d Author: Chesnay Schepler AuthorDate: Mon Sep 12 20:08:34 2022 +0200 Initialize project --- .asf.yml | 20 1 file changed, 20 insertions(+) diff --git a/.asf.yml b/.asf.yml new file mode 100644 index 000..33536cd --- /dev/null +++ b/.asf.yml @@ -0,0 +1,20 @@ +github: + enabled_merge_buttons: +squash: true +merge: false +rebase: true + labels: +- flink +- opensearch +- connector +- datastream +- table +- sql + collaborators: +- flinkbot +notifications: + commits: commits@flink.apache.org + issues: iss...@flink.apache.org + pullrequests: iss...@flink.apache.org + jobs: bui...@flink.apache.org + jira_options: link label \ No newline at end of file
[flink-connector-opensearch] branch main created (now 55dd8d5)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/flink-connector-opensearch.git at 55dd8d5 Initialize project This branch includes the following new commits: new 55dd8d5 Initialize project The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[flink] branch master updated (36396f789cd -> b8fc7f34d2c)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 36396f789cd [FLINK-29217][tests] Guarantee checkpoint order in OC test add b8fc7f34d2c [FLINK-29248][rpc] Remove RpcService#fenceRpcServer No new revisions were added by this update. Summary of changes: .../flink/runtime/rpc/akka/AkkaRpcService.java | 34 -- .../org/apache/flink/runtime/rpc/RpcService.java | 14 - .../flink/runtime/rpc/TestingRpcService.java | 5 .../OperatorEventSendingCheckpointITCase.java | 6 4 files changed, 59 deletions(-)
[flink-web] 02/02: Rebuild website
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 25f702bdc861524ac1560399822e9cceb08681f7 Author: Chesnay Schepler AuthorDate: Sat Sep 10 00:13:14 2022 +0200 Rebuild website --- content/blog/feed.xml| 15 +++ content/news/2022/09/08/akka-license-change.html | 15 +++ 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/content/blog/feed.xml b/content/blog/feed.xml index ed23f2c91..3decd4831 100644 --- a/content/blog/feed.xml +++ b/content/blog/feed.xml @@ -41,12 +41,19 @@ Historically Akka has been incredibly stable, and combined with our limited use h2 id=what-if-a-new-security-vulnerabilities-is-found-in-akka-26What if a new security vulnerabilities is found in Akka 2.6?/h2 -pThat is the big unknown./p +pdelThat is the big unknown./del/p -pEven though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that a href=https://github.com/akka/akka/pull/31561#issuecomment-12392176022.6 will no longer be supported/a from that point onwards.br / -Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6./p +pdelEven though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that a href=https://github.com/akka/akka/pull/31561#issuecomment-12392176022.6 will no longer be supported/a from that point onwards.br / +Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6./del/p -pWe cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time./p +pdelWe cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time./del/p + +pstrongUpdate - September 9th/strong: Akka 2.6 will continue to receive critical security updates and critical bug fixes under the current Apache 2 license until a href=https://www.lightbend.com/akka/license-faqSeptember of 2023/a./p + +blockquote + pstrongWill critical vulnerabilities and bugs be patched in 2.6.x?/strongbr / +Yes, critical security updates and critical bugs will be patched in Akka v2.6.x under the current Apache 2 license until September of 2023./p +/blockquote h1 id=how-does-flink-use-akkaHow does Flink use Akka?/h1 diff --git a/content/news/2022/09/08/akka-license-change.html b/content/news/2022/09/08/akka-license-change.html index 20f65411e..3bf27c48d 100644 --- a/content/news/2022/09/08/akka-license-change.html +++ b/content/news/2022/09/08/akka-license-change.html @@ -275,12 +275,19 @@ Historically Akka has been incredibly stable, and combined with our limited use What if a new security vulnerabilities is found in Akka 2.6? -That is the big unknown. +That is the big unknown. -Even though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that https://github.com/akka/akka/pull/31561#issuecomment-1239217602;>2.6 will no longer be supported from that point onwards. -Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6. +Even though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that https://github.com/akka/akka/pull/31561#issuecomment-1239217602;>2.6 will no longer be supported from that point onwards. +Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6. -We cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time. +We cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time. + +Update - September 9th: Akka 2.6 will continue to receive critical security updates and critical bug fixes under the current Apache 2 license until https://www.lightbend.com/akka/license-faq;>September of 2023. + + + Will critical vulnerabilities and bugs be patched in 2.6.x? +Yes, critical security updates and critical bugs will be patched in Akka v2.6.x under the current Apache 2 license until September of 2023. + How does Flink use Akka?
[flink-web] 01/02: Update Akka licensing blogpost
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git commit 1201d698cb72276f5a4c56ff8152818e0595ec65 Author: Chesnay Schepler AuthorDate: Sat Sep 10 00:13:07 2022 +0200 Update Akka licensing blogpost --- _posts/2022-09-08-akka-license-change.md | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/_posts/2022-09-08-akka-license-change.md b/_posts/2022-09-08-akka-license-change.md index 1ea6ba76b..7d54cd27c 100644 --- a/_posts/2022-09-08-akka-license-change.md +++ b/_posts/2022-09-08-akka-license-change.md @@ -40,12 +40,17 @@ Should a community fork be created (which at this time seems possible) we will s ## What if a new security vulnerabilities is found in Akka 2.6? -That is the big unknown. +~~That is the big unknown.~~ -Even though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that [2.6 will no longer be supported](https://github.com/akka/akka/pull/31561#issuecomment-1239217602) from that point onwards. -Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6. +~~Even though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that [2.6 will no longer be supported](https://github.com/akka/akka/pull/31561#issuecomment-1239217602) from that point onwards. +Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6.~~ -We cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time. +~~We cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time.~~ + +**Update - September 9th**: Akka 2.6 will continue to receive critical security updates and critical bug fixes under the current Apache 2 license until [September of 2023](https://www.lightbend.com/akka/license-faq). + +> **Will critical vulnerabilities and bugs be patched in 2.6.x?** +> Yes, critical security updates and critical bugs will be patched in Akka v2.6.x under the current Apache 2 license until September of 2023. # How does Flink use Akka?
[flink-web] branch asf-site updated (386514122 -> 25f702bdc)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git from 386514122 Rebuild website new 1201d698c Update Akka licensing blogpost new 25f702bdc Rebuild website The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: _posts/2022-09-08-akka-license-change.md | 13 + content/blog/feed.xml| 15 +++ content/news/2022/09/08/akka-license-change.html | 15 +++ 3 files changed, 31 insertions(+), 12 deletions(-)
[flink] branch release-1.16 updated: [FLINK-29211][hive][legal] Update 2.3.9 NOTICE
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch release-1.16 in repository https://gitbox.apache.org/repos/asf/flink.git The following commit(s) were added to refs/heads/release-1.16 by this push: new d09f3eb1f42 [FLINK-29211][hive][legal] Update 2.3.9 NOTICE d09f3eb1f42 is described below commit d09f3eb1f427f6f30bedf1919682362c47ca2915 Author: Chesnay Schepler AuthorDate: Fri Sep 9 11:39:42 2022 +0200 [FLINK-29211][hive][legal] Update 2.3.9 NOTICE --- .../src/main/resources/META-INF/NOTICE | 20 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/flink-connectors/flink-sql-connector-hive-2.3.9/src/main/resources/META-INF/NOTICE b/flink-connectors/flink-sql-connector-hive-2.3.9/src/main/resources/META-INF/NOTICE index 855692eabf8..f995fcfdd8c 100644 --- a/flink-connectors/flink-sql-connector-hive-2.3.9/src/main/resources/META-INF/NOTICE +++ b/flink-connectors/flink-sql-connector-hive-2.3.9/src/main/resources/META-INF/NOTICE @@ -16,6 +16,9 @@ See bundled license files for details. The bundled Apache Hive org.apache.hive:hive-exec dependency bundles the following dependencies under the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt) +- com.fasterxml.jackson.core:jackson-annotations:2.6.3 +- com.fasterxml.jackson.core:jackson-core:2.6.3 +- com.fasterxml.jackson.core:jackson-databind:2.6.3 - com.google.guava:guava:14.0.1 - com.googlecode.javaewah:JavaEWAH:0.3.2 - com.tdunning:json:1.8 @@ -24,8 +27,14 @@ the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt) - javax.jdo:jdo-api:3.0.1 - joda-time:joda-time:2.8.1 - net.sf.opencsv:opencsv:2.3 -- org.apache.avro:avro-mapred:1.7.7 -- org.apache.avro:avro:1.7.7 +- org.apache.avro:avro-guava-dependencies:1.8.2 +- org.apache.avro:avro-mapred:1.8.2 +- org.apache.avro:avro:1.8.2 +- org.apache.calcite:calcite-core:1.10.0 +- org.apache.calcite:calcite-druid:1.10.0 +- org.apache.calcite:calcite-linq4j:1.10.0 +- org.apache.calcite.avatica:avatica:1.8.0 +- org.apache.commons:commons-lang:2.6 - org.apache.commons:commons-lang3:3.1 - org.apache.hive.shims:hive-shims-0.23:2.3.9 - org.apache.hive.shims:hive-shims-common:2.3.9 @@ -40,9 +49,12 @@ the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt) - org.apache.orc:orc-core:1.3.4 - org.apache.orc:orc-tools:1.3.4 - org.apache.parquet:parquet-hadoop-bundle:1.8.1 -- org.apache.parquet:parquet-jackson:1.8.1 +- org.apache.thrift:libthrift:0.7.0 - org.apache.thrift:libthrift:0.9.3 +- org.apache.thrift:libfb303:0.9.3 +- org.codehaus.jackson:jackson-core-asl:1.9.11 - org.codehaus.jackson:jackson-core-asl:1.9.13 +- org.codehaus.jackson:jackson-mapper-asl:1.9.11 - org.codehaus.jackson:jackson-mapper-asl:1.9.13 - org.objenesis:objenesis:2.1 @@ -59,4 +71,4 @@ See bundled license files for details. The bundled Apache Hive org.apache.hive:hive-exec dependency bundles the following dependencies under the MIT/X11 license. See bundled license files for details. -- org.slf4j:slf4j-api:1.7.10 +- org.slf4j:slf4j-api:1.7.2
[flink] branch master updated (17d7c39bb2a -> ec2f3d99ac9)
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git from 17d7c39bb2a [FLINK-29047][k8s] Shade fabric8 kubernetes dependency with org.apache.flink.shaded prefix in flink-kubernetes add ec2f3d99ac9 [FLINK-29211][hive][legal] Update 2.3.9 NOTICE No new revisions were added by this update. Summary of changes: .../src/main/resources/META-INF/NOTICE | 20 1 file changed, 16 insertions(+), 4 deletions(-)
[flink-web] branch asf-site updated: Rebuild website
This is an automated email from the ASF dual-hosted git repository. chesnay pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/flink-web.git The following commit(s) were added to refs/heads/asf-site by this push: new 386514122 Rebuild website 386514122 is described below commit 38651412202a7b00a869fd6c9514c76e8d2301be Author: Chesnay Schepler AuthorDate: Thu Sep 8 19:30:42 2022 +0200 Rebuild website --- content/blog/feed.xml | 291 + content/blog/index.html| 41 +-- content/blog/page10/index.html | 36 ++- content/blog/page11/index.html | 36 ++- content/blog/page12/index.html | 42 +-- content/blog/page13/index.html | 46 ++-- content/blog/page14/index.html | 42 +-- content/blog/page15/index.html | 40 +-- content/blog/page16/index.html | 40 +-- content/blog/page17/index.html | 40 +-- content/blog/page18/index.html | 40 +-- content/blog/page19/index.html | 40 +-- content/blog/page2/index.html | 39 ++- content/blog/page20/index.html | 25 ++ content/blog/page3/index.html | 38 +-- content/blog/page4/index.html | 47 ++-- content/blog/page5/index.html | 49 ++-- content/blog/page6/index.html | 42 +-- content/blog/page7/index.html | 40 +-- content/blog/page8/index.html | 45 ++-- content/blog/page9/index.html | 43 ++- content/downloads.html | 18 +- content/index.html | 8 +- .../2022/09/08/akka-license-change.html} | 290 +--- content/zh/downloads.html | 18 +- content/zh/index.html | 8 +- 26 files changed, 660 insertions(+), 784 deletions(-) diff --git a/content/blog/feed.xml b/content/blog/feed.xml index 734493400..ed23f2c91 100644 --- a/content/blog/feed.xml +++ b/content/blog/feed.xml @@ -6,6 +6,66 @@ https://flink.apache.org/blog https://flink.apache.org/blog/feed.xml; rel="self" type="application/rss+xml" /> + +Regarding Akkas licensing change +pOn September 7th Lightbend announced a a href=https://www.lightbend.com/blog/why-we-are-changing-the-license-for-akkalicense change/a for the Akka project, the TL;DR being that you will need a commercial license to use future versions of Akka (2.7+) in production if you exceed a certain revenue threshold./p + +pWithin a few hours of the announcement several people reached out to the Flink project, worrying about the impact this has on Flink, as we use Akka internally./p + +pThe purpose of this blogpost is to clarify our position on the matter./p + +pPlease be aware that this topic is still quite fresh, and things are subject to change.br / +Should anything significant change we will amend this blogpost and inform you via the usual channels./p + +h1 id=give-me-the-short-versionGive me the short version/h1 + +pFlink is not in any immediate danger and we will ensure that users are not affected by this change./p + +pThe licensing of Flink will not change; it will stay Apache-licensed and will only contain dependencies that are compatible with it./p + +pWe will not use Akka versions with the new license./p + +h1 id=whats-the-plan-going-forwardWhat’s the plan going forward?/h1 + +pemFor now/em, we’ll stay on Akka 2.6, the current latest version that is still available under the original license. +Historically Akka has been incredibly stable, and combined with our limited use of features, we do not expect this to be a problem./p + +pMeanwhile, we will/p + +ul + liobserve how the situation unfolds (in particular w.r.t. community forks)/li + lilook into a replacement for Akka./li +/ul + +pShould a community fork be created (which at this time seems possible) we will switch to that fork in all likely-hood for 1.15+./p + +h2 id=what-if-a-new-security-vulnerabilities-is-found-in-akka-26What if a new security vulnerabilities is found in Akka 2.6?/h2 + +pThat is the big unknown./p + +pEven though we will be able to upgrade to 2.6.20 (the (apparently) last planned release for Akka 2.6) in Flink 1.17, the unfortunate reality is that a href=https://github.com/akka/akka/pull/31561#issuecomment-12392176022.6 will no longer be supported/a from that point onwards.br / +Should a CVE be discovered after that it is unlikely to be fixed in Akka 2.6./p + +pWe cannot provide a definitive answer as to how that case would be handled, as it depends on what the CVE is and/or whether a community fork already exists at the time./p + +h1 id=how-does-flink-use
