[GitHub] [flink] matriv commented on a change in pull request #17888: [Flink-24077] fix sporadic ut failing
matriv commented on a change in pull request #17888:
URL: https://github.com/apache/flink/pull/17888#discussion_r755214396
##
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/HBaseConnectorITCase.java
##
@@ -250,13 +252,25 @@ public void testTableSink() throws Exception {
+ " FROM "
+ TEST_TABLE_1;
+TableResult tableResult = tEnv.executeSql(query);
+
// wait to finish
-tEnv.executeSql(query).await();
+tableResult.await();
+
+assertEquals(
+"Expected INSERT rowKind", RowKind.INSERT,
tableResult.collect().next().getKind());
// start a batch scan job to verify contents in HBase table
TableEnvironment batchEnv = TableEnvironment.create(batchSettings);
batchEnv.executeSql(table2DDL);
+Table countTable =
+batchEnv.sqlQuery(
+"SELECT " + " count(h.rowkey) " + "FROM " +
TEST_TABLE_2 + " AS h");
+
+// check the count is right
Review comment:
I think the comment is superfluous, the assertion is obvious.
##
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/HBaseConnectorITCase.java
##
@@ -250,13 +252,25 @@ public void testTableSink() throws Exception {
+ " FROM "
+ TEST_TABLE_1;
+TableResult tableResult = tEnv.executeSql(query);
+
// wait to finish
-tEnv.executeSql(query).await();
+tableResult.await();
+
+assertEquals(
+"Expected INSERT rowKind", RowKind.INSERT,
tableResult.collect().next().getKind());
// start a batch scan job to verify contents in HBase table
TableEnvironment batchEnv = TableEnvironment.create(batchSettings);
batchEnv.executeSql(table2DDL);
+Table countTable =
+batchEnv.sqlQuery(
+"SELECT " + " count(h.rowkey) " + "FROM " +
TEST_TABLE_2 + " AS h");
Review comment:
```suggestion
"SELECT COUNT(h.rowkey) FROM " + TEST_TABLE_2 + "
AS h");
```
##
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/HBaseConnectorITCase.java
##
@@ -307,12 +327,25 @@ public void testTableSourceSinkWithDDL() throws Exception
{
+ " family4"
+ " from "
+ TEST_TABLE_1;
+
+TableResult tableResult = tEnv.executeSql(insertStatement);
+
// wait to finish
-tEnv.executeSql(insertStatement).await();
+tableResult.await();
+
+assertEquals(
+"Expected INSERT rowKind", RowKind.INSERT,
tableResult.collect().next().getKind());
// start a batch scan job to verify contents in HBase table
TableEnvironment batchEnv = TableEnvironment.create(batchSettings);
batchEnv.executeSql(table3DDL);
+
+Table countTable =
+batchEnv.sqlQuery(
+"SELECT " + " count(h.rowkey) " + "FROM " +
TEST_TABLE_3 + " AS h");
Review comment:
```suggestion
"SELECT COUNT(h.rowkey) FROM " + TEST_TABLE_3 + " AS
h");
```
##
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/HBaseConnectorITCase.java
##
@@ -329,9 +362,13 @@ public void testTableSourceSinkWithDDL() throws Exception {
+ " FROM "
+ TEST_TABLE_3
+ " AS h";
-Iterator collected = tEnv.executeSql(query).collect();
+
+TableResult tableResult3 = tEnv.executeSql(query);
Review comment:
```suggestion
TableResult tableResult2 = tEnv.executeSql(query);
```
it's a different method, no need to use `3` I guess.
##
File path:
flink-connectors/flink-connector-hbase-2.2/src/test/java/org/apache/flink/connector/hbase2/HBaseConnectorITCase.java
##
@@ -307,12 +327,25 @@ public void testTableSourceSinkWithDDL() throws Exception
{
+ " family4"
+ " from "
+ TEST_TABLE_1;
+
+TableResult tableResult = tEnv.executeSql(insertStatement);
+
// wait to finish
-tEnv.executeSql(insertStatement).await();
+tableResult.await();
+
+assertEquals(
+"Expected INSERT rowKind", RowKind.INSERT,
tableResult.collect().next().getKind());
// start a batch scan job to verify contents in HBase table
TableEnvironment batchEnv = TableEnvironment.create(batchSettings);
batchEnv.executeSql(table3DDL);
+
+Table countTable =
+batchEnv.sqlQuery(
+"SELECT " + " count(h.ro
[GitHub] [flink] flinkbot edited a comment on pull request #17133: [FLINK-24138] Architectural tests
flinkbot edited a comment on pull request #17133: URL: https://github.com/apache/flink/pull/17133#issuecomment-912323602 ## CI report: * cfb98c4cc708adb38ced719788af16a50625a1c1 UNKNOWN * 7f5fbf7c933209fd8cb7f8681499ddadaaa4eeef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26796) * 994ef1ae7de8c68688a26a0a9cbd24acb87a46f7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] nicoweidner commented on pull request #17855: [FLINK-24983][build] Upgrade surefire to 3.0.5-M5
nicoweidner commented on pull request #17855: URL: https://github.com/apache/flink/pull/17855#issuecomment-976675499 > Oh it's definitely related. I'm wondering why I never saw that on CI with Java 17... > > but more to the point, it's because the `integration-tests` surefire execution is setup to run anything that isn't a unit test, and it apparently throws up when dealing with certain scala constructs... Yeah, I suspected that some behavior changed with the new surefire version - but I couldn't figure out where this malformed class name that it complains about comes from... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17857: [FLINK-24985][tests] Relax assumptions of stacktrace layout
flinkbot edited a comment on pull request #17857: URL: https://github.com/apache/flink/pull/17857#issuecomment-975401663 ## CI report: * 7fe5c7914e7831df87b81e5229193737bb123bd2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26830) * 17691f4330fb8017970c9934c8e52ea91da7cdf4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingGe commented on pull request #17851: [FLINK-21214][kafka/IT] add retry rule for testScaleDownBeforeFirstCh…
JingGe commented on pull request #17851: URL: https://github.com/apache/flink/pull/17851#issuecomment-976672019 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24978) Upgrade ASM to 9.2
[ https://issues.apache.org/jira/browse/FLINK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-24978: --- Labels: pull-request-available (was: ) > Upgrade ASM to 9.2 > -- > > Key: FLINK-24978 > URL: https://issues.apache.org/jira/browse/FLINK-24978 > Project: Flink > Issue Type: Sub-task > Components: BuildSystem / Shaded >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0, shaded-15.0 > > > As per usual we need to bump flink-shaded-asm. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17133: [FLINK-24138] Architectural tests
flinkbot edited a comment on pull request #17133: URL: https://github.com/apache/flink/pull/17133#issuecomment-912323602 ## CI report: * cfb98c4cc708adb38ced719788af16a50625a1c1 UNKNOWN * 7f5fbf7c933209fd8cb7f8681499ddadaaa4eeef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26796) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-shaded] XComp commented on a change in pull request #103: [FLINK-24978] Upgrade ASM to 9.2
XComp commented on a change in pull request #103: URL: https://github.com/apache/flink-shaded/pull/103#discussion_r755208406 ## File path: flink-shaded-asm-9/src/main/resources/META-INF/NOTICE ## @@ -7,7 +7,7 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the BSD license. See bundled license files for details. -- org.ow2.asm:asm:7.1 -- org.ow2.asm:asm-analysis:7.1 -- org.ow2.asm:asm-commons:7.1 -- org.ow2.asm:asm-tree:7.1 \ No newline at end of file +- org.ow2.asm:asm:9.2 Review comment: I cannot mark the line directly but there is another location further up in this file which we might want to update: ``` flink-shaded-asm7 -> flink-shaded-asm-9 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on pull request #17855: [FLINK-24983][build] Upgrade surefire to 3.0.5-M5
zentol commented on pull request #17855: URL: https://github.com/apache/flink/pull/17855#issuecomment-976668983 Oh it's definitely related. I'm wondering why I never saw that on CI with Java 17... but more to the point, it's because the `integration-tests` surefire execution is setup to run anything that isn't a unit test, and it apparently throws up when dealing with certain scala constructs... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17888: [Flink-24077] fix sporadic ut failing
flinkbot edited a comment on pull request #17888: URL: https://github.com/apache/flink/pull/17888#issuecomment-976661814 ## CI report: * b6a5b1890492cd51454d4ab44a3bba2db0fe8444 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26946) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17133: [FLINK-24138] Architectural tests
flinkbot edited a comment on pull request #17133: URL: https://github.com/apache/flink/pull/17133#issuecomment-912323602 ## CI report: * cfb98c4cc708adb38ced719788af16a50625a1c1 UNKNOWN * 7f5fbf7c933209fd8cb7f8681499ddadaaa4eeef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26796) * 994ef1ae7de8c68688a26a0a9cbd24acb87a46f7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol edited a comment on pull request #17852: [FLINK-24979][build][hbase] Remove MaxPermSize setting
zentol edited a comment on pull request #17852:
URL: https://github.com/apache/flink/pull/17852#issuecomment-976661587
> Can we run into situations now where the fork number is not set when
generating the logs?
No, because mvn.forkNumber is still set in the root pom surefire
configuration.
> Why does [-Dmvn.forkNumber... ] not have any effect?
It does, but we can use use the configuration from the root pom.
> What happens to the expression ${sys:mvn.forkNumber} if the variable is
not set? Will it stay like this verbatim?
yes. You can actually find such a file in the azure artifacts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17888: [Flink-24077] fix sporadic ut failing
flinkbot commented on pull request #17888: URL: https://github.com/apache/flink/pull/17888#issuecomment-976661814 ## CI report: * b6a5b1890492cd51454d4ab44a3bba2db0fe8444 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17882: [FLINK-24971][BP-1.12] Adding retry mechanism in case git clone fails
flinkbot edited a comment on pull request #17882: URL: https://github.com/apache/flink/pull/17882#issuecomment-976334078 ## CI report: * a72b0de8df9c3eca5580d18034b754288ff92658 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26914) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on pull request #17852: [FLINK-24979][build][hbase] Remove MaxPermSize setting
zentol commented on pull request #17852:
URL: https://github.com/apache/flink/pull/17852#issuecomment-976661587
> Can we run into situations now where the fork number is not set when
generating the logs?
No, because mvn.forkNumber is still set in the root pom surefire
configuration.
> What happens to the expression ${sys:mvn.forkNumber} if the variable is
not set? Will it stay like this verbatim?
yes. You can actually find such a file in the azure artifacts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17660: [FLINK-24657][runtime] Added metric of the total real size of input/o…
flinkbot edited a comment on pull request #17660: URL: https://github.com/apache/flink/pull/17660#issuecomment-958945516 ## CI report: * 7f7705d6f236eedbf624fdcb448ca3234828157c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26927) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-24981) Fix JDK12+ profile activations
[ https://issues.apache.org/jira/browse/FLINK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-24981. Resolution: Fixed master: b5ce1a1ac25fc375f0623d9c63124fef38578dc0 > Fix JDK12+ profile activations > -- > > Key: FLINK-24981 > URL: https://issues.apache.org/jira/browse/FLINK-24981 > Project: Flink > Issue Type: Sub-task > Components: Build System >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > All Java11 profiles should be active for Java 11 and all subsequent Java > versions, but this currently only applies to the "main" profile in the root > pom. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17133: [FLINK-24138] Architectural tests
flinkbot edited a comment on pull request #17133: URL: https://github.com/apache/flink/pull/17133#issuecomment-912323602 ## CI report: * cfb98c4cc708adb38ced719788af16a50625a1c1 UNKNOWN * 7f5fbf7c933209fd8cb7f8681499ddadaaa4eeef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26796) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol merged pull request #17853: [FLINK-24981][build] Fix profile activations for JDK 12+
zentol merged pull request #17853: URL: https://github.com/apache/flink/pull/17853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17888: [Flink-24077] fix sporadic ut failing
flinkbot commented on pull request #17888: URL: https://github.com/apache/flink/pull/17888#issuecomment-976656426 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a415204749bba8a4d70164fb123b8275084ce37b (Tue Nov 23 14:47:23 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on a change in pull request #17133: [FLINK-24138] Architectural tests
zentol commented on a change in pull request #17133:
URL: https://github.com/apache/flink/pull/17133#discussion_r755189758
##
File path:
flink-architecture-tests/src/test/java/org/apache/flink/architecture/rules/ApiAnnotationRules.java
##
@@ -0,0 +1,168 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.architecture.rules;
+
+import org.apache.flink.annotation.Experimental;
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.annotation.Public;
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.annotation.VisibleForTesting;
+
+import com.tngtech.archunit.base.DescribedPredicate;
+import com.tngtech.archunit.core.domain.JavaMethodCall;
+import com.tngtech.archunit.junit.ArchTest;
+import com.tngtech.archunit.lang.ArchRule;
+
+import static
com.tngtech.archunit.core.domain.JavaClass.Predicates.resideOutsideOfPackage;
+import static
com.tngtech.archunit.core.domain.properties.CanBeAnnotated.Predicates.annotatedWith;
+import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.methods;
+import static com.tngtech.archunit.library.freeze.FreezingArchRule.freeze;
+import static org.apache.flink.architecture.common.Conditions.fulfill;
+import static
org.apache.flink.architecture.common.Conditions.haveLeafArgumentTypes;
+import static
org.apache.flink.architecture.common.Conditions.haveLeafReturnTypes;
+import static
org.apache.flink.architecture.common.GivenJavaClasses.javaClassesThat;
+import static
org.apache.flink.architecture.common.GivenJavaClasses.noJavaClassesThat;
+import static
org.apache.flink.architecture.common.Predicates.areDirectlyAnnotatedWithAtLeastOneOf;
+import static
org.apache.flink.architecture.common.SourcePredicates.areJavaClasses;
+import static
org.apache.flink.architecture.common.SourcePredicates.areProductionCode;
+
+/** Rules for API visibility annotations. */
+public class ApiAnnotationRules {
+
+@ArchTest
+public static final ArchRule ANNOTATED_APIS =
+freeze(
+javaClassesThat()
+.resideInAPackage("org.apache.flink..api..")
+.and()
+.resideOutsideOfPackage("..internal..")
+.and()
+.arePublic()
+.should(
+fulfill(
+
areDirectlyAnnotatedWithAtLeastOneOf(
+Internal.class,
+Experimental.class,
+PublicEvolving.class,
+Public.class,
+Deprecated.class)))
+.as(
+"Classes in API packages should have at
least one API visibility annotation."));
+
+/**
+ * This is a stronger requirement than {@link
+ * #PUBLIC_EVOLVING_API_METHODS_USE_ONLY_PUBLIC_EVOLVING_API_TYPES} and
ensures that {@link
+ * Public} APIs do not use {@link PublicEvolving} APIs.
+ */
+@ArchTest
+public static final ArchRule PUBLIC_API_METHODS_USE_ONLY_PUBLIC_API_TYPES =
+freeze(
+methods()
+.that()
+.areDeclaredInClassesThat(
+
areJavaClasses().and(annotatedWith(Public.class)))
+.and()
+.arePublic()
+.and()
+.areNotAnnotatedWith(PublicEvolving.class)
+.and()
+.areNotAnnotatedWith(Internal.class)
+.and()
+.areNotAnnotatedWith(Deprecated.class)
+.and()
+.areNotAnnotatedWith(Experimental.class)
+.should(
+haveLeafReturnTypes(
+
reside
[GitHub] [flink] flinkbot edited a comment on pull request #17653: FLINK SQL checkpoint不生效
flinkbot edited a comment on pull request #17653: URL: https://github.com/apache/flink/pull/17653#issuecomment-958686553 ## CI report: * a2e241da13d788d165fe7f106b38f707a1cbf79f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26918) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17133: [FLINK-24138] Architectural tests
flinkbot edited a comment on pull request #17133: URL: https://github.com/apache/flink/pull/17133#issuecomment-912323602 ## CI report: * cfb98c4cc708adb38ced719788af16a50625a1c1 UNKNOWN * 7f5fbf7c933209fd8cb7f8681499ddadaaa4eeef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26796) * 994ef1ae7de8c68688a26a0a9cbd24acb87a46f7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] JingGe opened a new pull request #17888: [Flink-24077] fix sporadic ut failing
JingGe opened a new pull request #17888: URL: https://github.com/apache/flink/pull/17888 ## What is the purpose of the change try to fix some unstable failing tests. ## Brief change log - add check of row count after insert - explicitly call await() for batch select ## Verifying this change This change is a trivial rework / test case improvement. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #17463: [Hotfix] Fix typos.
MartijnVisser commented on pull request #17463: URL: https://github.com/apache/flink/pull/17463#issuecomment-976644996 Please also make sure that the PR template is properly filled in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] MartijnVisser commented on a change in pull request #17463: [Hotfix] Fix typos.
MartijnVisser commented on a change in pull request #17463:
URL: https://github.com/apache/flink/pull/17463#discussion_r755184904
##
File path:
flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/table/IndexGenerator.java
##
@@ -34,6 +34,6 @@
*/
default void open() {}
-/** Generate index name according the the given row. */
+/** Generate index name according the given row. */
Review comment:
```suggestion
/** Generate index name according to the given row. */
```
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/api/writer/ResultPartitionWriter.java
##
@@ -38,7 +38,7 @@
* ResultPartitionWriter#fail(Throwable)} or {@link
ResultPartitionWriter#finish()}, it abruptly
* triggers failure and cancellation of production. In this case {@link
* ResultPartitionWriter#fail(Throwable)} still needs to be called afterwards
to fully release all
- * resources associated the the partition and propagate failure cause to the
consumer if possible.
+ * resources associated the partition and propagate failure cause to the
consumer if possible.
Review comment:
```suggestion
* resources associated with the partition and propagate failure cause to
the consumer if possible.
```
##
File path:
flink-connectors/flink-connector-elasticsearch-base/src/main/java/org/apache/flink/streaming/connectors/elasticsearch/index/IndexGenerator.java
##
@@ -33,6 +33,6 @@
*/
default void open() {}
-/** Generate index name according the the given row. */
+/** Generate index name according the given row. */
Review comment:
```suggestion
/** Generate index name according to the given row. */
```
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/MergingWindowSet.java
##
@@ -145,8 +145,8 @@ public void retireWindow(W window) {
*
* @param newWindow The new {@code Window} to add.
* @param mergeFunction The callback to be invoked in case a merge occurs.
- * @return The {@code Window} that new new {@code Window} ended up in.
This can also be the the
- * new {@code Window} itself in case no merge occurred.
+ * @return The {@code Window} that new new {@code Window} ended up in.
This can also be the new
+ * {@code Window} itself in case no merge occurred.
Review comment:
```suggestion
* @return The {@code Window} that new {@code Window} ended up in. This
can also be the new
* {@code Window} itself in case no merge occurred.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink-benchmarks] Aitozi commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
Aitozi commented on a change in pull request #41:
URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755188360
##
File path:
src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java
##
@@ -52,10 +54,25 @@
final ThreadLocalRandom random = ThreadLocalRandom.current();
@Param({"HEAP", "ROCKSDB"})
-protected StateBackendBenchmarkUtils.StateBackendType backendType;
+private StateBackendBenchmarkUtils.StateBackendType backendType;
+
+@Param({StateBenchmarkConstants.defaultStateDataDirPath})
+private String stateDataDirPath;
Review comment:
ok, I will provide a single configuration implementation .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17887: [FLINK-25020] Properly forward exception
flinkbot edited a comment on pull request #17887: URL: https://github.com/apache/flink/pull/17887#issuecomment-976615140 ## CI report: * 15a77538ee4fdbc85ca6c875ae3797b0a9b2d9e5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26944) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17670: [FLINK-24760][docs] Update user document for batch window tvf
flinkbot edited a comment on pull request #17670: URL: https://github.com/apache/flink/pull/17670#issuecomment-960539433 ## CI report: * 7c571626f1139e8af963bc3314de12a801587f52 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26926) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-16931) Large _metadata file lead to JobManager not responding when restart
[
https://issues.apache.org/jira/browse/FLINK-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-16931:
---
Priority: Not a Priority (was: Minor)
> Large _metadata file lead to JobManager not responding when restart
> ---
>
> Key: FLINK-16931
> URL: https://issues.apache.org/jira/browse/FLINK-16931
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.9.2, 1.10.0, 1.11.0, 1.12.0
>Reporter: Lu Niu
>Priority: Not a Priority
> Labels: auto-unassigned, stale-minor
>
> When _metadata file is big, JobManager could never recover from checkpoint.
> It fall into a loop that fetch checkpoint -> JM timeout -> restart. Here is
> related log:
> {code:java}
> 2020-04-01 17:08:25,689 INFO
> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore -
> Recovering checkpoints from ZooKeeper.
> 2020-04-01 17:08:25,698 INFO
> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore - Found
> 3 checkpoints in ZooKeeper.
> 2020-04-01 17:08:25,698 INFO
> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore -
> Trying to fetch 3 checkpoints from storage.
> 2020-04-01 17:08:25,698 INFO
> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore -
> Trying to retrieve checkpoint 50.
> 2020-04-01 17:08:48,589 INFO
> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore -
> Trying to retrieve checkpoint 51.
> 2020-04-01 17:09:12,775 INFO org.apache.flink.yarn.YarnResourceManager - The
> heartbeat of JobManager with id 02500708baf0bb976891c391afd3d7d5 timed out.
> {code}
> Digging into the code, looks like ExecutionGraph::restart runs in JobMaster
> main thread and finally calls
> ZooKeeperCompletedCheckpointStore::retrieveCompletedCheckpoint which download
> file form DFS. The main thread is basically blocked for a while because of
> this. One possible solution is to making the downloading part async. More
> things might need to consider as the original change tries to make it
> single-threaded. [https://github.com/apache/flink/pull/7568]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink-benchmarks] pnowojski commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
pnowojski commented on a change in pull request #41:
URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755179227
##
File path:
src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java
##
@@ -52,10 +54,25 @@
final ThreadLocalRandom random = ThreadLocalRandom.current();
@Param({"HEAP", "ROCKSDB"})
-protected StateBackendBenchmarkUtils.StateBackendType backendType;
+private StateBackendBenchmarkUtils.StateBackendType backendType;
+
+@Param({StateBenchmarkConstants.defaultStateDataDirPath})
+private String stateDataDirPath;
Review comment:
I think either of those options would work, depends on what we want to
achieve? I think using `@Params` makes sense if we really want to present the
results from different configurations side by side, especially in the WebUI or
in the continuous regression testing scripts. However if there are many many
different constants that you would like to change, it probably doesn't make
sense to keep them all separately as `@Params`, as it would simply explode the
number of tests (jmh is testing all combinations of params).
Having said all of that, a dedicated config file might be a not a bad
solution. For now there can be a single configuration file that anyone can
overwrite as he chooses. If we ever decide to present the results from
different configs side by side, we can do this via `@Param` `preset`, where
`preset` would be a name of one of those config files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17839: [FLINK-24859] document new connectors formats
flinkbot edited a comment on pull request #17839: URL: https://github.com/apache/flink/pull/17839#issuecomment-974134553 ## CI report: * Unknown: [CANCELED](TBD) * d066843917e65f45a3a7299ae58b9ef5bc8ee782 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26943) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17887: [FLINK-25020] Properly forward exception
flinkbot commented on pull request #17887: URL: https://github.com/apache/flink/pull/17887#issuecomment-976615140 ## CI report: * 15a77538ee4fdbc85ca6c875ae3797b0a9b2d9e5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17885: [FLINK-24772][docs] Update documentation for individual window table-value function
flinkbot edited a comment on pull request #17885: URL: https://github.com/apache/flink/pull/17885#issuecomment-976414441 ## CI report: * 9e922727fac6574c00a3b9120128a798f61147f3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26925) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25020) FlinkSecurityManager can swallow exception
[ https://issues.apache.org/jira/browse/FLINK-25020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25020: --- Labels: pull-request-available (was: ) > FlinkSecurityManager can swallow exception > -- > > Key: FLINK-25020 > URL: https://issues.apache.org/jira/browse/FLINK-25020 > Project: Flink > Issue Type: Bug > Components: API / Core >Affects Versions: 1.13.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > If the security manager cannot be set then > FlinkSecurityManager#setFromConfiguration swallows the exception, because it > is passed to String.format instead of the IllegalConfigurationException > constructor. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot commented on pull request #17887: [FLINK-25020] Properly forward exception
flinkbot commented on pull request #17887: URL: https://github.com/apache/flink/pull/17887#issuecomment-976612397 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 15a77538ee4fdbc85ca6c875ae3797b0a9b2d9e5 (Tue Nov 23 14:26:22 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] akalash commented on pull request #17796: [FLINK-24846][streaming] Closing the mailbox only after it drains all mails including recursive ones
akalash commented on pull request #17796:
URL: https://github.com/apache/flink/pull/17796#issuecomment-976609683
@pnowojski, I understand the problem something like this:
1. AsyncWaitOperator#processElement registers the timeout for the entry
2. endInput happens which waits for in-flight inputs but doesn't wait for
the registered timeout
3. finish happens
4. mailbox is suspend
5. timer fires which put mail to mailbox
(AsyncWaitOperator.ResultHandler#processInMailbox)
6. all timers close
7. mailbox is marked as close which forbids adding new mails
8. drain is called which execute mail described above which recursively try
to add new mail to mailbox which fails
In my opinion, it is impossible to be sure that there are no new data
possible until we close all potential sources of data like timers. So in this
PR I close all timers which should guarantee that no new outside data is
possible then I drain mailbox, and only then I mark the mailbox as closed.
It is difficult to say something about `finish ` contract because it indeed
guarantees that no new records from the channel are possible but it doesn't
guarantee that records from another place(like timer service) are impossible.
If we think that it is not correct we should rewrite
`AsyncWaitOperator#waitInFlightInputsFinished` in such a way that it waits for
time service as well(which is not so easy to do) or we can ignore all time
service events after `endInput` was received. For example, we can rewrite
`AsyncWaitOperator.ResultHandler#timerTriggered` to:
```
private void timerTriggered() throws Exception {
if (!completed.get() && !endInput.get()) {
userFunction.timeout(inputRecord.getValue(), this);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17839: [FLINK-24859] document new connectors formats
flinkbot edited a comment on pull request #17839: URL: https://github.com/apache/flink/pull/17839#issuecomment-974134553 ## CI report: * Unknown: [CANCELED](TBD) * d066843917e65f45a3a7299ae58b9ef5bc8ee782 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25020) FlinkSecurityManager can swallow exception
Chesnay Schepler created FLINK-25020: Summary: FlinkSecurityManager can swallow exception Key: FLINK-25020 URL: https://issues.apache.org/jira/browse/FLINK-25020 Project: Flink Issue Type: Bug Components: API / Core Affects Versions: 1.13.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: 1.15.0 If the security manager cannot be set then FlinkSecurityManager#setFromConfiguration swallows the exception, because it is passed to String.format instead of the IllegalConfigurationException constructor. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] echauchot commented on pull request #17839: [FLINK-24859] document new connectors formats
echauchot commented on pull request #17839: URL: https://github.com/apache/flink/pull/17839#issuecomment-976603434 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] XComp commented on a change in pull request #17857: [FLINK-24985][tests] Relax assumptions of stacktrace layout
XComp commented on a change in pull request #17857:
URL: https://github.com/apache/flink/pull/17857#discussion_r755157198
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPoolDestroyTest.java
##
@@ -119,8 +119,15 @@ public void testDestroyWhileBlockingRequest() throws
Exception {
*/
public static boolean isInBlockingBufferRequest(StackTraceElement[]
stackTrace) {
if (stackTrace.length >= 8) {
-return stackTrace[5].getMethodName().equals("get")
-&&
stackTrace[7].getClassName().equals(LocalBufferPool.class.getName());
+for (int x = 0; x < stackTrace.length - 2; x++) {
+if (stackTrace[x].getMethodName().equals("get")
+&& stackTrace[x + 2]
+.getClassName()
+.equals(LocalBufferPool.class.getName())) {
+return true;
+}
+}
+return false;
Review comment:
That return could be moved outside of the if block if its else branch is
removed as well. ¯\_(ツ)_/¯
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17789: [FLINK-24351][docs] Translate "JSON Function" pages into Chinese
flinkbot edited a comment on pull request #17789: URL: https://github.com/apache/flink/pull/17789#issuecomment-968290372 ## CI report: * 3db6df5e2bfaf867f718ed0fb8d2929511a1e85c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26920) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17865: [FLINK-24993][core] Harden ExceptionUtils against changes to exceptions
flinkbot edited a comment on pull request #17865: URL: https://github.com/apache/flink/pull/17865#issuecomment-975516895 ## CI report: * 707f22044b3dc3ef8c61a49b892f3515df6a8644 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26849) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17851: [FLINK-21214][kafka/IT] add retry rule for testScaleDownBeforeFirstCh…
flinkbot edited a comment on pull request #17851: URL: https://github.com/apache/flink/pull/17851#issuecomment-975348551 ## CI report: * d345c99d5d16c997c423941618e5441d17ba3f9f Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26923) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17848: [FLINK-24583] [connectors/elasticsearch] Improve test stability by blocking until ES has acknowledged all records
flinkbot edited a comment on pull request #17848: URL: https://github.com/apache/flink/pull/17848#issuecomment-975246406 ## CI report: * ab1fe6e89640d5ebc7c2f8329fdfb1ff22694b49 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26809) * 96fc657d42690427da941acac84f9c895262bc24 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26942) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24978) Upgrade ASM to 9.2
[ https://issues.apache.org/jira/browse/FLINK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448021#comment-17448021 ] Chesnay Schepler commented on FLINK-24978: -- PRs should be detected automatically. > Upgrade ASM to 9.2 > -- > > Key: FLINK-24978 > URL: https://issues.apache.org/jira/browse/FLINK-24978 > Project: Flink > Issue Type: Sub-task > Components: BuildSystem / Shaded >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Fix For: 1.15.0, shaded-15.0 > > > As per usual we need to bump flink-shaded-asm. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17865: [FLINK-24993][core] Harden ExceptionUtils against changes to exceptions
flinkbot edited a comment on pull request #17865: URL: https://github.com/apache/flink/pull/17865#issuecomment-975516895 ## CI report: * 707f22044b3dc3ef8c61a49b892f3515df6a8644 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26849) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17848: [FLINK-24583] [connectors/elasticsearch] Improve test stability by blocking until ES has acknowledged all records
flinkbot edited a comment on pull request #17848: URL: https://github.com/apache/flink/pull/17848#issuecomment-975246406 ## CI report: * ab1fe6e89640d5ebc7c2f8329fdfb1ff22694b49 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26809) * 96fc657d42690427da941acac84f9c895262bc24 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on pull request #17865: [FLINK-24993][core] Harden ExceptionUtils against changes to exceptions
zentol commented on pull request #17865: URL: https://github.com/apache/flink/pull/17865#issuecomment-976576232 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17847: [FLINK-6573][Connectors/Common ] Flink MongoDB Connector
flinkbot edited a comment on pull request #17847: URL: https://github.com/apache/flink/pull/17847#issuecomment-974817636 ## CI report: * 8364104af21c6c012779149008980072b18cb87c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26933) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25019) Remove misleading comment regarding flink-shaded-netty re-use
[ https://issues.apache.org/jira/browse/FLINK-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-25019: - Priority: Minor (was: Major) > Remove misleading comment regarding flink-shaded-netty re-use > - > > Key: FLINK-25019 > URL: https://issues.apache.org/jira/browse/FLINK-25019 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: shaded-15.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Minor > Fix For: shaded-15.0 > > > In FLINK-24447 we modified Zookeeper to bundle Netty, but added a comment > stating that we re-use flink-shaded netty, which is incorrect. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25019) Remove misleading comment regarding flink-shaded-netty re-use
[ https://issues.apache.org/jira/browse/FLINK-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-25019: - Summary: Remove misleading comment regarding flink-shaded-netty re-use (was: Incorrect netty shading prefix) > Remove misleading comment regarding flink-shaded-netty re-use > - > > Key: FLINK-25019 > URL: https://issues.apache.org/jira/browse/FLINK-25019 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: shaded-15.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Fix For: shaded-15.0 > > > In FLINK-24447 we modified Zookeeper to relocate Netty, with the goal being > to reuse flink-shaded-netty if required. We messed up the shading prefix > however. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25019) Remove misleading comment regarding flink-shaded-netty re-use
[ https://issues.apache.org/jira/browse/FLINK-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-25019: - Description: In FLINK-24447 we modified Zookeeper to bundle Netty, but added a comment stating that we re-use flink-shaded netty, which is incorrect. (was: In FLINK-24447 we modified Zookeeper to relocate Netty, with the goal being to reuse flink-shaded-netty if required. We messed up the shading prefix however.) > Remove misleading comment regarding flink-shaded-netty re-use > - > > Key: FLINK-25019 > URL: https://issues.apache.org/jira/browse/FLINK-25019 > Project: Flink > Issue Type: Bug > Components: BuildSystem / Shaded >Affects Versions: shaded-15.0 >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Fix For: shaded-15.0 > > > In FLINK-24447 we modified Zookeeper to bundle Netty, but added a comment > stating that we re-use flink-shaded netty, which is incorrect. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25019) Incorrect netty shading prefix
Chesnay Schepler created FLINK-25019: Summary: Incorrect netty shading prefix Key: FLINK-25019 URL: https://issues.apache.org/jira/browse/FLINK-25019 Project: Flink Issue Type: Bug Components: BuildSystem / Shaded Affects Versions: shaded-15.0 Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: shaded-15.0 In FLINK-24447 we modified Zookeeper to relocate Netty, with the goal being to reuse flink-shaded-netty if required. We messed up the shading prefix however. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (FLINK-24978) Upgrade ASM to 9.2
[
https://issues.apache.org/jira/browse/FLINK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448018#comment-17448018
]
Matthias edited comment on FLINK-24978 at 11/23/21, 1:49 PM:
-
Doesn't the PR detection work on the {{apache/flink-shaded}} repository? FYI:
The corresponding PR is: https://github.com/apache/flink-shaded/pull/103
was (Author: mapohl):
Doesn't the PR detection work on the {{apache/flink-shaded}} repository? The
corresponding PR is: https://github.com/apache/flink-shaded/pull/103
> Upgrade ASM to 9.2
> --
>
> Key: FLINK-24978
> URL: https://issues.apache.org/jira/browse/FLINK-24978
> Project: Flink
> Issue Type: Sub-task
> Components: BuildSystem / Shaded
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.15.0, shaded-15.0
>
>
> As per usual we need to bump flink-shaded-asm.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-24978) Upgrade ASM to 9.2
[
https://issues.apache.org/jira/browse/FLINK-24978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448018#comment-17448018
]
Matthias commented on FLINK-24978:
--
Doesn't the PR detection work on the {{apache/flink-shaded}} repository? The
corresponding PR is: https://github.com/apache/flink-shaded/pull/103
> Upgrade ASM to 9.2
> --
>
> Key: FLINK-24978
> URL: https://issues.apache.org/jira/browse/FLINK-24978
> Project: Flink
> Issue Type: Sub-task
> Components: BuildSystem / Shaded
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.15.0, shaded-15.0
>
>
> As per usual we need to bump flink-shaded-asm.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-24920) Update README examples
[ https://issues.apache.org/jira/browse/FLINK-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448016#comment-17448016 ] Martijn Visser commented on FLINK-24920: [~Sergey Nuyanzin] I think there's indeed some more missing in the README, I've updated this ticket to specifically mention the examples from the README. I've also taken a look at your PR. > Update README examples > -- > > Key: FLINK-24920 > URL: https://issues.apache.org/jira/browse/FLINK-24920 > Project: Flink > Issue Type: Sub-task >Reporter: Martijn Visser >Priority: Major > > Flink's README contains outdated examples (DataSet) and no examples for Table > API. We should at least update the examples -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-24920) Update README examples
[ https://issues.apache.org/jira/browse/FLINK-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-24920: --- Summary: Update README examples (was: Update README) > Update README examples > -- > > Key: FLINK-24920 > URL: https://issues.apache.org/jira/browse/FLINK-24920 > Project: Flink > Issue Type: Sub-task >Reporter: Martijn Visser >Priority: Major > > Flink's README contains outdated examples (DataSet) and no examples for Table > API. We should at least update the examples -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17666: [FLINK-21327][table-planner-blink] Support window TVF in batch mode
flinkbot edited a comment on pull request #17666: URL: https://github.com/apache/flink/pull/17666#issuecomment-960514675 ## CI report: * 51b6f01db6e073d4f943fca2d0aba057bbabac72 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26917) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17554: [FLINK-24624][Kubernetes]Kill cluster when starting kubernetes session or application cluster failed
flinkbot edited a comment on pull request #17554: URL: https://github.com/apache/flink/pull/17554#issuecomment-950550174 ## CI report: * cf579eec3aa9afc68f2cf9f5b77d5805ec4496b4 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26909) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] MartijnVisser commented on a change in pull request #17839: [FLINK-24859] document new connectors formats
MartijnVisser commented on a change in pull request #17839:
URL: https://github.com/apache/flink/pull/17839#discussion_r755110746
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
Review comment:
```suggestion
# Parquet
```
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink supports reading [parquet](https://parquet.apache.org/) files and
producing [Flink
rows](https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/types/Row.html).
+To use that format you need to add the dependency below to your project:
+
+```xml
+{{< artifact flink-parquet >}}
+```
+
+The DataStream API is now compatible with both batch and streaming execution
modes.
Review comment:
This could imply that adding the dependency is necessary to make the
DataStream API compatible with both batch & streaming mode :) I think we can
remove this sentence.
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink supports reading [parquet](https://parquet.apache.org/) files and
producing [Flink
rows](https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/types/Row.html).
Review comment:
I'm wondering if it makes sense to include 'producing Flink rows' here.
Doesn't Flink "just" support reading and writing Parquet files? Rows are just
an intermediary concept of Flink itself?
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink supports reading [parquet](https://parquet.apache.org/) files and
producing [Flink
rows](https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/types/Row.html).
+To use that format you need to add the dependency below to your project:
Review comment:
```suggestion
To use the format you need to add the Flink Parquet dependency to your
project.
```
##
File path: docs/content/docs/connectors/datastream/formats/text_files.md
##
@@ -0,0 +1,69 @@
+---
+title: "Text files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/text_files.html
+- /apis/streaming/connectors/formats/text_files.html
+---
+
+
+
+# Text files format
Review comment:
I don't have a better suggestion, but I think `Text files format` is
ambiguous. It could be CSV, JSON, AVRO etc. WDYT?
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink supports reading [parquet](https://parquet.apache.org/) files and
producing [Flink
rows](https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/types/Row.html).
+To use that format you need to add the dependency below to your project:
+
+```xml
+{{< artifact flink-parquet >}}
+```
+
+The DataStream API is now compatible with both batch and streaming execution
modes.
+And this format is compatible with the new Source that can be used in both
modes.
+Thus, you can use this format in two ways:
+- Bounded read for batch mode
+- Continuous read for streaming mode: monitors a directory for new files that
appear
+
+**Bounded read example**:
+
+In this example we create a DataStream containing parquet records as flink
Rows. We project the schema to read only certain fields ("f7", "f4" and "f99").
Review comment:
```suggestion
In this example we create a DataStream-based application containing Parquet
records as Flink Rows. We project the schema to read only certain fields ("f7",
"f4" and "f99").
```
##
File path: docs/content/docs/connectors/datastream/formats/parquet.md
##
@@ -0,0 +1,99 @@
+---
+title: "Parquet files"
+weight: 4
+type: docs
+aliases:
+- /dev/connectors/formats/parquet.html
+- /apis/streaming/connectors/formats/parquet.html
+---
+
+
+
+# Parquet format
+
+Flink supports reading [parquet](https://parquet.apache.org/) files and
producing [Flink
rows](https://nightlies
[jira] [Created] (FLINK-25018) Upgrade Curator to 5.2.0
Chesnay Schepler created FLINK-25018: Summary: Upgrade Curator to 5.2.0 Key: FLINK-25018 URL: https://issues.apache.org/jira/browse/FLINK-25018 Project: Flink Issue Type: Improvement Components: BuildSystem / Shaded Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: shaded-15.0 Upgrade Curator to 5.2.0 to properly fix FLINK-24543. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25017) Drop Zookeeper 3.4
[ https://issues.apache.org/jira/browse/FLINK-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25017: --- Labels: pull-request-available (was: ) > Drop Zookeeper 3.4 > -- > > Key: FLINK-25017 > URL: https://issues.apache.org/jira/browse/FLINK-25017 > Project: Flink > Issue Type: Improvement > Components: BuildSystem / Shaded >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: shaded-15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17538: [FLINK-24325][connectors/elasticsearch] Create Elasticsearch 6 Sink
flinkbot edited a comment on pull request #17538: URL: https://github.com/apache/flink/pull/17538#issuecomment-948574914 ## CI report: * 2d2e314314c0aca9b334ff09dd744d075b47cb53 UNKNOWN * 07d1221acb25978fb79a84beed8e81122e40daca Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26916) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-shaded] zentol opened a new pull request #104: [FLINK-25017] Drop Zookeeper 3.4
zentol opened a new pull request #104: URL: https://github.com/apache/flink-shaded/pull/104 We no longer need to publish a 3.4 artifact, because there will be no future patches to ZK 3.4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24905) KDS implementation of Async Sink Table API
[
https://issues.apache.org/jira/browse/FLINK-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448010#comment-17448010
]
Arvid Heise commented on FLINK-24905:
-
Table API is not giving a strong guarantee for state upgrades in 1.14 but that
may change later. So to be safe, it would be awesome if the state can be
migrated (if there is any).
We should keep options aligned between the versions, you can use fallback keys
to maintain backward compatibility.
I don't see a need to support the old connector unless some features are yet to
be migrated. If they really want to use the old connector, they can just use
sql-connector-kinesis-1.14.jar. In the end, we want to get rid of the old
connector rather sooner than later and then the option
{{sink.use-kinesis-producer}} is deprecated. This may cause more friction than
having a drop-in replacement.
> KDS implementation of Async Sink Table API
> --
>
> Key: FLINK-24905
> URL: https://issues.apache.org/jira/browse/FLINK-24905
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Kinesis
>Reporter: Zichen Liu
>Assignee: Ahmed Hamdy
>Priority: Major
> Fix For: 1.15.0
>
>
> h2. Motivation
> *User stories:*
> As a Flink user, I’d like to use the Table API for the new Kinesis Data
> Streams sink.
> *Scope:*
> * Introduce {{AsyncDynamicTableSink}} that enables Sinking Tables into Async
> Implementations.
> * Implement a new {{KinesisDynamicTableSink}} that uses
> {{KinesisDataStreamSink}} Async Implementation and implements
> {{{}AsyncDynamicTableSink{}}}.
> * The implementation introduces Async Sink configurations as optional
> options in the table definition, with default values derived from the
> {{KinesisDataStream}} default values.
> * Unit/Integration testing. modify KinesisTableAPI tests for the new
> implementation, add unit tests for {{AsyncDynamicTableSink}} and
> {{KinesisDynamicTableSink}} and {{{}KinesisDynamicTableSinkFactory{}}}.
> * Java / code-level docs.
> h2. References
> More details to be found
> [https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25017) Drop Zookeeper 3.4
[ https://issues.apache.org/jira/browse/FLINK-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448011#comment-17448011 ] Chesnay Schepler commented on FLINK-25017: -- ZK 3.4 is no longer being maintained, and we already published the latest version in flink-shaded 14. We can drop it from flink-shaded, and then look iunto upgrading to ZK 3.6 and curator 5. > Drop Zookeeper 3.4 > -- > > Key: FLINK-25017 > URL: https://issues.apache.org/jira/browse/FLINK-25017 > Project: Flink > Issue Type: Improvement > Components: BuildSystem / Shaded >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Fix For: shaded-15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25017) Drop Zookeeper 3.4
Chesnay Schepler created FLINK-25017: Summary: Drop Zookeeper 3.4 Key: FLINK-25017 URL: https://issues.apache.org/jira/browse/FLINK-25017 Project: Flink Issue Type: Improvement Components: BuildSystem / Shaded Reporter: Chesnay Schepler Assignee: Chesnay Schepler Fix For: shaded-15.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink-benchmarks] Aitozi commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
Aitozi commented on a change in pull request #41:
URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755120413
##
File path:
src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java
##
@@ -52,10 +54,25 @@
final ThreadLocalRandom random = ThreadLocalRandom.current();
@Param({"HEAP", "ROCKSDB"})
-protected StateBackendBenchmarkUtils.StateBackendType backendType;
+private StateBackendBenchmarkUtils.StateBackendType backendType;
+
+@Param({StateBenchmarkConstants.defaultStateDataDirPath})
+private String stateDataDirPath;
Review comment:
I didn't notice it. Now I prefer to not use `@Param` for this . Instead
of using environmental variable, Can we add a benchmark conf as I mentioned in
[FLINK-24967](https://issues.apache.org/jira/projects/FLINK/issues/FLINK-24967)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #17607: [FLINK-24543][runtime] Avoid possible inconsistencies of ZookeeperSta…
dmvk commented on a change in pull request #17607: URL: https://github.com/apache/flink/pull/17607#discussion_r755120267 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java ## @@ -87,18 +88,19 @@ private static final Logger LOG = LoggerFactory.getLogger(ZooKeeperStateHandleStore.class); -@VisibleForTesting -static final Set> PRE_COMMIT_EXCEPTIONS = -newHashSet( -KeeperException.NodeExistsException.class, -KeeperException.BadArgumentsException.class, -KeeperException.NoNodeException.class, -KeeperException.NoAuthException.class, -KeeperException.BadVersionException.class, -KeeperException.AuthFailedException.class, -KeeperException.InvalidACLException.class, -KeeperException.SessionMovedException.class, -KeeperException.NotReadOnlyException.class); +/** Pre-commit exceptions that don't imply data inconsistency. */ +private static final Set> SAFE_PRE_COMMIT_EXCEPTIONS = +new HashSet<>( +Arrays.asList( +KeeperException.NodeExistsException.class, +KeeperException.BadArgumentsException.class, +KeeperException.NoNodeException.class, +KeeperException.NoAuthException.class, +KeeperException.BadVersionException.class, +KeeperException.AuthFailedException.class, +KeeperException.InvalidACLException.class, +KeeperException.SessionMovedException.class, +KeeperException.NotReadOnlyException.class)); Review comment: One more suggestion, let's merge it as is and we should follow-up in different PR. It's still a big improvement from the original state and we'd need the patch for the transaction code path anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #17607: [FLINK-24543][runtime] Avoid possible inconsistencies of ZookeeperSta…
dmvk commented on a change in pull request #17607: URL: https://github.com/apache/flink/pull/17607#discussion_r755118309 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java ## @@ -87,18 +88,19 @@ private static final Logger LOG = LoggerFactory.getLogger(ZooKeeperStateHandleStore.class); -@VisibleForTesting -static final Set> PRE_COMMIT_EXCEPTIONS = -newHashSet( -KeeperException.NodeExistsException.class, -KeeperException.BadArgumentsException.class, -KeeperException.NoNodeException.class, -KeeperException.NoAuthException.class, -KeeperException.BadVersionException.class, -KeeperException.AuthFailedException.class, -KeeperException.InvalidACLException.class, -KeeperException.SessionMovedException.class, -KeeperException.NotReadOnlyException.class); +/** Pre-commit exceptions that don't imply data inconsistency. */ +private static final Set> SAFE_PRE_COMMIT_EXCEPTIONS = +new HashSet<>( +Arrays.asList( +KeeperException.NodeExistsException.class, +KeeperException.BadArgumentsException.class, +KeeperException.NoNodeException.class, +KeeperException.NoAuthException.class, +KeeperException.BadVersionException.class, +KeeperException.AuthFailedException.class, +KeeperException.InvalidACLException.class, +KeeperException.SessionMovedException.class, +KeeperException.NotReadOnlyException.class)); Review comment: I'm just desperate here... I'd say except `NoNodeException` and `BadArgumentsException` all of them are "potentially" unsafe if preceded by connection loss. I'd be in favor of removing this list completely as it's really hard to reason about. I'd even suggest reverting https://issues.apache.org/jira/browse/FLINK-22494 and coming up with a different solution. (discarding checkpoint & validating metadata before recovery) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-24990) LookupJoinITCase fails on Java 17
[
https://issues.apache.org/jira/browse/FLINK-24990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-24990.
Resolution: Fixed
master: f4b2153b82b3b0dc1c65fa33550a23380e4185b6
> LookupJoinITCase fails on Java 17
> -
>
> Key: FLINK-24990
> URL: https://issues.apache.org/jira/browse/FLINK-24990
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner, Tests
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> The UserDefinedFunctionHelper validates that a given function is public.
> The {{InMemory[Async]LookupFunction}} classes are not public however.
> Probably some Java<->Scala interplay that causes this to not be detected at
> this time.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-25005) Add java17-target profile
[ https://issues.apache.org/jira/browse/FLINK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-25005. Resolution: Fixed master: 63707c862b33e4da5c7e5b600361c5bb30078738 > Add java17-target profile > - > > Key: FLINK-25005 > URL: https://issues.apache.org/jira/browse/FLINK-25005 > Project: Flink > Issue Type: Sub-task > Components: Build System >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > Add a new profile analogous to the java11-target profile. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] zentol merged pull request #17869: [FLINK-25005][build] Add java17-target profile
zentol merged pull request #17869: URL: https://github.com/apache/flink/pull/17869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol merged pull request #17862: [FLINK-24990][tests] Make InMemory[Async]LookupFunction public
zentol merged pull request #17862: URL: https://github.com/apache/flink/pull/17862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-benchmarks] Aitozi commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
Aitozi commented on a change in pull request #41: URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755114500 ## File path: src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java ## @@ -20,15 +20,17 @@ import org.apache.flink.benchmark.BenchmarkBase; import org.apache.flink.contrib.streaming.state.benchmark.StateBackendBenchmarkUtils; import org.apache.flink.runtime.state.KeyedStateBackend; - Review comment: updated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-benchmarks] Aitozi commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
Aitozi commented on a change in pull request #41: URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755114314 ## File path: src/main/java/org/apache/flink/state/benchmark/ListStateBenchmark.java ## @@ -26,6 +26,7 @@ import org.openjdk.jmh.annotations.Level; import org.openjdk.jmh.annotations.Setup; import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.infra.BenchmarkParams; Review comment: removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] nicoweidner commented on pull request #17862: [FLINK-24990][tests] Make InMemory[Async]LookupFunction public
nicoweidner commented on pull request #17862: URL: https://github.com/apache/flink/pull/17862#issuecomment-976519118 > I can give you a branch though where this is setup if you want. It's fine, I just looked up the mentioned code (UserDefinedFunctionHelper). I just thought at first I would simply try it out to verify, which didn't quite work -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17653: FLINK SQL checkpoint不生效
flinkbot edited a comment on pull request #17653: URL: https://github.com/apache/flink/pull/17653#issuecomment-958686553 ## CI report: * 29429eaf63b982ce386614cb90caeb70697cefb2 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26907) * a2e241da13d788d165fe7f106b38f707a1cbf79f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26918) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) * 87ea7d97858a671be194c5155decb275b5443792 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26940) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #17607: [FLINK-24543][runtime] Avoid possible inconsistencies of ZookeeperSta…
dmvk commented on a change in pull request #17607: URL: https://github.com/apache/flink/pull/17607#discussion_r755105408 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java ## @@ -87,18 +88,19 @@ private static final Logger LOG = LoggerFactory.getLogger(ZooKeeperStateHandleStore.class); -@VisibleForTesting -static final Set> PRE_COMMIT_EXCEPTIONS = -newHashSet( -KeeperException.NodeExistsException.class, -KeeperException.BadArgumentsException.class, -KeeperException.NoNodeException.class, -KeeperException.NoAuthException.class, -KeeperException.BadVersionException.class, -KeeperException.AuthFailedException.class, -KeeperException.InvalidACLException.class, -KeeperException.SessionMovedException.class, -KeeperException.NotReadOnlyException.class); +/** Pre-commit exceptions that don't imply data inconsistency. */ +private static final Set> SAFE_PRE_COMMIT_EXCEPTIONS = +new HashSet<>( +Arrays.asList( +KeeperException.NodeExistsException.class, Review comment: Replace should never throw this exception (as it works only when node exists). After this change neither should `addAndLock`, so I'd say it should be safe to remove it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #17728: [FLINK-24832][tests][testinfrastructure] Update JUnit5 to v5.8.1
MartijnVisser commented on pull request #17728: URL: https://github.com/apache/flink/pull/17728#issuecomment-976504505 @AHeise Please merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) * 87ea7d97858a671be194c5155decb275b5443792 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-benchmarks] pnowojski commented on a change in pull request #41: [FLINK-24918][Runtime/State Backends]Support to specify the data dir for state benchmark
pnowojski commented on a change in pull request #41:
URL: https://github.com/apache/flink-benchmarks/pull/41#discussion_r755090811
##
File path: README.md
##
@@ -31,14 +31,33 @@ There're mainly two ways:
mvn -Dflink.version= clean package exec:exec \
-Dbenchmarks=""
```
-An example flink version can be -Dflink.version=1.12-SNAPSHOT.
+
+An example flink version can be -Dflink.version=1.12-SNAPSHOT.
+
+3. Run the uber jar directly like:
+
+```
+java -jar target/benchmarks.jar -rf csv ""
+```
We also support to run each benchmark once (with only one fork and one
iteration) for testing, with below command:
```
mvn test -P test
```
+## Parameters
+
+There are some built-in params to run different benchmarks, these can be
shown/overridden from the command line.
Review comment:
Missed `params`
##
File path:
src/main/java/org/apache/flink/state/benchmark/ListStateBenchmark.java
##
@@ -26,6 +26,7 @@
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.TearDown;
+import org.openjdk.jmh.infra.BenchmarkParams;
Review comment:
Why is this needed?
##
File path:
src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java
##
@@ -20,15 +20,17 @@
import org.apache.flink.benchmark.BenchmarkBase;
import
org.apache.flink.contrib.streaming.state.benchmark.StateBackendBenchmarkUtils;
import org.apache.flink.runtime.state.KeyedStateBackend;
-
Review comment:
Are you using the same formatting settings as for the main Flink
repository?
https://nightlies.apache.org/flink/flink-docs-master/docs/flinkdev/ide_setup/#code-formatting
##
File path: README.md
##
@@ -31,14 +31,33 @@ There're mainly two ways:
mvn -Dflink.version= clean package exec:exec \
-Dbenchmarks=""
```
-An example flink version can be -Dflink.version=1.12-SNAPSHOT.
+
+An example flink version can be -Dflink.version=1.12-SNAPSHOT.
+
+3. Run the uber jar directly like:
+
+```
+java -jar target/benchmarks.jar -rf csv ""
+```
We also support to run each benchmark once (with only one fork and one
iteration) for testing, with below command:
```
mvn test -P test
```
+## Parameters
+
+There are some built-in params to run different benchmarks, these can be
shown/overridden from the command line.
+
+```
+# show all the params combination for the
Review comment:
Missed `params`
##
File path:
src/main/java/org/apache/flink/state/benchmark/StateBenchmarkBase.java
##
@@ -52,10 +54,25 @@
final ThreadLocalRandom random = ThreadLocalRandom.current();
@Param({"HEAP", "ROCKSDB"})
-protected StateBackendBenchmarkUtils.StateBackendType backendType;
+private StateBackendBenchmarkUtils.StateBackendType backendType;
+
+@Param({StateBenchmarkConstants.defaultStateDataDirPath})
+private String stateDataDirPath;
Review comment:
Won't this in combination with
https://github.com/apache/flink-benchmarks/blob/master/save_jmh_result.py#L72:L76
change the names of the uploaded benchmarks to the [code
speed](http://codespeed.dak8s.net:8000/changes/)?
Do we really want this to be reported like this? Won't this clog the results
and the WebUI in particular?
Secondly, even if we want this, we would have to manually rename all of the
benchmarks in the database to preserve continuity of the results in the graphs.
Maybe it's actually better idea after all, to not use jmh's `@Param` for
this? Maybe simply use an environmental variable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17754: [FLINK-24861][connector][jdbc] Fix false cache lookup for empty data
flinkbot edited a comment on pull request #17754: URL: https://github.com/apache/flink/pull/17754#issuecomment-965189525 ## CI report: * b3703f43ebbe0c33d7ca3a607b7cd6fb49169e5b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26584) * 5c617cbc1802c0dd3c061bc84fae49de524e7584 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26928) * 24c7450ef162ec4530af791b1ef490cd2fe0f201 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26938) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] dmvk commented on a change in pull request #17607: [FLINK-24543][runtime] Avoid possible inconsistencies of ZookeeperSta…
dmvk commented on a change in pull request #17607:
URL: https://github.com/apache/flink/pull/17607#discussion_r755095766
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java
##
@@ -87,18 +88,19 @@
private static final Logger LOG =
LoggerFactory.getLogger(ZooKeeperStateHandleStore.class);
-@VisibleForTesting
-static final Set> PRE_COMMIT_EXCEPTIONS =
-newHashSet(
-KeeperException.NodeExistsException.class,
-KeeperException.BadArgumentsException.class,
-KeeperException.NoNodeException.class,
-KeeperException.NoAuthException.class,
-KeeperException.BadVersionException.class,
-KeeperException.AuthFailedException.class,
-KeeperException.InvalidACLException.class,
-KeeperException.SessionMovedException.class,
-KeeperException.NotReadOnlyException.class);
+/** Pre-commit exceptions that don't imply data inconsistency. */
+private static final Set>
SAFE_PRE_COMMIT_EXCEPTIONS =
+new HashSet<>(
+Arrays.asList(
+KeeperException.NodeExistsException.class,
+KeeperException.BadArgumentsException.class,
+KeeperException.NoNodeException.class,
+KeeperException.NoAuthException.class,
+KeeperException.BadVersionException.class,
+KeeperException.AuthFailedException.class,
+KeeperException.InvalidACLException.class,
+KeeperException.SessionMovedException.class,
+KeeperException.NotReadOnlyException.class));
Review comment:
I'd say in very rare conditions it could happen (if connection loss
happens right before retries are exhausted). We default to 3 retries with
exponential backoff.
We could also argue that this could happen `AuthFailed` /
`InvalidACLException` when there is some manual operation on the broker side.
Eg. sys-admin changes the ACL
Description of `SessionMovedException`
```
SESSIONMOVED
Session moved to another server, so operation is ignored
```
From `RetryLoop`:
```
public static boolean shouldRetry(int rc) {
return rc == Code.CONNECTIONLOSS.intValue() || rc ==
Code.OPERATIONTIMEOUT.intValue() || rc == Code.SESSIONMOVED.intValue() || rc ==
Code.SESSIONEXPIRED.intValue() || rc == -13;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) * 87ea7d97858a671be194c5155decb275b5443792 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] zhipeng93 commented on pull request #28: [Flink-24556] Add Estimator and Transformer for logistic regression
zhipeng93 commented on pull request #28: URL: https://github.com/apache/flink-ml/pull/28#issuecomment-976486651 Hi@lindong28 , thanks for the review and I have address your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-24989) Upgrade shade-plugin to 3.2.4
[ https://issues.apache.org/jira/browse/FLINK-24989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-24989. Resolution: Fixed master: a26cd47c1a1332ded2ecdeb6cabca7a3e589f4e4 > Upgrade shade-plugin to 3.2.4 > - > > Key: FLINK-24989 > URL: https://issues.apache.org/jira/browse/FLINK-24989 > Project: Flink > Issue Type: Sub-task > Components: Build System >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (FLINK-24988) Upgrade lombok to 1.18.22
[ https://issues.apache.org/jira/browse/FLINK-24988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-24988. Resolution: Fixed master: b485560c1129a303eab8655e97d56d7bc8e8bfe7 > Upgrade lombok to 1.18.22 > - > > Key: FLINK-24988 > URL: https://issues.apache.org/jira/browse/FLINK-24988 > Project: Flink > Issue Type: Sub-task > Components: Tests >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > Our current Lombok version fails on Java 17 due to illegal accesses to JDK > internals. Newer versions of Lombok do some hacks at runtime to resolve the > issue... -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] zentol merged pull request #17860: [FLINK-24988][build] Upgrade lombok to 1.18.22
zentol merged pull request #17860: URL: https://github.com/apache/flink/pull/17860 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17754: [FLINK-24861][connector][jdbc] Fix false cache lookup for empty data
flinkbot edited a comment on pull request #17754: URL: https://github.com/apache/flink/pull/17754#issuecomment-965189525 ## CI report: * b3703f43ebbe0c33d7ca3a607b7cd6fb49169e5b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26584) * 5c617cbc1802c0dd3c061bc84fae49de524e7584 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26928) * 24c7450ef162ec4530af791b1ef490cd2fe0f201 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol merged pull request #17861: [FLINK-24989][build] Bump shade-plugin to 3.2.4
zentol merged pull request #17861: URL: https://github.com/apache/flink/pull/17861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] zhipeng93 commented on a change in pull request #28: [Flink-24556] Add Estimator and Transformer for logistic regression
zhipeng93 commented on a change in pull request #28:
URL: https://github.com/apache/flink-ml/pull/28#discussion_r755088231
##
File path:
flink-ml-lib/src/main/java/org/apache/flink/ml/common/param/HasL2.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.ml.common.param;
+
+import org.apache.flink.ml.param.DoubleParam;
+import org.apache.flink.ml.param.Param;
+import org.apache.flink.ml.param.ParamValidators;
+import org.apache.flink.ml.param.WithParams;
+
+/** Interface for the shared L2 regularization param. */
+public interface HasL2 extends WithParams {
Review comment:
Good observation! I prefer using `HasReg` and `HasElasticNet` to
spefcify the three modes, similar as Spark.
I have renamed `HasL2` as `HasReg`. Given that we are not using
`HasElasticNet` in LogisticRegression, we can add this later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] XComp commented on pull request #17858: [FLINK-24986][tests] Replace anonymous StreamStateHandle class
XComp commented on pull request #17858: URL: https://github.com/apache/flink/pull/17858#issuecomment-976480406 The test failure is related to [FLINK-23525](https://issues.apache.org/jira/browse/FLINK-23525) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17847: [FLINK-6573][Connectors/Common ] Flink MongoDB Connector
flinkbot edited a comment on pull request #17847: URL: https://github.com/apache/flink/pull/17847#issuecomment-974817636 ## CI report: * 08d4751f06377557fc6a8b2fc9db6008f17bd23d Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26924) * 8364104af21c6c012779149008980072b18cb87c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26933) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] zhipeng93 commented on a change in pull request #28: [Flink-24556] Add Estimator and Transformer for logistic regression
zhipeng93 commented on a change in pull request #28:
URL: https://github.com/apache/flink-ml/pull/28#discussion_r755088231
##
File path:
flink-ml-lib/src/main/java/org/apache/flink/ml/common/param/HasL2.java
##
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.ml.common.param;
+
+import org.apache.flink.ml.param.DoubleParam;
+import org.apache.flink.ml.param.Param;
+import org.apache.flink.ml.param.ParamValidators;
+import org.apache.flink.ml.param.WithParams;
+
+/** Interface for the shared L2 regularization param. */
+public interface HasL2 extends WithParams {
Review comment:
Good observation! I prefer using `HasReg` and `HasElasticNet` to
spefcify the three modes, similar as Spark.
I have renamed `HasL2` as HasReg`. Given that we are not using
`HasElasticNet` in LogisticRegression, we can add this later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors/kinesis] Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * e0d7451523be3976f3c8695a0f3c9c782a76fddf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26865) * c5fdc0470d9f96ed3aaa859c304ff2e78319f2bc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26932) * 87ea7d97858a671be194c5155decb275b5443792 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
