[GitHub] [flink] flinkbot edited a comment on pull request #14738: [FLINK-20947][planner] Fix idle source doesn't work when pushing watermark into the source
flinkbot edited a comment on pull request #14738: URL: https://github.com/apache/flink/pull/14738#issuecomment-766300412 ## CI report: * ee016150b9a4dce6af1f3e060c4c0708fcfb0940 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12418) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
flinkbot edited a comment on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766281483 ## CI report: * 399f06e14079a512d35814508b6f7598d7d175ba Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12417) * bd617892bfec1db1654606355041a6e4b9050304 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14738: [FLINK-20947][planner] Fix idle source doesn't work when pushing watermark into the source
flinkbot edited a comment on pull request #14738: URL: https://github.com/apache/flink/pull/14738#issuecomment-766300412 ## CI report: * ee016150b9a4dce6af1f3e060c4c0708fcfb0940 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12418) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14738: [FLINK-20947][planner] Fix idle source doesn't work when pushing watermark into the source
flinkbot commented on pull request #14738: URL: https://github.com/apache/flink/pull/14738#issuecomment-766300412 ## CI report: * ee016150b9a4dce6af1f3e060c4c0708fcfb0940 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14738: [FLINK-20947][planner] Fix idle source doesn't work when pushing watermark into the source
flinkbot commented on pull request #14738: URL: https://github.com/apache/flink/pull/14738#issuecomment-766299296 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ee016150b9a4dce6af1f3e060c4c0708fcfb0940 (Sun Jan 24 06:19:38 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 opened a new pull request #14738: [FLINK-20947][planner] Fix idle source doesn't work when pushing watermark into the source
fsk119 opened a new pull request #14738: URL: https://github.com/apache/flink/pull/14738 ## What is the purpose of the change *Fix idle source doesn't work when pushing watermark strategy into table source.* ## Brief change log - *Fix the bug.* ## Verifying this change This change added tests and can be verified as follows: - *Idle source plan test.* - *Idle source IT cases.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21107) Move test base classes from flink-tests to flink-test-utils
[ https://issues.apache.org/jira/browse/FLINK-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenqiu Huang closed FLINK-21107. - Resolution: Won't Fix > Move test base classes from flink-tests to flink-test-utils > --- > > Key: FLINK-21107 > URL: https://issues.apache.org/jira/browse/FLINK-21107 > Project: Flink > Issue Type: Improvement > Components: Tests >Affects Versions: 1.12.0 >Reporter: Zhenqiu Huang >Priority: Minor > > There are several test help classes are defined in flink-tests and used in > modules in flink-streaming-scala, flink-table-planner-blink: > For example: > org.apache.flink.test.streaming.runtime.util.TestListResultSink > org.apache.flink.api.scala.completeness.ScalaAPICompletenessTestBase > org.apache.flink.test.util.SuccessException > But actually, flink-tests module is a place for ITCases. If we want to add > ITCase for classes (for example remote udf) of flink-table-planner-blink into > flink-tests, the cycle reference will be created. > Thus, I would suggest moving those helper classes from flink-tests to > flink-test-utils. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21107) Move test base classes from flink-tests to flink-test-utils
Zhenqiu Huang created FLINK-21107: - Summary: Move test base classes from flink-tests to flink-test-utils Key: FLINK-21107 URL: https://issues.apache.org/jira/browse/FLINK-21107 Project: Flink Issue Type: Improvement Components: Tests Affects Versions: 1.12.0 Reporter: Zhenqiu Huang There are several test help classes are defined in flink-tests and used in modules in flink-streaming-scala, flink-table-planner-blink: For example: org.apache.flink.test.streaming.runtime.util.TestListResultSink org.apache.flink.api.scala.completeness.ScalaAPICompletenessTestBase org.apache.flink.test.util.SuccessException But actually, flink-tests module is a place for ITCases. If we want to add ITCase for classes (for example remote udf) of flink-table-planner-blink into flink-tests, the cycle reference will be created. Thus, I would suggest moving those helper classes from flink-tests to flink-test-utils. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink-shaded] HuangXingBo edited a comment on pull request #93: [FLINK-21020][jackson] Bump version to 2.12.1
HuangXingBo edited a comment on pull request #93: URL: https://github.com/apache/flink-shaded/pull/93#issuecomment-765901405 I have tested this flink-shaded version with flink (master version and release-1.12) in my private Azure. the test commit in master: https://github.com/HuangXingBo/flink/commit/554cecd092594e155a7b0b140b675fe0b47fe72d the test result in master: https://dev.azure.com/hxbks2ks/FLINK-TEST/_build/results?buildId=808&view=results the test result in release-1.12: https://dev.azure.com/hxbks2ks/FLINK-TEST/_build/results?buildId=810&view=results This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
flinkbot edited a comment on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766281483 ## CI report: * 399f06e14079a512d35814508b6f7598d7d175ba Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12417) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
flinkbot edited a comment on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766281483 ## CI report: * 399f06e14079a512d35814508b6f7598d7d175ba Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12417) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
flinkbot commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766281483 ## CI report: * 399f06e14079a512d35814508b6f7598d7d175ba UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong commented on pull request #14710: [FLINK-19393][docs-zh] Translate the 'SQL Hints' page of 'Table API & SQL' into Chinese
wuchong commented on pull request #14710: URL: https://github.com/apache/flink/pull/14710#issuecomment-766280734 cc @fsk119 , could you help to review this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
wuchong commented on a change in pull request #14735:
URL: https://github.com/apache/flink/pull/14735#discussion_r563227615
##
File path:
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/planner/runtime/stream/table/TimeAttributesITCase.scala
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.runtime.stream.table
+
+import org.apache.flink.api.common.eventtime.{SerializableTimestampAssigner,
WatermarkStrategy}
+import org.apache.flink.api.scala.createTypeInformation
+import org.apache.flink.runtime.client.JobExecutionException
+import org.apache.flink.table.api._
+import
org.apache.flink.table.planner.runtime.utils.StreamingWithStateTestBase.StateBackendMode
+import
org.apache.flink.table.planner.runtime.utils.{StreamingWithStateTestBase,
TestingAppendSink}
+import org.apache.flink.types.Row
+import org.junit.Assert.{assertEquals, fail}
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.Parameterized
+
+import java.time.{Duration, Instant, LocalDateTime, ZoneOffset}
+
+@RunWith(classOf[Parameterized])
+class TimeAttributesITCase(mode: StateBackendMode) extends
StreamingWithStateTestBase(mode) {
+
+ @Test
+ def testMissingTimeAttributeThrowsCorrectException(): Unit = {
+val data = List(1L -> "hello", 2L -> "world")
+val stream = env.fromCollection[(Long, String)](data)
+
+tEnv.createTemporaryView("test", stream, $"event_time".rowtime(), $"data")
+val result = tEnv.sqlQuery("SELECT * FROM test")
+
+val sink = new TestingAppendSink()
+tEnv.toAppendStream[Row](result).addSink(sink)
+try {
+ env.execute()
+} catch {
+ case je: JobExecutionException =>
+val innerCause = je.getCause.getCause
+assert(innerCause.isInstanceOf[RuntimeException])
+assertEquals(
+ "Rowtime timestamp is not defined. Please make sure that a " +
+"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
+"time characteristic.",
+ innerCause.getMessage)
+ case e: Exception => fail(s"Expected JobExecutionException, received $e")
Review comment:
Can be simplify to
```scala
case t: Throwable =>
assertThat(
t,
containsMessage("Rowtime timestamp is not defined. Please make
sure that a " +
"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
"time characteristic."))
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
wuchong commented on a change in pull request #14735:
URL: https://github.com/apache/flink/pull/14735#discussion_r563227674
##
File path:
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/planner/runtime/stream/table/TimeAttributesITCase.scala
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.runtime.stream.table
+
+import org.apache.flink.api.common.eventtime.{SerializableTimestampAssigner,
WatermarkStrategy}
+import org.apache.flink.api.scala.createTypeInformation
+import org.apache.flink.runtime.client.JobExecutionException
+import org.apache.flink.table.api._
+import
org.apache.flink.table.planner.runtime.utils.StreamingWithStateTestBase.StateBackendMode
+import
org.apache.flink.table.planner.runtime.utils.{StreamingWithStateTestBase,
TestingAppendSink}
+import org.apache.flink.types.Row
+import org.junit.Assert.{assertEquals, fail}
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.Parameterized
+
+import java.time.{Duration, Instant, LocalDateTime, ZoneOffset}
+
+@RunWith(classOf[Parameterized])
+class TimeAttributesITCase(mode: StateBackendMode) extends
StreamingWithStateTestBase(mode) {
+
+ @Test
+ def testMissingTimeAttributeThrowsCorrectException(): Unit = {
+val data = List(1L -> "hello", 2L -> "world")
+val stream = env.fromCollection[(Long, String)](data)
+
+tEnv.createTemporaryView("test", stream, $"event_time".rowtime(), $"data")
+val result = tEnv.sqlQuery("SELECT * FROM test")
+
+val sink = new TestingAppendSink()
+tEnv.toAppendStream[Row](result).addSink(sink)
+try {
+ env.execute()
Review comment:
Should add `fail("should fail")` to make sure there is a runtime
exception when running the query.
##
File path:
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/planner/runtime/stream/table/TimeAttributesITCase.scala
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.runtime.stream.table
+
+import org.apache.flink.api.common.eventtime.{SerializableTimestampAssigner,
WatermarkStrategy}
+import org.apache.flink.api.scala.createTypeInformation
+import org.apache.flink.runtime.client.JobExecutionException
+import org.apache.flink.table.api._
+import
org.apache.flink.table.planner.runtime.utils.StreamingWithStateTestBase.StateBackendMode
+import
org.apache.flink.table.planner.runtime.utils.{StreamingWithStateTestBase,
TestingAppendSink}
+import org.apache.flink.types.Row
+import org.junit.Assert.{assertEquals, fail}
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.junit.runners.Parameterized
+
+import java.time.{Duration, Instant, LocalDateTime, ZoneOffset}
+
+@RunWith(classOf[Parameterized])
+class TimeAttributesITCase(mode: StateBackendMode) extends
StreamingWithStateTestBase(mode) {
+
+ @Test
+ def testMissingTimeAttributeThrowsCorrectException(): Unit = {
+val data = List(1L -> "hello", 2L -> "world")
+val stream = env.fromCollection[(Long, String)](data)
+
+tEnv.createTemporaryView("test", stream, $"event_time".rowtime(), $"data")
+val result = tEnv.sqlQuery("SELECT * FROM test")
+
+val sink = new TestingAppendSink()
+tEnv.toAppendStream[Row](result).addSink(sink)
+try {
+ env.execute()
+} catch {
+ case je: JobExecutionE
[GitHub] [flink] flinkbot commented on pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
flinkbot commented on pull request #14737: URL: https://github.com/apache/flink/pull/14737#issuecomment-766279661 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 399f06e14079a512d35814508b6f7598d7d175ba (Sun Jan 24 02:36:14 UTC 2021) **Warnings:** * **4 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] mohitpali opened a new pull request #14737: [FLINK-19667] Add AWS Glue Schema Registry integration
mohitpali opened a new pull request #14737: URL: https://github.com/apache/flink/pull/14737 ## What is the purpose of the change _The AWS Glue Schema Registry is a new feature of AWS Glue that allows you to centrally discover, control, and evolve data stream schemas. This request is to add a new format to launch an integration for Apache Flink with AWS Glue Schema Registry._ ## Brief change log * _Added `flink-avro-glue-schema-registry` module under `flink-formats`_ * _Added end-to-end test named `flink-glue-schema-registry-test` for the new module_ ## Verifying this change This change added tests and can be verified as follows: * _Added integration tests for end-to-end deployment_ ## Does this pull request potentially affect one of the following parts: * Dependencies (does it add or upgrade a dependency): (yes) * The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) * The serializers: (yes) * The runtime per-record code paths (performance sensitive): (don't know) * Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no) * The S3 file system connector: (no) ## Documentation * Does this pull request introduce a new feature? (yes) * If yes, how is the feature documented? (JavaDocs) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] RocMarshal commented on pull request #14710: [FLINK-19393][docs-zh] Translate the 'SQL Hints' page of 'Table API & SQL' into Chinese
RocMarshal commented on pull request #14710: URL: https://github.com/apache/flink/pull/14710#issuecomment-766276114 @wuchong Could you help me to review this PR? Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14736: [FLINK-21104][network] Ensure that converted input channels start in the correct persisting state.
flinkbot edited a comment on pull request #14736: URL: https://github.com/apache/flink/pull/14736#issuecomment-766158250 ## CI report: * ea16029a49bbd0716dd2f0c4c4d8fc43e957c316 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12413) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21106) google-java-format Intellij Plugin 1.7.0.5 causes UnsupportedOperationException in IntelliJ
[
https://issues.apache.org/jira/browse/FLINK-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-21106:
-
Description:
There's a problem with {{google-java-format}} Intellij plugin version
{{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
new Java class file. Besides the exception, an error dialog pops up and the
newly created file is not properly formatted. A simple reformat solves the
issue.
This problem is caused by a bug that got fixed in the {{google-java-format}}
plugin's codebase in
[45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
Unfortunately, this fix got released with the plugin version {{1.8.0.1}} which
we cannot upgrade to due to our limitations on sticking to Java 8 for now (see
FLINK-20803).
was:
There's a problem with {{google-java-format}} Intellij plugin version
{{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
new Java class file. Besides the exception, an error dialog pops up and the
newly created file is not properly formatted. A simple reformat solves the
issue.
This problem is caused by a bug that got fixed in the {{google-java-format}}
plugin's codebase in
[45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
Unfortunately, this fix got released with the plugin version {{1.8.0.1}} which
we cannot upgrade to due to our limitations on sticking to Java 8 for now.
> google-java-format Intellij Plugin 1.7.0.5 causes
> UnsupportedOperationException in IntelliJ
> ---
>
> Key: FLINK-21106
> URL: https://issues.apache.org/jira/browse/FLINK-21106
> Project: Flink
> Issue Type: Bug
>Reporter: Matthias
>Priority: Minor
> Attachments: google-java-format.zip
>
>
> There's a problem with {{google-java-format}} Intellij plugin version
> {{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
> new Java class file. Besides the exception, an error dialog pops up and the
> newly created file is not properly formatted. A simple reformat solves the
> issue.
> This problem is caused by a bug that got fixed in the {{google-java-format}}
> plugin's codebase in
> [45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
> Unfortunately, this fix got released with the plugin version {{1.8.0.1}}
> which we cannot upgrade to due to our limitations on sticking to Java 8 for
> now (see FLINK-20803).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-21104) UnalignedCheckpointITCase.execute failed with "IllegalStateException"
[
https://issues.apache.org/jira/browse/FLINK-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arvid Heise reassigned FLINK-21104:
---
Assignee: Arvid Heise
> UnalignedCheckpointITCase.execute failed with "IllegalStateException"
> -
>
> Key: FLINK-21104
> URL: https://issues.apache.org/jira/browse/FLINK-21104
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.13.0, 1.12.2
>Reporter: Huang Xingbo
>Assignee: Arvid Heise
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.13.0, 1.12.2
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12383&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=f508e270-48d6-5f1e-3138-42a17e0714f0]
> {code:java}
> 2021-01-22T15:17:34.6711152Z [ERROR] execute[Parallel union, p =
> 10](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time
> elapsed: 3.903 s <<< ERROR!
> 2021-01-22T15:17:34.6711736Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-01-22T15:17:34.6712204Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-01-22T15:17:34.6712779Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-01-22T15:17:34.6713344Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-01-22T15:17:34.6713816Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-01-22T15:17:34.6714454Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6714952Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6715472Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238)
> 2021-01-22T15:17:34.6716026Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-01-22T15:17:34.6716631Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-01-22T15:17:34.6717128Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6717616Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6718105Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-01-22T15:17:34.6718596Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-01-22T15:17:34.6718973Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-01-22T15:17:34.6719364Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-01-22T15:17:34.6719748Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-01-22T15:17:34.6720155Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6720641Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-01-22T15:17:34.6721236Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-01-22T15:17:34.6721706Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-01-22T15:17:34.6722205Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-01-22T15:17:34.6722663Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-01-22T15:17:34.6723214Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-01-22T15:17:34.6723723Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-01-22T15:17:34.6724146Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-01-22T15:17:34.6724726Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6725198Z at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-01-22T15:17:34.6725861Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6726525Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727278Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727773Z at
> scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
> 2021-01-22T15:17:34.6728484Z at
> akka.dispatch.Batching
[jira] [Commented] (FLINK-21106) google-java-format Intellij Plugin 1.7.0.5 causes UnsupportedOperationException in IntelliJ
[
https://issues.apache.org/jira/browse/FLINK-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270736#comment-17270736
]
Matthias commented on FLINK-21106:
--
I worked around the issue for now by applying the patch below to the most
recent {{google-java-format}} plugin code and build the plugin from source (it
essentially reverts [(it essentially reverts
[c2c8a51|https://github.com/google/google-java-format/commit/c2c8a517c92fbf3553e53524057cd2ad8ae62962]):
{code:java}
diff --git a/idea_plugin/build.gradle b/idea_plugin/build.gradle
index 6ec1c21..32eb5a0 100644
--- a/idea_plugin/build.gradle
+++ b/idea_plugin/build.gradle
@@ -15,7 +15,7 @@
*/
plugins {
- id "org.jetbrains.intellij" version "0.4.22"
+ id "org.jetbrains.intellij" version "0.4.18"
}
repositories {
@@ -23,7 +23,7 @@ repositories {
}
ext {
- googleJavaFormatVersion = '1.9'
+ googleJavaFormatVersion = '1.7'
}
apply plugin: 'org.jetbrains.intellij'
@@ -37,7 +37,7 @@ intellij {
patchPluginXml {
pluginDescription = "Formats source code using the google-java-format tool.
This version of " +
"the plugin uses version ${googleJavaFormatVersion} of
the tool."
- version = "${googleJavaFormatVersion}.0.0"
+ version = "${googleJavaFormatVersion}.0.5"
sinceBuild = '201'
untilBuild = ''
}
{code}
You can find the patched plugin attached to this Jira issue.
[^google-java-format.zip]
> google-java-format Intellij Plugin 1.7.0.5 causes
> UnsupportedOperationException in IntelliJ
> ---
>
> Key: FLINK-21106
> URL: https://issues.apache.org/jira/browse/FLINK-21106
> Project: Flink
> Issue Type: Bug
>Reporter: Matthias
>Priority: Minor
> Attachments: google-java-format.zip
>
>
> There's a problem with {{google-java-format}} Intellij plugin version
> {{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
> new Java class file. Besides the exception, an error dialog pops up and the
> newly created file is not properly formatted. A simple reformat solves the
> issue.
> This problem is caused by a bug that got fixed in the {{google-java-format}}
> plugin's codebase in
> [45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
> Unfortunately, this fix got released with the plugin version {{1.8.0.1}}
> which we cannot upgrade to due to our limitations on sticking to Java 8 for
> now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21106) google-java-format Intellij Plugin 1.7.0.5 causes UnsupportedOperationException in IntelliJ
[
https://issues.apache.org/jira/browse/FLINK-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-21106:
-
Attachment: google-java-format.zip
> google-java-format Intellij Plugin 1.7.0.5 causes
> UnsupportedOperationException in IntelliJ
> ---
>
> Key: FLINK-21106
> URL: https://issues.apache.org/jira/browse/FLINK-21106
> Project: Flink
> Issue Type: Bug
>Reporter: Matthias
>Priority: Minor
> Attachments: google-java-format.zip
>
>
> There's a problem with {{google-java-format}} Intellij plugin version
> {{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
> new Java class file. Besides the exception, an error dialog pops up and the
> newly created file is not properly formatted. A simple reformat solves the
> issue.
> This problem is caused by a bug that got fixed in the {{google-java-format}}
> plugin's codebase in
> [45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
> Unfortunately, this fix got released with the plugin version {{1.8.0.1}}
> which we cannot upgrade to due to our limitations on sticking to Java 8 for
> now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-21106) google-java-format Intellij Plugin 1.7.0.5 causes UnsupportedOperationException in IntelliJ
Matthias created FLINK-21106:
Summary: google-java-format Intellij Plugin 1.7.0.5 causes
UnsupportedOperationException in IntelliJ
Key: FLINK-21106
URL: https://issues.apache.org/jira/browse/FLINK-21106
Project: Flink
Issue Type: Bug
Reporter: Matthias
There's a problem with {{google-java-format}} Intellij plugin version
{{1.7.0.5}} that causes an {{UnsupportedOperationException}} when creating a
new Java class file. Besides the exception, an error dialog pops up and the
newly created file is not properly formatted. A simple reformat solves the
issue.
This problem is caused by a bug that got fixed in the {{google-java-format}}
plugin's codebase in
[45fb41a|https://github.com/google/google-java-format/commit/45fb41a7bac3dfe0726601ceb87d1c17bbf494ec].
Unfortunately, this fix got released with the plugin version {{1.8.0.1}} which
we cannot upgrade to due to our limitations on sticking to Java 8 for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14736: [FLINK-21104][network] Ensure that converted input channels start in the correct persisting state.
flinkbot edited a comment on pull request #14736: URL: https://github.com/apache/flink/pull/14736#issuecomment-766158250 ## CI report: * ea16029a49bbd0716dd2f0c4c4d8fc43e957c316 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12413) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766094970 ## CI report: * 95137f07f2172150b444e9d3e4ce0b5cece22e2d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12412) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14734: [FLINK-21066][runtime][checkpoint] Refactor CheckpointCoordinator to compute tasks to trigger/ack/commit dynamically
flinkbot edited a comment on pull request #14734: URL: https://github.com/apache/flink/pull/14734#issuecomment-766092219 ## CI report: * 721dd38bb286722cbb1a6925dbedbb3614930b20 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12410) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14736: [FLINK-21104][network] Ensure that converted input channels start in the correct persisting state.
flinkbot commented on pull request #14736: URL: https://github.com/apache/flink/pull/14736#issuecomment-766158250 ## CI report: * ea16029a49bbd0716dd2f0c4c4d8fc43e957c316 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21104) UnalignedCheckpointITCase.execute failed with "IllegalStateException"
[
https://issues.apache.org/jira/browse/FLINK-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-21104:
---
Labels: pull-request-available test-stability (was: test-stability)
> UnalignedCheckpointITCase.execute failed with "IllegalStateException"
> -
>
> Key: FLINK-21104
> URL: https://issues.apache.org/jira/browse/FLINK-21104
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.13.0, 1.12.2
>Reporter: Huang Xingbo
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.13.0, 1.12.2
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12383&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=f508e270-48d6-5f1e-3138-42a17e0714f0]
> {code:java}
> 2021-01-22T15:17:34.6711152Z [ERROR] execute[Parallel union, p =
> 10](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time
> elapsed: 3.903 s <<< ERROR!
> 2021-01-22T15:17:34.6711736Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-01-22T15:17:34.6712204Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-01-22T15:17:34.6712779Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-01-22T15:17:34.6713344Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-01-22T15:17:34.6713816Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-01-22T15:17:34.6714454Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6714952Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6715472Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238)
> 2021-01-22T15:17:34.6716026Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-01-22T15:17:34.6716631Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-01-22T15:17:34.6717128Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6717616Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6718105Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-01-22T15:17:34.6718596Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-01-22T15:17:34.6718973Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-01-22T15:17:34.6719364Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-01-22T15:17:34.6719748Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-01-22T15:17:34.6720155Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6720641Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-01-22T15:17:34.6721236Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-01-22T15:17:34.6721706Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-01-22T15:17:34.6722205Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-01-22T15:17:34.6722663Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-01-22T15:17:34.6723214Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-01-22T15:17:34.6723723Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-01-22T15:17:34.6724146Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-01-22T15:17:34.6724726Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6725198Z at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-01-22T15:17:34.6725861Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6726525Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727278Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727773Z at
> scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
> 2021-01-22T15:17:34.6728484Z at
> akka.dispat
[GitHub] [flink] flinkbot commented on pull request #14736: [FLINK-21104][network] Ensure that converted input channels start in the correct persisting state.
flinkbot commented on pull request #14736: URL: https://github.com/apache/flink/pull/14736#issuecomment-766156866 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 491edd1384ed492c449404cc0556fc36a60d5a53 (Sat Jan 23 18:27:57 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] AHeise opened a new pull request #14736: [FLINK-20654][network] Ensure that converted input channels start in the correct persisting state.
AHeise opened a new pull request #14736: URL: https://github.com/apache/flink/pull/14736 ## What is the purpose of the change When a checkpoint is aborted, the ChannelStatePersister of an input channel is stopped for that checkpoint. A barrier that arrives after that is simply ignored. However, if a channel is still being recovered during the abortion, the information that the checkpoint is stopped is lost for that particular channel. ## Brief change log - This commit initializes converted channels with that state correctly. - Additionally, this commit includes test that helped to verify that the ChannelStatePersister is actually working correctly now. ## Verifying this change Stabilizes the UnalignedCheckpointITCase and added 2 unit tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (**yes** / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766094970 ## CI report: * 235d61be4bb5ed158c109a11de845f73c077c0fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12411) * 95137f07f2172150b444e9d3e4ce0b5cece22e2d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12412) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766094970 ## CI report: * 235d61be4bb5ed158c109a11de845f73c077c0fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12411) * 95137f07f2172150b444e9d3e4ce0b5cece22e2d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * de291068835c9b7540c3a0544c6dd589f0c3ada2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12408) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] AHeise merged pull request #414: Add a warning for corrupt unaligned checkpoints in 1.12 release blog post.
AHeise merged pull request #414: URL: https://github.com/apache/flink-web/pull/414 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766094970 ## CI report: * 235d61be4bb5ed158c109a11de845f73c077c0fe Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12411) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
flinkbot edited a comment on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-762224447 ## CI report: * f26efadf325bb52d3bc8b2711fc83a169ad61ca5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12409) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14734: [FLINK-21066][runtime][checkpoint] Refactor CheckpointCoordinator to compute tasks to trigger/ack/commit dynamically
flinkbot edited a comment on pull request #14734: URL: https://github.com/apache/flink/pull/14734#issuecomment-766092219 ## CI report: * 721dd38bb286722cbb1a6925dbedbb3614930b20 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12410) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot commented on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766094970 ## CI report: * 235d61be4bb5ed158c109a11de845f73c077c0fe UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
flinkbot edited a comment on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-762224447 ## CI report: * 5ddc226c2aa4fd063cc173a65eb887f9ac0247ba Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12192) * f26efadf325bb52d3bc8b2711fc83a169ad61ca5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12409) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot commented on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766092304 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 235d61be4bb5ed158c109a11de845f73c077c0fe (Sat Jan 23 15:00:49 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] anlen321 commented on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
anlen321 commented on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-766092286 Hi @leonardBang I have submitted the new commit, please help to review it.thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14734: [FLINK-21066][runtime][checkpoint] Refactor CheckpointCoordinator to compute tasks to trigger/ack/commit dynamically
flinkbot commented on pull request #14734: URL: https://github.com/apache/flink/pull/14734#issuecomment-766092219 ## CI report: * 721dd38bb286722cbb1a6925dbedbb3614930b20 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * ef5b18453605da53ca9c9ce0c7fce9b7361f67a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12406) * de291068835c9b7540c3a0544c6dd589f0c3ada2 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12408) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
flinkbot edited a comment on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-762224447 ## CI report: * 5ddc226c2aa4fd063cc173a65eb887f9ac0247ba Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12192) * f26efadf325bb52d3bc8b2711fc83a169ad61ca5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] anlen321 removed a comment on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
anlen321 removed a comment on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-766091231 > I'll take a look this PR, @wuchong . > > @anlen321 Looks like you fixed the conflicts with `git merge` command, it's recommended to use `git rebase master` command to fix the conflicts, could you move your change to single commit and then we will help review? Currently your two commits are messed up with others', it's hard to read. >  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] anlen321 commented on pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
anlen321 commented on pull request #14684: URL: https://github.com/apache/flink/pull/14684#issuecomment-766091231 > I'll take a look this PR, @wuchong . > > @anlen321 Looks like you fixed the conflicts with `git merge` command, it's recommended to use `git rebase master` command to fix the conflicts, could you move your change to single commit and then we will help review? Currently your two commits are messed up with others', it's hard to read. >  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] anlen321 closed pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
anlen321 closed pull request #14684: URL: https://github.com/apache/flink/pull/14684 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on pull request #14735: URL: https://github.com/apache/flink/pull/14735#issuecomment-766090546 @wuchong Please re-check. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov opened a new pull request #14735: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov opened a new pull request #14735: URL: https://github.com/apache/flink/pull/14735 Fix an erroneous NPE created when using a table from a DataStream that did not assign timestamps and watermarks ## Brief change log - Fix direct access to `ctx.timestamp()`, validate that it has a timestamp before ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: (Maybe, not sure) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-766089520 @wuchong I'm going to re-open this PR under a fresh branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov closed pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov closed pull request #14633: URL: https://github.com/apache/flink/pull/14633 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563159502
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -909,6 +908,62 @@ object GenerateUtils {
|}
""".stripMargin
compares += code
+})
+compares.mkString
+ }
+
+ /**
+ * Generates code for comparing row keys.
+ * TODO: Remove this after [[MultiFieldRangeBoundComparatorCodeGenerator]]
+ * is refactored using SortSpec.
+ */
+ def generateRowCompare(
Review comment:
@wuchong Did you rebase off master? Diffing master I see this was added.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14734: [FLINK-21066][runtime][checkpoint] Refactor CheckpointCoordinator to compute tasks to trigger/ack/commit dynamically
flinkbot commented on pull request #14734: URL: https://github.com/apache/flink/pull/14734#issuecomment-766088540 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 721dd38bb286722cbb1a6925dbedbb3614930b20 (Sat Jan 23 14:33:14 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21066) Refactor the CheckpointCoordinator to compute the tasks to trigger/wait/commit dynamically
[ https://issues.apache.org/jira/browse/FLINK-21066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-21066: --- Labels: pull-request-available (was: ) > Refactor the CheckpointCoordinator to compute the tasks to > trigger/wait/commit dynamically > -- > > Key: FLINK-21066 > URL: https://issues.apache.org/jira/browse/FLINK-21066 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream, Runtime / Checkpointing >Reporter: Yun Gao >Assignee: Yun Gao >Priority: Major > Labels: pull-request-available > > Currently Flink would identify the tasks to trigger/wait/commit in compile > phase, and pass them to the CheckpointCoordinator via JobGraph and cache them > in CheckpointCoordinator. However, to support checkpoint after some tasks > finished, we might need to recompute the tasks to trigger/wait/commit for > each checkpoint, thus we would first refactor CheckpointCoordinator to > compute the tasks to trigger/wait/commit dynamically, but keeps the computing > logic unchange (e.g., tasks to trigger are source tasks and tasks to > wait/commit are all tasks) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] gaoyunhaii opened a new pull request #14734: [FLINK-21066][runtime][checkpoint] Refactor CheckpointCoordinator to compute tasks to trigger/ack/commit dynamically
gaoyunhaii opened a new pull request #14734: URL: https://github.com/apache/flink/pull/14734 ## What is the purpose of the change This PR refactors the CheckpointCoordinator logic to compute the tasks to trigger/wait/commit dynamically. Previously it is done during compile phase and passed to CheckpointCoordinator via JobGraph. This is a premise for support considering finished tasks into checkpoint, since considering finished tasks would cause each checkpoint might need to trigger different tasks and have to re-compute at each checkpoint ## Brief change log - 721dd38bb286722cbb1a6925dbedbb3614930b20 did the refactor. ## Verifying this change This change is a code refactor and could be covered by existing tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * ef5b18453605da53ca9c9ce0c7fce9b7361f67a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12406) * de291068835c9b7540c3a0544c6dd589f0c3ada2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-759451108 ## CI report: * 4740cf1d2d0a16700698f5f2108ee4a310d942dd UNKNOWN * b0e28744da86b772436f806aa24a99f5e7581b6b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12320) * 1f8df030fc2ed7fab4d134ef9bddcde2f7d413da UNKNOWN * d0dd72f721d54953ef805c065a486ac73c18b2ff UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563158546
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -909,6 +908,62 @@ object GenerateUtils {
|}
""".stripMargin
compares += code
+})
+compares.mkString
+ }
+
+ /**
+ * Generates code for comparing row keys.
+ * TODO: Remove this after [[MultiFieldRangeBoundComparatorCodeGenerator]]
+ * is refactored using SortSpec.
+ */
+ def generateRowCompare(
Review comment:
Weird, I did not add that. Will revert.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-21104) UnalignedCheckpointITCase.execute failed with "IllegalStateException"
[
https://issues.apache.org/jira/browse/FLINK-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270662#comment-17270662
]
Arvid Heise commented on FLINK-21104:
-
Looking into it.
> UnalignedCheckpointITCase.execute failed with "IllegalStateException"
> -
>
> Key: FLINK-21104
> URL: https://issues.apache.org/jira/browse/FLINK-21104
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.13.0, 1.12.2
>Reporter: Huang Xingbo
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.13.0, 1.12.2
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12383&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=f508e270-48d6-5f1e-3138-42a17e0714f0]
> {code:java}
> 2021-01-22T15:17:34.6711152Z [ERROR] execute[Parallel union, p =
> 10](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time
> elapsed: 3.903 s <<< ERROR!
> 2021-01-22T15:17:34.6711736Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-01-22T15:17:34.6712204Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-01-22T15:17:34.6712779Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-01-22T15:17:34.6713344Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-01-22T15:17:34.6713816Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-01-22T15:17:34.6714454Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6714952Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6715472Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238)
> 2021-01-22T15:17:34.6716026Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-01-22T15:17:34.6716631Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-01-22T15:17:34.6717128Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6717616Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6718105Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-01-22T15:17:34.6718596Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-01-22T15:17:34.6718973Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-01-22T15:17:34.6719364Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-01-22T15:17:34.6719748Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-01-22T15:17:34.6720155Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6720641Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-01-22T15:17:34.6721236Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-01-22T15:17:34.6721706Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-01-22T15:17:34.6722205Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-01-22T15:17:34.6722663Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-01-22T15:17:34.6723214Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-01-22T15:17:34.6723723Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-01-22T15:17:34.6724146Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-01-22T15:17:34.6724726Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6725198Z at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-01-22T15:17:34.6725861Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6726525Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727278Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727773Z at
> scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
> 2021-01-22T15:17:34.6728484Z at
> akka.dispatch.BatchingExecutor$Blockabl
[GitHub] [flink] wuchong commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
wuchong commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563157652
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -909,6 +908,62 @@ object GenerateUtils {
|}
""".stripMargin
compares += code
+})
+compares.mkString
+ }
+
+ /**
+ * Generates code for comparing row keys.
+ * TODO: Remove this after [[MultiFieldRangeBoundComparatorCodeGenerator]]
+ * is refactored using SortSpec.
+ */
+ def generateRowCompare(
Review comment:
It seems this method is not used. Is it introduced by accident?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-759451108 ## CI report: * 4740cf1d2d0a16700698f5f2108ee4a310d942dd UNKNOWN * b0e28744da86b772436f806aa24a99f5e7581b6b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12320) * 1f8df030fc2ed7fab4d134ef9bddcde2f7d413da UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563156638
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -512,18 +512,21 @@ object GenerateUtils {
contextTerm: String): GeneratedExpression = {
val resultType = new TimestampType(true, TimestampKind.ROWTIME, 3)
val resultTypeTerm = primitiveTypeTermForType(resultType)
-val Seq(resultTerm, nullTerm) = ctx.addReusableLocalVariables(
+val Seq(resultTerm, nullTerm, timestamp) = ctx.addReusableLocalVariables(
(resultTypeTerm, "result"),
- ("boolean", "isNull"))
+ ("boolean", "isNull"),
+ ("Long", "tmp")
+)
val accessCode =
s"""
- |$resultTerm =
$TIMESTAMP_DATA.fromEpochMillis($contextTerm.timestamp());
- |if ($resultTerm == null) {
- | throw new RuntimeException("Rowtime timestamp is null. Please make
sure that a " +
- |"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
- |"time characteristic.");
+ |$timestamp = $contextTerm.timestamp();
+ |if ($timestamp == null) {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
wuchong commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563155612
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -512,18 +512,21 @@ object GenerateUtils {
contextTerm: String): GeneratedExpression = {
val resultType = new TimestampType(true, TimestampKind.ROWTIME, 3)
val resultTypeTerm = primitiveTypeTermForType(resultType)
-val Seq(resultTerm, nullTerm) = ctx.addReusableLocalVariables(
+val Seq(resultTerm, nullTerm, timestamp) = ctx.addReusableLocalVariables(
(resultTypeTerm, "result"),
- ("boolean", "isNull"))
+ ("boolean", "isNull"),
+ ("Long", "tmp")
+)
val accessCode =
s"""
- |$resultTerm =
$TIMESTAMP_DATA.fromEpochMillis($contextTerm.timestamp());
- |if ($resultTerm == null) {
- | throw new RuntimeException("Rowtime timestamp is null. Please make
sure that a " +
- |"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
- |"time characteristic.");
+ |$timestamp = $contextTerm.timestamp();
+ |if ($timestamp == null) {
Review comment:
The cost of get timestamp is negligible. Therefore, I prefer the simper
way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov edited a comment on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov edited a comment on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-766080388 @wuchong Apparently not all interfaces of `Context` support `element`. I refactored the code to just do a `null` check on the returned timestamp. Can you please re-run CI? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563155265
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -512,18 +512,21 @@ object GenerateUtils {
contextTerm: String): GeneratedExpression = {
val resultType = new TimestampType(true, TimestampKind.ROWTIME, 3)
val resultTypeTerm = primitiveTypeTermForType(resultType)
-val Seq(resultTerm, nullTerm) = ctx.addReusableLocalVariables(
+val Seq(resultTerm, nullTerm, timestamp) = ctx.addReusableLocalVariables(
(resultTypeTerm, "result"),
- ("boolean", "isNull"))
+ ("boolean", "isNull"),
+ ("Long", "tmp")
+)
val accessCode =
s"""
- |$resultTerm =
$TIMESTAMP_DATA.fromEpochMillis($contextTerm.timestamp());
- |if ($resultTerm == null) {
- | throw new RuntimeException("Rowtime timestamp is null. Please make
sure that a " +
- |"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
- |"time characteristic.");
+ |$timestamp = $contextTerm.timestamp();
+ |if ($timestamp == null) {
Review comment:
@wuchong Right, it's either this or retrieving the timestamp twice.
Whichever is preferred.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
wuchong commented on a change in pull request #14633:
URL: https://github.com/apache/flink/pull/14633#discussion_r563155187
##
File path:
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/codegen/GenerateUtils.scala
##
@@ -512,18 +512,21 @@ object GenerateUtils {
contextTerm: String): GeneratedExpression = {
val resultType = new TimestampType(true, TimestampKind.ROWTIME, 3)
val resultTypeTerm = primitiveTypeTermForType(resultType)
-val Seq(resultTerm, nullTerm) = ctx.addReusableLocalVariables(
+val Seq(resultTerm, nullTerm, timestamp) = ctx.addReusableLocalVariables(
(resultTypeTerm, "result"),
- ("boolean", "isNull"))
+ ("boolean", "isNull"),
+ ("Long", "tmp")
+)
val accessCode =
s"""
- |$resultTerm =
$TIMESTAMP_DATA.fromEpochMillis($contextTerm.timestamp());
- |if ($resultTerm == null) {
- | throw new RuntimeException("Rowtime timestamp is null. Please make
sure that a " +
- |"proper TimestampAssigner is defined and the stream environment
uses the EventTime " +
- |"time characteristic.");
+ |$timestamp = $contextTerm.timestamp();
+ |if ($timestamp == null) {
Review comment:
I simple way for this problem can be `if ($contextTerm.timestamp() ==
null)`, then you don't need to declare a tmp variable.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
flinkbot edited a comment on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-759451108 ## CI report: * 4740cf1d2d0a16700698f5f2108ee4a310d942dd UNKNOWN * b0e28744da86b772436f806aa24a99f5e7581b6b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12320) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-766081220 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20947) Idle source doesn't work when pushing watermark into the source
[
https://issues.apache.org/jira/browse/FLINK-20947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-20947:
Summary: Idle source doesn't work when pushing watermark into the source
(was: Unable to trigger Hive partition commit when use table sql to write data
from Kafka into a Hive table)
> Idle source doesn't work when pushing watermark into the source
> ---
>
> Key: FLINK-20947
> URL: https://issues.apache.org/jira/browse/FLINK-20947
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.0
>Reporter: Weijia Xu
>Assignee: Shengkai Fang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0, 1.12.2
>
>
> I use table sql to create stream with kafka source, and write data from Kafka
> into a Hive partitioned table.
> The following sql to create kafka table:
> {code:java}
> // code placeholder
> tableEnv.executeSql(
> "CREATE TABLE stream_tmp.kafka_tmp (`messageId` STRING, `message_type`
> STRING,`payload` STRING,`timestamp` BIGINT, " +
> " procTime AS PROCTIME()," +
> " eventTime AS TO_TIMESTAMP(FROM_UNIXTIME(`timestamp` /
> 1000,'-MM-dd HH:mm:ss'))," +
> " WATERMARK FOR eventTime AS eventTime - INTERVAL '15' SECOND )"
> +
> " WITH ('connector' = 'kafka'," +
> " 'topic' = 'XXX-topic'," +
> " 'properties.bootstrap.servers'='kafka-server:9092'," +
> " 'properties.group.id' = 'XXX-group_id'," +
> " 'scan.startup.mode' = 'latest-offset'," +
> " 'format' = 'json'," +
> " 'json.fail-on-missing-field' = 'false'," +
> " 'json.ignore-parse-errors' = 'true' )"
> );{code}
>
> And, the following sql to create Hive table:
> {code:java}
> // code placeholder
> tableEnv.executeSql(
> "CREATE TABLE hive_tmp.kafka_hive_tmp (`message_id`
> STRING,`message_type` STRING,`payload` STRING,`event_ts` BIGINT ) " +
> " PARTITIONED BY (ts_date STRING,ts_hour STRING, ts_minute
> STRING)" +
> " STORED AS PARQUET TBLPROPERTIES (" +
> " 'sink.partition-commit.trigger' = 'partition-time'," +
> " 'sink.partition-commit.delay' = '1 min'," +
> " 'sink.partition-commit.policy.kind'='metastore,success-file',"
> +
> " 'sink.partition-commit.success-file.name'='_SUCCESS'," +
> " 'partition.time-extractor.timestamp-pattern' = '$ts_date
> $ts_hour:$ts_minute:00')");
> {code}
>
> For the kafka topic used above, which has multi partitions, with some of
> the partitions there's data coming in, while other partitions with no data
> coming.
> I noticed that there's config "_table.exec.source.idle-timeout_" can handle
> the situation for the "idle" source partition, but event though set this
> config, it still cannot trigger the Hive partition commit (that means the
> "_SUCCESS" file will not be created for the partition).
> After do the analysis for this issue, find the root cause is that the
> watermark for the "idle" partition will not advance, which cause the Hive
> partition cannot be committed.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] YuvalItzchakov edited a comment on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov edited a comment on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-766080388 @wuchong Apparently not all interfaces of `Context` support `hasTimestamp`. I refactored the code to just do a `null` check on the returned timestamp. Can you please re-run CI? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YuvalItzchakov commented on pull request #14633: [FLINK-20961][table-planner-blink] Fix NPE when no assigned timestamp defined
YuvalItzchakov commented on pull request #14633: URL: https://github.com/apache/flink/pull/14633#issuecomment-766080388 @wuchong Apparently not all interfaces of `Context` support `hasTimestamp`. I refactored the code to just do a `null` check on the returned timestamp. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * ef5b18453605da53ca9c9ce0c7fce9b7361f67a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12406) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * 35ecb423c9f73b3797455995731e29b72a380e88 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12404) * ef5b18453605da53ca9c9ce0c7fce9b7361f67a3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12406) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-shaded] HuangXingBo commented on pull request #93: [FLINK-21020][jackson] Bump version to 2.12.1
HuangXingBo commented on pull request #93: URL: https://github.com/apache/flink-shaded/pull/93#issuecomment-765901405 I have tested this flink-shaded version with flink (master version) in my private Azure. the test commit: https://github.com/HuangXingBo/flink/commit/554cecd092594e155a7b0b140b675fe0b47fe72d the test result: https://dev.azure.com/hxbks2ks/FLINK-TEST/_build/results?buildId=808&view=results This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * 35ecb423c9f73b3797455995731e29b72a380e88 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12404) * ef5b18453605da53ca9c9ce0c7fce9b7361f67a3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-21105) Rename ExecEdge to InputProperty
godfrey he created FLINK-21105: -- Summary: Rename ExecEdge to InputProperty Key: FLINK-21105 URL: https://issues.apache.org/jira/browse/FLINK-21105 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: godfrey he Assignee: godfrey he Fix For: 1.13.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-20435) Refactor ExecNode and ExecEdge
[
https://issues.apache.org/jira/browse/FLINK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-20435:
---
Description:
Currently, there are many improvements about ExecNode:
1. simplify type parameter of {{ExecNode}}. Currently,
{{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
indicates the node is a batch node or a streaming node. While in the future, a
plan may contain both batch nodes and stream node. The type parameter can be
removed, and we will use PlannerBase instead.
2. port the implementation of ExecNodes to Java
3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, an execution node extends both from {{FlinkPhysicalRel}} and
{{ExecNode}}. After a physical node is converted to an exec node, many
parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With more
optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and simpler.
So we will separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, the ExecEdge represents the properties for the input of an operator,
the properties include
1. required data distribute for an input (the input corresponds to the
[Input|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/Input.java])
2. DamBehavior which describes the behaviors how an in record may trigger the
output of the target operator.
3. the priority of this input read by the target operator
So ExecEdge should be rename to InputProperty, and we will re-introduce
ExecEdge which describes how to connect two ExecNodes, and how to shuffle the
data between two ExecNodes. Its role should be similar to StreamEdge.
This is an umbrella issue, we will create more related sub-tasks.
was:
Currently, there are many improvements about ExecNode:
1. simplify type parameter of {{ExecNode}}. Currently,
{{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
indicates the node is a batch node or a streaming node. While in the future, a
plan may contain both batch nodes and stream node. The type parameter can be
removed, and we will use PlannerBase instead.
2. port the implementation of ExecNodes to Java
3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, an execution node extends both from {{FlinkPhysicalRel}} and
{{ExecNode}}. After a physical node is converted to an exec node, many
parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With more
optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and simpler.
So we will separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, the ExecEdge represents the input properties of an operator, the
properties include
1. required data distribute for an input (the input corresponds to the Input)
2. DamBehavior
This is an umbrella issue, we will create more related sub-tasks.
> Refactor ExecNode and ExecEdge
> --
>
> Key: FLINK-20435
> URL: https://issues.apache.org/jira/browse/FLINK-20435
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>
> Currently, there are many improvements about ExecNode:
> 1. simplify type parameter of {{ExecNode}}. Currently,
> {{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
> parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
> indicates the node is a batch node or a streaming node. While in the future,
> a plan may contain both batch nodes and stream node. The type parameter can
> be removed, and we will use PlannerBase instead.
> 2. port the implementation of ExecNodes to Java
> 3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
> Currently, an execution node extends both from {{FlinkPhysicalRel}} and
> {{ExecNode}}. After a physical node is converted to an exec node, many
> parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With
> more optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and
> simpler. So we will separate the implementation of {{FlinkPhysicalRel}} and
> {{ExecNode}}.
> Currently, the ExecEdge represents the properties for the input of an
> operator, the properties include
> 1. required data distribute for an input (the input corresponds to the
> [Input|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/Input.java])
> 2. DamBehavior which describes the behaviors how an in record may trigger
[jira] [Updated] (FLINK-20435) Refactor ExecNode and ExecEdge
[
https://issues.apache.org/jira/browse/FLINK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-20435:
---
Description:
Currently, there are many improvements about ExecNode:
1. simplify type parameter of {{ExecNode}}. Currently,
{{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
indicates the node is a batch node or a streaming node. While in the future, a
plan may contain both batch nodes and stream node. The type parameter can be
removed, and we will use PlannerBase instead.
2. port the implementation of ExecNodes to Java
3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, an execution node extends both from {{FlinkPhysicalRel}} and
{{ExecNode}}. After a physical node is converted to an exec node, many
parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With more
optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and simpler.
So we will separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, the ExecEdge represents the input properties of an operator, the
properties include
1. required data distribute for an input (the input corresponds to the Input)
2. DamBehavior
This is an umbrella issue, we will create more related sub-tasks.
was:
Currently, there are many improvements about ExecNode:
1. simplify type parameter of {{ExecNode}}. Currently,
{{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
indicates the node is a batch node or a streaming node. While in the future, a
plan may contain both batch nodes and stream node. The type parameter can be
removed, and we will use PlannerBase instead.
2. port the implementation of ExecNodes to Java
3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
Currently, an execution node extends both from {{FlinkPhysicalRel}} and
{{ExecNode}}. After a physical node is converted to an exec node, many
parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With more
optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and simpler.
So we will separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
This is an umbrella issue, we will create more related sub-tasks.
> Refactor ExecNode and ExecEdge
> --
>
> Key: FLINK-20435
> URL: https://issues.apache.org/jira/browse/FLINK-20435
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>
> Currently, there are many improvements about ExecNode:
> 1. simplify type parameter of {{ExecNode}}. Currently,
> {{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
> parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
> indicates the node is a batch node or a streaming node. While in the future,
> a plan may contain both batch nodes and stream node. The type parameter can
> be removed, and we will use PlannerBase instead.
> 2. port the implementation of ExecNodes to Java
> 3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
> Currently, an execution node extends both from {{FlinkPhysicalRel}} and
> {{ExecNode}}. After a physical node is converted to an exec node, many
> parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With
> more optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and
> simpler. So we will separate the implementation of {{FlinkPhysicalRel}} and
> {{ExecNode}}.
> Currently, the ExecEdge represents the input properties of an operator, the
> properties include
> 1. required data distribute for an input (the input corresponds to the Input)
> 2. DamBehavior
> This is an umbrella issue, we will create more related sub-tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * 35ecb423c9f73b3797455995731e29b72a380e88 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12404) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20435) Refactor ExecNode and ExecEdge
[
https://issues.apache.org/jira/browse/FLINK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-20435:
---
Summary: Refactor ExecNode and ExecEdge (was: Refactor ExecNode)
> Refactor ExecNode and ExecEdge
> --
>
> Key: FLINK-20435
> URL: https://issues.apache.org/jira/browse/FLINK-20435
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
>
> Currently, there are many improvements about ExecNode:
> 1. simplify type parameter of {{ExecNode}}. Currently,
> {{ExecNode#translateToPlan}} takes {{BatchPlanner}} or {{StreamPlanner}} as a
> parameter, so {{ExecNode}} has a type parameter {{E <: Planner}}, which
> indicates the node is a batch node or a streaming node. While in the future,
> a plan may contain both batch nodes and stream node. The type parameter can
> be removed, and we will use PlannerBase instead.
> 2. port the implementation of ExecNodes to Java
> 3. separate the implementation of {{FlinkPhysicalRel}} and {{ExecNode}}.
> Currently, an execution node extends both from {{FlinkPhysicalRel}} and
> {{ExecNode}}. After a physical node is converted to an exec node, many
> parameters are unnecessary, such as: RelOptCluster, RelTraitSet, etc. With
> more optimizations on {{ExecNode}}, We need {{ExecNode}} to be cleaner and
> simpler. So we will separate the implementation of {{FlinkPhysicalRel}} and
> {{ExecNode}}.
> This is an umbrella issue, we will create more related sub-tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #13377: [FLINK-18592] bugfix for StreamingFileSink
flinkbot edited a comment on pull request #13377: URL: https://github.com/apache/flink/pull/13377#issuecomment-691669199 ## CI report: * e50494c67acde3b849686106cf926a28f1b32685 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12405) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot edited a comment on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * 35ecb423c9f73b3797455995731e29b72a380e88 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12404) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21036) Consider removing automatic configuration fo number of slots from docker
[
https://issues.apache.org/jira/browse/FLINK-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270590#comment-17270590
]
Yang Wang commented on FLINK-21036:
---
For Kubernetes use case, we suggest our users to create a flink-conf
ConfigMap[1] and then mount it to JobManager and TaskManager. It is a k8s
natural way to set the configuration. We do not need to set them via env or
container args.
[1].
https://ci.apache.org/projects/flink/flink-docs-master/deployment/resource-providers/standalone/kubernetes.html#common-cluster-resource-definitions
> Consider removing automatic configuration fo number of slots from docker
>
>
> Key: FLINK-21036
> URL: https://issues.apache.org/jira/browse/FLINK-21036
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Scripts
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> The {{docker-entrypoint.sh}} supports setting the number of task slots via
> the {{TASK_MANAGER_NUMBER_OF_TASK_SLOTS}} environment variable, which
> defaults to the number of cpu cores via {{$(grep -c ^processor
> /proc/cpuinfo)}}.
> The environment variable itself is redundant nowadays since we introduced
> {{FLINK_PROPERTIES}}, and is no longer documented.
> Defaulting to the number of CPU cores can be considered convenience, but it
> seems odd to have this specific to docker while the distribution defaults to
> {{1}}.
> The bigger issue in my mind though is that this creates a configuration
> mismatch between the Job- and TaskManager processes; the ResourceManager
> specifically needs to know how many slots a worker has to make decisions
> about redundancy and allocating resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #13377: [FLINK-18592] bugfix for StreamingFileSink
flinkbot edited a comment on pull request #13377: URL: https://github.com/apache/flink/pull/13377#issuecomment-691669199 ## CI report: * 5133b8e8e8ab5c63b96f23b024ebef1252f5d128 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6474) * e50494c67acde3b849686106cf926a28f1b32685 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wangjian2019 commented on pull request #13377: [FLINK-18592] bugfix for StreamingFileSink
wangjian2019 commented on pull request #13377: URL: https://github.com/apache/flink/pull/13377#issuecomment-765889444 resolve conflict This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot commented on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765889414 ## CI report: * 35ecb423c9f73b3797455995731e29b72a380e88 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14729: [FLINK-21092][FLINK-21093][FLINK-21094][FLINK-21096][table-planner-blink] Support ExecNode plan serialization/deserialization for `IN
flinkbot edited a comment on pull request #14729: URL: https://github.com/apache/flink/pull/14729#issuecomment-765399004 ## CI report: * ef724e57e7676c154c8f0d6da93a811e9d411051 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12402) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14729: [FLINK-21092][FLINK-21093][FLINK-21094][FLINK-21096][table-planner-blink] Support ExecNode plan serialization/deserialization for `IN
flinkbot edited a comment on pull request #14729: URL: https://github.com/apache/flink/pull/14729#issuecomment-765399004 ## CI report: * ef724e57e7676c154c8f0d6da93a811e9d411051 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12402) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14732: Release 1.12 The Flink task was committed to YARN, but the prompt（NoSuchMethod:Lorg/apache/hadoop/security/UserGroupInformation） fail
flinkbot edited a comment on pull request #14732: URL: https://github.com/apache/flink/pull/14732#issuecomment-765882051 ## CI report: * b866f88656da7ca5f6b962c234f7c2a07d58fe99 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12403) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] godfreyhe commented on pull request #14729: [FLINK-21092][FLINK-21093][FLINK-21094][FLINK-21096][table-planner-blink] Support ExecNode plan serialization/deserialization for `INSERT I
godfreyhe commented on pull request #14729: URL: https://github.com/apache/flink/pull/14729#issuecomment-765887150 @flinkbot run azure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
flinkbot commented on pull request #14733: URL: https://github.com/apache/flink/pull/14733#issuecomment-765886914 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 109b78814bd15e98f31a858175c7e50793b7d83a (Sat Jan 23 08:11:20 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20968) Remove legacy exec nodes
[ https://issues.apache.org/jira/browse/FLINK-20968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20968: --- Labels: pull-request-available (was: ) > Remove legacy exec nodes > > > Key: FLINK-20968 > URL: https://issues.apache.org/jira/browse/FLINK-20968 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: godfrey he >Assignee: godfrey he >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] godfreyhe opened a new pull request #14733: [FLINK-20968][table-planner-blink] Remove legacy exec nodes
godfreyhe opened a new pull request #14733: URL: https://github.com/apache/flink/pull/14733 ## What is the purpose of the change *Since all exec node do not extend from LegacyExecNodeBase, we can remove them (LegacyExecNodeBase, LegacyBatchExecNode and LegacyStreamExecNode). And this pr also do some code clean up, such as: rename ExecNode#getDesc to ExecNode#getDescription, fix some typos* ## Brief change log - *Remove legacy exec nodes* - *Rename ExecNode#getDesc to ExecNode#getDescription* - *Move CommonPhysicalJoin and CommonPhysicalLookupJoin to `nodes.physical.common` package* - *Minor code cleanup for some exec nodes* ## Verifying this change This change is code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21096) Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json plan and deserialize json plan to ExecNodeGraph
[
https://issues.apache.org/jira/browse/FLINK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-21096:
---
Description:
ExecNodeGraphJsonPlanGenerator is an utility class that can generate json plan
based on given ExecNodeGraph or generate ExecNodeGraph based on given json
plan.
Based on FLINK-21092, FLINK-21093, FLINK-21094 and this jira, we can get the
json plan for {{insert into MySink select * from MyTable}}, we also can submit
the json plan and get the execution result.
was:
ExecNodeGraphJsonPlanGenerator is an utility class that can generate json plan
based on given ExecNodeGraph or generate ExecNodeGraph based on given json
plan.
Based on FLINK-21092, FLINK-21093, FLINK-21094 and this jira, we can get the
json plan for {{insert into MySink select * from MyTable}}, we also can submit
the json plan and get the result.
> Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json
> plan and deserialize json plan to ExecNodeGraph
> ---
>
> Key: FLINK-21096
> URL: https://issues.apache.org/jira/browse/FLINK-21096
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
> Fix For: 1.13.0
>
>
> ExecNodeGraphJsonPlanGenerator is an utility class that can generate json
> plan based on given ExecNodeGraph or generate ExecNodeGraph based on given
> json plan.
> Based on FLINK-21092, FLINK-21093, FLINK-21094 and this jira, we can get the
> json plan for {{insert into MySink select * from MyTable}}, we also can
> submit the json plan and get the execution result.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21096) Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json plan and deserialize json plan to ExecNodeGraph
[
https://issues.apache.org/jira/browse/FLINK-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-21096:
---
Description:
ExecNodeGraphJsonPlanGenerator is an utility class that can generate json plan
based on given ExecNodeGraph or generate ExecNodeGraph based on given json
plan.
Based on FLINK-21092, FLINK-21093, FLINK-21094 and this jira, we can get the
json plan for {{insert into MySink select * from MyTable}}, we also can submit
the json plan and get the result.
> Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json
> plan and deserialize json plan to ExecNodeGraph
> ---
>
> Key: FLINK-21096
> URL: https://issues.apache.org/jira/browse/FLINK-21096
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
>Reporter: godfrey he
>Assignee: godfrey he
>Priority: Major
> Fix For: 1.13.0
>
>
> ExecNodeGraphJsonPlanGenerator is an utility class that can generate json
> plan based on given ExecNodeGraph or generate ExecNodeGraph based on given
> json plan.
> Based on FLINK-21092, FLINK-21093, FLINK-21094 and this jira, we can get the
> json plan for {{insert into MySink select * from MyTable}}, we also can
> submit the json plan and get the result.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
