[GitHub] [flink] flinkbot edited a comment on pull request #13474: [FLINK-19391][network] Moved notification during subpartition request to the requester.
flinkbot edited a comment on pull request #13474: URL: https://github.com/apache/flink/pull/13474#issuecomment-698189239 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit de0facc9fcb98ce11cc2c30df6524fe4fb09d1ed (Fri Feb 19 07:26:19 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13460: [FLINK-19364][python] Add Batch Physical Pandas Group Window Aggregate Rule and RelNode
flinkbot edited a comment on pull request #13460: URL: https://github.com/apache/flink/pull/13460#issuecomment-697191079 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a167ead3f80c642c8e5cc6098deec39b0d9a6899 (Fri Feb 19 07:26:08 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13003: [FLINK-18737][docs]translate jdbc connector
flinkbot edited a comment on pull request #13003: URL: https://github.com/apache/flink/pull/13003#issuecomment-664836878 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 19d4a1375947b925e86b02fb4fe67d313fed43e2 (Fri Feb 19 07:26:01 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12454: [FLINK-17091][Arvo] Adapt Avro record conversion to new timestamp bridged classes
flinkbot edited a comment on pull request #12454: URL: https://github.com/apache/flink/pull/12454#issuecomment-637988260 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 498d42db35e412ad834d8b0c70d6404d0de4c339 (Fri Feb 19 07:26:05 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13416: [FLINK-19179] Extend managed memory fraction calculation for various use cases.
flinkbot edited a comment on pull request #13416: URL: https://github.com/apache/flink/pull/13416#issuecomment-694667551 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9bad008ffbab45dbefbba394eda9c728e531c5ef (Fri Feb 19 07:26:12 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13437: [FLINK-19189][runtime] Enable pipelined scheduling by default
flinkbot edited a comment on pull request #13437: URL: https://github.com/apache/flink/pull/13437#issuecomment-695936739 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 589cb4d4a9b4df102404c961903e4c02144bc0a4 (Fri Feb 19 07:24:46 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12768: [FLINK-17804][parquet] Follow Parquet spec when decoding DECIMAL
flinkbot edited a comment on pull request #12768: URL: https://github.com/apache/flink/pull/12768#issuecomment-649137157 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a2efe0f32d68f1c563e1825ec17e10c4229d44fa (Fri Feb 19 07:24:56 UTC 2021) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-17804).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #9356: [FLINK-13340][kafka][table] Add 'topics' and 'subscriptionPattern' option for Flink Kafka connector
flinkbot edited a comment on pull request #9356: URL: https://github.com/apache/flink/pull/9356#issuecomment-518115797 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 994378a936cb3d0e91dd78607e81229c4680e7d6 (Fri Feb 19 07:24:49 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13468: fix properties type cast error
flinkbot edited a comment on pull request #13468: URL: https://github.com/apache/flink/pull/13468#issuecomment-698096647 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 041db6f59cb68e8c868dae42df3b76c12cf18db2 (Fri Feb 19 07:24:53 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13228: [FLINK-19026][network] Improve threading model of CheckpointBarrierUnaligner
flinkbot edited a comment on pull request #13228: URL: https://github.com/apache/flink/pull/13228#issuecomment-679094462 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 36c366400126cb68f0a8c8cf14b5c5e98ef2b58d (Fri Feb 19 07:24:38 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13445: [FLINK-19331][state-processor-api] Native resource leak when working with RocksDB
flinkbot edited a comment on pull request #13445: URL: https://github.com/apache/flink/pull/13445#issuecomment-696385131 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit be6abbabc63e0d605e7319fa87b3f84c27cf2ec5 (Fri Feb 19 07:24:27 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13422: [FLINK-19286][runtime] Improve pipelined region scheduling performance
flinkbot edited a comment on pull request #13422: URL: https://github.com/apache/flink/pull/13422#issuecomment-694779455 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 62e3ab653c5b01e5ad7b750e1db97d3d066ef287 (Fri Feb 19 07:24:42 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13357: [FLINK-19165] Refactor the UnilateralSortMerger
flinkbot edited a comment on pull request #13357: URL: https://github.com/apache/flink/pull/13357#issuecomment-689028555 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f02440e449ea6ebefbd3eae151bf30ae0cebed72 (Fri Feb 19 07:24:24 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on a change in pull request #14950: [FLINK-21347][coordination] Extract interface from ExecutionGraph
zentol commented on a change in pull request #14950:
URL: https://github.com/apache/flink/pull/14950#discussion_r578977899
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/declarative/StateTrackingMockExecutionGraph.java
##
@@ -0,0 +1,459 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.declarative;
+
+import org.apache.flink.api.common.ArchivedExecutionConfig;
+import org.apache.flink.api.common.ExecutionConfig;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.api.common.JobStatus;
+import org.apache.flink.api.common.accumulators.Accumulator;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.accumulators.AccumulatorSnapshot;
+import org.apache.flink.runtime.accumulators.StringifiedAccumulatorResult;
+import org.apache.flink.runtime.blob.BlobWriter;
+import org.apache.flink.runtime.blob.PermanentBlobKey;
+import org.apache.flink.runtime.checkpoint.CheckpointCoordinator;
+import org.apache.flink.runtime.checkpoint.CheckpointIDCounter;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsTracker;
+import org.apache.flink.runtime.checkpoint.CheckpointsCleaner;
+import org.apache.flink.runtime.checkpoint.CompletedCheckpointStore;
+import org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook;
+import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.ErrorInfo;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionAttemptID;
+import org.apache.flink.runtime.executiongraph.ExecutionDeploymentListener;
+import org.apache.flink.runtime.executiongraph.ExecutionGraph;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.executiongraph.IntermediateResult;
+import org.apache.flink.runtime.executiongraph.JobInformation;
+import org.apache.flink.runtime.executiongraph.JobStatusListener;
+import org.apache.flink.runtime.executiongraph.TaskExecutionStateTransition;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.ResultPartitionAvailabilityChecker;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.partitionrelease.PartitionReleaseStrategy;
+import org.apache.flink.runtime.io.network.partition.JobMasterPartitionTracker;
+import org.apache.flink.runtime.io.network.partition.ResultPartitionID;
+import org.apache.flink.runtime.jobgraph.IntermediateDataSetID;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+import org.apache.flink.runtime.jobgraph.ScheduleMode;
+import
org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration;
+import org.apache.flink.runtime.query.KvStateLocationRegistry;
+import org.apache.flink.runtime.scheduler.InternalFailuresListener;
+import org.apache.flink.runtime.scheduler.strategy.SchedulingTopology;
+import org.apache.flink.runtime.shuffle.ShuffleMaster;
+import org.apache.flink.runtime.state.CheckpointStorage;
+import org.apache.flink.runtime.state.StateBackend;
+import org.apache.flink.types.Either;
+import org.apache.flink.util.OptionalFailure;
+import org.apache.flink.util.SerializedValue;
+
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
+import java.util.concurrent.ForkJoinPool;
+
+/**
+ * Mocked ExecutionGraph which (partially) tracks the job status, and provides
some basic mocks to
+ * create an {@link
org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph} from this
+ * ExecutionGraph.
+ */
+class StateTrackingMockExecutionGraph implements ExecutionGraph {
+private JobStatus state = JobStatus.INITIALIZING;
+private final
[GitHub] [flink] flinkbot edited a comment on pull request #13358: [FLINK-11779] CLI ignores -m parameter if high-availability is ZOOKEEPER
flinkbot edited a comment on pull request #13358: URL: https://github.com/apache/flink/pull/13358#issuecomment-689269505 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 34e6524648d45b6806245e8e44c52c3321f2c21e (Fri Feb 19 07:24:31 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13455: [FLINK-19361][hive] Create a synchronized metastore client to talk to…
flinkbot edited a comment on pull request #13455: URL: https://github.com/apache/flink/pull/13455#issuecomment-696719238 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a65e5efb173508929ee6ff9cddb7a3ac8cb38f04 (Fri Feb 19 07:24:11 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #8741: [FLINK-12752] Add Option to Pass Seed for JobID Hash for StandaloneJobClusterEntrypoint
flinkbot edited a comment on pull request #8741: URL: https://github.com/apache/flink/pull/8741#issuecomment-502078258 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c633d83e46d8618812f649d78b1be99450c901a6 (Fri Feb 19 07:24:19 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #11359: [FLINK-16095] [docs-zh] Translate "Modules" page of "Table API & SQL" into Chinese
flinkbot edited a comment on pull request #11359: URL: https://github.com/apache/flink/pull/11359#issuecomment-596915123 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cbd904d8f233ecc0813c96f598540973a5e0b921 (Fri Feb 19 07:24:15 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #10354: [FLINK-14729][connectors] Multi-topics consuming from KafkaTableSource
flinkbot edited a comment on pull request #10354: URL: https://github.com/apache/flink/pull/10354#issuecomment-559645834 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 39f094776ea236e08b18c471c36fc784d8e85bbc (Fri Feb 19 07:24:08 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14729).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13405: [FLINK-19270] Extract an inteface from AbstractKeyedStateBackend
flinkbot edited a comment on pull request #13405: URL: https://github.com/apache/flink/pull/13405#issuecomment-694050574 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ec745aa11eb24be7e2bfc6bb5da93f2e24a1e6cc (Fri Feb 19 07:22:52 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13289: [FLINK-18548][table-planner] support flexible syntax for Temporal table join
flinkbot edited a comment on pull request #13289: URL: https://github.com/apache/flink/pull/13289#issuecomment-683886127 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ef462e3d6a961f1d353a84e8ae55982c97c67a8e (Fri Feb 19 07:23:04 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13414: [FLINK-19227][Table SQL / API] The catalog is still created after opening failed in catalog registering
flinkbot edited a comment on pull request #13414: URL: https://github.com/apache/flink/pull/13414#issuecomment-694640971 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit b392f6aff314ad8acd9e6d720e390cfc00e026ef (Fri Feb 19 07:23:00 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13420: [FLINK-19229][python] Introduce the PythonStreamGroupAggregateOperator for Python UDAF.
flinkbot edited a comment on pull request #13420: URL: https://github.com/apache/flink/pull/13420#issuecomment-694771096 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 7571b70b754ee4e740b3a3777c506fb8f5e67c40 (Fri Feb 19 07:22:56 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13429: [FLINK-19140] Fix UDTF documentation which uses wrong alias
flinkbot edited a comment on pull request #13429: URL: https://github.com/apache/flink/pull/13429#issuecomment-695771100 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8a1269960098af0f93c2107989391e692588f104 (Fri Feb 19 07:22:44 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13430: [BP-1.11][FLINK-19140] Fix UDTF documentation which uses wrong alias
flinkbot edited a comment on pull request #13430: URL: https://github.com/apache/flink/pull/13430#issuecomment-695771924 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6f69d65d79928c4e3189aae01a5c3f220fb0e5d2 (Fri Feb 19 07:22:41 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #8331: [FLINK-11987][flink-connector-kafka-0.11] convert NEXT_TRANSACTIONAL_ID_HINT_DESCRIPTOR to a non-static variabl…
flinkbot edited a comment on pull request #8331: URL: https://github.com/apache/flink/pull/8331#issuecomment-488582162 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 508fea5e338237f096ebbe05979e1c95b7e07719 (Fri Feb 19 07:22:48 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13332: [FLINK-19128][sql-client] Remove the runtime execution configuration in sql-client-defaults.yaml
flinkbot edited a comment on pull request #13332: URL: https://github.com/apache/flink/pull/13332#issuecomment-687611369 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 8f1ae1192b3809aac8728de1bdb9d5c6bce36f9a (Fri Feb 19 07:22:34 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13446: [FLINK-18779][table sql/planner]Support the SupportsFilterPushDown for LookupTableSource
flinkbot edited a comment on pull request #13446: URL: https://github.com/apache/flink/pull/13446#issuecomment-696484481 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 41d632a55eeed78fdeb78a23ba5cc68ca12fc1f1 (Fri Feb 19 07:22:30 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #8142: [FLINK-12148][clients] Give precedence to specified local jar
flinkbot edited a comment on pull request #8142: URL: https://github.com/apache/flink/pull/8142#issuecomment-481692125 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4c0c4f1221bb153cde03b03b46992615c399a86a (Fri Feb 19 07:22:37 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13427: [FLINK-17767][table-planner-blink]Tumbling/Sliding window aggregate support window start offset in batch and streaming mode
flinkbot edited a comment on pull request #13427: URL: https://github.com/apache/flink/pull/13427#issuecomment-695763230 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 85c69f8e4c0c59ca25be120811b81acdad2486a4 (Fri Feb 19 07:22:19 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13442: [FLINK-19321][Table SQL / Runtime]CollectSinkFunction does not define serialVersionUID
flinkbot edited a comment on pull request #13442: URL: https://github.com/apache/flink/pull/13442#issuecomment-696189196 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5357cf4726cec3b04d6bfa54a6b95932d380cf1f (Fri Feb 19 07:22:27 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13392: [FLINK-18713][table-planner-blink] Change duration configOption to duration type
flinkbot edited a comment on pull request #13392: URL: https://github.com/apache/flink/pull/13392#issuecomment-692536820 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9f86e01988e6d8ad8b904670f66713ab21e485d6 (Fri Feb 19 07:22:23 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13444: [BP-1.11][FLINK-19244] CsvRowDataDeserializationSchema throws cast ex…
flinkbot edited a comment on pull request #13444: URL: https://github.com/apache/flink/pull/13444#issuecomment-696202108 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 0a8312fc1af9600f6f0f247dbf540ec4848d02c1 (Fri Feb 19 07:22:15 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on a change in pull request #14950: [FLINK-21347][coordination] Extract interface from ExecutionGraph
zentol commented on a change in pull request #14950:
URL: https://github.com/apache/flink/pull/14950#discussion_r578976880
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/declarative/StateTrackingMockExecutionGraph.java
##
@@ -0,0 +1,459 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.declarative;
+
+import org.apache.flink.api.common.ArchivedExecutionConfig;
+import org.apache.flink.api.common.ExecutionConfig;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.api.common.JobStatus;
+import org.apache.flink.api.common.accumulators.Accumulator;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.accumulators.AccumulatorSnapshot;
+import org.apache.flink.runtime.accumulators.StringifiedAccumulatorResult;
+import org.apache.flink.runtime.blob.BlobWriter;
+import org.apache.flink.runtime.blob.PermanentBlobKey;
+import org.apache.flink.runtime.checkpoint.CheckpointCoordinator;
+import org.apache.flink.runtime.checkpoint.CheckpointIDCounter;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsTracker;
+import org.apache.flink.runtime.checkpoint.CheckpointsCleaner;
+import org.apache.flink.runtime.checkpoint.CompletedCheckpointStore;
+import org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook;
+import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.ErrorInfo;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionAttemptID;
+import org.apache.flink.runtime.executiongraph.ExecutionDeploymentListener;
+import org.apache.flink.runtime.executiongraph.ExecutionGraph;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.executiongraph.IntermediateResult;
+import org.apache.flink.runtime.executiongraph.JobInformation;
+import org.apache.flink.runtime.executiongraph.JobStatusListener;
+import org.apache.flink.runtime.executiongraph.TaskExecutionStateTransition;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.ResultPartitionAvailabilityChecker;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.partitionrelease.PartitionReleaseStrategy;
+import org.apache.flink.runtime.io.network.partition.JobMasterPartitionTracker;
+import org.apache.flink.runtime.io.network.partition.ResultPartitionID;
+import org.apache.flink.runtime.jobgraph.IntermediateDataSetID;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+import org.apache.flink.runtime.jobgraph.ScheduleMode;
+import
org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration;
+import org.apache.flink.runtime.query.KvStateLocationRegistry;
+import org.apache.flink.runtime.scheduler.InternalFailuresListener;
+import org.apache.flink.runtime.scheduler.strategy.SchedulingTopology;
+import org.apache.flink.runtime.shuffle.ShuffleMaster;
+import org.apache.flink.runtime.state.CheckpointStorage;
+import org.apache.flink.runtime.state.StateBackend;
+import org.apache.flink.types.Either;
+import org.apache.flink.util.OptionalFailure;
+import org.apache.flink.util.SerializedValue;
+
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
+import java.util.concurrent.ForkJoinPool;
+
+/**
+ * Mocked ExecutionGraph which (partially) tracks the job status, and provides
some basic mocks to
+ * create an {@link
org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph} from this
+ * ExecutionGraph.
+ */
+class StateTrackingMockExecutionGraph implements ExecutionGraph {
+private JobStatus state = JobStatus.INITIALIZING;
+private final
[GitHub] [flink] flinkbot edited a comment on pull request #13426: [FLINK-19244] CsvRowDataDeserializationSchema throws cast exception : Row length mismatch.
flinkbot edited a comment on pull request #13426: URL: https://github.com/apache/flink/pull/13426#issuecomment-695220756 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 622a545952e0c527b145ce99194067887d75f714 (Fri Feb 19 07:21:12 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13436: [FLINK-19301][python] Improve the package structure of Python DataStream API
flinkbot edited a comment on pull request #13436: URL: https://github.com/apache/flink/pull/13436#issuecomment-695928048 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit aabbc32fdf457d00564976ddf217f7872ca7878a (Fri Feb 19 07:20:59 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13401: [FLINK-19161][file connector] Add first version of the FLIP-27 File Source
flinkbot edited a comment on pull request #13401: URL: https://github.com/apache/flink/pull/13401#issuecomment-693272292 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit d76795d2b9cec4b3a8c3efcdbc3062ec4e71d427 (Fri Feb 19 07:21:08 UTC 2021) **Warnings:** * **7 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13433: [BP-1.11][FLINK-19280][jdbc] Fix option "sink.buffer-flush.max-rows" for JDBC can't be disabled by setting to zero
flinkbot edited a comment on pull request #13433: URL: https://github.com/apache/flink/pull/13433#issuecomment-695888255 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9a754ba34826f47a7705b2f04a7d56dd36ec81b8 (Fri Feb 19 07:21:03 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13435: [FLINK-19299][tests] Fix that NettyShuffleEnvironmentBuilder#setBufferSize does not take effect
flinkbot edited a comment on pull request #13435: URL: https://github.com/apache/flink/pull/13435#issuecomment-695891985 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit d1facaadfa12109f252a66bd76c0e849c705751a (Fri Feb 19 07:20:56 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13432: [FLINK-19280][connectors] fix the bug of the option "sink.buffer-flush.max-rows" for JDBC can't be disabled by set to zero
flinkbot edited a comment on pull request #13432: URL: https://github.com/apache/flink/pull/13432#issuecomment-695807197 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 3061480f6d6893693e110f421e1aceabf1bd83c7 (Fri Feb 19 07:20:42 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13189: [FLINK-18661][Kinesis] Stream consumer Registration/Deregistration
flinkbot edited a comment on pull request #13189: URL: https://github.com/apache/flink/pull/13189#issuecomment-675341370 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 05be61208d813aa603b5acff524c79d663579ec1 (Fri Feb 19 07:20:47 UTC 2021) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13102: [FLINK-18515][Kinesis] Adding FanOutRecordPublisher for Kinesis EFO support
flinkbot edited a comment on pull request #13102: URL: https://github.com/apache/flink/pull/13102#issuecomment-671232457 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 424e9b3074c5e67066aa1fdd41f14bb2fbae (Fri Feb 19 07:20:52 UTC 2021) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12962: [FLINK-18694] Add unaligned checkpoint config to web ui
flinkbot edited a comment on pull request #12962: URL: https://github.com/apache/flink/pull/12962#issuecomment-662518124 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f5acc063125e57546cd7717106da2a1f642e841c (Fri Feb 19 07:20:34 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13275: [FLINK-19064][hbase] HBaseRowDataInputFormat is leaking resources
flinkbot edited a comment on pull request #13275: URL: https://github.com/apache/flink/pull/13275#issuecomment-682387017 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4a1bd39b87efb3680010a3ba9a9fce8485411546 (Fri Feb 19 07:20:39 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13417: [FLINK-FLINK][docs] Update the version in docs/_config.yml to be the latest released minor version
flinkbot edited a comment on pull request #13417: URL: https://github.com/apache/flink/pull/13417#issuecomment-694697532 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 50257769477d0a4c10f4fb1ec45f5c460e4d0b2c (Fri Feb 19 07:20:27 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13396: [FLINK-19248] add missing lastJobExecutionResult assignment after job finished in ContexEnvironment
flinkbot edited a comment on pull request #13396: URL: https://github.com/apache/flink/pull/13396#issuecomment-693187999 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 3eedbea46311fafbd37fc107d892e9fa906bf451 (Fri Feb 19 07:20:30 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13418: [FLINK-FLINK][docs] Update the version in docs/_config.yml to be the latest released minor version
flinkbot edited a comment on pull request #13418: URL: https://github.com/apache/flink/pull/13418#issuecomment-694697519 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4dbb1a782deb4f1750186490dbe6e8857e5a2d0a (Fri Feb 19 07:20:23 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13384: [FLINK-19224][state-processor-api] Support reading window operator state
flinkbot edited a comment on pull request #13384: URL: https://github.com/apache/flink/pull/13384#issuecomment-692328370 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 98b77dffa6ea7b1da3940d9c5cceb5e8d7a73e82 (Fri Feb 19 07:20:19 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zentol commented on a change in pull request #14950: [FLINK-21347][coordination] Extract interface from ExecutionGraph
zentol commented on a change in pull request #14950:
URL: https://github.com/apache/flink/pull/14950#discussion_r578976098
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/declarative/StateTrackingMockExecutionGraph.java
##
@@ -0,0 +1,459 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.declarative;
+
+import org.apache.flink.api.common.ArchivedExecutionConfig;
+import org.apache.flink.api.common.ExecutionConfig;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.api.common.JobStatus;
+import org.apache.flink.api.common.accumulators.Accumulator;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.accumulators.AccumulatorSnapshot;
+import org.apache.flink.runtime.accumulators.StringifiedAccumulatorResult;
+import org.apache.flink.runtime.blob.BlobWriter;
+import org.apache.flink.runtime.blob.PermanentBlobKey;
+import org.apache.flink.runtime.checkpoint.CheckpointCoordinator;
+import org.apache.flink.runtime.checkpoint.CheckpointIDCounter;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsTracker;
+import org.apache.flink.runtime.checkpoint.CheckpointsCleaner;
+import org.apache.flink.runtime.checkpoint.CompletedCheckpointStore;
+import org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook;
+import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.ErrorInfo;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionAttemptID;
+import org.apache.flink.runtime.executiongraph.ExecutionDeploymentListener;
+import org.apache.flink.runtime.executiongraph.ExecutionGraph;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.executiongraph.IntermediateResult;
+import org.apache.flink.runtime.executiongraph.JobInformation;
+import org.apache.flink.runtime.executiongraph.JobStatusListener;
+import org.apache.flink.runtime.executiongraph.TaskExecutionStateTransition;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.ResultPartitionAvailabilityChecker;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.partitionrelease.PartitionReleaseStrategy;
+import org.apache.flink.runtime.io.network.partition.JobMasterPartitionTracker;
+import org.apache.flink.runtime.io.network.partition.ResultPartitionID;
+import org.apache.flink.runtime.jobgraph.IntermediateDataSetID;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+import org.apache.flink.runtime.jobgraph.ScheduleMode;
+import
org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration;
+import org.apache.flink.runtime.query.KvStateLocationRegistry;
+import org.apache.flink.runtime.scheduler.InternalFailuresListener;
+import org.apache.flink.runtime.scheduler.strategy.SchedulingTopology;
+import org.apache.flink.runtime.shuffle.ShuffleMaster;
+import org.apache.flink.runtime.state.CheckpointStorage;
+import org.apache.flink.runtime.state.StateBackend;
+import org.apache.flink.types.Either;
+import org.apache.flink.util.OptionalFailure;
+import org.apache.flink.util.SerializedValue;
+
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
+import java.util.concurrent.ForkJoinPool;
+
+/**
+ * Mocked ExecutionGraph which (partially) tracks the job status, and provides
some basic mocks to
+ * create an {@link
org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph} from this
+ * ExecutionGraph.
+ */
+class StateTrackingMockExecutionGraph implements ExecutionGraph {
+private JobStatus state = JobStatus.INITIALIZING;
+private final
[GitHub] [flink] zentol commented on a change in pull request #14950: [FLINK-21347][coordination] Extract interface from ExecutionGraph
zentol commented on a change in pull request #14950:
URL: https://github.com/apache/flink/pull/14950#discussion_r578975860
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/declarative/StateTrackingMockExecutionGraph.java
##
@@ -0,0 +1,459 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.scheduler.declarative;
+
+import org.apache.flink.api.common.ArchivedExecutionConfig;
+import org.apache.flink.api.common.ExecutionConfig;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.api.common.JobStatus;
+import org.apache.flink.api.common.accumulators.Accumulator;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.JobException;
+import org.apache.flink.runtime.accumulators.AccumulatorSnapshot;
+import org.apache.flink.runtime.accumulators.StringifiedAccumulatorResult;
+import org.apache.flink.runtime.blob.BlobWriter;
+import org.apache.flink.runtime.blob.PermanentBlobKey;
+import org.apache.flink.runtime.checkpoint.CheckpointCoordinator;
+import org.apache.flink.runtime.checkpoint.CheckpointIDCounter;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot;
+import org.apache.flink.runtime.checkpoint.CheckpointStatsTracker;
+import org.apache.flink.runtime.checkpoint.CheckpointsCleaner;
+import org.apache.flink.runtime.checkpoint.CompletedCheckpointStore;
+import org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook;
+import org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.ErrorInfo;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionAttemptID;
+import org.apache.flink.runtime.executiongraph.ExecutionDeploymentListener;
+import org.apache.flink.runtime.executiongraph.ExecutionGraph;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.executiongraph.IntermediateResult;
+import org.apache.flink.runtime.executiongraph.JobInformation;
+import org.apache.flink.runtime.executiongraph.JobStatusListener;
+import org.apache.flink.runtime.executiongraph.TaskExecutionStateTransition;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.ResultPartitionAvailabilityChecker;
+import
org.apache.flink.runtime.executiongraph.failover.flip1.partitionrelease.PartitionReleaseStrategy;
+import org.apache.flink.runtime.io.network.partition.JobMasterPartitionTracker;
+import org.apache.flink.runtime.io.network.partition.ResultPartitionID;
+import org.apache.flink.runtime.jobgraph.IntermediateDataSetID;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+import org.apache.flink.runtime.jobgraph.ScheduleMode;
+import
org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration;
+import org.apache.flink.runtime.query.KvStateLocationRegistry;
+import org.apache.flink.runtime.scheduler.InternalFailuresListener;
+import org.apache.flink.runtime.scheduler.strategy.SchedulingTopology;
+import org.apache.flink.runtime.shuffle.ShuffleMaster;
+import org.apache.flink.runtime.state.CheckpointStorage;
+import org.apache.flink.runtime.state.StateBackend;
+import org.apache.flink.types.Either;
+import org.apache.flink.util.OptionalFailure;
+import org.apache.flink.util.SerializedValue;
+
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.Executor;
+import java.util.concurrent.ForkJoinPool;
+
+/**
+ * Mocked ExecutionGraph which (partially) tracks the job status, and provides
some basic mocks to
+ * create an {@link
org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph} from this
+ * ExecutionGraph.
+ */
+class StateTrackingMockExecutionGraph implements ExecutionGraph {
+private JobStatus state = JobStatus.INITIALIZING;
+private final
[GitHub] [flink] flinkbot edited a comment on pull request #14868: [FLINK-21326][runtime] Optimize building topology when initializing ExecutionGraph
flinkbot edited a comment on pull request #14868: URL: https://github.com/apache/flink/pull/14868#issuecomment-773192044 ## CI report: * 997d4dddfc4d6ae2f45d80102655d02f09168d1b Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13480) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21380) Restarting/Canceling/Failing expose globally-terminal EG
[
https://issues.apache.org/jira/browse/FLINK-21380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-21380.
Resolution: Fixed
master: 2a78440bd333a68c24acfeea016f35ec63bbe3f9
> Restarting/Canceling/Failing expose globally-terminal EG
>
>
> Key: FLINK-21380
> URL: https://issues.apache.org/jira/browse/FLINK-21380
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> When the declarative scheduler is in the Restarting state it can expose a
> fully Canceled EG, because it only overrides {{getJobStatus}}, but the
> archived execution graph still contains the Canceled job state.
> This breaks the contract that the EG only transitions into a terminal state
> once, and never transitions into another state afterwards.
> Related to that the Canceling/Failing states can also expose such an EG
> because {{State#onGloballyTerminalState}} is called asynchronously. A
> globally-terminal EG should only be visible in the Finished state however.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] zentol merged pull request #14951: [FLINK-21380][coordination] Hide terminal ExecutionGraph in StateWithExecutionGraph
zentol merged pull request #14951: URL: https://github.com/apache/flink/pull/14951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21360) Add resourceTimeout configuration
[ https://issues.apache.org/jira/browse/FLINK-21360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler closed FLINK-21360. Resolution: Fixed master: 3dbb18491aa81af29f0e3b3ee0b6dd58ce02ef76 > Add resourceTimeout configuration > - > > Key: FLINK-21360 > URL: https://issues.apache.org/jira/browse/FLINK-21360 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: Robert Metzger >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > resourceTimeout is currently a hardcoded value. Make it configurable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zentol merged pull request #14928: [FLINK-21360][coordination] Make resource timeout configurable
zentol merged pull request #14928: URL: https://github.com/apache/flink/pull/14928 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-21414) javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
Bhagi created FLINK-21414:
-
Summary: javax.net.ssl.SSLHandshakeException: Received fatal
alert: certificate_unknown
Key: FLINK-21414
URL: https://issues.apache.org/jira/browse/FLINK-21414
Project: Flink
Issue Type: Bug
Components: Deployment / Kubernetes
Affects Versions: 1.12.0
Environment: flink@flink-jobmanager-5d9594b76c-t6cc8:~$ cat
/etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
Reporter: Bhagi
Fix For: 1.12.0
Attachments: image-2021-02-19-12-26-27-296.png
Enabled High availability for FLink with kubernetes API with Leader election
and retirieval mechanism,After configuring HA parameters with Leader election
and Leader retrival ..I started getting this error in job manager logs.
+error logs in job manager logs:+
at
org.apache.flink.shaded.netty4.io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
... 16 more
2021-02-19 06:11:04,358 WARN
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint [] - Unhandled
exception
org.apache.flink.shaded.netty4.io.netty.handler.codec.DecoderException:
javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
~[flink-dist_2.11-1.12.0.jar:1.12.0]
at org
+UI is throwing error:+
{"errors":["Service temporarily unavailable due to an ongoing leader election.
Please refresh."]}[link title|[http://example.com]|http://example.com]/]
!image-2021-02-19-12-26-27-296.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14897: [FLINK-21221][runtime] Deduplication for multiple ResourceCounters
flinkbot edited a comment on pull request #14897: URL: https://github.com/apache/flink/pull/14897#issuecomment-775050448 ## CI report: * 06c2bdca6652a653ece66c02c5a96d79326662e6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13481) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14798: [FLINK-21187] Provide exception history for root causes
flinkbot edited a comment on pull request #14798: URL: https://github.com/apache/flink/pull/14798#issuecomment-769223911 ## CI report: * af8de65a4905ae47ab704a75acfcd1b2e897d915 UNKNOWN * 772e076ed9d773e16713236c942f4e30b659d6eb UNKNOWN * 854b645872b5596d2cc618c65dac0228c616eec2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13478) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
flinkbot edited a comment on pull request #14943: URL: https://github.com/apache/flink/pull/14943#issuecomment-779315285 ## CI report: * f80b97c479ec82663e971553c54c8a6cda3122ff UNKNOWN * c9be932fec6390ffb103c000550cbc85747c266e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13461) * 8584b9ca7b9cf29f0f845aafe422f356e068cfe6 UNKNOWN * 236b938b10e8bc442e905bb5309ca0eb9f238eba Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13485) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
flinkbot edited a comment on pull request #14943: URL: https://github.com/apache/flink/pull/14943#issuecomment-779315285 ## CI report: * f80b97c479ec82663e971553c54c8a6cda3122ff UNKNOWN * c9be932fec6390ffb103c000550cbc85747c266e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13461) * 8584b9ca7b9cf29f0f845aafe422f356e068cfe6 UNKNOWN * 236b938b10e8bc442e905bb5309ca0eb9f238eba UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
flinkbot edited a comment on pull request #14943: URL: https://github.com/apache/flink/pull/14943#issuecomment-779315285 ## CI report: * f80b97c479ec82663e971553c54c8a6cda3122ff UNKNOWN * c9be932fec6390ffb103c000550cbc85747c266e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13461) * 8584b9ca7b9cf29f0f845aafe422f356e068cfe6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] curcur commented on pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on pull request #14943: URL: https://github.com/apache/flink/pull/14943#issuecomment-781830344 rebase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943: URL: https://github.com/apache/flink/pull/14943#discussion_r578936411 ## File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/changelog/state/ChangelogAggregatingState.java ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.changelog.state; Review comment: I do not mind, either way. Let's maybe decide this after your API changes are finalized as well? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14964: [FLINK-21410][docs] Document checkpoint interval trade-offs
flinkbot edited a comment on pull request #14964: URL: https://github.com/apache/flink/pull/14964#issuecomment-781637879 ## CI report: * 63cafd0fe1c8c1c43e69f5acf5faf1256073dcea Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13477) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578932063
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java
##
@@ -242,6 +265,53 @@ public static StateBackend
fromApplicationOrConfigOrDefault(
return backend;
}
+public static StateBackend loadStateBackend(
+@Nullable StateBackend fromApplication,
+Configuration config,
+ClassLoader classLoader,
+@Nullable Logger logger)
+throws IllegalConfigurationException, DynamicCodeLoadingException,
IOException {
+
+final StateBackend backend =
+fromApplicationOrConfigOrDefault(fromApplication, config,
classLoader, logger);
+
+if (config.get(CheckpointingOptions.ENABLE_STATE_CHANGE_LOG)) {
+
+Preconditions.checkArgument(
+backend instanceof DelegatedStateBackend, "backend is not
delegable");
+
+LOG.info(
+"Delegate State Backend is used, and the root State
Backend is {}",
+backend.getClass().getSimpleName());
+
+// ChangelogStateBackend resides in a separate module, load it
using reflection
+try {
+Constructor constructor =
+Class.forName(CHANGELOG_STATE_BACKEND, false,
classLoader)
+.asSubclass(DelegateStateBackend.class)
+.getConstructor(DelegatedStateBackend.class);
+
+Class.forName(ROCKSDB_STATE_BACKEND_FACTORY, false,
classLoader);
Review comment:
Thanks for the careful review, it may cause bugs that are
strange/difficult to debug.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578931859
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java
##
@@ -242,6 +265,53 @@ public static StateBackend
fromApplicationOrConfigOrDefault(
return backend;
}
+public static StateBackend loadStateBackend(
+@Nullable StateBackend fromApplication,
+Configuration config,
+ClassLoader classLoader,
+@Nullable Logger logger)
+throws IllegalConfigurationException, DynamicCodeLoadingException,
IOException {
+
+final StateBackend backend =
+fromApplicationOrConfigOrDefault(fromApplication, config,
classLoader, logger);
+
+if (config.get(CheckpointingOptions.ENABLE_STATE_CHANGE_LOG)) {
+
+Preconditions.checkArgument(
+backend instanceof DelegatedStateBackend, "backend is not
delegable");
+
+LOG.info(
+"Delegate State Backend is used, and the root State
Backend is {}",
+backend.getClass().getSimpleName());
+
+// ChangelogStateBackend resides in a separate module, load it
using reflection
+try {
+Constructor constructor =
+Class.forName(CHANGELOG_STATE_BACKEND, false,
classLoader)
+.asSubclass(DelegateStateBackend.class)
+.getConstructor(DelegatedStateBackend.class);
+
+Class.forName(ROCKSDB_STATE_BACKEND_FACTORY, false,
classLoader);
Review comment:
I've added it to test classloader, it should be removed. Sorry.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578931575
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegateStateBackend.java
##
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+/**
+ * An abstract base class for delegated state backend.
+ *
+ * As its name, it should include a state backend to delegate, which can be
one of the following:
+ * {@code MemoryStateBackend}, {code FsStateBackend} and {@code
RocksDBStateBackend}.
+ */
+@Internal
+public abstract class DelegateStateBackend implements StateBackend,
java.io.Serializable {
Review comment:
Explained above already.
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegateStateBackend.java
##
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+/**
+ * An abstract base class for delegated state backend.
+ *
+ * As its name, it should include a state backend to delegate, which can be
one of the following:
+ * {@code MemoryStateBackend}, {code FsStateBackend} and {@code
RocksDBStateBackend}.
+ */
+@Internal
+public abstract class DelegateStateBackend implements StateBackend,
java.io.Serializable {
Review comment:
Explained above
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578904214
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java
##
@@ -204,6 +222,11 @@ public static StateBackend
fromApplicationOrConfigOrDefault(
// (1) the application defined state backend has precedence
if (fromApplication != null) {
+
+checkArgument(
+!(fromApplication instanceof DelegateStateBackend),
+"DelegateStateBackend can not be delegated!");
Review comment:
Yes
The intention here is to make sure `loadStateBackendFromConfig` only returns
unwrapped state backend.
I can add another check before actual wrapping.
Why not?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * c7e6b28b249f85cf52740d5201a769e0982a60aa UNKNOWN * bebd298009b12a9d5ac6518902f5534f8e00ff32 UNKNOWN * c69c79df69c5fa912b6d16e3cfa40edc3b9b82f2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13438) * eb6c10b0d339bfc92a540314e7c58cbf11a70dd9 UNKNOWN * ea70d7913106bb528aaf3e775679619a9f241be9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13484) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * c7e6b28b249f85cf52740d5201a769e0982a60aa UNKNOWN * bebd298009b12a9d5ac6518902f5534f8e00ff32 UNKNOWN * c69c79df69c5fa912b6d16e3cfa40edc3b9b82f2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13438) * eb6c10b0d339bfc92a540314e7c58cbf11a70dd9 UNKNOWN * ea70d7913106bb528aaf3e775679619a9f241be9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
> I'm curious why didn't you add unwrap method to the StateBackend
interface?
I am not saying `unwrap` is not good, I just eel unwrap is not a concept
belonging to `StateBackend `, instead it is a concept belonging to
`DelegateStateBackend`
Currently, you can think of `getDelegatedStateBackend` in
`DelegateStateBackend` a simplified version of `unwrap`.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the problem: for example, the match check can be
done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> Every new backend must be marked explicitly
Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
> I'm curious why didn't you add unwrap method to the StateBackend
interface?
I am not saying `unwrap` is not good, I just eel unwrap is not a concept
belonging to `StateBackend `, instead it is a concept belonging to
`DelegateStateBackend`
Currently, you can think of `getDelegatedStateBackend` in
`DelegateStateBackend` a simplified version of `unwrap`.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the first problem: for example, the match check can
be done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> Every new backend must be marked explicitly
Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
> I'm curious why didn't you add unwrap method to the StateBackend
interface?
I am not saying `unwrap` is not good, I just eel unwrap is not a concept
belonging to `StateBackend `, instead it is a concept belonging to
`DelegateStateBackend`
Currently, you can think of `getDelegatedStateBackend` in
`DelegateStateBackend` a simplified version of `unwrap`.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
For the delegatee/delegated matching, I do not see how adding unwrap in
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
1. I am not saying `unwrap` is not good, I just eel unwrap is not a
concept belonging to `StateBackend `, instead it is a concept belonging to
`DelegateStateBackend`
2. Currently, you can think of `getDelegatedStateBackend` in
`DelegateStateBackend` a simplified version of `unwrap`.
> I'm curious why didn't you add unwrap method to the StateBackend interface?
1. For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the first problem: for example, the match check can
be done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
2. Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
> Every new backend must be marked explicitly
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
1. I am not saying `unwrap` is not good, I just eel unwrap is not a
concept belonging to `StateBackend `, instead it is a concept belonging to
DelegateStateBackend`
2. Currently, you can think of getDelegatedStateBackend in
DelegateStateBackend a simplified version of unwrapping.
> I'm curious why didn't you add unwrap method to the StateBackend interface?
1. For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the first problem: for example, the match check can
be done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
2. Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
> Every new backend must be marked explicitly
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
1. I am not saying `unwrap` is not good, but instead, I feel unwrap is
not a concept belonging to `StateBackend `, instead it is a concept belonging
to DelegateStateBackend`
2. Currently, you can think of getDelegatedStateBackend in
DelegateStateBackend a simplified version of unwrapping.
1. For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the first problem: for example, the match check can
be done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
2. Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
> Every new backend must be marked explicitly
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578909351
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/delegate/DelegatedStateBackend.java
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state.delegate;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.runtime.state.StateBackend;
+
+@Internal
+/**
+ * An interface for DelegatedStateBackend. A state backend to be delegated
must implement this
+ * interface.
+ */
+public interface DelegatedStateBackend extends StateBackend {}
Review comment:
1. I am saying `unwrap` is not good, but instead, I feel unwrap is not a
concept belonging to `StateBackend `, instead it is a concept belonging to
DelegateStateBackend`
2. Currently, you can think of getDelegatedStateBackend in
DelegateStateBackend a simplified version of unwrapping.
1. For the delegatee/delegated matching, I do not see how adding unwrap in
`StateBackend` can be simpler than the current approach; and there are several
different ways we can solve the first problem: for example, the match check can
be done when loading explicit delegatee; Or we can put the matching logic in
DelegateStateBackend.
> If we add some new delegatee then it's impossible to distinguish for which
delegatee given backend can be delegated. For example, we add a new delegating
backend with extended logging. How RocksDbBackend can signal that it can NOT
work with it?
2. Yes, it should. Not every state backend is delegable. When implementing a
new state backend, it should register the information somewhere whether it is
delegable and who can delegate it. Such logic should be put within
DelegatedStateBackend, and DelegateStateBackend should use such information to
double check. That's the intension to introduce Interface DelegatedStateBackend
and Abstract Class DelegateStateBackend
> Every new backend must be marked explicitly
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] gaoyunhaii commented on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-781798040 Hi Roman @rkhachatryan very thanks for the review! I updated the PR according to the comments~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gaoyunhaii commented on a change in pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on a change in pull request #14740:
URL: https://github.com/apache/flink/pull/14740#discussion_r578907964
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/DefaultCheckpointPlanCalculator.java
##
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.checkpoint;
+
+import org.apache.flink.annotation.VisibleForTesting;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionEdge;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.jobgraph.DistributionPattern;
+import org.apache.flink.runtime.jobgraph.JobEdge;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.stream.Collectors;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * Default implementation for {@link CheckpointPlanCalculator}. If all tasks
are running, it
+ * directly marks all the sources as tasks to trigger, otherwise it would try
to find the running
+ * tasks without running processors as tasks to trigger.
+ */
+public class DefaultCheckpointPlanCalculator implements
CheckpointPlanCalculator {
+
+private final JobID jobId;
+
+private final CheckpointPlanCalculatorContext context;
+
+private final List jobVerticesInTopologyOrder = new
ArrayList<>();
+
+private final List allTasks = new ArrayList<>();
+
+private final List sourceTasks = new ArrayList<>();
+
+public DefaultCheckpointPlanCalculator(
+JobID jobId,
+CheckpointPlanCalculatorContext context,
+Iterable jobVerticesInTopologyOrderIterable) {
+
+this.jobId = checkNotNull(jobId);
+this.context = checkNotNull(context);
+
+checkNotNull(jobVerticesInTopologyOrderIterable);
+jobVerticesInTopologyOrderIterable.forEach(
+jobVertex -> {
+jobVerticesInTopologyOrder.add(jobVertex);
+
allTasks.addAll(Arrays.asList(jobVertex.getTaskVertices()));
+
+if (jobVertex.getJobVertex().isInputVertex()) {
+
sourceTasks.addAll(Arrays.asList(jobVertex.getTaskVertices()));
+}
+});
+}
+
+@Override
+public CompletableFuture calculateCheckpointPlan() {
+return CompletableFuture.supplyAsync(
+() -> {
+try {
+checkAllTasksInitiated();
+
+CheckpointPlan result =
+context.hasFinishedTasks()
+? calculateAfterTasksFinished()
+: calculateWithAllTasksRunning();
+
+checkTasksStarted(result.getTasksToTrigger());
+
+return result;
+} catch (Throwable throwable) {
+throw new CompletionException(throwable);
+}
+},
+context.getMainExecutor());
+}
+
+/**
+ * Checks if all tasks are attached with the current Execution already.
This method should be
+ * called from JobMaster main thread executor.
+ *
+ * @throws CheckpointException if some tasks do not have attached
Execution.
+ */
+private void checkAllTasksInitiated() throws CheckpointException {
+for (ExecutionVertex task : allTasks) {
+if (task.getCurrentExecutionAttempt() == null) {
+throw new CheckpointException(
+String.format(
+"task %s of job %s is not being executed
[GitHub] [flink] flinkbot edited a comment on pull request #14891: [FLINK-21289][deployment] FIX missing load pipeline.classpaths in app…
flinkbot edited a comment on pull request #14891: URL: https://github.com/apache/flink/pull/14891#issuecomment-774628149 ## CI report: * 1976f8fa0310d74fccbd2ccf0ccb6f7b904a60a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13430) * c4db5e1f7e1adbb6b4c21c4dccf211728fbc2f78 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13483) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * c7e6b28b249f85cf52740d5201a769e0982a60aa UNKNOWN * bebd298009b12a9d5ac6518902f5534f8e00ff32 UNKNOWN * c69c79df69c5fa912b6d16e3cfa40edc3b9b82f2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13438) * eb6c10b0d339bfc92a540314e7c58cbf11a70dd9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578904214
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java
##
@@ -204,6 +222,11 @@ public static StateBackend
fromApplicationOrConfigOrDefault(
// (1) the application defined state backend has precedence
if (fromApplication != null) {
+
+checkArgument(
+!(fromApplication instanceof DelegateStateBackend),
+"DelegateStateBackend can not be delegated!");
Review comment:
Why not?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] curcur commented on a change in pull request #14943: [FLINK-21354] Implement a StateChangelogStateBackend to forward state changes to St…
curcur commented on a change in pull request #14943:
URL: https://github.com/apache/flink/pull/14943#discussion_r578903917
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStorageLoader.java
##
@@ -162,14 +163,19 @@ public static CheckpointStorage load(
Preconditions.checkNotNull(classLoader, "classLoader");
Preconditions.checkNotNull(configuredStateBackend, "statebackend");
-if (configuredStateBackend instanceof CheckpointStorage) {
+StateBackend rootStateBackend =
+(configuredStateBackend instanceof DelegateStateBackend)
+? ((DelegateStateBackend)
configuredStateBackend).getDelegatedStateBackend()
+: configuredStateBackend;
Review comment:
DelegateStateBackend (Abstract class) is added exactly for that purpose.
New delegating/proxying only needs to extends DelegateStateBackend, like
ChangelogStateBackend.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14891: [FLINK-21289][deployment] FIX missing load pipeline.classpaths in app…
flinkbot edited a comment on pull request #14891: URL: https://github.com/apache/flink/pull/14891#issuecomment-774628149 ## CI report: * 1976f8fa0310d74fccbd2ccf0ccb6f7b904a60a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13430) * c4db5e1f7e1adbb6b4c21c4dccf211728fbc2f78 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21413) TtlMapState and TtlListState cannot be clean completely with Filesystem StateBackend
[
https://issues.apache.org/jira/browse/FLINK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiayi Liao updated FLINK-21413:
---
Description:
Take the #TtlMapState as an example,
{code:java}
public Map> getUnexpiredOrNull(@Nonnull Map>
ttlValue) {
Map> unexpired = new HashMap<>();
TypeSerializer> valueSerializer =
((MapSerializer>)
original.getValueSerializer()).getValueSerializer();
for (Map.Entry> e : ttlValue.entrySet()) {
if (!expired(e.getValue())) {
// we have to do the defensive copy to update the value
unexpired.put(e.getKey(),
valueSerializer.copy(e.getValue()));
}
}
return ttlValue.size() == unexpired.size() ? ttlValue : unexpired;
}
{code}
The returned value will never be null and the #StateEntry will exists forever,
which leads to memory leak if the key's range of the stream is very large.
Below we can see that 20+ millison uncleared TtlStateMap could take up several
GB memory.
!image-2021-02-19-11-13-58-672.png!
was:
Take the #TtlMapState as an example,
{code:java}
public Map> getUnexpiredOrNull(@Nonnull Map>
ttlValue) {
Map> unexpired = new HashMap<>();
TypeSerializer> valueSerializer =
((MapSerializer>)
original.getValueSerializer()).getValueSerializer();
for (Map.Entry> e : ttlValue.entrySet()) {
if (!expired(e.getValue())) {
// we have to do the defensive copy to update the value
unexpired.put(e.getKey(),
valueSerializer.copy(e.getValue()));
}
}
return ttlValue.size() == unexpired.size() ? ttlValue : unexpired;
}
{code}
The returned value will never be null and the #StateEntry will exists forever,
which leads to memory leak if the key's range of the stream is very large.
Below we can see that 20+ millison uncleared TtlStateMap could take up several
GB memory.
!image-2021-02-19-11-13-38-691.png!
> TtlMapState and TtlListState cannot be clean completely with Filesystem
> StateBackend
>
>
> Key: FLINK-21413
> URL: https://issues.apache.org/jira/browse/FLINK-21413
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
>Affects Versions: 1.9.0
>Reporter: Jiayi Liao
>Priority: Major
> Attachments: image-2021-02-19-11-13-58-672.png
>
>
> Take the #TtlMapState as an example,
>
> {code:java}
> public Map> getUnexpiredOrNull(@Nonnull Map TtlValue> ttlValue) {
> Map> unexpired = new HashMap<>();
> TypeSerializer> valueSerializer =
> ((MapSerializer>)
> original.getValueSerializer()).getValueSerializer();
> for (Map.Entry> e : ttlValue.entrySet()) {
> if (!expired(e.getValue())) {
> // we have to do the defensive copy to update the
> value
> unexpired.put(e.getKey(),
> valueSerializer.copy(e.getValue()));
> }
> }
> return ttlValue.size() == unexpired.size() ? ttlValue : unexpired;
> }
> {code}
>
> The returned value will never be null and the #StateEntry will exists
> forever, which leads to memory leak if the key's range of the stream is very
> large. Below we can see that 20+ millison uncleared TtlStateMap could take up
> several GB memory.
>
> !image-2021-02-19-11-13-58-672.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21413) TtlMapState and TtlListState cannot be clean completely with Filesystem StateBackend
[
https://issues.apache.org/jira/browse/FLINK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiayi Liao updated FLINK-21413:
---
Attachment: image-2021-02-19-11-13-58-672.png
> TtlMapState and TtlListState cannot be clean completely with Filesystem
> StateBackend
>
>
> Key: FLINK-21413
> URL: https://issues.apache.org/jira/browse/FLINK-21413
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
>Affects Versions: 1.9.0
>Reporter: Jiayi Liao
>Priority: Major
> Attachments: image-2021-02-19-11-13-58-672.png
>
>
> Take the #TtlMapState as an example,
>
> {code:java}
> public Map> getUnexpiredOrNull(@Nonnull Map TtlValue> ttlValue) {
> Map> unexpired = new HashMap<>();
> TypeSerializer> valueSerializer =
> ((MapSerializer>)
> original.getValueSerializer()).getValueSerializer();
> for (Map.Entry> e : ttlValue.entrySet()) {
> if (!expired(e.getValue())) {
> // we have to do the defensive copy to update the
> value
> unexpired.put(e.getKey(),
> valueSerializer.copy(e.getValue()));
> }
> }
> return ttlValue.size() == unexpired.size() ? ttlValue : unexpired;
> }
> {code}
>
> The returned value will never be null and the #StateEntry will exists
> forever, which leads to memory leak if the key's range of the stream is very
> large. Below we can see that 20+ millison uncleared TtlStateMap could take up
> several GB memory.
>
> !image-2021-02-19-11-13-38-691.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-21413) TtlMapState and TtlListState cannot be clean completely with Filesystem StateBackend
Jiayi Liao created FLINK-21413:
--
Summary: TtlMapState and TtlListState cannot be clean completely
with Filesystem StateBackend
Key: FLINK-21413
URL: https://issues.apache.org/jira/browse/FLINK-21413
Project: Flink
Issue Type: Bug
Components: Runtime / State Backends
Affects Versions: 1.9.0
Reporter: Jiayi Liao
Take the #TtlMapState as an example,
{code:java}
public Map> getUnexpiredOrNull(@Nonnull Map>
ttlValue) {
Map> unexpired = new HashMap<>();
TypeSerializer> valueSerializer =
((MapSerializer>)
original.getValueSerializer()).getValueSerializer();
for (Map.Entry> e : ttlValue.entrySet()) {
if (!expired(e.getValue())) {
// we have to do the defensive copy to update the value
unexpired.put(e.getKey(),
valueSerializer.copy(e.getValue()));
}
}
return ttlValue.size() == unexpired.size() ? ttlValue : unexpired;
}
{code}
The returned value will never be null and the #StateEntry will exists forever,
which leads to memory leak if the key's range of the stream is very large.
Below we can see that 20+ millison uncleared TtlStateMap could take up several
GB memory.
!image-2021-02-19-11-13-38-691.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-21412) pyflink DataTypes.DECIMAL is not available
awayne created FLINK-21412:
--
Summary: pyflink DataTypes.DECIMAL is not available
Key: FLINK-21412
URL: https://issues.apache.org/jira/browse/FLINK-21412
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.12.1
Environment: python 3.7.5
pyflink 1.12.1
Reporter: awayne
when i use DataTypes.DECIMAL in udaf
File "/home/ubuntu/pyflenv/lib/python3.7/site-packages/pyflink/table/types.py",
line 2025, in _to_java_data_type
_to_java_data_type(data_type._element_type))
File
"/home/ubuntu/pyflenv/lib/python3.7/site-packages/pyflink/table/types.py", line
1964, in _to_java_data_type
j_data_type = JDataTypes.Decimal(data_type.precision, data_type.scale)
File "/home/ubuntu/pyflenv/lib/python3.7/site-packages/py4j/java_gateway.py",
line 1516, in __getattr__
"\{0}.\{1} does not exist in the JVM".format(self._fqn, name))
py4j.protocol.Py4JError: org.apache.flink.table.api.DataTypes.Decimal does not
exist in the JVM
in pyflink\table\types.py
line 1963-1964
elif isinstance(data_type, DecimalType):
j_data_type = JDataTypes.{color:#FF}Decimal{color}(data_type.precision,
data_type.scale)
in java should be called "DECIMAL"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-21370) flink-1.12.1 - JDBCExecutionOptions defaults config JDBCDynamicTableFactory default config is not consistent
[
https://issues.apache.org/jira/browse/FLINK-21370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 closed FLINK-21370.
--
Resolution: Not A Problem
> flink-1.12.1 - JDBCExecutionOptions defaults config JDBCDynamicTableFactory
> default config is not consistent
>
>
> Key: FLINK-21370
> URL: https://issues.apache.org/jira/browse/FLINK-21370
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC
>Affects Versions: 1.12.1
>Reporter: 谢波
>Priority: Major
> Labels: pull-request-available
>
> When i test jdbc sink with kafka src, the data is not sink to db, and then i
> find out JDBCExecutionOptions defaults config JDBCDynamicTableFactory default
> config is not consistent.
> JDBCExecutionOptions
> public static final int DEFAULT_MAX_RETRY_TIMES = 3;
> private static final int DEFAULT_INTERVAL_MILLIS = 0;
> public static final int DEFAULT_SIZE = 5000;
>
> JDBCDynamicTableFactory
>
> // write config options
> private static final ConfigOption SINK_BUFFER_FLUSH_MAX_ROWS =
> ConfigOptions.key("sink.buffer-flush.max-rows")
> .intType()
> .defaultValue(100)
> .withDescription(
> "the flush max size (includes all append, upsert and delete records), over
> this number"
> + " of records, will flush data. The default value is 100.");
> private static final ConfigOption SINK_BUFFER_FLUSH_INTERVAL =
> ConfigOptions.key("sink.buffer-flush.interval")
> .durationType()
> .defaultValue(Duration.ofSeconds(1))
> .withDescription(
> "the flush interval mills, over this time, asynchronous threads will flush
> data. The "
> + "default value is 1s.");
> private static final ConfigOption SINK_MAX_RETRIES =
> ConfigOptions.key("sink.max-retries")
> .intType()
> .defaultValue(3)
> .withDescription("the max retry times if writing records to database
> failed.");
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-21049) query_configuration.html content out date
[
https://issues.apache.org/jira/browse/FLINK-21049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 closed FLINK-21049.
--
Fix Version/s: 1.13.0
Resolution: Fixed
> query_configuration.html content out date
> -
>
> Key: FLINK-21049
> URL: https://issues.apache.org/jira/browse/FLINK-21049
> Project: Flink
> Issue Type: Bug
> Components: Documentation, Table SQL / API
>Affects Versions: 1.12.1
>Reporter: 谢波
>Priority: Major
> Fix For: 1.13.0
>
>
> [https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/streaming/query_configuration.html]
> code samples is out date.
>
> {{tConfig.setIdleStateRetentionTime(Time.hours(12), Time.hours(24))}}
> {{}}
> Specifies a minimum and a maximum time interval for how long idle state,
> i.e., state which was not updated, will be retained. State will never be
> cleared until it was idle for less than the minimum time and will never be
> kept if it was idle for more than the maximum time.
> When new data arrives for previously cleaned-up state, the new data will be
> handled as if it was the first data. This can result in previous results
> being overwritten.
> Set to 0 (zero) to never clean-up the state.
> NOTE: Cleaning up state requires additional bookkeeping which becomes less
> expensive for larger differences of minTime and maxTime. The difference
> between minTime and maxTime must be at least 5 minutes.
> NOTE: Currently maxTime will be ignored and it will automatically derived
> from minTime as 1.5 x minTime.
> Deprecated
> use setIdleStateRetention(Duration) instead.
> Params:
> minTime – The minimum time interval for which idle state is retained. Set to
> 0 (zero) to never clean-up the state.
> maxTime – The maximum time interval for which idle state is retained. Must be
> at least 5 minutes greater than minTime. Set to 0 (zero) to never clean-up
> the state
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] anlen321 commented on a change in pull request #14684: [FLINK-20460][Connector-HBase] Support async lookup for HBase connector
anlen321 commented on a change in pull request #14684:
URL: https://github.com/apache/flink/pull/14684#discussion_r57288
##
File path:
flink-connectors/flink-connector-hbase-base/src/main/java/org/apache/flink/connector/hbase/options/HBaseOptions.java
##
@@ -94,6 +94,33 @@
+ "Can be set to '0' to disable it. Note,
both 'sink.buffer-flush.max-size' and 'sink.buffer-flush.max-rows' "
+ "can be set to '0' with the flush
interval set allowing for complete async processing of buffered actions.");
+public static final ConfigOption LOOKUP_ASYNC =
+ConfigOptions.key("lookup.async")
+.booleanType()
+.defaultValue(false)
+.withDescription("whether to set async lookup.");
+
+public static final ConfigOption LOOKUP_CACHE_MAX_ROWS =
+ConfigOptions.key("lookup.cache.max-rows")
+.longType()
+.defaultValue(-1L)
+.withDescription(
+"the max number of rows of lookup cache, over this value, the
oldest rows will "
++ "be eliminated. \"cache.max-rows\" and \"cache.ttl\"
options must all be specified if any of them is "
++ "specified. Cache is not enabled as default.");
+
+public static final ConfigOption LOOKUP_CACHE_TTL =
+ConfigOptions.key("lookup.cache.ttl")
+.durationType()
+.defaultValue(Duration.ofSeconds(0))
+.withDescription("the cache time to live.");
+
+public static final ConfigOption LOOKUP_MAX_RETRIES =
+ConfigOptions.key("lookup.max-retries")
+.intType()
+.defaultValue(3)
+.withDescription("the max retry times if lookup database failed.");
Review comment:
@leonardBang hello ,Can I support cache for HBaseRowDataLookupFunction
in the current patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14944: [FLINK-21297] Support 'LOAD/UNLOAD MODULE' syntax
flinkbot edited a comment on pull request #14944: URL: https://github.com/apache/flink/pull/14944#issuecomment-779315396 ## CI report: * 2ee9ac64332d3547bf934d7cd17f73ee5404c8b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13459) * 073e7d7aea71ed9b4b2806821a7172a92b9f26ec Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13482) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14897: [FLINK-21221][runtime] Deduplication for multiple ResourceCounters
flinkbot edited a comment on pull request #14897: URL: https://github.com/apache/flink/pull/14897#issuecomment-775050448 ## CI report: * 35644082e06622b2182b6fb68d9a30df167d5c11 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13432) * 06c2bdca6652a653ece66c02c5a96d79326662e6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13481) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20427) Remove CheckpointConfig.setPreferCheckpointForRecovery because it can lead to data loss
[
https://issues.apache.org/jira/browse/FLINK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286823#comment-17286823
]
Rex Remind commented on FLINK-20427:
Cool, thanks!
> Remove CheckpointConfig.setPreferCheckpointForRecovery because it can lead to
> data loss
> ---
>
> Key: FLINK-20427
> URL: https://issues.apache.org/jira/browse/FLINK-20427
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, Runtime / Checkpointing
>Affects Versions: 1.12.0
>Reporter: Till Rohrmann
>Priority: Critical
> Fix For: 1.13.0
>
>
> The {{CheckpointConfig.setPreferCheckpointForRecovery}} allows to configure
> whether Flink prefers checkpoints for recovery if the
> {{CompletedCheckpointStore}} contains savepoints and checkpoints. This is
> problematic because due to this feature, Flink might prefer older checkpoints
> over newer savepoints for recovery. Since some components expect that the
> always the latest checkpoint/savepoint is used (e.g. the
> {{SourceCoordinator}}), it breaks assumptions and can lead to
> {{SourceSplits}} which are not read. This effectively means that the system
> loses data. Similarly, this behaviour can cause that exactly once sinks might
> output results multiple times which violates the processing guarantees.
> Hence, I believe that we should remove this setting because it changes
> Flink's behaviour in some very significant way potentially w/o the user
> noticing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] gaoyunhaii commented on a change in pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on a change in pull request #14740:
URL: https://github.com/apache/flink/pull/14740#discussion_r578879271
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -651,39 +681,52 @@ private void
startTriggeringCheckpoint(CheckpointTriggerRequest request) {
}
}
+private CompletableFuture calculateCheckpointPlan() {
+return checkpointPlanCalculator
+.calculateCheckpointPlan()
+// Disable checkpoints after tasks finished according to the
flag.
+.thenApplyAsync(
+plan -> {
+if (!allowCheckpointsAfterTasksFinished
+&& !plan.getFinishedTasks().isEmpty()) {
+throw new CompletionException(
Review comment:
I also think move the check before computing would be more reasonable. I
have modified accordingly. For the long run, we would remove the judgement
after we enabled the whole functionality~
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14944: [FLINK-21297] Support 'LOAD/UNLOAD MODULE' syntax
flinkbot edited a comment on pull request #14944: URL: https://github.com/apache/flink/pull/14944#issuecomment-779315396 ## CI report: * 2ee9ac64332d3547bf934d7cd17f73ee5404c8b1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13459) * 073e7d7aea71ed9b4b2806821a7172a92b9f26ec UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14897: [FLINK-21221][runtime] Deduplication for multiple ResourceCounters
flinkbot edited a comment on pull request #14897: URL: https://github.com/apache/flink/pull/14897#issuecomment-775050448 ## CI report: * 35644082e06622b2182b6fb68d9a30df167d5c11 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13432) * 06c2bdca6652a653ece66c02c5a96d79326662e6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gaoyunhaii commented on a change in pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on a change in pull request #14740:
URL: https://github.com/apache/flink/pull/14740#discussion_r578879271
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##
@@ -651,39 +681,52 @@ private void
startTriggeringCheckpoint(CheckpointTriggerRequest request) {
}
}
+private CompletableFuture calculateCheckpointPlan() {
+return checkpointPlanCalculator
+.calculateCheckpointPlan()
+// Disable checkpoints after tasks finished according to the
flag.
+.thenApplyAsync(
+plan -> {
+if (!allowCheckpointsAfterTasksFinished
+&& !plan.getFinishedTasks().isEmpty()) {
+throw new CompletionException(
Review comment:
I also think move the check before computing would be more reasonable. I
have modified accordingly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #14897: [FLINK-21221][runtime] Deduplication for multiple ResourceCounters
KarmaGYZ commented on pull request #14897: URL: https://github.com/apache/flink/pull/14897#issuecomment-781765733 Thanks for the new comments @tillrohrmann . PR updated according to it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #14897: [FLINK-21221][runtime] Deduplication for multiple ResourceCounters
KarmaGYZ commented on a change in pull request #14897:
URL: https://github.com/apache/flink/pull/14897#discussion_r578874000
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/BiDirectionalResourceToRequirementMapping.java
##
@@ -74,23 +81,26 @@ private static void internalDecrementCount(
"Attempting to decrement count of %s->%s, but
primary key was unknown.",
resourceProfile,
secondaryKey);
-resourceCounter.decrementCount(secondaryKey, decrement);
-return resourceCounter.isEmpty() ? null : resourceCounter;
+final ResourceCounter newCounter =
+resourceCounter.subtract(secondaryKey, decrement);
+return newCounter.isEmpty() ? null : newCounter;
});
}
-public Map
getResourcesFulfilling(ResourceProfile requirement) {
+public Collection>
getResourcesFulfilling(
Review comment:
Yes, I think we can directly return the `ResourceCounter` since it is
immutable now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] gaoyunhaii commented on a change in pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on a change in pull request #14740:
URL: https://github.com/apache/flink/pull/14740#discussion_r578872539
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/DefaultCheckpointPlanCalculator.java
##
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.checkpoint;
+
+import org.apache.flink.annotation.VisibleForTesting;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.execution.ExecutionState;
+import org.apache.flink.runtime.executiongraph.Execution;
+import org.apache.flink.runtime.executiongraph.ExecutionEdge;
+import org.apache.flink.runtime.executiongraph.ExecutionJobVertex;
+import org.apache.flink.runtime.executiongraph.ExecutionVertex;
+import org.apache.flink.runtime.jobgraph.DistributionPattern;
+import org.apache.flink.runtime.jobgraph.JobEdge;
+import org.apache.flink.runtime.jobgraph.JobVertex;
+import org.apache.flink.runtime.jobgraph.JobVertexID;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.stream.Collectors;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * Default implementation for {@link CheckpointPlanCalculator}. If all tasks
are running, it
+ * directly marks all the sources as tasks to trigger, otherwise it would try
to find the running
+ * tasks without running processors as tasks to trigger.
+ */
+public class DefaultCheckpointPlanCalculator implements
CheckpointPlanCalculator {
+
+private final JobID jobId;
+
+private final CheckpointPlanCalculatorContext context;
+
+private final List jobVerticesInTopologyOrder = new
ArrayList<>();
+
+private final List allTasks = new ArrayList<>();
+
+private final List sourceTasks = new ArrayList<>();
+
+public DefaultCheckpointPlanCalculator(
+JobID jobId,
+CheckpointPlanCalculatorContext context,
+Iterable jobVerticesInTopologyOrderIterable) {
+
+this.jobId = checkNotNull(jobId);
+this.context = checkNotNull(context);
+
+checkNotNull(jobVerticesInTopologyOrderIterable);
+jobVerticesInTopologyOrderIterable.forEach(
+jobVertex -> {
+jobVerticesInTopologyOrder.add(jobVertex);
+
allTasks.addAll(Arrays.asList(jobVertex.getTaskVertices()));
+
+if (jobVertex.getJobVertex().isInputVertex()) {
+
sourceTasks.addAll(Arrays.asList(jobVertex.getTaskVertices()));
+}
+});
+}
+
+@Override
+public CompletableFuture calculateCheckpointPlan() {
+return CompletableFuture.supplyAsync(
+() -> {
+try {
+checkAllTasksInitiated();
+
+CheckpointPlan result =
+context.hasFinishedTasks()
+? calculateAfterTasksFinished()
+: calculateWithAllTasksRunning();
+
+checkTasksStarted(result.getTasksToTrigger());
+
+return result;
+} catch (Throwable throwable) {
+throw new CompletionException(throwable);
+}
+},
+context.getMainExecutor());
+}
+
+/**
+ * Checks if all tasks are attached with the current Execution already.
This method should be
+ * called from JobMaster main thread executor.
+ *
+ * @throws CheckpointException if some tasks do not have attached
Execution.
+ */
+private void checkAllTasksInitiated() throws CheckpointException {
+for (ExecutionVertex task : allTasks) {
+if (task.getCurrentExecutionAttempt() == null) {
+throw new CheckpointException(
+String.format(
+"task %s of job %s is not being executed
