[GitHub] [flink] flinkbot commented on pull request #15890: [BP-1.12][FLINK-17857]Make K8s e2e tests could run on Mac
flinkbot commented on pull request #15890: URL: https://github.com/apache/flink/pull/15890#issuecomment-837869596 ## CI report: * a3b9bae7fb2a6e4e8b65c3a78b0c1d12591a44ce UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15889: [BP-1.13][FLINK-17857]Make K8s e2e tests could run on Mac
flinkbot commented on pull request #15889: URL: https://github.com/apache/flink/pull/15889#issuecomment-837869382 ## CI report: * e2e703b430cbdfd387de4e2437a981307731027b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
flinkbot edited a comment on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-824189664 ## CI report: * f2fe08ada02c5c9e20ed397163d3ab7a34594994 UNKNOWN * ce0e120bbd288c77f3dd5e2befc30b117123e75f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17614) * 3afca28d455517a4584bd2e5c4637385023ad93e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17831) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15664: [FLINK-22155][table-api][parser] fix explain insert or select two case,sql convert separate selection.
flinkbot edited a comment on pull request #15664: URL: https://github.com/apache/flink/pull/15664#issuecomment-822263432 ## CI report: * ed977f98e6c66aba497c85379927cd677461c6c2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17789) * 3eeeae2cf1aac18495743e56f9699df7bee1a30c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17830) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14012: [FLINK-17857][test] Make K8s e2e tests could run on Mac
flinkbot edited a comment on pull request #14012: URL: https://github.com/apache/flink/pull/14012#issuecomment-724568763 ## CI report: * ec6bda8748f9a4ab81fe59cb1e41beeb53df263e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17698) * ff7f612cdc6e041f05de9f08f1aadbd103413661 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22464) OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure hangs with `AdaptiveScheduler`
[
https://issues.apache.org/jira/browse/FLINK-22464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-22464:
---
Component/s: Runtime / Coordination
> OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure
> hangs with `AdaptiveScheduler`
> --
>
> Key: FLINK-22464
> URL: https://issues.apache.org/jira/browse/FLINK-22464
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.14.0
>Reporter: Guowei Ma
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17178=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=a0a633b8-47ef-5c5a-2806-3c13b9e48228=8171
> {code:java}
> 2021-05-10T02:56:09.3603584Z "main" #1 prio=5 os_prio=0
> tid=0x7f677000b800 nid=0x40e4 waiting on condition [0x7f6776cc8000]
> 2021-05-10T02:56:09.3604176Zjava.lang.Thread.State: TIMED_WAITING
> (sleeping)
> 2021-05-10T02:56:09.3604468Z at java.lang.Thread.sleep(Native Method)
> 2021-05-10T02:56:09.3604925Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.sleepBeforeRetry(CollectResultFetcher.java:237)
> 2021-05-10T02:56:09.3605582Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher.next(CollectResultFetcher.java:113)
> 2021-05-10T02:56:09.3606205Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:106)
> 2021-05-10T02:56:09.3606924Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
> 2021-05-10T02:56:09.3607469Z at
> org.apache.flink.streaming.api.datastream.DataStream.executeAndCollect(DataStream.java:1320)
> 2021-05-10T02:56:09.3607996Z at
> org.apache.flink.streaming.api.datastream.DataStream.executeAndCollect(DataStream.java:1303)
> 2021-05-10T02:56:09.3608616Z at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.runTest(OperatorEventSendingCheckpointITCase.java:223)
> 2021-05-10T02:56:09.3609378Z at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure(OperatorEventSendingCheckpointITCase.java:135)
> 2021-05-10T02:56:09.3609968Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-05-10T02:56:09.3610386Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-05-10T02:56:09.3610858Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-05-10T02:56:09.3611295Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-05-10T02:56:09.3611703Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-05-10T02:56:09.3612207Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-05-10T02:56:09.3612774Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-05-10T02:56:09.3613470Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-05-10T02:56:09.3613930Z at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2021-05-10T02:56:09.3614401Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-05-10T02:56:09.3614770Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-05-10T02:56:09.3615138Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-05-10T02:56:09.3615584Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-05-10T02:56:09.3616070Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-05-10T02:56:09.3616487Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-05-10T02:56:09.3616962Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-05-10T02:56:09.3617361Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-05-10T02:56:09.3617785Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-05-10T02:56:09.3618209Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-05-10T02:56:09.3618635Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-05-10T02:56:09.3619101Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-05-10T02:56:09.3619507Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2021-05-10T02:56:09.3619879Z at
>
[GitHub] [flink] rmetzger commented on a change in pull request #15728: [FLINK-22379][runtime] CheckpointCoordinator checks the state of all …
rmetzger commented on a change in pull request #15728:
URL: https://github.com/apache/flink/pull/15728#discussion_r629857371
##
File path:
flink-libraries/flink-state-processing-api/src/test/java/org/apache/flink/state/api/SavepointWindowReaderITCase.java
##
@@ -348,24 +359,30 @@ public void testApplyWindowStateReader() throws Exception
{
public void testApplyEvictorWindowStateReader() throws Exception {
String savepointPath =
takeSavepoint(
-numbers,
-source -> {
+WaitingWindowAssigner.wrap(
+
TumblingEventTimeWindows.of(Time.milliseconds(10))),
+(windowAssigner) -> {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setStateBackend(getStateBackend());
env.setParallelism(4);
-env.addSource(source)
-.rebalance()
-.assignTimestampsAndWatermarks(
-
WatermarkStrategy.noWatermarks()
-
.withTimestampAssigner((event, timestamp) -> 0))
-.keyBy(id -> id)
-
.window(TumblingEventTimeWindows.of(Time.milliseconds(10)))
-.evictor(new NoOpEvictor<>())
-.apply(new NoOpWindowFunction())
-.uid(uid)
-.addSink(new DiscardingSink<>());
+try {
+env.addSource(createSource(numbers))
+.rebalance()
+.assignTimestampsAndWatermarks(
+
WatermarkStrategy.noWatermarks()
+.withTimestampAssigner(
+(event,
timestamp) -> 0))
+.keyBy(id -> id)
+.window(windowAssigner)
+.evictor(new NoOpEvictor<>())
+.apply(new NoOpWindowFunction())
+.uid(uid)
+.addSink(new DiscardingSink<>());
+} catch (Exception e) {
+e.printStackTrace();
+}
Review comment:
Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15890: [BP-1.12][FLINK-17857]Make K8s e2e tests could run on Mac
flinkbot commented on pull request #15890: URL: https://github.com/apache/flink/pull/15890#issuecomment-83783 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a3b9bae7fb2a6e4e8b65c3a78b0c1d12591a44ce (Tue May 11 05:20:54 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15889: [BP-1.13][FLINK-17857]Make K8s e2e tests could run on Mac
flinkbot commented on pull request #15889: URL: https://github.com/apache/flink/pull/15889#issuecomment-837835694 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit e2e703b430cbdfd387de4e2437a981307731027b (Tue May 11 05:18:56 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wangyang0918 opened a new pull request #15890: [BP-1.12][FLINK-17857]Make K8s e2e tests could run on Mac
wangyang0918 opened a new pull request #15890: URL: https://github.com/apache/flink/pull/15890 Backport #14012 to release-1.12. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wangyang0918 opened a new pull request #15889: [BP-1.13][FLINK-17857]Make K8s e2e tests could run on Mac
wangyang0918 opened a new pull request #15889: URL: https://github.com/apache/flink/pull/15889 Backport #14012 to release-1.13. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wangyang0918 commented on pull request #14012: [FLINK-17857][test] Make K8s e2e tests could run on Mac
wangyang0918 commented on pull request #14012: URL: https://github.com/apache/flink/pull/14012#issuecomment-837827708 Rebase latest master since FLINK-22577 is merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15848: [FLINK-22586][table] Improve the precision dedivation for decimal ari…
flinkbot edited a comment on pull request #15848: URL: https://github.com/apache/flink/pull/15848#issuecomment-833490569 ## CI report: * 99a21fd00339a5c0f0700c493095c745ac1935c0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17723) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17660) * 694aaee49213ced8bc17bc35b1439743842fa951 UNKNOWN * a1eda92b41a825be3259df4a57c7dbb0cc091281 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17828) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15664: [FLINK-22155][table-api][parser] fix explain insert or select two case,sql convert separate selection.
flinkbot edited a comment on pull request #15664: URL: https://github.com/apache/flink/pull/15664#issuecomment-822263432 ## CI report: * ed977f98e6c66aba497c85379927cd677461c6c2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17789) * 3eeeae2cf1aac18495743e56f9699df7bee1a30c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15848: [FLINK-22586][table] Improve the precision dedivation for decimal ari…
flinkbot edited a comment on pull request #15848: URL: https://github.com/apache/flink/pull/15848#issuecomment-833490569 ## CI report: * 99a21fd00339a5c0f0700c493095c745ac1935c0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17723) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17660) * 694aaee49213ced8bc17bc35b1439743842fa951 UNKNOWN * a1eda92b41a825be3259df4a57c7dbb0cc091281 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15858: [FLINK-22067][tests] Wait for vertices to start using API
flinkbot edited a comment on pull request #15858: URL: https://github.com/apache/flink/pull/15858#issuecomment-834799316 ## CI report: * 536a3f0e34b5840b019cf571cf85d4459b52c3a7 UNKNOWN * 2740e54220264d446e0640b4ec46eaf0b4bb1db0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17821) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-22525) The zone id in exception message should be GMT+08:00 instead of GMT+8:00
[
https://issues.apache.org/jira/browse/FLINK-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341718#comment-17341718
]
Jark Wu edited comment on FLINK-22525 at 5/11/21, 4:23 AM:
---
Fixed in
- master: 92512869889ff8e77e1e8030e7014cf7c5119ce4
- release-1.13: b5f375dee021fb7a1294908e6a70553adfb3a81a
was (Author: jark):
Fixed in
- master: 92512869889ff8e77e1e8030e7014cf7c5119ce4
- release-1.13: TODO
> The zone id in exception message should be GMT+08:00 instead of GMT+8:00
>
>
> Key: FLINK-22525
> URL: https://issues.apache.org/jira/browse/FLINK-22525
> Project: Flink
> Issue Type: Bug
> Components: Documentation, Table SQL / API
>Affects Versions: 1.13.0
>Reporter: Leonard Xu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> {code:java}
> Flink SQL> SET table.local-time-zone=UTC+3;
> Flink SQL> select current_row_timestamp();
> [ERROR] Could not execute SQL statement. Reason:
> java.lang.IllegalArgumentException: The supported Zone ID is either a full
> name such as 'America/Los_Angeles', or a custom timezone id such as
> 'GMT-8:00', but configured Zone ID is 'UTC+3'.
> {code}
> The valid zoned should be 'GMT-08:00'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-22525) The zone id in exception message should be GMT+08:00 instead of GMT+8:00
[
https://issues.apache.org/jira/browse/FLINK-22525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22525.
---
Assignee: Leonard Xu
Resolution: Fixed
> The zone id in exception message should be GMT+08:00 instead of GMT+8:00
>
>
> Key: FLINK-22525
> URL: https://issues.apache.org/jira/browse/FLINK-22525
> Project: Flink
> Issue Type: Bug
> Components: Documentation, Table SQL / API
>Affects Versions: 1.13.0
>Reporter: Leonard Xu
>Assignee: Leonard Xu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> {code:java}
> Flink SQL> SET table.local-time-zone=UTC+3;
> Flink SQL> select current_row_timestamp();
> [ERROR] Could not execute SQL statement. Reason:
> java.lang.IllegalArgumentException: The supported Zone ID is either a full
> name such as 'America/Los_Angeles', or a custom timezone id such as
> 'GMT-8:00', but configured Zone ID is 'UTC+3'.
> {code}
> The valid zoned should be 'GMT-08:00'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #15874: [FLINK-22525][table-api] Fix gmt format in Flink from GMT-8:00 to GMT-08:00
wuchong merged pull request #15874: URL: https://github.com/apache/flink/pull/15874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15848: [FLINK-22586][table] Improve the precision dedivation for decimal ari…
flinkbot edited a comment on pull request #15848: URL: https://github.com/apache/flink/pull/15848#issuecomment-833490569 ## CI report: * 99a21fd00339a5c0f0700c493095c745ac1935c0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17723) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17660) * 694aaee49213ced8bc17bc35b1439743842fa951 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
flinkbot edited a comment on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-824189664 ## CI report: * f2fe08ada02c5c9e20ed397163d3ab7a34594994 UNKNOWN * ce0e120bbd288c77f3dd5e2befc30b117123e75f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17614) * 3afca28d455517a4584bd2e5c4637385023ad93e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] hehuiyuan commented on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
hehuiyuan commented on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-837753830 > Hi @hehuiyuan , I mean you could include the parquet file in your PR, and use it in the test. The file should contain minimum data that is sufficient to reproduce the issue. get. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15813: [FLINK-22519][flink-python] support tar python archives
flinkbot edited a comment on pull request #15813: URL: https://github.com/apache/flink/pull/15813#issuecomment-829753030 ## CI report: * fd020167f550e9c4c3b70d0a1dbe9d35b600d7cf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17826) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15888: [FLINK-22624][runtime] Utilize the remain resource of new pending tas…
flinkbot edited a comment on pull request #15888: URL: https://github.com/apache/flink/pull/15888#issuecomment-837704287 ## CI report: * 327482a8fb754651f21f27eb7d3e277c4861 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17827) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15813: [FLINK-22519][flink-python] support tar python archives
flinkbot edited a comment on pull request #15813: URL: https://github.com/apache/flink/pull/15813#issuecomment-829753030 ## CI report: * f1a3fdefdb723f03636fb7a73a185b69bd93b57d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17808) * fd020167f550e9c4c3b70d0a1dbe9d35b600d7cf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17826) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22534) Set delegation token's service name as credential alias
[ https://issues.apache.org/jira/browse/FLINK-22534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342268#comment-17342268 ] Junfan Zhang commented on FLINK-22534: -- Any ideas on it? [~lirui] [~mapohl] > Set delegation token's service name as credential alias > --- > > Key: FLINK-22534 > URL: https://issues.apache.org/jira/browse/FLINK-22534 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hadoop Compatibility >Reporter: Junfan Zhang >Assignee: Junfan Zhang >Priority: Major > Labels: pull-request-available > Attachments: debug2.PNG > > > h4. What > Set the Hadoop delegation token's service name as credential alias. > h4. Why > In current implementation, Flink will use delegation token's service name or > identifer as credential alias, refer to Flink code > [HadoopModule|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java#L101] > and [Yarn > Utils|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L209]. > Firstly, I think we could use the same way to set credential alias, like > delegation token's service name. It will be more clear. > Secondly, when fetching HDFS delegation token and then inject all tokens to > current UserGroupInformation in Hadoop HDFS HA mode, it will cause the > problem of overwriting the different delegation tokens with the same > identifier, [refer to code > here|https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java#L209]. > h5. When does the same identifier delegation tokens appear? > When in HDFS HA mode, Hadoop HA delegation tokens will have the same > identifier(Refer to HDFS-9276), but its' service name is different. So we can > use service name as alias. > The following figure from HDFS-9276 can show that the identifier of HA > delegation token is the same. > !debug2.PNG! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #15888: [FLINK-22624][runtime] Utilize the remain resource of new pending tas…
flinkbot commented on pull request #15888: URL: https://github.com/apache/flink/pull/15888#issuecomment-837704287 ## CI report: * 327482a8fb754651f21f27eb7d3e277c4861 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15877: [FLINK-22612][python] Restructure the coders in PyFlink
flinkbot edited a comment on pull request #15877: URL: https://github.com/apache/flink/pull/15877#issuecomment-836433504 ## CI report: * ae350a48301e7483cdd6b2856f674e7628b80bce Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17798) * 816de487302903060345a5696a3ef548dc82b69d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17824) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15887: [FLINK-22618][runtime] Fix incorrect free resource metrics of task ma…
flinkbot edited a comment on pull request #15887: URL: https://github.com/apache/flink/pull/15887#issuecomment-837684409 ## CI report: * 791cd20391c7bc64f3d2ca77ba6d73c56c3acc79 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17825) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15813: [FLINK-22519][flink-python] support tar python archives
flinkbot edited a comment on pull request #15813: URL: https://github.com/apache/flink/pull/15813#issuecomment-829753030 ## CI report: * f1a3fdefdb723f03636fb7a73a185b69bd93b57d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17808) * fd020167f550e9c4c3b70d0a1dbe9d35b600d7cf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #15812: remove slotpoolImpl
KarmaGYZ commented on pull request #15812: URL: https://github.com/apache/flink/pull/15812#issuecomment-837699726 @tillrohrmann Hi, I think now we can remove the `SlotManagerImpl`. As this PR is large enough, we can move that effort out of the scope of this PR. WDYT? If so, I'd like to file a ticket. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15887: [FLINK-22618][runtime] Fix incorrect free resource metrics of task ma…
flinkbot commented on pull request #15887: URL: https://github.com/apache/flink/pull/15887#issuecomment-837684409 ## CI report: * 791cd20391c7bc64f3d2ca77ba6d73c56c3acc79 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15877: [FLINK-22612][python] Restructure the coders in PyFlink
flinkbot edited a comment on pull request #15877: URL: https://github.com/apache/flink/pull/15877#issuecomment-836433504 ## CI report: * ae350a48301e7483cdd6b2856f674e7628b80bce Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17798) * 816de487302903060345a5696a3ef548dc82b69d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YikSanChan commented on a change in pull request #15813: [FLINK-22519][flink-python] support tar python archives
YikSanChan commented on a change in pull request #15813: URL: https://github.com/apache/flink/pull/15813#discussion_r629809218 ## File path: flink-python/src/main/java/org/apache/flink/python/util/DecompressUtils.java ## @@ -0,0 +1,12 @@ +package org.apache.flink.python.util; Review comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15888: [FLINK-22624][runtime] Utilize the remain resource of new pending tas…
flinkbot commented on pull request #15888: URL: https://github.com/apache/flink/pull/15888#issuecomment-837682553 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 327482a8fb754651f21f27eb7d3e277c4861 (Tue May 11 02:45:58 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] YikSanChan commented on a change in pull request #15813: [FLINK-22519][flink-python] support tar python archives
YikSanChan commented on a change in pull request #15813:
URL: https://github.com/apache/flink/pull/15813#discussion_r629337523
##
File path:
flink-python/src/main/java/org/apache/flink/python/env/beam/ProcessPythonEnvironmentManager.java
##
@@ -326,9 +327,21 @@ private void constructArchivesDirectory(Map env) throws IOExcept
// extract archives to archives directory
for (Map.Entry entry :
dependencyInfo.getArchives().entrySet()) {
-ZipUtils.extractZipFileWithPermissions(
-entry.getKey(),
-String.join(File.separator, archivesDirectory,
entry.getValue()));
+String inFilePath = entry.getKey();
+String targetDirPath =
+String.join(File.separator, archivesDirectory,
entry.getValue());
+if (inFilePath.endsWith(".zip") ||
inFilePath.endsWith(".jar")) {
Review comment:
I am not sure if all filesystems are case-insensitive
##
File path:
flink-python/src/main/java/org/apache/flink/python/env/beam/ProcessPythonEnvironmentManager.java
##
@@ -326,9 +327,21 @@ private void constructArchivesDirectory(Map env) throws IOExcept
// extract archives to archives directory
for (Map.Entry entry :
dependencyInfo.getArchives().entrySet()) {
-ZipUtils.extractZipFileWithPermissions(
-entry.getKey(),
-String.join(File.separator, archivesDirectory,
entry.getValue()));
+String inFilePath = entry.getKey();
+String targetDirPath =
+String.join(File.separator, archivesDirectory,
entry.getValue());
+if (inFilePath.endsWith(".zip") ||
inFilePath.endsWith(".jar")) {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] YikSanChan commented on a change in pull request #15813: [FLINK-22519][flink-python] support tar python archives
YikSanChan commented on a change in pull request #15813:
URL: https://github.com/apache/flink/pull/15813#discussion_r629808846
##
File path:
flink-python/src/main/java/org/apache/flink/python/env/beam/ProcessPythonEnvironmentManager.java
##
@@ -326,9 +327,21 @@ private void constructArchivesDirectory(Map env) throws IOExcept
// extract archives to archives directory
for (Map.Entry entry :
dependencyInfo.getArchives().entrySet()) {
-ZipUtils.extractZipFileWithPermissions(
-entry.getKey(),
-String.join(File.separator, archivesDirectory,
entry.getValue()));
+String filePath = entry.getKey();
+if (filePath.endsWith(".zip") || filePath.endsWith(".jar")) {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-22624) Default resource allocation strategy will allocate more pending task managers than demand
[
https://issues.apache.org/jira/browse/FLINK-22624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-22624:
---
Labels: pull-request-available (was: )
> Default resource allocation strategy will allocate more pending task managers
> than demand
> -
>
> Key: FLINK-22624
> URL: https://issues.apache.org/jira/browse/FLINK-22624
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Yangze Guo
>Assignee: Yangze Guo
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> When the {{DefaultResourceAllocationStrategy}} try to fulfill a requirement
> with allocating new pending task managers. The remaining resource of those
> task managers will never be used to fulfill other requirement, which hurt
> resource utilization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] KarmaGYZ opened a new pull request #15888: [FLINK-22624][runtime] Utilize the remain resource of new pending tas…
KarmaGYZ opened a new pull request #15888: URL: https://github.com/apache/flink/pull/15888 …k managers to fulfill requirement in DefaultResourceAllocationStrategy ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-22624) Default resource allocation strategy will allocate more pending task managers than demand
[
https://issues.apache.org/jira/browse/FLINK-22624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song reassigned FLINK-22624:
Assignee: Yangze Guo
> Default resource allocation strategy will allocate more pending task managers
> than demand
> -
>
> Key: FLINK-22624
> URL: https://issues.apache.org/jira/browse/FLINK-22624
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Yangze Guo
>Assignee: Yangze Guo
>Priority: Major
> Fix For: 1.14.0, 1.13.1
>
>
> When the {{DefaultResourceAllocationStrategy}} try to fulfill a requirement
> with allocating new pending task managers. The remaining resource of those
> task managers will never be used to fulfill other requirement, which hurt
> resource utilization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-22624) Default resource allocation strategy will allocate more pending task managers than demand
Yangze Guo created FLINK-22624:
--
Summary: Default resource allocation strategy will allocate more
pending task managers than demand
Key: FLINK-22624
URL: https://issues.apache.org/jira/browse/FLINK-22624
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Yangze Guo
Fix For: 1.14.0, 1.13.1
When the {{DefaultResourceAllocationStrategy}} try to fulfill a requirement
with allocating new pending task managers. The remaining resource of those task
managers will never be used to fulfill other requirement, which hurt resource
utilization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-22523) TUMBLE TVF should throw helpful exception when specifying second interval parameter
[
https://issues.apache.org/jira/browse/FLINK-22523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22523.
---
Resolution: Fixed
Fixed in
- master: 3d201442cee372856f2c919dee38d29bf08b77f4
- release-1.13: 1088b8726732d5121a40a88f38e2fe0bcefffb37
> TUMBLE TVF should throw helpful exception when specifying second interval
> parameter
> ---
>
> Key: FLINK-22523
> URL: https://issues.apache.org/jira/browse/FLINK-22523
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> Currently, the following query can run and no exception is thrown.
> However, the second interval parameter (i.e. the offset parameter) is not
> supported yet. We should throw a exception for this.
> {code:sql}
> select
> date_format(window_end, '-MM-dd') as date_str,
> date_format(window_end, 'HH:mm') as time_str,
> count(distinct user_id) as uv
> from table(tumble(table user_behavior, descriptor(ts), interval '10' minute,
> interval '1' day))
> group by window_start, window_end;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22523) TUMBLE TVF should throw helpful exception when specifying second interval parameter
[
https://issues.apache.org/jira/browse/FLINK-22523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-22523:
Component/s: Table SQL / API
> TUMBLE TVF should throw helpful exception when specifying second interval
> parameter
> ---
>
> Key: FLINK-22523
> URL: https://issues.apache.org/jira/browse/FLINK-22523
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API, Table SQL / Planner
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> Currently, the following query can run and no exception is thrown.
> However, the second interval parameter (i.e. the offset parameter) is not
> supported yet. We should throw a exception for this.
> {code:sql}
> select
> date_format(window_end, '-MM-dd') as date_str,
> date_format(window_end, 'HH:mm') as time_str,
> count(distinct user_id) as uv
> from table(tumble(table user_behavior, descriptor(ts), interval '10' minute,
> interval '1' day))
> group by window_start, window_end;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #15803: [FLINK-22523][table-planner-blink] TUMBLE TVF should throw helpful exception when specifying second interval parameter
wuchong merged pull request #15803: URL: https://github.com/apache/flink/pull/15803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong commented on pull request #15803: [FLINK-22523][table-planner-blink] TUMBLE TVF should throw helpful exception when specifying second interval parameter
wuchong commented on pull request #15803: URL: https://github.com/apache/flink/pull/15803#issuecomment-837655471 The failed test case is not related to this PR. Merging... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-22313) Redundant CAST in plan when selecting window start and window end in window agg
[
https://issues.apache.org/jira/browse/FLINK-22313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22313.
---
Fix Version/s: 1.14.0
Resolution: Fixed
Fixed in master: 3c342c93ea56a326e13fd3caf48b6a46e6604de7
> Redundant CAST in plan when selecting window start and window end in window
> agg
> ---
>
> Key: FLINK-22313
> URL: https://issues.apache.org/jira/browse/FLINK-22313
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Affects Versions: 1.13.0
>Reporter: Caizhi Weng
>Assignee: Jark Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> Add the following test case to
> {{org.apache.flink.table.planner.plan.stream.sql.agg.WindowAggregateTest}} to
> reproduce this bug.
> {code:scala}
> @Test
> def testSessionFunction(): Unit = {
> val sql =
> """
> |SELECT
> |COUNT(*),
> |SESSION_START(proctime, INTERVAL '15' MINUTE),
> |SESSION_END(proctime, INTERVAL '15' MINUTE)
> |FROM MyTable
> |GROUP BY SESSION(proctime, INTERVAL '15' MINUTE)
> """.stripMargin
> util.verifyExecPlan(sql)
> }
> {code}
> The produced plan is
> {code}
> Calc(select=[EXPR$0, CAST(w$start) AS EXPR$1, CAST(w$end) AS EXPR$2])
> +- GroupWindowAggregate(window=[SessionGroupWindow('w$, proctime, 90)],
> properties=[w$start, w$end, w$proctime], select=[COUNT(*) AS EXPR$0,
> start('w$) AS w$start, end('w$) AS w$end, proctime('w$) AS w$proctime])
>+- Exchange(distribution=[single])
> +- Calc(select=[proctime])
> +- WatermarkAssigner(rowtime=[rowtime], watermark=[(rowtime -
> 1000:INTERVAL SECOND)])
> +- Calc(select=[PROCTIME() AS proctime, rowtime])
>+- TableSourceScan(table=[[default_catalog, default_database,
> MyTable, project=[rowtime]]], fields=[rowtime])
> {code}
> This is because the nullability indicated by
> {{PlannerWindowStart#getResultType}} and
> {{SqlGroupedWindowFunction#WindowStartEndReturnTypeInference}} are different.
> Actually time attribute and window start / end should always be not null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #15806: [FLINK-22313][table-planner-blink] Redundant CAST in plan when selecting window start and window end in window agg
wuchong merged pull request #15806: URL: https://github.com/apache/flink/pull/15806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22618) Fix incorrect free resource metrics of task managers
[
https://issues.apache.org/jira/browse/FLINK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yangze Guo updated FLINK-22618:
---
Labels: pull-request-available starter (was: pull-request-available)
> Fix incorrect free resource metrics of task managers
>
>
> Key: FLINK-22618
> URL: https://issues.apache.org/jira/browse/FLINK-22618
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Yangze Guo
>Priority: Major
> Labels: pull-request-available, starter
> Fix For: 1.14.0, 1.13.1
>
>
> In FLINK-21177, the {{FineGrainedSlotManager#getFreeResourceOf}} wrongly
> return the total resource.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #15887: [FLINK-22618][runtime] Fix incorrect free resource metrics of task ma…
flinkbot commented on pull request #15887: URL: https://github.com/apache/flink/pull/15887#issuecomment-837644601 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 791cd20391c7bc64f3d2ca77ba6d73c56c3acc79 (Tue May 11 02:08:47 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-22618).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22618) Fix incorrect free resource metrics of task managers
[
https://issues.apache.org/jira/browse/FLINK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-22618:
---
Labels: pull-request-available (was: )
> Fix incorrect free resource metrics of task managers
>
>
> Key: FLINK-22618
> URL: https://issues.apache.org/jira/browse/FLINK-22618
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Yangze Guo
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1
>
>
> In FLINK-21177, the {{FineGrainedSlotManager#getFreeResourceOf}} wrongly
> return the total resource.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] paul8263 opened a new pull request #15887: [FLINK-22618][runtime] Fix incorrect free resource metrics of task ma…
paul8263 opened a new pull request #15887: URL: https://github.com/apache/flink/pull/15887 ## What is the purpose of the change Fix incorrect free resource metrics of task managers ## Brief change log Changed method "getFreeResourceOf" in flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManager.java by invoking the correct method in taskManagerTracker. ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] cshuo commented on pull request #15848: [FLINK-22586][table] Improve the precision dedivation for decimal ari…
cshuo commented on pull request #15848: URL: https://github.com/apache/flink/pull/15848#issuecomment-837638785 > @cshuo Can you take a look to test failure? I'm checking... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342242#comment-17342242 ] Xintong Song commented on FLINK-19481: -- [~galenwarren], I think the RecoverableWriter implementation is beneficial regardless of which (or both) file system implementation we use. Meantime, I'm still a bit unsure about introducing another fs implementation. We probably should not block a definite improvement on an uncertain thread. Even we decide to introduce a native gcs fs implementation, there's only a small fraction of FLINK-11838 needs further changes. I think we can make those changes when we indeed introduce the native gcs fs. > Add support for a flink native GCS FileSystem > - > > Key: FLINK-19481 > URL: https://issues.apache.org/jira/browse/FLINK-19481 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem, FileSystems >Affects Versions: 1.12.0 >Reporter: Ben Augarten >Priority: Minor > Labels: auto-deprioritized-major > > Currently, GCS is supported but only by using the hadoop connector[1] > > The objective of this improvement is to add support for checkpointing to > Google Cloud Storage with the Flink File System, > > This would allow the `gs://` scheme to be used for savepointing and > checkpointing. Long term, it would be nice if we could use the GCS FileSystem > as a source and sink in flink jobs as well. > > Long term, I hope that implementing a flink native GCS FileSystem will > simplify usage of GCS because the hadoop FileSystem ends up bringing in many > unshaded dependencies. > > [1] > [https://github.com/GoogleCloudDataproc/hadoop-connectors|https://github.com/GoogleCloudDataproc/hadoop-connectors)] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-22353) Add CPU metrics of TaskManager and JobManager into Flink WebUI
[ https://issues.apache.org/jira/browse/FLINK-22353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roc Marshal closed FLINK-22353. --- Resolution: Abandoned > Add CPU metrics of TaskManager and JobManager into Flink WebUI > -- > > Key: FLINK-22353 > URL: https://issues.apache.org/jira/browse/FLINK-22353 > Project: Flink > Issue Type: Improvement > Components: Runtime / Metrics, Runtime / Web Frontend >Reporter: Roc Marshal >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15858: [FLINK-22067][tests] Wait for vertices to start using API
flinkbot edited a comment on pull request #15858: URL: https://github.com/apache/flink/pull/15858#issuecomment-834799316 ## CI report: * 16e61d4169ad70214e2d8a503e100ac566b6800e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17720) * 536a3f0e34b5840b019cf571cf85d4459b52c3a7 UNKNOWN * 2740e54220264d446e0640b4ec46eaf0b4bb1db0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17821) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15885: [FLINK-22376][runtime] RecoveredChannelStateHandler recycles the buffer if it was created inside and doesn't recycle if it was passed
flinkbot edited a comment on pull request #15885: URL: https://github.com/apache/flink/pull/15885#issuecomment-837001445 ## CI report: * ffab5d3ccda17f4ec347528d851ba39d5a59cca6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17815) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15858: [FLINK-22067][tests] Wait for vertices to start using API
flinkbot edited a comment on pull request #15858: URL: https://github.com/apache/flink/pull/15858#issuecomment-834799316 ## CI report: * 16e61d4169ad70214e2d8a503e100ac566b6800e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17720) * 536a3f0e34b5840b019cf571cf85d4459b52c3a7 UNKNOWN * 2740e54220264d446e0640b4ec46eaf0b4bb1db0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15858: [FLINK-22067][tests] Wait for vertices to start using API
flinkbot edited a comment on pull request #15858: URL: https://github.com/apache/flink/pull/15858#issuecomment-834799316 ## CI report: * 16e61d4169ad70214e2d8a503e100ac566b6800e Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17720) * 536a3f0e34b5840b019cf571cf85d4459b52c3a7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #15728: [FLINK-22379][runtime] CheckpointCoordinator checks the state of all …
rkhachatryan commented on a change in pull request #15728:
URL: https://github.com/apache/flink/pull/15728#discussion_r629693902
##
File path:
flink-libraries/flink-state-processing-api/src/test/java/org/apache/flink/state/api/SavepointWindowReaderITCase.java
##
@@ -348,24 +359,30 @@ public void testApplyWindowStateReader() throws Exception
{
public void testApplyEvictorWindowStateReader() throws Exception {
String savepointPath =
takeSavepoint(
-numbers,
-source -> {
+WaitingWindowAssigner.wrap(
+
TumblingEventTimeWindows.of(Time.milliseconds(10))),
+(windowAssigner) -> {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.setStateBackend(getStateBackend());
env.setParallelism(4);
-env.addSource(source)
-.rebalance()
-.assignTimestampsAndWatermarks(
-
WatermarkStrategy.noWatermarks()
-
.withTimestampAssigner((event, timestamp) -> 0))
-.keyBy(id -> id)
-
.window(TumblingEventTimeWindows.of(Time.milliseconds(10)))
-.evictor(new NoOpEvictor<>())
-.apply(new NoOpWindowFunction())
-.uid(uid)
-.addSink(new DiscardingSink<>());
+try {
+env.addSource(createSource(numbers))
+.rebalance()
+.assignTimestampsAndWatermarks(
+
WatermarkStrategy.noWatermarks()
+.withTimestampAssigner(
+(event,
timestamp) -> 0))
+.keyBy(id -> id)
+.window(windowAssigner)
+.evictor(new NoOpEvictor<>())
+.apply(new NoOpWindowFunction())
+.uid(uid)
+.addSink(new DiscardingSink<>());
+} catch (Exception e) {
+e.printStackTrace();
+}
Review comment:
I agree, will remove it in #15858.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (FLINK-22067) SavepointWindowReaderITCase.testApplyEvictorWindowStateReader
[
https://issues.apache.org/jira/browse/FLINK-22067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan reassigned FLINK-22067:
-
Assignee: Roman Khachatryan
> SavepointWindowReaderITCase.testApplyEvictorWindowStateReader
> -
>
> Key: FLINK-22067
> URL: https://issues.apache.org/jira/browse/FLINK-22067
> Project: Flink
> Issue Type: Bug
> Components: API / State Processor
>Affects Versions: 1.13.0, 1.14.0
>Reporter: Till Rohrmann
>Assignee: Roman Khachatryan
>Priority: Blocker
> Labels: auto-deprioritized-critical, pull-request-available,
> test-stability
> Attachments: isolated_logs_builD_9072.log
>
>
> The test case
> {{SavepointWindowReaderITCase.testApplyEvictorWindowStateReader}} failed on
> AZP with:
> {code}
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.takeSavepoint(SavepointTestBase.java:69)
> ... 33 more
> Caused by: java.util.concurrent.TimeoutException: Invocation of public
> default java.util.concurrent.CompletableFuture
> org.apache.flink.runtime.webmonitor.RestfulGateway.triggerSavepoint(org.apache.flink.api.common.JobID,java.lang.String,boolean,org.apache.flink.api.common.time.Time)
> timed out.
> at com.sun.proxy.$Proxy32.triggerSavepoint(Unknown Source)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.lambda$triggerSavepoint$8(MiniCluster.java:716)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> at
> java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628)
> at
> java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.runDispatcherCommand(MiniCluster.java:751)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.triggerSavepoint(MiniCluster.java:714)
> at
> org.apache.flink.client.program.MiniClusterClient.triggerSavepoint(MiniClusterClient.java:101)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.triggerSavepoint(SavepointTestBase.java:93)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.lambda$takeSavepoint$0(SavepointTestBase.java:68)
> at
> java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:966)
> at
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1646)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1632)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> Caused by: akka.pattern.AskTimeoutException: Ask timed out on
> [Actor[akka://flink/user/rpc/dispatcher_2#-390276455]] after [1 ms].
> Message of type [org.apache.flink.runtime.rpc.messages.LocalFencedMessage]. A
> typical reason for `AskTimeoutException` is that the recipient actor didn't
> send a reply.
> at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635)
> at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635)
> at
> akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:648)
> at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)
> at
> scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
> at
> scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
> at
> scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
> at
> akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15809=logs=b2f046ab-ae17-5406-acdc-240be7e870e4=93e5ae06-d194-513d-ba8d-150ef6da1d7c=9197
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-22481) RocksDBStateBackendWindowITCase times out on Azure
[
https://issues.apache.org/jira/browse/FLINK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Khachatryan closed FLINK-22481.
-
Resolution: Duplicate
Closing as duplicate of FLINK-22067.
> RocksDBStateBackendWindowITCase times out on Azure
> --
>
> Key: FLINK-22481
> URL: https://issues.apache.org/jira/browse/FLINK-22481
> Project: Flink
> Issue Type: Bug
> Components: API / State Processor
>Affects Versions: 1.13.0
>Reporter: Dawid Wysakowicz
>Priority: Major
> Labels: pull-request-available, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17215=logs=b2f046ab-ae17-5406-acdc-240be7e870e4=93e5ae06-d194-513d-ba8d-150ef6da1d7c=9161
> {code}
> java.lang.RuntimeException: Failed to take savepoint
> Apr 26 11:07:04 at
> org.apache.flink.state.api.utils.SavepointTestBase.takeSavepoint(SavepointTestBase.java:71)
> Apr 26 11:07:04 at
> org.apache.flink.state.api.utils.SavepointTestBase.takeSavepoint(SavepointTestBase.java:46)
> Apr 26 11:07:04 at
> org.apache.flink.state.api.SavepointWindowReaderITCase.testAggregateWindowStateReader(SavepointWindowReaderITCase.java:146)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> Apr 26 11:07:04 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 26 11:07:04 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 26 11:07:04 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Apr 26 11:07:04 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> Apr 26 11:07:04 at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Apr 26 11:07:04 Caused by: java.util.concurrent.ExecutionException:
> java.util.concurrent.TimeoutException: Invocation of public default
> java.util.concurrent.CompletableFuture
> org.apache.flink.runtime.webmonitor.RestfulGateway.triggerSavepoint(org.apache.flink.api.common.JobID,java.lang.String,boolean,org.apache.flink.api.common.time.Time)
> timed out.
> Apr 26 11:07:04 at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> Apr 26 11:07:04 at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
> Apr 26 11:07:04 at
> org.apache.flink.state.api.utils.SavepointTestBase.takeSavepoint(SavepointTestBase.java:69)
> Apr 26 11:07:04 ... 34 more
> Apr 26 11:07:04 Caused by: java.util.concurrent.TimeoutException: Invocation
> of public default java.util.concurrent.CompletableFuture
> org.apache.flink.runtime.webmonitor.RestfulGateway.triggerSavepoint(org.apache.flink.api.common.JobID,java.lang.String,boolean,org.apache.flink.api.common.time.Time)
> timed out.
> Apr 26 11:07:04 at com.sun.proxy.$Proxy32.triggerSavepoint(Unknown
> Source)
> Apr 26 11:07:04 at
> org.apache.flink.runtime.minicluster.MiniCluster.lambda$triggerSavepoint$8(MiniCluster.java:716)
> Apr 26 11:07:04 at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> Apr 26 11:07:04 at
> java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628)
> Apr 26 11:07:04 at
> java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996)
> Apr 26 11:07:04 at
> org.apache.flink.runtime.minicluster.MiniCluster.runDispatcherCommand(MiniCluster.java:751)
> Apr 26 11:07:04 at
> org.apache.flink.runtime.minicluster.MiniCluster.triggerSavepoint(MiniCluster.java:714)
> Apr 26 11:07:04 at
>
[GitHub] [flink] rkhachatryan commented on pull request #15858: [FLINK-22067][tests] Wait for vertices to start using API
rkhachatryan commented on pull request #15858: URL: https://github.com/apache/flink/pull/15858#issuecomment-837356465 Thanks for reviewing @sjwiesman. I've removed `WaitingFunction`, `WaitingWindowAssigner` and unnecessary now arguments, PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] AHeise opened a new pull request #446: Added release 1.12.4
AHeise opened a new pull request #446: URL: https://github.com/apache/flink-web/pull/446 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * b979ed53178630a7bf881d3e021993d0ae803449 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17814) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15884: [FLINK-22266] Fix stop-with-savepoint operation in AdaptiveScheduler
flinkbot edited a comment on pull request #15884: URL: https://github.com/apache/flink/pull/15884#issuecomment-836859750 ## CI report: * e28e6f0fae5e5688149d613d93aa621d24923de4 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17813) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #15858: [WIP][FLINK-22067][tests] Wait for vertices to start using API
rkhachatryan commented on a change in pull request #15858:
URL: https://github.com/apache/flink/pull/15858#discussion_r629647704
##
File path:
flink-libraries/flink-state-processing-api/src/test/java/org/apache/flink/state/api/utils/SavepointTestBase.java
##
@@ -56,16 +64,33 @@
try {
JobID jobID = client.submitJob(jobGraph).get();
-return CompletableFuture.runAsync(waitingFunction::await)
-.thenCompose(ignore -> triggerSavepoint(client, jobID))
-.get(5, TimeUnit.MINUTES);
+waitForAllRunningOrSomeTerminal(jobID);
+
+return triggerSavepoint(client, jobID).get(5, TimeUnit.MINUTES);
} catch (Exception e) {
throw new RuntimeException("Failed to take savepoint", e);
} finally {
client.cancel(jobId);
}
}
+private void waitForAllRunningOrSomeTerminal(JobID jobID) throws Exception
{
+while (true) {
Review comment:
I think it's better to rely on CI timeout (this topic was recently
discussed
[here](http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Using-timeouts-in-JUnit-tests-tp50449p50601.html)).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15886: [hotfix] [docs] Fix typo in k8s docs
flinkbot edited a comment on pull request #15886: URL: https://github.com/apache/flink/pull/15886#issuecomment-837066644 ## CI report: * c78d77a6effa454bc4c4a02b05d9037b5f4c5059 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17816) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15882: [FLINK-22574] Adaptive Scheduler: Fix cancellation while in Restarting state
flinkbot edited a comment on pull request #15882: URL: https://github.com/apache/flink/pull/15882#issuecomment-836740251 ## CI report: * 89dbb13e84a099927fe0eb1ac7e5e7bcdb79f08f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17811) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342087#comment-17342087 ] Galen Warren commented on FLINK-19481: -- I wanted to check in here. Should I wait until this question is resolved before proceeding with the PR? Personally, my preference would be to see Flink HadoopFileSystem + GoogleHadoopFileSystem as at least _an_ option for the file system implementation, just because those components seem to be well established. I'm not opposed to an alternate implementation, though, i.e. as has been done with S3. If that's the path we're going down, it might mean some changes for the code in the PR I'm working on, hence the question. > Add support for a flink native GCS FileSystem > - > > Key: FLINK-19481 > URL: https://issues.apache.org/jira/browse/FLINK-19481 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem, FileSystems >Affects Versions: 1.12.0 >Reporter: Ben Augarten >Priority: Minor > Labels: auto-deprioritized-major > > Currently, GCS is supported but only by using the hadoop connector[1] > > The objective of this improvement is to add support for checkpointing to > Google Cloud Storage with the Flink File System, > > This would allow the `gs://` scheme to be used for savepointing and > checkpointing. Long term, it would be nice if we could use the GCS FileSystem > as a source and sink in flink jobs as well. > > Long term, I hope that implementing a flink native GCS FileSystem will > simplify usage of GCS because the hadoop FileSystem ends up bringing in many > unshaded dependencies. > > [1] > [https://github.com/GoogleCloudDataproc/hadoop-connectors|https://github.com/GoogleCloudDataproc/hadoop-connectors)] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15886: [hotfix] [docs] Fix typo in k8s docs
flinkbot edited a comment on pull request #15886: URL: https://github.com/apache/flink/pull/15886#issuecomment-837066644 ## CI report: * c78d77a6effa454bc4c4a02b05d9037b5f4c5059 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17816) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22133) SplitEmumerator does not provide checkpoint id in snapshot
[ https://issues.apache.org/jira/browse/FLINK-22133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342063#comment-17342063 ] Thomas Weise commented on FLINK-22133: -- Hi Becket, I think the suggestion from Stephan would be the best approach. It doesn't break existing users and at the same time allows to build customizations in a 1.13 compatible manner. PublicEvolving should not be modified in binary incompatible way between patch releases and the japicmp check guards that. So if we were to make the same change as in 1.13, we would require an exclusion. But I think we better stick with the compatibility guarantee. > SplitEmumerator does not provide checkpoint id in snapshot > -- > > Key: FLINK-22133 > URL: https://issues.apache.org/jira/browse/FLINK-22133 > Project: Flink > Issue Type: Bug > Components: Connectors / Common >Affects Versions: 1.12.0 >Reporter: Brian Zhou >Assignee: Stephan Ewen >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > In ExternallyInducedSource API, the checkpoint trigger exposes the checkpoint > Id for the external client to identify the checkpoint. However, in the > FLIP-27 source, the SplitEmumerator::snapshot() is a no-arg method. The > connector cannot track the checkpoint ID from Flink -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #15886: [hotfix] [docs] Fix typo in k8s docs
flinkbot commented on pull request #15886: URL: https://github.com/apache/flink/pull/15886#issuecomment-837066644 ## CI report: * c78d77a6effa454bc4c4a02b05d9037b5f4c5059 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20498) SQLClientSchemaRegistryITCase.testReading test timed out after 120000 milliseconds
[
https://issues.apache.org/jira/browse/FLINK-20498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342055#comment-17342055
]
Robert Metzger commented on FLINK-20498:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17804=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529
> SQLClientSchemaRegistryITCase.testReading test timed out after 12
> milliseconds
> --
>
> Key: FLINK-20498
> URL: https://issues.apache.org/jira/browse/FLINK-20498
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Huang Xingbo
>Priority: Major
> Labels: test-stability
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10548=logs=739e6eac-8312-5d31-d437-294c4d26fced=a68b8d89-50e9-5977-4500-f4fde4f57f9b]
> {code:java}
> 2020-12-06T02:06:38.6416440Z Dec 06 02:06:38
> org.junit.runners.model.TestTimedOutException: test timed out after 12
> milliseconds
> 2020-12-06T02:06:38.6417052Z Dec 06 02:06:38 at java.lang.Object.wait(Native
> Method)
> 2020-12-06T02:06:38.6417586Z Dec 06 02:06:38 at
> java.lang.Thread.join(Thread.java:1252)
> 2020-12-06T02:06:38.6418170Z Dec 06 02:06:38 at
> java.lang.Thread.join(Thread.java:1326)
> 2020-12-06T02:06:38.6418788Z Dec 06 02:06:38 at
> org.apache.kafka.clients.admin.KafkaAdminClient.close(KafkaAdminClient.java:541)
> 2020-12-06T02:06:38.6419463Z Dec 06 02:06:38 at
> org.apache.kafka.clients.admin.Admin.close(Admin.java:96)
> 2020-12-06T02:06:38.6420277Z Dec 06 02:06:38 at
> org.apache.kafka.clients.admin.Admin.close(Admin.java:79)
> 2020-12-06T02:06:38.6420973Z Dec 06 02:06:38 at
> org.apache.flink.tests.util.kafka.KafkaContainerClient.createTopic(KafkaContainerClient.java:76)
> 2020-12-06T02:06:38.6421797Z Dec 06 02:06:38 at
> org.apache.flink.tests.util.kafka.SQLClientSchemaRegistryITCase.testReading(SQLClientSchemaRegistryITCase.java:109)
> 2020-12-06T02:06:38.6422517Z Dec 06 02:06:38 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-12-06T02:06:38.6423173Z Dec 06 02:06:38 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-12-06T02:06:38.6423990Z Dec 06 02:06:38 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-12-06T02:06:38.6424656Z Dec 06 02:06:38 at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-12-06T02:06:38.6425321Z Dec 06 02:06:38 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2020-12-06T02:06:38.6426057Z Dec 06 02:06:38 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2020-12-06T02:06:38.6426766Z Dec 06 02:06:38 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2020-12-06T02:06:38.6427478Z Dec 06 02:06:38 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2020-12-06T02:06:38.6428232Z Dec 06 02:06:38 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> 2020-12-06T02:06:38.6428999Z Dec 06 02:06:38 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> 2020-12-06T02:06:38.6429707Z Dec 06 02:06:38 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 2020-12-06T02:06:38.6430292Z Dec 06 02:06:38 at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 9a256ec6b644a0f795376076f4e3ff4e85aaad1e UNKNOWN * b7dca63a2e98f09c8c780a21091b5268d897b220 UNKNOWN * 2b7bb05a4dbbb0883db566589dfe1975fdbb279d UNKNOWN * 4fd134403d535e074edf3b6a82b540e88709f4ec Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17807) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on pull request #15883: [FLINK-22556][ci] Extend JarFileChecker to search for traces of incompatible licenses
rmetzger commented on pull request #15883: URL: https://github.com/apache/flink/pull/15883#issuecomment-837057278 I tested the functionality of this change here: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=9085=logs=9401bf33-03c4-5a24-83fe-e51d75db73ef=3965ec8d-ff2e-54b6-c0a6-70172f3e6f55 ✅ Once the build is passing and my comments are resolved, I'll approve the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] rmetzger commented on a change in pull request #15883: [FLINK-22556][ci] Extend JarFileChecker to search for traces of incompatible licenses
rmetzger commented on a change in pull request #15883:
URL: https://github.com/apache/flink/pull/15883#discussion_r629561789
##
File path:
tools/ci/java-ci-tools/src/main/java/org/apache/flink/tools/ci/licensecheck/JarFileChecker.java
##
@@ -134,6 +140,91 @@ private static boolean licenseFileExistsAndIsValid(Path
licenseFile, Path jar)
return true;
}
+private static int getFilesWithIncompatibleLicenses(Path jar, Path
jarRoot) throws IOException {
+return findNonBinaryFilesContainingText(
+jar,
+jarRoot,
+asPatterns(
+"GNU Lesser General Public License",
+"GNU General Public License",
+"GPL", // also detects LGPL
+"GNU Affero General Public License",
+"Amazon Software License",
+"Confluent Community License Agreement Version 1.0",
+"Don’t be evil") // can sometimes be found in "funny"
licenses
+);
+}
+
+private static Collection asPatterns(String... texts) {
+return Stream.of(texts)
+.map(JarFileChecker::asPatternWithPotentialLineBreaks)
+.collect(Collectors.toList());
+}
+
+private static Pattern asPatternWithPotentialLineBreaks(String text) {
+return Pattern.compile(text.toLowerCase(Locale.ROOT).replaceAll(" ", "
?R?[s/#]*"));
+}
+
+private static int findNonBinaryFilesContainingText(
+Path jar, Path jarRoot, Collection forbidden) throws
IOException {
+try (Stream files = Files.walk(jarRoot)) {
+return files.filter(path -> !path.equals(jarRoot))
+.filter(path -> !Files.isDirectory(path))
+.filter(JarFileChecker::isNoClassFile)
+// frequent false-positives due to dual-licensing;
generated by maven
+.filter(path -> !getFileName(path).equals("dependencies"))
+// false-positives due to dual-licensing; use startsWith
to cover .txt/.md files
+.filter(path -> !getFileName(path).startsWith("license"))
+// false-positives due to optional components; startsWith
covers .txt/.md files
+.filter(path -> !getFileName(path).startsWith("notice"))
+// dual-licensed under GPL 2 and CDDL 1.1
+// contained in hadoop/presto S3 FS and flink-dist
+.filter(path -> !pathStartsWith(path,
"/META-INF/versions/11/javax/xml/bind"))
+.filter(path -> !(isJavaxManifest(jar, path)))
+// dual-licensed under GPL 2 and EPL 2.0
+// contained in sql-avro-confluent-registry
+.filter(path -> !pathStartsWith(path,
"/org/glassfish/jersey/internal"))
+.map(
+path -> {
+try {
+final String fileContents;
+try {
+fileContents =
+Files.readString(path,
StandardCharsets.UTF_8)
+
.toLowerCase(Locale.ROOT);
+} catch (MalformedInputException mie) {
+// binary file
+return 0;
+}
+
+int violations = 0;
+for (Pattern text : forbidden) {
+if (text.matcher(fileContents).find())
{
+// do not count individual
violations because it can be
+// confusing when checking with
aliases for the same
+// license
+violations = 1;
+LOG.error(
+"File '{}' in jar '{}'
contains forbidden text '{}'.",
Review comment:
```suggestion
"File '{}' in jar '{}'
contains match with forbidden regex '{}'.",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15886: [hotfix] [docs] Fix typo in k8s docs
flinkbot commented on pull request #15886: URL: https://github.com/apache/flink/pull/15886#issuecomment-837043855 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit c78d77a6effa454bc4c4a02b05d9037b5f4c5059 (Mon May 10 17:51:46 UTC 2021) **Warnings:** * Documentation files were touched, but no `docs/content.zh/` files: Update Chinese documentation or file Jira ticket. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22613) FlinkKinesisITCase.testStopWithSavepoint fails
[
https://issues.apache.org/jira/browse/FLINK-22613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342044#comment-17342044
]
Matthias commented on FLINK-22613:
--
https://dev.azure.com/mapohl/flink/_build/results?buildId=420=logs=dafbab6d-4616-5d7b-ee37-3c54e4828fd7=777327ab-6d4e-582e-3e76-4a9391c57e59
> FlinkKinesisITCase.testStopWithSavepoint fails
> --
>
> Key: FLINK-22613
> URL: https://issues.apache.org/jira/browse/FLINK-22613
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
>Affects Versions: 1.13.0, 1.12.3
>Reporter: Guowei Ma
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-05-10T03:09:18.4601182Z May 10 03:09:18 [ERROR]
> testStopWithSavepoint(org.apache.flink.streaming.connectors.kinesis.FlinkKinesisITCase)
> Time elapsed: 3.526 s <<< FAILURE!
> 2021-05-10T03:09:18.4601884Z May 10 03:09:18 java.lang.AssertionError:
> 2021-05-10T03:09:18.4605902Z May 10 03:09:18
> 2021-05-10T03:09:18.4616154Z May 10 03:09:18 Expected: a collection with size
> a value less than <10>
> 2021-05-10T03:09:18.4616818Z May 10 03:09:18 but: collection size <10>
> was equal to <10>
> 2021-05-10T03:09:18.4618087Z May 10 03:09:18 at
> org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> 2021-05-10T03:09:18.4618702Z May 10 03:09:18 at
> org.junit.Assert.assertThat(Assert.java:956)
> 2021-05-10T03:09:18.4619467Z May 10 03:09:18 at
> org.junit.Assert.assertThat(Assert.java:923)
> 2021-05-10T03:09:18.4620391Z May 10 03:09:18 at
> org.apache.flink.streaming.connectors.kinesis.FlinkKinesisITCase.testStopWithSavepoint(FlinkKinesisITCase.java:126)
> 2021-05-10T03:09:18.4621115Z May 10 03:09:18 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-05-10T03:09:18.4621751Z May 10 03:09:18 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-05-10T03:09:18.4622475Z May 10 03:09:18 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-05-10T03:09:18.4623142Z May 10 03:09:18 at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-05-10T03:09:18.4623783Z May 10 03:09:18 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-05-10T03:09:18.4624514Z May 10 03:09:18 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-05-10T03:09:18.4625246Z May 10 03:09:18 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-05-10T03:09:18.4625967Z May 10 03:09:18 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-05-10T03:09:18.4626671Z May 10 03:09:18 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-05-10T03:09:18.4627349Z May 10 03:09:18 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> 2021-05-10T03:09:18.4627979Z May 10 03:09:18 at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-05-10T03:09:18.4628582Z May 10 03:09:18 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-05-10T03:09:18.4629251Z May 10 03:09:18 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-05-10T03:09:18.4629950Z May 10 03:09:18 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-05-10T03:09:18.4630616Z May 10 03:09:18 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-05-10T03:09:18.4631339Z May 10 03:09:18 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-05-10T03:09:18.4631986Z May 10 03:09:18 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-05-10T03:09:18.4632630Z May 10 03:09:18 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2021-05-10T03:09:18.4633269Z May 10 03:09:18 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2021-05-10T03:09:18.4634016Z May 10 03:09:18 at
> org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30)
> 2021-05-10T03:09:18.4634786Z May 10 03:09:18 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> 2021-05-10T03:09:18.4635412Z May 10 03:09:18 at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-05-10T03:09:18.4635995Z May 10 03:09:18 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2021-05-10T03:09:18.4636656Z May 10 03:09:18 at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> 2021-05-10T03:09:18.4637398Z May 10 03:09:18 at
>
[GitHub] [flink] kbohinski opened a new pull request #15886: [hotfix] [docs] Fix typo in k8s docs
kbohinski opened a new pull request #15886: URL: https://github.com/apache/flink/pull/15886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15885: [FLINK-22376][runtime] RecoveredChannelStateHandler recycles the buffer if it was created inside and doesn't recycle if it was passed
flinkbot edited a comment on pull request #15885: URL: https://github.com/apache/flink/pull/15885#issuecomment-837001445 ## CI report: * ffab5d3ccda17f4ec347528d851ba39d5a59cca6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17815) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15873: [FLINK-22413][runtime-web] Hide Checkpointing page in the UI for batch jobs
flinkbot edited a comment on pull request #15873: URL: https://github.com/apache/flink/pull/15873#issuecomment-836246953 ## CI report: * 162666773fa72d571a08b1b0d55e49355f2dcca6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17805) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15885: [FLINK-22376][runtime] RecoveredChannelStateHandler recycles the buffer if it was created inside and doesn't recycle if it was passed as par
flinkbot commented on pull request #15885: URL: https://github.com/apache/flink/pull/15885#issuecomment-837001445 ## CI report: * ffab5d3ccda17f4ec347528d851ba39d5a59cca6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * 61748f522c99468ce7c5b3dd37b11319f3c7b977 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17695) * b979ed53178630a7bf881d3e021993d0ae803449 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17814) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15885: [FLINK-22376][runtime] RecoveredChannelStateHandler recycles the buffer if it was created inside and doesn't recycle if it was passed as par
flinkbot commented on pull request #15885: URL: https://github.com/apache/flink/pull/15885#issuecomment-836966255 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ffab5d3ccda17f4ec347528d851ba39d5a59cca6 (Mon May 10 16:57:54 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-22376).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22376) SequentialChannelStateReaderImpl may recycle buffer twice
[ https://issues.apache.org/jira/browse/FLINK-22376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-22376: --- Labels: pull-request-available (was: ) > SequentialChannelStateReaderImpl may recycle buffer twice > - > > Key: FLINK-22376 > URL: https://issues.apache.org/jira/browse/FLINK-22376 > Project: Flink > Issue Type: Bug > Components: Runtime / Network, Runtime / Task >Affects Versions: 1.13.0 >Reporter: Roman Khachatryan >Priority: Critical > Labels: pull-request-available > Fix For: 1.14.0, 1.13.1 > > > In ChannelStateChunkReader.readChunk in case of error buffer is recycled in > the catch block. However, it might already have been recycled in > stateHandler.recover(). > Using minor priority, as this only affects already failing path. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] akalash opened a new pull request #15885: [FLINK-22376][runtime] RecoveredChannelStateHandler recycles the buffer if it was created inside and doesn't recycle if it was passed as para
akalash opened a new pull request #15885: URL: https://github.com/apache/flink/pull/15885 ## What is the purpose of the change *PR organizes the order of recycling of buffer inside of RecoveredChannelStateHandler * ## Brief change log - *BufferBuilder is able to recycle itself if there is no consumer was created* - *InputChannelRecoveredStateHandler doesn't recycle buffer inside of recover* - *ResultSubpartitionRecoveredStateHandler create buffer consumer inside recover rather than receiving it as parameter* ## Verifying this change This change added tests and can be verified as follows: - *Added tests for BufferBuilder, InputChannelRecoveredStateHandler, ResultSubpartitionRecoveredStateHandler* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15880: [BP-1.13][FLINK-22566][test] Adds NodeManager log extraction to YARN-related e2e tests
flinkbot edited a comment on pull request #15880: URL: https://github.com/apache/flink/pull/15880#issuecomment-836553806 ## CI report: * 632e575bb187fbde54ef7660f341feda590494a9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17800) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * 61748f522c99468ce7c5b3dd37b11319f3c7b977 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17695) * b979ed53178630a7bf881d3e021993d0ae803449 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15884: [FLINK-22266] Fix stop-with-savepoint operation in AdaptiveScheduler
flinkbot edited a comment on pull request #15884: URL: https://github.com/apache/flink/pull/15884#issuecomment-836859750 ## CI report: * e28e6f0fae5e5688149d613d93aa621d24923de4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17813) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15881: [BP-1.12][FLINK-22566][test] Adds NodeManager log extraction to YARN-related e2e tests
flinkbot edited a comment on pull request #15881: URL: https://github.com/apache/flink/pull/15881#issuecomment-836553947 ## CI report: * 52fa6b853cb34075be048e970d5195e48481dd84 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17801) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 9a256ec6b644a0f795376076f4e3ff4e85aaad1e UNKNOWN * b7dca63a2e98f09c8c780a21091b5268d897b220 UNKNOWN * 2b7bb05a4dbbb0883db566589dfe1975fdbb279d UNKNOWN * daaa01d17c62036d59504878a2f5e3d8812094f7 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17799) * 4fd134403d535e074edf3b6a82b540e88709f4ec Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17807) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] sjwiesman commented on a change in pull request #15858: [WIP][FLINK-22067][tests] Wait for vertices to start using API
sjwiesman commented on a change in pull request #15858:
URL: https://github.com/apache/flink/pull/15858#discussion_r629482995
##
File path:
flink-libraries/flink-state-processing-api/src/test/java/org/apache/flink/state/api/utils/SavepointTestBase.java
##
@@ -56,16 +64,33 @@
try {
JobID jobID = client.submitJob(jobGraph).get();
-return CompletableFuture.runAsync(waitingFunction::await)
-.thenCompose(ignore -> triggerSavepoint(client, jobID))
-.get(5, TimeUnit.MINUTES);
+waitForAllRunningOrSomeTerminal(jobID);
+
+return triggerSavepoint(client, jobID).get(5, TimeUnit.MINUTES);
} catch (Exception e) {
throw new RuntimeException("Failed to take savepoint", e);
} finally {
client.cancel(jobId);
}
}
+private void waitForAllRunningOrSomeTerminal(JobID jobID) throws Exception
{
+while (true) {
Review comment:
Should this have some sort of timeout? Or are we happy to let the CI
kill the process after some time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] sjwiesman commented on a change in pull request #15858: [WIP][FLINK-22067][tests] Wait for vertices to start using API
sjwiesman commented on a change in pull request #15858:
URL: https://github.com/apache/flink/pull/15858#discussion_r629482322
##
File path:
flink-libraries/flink-state-processing-api/src/test/java/org/apache/flink/state/api/utils/SavepointTestBase.java
##
@@ -33,15 +35,21 @@
import java.io.IOException;
import java.util.Arrays;
import java.util.Collection;
+import java.util.EnumSet;
+import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.flink.runtime.execution.ExecutionState.RUNNING;
/** A test base that includes utilities for taking a savepoint. */
public abstract class SavepointTestBase extends AbstractTestBase {
public String takeSavepoint(
-T waitingFunction, Function
jobGraphFactory) {
+T waitingFunction /*todo: inline and delete class?*/,
Review comment:
If it can be safely inlined then +1. The more we can simplify the
better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-22067) SavepointWindowReaderITCase.testApplyEvictorWindowStateReader
[
https://issues.apache.org/jira/browse/FLINK-22067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341983#comment-17341983
]
Seth Wiesman commented on FLINK-22067:
--
[~roman_khachatryan] I agree FLINK-22481 is likely a duplicate. I've added a
few notes to the draft PR but the approach makes sense to me and I'm happy to
drop WaitingFunction / simplify the test infrastructure here.
> SavepointWindowReaderITCase.testApplyEvictorWindowStateReader
> -
>
> Key: FLINK-22067
> URL: https://issues.apache.org/jira/browse/FLINK-22067
> Project: Flink
> Issue Type: Bug
> Components: API / State Processor
>Affects Versions: 1.13.0, 1.14.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Labels: auto-deprioritized-critical, pull-request-available,
> test-stability
> Attachments: isolated_logs_builD_9072.log
>
>
> The test case
> {{SavepointWindowReaderITCase.testApplyEvictorWindowStateReader}} failed on
> AZP with:
> {code}
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.takeSavepoint(SavepointTestBase.java:69)
> ... 33 more
> Caused by: java.util.concurrent.TimeoutException: Invocation of public
> default java.util.concurrent.CompletableFuture
> org.apache.flink.runtime.webmonitor.RestfulGateway.triggerSavepoint(org.apache.flink.api.common.JobID,java.lang.String,boolean,org.apache.flink.api.common.time.Time)
> timed out.
> at com.sun.proxy.$Proxy32.triggerSavepoint(Unknown Source)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.lambda$triggerSavepoint$8(MiniCluster.java:716)
> at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> at
> java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:628)
> at
> java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1996)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.runDispatcherCommand(MiniCluster.java:751)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.triggerSavepoint(MiniCluster.java:714)
> at
> org.apache.flink.client.program.MiniClusterClient.triggerSavepoint(MiniClusterClient.java:101)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.triggerSavepoint(SavepointTestBase.java:93)
> at
> org.apache.flink.state.api.utils.SavepointTestBase.lambda$takeSavepoint$0(SavepointTestBase.java:68)
> at
> java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:966)
> at
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
> at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1646)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1632)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> Caused by: akka.pattern.AskTimeoutException: Ask timed out on
> [Actor[akka://flink/user/rpc/dispatcher_2#-390276455]] after [1 ms].
> Message of type [org.apache.flink.runtime.rpc.messages.LocalFencedMessage]. A
> typical reason for `AskTimeoutException` is that the recipient actor didn't
> send a reply.
> at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635)
> at akka.pattern.PromiseActorRef$$anonfun$2.apply(AskSupport.scala:635)
> at
> akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:648)
> at akka.actor.Scheduler$$anon$4.run(Scheduler.scala:205)
> at
> scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:601)
> at
> scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
> at
> scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:599)
> at
> akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:328)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.executeBucket$1(LightArrayRevolverScheduler.scala:279)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.nextTick(LightArrayRevolverScheduler.scala:283)
> at
> akka.actor.LightArrayRevolverScheduler$$anon$4.run(LightArrayRevolverScheduler.scala:235)
> at java.lang.Thread.run(Thread.java:748)
> {code}
>
[GitHub] [flink] flinkbot edited a comment on pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
flinkbot edited a comment on pull request #11877: URL: https://github.com/apache/flink/pull/11877#issuecomment-618273998 ## CI report: * 4f4ce33b815167b23c8ee3dc8d65e1e97cc0cd07 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17794) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tillrohrmann commented on pull request #15832: [FLINK-22494][ha] Introduce PossibleInconsistentStateException
tillrohrmann commented on pull request #15832: URL: https://github.com/apache/flink/pull/15832#issuecomment-836866043 Feel free to also open the PRs for the backports so that we get CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tillrohrmann commented on a change in pull request #15832: [FLINK-22494][ha] Introduce PossibleInconsistentStateException
tillrohrmann commented on a change in pull request #15832:
URL: https://github.com/apache/flink/pull/15832#discussion_r628363445
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/persistence/PossibleInconsistentStateException.java
##
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.persistence;
+
+/**
+ * {@code PossibleInconsistentStateException} represents errors that might
have lead to an
+ * inconsistent state within the HA resources.
+ */
+public class PossibleInconsistentStateException extends Exception {
Review comment:
Shall we let this exception extend from `FlinkException`?
##
File path:
flink-kubernetes/src/main/java/org/apache/flink/kubernetes/highavailability/KubernetesStateHandleStore.java
##
@@ -114,21 +118,28 @@ public KubernetesStateHandleStore(
* @param key Key in ConfigMap
* @param state State to be added
* @throws AlreadyExistException if the name already exists
+ * @throws PossibleInconsistentStateException if the write-to-Kubernetes
operation failed. This
+ * indicates that it's not clear whether the new state was
successfully written to
+ * Kubernetes or not. No state was discarded. Proper error handling
has to be applied on the
+ * caller's side.
* @throws Exception if persisting state or writing state handle failed
*/
@Override
-public RetrievableStateHandle addAndLock(String key, T state) throws
Exception {
+public RetrievableStateHandle addAndLock(String key, T state)
+throws PossibleInconsistentStateException, Exception {
checkNotNull(key, "Key in ConfigMap.");
checkNotNull(state, "State.");
final RetrievableStateHandle storeHandle = storage.store(state);
-boolean success = false;
+final byte[] serializedStoreHandle = serializeStateHandle(storeHandle);
+// initialize flag to serve the failure case
+boolean discardState = true;
try {
-final byte[] serializedStoreHandle =
InstantiationUtil.serializeObject(storeHandle);
-success =
-kubeClient
+// a successful operation will result in the state not being
discarded
+discardState =
+!kubeClient
Review comment:
I think this is personal taste: I find it easier to think about it with
`success` and w/o the negation here.
##
File path:
flink-kubernetes/src/main/java/org/apache/flink/kubernetes/highavailability/KubernetesStateHandleStore.java
##
@@ -470,6 +508,26 @@ public String toString() {
return this.getClass().getSimpleName() + "{configMapName='" +
configMapName + "'}";
}
+private byte[] serializeStateHandle(RetrievableStateHandle
retrievableStateHandle)
+throws Exception {
+try {
+// Serialize the state handle. This writes the state to the
backend.
+return InstantiationUtil.serializeObject(retrievableStateHandle);
+} catch (Exception e) {
+discardState(retrievableStateHandle);
Review comment:
I think here it would be good to catch a possible exception and add it
as a suppressed exception to `e`. Otherwise, we might lose it.
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java
##
@@ -196,27 +231,74 @@ public void replace(String pathInZooKeeper,
IntegerResourceVersion expectedVersi
RetrievableStateHandle newStateHandle = storage.store(state);
-boolean success = false;
+final byte[] serializedStateHandle =
serializeStateHandle(newStateHandle);
+// initialize flags to serve the failure case
+boolean discardOldState = false;
+boolean discardNewState = true;
try {
-// Serialize the new state handle. This writes the state to the
backend.
-byte[] serializedStateHandle =
InstantiationUtil.serializeObject(newStateHandle);
+setStateHandle(path, serializedStateHandle,
expectedVersion.getValue());
+
+// swap subject
[GitHub] [flink] flinkbot edited a comment on pull request #15882: [FLINK-22574] Adaptive Scheduler: Fix cancellation while in Restarting state
flinkbot edited a comment on pull request #15882: URL: https://github.com/apache/flink/pull/15882#issuecomment-836740251 ## CI report: * 36372cc0645878382994753f450bbb7ace00142f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17809) * 89dbb13e84a099927fe0eb1ac7e5e7bcdb79f08f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17811) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15884: [FLINK-22266] Fix stop-with-savepoint operation in AdaptiveScheduler
flinkbot commented on pull request #15884: URL: https://github.com/apache/flink/pull/15884#issuecomment-836859750 ## CI report: * e28e6f0fae5e5688149d613d93aa621d24923de4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15883: [FLINK-22556][ci] Extend JarFileChecker to search for traces of incompatible licenses
flinkbot edited a comment on pull request #15883: URL: https://github.com/apache/flink/pull/15883#issuecomment-836815823 ## CI report: * d6632569dd2adc8ce52aee2fd2f728425137e0f0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17812) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
