[GitHub] [flink] flinkbot edited a comment on pull request #14724: [FLINK-19360] Fink startup fails when $JAVA_HOME have space
flinkbot edited a comment on pull request #14724: URL: https://github.com/apache/flink/pull/14724#issuecomment-765052737 ## CI report: * 79d610f87c58b7c392d4a6cd46fb24eb9f7850c8 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12688) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] PatrickRen commented on pull request #14789: [FLINK-21159][connector/kafka] Send NoMoreSplitsEvent to all readers even if the reader is not assigned with any splits
PatrickRen commented on pull request #14789: URL: https://github.com/apache/flink/pull/14789#issuecomment-770181843 I noticed a race condition in my modification: the variable ```noMoreNewPartitionSplits``` may not be set when ```addReader``` is invoked because they are in different threads, which will cause not sending NoMoreSplitsEvent to some readers. I'll make another commit later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YufeiLiu updated FLINK-21208:
-
Issue Type: Bug (was: Improvement)
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 978, in process_bundle
> element.data)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 218, in process_encoded
> self.output(decoded_value)
> File "apache_beam/runners/worker/operations.py", line 330, in
> apache_beam.runners.worker.operations.Operation.output
> File "apache_beam/runners/worker/operations.py", line 332, in
> apache_beam.runners.worker.operations.Operation.output
> File "apache_beam/run
[jira] [Updated] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YufeiLiu updated FLINK-21208:
-
Affects Version/s: 1.12.1
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 978, in process_bundle
> element.data)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 218, in process_encoded
> self.output(decoded_value)
> File "apache_beam/runners/worker/operations.py", line 330, in
> apache_beam.runners.worker.operations.Operation.output
> File "apache_beam/runners/worker/operations.py", line 332, in
> apache_beam.runners.worker.operations.Operation.output
> File "apache_beam/runne
[jira] [Commented] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275546#comment-17275546
]
YufeiLiu commented on FLINK-21208:
--
I've found that beam worker wasn't exit after job failed. And this bug doesn't
appear if I kill all worker process.
cc [~dian.fu]]
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 978, in process_bundle
> element.data)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 218, in process_encoded
> self.output(decoded_value)
> File "apache_beam/runners/worker/operations.py", line 330, in
> apache_beam.runners.worker.operations.Operat
[GitHub] [flink] flinkbot edited a comment on pull request #14813: [FLINK-21207] Fix 'csv.disable-quote-character' can not take effect during deserial…
flinkbot edited a comment on pull request #14813: URL: https://github.com/apache/flink/pull/14813#issuecomment-769867480 ## CI report: * e5f633baea6ed961c21635feedab97683c429def Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12689) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] pnowojski merged pull request #14797: [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
pnowojski merged pull request #14797: URL: https://github.com/apache/flink/pull/14797 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14816: [FLINK-21202][python] Introduce TimeWindow and CountWindow in PyFlink
flinkbot edited a comment on pull request #14816: URL: https://github.com/apache/flink/pull/14816#issuecomment-770164177 ## CI report: * daece76bc00e4dd9aab27b59c16fd67d57cc24d6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12690) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275546#comment-17275546
]
YufeiLiu edited comment on FLINK-21208 at 1/30/21, 10:25 AM:
-
I found that have start more than one beam worker, and seems share the same
stream, only write schema at begin of stream, the second worker throw this
error. Another question is why beam worker didn't exit after job failure.
cc [~dian.fu]]
was (Author: liuyufei):
I've found that beam worker wasn't exit after job failed. And this bug doesn't
appear if I kill all worker process.
cc [~dian.fu]]
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 978, in process_b
[GitHub] [flink] pnowojski opened a new pull request #14817: backport [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
pnowojski opened a new pull request #14817: URL: https://github.com/apache/flink/pull/14817 This is a backport of https://github.com/apache/flink/pull/14797 to release-1.12 branch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14817: backport [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
flinkbot commented on pull request #14817: URL: https://github.com/apache/flink/pull/14817#issuecomment-770192564 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5c33a060b8790633c3f49761772384fae4e6768c (Sat Jan 30 10:47:15 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21104) UnalignedCheckpointITCase.execute failed with "IllegalStateException"
[
https://issues.apache.org/jira/browse/FLINK-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275551#comment-17275551
]
Piotr Nowojski commented on FLINK-21104:
fix merged to master as a6b909c3e59..a4ccc6e63ca (this range includes
a23744795d9 for FLINK-20654).
Backport to 1.12 pending: https://github.com/apache/flink/pull/14817
> UnalignedCheckpointITCase.execute failed with "IllegalStateException"
> -
>
> Key: FLINK-21104
> URL: https://issues.apache.org/jira/browse/FLINK-21104
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.12.2, 1.13.0
>Reporter: Huang Xingbo
>Assignee: Piotr Nowojski
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.12.2, 1.13.0
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12383&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=f508e270-48d6-5f1e-3138-42a17e0714f0]
> {code:java}
> 2021-01-22T15:17:34.6711152Z [ERROR] execute[Parallel union, p =
> 10](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time
> elapsed: 3.903 s <<< ERROR!
> 2021-01-22T15:17:34.6711736Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-01-22T15:17:34.6712204Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-01-22T15:17:34.6712779Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-01-22T15:17:34.6713344Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-01-22T15:17:34.6713816Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-01-22T15:17:34.6714454Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6714952Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6715472Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238)
> 2021-01-22T15:17:34.6716026Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-01-22T15:17:34.6716631Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-01-22T15:17:34.6717128Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-01-22T15:17:34.6717616Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-01-22T15:17:34.6718105Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-01-22T15:17:34.6718596Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-01-22T15:17:34.6718973Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-01-22T15:17:34.6719364Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-01-22T15:17:34.6719748Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-01-22T15:17:34.6720155Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6720641Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-01-22T15:17:34.6721236Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-01-22T15:17:34.6721706Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-01-22T15:17:34.6722205Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-01-22T15:17:34.6722663Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-01-22T15:17:34.6723214Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-01-22T15:17:34.6723723Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-01-22T15:17:34.6724146Z at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-01-22T15:17:34.6724726Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-01-22T15:17:34.6725198Z at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-01-22T15:17:34.6725861Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6726525Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
> 2021-01-22T15:17:34.6727278Z at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run
[jira] [Commented] (FLINK-20654) Unaligned checkpoint recovery may lead to corrupted data stream
[ https://issues.apache.org/jira/browse/FLINK-20654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275552#comment-17275552 ] Piotr Nowojski commented on FLINK-20654: yet another fix merged to master as a23744795d9 Backport to 1.12 pending: https://github.com/apache/flink/pull/14817 > Unaligned checkpoint recovery may lead to corrupted data stream > --- > > Key: FLINK-20654 > URL: https://issues.apache.org/jira/browse/FLINK-20654 > Project: Flink > Issue Type: Bug > Components: Runtime / Checkpointing >Affects Versions: 1.12.0, 1.12.1 >Reporter: Arvid Heise >Assignee: Piotr Nowojski >Priority: Blocker > Labels: pull-request-available, test-stability > Fix For: 1.12.2, 1.13.0 > > > Fix of FLINK-20433 shows potential corruption after recovery for all > variations of UnalignedCheckpointITCase. > To reproduce, run UCITCase a couple hundreds times. The issue showed for me > in: > - execute [Parallel union, p = 5] > - execute [Parallel union, p = 10] > - execute [Parallel cogroup, p = 5] > - execute [parallel pipeline with remote channels, p = 5] > with decreasing frequency. > The issue manifests as one of the following issues: > - stream corrupted exception > - EOF exception > - assertion failure in NUM_LOST or NUM_OUT_OF_ORDER > - (for union) ArithmeticException overflow (because the number that should be > [0;10] has been mis-deserialized) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #14817: backport [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
flinkbot commented on pull request #14817: URL: https://github.com/apache/flink/pull/14817#issuecomment-770195122 ## CI report: * 3da70442b1cd80555f9c6bab5fdf9903cca9892d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-16017) Improve attachJobGraph Performance
[
https://issues.apache.org/jira/browse/FLINK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275554#comment-17275554
]
Zhu Zhu commented on FLINK-16017:
-
[~rmetzger] yes it is already fixed via FLINK-17180. Will close it.
> Improve attachJobGraph Performance
> --
>
> Key: FLINK-16017
> URL: https://issues.apache.org/jira/browse/FLINK-16017
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Jiayi Liao
>Priority: Major
>
> Currently {{RegionPartitionReleaseStrategy}} and
> {{AdaptedRestartPipelinedRegionStrategyNG}} both need to compute distinct
> pipelined regions, which affects the performance due to the duplicate
> calculating.
> The best idea that comes to my mind so far is calculating distinct pipelined
> regions in {{DefaultExecutionTopology}} so that we can remove the duplicate
> calculating.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-16017) Improve attachJobGraph Performance
[
https://issues.apache.org/jira/browse/FLINK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu closed FLINK-16017.
---
Resolution: Duplicate
> Improve attachJobGraph Performance
> --
>
> Key: FLINK-16017
> URL: https://issues.apache.org/jira/browse/FLINK-16017
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: Jiayi Liao
>Priority: Major
>
> Currently {{RegionPartitionReleaseStrategy}} and
> {{AdaptedRestartPipelinedRegionStrategyNG}} both need to compute distinct
> pipelined regions, which affects the performance due to the duplicate
> calculating.
> The best idea that comes to my mind so far is calculating distinct pipelined
> regions in {{DefaultExecutionTopology}} so that we can remove the duplicate
> calculating.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14817: backport [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
flinkbot edited a comment on pull request #14817: URL: https://github.com/apache/flink/pull/14817#issuecomment-770195122 ## CI report: * 3da70442b1cd80555f9c6bab5fdf9903cca9892d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12692) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14799: [WIP] Wrapped StateBackend to forward state changes to StateChangleLog
flinkbot edited a comment on pull request #14799: URL: https://github.com/apache/flink/pull/14799#issuecomment-769554549 ## CI report: * 7ee60d64feb3da9e9fc24e08482260d4b021c65f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12652) * 89066112157569ad58908e401696b45a56f48fc3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14799: [WIP] Wrapped StateBackend to forward state changes to StateChangleLog
flinkbot edited a comment on pull request #14799: URL: https://github.com/apache/flink/pull/14799#issuecomment-769554549 ## CI report: * 7ee60d64feb3da9e9fc24e08482260d4b021c65f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12652) * 89066112157569ad58908e401696b45a56f48fc3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12693) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14817: backport [FLINK-20654][FLINK-21104][network] Fix couple bugs in the handling of unaligned checkpoints and cancellations.
flinkbot edited a comment on pull request #14817: URL: https://github.com/apache/flink/pull/14817#issuecomment-770195122 ## CI report: * 3da70442b1cd80555f9c6bab5fdf9903cca9892d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12692) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14799: [WIP] Wrapped StateBackend to forward state changes to StateChangleLog
flinkbot edited a comment on pull request #14799: URL: https://github.com/apache/flink/pull/14799#issuecomment-769554549 ## CI report: * 89066112157569ad58908e401696b45a56f48fc3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12693) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Issue Comment Deleted] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YufeiLiu updated FLINK-21208:
-
Comment: was deleted
(was: I found that have start more than one beam worker, and seems share the
same stream, only write schema at begin of stream, the second worker throw this
error. Another question is why beam worker didn't exit after job failure.
cc [~dian.fu]])
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 978, in process_bundle
> element.data)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/bundle_processor.py",
> line 218, in process_encoded
> self.output(decoded_value)
> File "apache_beam/runners/
[GitHub] [flink] gaoyunhaii commented on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
gaoyunhaii commented on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-770215757 Very thanks @guoweiM for the comments and I have update the algorithm accordingly This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * 15a1f2beef1ddf95687fb4a53cf8a6f906559836 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12478) * 2ab49d334140ab47d7eb144964fac540e6f98444 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14754: [FLINK-21127][runtime][checkpoint] Stores finished status for fully finished operators in checkpoint
flinkbot edited a comment on pull request #14754: URL: https://github.com/apache/flink/pull/14754#issuecomment-766965862 ## CI report: * 37484933592e8b5a4f5cf83425ac0b7b612c7480 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12479) * 7a0a38b9960949cedae60f4574edc8382841c8ac UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * 15a1f2beef1ddf95687fb4a53cf8a6f906559836 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12478) * 2ab49d334140ab47d7eb144964fac540e6f98444 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12694) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14754: [FLINK-21127][runtime][checkpoint] Stores finished status for fully finished operators in checkpoint
flinkbot edited a comment on pull request #14754: URL: https://github.com/apache/flink/pull/14754#issuecomment-766965862 ## CI report: * 7a0a38b9960949cedae60f4574edc8382841c8ac Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12695) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gaoyunhaii opened a new pull request #14818: [FLINK-21081][runtime][checkpoint] Re-compute the tasks to trigger if some tasks finished before get triggered
gaoyunhaii opened a new pull request #14818: URL: https://github.com/apache/flink/pull/14818 ## What is the purpose of the change This PR tries to re-trigger the descendent tasks if needed when some tasks finished before get triggered successfully. ## Brief change log - 8c90e675f53a6fd1ae52cf36350b55416470c7b3 refactors the `CheckpointCoordinator` to let `CheckpointCoordinator` decide the thread used to deal with tasks' report. This ensures the `CheckpointCoordinator` has an opportunity to record the in-process reports. - 127c825b086444687af34685da04e72e9fd6bc2c implements the re-triggering logic. ## Verifying this change This change added tests and can be verified via added unit tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive):**no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21081) Re-compute tasks to trigger when tasks get triggered before finished
[ https://issues.apache.org/jira/browse/FLINK-21081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-21081: --- Labels: pull-request-available (was: ) > Re-compute tasks to trigger when tasks get triggered before finished > > > Key: FLINK-21081 > URL: https://issues.apache.org/jira/browse/FLINK-21081 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream, Runtime / Checkpointing >Reporter: Yun Gao >Priority: Major > Labels: pull-request-available > > When considering checkpoint while tasks get to finished, > CheckpointCoordinator computes the tasks to trigger based on the current > finish status, then during trigger, some more tasks might finished. In this > case, CheckpointCoordinator would need to re-compute tasks to trigger and > re-trigger the newly involved tasks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #14818: [FLINK-21081][runtime][checkpoint] Re-compute the tasks to trigger if some tasks finished before get triggered
flinkbot commented on pull request #14818: URL: https://github.com/apache/flink/pull/14818#issuecomment-770226619 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 127c825b086444687af34685da04e72e9fd6bc2c (Sat Jan 30 15:14:47 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-21081).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14818: [FLINK-21081][runtime][checkpoint] Re-compute the tasks to trigger if some tasks finished before get triggered
flinkbot commented on pull request #14818: URL: https://github.com/apache/flink/pull/14818#issuecomment-770227636 ## CI report: * 127c825b086444687af34685da04e72e9fd6bc2c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14818: [FLINK-21081][runtime][checkpoint] Re-compute the tasks to trigger if some tasks finished before get triggered
flinkbot edited a comment on pull request #14818: URL: https://github.com/apache/flink/pull/14818#issuecomment-770227636 ## CI report: * 127c825b086444687af34685da04e72e9fd6bc2c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12696) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275661#comment-17275661
]
YufeiLiu commented on FLINK-21208:
--
I've made some discoveries, I found beam has a config
{{DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S}}=60s, periodicity
shutdown inactive processor, and initialize a new processor need read schema
data from the begining of the stream. The stream was generated by
{{ArrowSerializer}} in flink operator and only serialize schema data once,
afterwards will only write records data.
So, if processor was evict from cache, beam will try to initialize a new
processor with the input stream(only records), and thow expected schema but was
record batch.
cc [~dian.fu]
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 253, in _execute
> response = task()
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 310, in
> lambda: self.create_worker().do_instruction(request), request)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 480, in do_instruction
> getattr(request, request_type), request.instruction_id)
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker.py",
> line 515, in process_bundle
> bundle_processor.process_bundle(instruction_id))
> File
> "/Users/l
[jira] [Comment Edited] (FLINK-21208) pyarrow exception when using window with pandas udaf
[
https://issues.apache.org/jira/browse/FLINK-21208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275661#comment-17275661
]
YufeiLiu edited comment on FLINK-21208 at 1/30/21, 4:01 PM:
I've made some discoveries, I found beam has a config
{{DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S}}=60s in
{{sdk_worker.py}}, periodicity shutdown inactive processor, and initialize a
new processor need read schema data from the begining of the stream. The stream
was generated by {{ArrowSerializer}} in flink operator and only serialize
schema data once, afterwards will only write records data.
So, if processor was evict from cache, beam will try to initialize a new
processor with the input stream(only records), and thow expected schema but was
record batch.
cc [~dian.fu]
was (Author: liuyufei):
I've made some discoveries, I found beam has a config
{{DEFAULT_BUNDLE_PROCESSOR_CACHE_SHUTDOWN_THRESHOLD_S}}=60s, periodicity
shutdown inactive processor, and initialize a new processor need read schema
data from the begining of the stream. The stream was generated by
{{ArrowSerializer}} in flink operator and only serialize schema data once,
afterwards will only write records data.
So, if processor was evict from cache, beam will try to initialize a new
processor with the input stream(only records), and thow expected schema but was
record batch.
cc [~dian.fu]
> pyarrow exception when using window with pandas udaf
>
>
> Key: FLINK-21208
> URL: https://issues.apache.org/jira/browse/FLINK-21208
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.12.0, 1.12.1
>Reporter: YufeiLiu
>Priority: Major
>
> I write a pyflink demo and execute in local environment, the logic is
> simple:generate records and aggerate in 100s tumle window, using a pandas
> udaf.
> But the job failed after several minutes, I don't think it's a resource
> problem because the amount of data is small, here is the error trace.
> {code:java}
> Caused by: org.apache.flink.streaming.runtime.tasks.AsynchronousException:
> Caught exception while processing timer.
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$StreamTaskAsyncExceptionHandler.handleAsyncException(StreamTask.java:1108)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.handleAsyncException(StreamTask.java:1082)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1213)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$null$17(StreamTask.java:1202)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:78)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:302)
> at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:184)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:575)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:539)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:722)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: TimerException{java.lang.RuntimeException: Failed to close remote
> bundle}
> ... 11 more
> Caused by: java.lang.RuntimeException: Failed to close remote bundle
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:371)
> at
> org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.flush(BeamPythonFunctionRunner.java:325)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.invokeFinishBundle(AbstractPythonFunctionOperator.java:291)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.checkInvokeFinishBundleByTime(AbstractPythonFunctionOperator.java:285)
> at
> org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator.lambda$open$0(AbstractPythonFunctionOperator.java:134)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invokeProcessingTimeCallback(StreamTask.java:1211)
> ... 10 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error received from SDK harness for instruction
> 3: Traceback (most recent call last):
> File
> "/Users/lyf/Library/Python/3.7/lib/python/site-packages/apache_beam/runners/worker/sdk_worker
[jira] [Comment Edited] (FLINK-21164) Jar handlers don't cleanup temporarily extracted jars
[
https://issues.apache.org/jira/browse/FLINK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275242#comment-17275242
]
Chesnay Schepler edited comment on FLINK-21164 at 1/30/21, 4:20 PM:
master: 30f6964a9b72c79925c62f952a4559f2e48bdf06
1.12: 43661b5ad6344fab221540622555e8e2ae81019e
1.11: 1aae7f724ca6d24c8d6205552ead72c06cb14e8d
was (Author: zentol):
master: 30f6964a9b72c79925c62f952a4559f2e48bdf06
1.12: 43661b5ad6344fab221540622555e8e2ae81019e
> Jar handlers don't cleanup temporarily extracted jars
> -
>
> Key: FLINK-21164
> URL: https://issues.apache.org/jira/browse/FLINK-21164
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / REST
>Affects Versions: 1.11.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.4, 1.12.2, 1.13.0
>
>
> The jar handlers create a PackagedProgram when running a job or retrieving
> the execution plan. When a PackagedProgram is created and jars contained
> within a jar are automatically extracted to a temporary directory.
> This directory is not automatically cleaned up; this must be done manually.
> The CliFrontend does that within {{#run}}, but the jar handlers do not,
> leading to jars piling up in the filesystem.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-21164) Jar handlers don't cleanup temporarily extracted jars
[
https://issues.apache.org/jira/browse/FLINK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-21164.
Resolution: Fixed
> Jar handlers don't cleanup temporarily extracted jars
> -
>
> Key: FLINK-21164
> URL: https://issues.apache.org/jira/browse/FLINK-21164
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / REST
>Affects Versions: 1.11.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.4, 1.12.2, 1.13.0
>
>
> The jar handlers create a PackagedProgram when running a job or retrieving
> the execution plan. When a PackagedProgram is created and jars contained
> within a jar are automatically extracted to a temporary directory.
> This directory is not automatically cleaned up; this must be done manually.
> The CliFrontend does that within {{#run}}, but the jar handlers do not,
> leading to jars piling up in the filesystem.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * 5bd74e2865ec431f3bf2528f5d85969ca51714fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12684) * 4447d44df71a1bb5b2ac672bfd9736d88dd3183f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14740: [FLINK-21067][runtime][checkpoint] Modify the logic of computing which tasks to trigger/ack/commit to support finished tasks
flinkbot edited a comment on pull request #14740: URL: https://github.com/apache/flink/pull/14740#issuecomment-766340750 ## CI report: * 2ab49d334140ab47d7eb144964fac540e6f98444 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12694) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * 5bd74e2865ec431f3bf2528f5d85969ca51714fb Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12684) * 4447d44df71a1bb5b2ac672bfd9736d88dd3183f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12698) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-9389) Improve error message when cancelling job in state canceled via REST API
[
https://issues.apache.org/jira/browse/FLINK-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275710#comment-17275710
]
DILJIT POOVAKOTIL RAMACHANDRAN commented on FLINK-9389:
---
[~uce] I tried reproducing this on my local machine, it still appears. However
I see a bunch of java stack trace with the error message which is kind of ug.
Can I work on this?
The current error response from the api looks like:
{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Job
could not be found.\n\tat
org.apache.flink.runtime.rest.handler.job.JobCancellationHandler.lambda$handleRequest$0(JobCancellationHandler.java:119)\n\tat
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)\n\tat
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)\n\tat
java.util.concurrent.CompletableFuture.completeExcept
> Improve error message when cancelling job in state canceled via REST API
>
>
> Key: FLINK-9389
> URL: https://issues.apache.org/jira/browse/FLINK-9389
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / REST
>Affects Versions: 1.5.0
>Reporter: Ufuk Celebi
>Priority: Minor
> Labels: starter
>
> - Request job that has been already cancelled
> {code}
> $ curl http://localhost:8081/jobs/b577a914ecdf710d4b93a84105dea0c9
> {"jid":"b577a914ecdf710d4b93a84105dea0c9","name":"CarTopSpeedWindowingExample","isStoppable":false,"state":"CANCELED",
> ...omitted...}
> {code}
> - Cancel job again
> {code}
> $ curl -XPATCH
> http://localhost:8081/jobs/b577a914ecdf710d4b93a84105dea0c9\?mode\=cancel
> {"errors":["Job could not be found."]} (HTTP 404)
> {code}
> Since the actual job resource still exists, I think status code 404 is
> confusing. I think a better message should indicate that the job is already
> in state CANCELED.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-9389) Improve error message when cancelling job in state canceled via REST API
[
https://issues.apache.org/jira/browse/FLINK-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275710#comment-17275710
]
DILJIT POOVAKOTIL RAMACHANDRAN edited comment on FLINK-9389 at 1/30/21, 8:50 PM:
-
[~uce] I tried reproducing this on my local machine, it still appears. However
I see a bunch of java stack trace with the error message which is kind of ug.
Can I work on this?
The current error response from the api looks like following:
{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Job
could not be found.\n\tat
org.apache.flink.runtime.rest.handler.job.JobCancellationHandler.lambda$handleRequest$0(JobCancellationHandler.java:119)\n\tat
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)\n\tat
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)\n\tat
java.util.concurrent.CompletableFuture.completeExcept
was (Author: diljitpr_dev):
[~uce] I tried reproducing this on my local machine, it still appears. However
I see a bunch of java stack trace with the error message which is kind of ug.
Can I work on this?
The current error response from the api looks like:
{"errors":["org.apache.flink.runtime.rest.handler.RestHandlerException: Job
could not be found.\n\tat
org.apache.flink.runtime.rest.handler.job.JobCancellationHandler.lambda$handleRequest$0(JobCancellationHandler.java:119)\n\tat
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836)\n\tat
java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811)\n\tat
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)\n\tat
java.util.concurrent.CompletableFuture.completeExcept
> Improve error message when cancelling job in state canceled via REST API
>
>
> Key: FLINK-9389
> URL: https://issues.apache.org/jira/browse/FLINK-9389
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / REST
>Affects Versions: 1.5.0
>Reporter: Ufuk Celebi
>Priority: Minor
> Labels: starter
>
> - Request job that has been already cancelled
> {code}
> $ curl http://localhost:8081/jobs/b577a914ecdf710d4b93a84105dea0c9
> {"jid":"b577a914ecdf710d4b93a84105dea0c9","name":"CarTopSpeedWindowingExample","isStoppable":false,"state":"CANCELED",
> ...omitted...}
> {code}
> - Cancel job again
> {code}
> $ curl -XPATCH
> http://localhost:8081/jobs/b577a914ecdf710d4b93a84105dea0c9\?mode\=cancel
> {"errors":["Job could not be found."]} (HTTP 404)
> {code}
> Since the actual job resource still exists, I think status code 404 is
> confusing. I think a better message should indicate that the job is already
> in state CANCELED.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * 4447d44df71a1bb5b2ac672bfd9736d88dd3183f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12698) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * 4447d44df71a1bb5b2ac672bfd9736d88dd3183f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12698) * e7dbab13079f1a1f2350320930cc216c8ed42759 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * 4447d44df71a1bb5b2ac672bfd9736d88dd3183f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12698) * e7dbab13079f1a1f2350320930cc216c8ed42759 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12702) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21213) Elasticsearch (v5.3.3) sink end-to-end test' failed
[ https://issues.apache.org/jira/browse/FLINK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-21213: -- Labels: test-stability (was: ) > Elasticsearch (v5.3.3) sink end-to-end test' failed > > > Key: FLINK-21213 > URL: https://issues.apache.org/jira/browse/FLINK-21213 > Project: Flink > Issue Type: Bug > Components: Connectors / ElasticSearch >Affects Versions: 1.13.0 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12686&view=logs&j=68a897ab-3047-5660-245a-cce8f83859f6&t=d47e27f5-9721-5d5f-1cf3-62adbf3d115d] > > Checking for non-empty .out files... > No non-empty .out files. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21213) Elasticsearch (v5.3.3) sink end-to-end test' failed
Guowei Ma created FLINK-21213: - Summary: Elasticsearch (v5.3.3) sink end-to-end test' failed Key: FLINK-21213 URL: https://issues.apache.org/jira/browse/FLINK-21213 Project: Flink Issue Type: Bug Components: Connectors / ElasticSearch Affects Versions: 1.13.0 Reporter: Guowei Ma [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12686&view=logs&j=68a897ab-3047-5660-245a-cce8f83859f6&t=d47e27f5-9721-5d5f-1cf3-62adbf3d115d] Checking for non-empty .out files... No non-empty .out files. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21214) FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint Failed
Guowei Ma created FLINK-21214: - Summary: FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint Failed Key: FLINK-21214 URL: https://issues.apache.org/jira/browse/FLINK-21214 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.12.0 Reporter: Guowei Ma [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12687&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5] [ERROR] testScaleDownBeforeFirstCheckpoint(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase) Time elapsed: 62.857 s <<< ERROR! org.apache.kafka.common.errors.TimeoutException: org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21215) UnalignedCheckpointITCase.execute
Guowei Ma created FLINK-21215: - Summary: UnalignedCheckpointITCase.execute Key: FLINK-21215 URL: https://issues.apache.org/jira/browse/FLINK-21215 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.13.0 Reporter: Guowei Ma [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12691&view=logs&j=34f41360-6c0d-54d3-11a1-0292a2def1d9&t=2d56e022-1ace-542f-bf1a-b37dd63243f2&l=9146] ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 101.136 s <<< FAILURE! - in org.apache.flink.test.checkpointing.UnalignedCheckpointITCase [ERROR] execute[parallel pipeline with local channels, p = 5](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time elapsed: 2.263 s <<< ERROR! org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137) at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046) at akka.dispatch.OnComplete.internal(Future.scala:264) at akka.dispatch.OnComplete.internal(Future.scala:261) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:191) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36) at org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73) at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252) at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572) at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22) at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21) at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436) at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-21215) UnalignedCheckpointITCase.execute
[ https://issues.apache.org/jira/browse/FLINK-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-21215: -- Description: [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12691&view=logs&j=34f41360-6c0d-54d3-11a1-0292a2def1d9&t=2d56e022-1ace-542f-bf1a-b37dd63243f2&l=9146] ... 4 more Caused by: org.apache.flink.util.FlinkRuntimeException: Exceeded checkpoint tolerable failure threshold. at org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleCheckpointException(CheckpointFailureManager.java:98) at org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleTaskLevelCheckpointException(CheckpointFailureManager.java:84) at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.abortPendingCheckpoint(CheckpointCoordinator.java:1930) at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:1007) at org.apache.flink.runtime.scheduler.SchedulerBase.lambda$declineCheckpoint$9(SchedulerBase.java:1009) at org.apache.flink.runtime.scheduler.SchedulerBase.lambda$processCheckpointCoordinatorMessage$10(SchedulerBase.java:1025) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) was: [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12691&view=logs&j=34f41360-6c0d-54d3-11a1-0292a2def1d9&t=2d56e022-1ace-542f-bf1a-b37dd63243f2&l=9146] ERROR] Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 101.136 s <<< FAILURE! - in org.apache.flink.test.checkpointing.UnalignedCheckpointITCase [ERROR] execute[parallel pipeline with local channels, p = 5](org.apache.flink.test.checkpointing.UnalignedCheckpointITCase) Time elapsed: 2.263 s <<< ERROR! org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137) at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:238) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046) at akka.dispatch.OnComplete.internal(Future.scala:264) at akka.dispatch.OnComplete.internal(Future.scala:261) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:191) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36) at org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73) at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252) at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572) at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22) at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21) at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436) at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91) at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(B
[jira] [Updated] (FLINK-21215) UnalignedCheckpointITCase.execute Failed
[ https://issues.apache.org/jira/browse/FLINK-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-21215: -- Summary: UnalignedCheckpointITCase.execute Failed (was: UnalignedCheckpointITCase.execute) > UnalignedCheckpointITCase.execute Failed > > > Key: FLINK-21215 > URL: https://issues.apache.org/jira/browse/FLINK-21215 > Project: Flink > Issue Type: Bug > Components: Runtime / Checkpointing >Affects Versions: 1.13.0 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12691&view=logs&j=34f41360-6c0d-54d3-11a1-0292a2def1d9&t=2d56e022-1ace-542f-bf1a-b37dd63243f2&l=9146] > > > ... 4 more > Caused by: org.apache.flink.util.FlinkRuntimeException: Exceeded checkpoint > tolerable failure threshold. > at > org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleCheckpointException(CheckpointFailureManager.java:98) > > at > org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleTaskLevelCheckpointException(CheckpointFailureManager.java:84) > > at > org.apache.flink.runtime.checkpoint.CheckpointCoordinator.abortPendingCheckpoint(CheckpointCoordinator.java:1930) > > at > org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:1007) > > at > org.apache.flink.runtime.scheduler.SchedulerBase.lambda$declineCheckpoint$9(SchedulerBase.java:1009) > > at > org.apache.flink.runtime.scheduler.SchedulerBase.lambda$processCheckpointCoordinatorMessage$10(SchedulerBase.java:1025) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-21216) StreamPandasConversionTests Fails
Guowei Ma created FLINK-21216: - Summary: StreamPandasConversionTests Fails Key: FLINK-21216 URL: https://issues.apache.org/jira/browse/FLINK-21216 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.13.0 Reporter: Guowei Ma [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12699&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3] === FAILURES === ___ StreamPandasConversionTests.test_empty_to_pandas ___ self = def test_empty_to_pandas(self): > table = self.t_env.from_pandas(self.pdf, self.data_type) pyflink/table/tests/test_pandas_conversion.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ pyflink/table/table_environment.py:1462: in from_pandas arrow_schema = pa.Schema.from_pandas(pdf, preserve_index=False) pyarrow/types.pxi:1315: in pyarrow.lib.Schema.from_pandas ??? .tox/py37-cython/lib/python3.7/site-packages/pyarrow/pandas_compat.py:519: in dataframe_to_types type_ = pa.lib._ndarray_to_arrow_type(values, type_) pyarrow/array.pxi:53: in pyarrow.lib._ndarray_to_arrow_type ??? pyarrow/array.pxi:64: in pyarrow.lib._ndarray_to_type ??? _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ??? E pyarrow.lib.ArrowTypeError: Did not pass numpy.dtype object pyarrow/error.pxi:108: ArrowTypeError _ StreamPandasConversionTests.test_from_pandas _ self = def test_from_pandas(self): > table = self.t_env.from_pandas(self.pdf, self.data_type, 5) pyflink/table/tests/test_pandas_conversion.py:120: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21215) UnalignedCheckpointITCase.execute Failed
[ https://issues.apache.org/jira/browse/FLINK-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275756#comment-17275756 ] Guowei Ma commented on FLINK-21215: --- https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12699&view=logs&j=02c4e775-43bf-5625-d1cc-542b5209e072&t=e5961b24-88d9-5c77-efd3-955422674c25 > UnalignedCheckpointITCase.execute Failed > > > Key: FLINK-21215 > URL: https://issues.apache.org/jira/browse/FLINK-21215 > Project: Flink > Issue Type: Bug > Components: Runtime / Checkpointing >Affects Versions: 1.13.0 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12691&view=logs&j=34f41360-6c0d-54d3-11a1-0292a2def1d9&t=2d56e022-1ace-542f-bf1a-b37dd63243f2&l=9146] > > > ... 4 more > Caused by: org.apache.flink.util.FlinkRuntimeException: Exceeded checkpoint > tolerable failure threshold. > at > org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleCheckpointException(CheckpointFailureManager.java:98) > > at > org.apache.flink.runtime.checkpoint.CheckpointFailureManager.handleTaskLevelCheckpointException(CheckpointFailureManager.java:84) > > at > org.apache.flink.runtime.checkpoint.CheckpointCoordinator.abortPendingCheckpoint(CheckpointCoordinator.java:1930) > > at > org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveDeclineMessage(CheckpointCoordinator.java:1007) > > at > org.apache.flink.runtime.scheduler.SchedulerBase.lambda$declineCheckpoint$9(SchedulerBase.java:1009) > > at > org.apache.flink.runtime.scheduler.SchedulerBase.lambda$processCheckpointCoordinatorMessage$10(SchedulerBase.java:1025) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20661) Elasticsearch6DynamicSinkITCase.testWritingDocuments test failed with ConnectException
[
https://issues.apache.org/jira/browse/FLINK-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275757#comment-17275757
]
Guowei Ma commented on FLINK-20661:
---
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12700&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=34f486e1-e1e4-5dd2-9c06-bfdd9b9c74a8]
Caused by: org.elasticsearch.ElasticsearchStatusException: method [HEAD], host
[[http://127.0.0.1:9200],|http://127.0.0.1:9200]%2C/] URI [/], status line
[HTTP/1.1 503 Service Unavailable]
at
org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:625)
at
org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:535)
at
org.elasticsearch.client.RestHighLevelClient.ping(RestHighLevelClient.java:275)
at
org.apache.flink.streaming.connectors.elasticsearch6.Elasticsearch6ApiCallBridge.verifyClientConnection(Elasticsearch6ApiCallBridge.java:134)
at
org.apache.flink.streaming.connectors.elasticsearch6.Elasticsearch6ApiCallBridge.verifyClientConnection(Elasticsearch6ApiCallBridge.java:45)
at
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.open(ElasticsearchSinkBase.java:322)
at
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at
org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:74)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:307)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$1(StreamTask.java:537)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:495)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:558)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:752)
at org.apache.flink.runtime.taskmanager.Task.run(Task.jav
> Elasticsearch6DynamicSinkITCase.testWritingDocuments test failed with
> ConnectException
> --
>
> Key: FLINK-20661
> URL: https://issues.apache.org/jira/browse/FLINK-20661
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ElasticSearch
>Affects Versions: 1.11.0, 1.12.0, 1.13.0
>Reporter: Huang Xingbo
>Priority: Major
> Labels: test-stability
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10989&view=logs&j=ba53eb01-1462-56a3-8e98-0dd97fbcaab5&t=eb5f4d19-2d2d-5856-a4ce-acf5f904a994]
> {code:java}
> 2020-12-17T22:52:41.2992508Z [ERROR] Tests run: 4, Failures: 0, Errors: 1,
> Skipped: 0, Time elapsed: 38.878 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.elasticsearch.table.Elasticsearch6DynamicSinkITCase
> 2020-12-17T22:52:41.2999076Z [ERROR]
> testWritingDocuments(org.apache.flink.streaming.connectors.elasticsearch.table.Elasticsearch6DynamicSinkITCase)
> Time elapsed: 16.409 s <<< ERROR!
> 2020-12-17T22:52:41.3008441Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2020-12-17T22:52:41.3009290Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:147)
> 2020-12-17T22:52:41.3048924Z at
> org.apache.flink.client.program.PerJobMiniClusterFactory$PerJobMiniClusterJobClient.lambda$getJobExecutionResult$2(PerJobMiniClusterFactory.java:186)
> 2020-12-17T22:52:41.3058938Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2020-12-17T22:52:41.3067969Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2020-12-17T22:52:41.3080564Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2020-12-17T22:52:41.3098938Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2020-12-17T22:52:41.3128311Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:229)
> 2020-12-17T22:52:41.3141102Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2020-12-17T22:52:41.3168389Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2020-12-17T22:52:41.3178382Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2020-12-17T22:52:41.3179506Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2020-12-17T22:52:41.3180
[jira] [Created] (FLINK-21217) Resuming Savepoint (rocks, scale up, rocks timers) end-to-end test
Guowei Ma created FLINK-21217: - Summary: Resuming Savepoint (rocks, scale up, rocks timers) end-to-end test Key: FLINK-21217 URL: https://issues.apache.org/jira/browse/FLINK-21217 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.13.0 Reporter: Guowei Ma [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12664&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529] Jan 29 15:20:55 [FAIL] 'Resuming Savepoint (rocks, scale up, rocks timers) end-to-end test' failed after 0 minutes and 37 seconds! Test exited with exit code 0 but the logs contained errors, exceptions or non-empty .out files -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-20523) OffsetsInitializerTest test failed with "java.net.BindException: Address already in use"
[
https://issues.apache.org/jira/browse/FLINK-20523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275760#comment-17275760
]
Guowei Ma commented on FLINK-20523:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12653&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5
> OffsetsInitializerTest test failed with "java.net.BindException: Address
> already in use"
>
>
> Key: FLINK-20523
> URL: https://issues.apache.org/jira/browse/FLINK-20523
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.12.0, 1.13.0
>Reporter: Huang Xingbo
>Priority: Major
> Labels: test-stability
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10597&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5]
> {code:java}
> 2020-12-07T16:26:03.5696966Z [ERROR] Tests run: 2, Failures: 0, Errors: 2,
> Skipped: 0, Time elapsed: 2.011 s <<< FAILURE! - in
> org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializerTest
> 2020-12-07T16:26:03.5702474Z [ERROR]
> org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializerTest
> Time elapsed: 2.011 s <<< ERROR!
> 2020-12-07T16:26:03.5712217Z java.net.BindException: Address already in use
> 2020-12-07T16:26:03.5712769Z at sun.nio.ch.Net.bind0(Native Method)
> 2020-12-07T16:26:03.5713147Z at sun.nio.ch.Net.bind(Net.java:461)
> 2020-12-07T16:26:03.5713555Z at sun.nio.ch.Net.bind(Net.java:453)
> 2020-12-07T16:26:03.5714096Z at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
> 2020-12-07T16:26:03.5714673Z at
> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
> 2020-12-07T16:26:03.5715245Z at
> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
> 2020-12-07T16:26:03.5715890Z at
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
> 2020-12-07T16:26:03.5716585Z at
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:120)
> 2020-12-07T16:26:03.5717279Z at
> org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:93)
> 2020-12-07T16:26:03.5717957Z at
> org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:148)
> 2020-12-07T16:26:03.5718486Z at java.lang.Thread.run(Thread.java:748)
> 2020-12-07T16:26:03.5718766Z
> 2020-12-07T16:26:03.5719289Z [ERROR]
> org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializerTest
> Time elapsed: 2.011 s <<< ERROR!
> 2020-12-07T16:26:03.5719896Z java.lang.NullPointerException
> 2020-12-07T16:26:03.5720659Z at
> org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializerTest.tearDown(OffsetsInitializerTest.java:60)
> 2020-12-07T16:26:03.5721565Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-12-07T16:26:03.5722148Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-12-07T16:26:03.5722806Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-12-07T16:26:03.5723381Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-12-07T16:26:03.5723940Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2020-12-07T16:26:03.5724417Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2020-12-07T16:26:03.5725153Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2020-12-07T16:26:03.5725608Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> 2020-12-07T16:26:03.5726019Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-12-07T16:26:03.5726429Z at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> 2020-12-07T16:26:03.5727029Z at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> 2020-12-07T16:26:03.5727524Z at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> 2020-12-07T16:26:03.5727986Z at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> 2020-12-07T16:26:03.5728491Z at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> 2020-12-07T16:26:03.5729009Z at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> 2020-12-07T16:26:03.5729467Z at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java
[jira] [Updated] (FLINK-21216) StreamPandasConversionTests Fails
[ https://issues.apache.org/jira/browse/FLINK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-21216: -- Affects Version/s: 1.12.0 > StreamPandasConversionTests Fails > - > > Key: FLINK-21216 > URL: https://issues.apache.org/jira/browse/FLINK-21216 > Project: Flink > Issue Type: Bug > Components: API / Python >Affects Versions: 1.12.0, 1.13.0 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12699&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3] > > === FAILURES > === > ___ StreamPandasConversionTests.test_empty_to_pandas > ___ > > self = > testMethod=test_empty_to_pandas> > > def test_empty_to_pandas(self): > > table = self.t_env.from_pandas(self.pdf, self.data_type) > > pyflink/table/tests/test_pandas_conversion.py:144: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > pyflink/table/table_environment.py:1462: in from_pandas > arrow_schema = pa.Schema.from_pandas(pdf, preserve_index=False) > pyarrow/types.pxi:1315: in pyarrow.lib.Schema.from_pandas > ??? > .tox/py37-cython/lib/python3.7/site-packages/pyarrow/pandas_compat.py:519: in > dataframe_to_types > type_ = pa.lib._ndarray_to_arrow_type(values, type_) > pyarrow/array.pxi:53: in pyarrow.lib._ndarray_to_arrow_type > ??? > pyarrow/array.pxi:64: in pyarrow.lib._ndarray_to_type > ??? > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > > > ??? > E pyarrow.lib.ArrowTypeError: Did not pass numpy.dtype object > > pyarrow/error.pxi:108: ArrowTypeError > _ StreamPandasConversionTests.test_from_pandas > _ > > self = > testMethod=test_from_pandas> > > def test_from_pandas(self): > > table = self.t_env.from_pandas(self.pdf, self.data_type, 5) > > pyflink/table/tests/test_pandas_conversion.py:120: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-21216) StreamPandasConversionTests Fails
[ https://issues.apache.org/jira/browse/FLINK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275765#comment-17275765 ] Guowei Ma commented on FLINK-21216: --- https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12701&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3 > StreamPandasConversionTests Fails > - > > Key: FLINK-21216 > URL: https://issues.apache.org/jira/browse/FLINK-21216 > Project: Flink > Issue Type: Bug > Components: API / Python >Affects Versions: 1.12.0, 1.13.0 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12699&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=8d78fe4f-d658-5c70-12f8-4921589024c3] > > === FAILURES > === > ___ StreamPandasConversionTests.test_empty_to_pandas > ___ > > self = > testMethod=test_empty_to_pandas> > > def test_empty_to_pandas(self): > > table = self.t_env.from_pandas(self.pdf, self.data_type) > > pyflink/table/tests/test_pandas_conversion.py:144: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > pyflink/table/table_environment.py:1462: in from_pandas > arrow_schema = pa.Schema.from_pandas(pdf, preserve_index=False) > pyarrow/types.pxi:1315: in pyarrow.lib.Schema.from_pandas > ??? > .tox/py37-cython/lib/python3.7/site-packages/pyarrow/pandas_compat.py:519: in > dataframe_to_types > type_ = pa.lib._ndarray_to_arrow_type(values, type_) > pyarrow/array.pxi:53: in pyarrow.lib._ndarray_to_arrow_type > ??? > pyarrow/array.pxi:64: in pyarrow.lib._ndarray_to_type > ??? > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ > > > ??? > E pyarrow.lib.ArrowTypeError: Did not pass numpy.dtype object > > pyarrow/error.pxi:108: ArrowTypeError > _ StreamPandasConversionTests.test_from_pandas > _ > > self = > testMethod=test_from_pandas> > > def test_from_pandas(self): > > table = self.t_env.from_pandas(self.pdf, self.data_type, 5) > > pyflink/table/tests/test_pandas_conversion.py:120: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] dijkwxyz opened a new pull request #14819: [FLINK-16152] Translate "Operator/index" into Chinese
dijkwxyz opened a new pull request #14819: URL: https://github.com/apache/flink/pull/14819 Finished previously incomplete translation for index.zh.md This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-16152) Translate "Operator/index" into Chinese
[ https://issues.apache.org/jira/browse/FLINK-16152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-16152: --- Labels: pull-request-available (was: ) > Translate "Operator/index" into Chinese > --- > > Key: FLINK-16152 > URL: https://issues.apache.org/jira/browse/FLINK-16152 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation >Reporter: Yun Gao >Assignee: Yun Gao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > The page is located at _docs/dev/stream/operators/index.zh.md_ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #14819: [FLINK-16152] Translate "Operator/index" into Chinese
flinkbot commented on pull request #14819: URL: https://github.com/apache/flink/pull/14819#issuecomment-770317299 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 3b55294a34fea9ba31744e8f7fb7456c3faf46ef (Sun Jan 31 03:03:59 UTC 2021) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20420) ES6 ElasticsearchSinkITCase failed due to no output for 900 seconds
[
https://issues.apache.org/jira/browse/FLINK-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275766#comment-17275766
]
Guowei Ma commented on FLINK-20420:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12701&view=logs&j=ba53eb01-1462-56a3-8e98-0dd97fbcaab5&t=bfbc6239-57a0-5db0-63f3-41551b4f7d51
> ES6 ElasticsearchSinkITCase failed due to no output for 900 seconds
> ---
>
> Key: FLINK-20420
> URL: https://issues.apache.org/jira/browse/FLINK-20420
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ElasticSearch
>Affects Versions: 1.12.0
>Reporter: Yun Tang
>Priority: Major
> Labels: test-stability
>
> Instance:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10249&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=03dca39c-73e8-5aaf-601d-328ae5c35f20&l=18821
> {code:java}
> Process produced no output for 900 seconds.
> ==
> ==
> The following Java processes are running (JPS)
> ==
> Picked up JAVA_TOOL_OPTIONS: -XX:+HeapDumpOnOutOfMemoryError
> 2274 Launcher
> 18260 Jps
> 15916 surefirebooter3434370240444055571.jar
> ==
> "main" #1 prio=5 os_prio=0 tid=0x7feec000b800 nid=0x3e2d runnable
> [0x7feec8541000]
>java.lang.Thread.State: RUNNABLE
> at org.testcontainers.shaded.okio.Buffer.indexOf(Buffer.java:1463)
> at
> org.testcontainers.shaded.okio.RealBufferedSource.indexOf(RealBufferedSource.java:352)
> at
> org.testcontainers.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)
> at
> org.testcontainers.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:224)
> at
> org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.readChunkSize(Http1ExchangeCodec.java:489)
> at
> org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.read(Http1ExchangeCodec.java:471)
> at
> org.testcontainers.shaded.okhttp3.internal.Util.skipAll(Util.java:204)
> at
> org.testcontainers.shaded.okhttp3.internal.Util.discard(Util.java:186)
> at
> org.testcontainers.shaded.okhttp3.internal.http1.Http1ExchangeCodec$ChunkedSource.close(Http1ExchangeCodec.java:511)
> at
> org.testcontainers.shaded.okio.ForwardingSource.close(ForwardingSource.java:43)
> at
> org.testcontainers.shaded.okhttp3.internal.connection.Exchange$ResponseBodySource.close(Exchange.java:313)
> at
> org.testcontainers.shaded.okio.RealBufferedSource.close(RealBufferedSource.java:476)
> at
> org.testcontainers.shaded.okhttp3.internal.Util.closeQuietly(Util.java:139)
> at
> org.testcontainers.shaded.okhttp3.ResponseBody.close(ResponseBody.java:192)
> at org.testcontainers.shaded.okhttp3.Response.close(Response.java:290)
> at
> org.testcontainers.shaded.com.github.dockerjava.okhttp.OkDockerHttpClient$OkResponse.close(OkDockerHttpClient.java:280)
> at
> org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$null$0(DefaultInvocationBuilder.java:272)
> at
> org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$$Lambda$87/1112018050.close(Unknown
> Source)
> at
> com.github.dockerjava.api.async.ResultCallbackTemplate.close(ResultCallbackTemplate.java:77)
> at
> org.testcontainers.utility.ResourceReaper.start(ResourceReaper.java:177)
> at
> org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:203)
> - locked <0x88fcbbf0> (a [Ljava.lang.Object;)
> at
> org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
> at
> org.testcontainers.LazyDockerClient.listImagesCmd(LazyDockerClient.java:12)
> at
> org.testcontainers.images.LocalImagesCache.maybeInitCache(LocalImagesCache.java:68)
> - locked <0x88fcb940> (a
> org.testcontainers.images.LocalImagesCache)
> at
> org.testcontainers.images.LocalImagesCache.get(LocalImagesCache.java:32)
> at
> org.testcontainers.images.AbstractImagePullPolicy.shouldPull(AbstractImagePullPolicy.java:18)
> at
> org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:66)
> at
> org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:27)
> at
> org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:17)
> - locked <0x890763d0> (a
> java.util.concurrent
[GitHub] [flink] flinkbot commented on pull request #14819: [FLINK-16152] Translate "Operator/index" into Chinese
flinkbot commented on pull request #14819: URL: https://github.com/apache/flink/pull/14819#issuecomment-770318086 ## CI report: * 3b55294a34fea9ba31744e8f7fb7456c3faf46ef UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14819: [FLINK-16152] Translate "Operator/index" into Chinese
flinkbot edited a comment on pull request #14819: URL: https://github.com/apache/flink/pull/14819#issuecomment-770318086 ## CI report: * 3b55294a34fea9ba31744e8f7fb7456c3faf46ef Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12703) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gaoyunhaii opened a new pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
gaoyunhaii opened a new pull request #14820: URL: https://github.com/apache/flink/pull/14820 ## What is the purpose of the change This PR enables triggering checkpoint barrier handle for non-source tasks. To support checkpoints after some tasks finished, non-source tasks might also receive RPC trigger of checkpoint after all their precedent tasks get finished. In this case, it need to trigger checkpoint barrier handler to insert barriers. This PR provides the functionality to notify checkpoint barrier on RPC trigger ## Brief change log *(for example:)* - a3daa2b88a7c1609daf2bb55742971eabe5dacf0 refactors the current implementation to expose CheckpointBarrierHandler. - b4baae6e24556aec841fb1c4fcedfec234e3c0cd notifies the non-source stream tasks' checkpoint barrier handler on RPC trigger. - 326d8751ac2aac4255b5e94da8e8f46a424c55c5 fixes the tests that using non-source task that directly perform checkpoint on trigger. ## Verifying this change Added UT to verify that checkpoint barrier handler get triggered on RPC checkpoint trigger. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-21085) Refactor StreamTask hierarchy to support triggering checkpoint via RPC for non-source tasks
[ https://issues.apache.org/jira/browse/FLINK-21085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-21085: --- Labels: pull-request-available (was: ) > Refactor StreamTask hierarchy to support triggering checkpoint via RPC for > non-source tasks > > > Key: FLINK-21085 > URL: https://issues.apache.org/jira/browse/FLINK-21085 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream, Runtime / Checkpointing >Reporter: Yun Gao >Priority: Major > Labels: pull-request-available > > For checkpoint considering finished tasks, non-source tasks might get > triggered for checkpoint via RPC if they become the new "root" tasks. In this > case, they would notify their CheckpointBarrierHandle for this checkpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
HuangZhenQiu commented on a change in pull request #14678: URL: https://github.com/apache/flink/pull/14678#discussion_r567364234 ## File path: docs/deployment/advanced/platform.md ## @@ -0,0 +1,49 @@ +--- +title: "Customizable Features for Platform Users" +nav-title: platform +nav-parent_id: advanced +nav-pos: 3 +--- + +Flink provides a set of customizable features for users to extend from the default behavior through the plugin framework. + +## Customize Failure Listener +For each of execution exceptions in a flink job, it will be passed to the job master. The default failure listener is only +to record the failure count and emit the metrics numJobFailure for the job. If you need an advanced classification on exceptions, +you can build a plugin to customize failure listener. For example, it can distinguish whether it is a flink runtime error or an +application user logic error. With the accurate metrics, you may have better idea about the platform level metrics, for example +failures due to network, platform reliability, etc. + + +# Implement a plugin for your custom failure listener Review comment: Agree. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
HuangZhenQiu commented on a change in pull request #14678: URL: https://github.com/apache/flink/pull/14678#discussion_r567364321 ## File path: docs/deployment/advanced/platform.md ## @@ -0,0 +1,49 @@ +--- +title: "Customizable Features for Platform Users" +nav-title: platform +nav-parent_id: advanced +nav-pos: 3 +--- +
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
HuangZhenQiu commented on a change in pull request #14678:
URL: https://github.com/apache/flink/pull/14678#discussion_r567364347
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/failurelistener/FailureListenerUtils.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.failurelistener;
+
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.core.failurelistener.FailureListener;
+import org.apache.flink.core.failurelistener.FailureListenerFactory;
+import org.apache.flink.core.plugin.PluginManager;
+import org.apache.flink.core.plugin.PluginUtils;
+import org.apache.flink.runtime.metrics.groups.JobManagerJobMetricGroup;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+
+/** Utils for creating failure listener. */
+public class FailureListenerUtils {
+
+public static List createFailureListener(
Review comment:
Updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] HuangZhenQiu commented on a change in pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
HuangZhenQiu commented on a change in pull request #14678:
URL: https://github.com/apache/flink/pull/14678#discussion_r567364389
##
File path:
flink-core/src/main/java/org/apache/flink/core/failurelistener/FailureListener.java
##
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.core.failurelistener;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.metrics.MetricGroup;
+
+/** Failure listener to customize the behavior for each type of failures
tracked in job manager. */
+@PublicEvolving
+public interface FailureListener {
+
+/**
+ * Initialize the FailureListener with MetricGroup.
+ *
+ * @param jobName the name job whose failure will be subscribed by the
listener
+ * @param metricGroup metrics group that the listener can add customized
metrics definition.
+ */
+void init(String jobName, MetricGroup metricGroup);
Review comment:
I just feel jobID is not needed for most of the cases initially. Added
accordingly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] HuangZhenQiu commented on pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
HuangZhenQiu commented on pull request #14678: URL: https://github.com/apache/flink/pull/14678#issuecomment-770324596 @rmetzger Thanks for these suggestions. Updated accordingly. Please review it again at your most convenient time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
flinkbot commented on pull request #14820: URL: https://github.com/apache/flink/pull/14820#issuecomment-770325573 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 326d8751ac2aac4255b5e94da8e8f46a424c55c5 (Sun Jan 31 04:51:56 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-21085).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21139) ThresholdMeterTest.testMarkMultipleEvents unstable
[
https://issues.apache.org/jira/browse/FLINK-21139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275780#comment-17275780
]
Zhenqiu Huang commented on FLINK-21139:
---
[~trohrmann] [~xintongsong]
The ManualClock is defined in flink-core. Currently, ThresholdMeter is defined
in flink-metrics. I feel it is not wise to make flink-metrics dependent on
flink-core module. Any suggestions about this?
> ThresholdMeterTest.testMarkMultipleEvents unstable
> --
>
> Key: FLINK-21139
> URL: https://issues.apache.org/jira/browse/FLINK-21139
> Project: Flink
> Issue Type: Bug
> Components: API / Core, Benchmarks, Build System, Tests
> Environment: $ java -version && javac -version
> openjdk version "11.0.8" 2020-07-14
> OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10-202007292333)
> Eclipse OpenJ9 VM AdoptOpenJDK (build master-dc7cbe4bd, JRE 11 Linux
> riscv-64-Bit Compressed References 20200729_78 (JIT disabled, AOT disabled)
> OpenJ9 - dc7cbe4bd
> OMR - 1c0299f20
> JCL - 94b9d6d2c6 based on jdk-11.0.8+10)
> javac 11.0.8
> $ uname -a
> Linux test-gdams-debian10-riscv64-1.adoptopenjdk.net
> 5.0.0-rc1-56210-g0a657e0d72f0 #1 SMP Fri May 15 18:05:26 EDT 2020 riscv64
> GNU/Linux
> $ cat /proc/cpuinfo
> processor : 0
> hart : 1
> isa : rv64imafdc
> mmu : sv39
> uarch : sifive,rocket0
>Reporter: Ivan Serdyuk
>Assignee: Zhenqiu Huang
>Priority: Critical
> Labels: test-stability
> Fix For: 1.13.0
>
>
> The test {{ThresholdMeterTest.testMarkMultipleEvents}} seems to be unstable.
> The version of Flink is
> ??$ git log -1??
> ??commit 6e77cfdff8a358adab4ab770a503197d95a64440 (HEAD -> master,
> origin/master, origin/HEAD)??
> ??Author: Roman Khachatryan ??
> ??Date: Fri Jan 22 14:39:48 2021 +0100??
> The test fails with the following exception:
> {code}
> Expected: a numeric value within <1.0E-6> of <40.0>
> but: <0.0> differed by <39.99>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> at org.junit.Assert.assertThat(Assert.java:956)
> at org.junit.Assert.assertThat(Assert.java:923)
> at
> org.apache.flink.metrics.ThresholdMeterTest.testMarkMultipleEvents(ThresholdMeterTest.java:58)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUni
[GitHub] [flink] flinkbot edited a comment on pull request #13912: [FLINK-19466][FLINK-19467][runtime / state backends] Add Flip-142 public interfaces and methods
flinkbot edited a comment on pull request #13912: URL: https://github.com/apache/flink/pull/13912#issuecomment-721398037 ## CI report: * e7dbab13079f1a1f2350320930cc216c8ed42759 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12702) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
flinkbot edited a comment on pull request #14678: URL: https://github.com/apache/flink/pull/14678#issuecomment-761905721 ## CI report: * cade20e85b29ca63c51383dca04976c1d9801042 UNKNOWN * 1a6bd7aa98b649bd119479247d34ea029b8ba636 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12400) * 11651466eb13c5c05d2cd0eed0ac08b9bf617185 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
flinkbot commented on pull request #14820: URL: https://github.com/apache/flink/pull/14820#issuecomment-770327711 ## CI report: * 326d8751ac2aac4255b5e94da8e8f46a424c55c5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14678: [FLINK-20833][runtime] Add pluggable failure listener in job manager
flinkbot edited a comment on pull request #14678: URL: https://github.com/apache/flink/pull/14678#issuecomment-761905721 ## CI report: * cade20e85b29ca63c51383dca04976c1d9801042 UNKNOWN * 11651466eb13c5c05d2cd0eed0ac08b9bf617185 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12704) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
flinkbot edited a comment on pull request #14820: URL: https://github.com/apache/flink/pull/14820#issuecomment-770327711 ## CI report: * 326d8751ac2aac4255b5e94da8e8f46a424c55c5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12705) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
flinkbot edited a comment on pull request #14820: URL: https://github.com/apache/flink/pull/14820#issuecomment-770327711 ## CI report: * 326d8751ac2aac4255b5e94da8e8f46a424c55c5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12705) * 5043cb6d95f503534b53bac93c451ae367e9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14820: [FLINK-21085][runtime][checkpoint] Allows triggering checkpoint barrier handler for non-source tasks
flinkbot edited a comment on pull request #14820: URL: https://github.com/apache/flink/pull/14820#issuecomment-770327711 ## CI report: * 5043cb6d95f503534b53bac93c451ae367e9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=12706) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21139) ThresholdMeterTest.testMarkMultipleEvents unstable
[
https://issues.apache.org/jira/browse/FLINK-21139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275790#comment-17275790
]
Xintong Song commented on FLINK-21139:
--
[~ZhenqiuHuang],
Maybe we can move {{ThreasholdMeter}} to {{org.apache.flink.runtime.metrics}}.
There's already {{TimerGauge}}, which also uses {{Clock}}.
> ThresholdMeterTest.testMarkMultipleEvents unstable
> --
>
> Key: FLINK-21139
> URL: https://issues.apache.org/jira/browse/FLINK-21139
> Project: Flink
> Issue Type: Bug
> Components: API / Core, Benchmarks, Build System, Tests
> Environment: $ java -version && javac -version
> openjdk version "11.0.8" 2020-07-14
> OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10-202007292333)
> Eclipse OpenJ9 VM AdoptOpenJDK (build master-dc7cbe4bd, JRE 11 Linux
> riscv-64-Bit Compressed References 20200729_78 (JIT disabled, AOT disabled)
> OpenJ9 - dc7cbe4bd
> OMR - 1c0299f20
> JCL - 94b9d6d2c6 based on jdk-11.0.8+10)
> javac 11.0.8
> $ uname -a
> Linux test-gdams-debian10-riscv64-1.adoptopenjdk.net
> 5.0.0-rc1-56210-g0a657e0d72f0 #1 SMP Fri May 15 18:05:26 EDT 2020 riscv64
> GNU/Linux
> $ cat /proc/cpuinfo
> processor : 0
> hart : 1
> isa : rv64imafdc
> mmu : sv39
> uarch : sifive,rocket0
>Reporter: Ivan Serdyuk
>Assignee: Zhenqiu Huang
>Priority: Critical
> Labels: test-stability
> Fix For: 1.13.0
>
>
> The test {{ThresholdMeterTest.testMarkMultipleEvents}} seems to be unstable.
> The version of Flink is
> ??$ git log -1??
> ??commit 6e77cfdff8a358adab4ab770a503197d95a64440 (HEAD -> master,
> origin/master, origin/HEAD)??
> ??Author: Roman Khachatryan ??
> ??Date: Fri Jan 22 14:39:48 2021 +0100??
> The test fails with the following exception:
> {code}
> Expected: a numeric value within <1.0E-6> of <40.0>
> but: <0.0> differed by <39.99>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> at org.junit.Assert.assertThat(Assert.java:956)
> at org.junit.Assert.assertThat(Assert.java:923)
> at
> org.apache.flink.metrics.ThresholdMeterTest.testMarkMultipleEvents(ThresholdMeterTest.java:58)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87)
> at
> org.apache.maven.surefire.junitcore.JUnit
[jira] [Created] (FLINK-21218) "state.checkpoints.dir" should be required only when "execution.checkpointing.interval" is specified
Dongwon Kim created FLINK-21218: --- Summary: "state.checkpoints.dir" should be required only when "execution.checkpointing.interval" is specified Key: FLINK-21218 URL: https://issues.apache.org/jira/browse/FLINK-21218 Project: Flink Issue Type: Improvement Components: Runtime / State Backends Reporter: Dongwon Kim Users have to specify "state.checkpoints.dir" even when to use a state backend as an unreliable per-key state storage. Thread in user ML : http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Setting-quot-unreliable-quot-RocksDB-state-backend-w-o-quot-execution-checkpointing-interval-quot-ans-td41003.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
