[GitHub] [flink] flinkbot edited a comment on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10764#issuecomment-570760745 ## CI report: * 5a909d7767744f8c9030965b58fc0fbaa1c0cffb Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143081445) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4093) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10764#issuecomment-570760745 ## CI report: * 5a909d7767744f8c9030965b58fc0fbaa1c0cffb Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143081445) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4093) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15472) Support SQL Client Gateway
[ https://issues.apache.org/jira/browse/FLINK-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] godfrey he updated FLINK-15472: --- Description: FLIP-91: https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway (was: FLIP-91: https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway design document: https://docs.google.com/document/d/1T7--664rts4t_4gjRPw937S9ln9Plf1yghNQ9IiHQtQ) > Support SQL Client Gateway > --- > > Key: FLINK-15472 > URL: https://issues.apache.org/jira/browse/FLINK-15472 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Client >Reporter: godfrey he >Priority: Major > > FLIP-91: > https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Client+Gateway -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy
flinkbot commented on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10764#issuecomment-570760745 ## CI report: * 5a909d7767744f8c9030965b58fc0fbaa1c0cffb UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy
flinkbot commented on issue #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10764#issuecomment-570759004 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5a909d7767744f8c9030965b58fc0fbaa1c0cffb (Sat Jan 04 05:42:46 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tweise opened a new pull request #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy
tweise opened a new pull request #10764: [FLINK-13027][fs-connector] SFS bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10764 backport of https://github.com/apache/flink/pull/10653 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tweise merged pull request #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
tweise merged pull request #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese
flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese URL: https://github.com/apache/flink/pull/10738#issuecomment-570073257 ## CI report: * 8b4f68255a0a91f32757394d86405e46f60118c5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142809001) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4023) * 827c1dc7e424d9e89d5573fe5a466682271803f8 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143077633) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4092) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups
flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups URL: https://github.com/apache/flink/pull/10716#issuecomment-569571340 ## CI report: * 68e5c62f5520559749307a6e2a2667434d5754cf Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609810) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3971) * 4900d684fedd60206559131a3d9eb2dde2c86125 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142978212) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4070) * eb8c0fc6d97f6e928e2a1c58bfb2c978096665e7 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143077006) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4091) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-568135707 ## CI report: * b36738feabf0ce5dc1feeed6cf7940d61e771a0e Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/141991090) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3835) * 56dda5aff74724f99f981a6359cfbcd974ccd207 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142001922) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3841) * 4da311951dfdf9248d98b523160eaee7136d3214 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142176372) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3866) * f82f5ebd560fd0528ed64a669030d4891ba2c0a5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143076282) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4088) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese
flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese URL: https://github.com/apache/flink/pull/10738#issuecomment-570073257 ## CI report: * 8b4f68255a0a91f32757394d86405e46f60118c5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142809001) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4023) * 827c1dc7e424d9e89d5573fe5a466682271803f8 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143077633) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4092) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default
flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default URL: https://github.com/apache/flink/pull/10717#issuecomment-569571352 ## CI report: * 3e35131da349b322ab298ec38f55bfdc21ba0e17 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609821) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3972) * 3abdbc17a2b4f5463396f8071e3076ff5b889274 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142978236) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4071) * 995435e15457f82ce7cdb0db9bdb63a8330c2d49 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142988515) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4080) * a3a115458459c1a04b4aab11d365f2bf01abcf25 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143076291) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4089) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143024027) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4084) * da8137a5c135ee86f12bd4cbd33f6aea77c06a14 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143077002) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4090) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese
flinkbot edited a comment on issue #10738: [FLINK-15385][table][docs] Translate SQL section of Table API into Chinese URL: https://github.com/apache/flink/pull/10738#issuecomment-570073257 ## CI report: * 8b4f68255a0a91f32757394d86405e46f60118c5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142809001) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4023) * 827c1dc7e424d9e89d5573fe5a466682271803f8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups
flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups URL: https://github.com/apache/flink/pull/10716#issuecomment-569571340 ## CI report: * 68e5c62f5520559749307a6e2a2667434d5754cf Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609810) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3971) * 4900d684fedd60206559131a3d9eb2dde2c86125 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142978212) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4070) * eb8c0fc6d97f6e928e2a1c58bfb2c978096665e7 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143077006) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4091) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-568135707 ## CI report: * b36738feabf0ce5dc1feeed6cf7940d61e771a0e Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/141991090) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3835) * 56dda5aff74724f99f981a6359cfbcd974ccd207 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142001922) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3841) * 4da311951dfdf9248d98b523160eaee7136d3214 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142176372) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3866) * f82f5ebd560fd0528ed64a669030d4891ba2c0a5 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143076282) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4088) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143024027) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4084) * da8137a5c135ee86f12bd4cbd33f6aea77c06a14 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143077002) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4090) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default
flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default URL: https://github.com/apache/flink/pull/10717#issuecomment-569571352 ## CI report: * 3e35131da349b322ab298ec38f55bfdc21ba0e17 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609821) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3972) * 3abdbc17a2b4f5463396f8071e3076ff5b889274 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142978236) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4071) * 995435e15457f82ce7cdb0db9bdb63a8330c2d49 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142988515) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4080) * a3a115458459c1a04b4aab11d365f2bf01abcf25 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143076291) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4089) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups
flinkbot edited a comment on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups URL: https://github.com/apache/flink/pull/10716#issuecomment-569571340 ## CI report: * 68e5c62f5520559749307a6e2a2667434d5754cf Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609810) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3971) * 4900d684fedd60206559131a3d9eb2dde2c86125 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142978212) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4070) * eb8c0fc6d97f6e928e2a1c58bfb2c978096665e7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-11964) Avoid hash collision of partition and bucket in HybridHashTable in Blink SQL
[ https://issues.apache.org/jira/browse/FLINK-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kurt Young reassigned FLINK-11964: -- Assignee: Jingsong Lee > Avoid hash collision of partition and bucket in HybridHashTable in Blink SQL > > > Key: FLINK-11964 > URL: https://issues.apache.org/jira/browse/FLINK-11964 > Project: Flink > Issue Type: Bug > Components: Table SQL / Runtime >Reporter: Jingsong Lee >Assignee: Jingsong Lee >Priority: Major > Fix For: 1.10.0 > > > In HybridHashTable, first select the corresponding partition according to > hashCode, and then select the bucket in the partition according to hashCode, > using the same hashCode can easily cause hash collision. > Consider doing some mix to hashCode when choosing bucket. > Like JDK HashMap, we can just XOR some shifted bits in the cheapest possible > way to reduce systematic lossage, as well as to incorporate impact of the > highest bits that would otherwise never be used in index calculations because > of table bounds. (bucket use power-of-two masking). Just like: (hash ^ > (hash >>> 16)) > In some cases, if a lot of conflicts occurred, this will lead to job hang, > because hash join will degenerate to nested loop join. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] xintongsong edited a comment on issue #10608: [FLINK-15300][Runtime] Fix sanity check to not fail if shuffle memory fraction is out of min/max range
xintongsong edited a comment on issue #10608: [FLINK-15300][Runtime] Fix sanity check to not fail if shuffle memory fraction is out of min/max range URL: https://github.com/apache/flink/pull/10608#issuecomment-570752333 @tillrohrmann I would consider `RangeFraction` as a data structure that serves `TaskExecutorResourceUtils` only rather than for general purpose. After all, it is a private internal class of `TaskExecutorResourceUtils` and it already assumes the range is represented in `MemorySize`. `fraction == 1` might be valid for a general purpose range-fraction. But for `TaskExecutorResourceUtils`, none of the task executor resource related fractions (managed, shuffle, jvm-overhead) should ever be 1. Managed memory or shuffle memory should never takes all of the total Flink memory, and JVM overhead should never takes all of the total process memory. Thus, IMO, it makes sense for `RangeFraction` to not accept `fraction == 1`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10608: [FLINK-15300][Runtime] Fix sanity check to not fail if shuffle memory fraction is out of min/max range
xintongsong commented on issue #10608: [FLINK-15300][Runtime] Fix sanity check to not fail if shuffle memory fraction is out of min/max range URL: https://github.com/apache/flink/pull/10608#issuecomment-570752333 @tillrohrmann I would consider `RangeFraction` as a data structure that serves `TaskExecutorResourceUtils` only rather than for general purpose. After all, it is a private internal class of `TaskExecutorResourceUtils` and it already assumes the range is represented in `MemorySize`. `fraction == 1` might be a valid for a general purpose range-fraction. But for `TaskExecutorResourceUtils`, none of the task executor resource related fractions (managed, shuffle, jvm-overhead) should ever be 1. Managed memory or shuffle memory should never takes all of the total Flink memory, and JVM overhead should never takes all of the total process memory. Thus, IMO, it makes sense for `RangeFraction` to not accept `fraction == 1`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-568135707 ## CI report: * b36738feabf0ce5dc1feeed6cf7940d61e771a0e Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/141991090) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3835) * 56dda5aff74724f99f981a6359cfbcd974ccd207 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142001922) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3841) * 4da311951dfdf9248d98b523160eaee7136d3214 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142176372) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3866) * f82f5ebd560fd0528ed64a669030d4891ba2c0a5 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143076282) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4088) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143024027) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4084) * da8137a5c135ee86f12bd4cbd33f6aea77c06a14 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups
xintongsong commented on issue #10716: [FLINK-15367][runtime] Handle backwards compatibility of "taskmanager.heap.size" differently for standalone / active setups URL: https://github.com/apache/flink/pull/10716#issuecomment-570750908 @tillrohrmann Thank you for the review. I've addressed your comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default
flinkbot edited a comment on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default URL: https://github.com/apache/flink/pull/10717#issuecomment-569571352 ## CI report: * 3e35131da349b322ab298ec38f55bfdc21ba0e17 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142609821) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3972) * 3abdbc17a2b4f5463396f8071e3076ff5b889274 Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142978236) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4071) * 995435e15457f82ce7cdb0db9bdb63a8330c2d49 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142988515) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4080) * a3a115458459c1a04b4aab11d365f2bf01abcf25 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
flinkbot edited a comment on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-568135707 ## CI report: * b36738feabf0ce5dc1feeed6cf7940d61e771a0e Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/141991090) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3835) * 56dda5aff74724f99f981a6359cfbcd974ccd207 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142001922) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3841) * 4da311951dfdf9248d98b523160eaee7136d3214 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142176372) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3866) * f82f5ebd560fd0528ed64a669030d4891ba2c0a5 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default
xintongsong commented on issue #10717: [FLINK-15369][runtime] MiniCluster use fixed network / managed memory sizes by default URL: https://github.com/apache/flink/pull/10717#issuecomment-570747981 @tillrohrmann, thanks for the review. I have addressed the comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yxu-valleytider commented on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
yxu-valleytider commented on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-570744150 Created [FLINK-15476](https://issues.apache.org/jira/browse/FLINK-15476) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15476) Update StreamingFileSink documentation -- bulk encoded writer now supports customized checkpoint policy
[
https://issues.apache.org/jira/browse/FLINK-15476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ying Xu updated FLINK-15476:
Component/s: Documentation
Description:
Per FLINK-13027, {{StreamingFileSink}}'s bulk encoded writer (created with
{{forBulkFormat}}) now supports customized checkpoint policies which roll file
at the checkpoint epoch.
The {{StreamingFileSink}} documentation needs to be updated accordingly.
Issue Type: Task (was: Improvement)
> Update StreamingFileSink documentation -- bulk encoded writer now supports
> customized checkpoint policy
> ---
>
> Key: FLINK-15476
> URL: https://issues.apache.org/jira/browse/FLINK-15476
> Project: Flink
> Issue Type: Task
> Components: Documentation
>Reporter: Ying Xu
>Priority: Major
>
> Per FLINK-13027, {{StreamingFileSink}}'s bulk encoded writer (created with
> {{forBulkFormat}}) now supports customized checkpoint policies which roll
> file at the checkpoint epoch.
> The {{StreamingFileSink}} documentation needs to be updated accordingly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15476) Update StreamingFileSink documentation -- bulk encoded writer now supports customized checkpoint policy
Ying Xu created FLINK-15476: --- Summary: Update StreamingFileSink documentation -- bulk encoded writer now supports customized checkpoint policy Key: FLINK-15476 URL: https://issues.apache.org/jira/browse/FLINK-15476 Project: Flink Issue Type: Improvement Reporter: Ying Xu -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] yxu-valleytider commented on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy
yxu-valleytider commented on issue #10653: [FLINK-13027][streaming]: StreamingFileSink bulk-encoded writer supports customized checkpoint policy URL: https://github.com/apache/flink/pull/10653#issuecomment-570741727 @tweise Thanks for the review. I can follow up with a separate PR addressing the documentation issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) * d61cca153ada8faf1ca0c59e9cb8ec8bd55ed39e Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143046561) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4087) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15473) Add support for Linux on ppc64le to MemoryArchitecture
[
https://issues.apache.org/jira/browse/FLINK-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ronald O. Edmark updated FLINK-15473:
-
Priority: Major (was: Minor)
> Add support for Linux on ppc64le to MemoryArchitecture
> --
>
> Key: FLINK-15473
> URL: https://issues.apache.org/jira/browse/FLINK-15473
> Project: Flink
> Issue Type: Improvement
> Environment: Red Hat, Ubuntu Linux on PowerPC.
>Reporter: Ronald O. Edmark
>Priority: Major
>
> Similar to issue FLINK-13449
> Please add "ppc64le" to MemoryArchitecture.java
> final List names64bit = Arrays.asList("amd64", "x86_64", "aarch64",
> "ppc64le");
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) * d61cca153ada8faf1ca0c59e9cb8ec8bd55ed39e Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143046561) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4087) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) * d61cca153ada8faf1ca0c59e9cb8ec8bd55ed39e Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143046561) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4087) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) * d61cca153ada8faf1ca0c59e9cb8ec8bd55ed39e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions
flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions URL: https://github.com/apache/flink/pull/10692#issuecomment-568924184 ## CI report: * cf3539fdec7d4ff9ad218545f3544b6cea3a2e24 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142331359) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3917) * c81d7ac0c294a6e05056ea11c15d645541f87f86 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142967001) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4066) * 032c103cc3737323789f835c3b78244dba2e3561 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143027591) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4086) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions
flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions URL: https://github.com/apache/flink/pull/10692#issuecomment-568924184 ## CI report: * cf3539fdec7d4ff9ad218545f3544b6cea3a2e24 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142331359) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3917) * c81d7ac0c294a6e05056ea11c15d645541f87f86 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142967001) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4066) * 032c103cc3737323789f835c3b78244dba2e3561 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143027591) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4086) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-564535655 ## CI report: * 252fe96f7a6254a18341f78cdb1bcbd24ae25a1c Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/140600093) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3473) * c44ddd079a072c171a1a10a075fd6f57610e2a4f Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/140628027) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3480) * f06f203456373d2685a1dc87996059ec6bde4993 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/141350358) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3657) * 962197a8d870bafcc7265cdd902657eddeddebd3 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142088890) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3854) * ba9e443d3a694bd741d69175ec4fe1e48cba1956 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142105543) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3862) * 786862ac623203275fd1b6fbf3e255abd233285f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142373804) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3935) * 9ffa9d4980fb49c40b5b1b8d2a00ef7aa43751de Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142981538) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4073) * 8201386c978807c5c1dea003e03cdfef36539ca4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142985614) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4074) * 6057b2d957ec44b73e06941ebf42388636e57a00 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143024073) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4085) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions
flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions URL: https://github.com/apache/flink/pull/10692#issuecomment-568924184 ## CI report: * cf3539fdec7d4ff9ad218545f3544b6cea3a2e24 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142331359) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3917) * c81d7ac0c294a6e05056ea11c15d645541f87f86 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142967001) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4066) * 032c103cc3737323789f835c3b78244dba2e3561 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143027591) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4086) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143024027) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4084) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions
flinkbot edited a comment on issue #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions URL: https://github.com/apache/flink/pull/10692#issuecomment-568924184 ## CI report: * cf3539fdec7d4ff9ad218545f3544b6cea3a2e24 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142331359) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3917) * c81d7ac0c294a6e05056ea11c15d645541f87f86 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/142967001) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4066) * 032c103cc3737323789f835c3b78244dba2e3561 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-564535655 ## CI report: * 252fe96f7a6254a18341f78cdb1bcbd24ae25a1c Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/140600093) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3473) * c44ddd079a072c171a1a10a075fd6f57610e2a4f Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/140628027) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3480) * f06f203456373d2685a1dc87996059ec6bde4993 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/141350358) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3657) * 962197a8d870bafcc7265cdd902657eddeddebd3 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142088890) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3854) * ba9e443d3a694bd741d69175ec4fe1e48cba1956 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142105543) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3862) * 786862ac623203275fd1b6fbf3e255abd233285f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142373804) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3935) * 9ffa9d4980fb49c40b5b1b8d2a00ef7aa43751de Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142981538) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4073) * 8201386c978807c5c1dea003e03cdfef36539ca4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142985614) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4074) * 6057b2d957ec44b73e06941ebf42388636e57a00 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143024073) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4085) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot edited a comment on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/143024027) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4084) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15475) Add isOutputTypeUsed() API to Transformation
[
https://issues.apache.org/jira/browse/FLINK-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rong Rong updated FLINK-15475:
--
Description:
Currently there's no way to "peek" into a `Transformation` object and see if
`typeUsed` has been set or not. The only way is to invoke the `setOutputType`
API and wrap around it with a try-catch block similar to:
{code:java}
try {
(SingleOutputStreamOperator)dataStream
.returns(myOutputType);
} catch (ValidationException ex) {
// ... handle exception when type has been used.
}
{code}
It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
check whether a particular transformation has a definitive output type set /
used or not.
was:
Currently there's no way to "peek" into a `Transformation` object and see if
`typeUsed` has been set or not. The only way is to invoke the `setOutputType`
API and wrap around it with a try-catch block. See:
It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
check whether a particular transformation has a definitive output type set /
used or not.
> Add isOutputTypeUsed() API to Transformation
>
>
> Key: FLINK-15475
> URL: https://issues.apache.org/jira/browse/FLINK-15475
> Project: Flink
> Issue Type: Improvement
> Components: API / Core, API / DataSet, API / DataStream
>Reporter: Rong Rong
>Assignee: Rong Rong
>Priority: Minor
>
> Currently there's no way to "peek" into a `Transformation` object and see if
> `typeUsed` has been set or not. The only way is to invoke the `setOutputType`
> API and wrap around it with a try-catch block similar to:
> {code:java}
> try {
> (SingleOutputStreamOperator)dataStream
> .returns(myOutputType);
> } catch (ValidationException ex) {
> // ... handle exception when type has been used.
> }
> {code}
> It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
> check whether a particular transformation has a definitive output type set /
> used or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
flinkbot edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-564535655 ## CI report: * 252fe96f7a6254a18341f78cdb1bcbd24ae25a1c Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/140600093) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3473) * c44ddd079a072c171a1a10a075fd6f57610e2a4f Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/140628027) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3480) * f06f203456373d2685a1dc87996059ec6bde4993 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/141350358) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3657) * 962197a8d870bafcc7265cdd902657eddeddebd3 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142088890) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3854) * ba9e443d3a694bd741d69175ec4fe1e48cba1956 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142105543) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3862) * 786862ac623203275fd1b6fbf3e255abd233285f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142373804) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3935) * 9ffa9d4980fb49c40b5b1b8d2a00ef7aa43751de Travis: [CANCELED](https://travis-ci.com/flink-ci/flink/builds/142981538) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4073) * 8201386c978807c5c1dea003e03cdfef36539ca4 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142985614) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4074) * 6057b2d957ec44b73e06941ebf42388636e57a00 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15475) Add isOutputTypeUsed() API to Transformation
[
https://issues.apache.org/jira/browse/FLINK-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rong Rong updated FLINK-15475:
--
Description:
Currently there's no way to "peek" into a `Transformation` object and see if
`typeUsed` has been set or not. The only way is to invoke the `setOutputType`
API and wrap around it with a try-catch block.
It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
check whether a particular transformation has a definitive output type set /
used or not.
was:
Currently there's no way to "peek" into a Transformation and see if OutputType
has been used or not. The only way is to invoke the {{setOutputType}} API and
wrap around it with a try-catch block.
It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
check whether a particular transformation has a definitive output type set /
used or not.
> Add isOutputTypeUsed() API to Transformation
>
>
> Key: FLINK-15475
> URL: https://issues.apache.org/jira/browse/FLINK-15475
> Project: Flink
> Issue Type: Improvement
> Components: API / Core, API / DataSet, API / DataStream
>Reporter: Rong Rong
>Assignee: Rong Rong
>Priority: Minor
>
> Currently there's no way to "peek" into a `Transformation` object and see if
> `typeUsed` has been set or not. The only way is to invoke the `setOutputType`
> API and wrap around it with a try-catch block.
> It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
> check whether a particular transformation has a definitive output type set /
> used or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15475) Add isOutputTypeUsed() API to Transformation
[ https://issues.apache.org/jira/browse/FLINK-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rong Rong updated FLINK-15475: -- Description: Currently there's no way to "peek" into a `Transformation` object and see if `typeUsed` has been set or not. The only way is to invoke the `setOutputType` API and wrap around it with a try-catch block. See: It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to check whether a particular transformation has a definitive output type set / used or not. was: Currently there's no way to "peek" into a `Transformation` object and see if `typeUsed` has been set or not. The only way is to invoke the `setOutputType` API and wrap around it with a try-catch block. It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to check whether a particular transformation has a definitive output type set / used or not. > Add isOutputTypeUsed() API to Transformation > > > Key: FLINK-15475 > URL: https://issues.apache.org/jira/browse/FLINK-15475 > Project: Flink > Issue Type: Improvement > Components: API / Core, API / DataSet, API / DataStream >Reporter: Rong Rong >Assignee: Rong Rong >Priority: Minor > > Currently there's no way to "peek" into a `Transformation` object and see if > `typeUsed` has been set or not. The only way is to invoke the `setOutputType` > API and wrap around it with a try-catch block. See: > It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to > check whether a particular transformation has a definitive output type set / > used or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] walterddr commented on issue #9373: [FLINK-13596][ml] Add two utils for Table transformations
walterddr commented on issue #9373: [FLINK-13596][ml] Add two utils for Table transformations URL: https://github.com/apache/flink/pull/9373#issuecomment-570632426 thanks @qiuxiafei for the contribution. I've added https://issues.apache.org/jira/browse/FLINK-15475 to track the improvement on the API front. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-15475) Add isOutputTypeUsed() API to Transformation
Rong Rong created FLINK-15475:
-
Summary: Add isOutputTypeUsed() API to Transformation
Key: FLINK-15475
URL: https://issues.apache.org/jira/browse/FLINK-15475
Project: Flink
Issue Type: Improvement
Components: API / Core, API / DataSet, API / DataStream
Reporter: Rong Rong
Assignee: Rong Rong
Currently there's no way to "peek" into a Transformation and see if OutputType
has been used or not. The only way is to invoke the {{setOutputType}} API and
wrap around it with a try-catch block.
It would be nice if we have a `isTypeUsed()` or `isOutputTypeUsed()` API to
check whether a particular transformation has a definitive output type set /
used or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] Myasuka commented on a change in pull request #10692: [FLINK-15387] Expose missing RocksDB properties out via RocksDBNativeMetricOptions
Myasuka commented on a change in pull request #10692: [FLINK-15387] Expose
missing RocksDB properties out via RocksDBNativeMetricOptions
URL: https://github.com/apache/flink/pull/10692#discussion_r362882872
##
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBNativeMetricOptions.java
##
@@ -153,6 +153,30 @@
.defaultValue(false)
.withDescription("Monitor the current actual delayed write
rate. 0 means no delay.");
+ public static final ConfigOption IS_WRITE_STOPPED =
ConfigOptions
+ .key(RocksDBProperty.IsWriteStopped.getConfigKey())
+ .booleanType()
+ .defaultValue(false)
+ .withDescription("Track whether write has been stopped in
RocksDB. Returns 1 if write has been stopped, 0 otherwise.");
Review comment:
I think a brief explanation would be helpful for users.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot commented on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570630824 ## CI report: * 19e2bf9244667af4490f1463bdf48ba40d02e5ed UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils
flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils URL: https://github.com/apache/flink/pull/10761#issuecomment-570553190 ## CI report: * 544ee0be382ed4beaa34e9c56c59efb6bc36a384 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142988493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4079) * b1fe280cb359d96df845e2eaa7ecfa1f0d03f832 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143012009) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4083) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-12462) Classloading fails in 1.8.0 when Scalatest is loaded first
[
https://issues.apache.org/jira/browse/FLINK-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007621#comment-17007621
]
sophie commented on FLINK-12462:
So the upgrade fails, u still use the workaround? [~gael]
> Classloading fails in 1.8.0 when Scalatest is loaded first
> --
>
> Key: FLINK-12462
> URL: https://issues.apache.org/jira/browse/FLINK-12462
> Project: Flink
> Issue Type: Bug
> Components: API / Scala
>Affects Versions: 1.8.0
>Reporter: Gaël Renoux
>Priority: Major
>
> When using Flink 1.8.0 and Scala 2.11.12, if Scalatest is loaded first and I
> use some class from the Scala library, I get a:
> java.lang.ClassNotFoundException: scala.reflect.io.JavaToolsPlatformArchive
> To reproduce:
> - start with the Scala Maven archetype
> (org.apache.flink:flink-quickstart-scala:1.8.0)
> - in the POM, add a Scalatest dependency with test scope, before the Flink
> dependencies (org.scalatest:scalatest_${scala.binary.version}:3.0.4:test)
> - in StreamingJob, add: env.fromElements[List[String]]()
> I got a similar issue using Scala 2.12.7: java.lang.ClassNotFoundException:
> scala.reflect.internal.SymbolTable$ReflectStats
> It works when changing the Flink version to 1.7.2. It also works if Flink is
> loaded first, but this can be a bit harder to ensure sometimes (our own POM
> has a few profile definitions).
> I also managed to make it work by forcing the following dependency:
> org.scala-lang:scala-reflect:${scala.version}. Looking at the dependency
> tree, it looks like Flink needs the most recent version of scala-reflect but
> somehow doesn't enforce it. When Scalatest is loaded first, the version it
> declares (wich is slightly older: 2.11.11 or 2.12.3) gets used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wangyang0918 commented on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
wangyang0918 commented on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-570628439 @tillrohrmann I have added a fixup to address your comments. Please take a look. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr closed pull request #9373: [FLINK-13596][ml] Add two utils for Table transformations
walterddr closed pull request #9373: [FLINK-13596][ml] Add two utils for Table transformations URL: https://github.com/apache/flink/pull/9373 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-12462) Classloading fails in 1.8.0 when Scalatest is loaded first
[
https://issues.apache.org/jira/browse/FLINK-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006936#comment-17006936
]
Gaël Renoux edited comment on FLINK-12462 at 1/3/20 4:44 PM:
-
I just checked on my project, since we've switched to Flink 1.9.1 a few weeks
ago. I removed the workaround, and it works with Scala 2.12.7 (version in the
ticket) as well as Scala 2.12.10 (current Scala 2.12 version).
EDIT: I spoke too soon, I messed up my test. It still doesn't work.
was (Author: gael):
I just checked on my project, since we've switched to Flink 1.9.1 a few weeks
ago. I removed the workaround, and it works with Scala 2.12.7 (version in the
ticket) as well as Scala 2.12.10 (current Scala 2.12 version).
EDIT: I spoke too soon, I messed up my test. It still doesn't work.
> Classloading fails in 1.8.0 when Scalatest is loaded first
> --
>
> Key: FLINK-12462
> URL: https://issues.apache.org/jira/browse/FLINK-12462
> Project: Flink
> Issue Type: Bug
> Components: API / Scala
>Affects Versions: 1.8.0
>Reporter: Gaël Renoux
>Priority: Major
>
> When using Flink 1.8.0 and Scala 2.11.12, if Scalatest is loaded first and I
> use some class from the Scala library, I get a:
> java.lang.ClassNotFoundException: scala.reflect.io.JavaToolsPlatformArchive
> To reproduce:
> - start with the Scala Maven archetype
> (org.apache.flink:flink-quickstart-scala:1.8.0)
> - in the POM, add a Scalatest dependency with test scope, before the Flink
> dependencies (org.scalatest:scalatest_${scala.binary.version}:3.0.4:test)
> - in StreamingJob, add: env.fromElements[List[String]]()
> I got a similar issue using Scala 2.12.7: java.lang.ClassNotFoundException:
> scala.reflect.internal.SymbolTable$ReflectStats
> It works when changing the Flink version to 1.7.2. It also works if Flink is
> loaded first, but this can be a bit harder to ensure sometimes (our own POM
> has a few profile definitions).
> I also managed to make it work by forcing the following dependency:
> org.scala-lang:scala-reflect:${scala.version}. Looking at the dependency
> tree, it looks like Flink needs the most recent version of scala-reflect but
> somehow doesn't enforce it. When Scalatest is loaded first, the version it
> declares (wich is slightly older: 2.11.11 or 2.12.3) gets used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (FLINK-12461) Flink 1.8 not working with Scala 2.12.8
[ https://issues.apache.org/jira/browse/FLINK-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaël Renoux updated FLINK-12461: Comment: was deleted (was: I messed up my test, Flink 1.9.1 has the same problem. It's still only compatible with Scala 2.12.7.) > Flink 1.8 not working with Scala 2.12.8 > --- > > Key: FLINK-12461 > URL: https://issues.apache.org/jira/browse/FLINK-12461 > Project: Flink > Issue Type: Bug > Components: API / Scala >Affects Versions: 1.8.0 >Reporter: Gaël Renoux >Assignee: Aljoscha Krettek >Priority: Major > > When using 1.8 with Scala 2.12.8 and trying to parse a scala.Map (not a > java.util.Map), I get a: java.lang.ClassNotFoundException: > scala.math.Ordering$$anon$9 > To reproduce: > - start with the Scala Maven archetype > (org.apache.flink:flink-quickstart-scala:1.8.0) > - in the POM, set the scala.version to 2.12.8 and the scala.binary.version > to 2.12 > - in StreamingJob, add: env.fromElements[Map[String, Int]]() > It works with Scala 2.12.7 (well, without putting anything in the job, it > fails with "No operators defined in streaming topology", which is expected). > I suspect this is linked to the binary incompatiblity of 2.12.8 with 2.12.7 > (see the release note of 2.12.8), so compiling Flink with 2.12.8 instead of > 2.12.7 might be enough (although it might stop working with 2.12.7?) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-12461) Flink 1.8 not working with Scala 2.12.8
[ https://issues.apache.org/jira/browse/FLINK-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006931#comment-17006931 ] Gaël Renoux edited comment on FLINK-12461 at 1/3/20 4:44 PM: - By the way, I just tested Scala 2.12.10 with Flink 1.9.1, and it works fine. Flink 1.8 is no longer the default choice for a new project, so I guess it's not a big deal anymore and this ticket can be closed. I still think the doc could use a compatibility page, though. EDIT: my bad, I messed up my test. Flink 1.9.1 has the same problem. It's still only compatible with Scala 2.12.7. was (Author: gael): By the way, I just tested Scala 2.12.10 with Flink 1.9.1, and it works fine. Flink 1.8 is no longer the default choice for a new project, so I guess it's not a big deal anymore and this ticket can be closed. I still think the doc could use a compatibility page, though. > Flink 1.8 not working with Scala 2.12.8 > --- > > Key: FLINK-12461 > URL: https://issues.apache.org/jira/browse/FLINK-12461 > Project: Flink > Issue Type: Bug > Components: API / Scala >Affects Versions: 1.8.0 >Reporter: Gaël Renoux >Assignee: Aljoscha Krettek >Priority: Major > > When using 1.8 with Scala 2.12.8 and trying to parse a scala.Map (not a > java.util.Map), I get a: java.lang.ClassNotFoundException: > scala.math.Ordering$$anon$9 > To reproduce: > - start with the Scala Maven archetype > (org.apache.flink:flink-quickstart-scala:1.8.0) > - in the POM, set the scala.version to 2.12.8 and the scala.binary.version > to 2.12 > - in StreamingJob, add: env.fromElements[Map[String, Int]]() > It works with Scala 2.12.7 (well, without putting anything in the job, it > fails with "No operators defined in streaming topology", which is expected). > I suspect this is linked to the binary incompatiblity of 2.12.8 with 2.12.7 > (see the release note of 2.12.8), so compiling Flink with 2.12.8 instead of > 2.12.7 might be enough (although it might stop working with 2.12.7?) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-12462) Classloading fails in 1.8.0 when Scalatest is loaded first
[
https://issues.apache.org/jira/browse/FLINK-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17006936#comment-17006936
]
Gaël Renoux edited comment on FLINK-12462 at 1/3/20 4:43 PM:
-
I just checked on my project, since we've switched to Flink 1.9.1 a few weeks
ago. I removed the workaround, and it works with Scala 2.12.7 (version in the
ticket) as well as Scala 2.12.10 (current Scala 2.12 version).
EDIT: I spoke too soon, I messed up my test. It still doesn't work.
was (Author: gael):
I just checked on my project, since we've switched to Flink 1.9.1 a few weeks
ago. I removed the workaround, and it works with Scala 2.12.7 (version in the
ticket) as well as Scala 2.12.10 (current Scala 2.12 version).
> Classloading fails in 1.8.0 when Scalatest is loaded first
> --
>
> Key: FLINK-12462
> URL: https://issues.apache.org/jira/browse/FLINK-12462
> Project: Flink
> Issue Type: Bug
> Components: API / Scala
>Affects Versions: 1.8.0
>Reporter: Gaël Renoux
>Priority: Major
>
> When using Flink 1.8.0 and Scala 2.11.12, if Scalatest is loaded first and I
> use some class from the Scala library, I get a:
> java.lang.ClassNotFoundException: scala.reflect.io.JavaToolsPlatformArchive
> To reproduce:
> - start with the Scala Maven archetype
> (org.apache.flink:flink-quickstart-scala:1.8.0)
> - in the POM, add a Scalatest dependency with test scope, before the Flink
> dependencies (org.scalatest:scalatest_${scala.binary.version}:3.0.4:test)
> - in StreamingJob, add: env.fromElements[List[String]]()
> I got a similar issue using Scala 2.12.7: java.lang.ClassNotFoundException:
> scala.reflect.internal.SymbolTable$ReflectStats
> It works when changing the Flink version to 1.7.2. It also works if Flink is
> loaded first, but this can be a bit harder to ensure sometimes (our own POM
> has a few profile definitions).
> I also managed to make it work by forcing the following dependency:
> org.scala-lang:scala-reflect:${scala.version}. Looking at the dependency
> tree, it looks like Flink needs the most recent version of scala-reflect but
> somehow doesn't enforce it. When Scalatest is loaded first, the version it
> declares (wich is slightly older: 2.11.11 or 2.12.3) gets used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (FLINK-12462) Classloading fails in 1.8.0 when Scalatest is loaded first
[
https://issues.apache.org/jira/browse/FLINK-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gaël Renoux updated FLINK-12462:
Comment: was deleted
(was: I spoke too soon, I messed up my test. It still doesn't work.)
> Classloading fails in 1.8.0 when Scalatest is loaded first
> --
>
> Key: FLINK-12462
> URL: https://issues.apache.org/jira/browse/FLINK-12462
> Project: Flink
> Issue Type: Bug
> Components: API / Scala
>Affects Versions: 1.8.0
>Reporter: Gaël Renoux
>Priority: Major
>
> When using Flink 1.8.0 and Scala 2.11.12, if Scalatest is loaded first and I
> use some class from the Scala library, I get a:
> java.lang.ClassNotFoundException: scala.reflect.io.JavaToolsPlatformArchive
> To reproduce:
> - start with the Scala Maven archetype
> (org.apache.flink:flink-quickstart-scala:1.8.0)
> - in the POM, add a Scalatest dependency with test scope, before the Flink
> dependencies (org.scalatest:scalatest_${scala.binary.version}:3.0.4:test)
> - in StreamingJob, add: env.fromElements[List[String]]()
> I got a similar issue using Scala 2.12.7: java.lang.ClassNotFoundException:
> scala.reflect.internal.SymbolTable$ReflectStats
> It works when changing the Flink version to 1.7.2. It also works if Flink is
> loaded first, but this can be a bit harder to ensure sometimes (our own POM
> has a few profile definitions).
> I also managed to make it work by forcing the following dependency:
> org.scala-lang:scala-reflect:${scala.version}. Looking at the dependency
> tree, it looks like Flink needs the most recent version of scala-reflect but
> somehow doesn't enforce it. When Scalatest is loaded first, the version it
> declares (wich is slightly older: 2.11.11 or 2.12.3) gets used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12461) Flink 1.8 not working with Scala 2.12.8
[ https://issues.apache.org/jira/browse/FLINK-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007618#comment-17007618 ] Gaël Renoux commented on FLINK-12461: - I messed up my test, Flink 1.9.1 has the same problem. It's still only compatible with Scala 2.12.7. > Flink 1.8 not working with Scala 2.12.8 > --- > > Key: FLINK-12461 > URL: https://issues.apache.org/jira/browse/FLINK-12461 > Project: Flink > Issue Type: Bug > Components: API / Scala >Affects Versions: 1.8.0 >Reporter: Gaël Renoux >Assignee: Aljoscha Krettek >Priority: Major > > When using 1.8 with Scala 2.12.8 and trying to parse a scala.Map (not a > java.util.Map), I get a: java.lang.ClassNotFoundException: > scala.math.Ordering$$anon$9 > To reproduce: > - start with the Scala Maven archetype > (org.apache.flink:flink-quickstart-scala:1.8.0) > - in the POM, set the scala.version to 2.12.8 and the scala.binary.version > to 2.12 > - in StreamingJob, add: env.fromElements[Map[String, Int]]() > It works with Scala 2.12.7 (well, without putting anything in the job, it > fails with "No operators defined in streaming topology", which is expected). > I suspect this is linked to the binary incompatiblity of 2.12.8 with 2.12.7 > (see the release note of 2.12.8), so compiling Flink with 2.12.8 instead of > 2.12.7 might be enough (although it might stop working with 2.12.7?) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-570588137 @tillrohrmann I take a serious consideration and tend to remove the windows escape code. Since the windows command does not have a very good compatibility. And in different version, we have to escape different characters. Although, using double quotes could satisfy almost the cases. `TaskManagerLoadingDynamicPropertiesITCase` may be a unstable test for different windows versions. I think the Yarn integrated tests in `flink-yarn-tests` have the similar problems. Go back to the changes in the PR, the `getDynamicPropertiesAsString` will only be used for Yarn and Kubernetes. Maybe the flink client is running on windows, the flink `YarnResourceManager`, `KubernetesResourceManager`, `YarnTaskExecutorRunner`, `KubernetesTaskExecutorRunner` are always running on unix-like OS. So the current implementation may go the wrong way and exceed what we want to test. Ignoring the `TaskManagerLoadingDynamicPropertiesITCase` test when running on windows is enough. ---Updated- From the hadoop document, it support windows for long time. So maybe some flink users are using hadoop on windows. Escaping the dynamic properties is necessary and i will try to escape with double quotes. `^ "` needs to be processed specially. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-12462) Classloading fails in 1.8.0 when Scalatest is loaded first
[
https://issues.apache.org/jira/browse/FLINK-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007615#comment-17007615
]
Gaël Renoux commented on FLINK-12462:
-
I spoke too soon, I messed up my test. It still doesn't work.
> Classloading fails in 1.8.0 when Scalatest is loaded first
> --
>
> Key: FLINK-12462
> URL: https://issues.apache.org/jira/browse/FLINK-12462
> Project: Flink
> Issue Type: Bug
> Components: API / Scala
>Affects Versions: 1.8.0
>Reporter: Gaël Renoux
>Priority: Major
>
> When using Flink 1.8.0 and Scala 2.11.12, if Scalatest is loaded first and I
> use some class from the Scala library, I get a:
> java.lang.ClassNotFoundException: scala.reflect.io.JavaToolsPlatformArchive
> To reproduce:
> - start with the Scala Maven archetype
> (org.apache.flink:flink-quickstart-scala:1.8.0)
> - in the POM, add a Scalatest dependency with test scope, before the Flink
> dependencies (org.scalatest:scalatest_${scala.binary.version}:3.0.4:test)
> - in StreamingJob, add: env.fromElements[List[String]]()
> I got a similar issue using Scala 2.12.7: java.lang.ClassNotFoundException:
> scala.reflect.internal.SymbolTable$ReflectStats
> It works when changing the Flink version to 1.7.2. It also works if Flink is
> loaded first, but this can be a bit harder to ensure sometimes (our own POM
> has a few profile definitions).
> I also managed to make it work by forcing the following dependency:
> org.scala-lang:scala-reflect:${scala.version}. Looking at the dependency
> tree, it looks like Flink needs the most recent version of scala-reflect but
> somehow doesn't enforce it. When Scalatest is loaded first, the version it
> declares (wich is slightly older: 2.11.11 or 2.12.3) gets used instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-570588137 @tillrohrmann I take a serious consideration and tend to remove the windows escape code. Since the windows command does not have a very good compatibility. And in different version, we have to escape different characters. Although, using double quotes could satisfy almost the cases. `TaskManagerLoadingDynamicPropertiesITCase` will be a unstable test in the future. Go back to the changes in the PR, the `getDynamicPropertiesAsString` will only be used for Yarn and Kubernetes. Maybe the flink client is running on windows, the flink `YarnResourceManager`, `KubernetesResourceManager`, `YarnTaskExecutorRunner`, `KubernetesTaskExecutorRunner` are always running on unix-like OS. So the current implementation may go the wrong way and exceed what we want to test. Ignoring the `TaskManagerLoadingDynamicPropertiesITCase` test when running on windows is enough. ---Updated- From the hadoop document, it support windows for long time. So maybe some flink users are using hadoop on windows. Escaping the dynamic properties is necessary and i will try to escape with double quotes. `^ "` needs to be processed specially. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils
flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils URL: https://github.com/apache/flink/pull/10761#issuecomment-570553190 ## CI report: * 544ee0be382ed4beaa34e9c56c59efb6bc36a384 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142988493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4079) * b1fe280cb359d96df845e2eaa7ecfa1f0d03f832 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143012009) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4083) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
flinkbot commented on issue #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763#issuecomment-570620627 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 19e2bf9244667af4490f1463bdf48ba40d02e5ed (Fri Jan 03 16:20:21 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14200) Temporal Table Function Joins do not work on Tables (only TableSources) on the query side
[
https://issues.apache.org/jira/browse/FLINK-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-14200:
---
Labels: pull-request-available (was: )
> Temporal Table Function Joins do not work on Tables (only TableSources) on
> the query side
> -
>
> Key: FLINK-14200
> URL: https://issues.apache.org/jira/browse/FLINK-14200
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.9.0
> Environment: Java 8, Scala 2.11, Flink 1.9
>Reporter: Benoît Paris
>Assignee: Jark Wu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.2
>
> Attachments:
> temporal-table-function-query-side-as-not-table-source.zip
>
>
> This only affects the Blink planner. The legacy planner works fine.
> With Orders as a TableSource, and Orders2 as a Table with the same content:
> {code:java}
> tEnv.registerTableSource("Orders", new FooSource(new String[] {"o_currency",
> "o_amount", "o_proctime"}));
> Table orders2 = tEnv.sqlQuery("SELECT * FROM Orders");
> tEnv.registerTable("Orders2", orders2);{code}
> This works (TableSource on the query side):
> {code:java}
> SELECT
> o_amount * r_amount AS amount
> FROM Orders
> , LATERAL TABLE (Rates(o_proctime))
> WHERE r_currency = o_currency{code}
> While this does not (Table on the query side):
> {code:java}
> SELECT
> o_amount * r_amount AS amount
> FROM Orders2
> , LATERAL TABLE (Rates(o_proctime))
> WHERE r_currency = o_currency{code}
> Throwing an NPE in FlinkRelBuilder, called from
> LogicalCorrelateToJoinFromTemporalTableFunctionRule. Attached is Java code
> for reproduction, along with the full log and stacktrace, and a pom.xml.
> EDIT: This may not be Table vs TableSource, maybe more a projection or a
> decorrelate issue? Don't know how the planner works well enough to
> characterize it better.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong opened a new pull request #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source
wuchong opened a new pull request #10763: [FLINK-14200][table] Fix NPE for Temporal Table Function Join when left side is a query instead of a source URL: https://github.com/apache/flink/pull/10763 ## What is the purpose of the change Fix NPE when left input is a query (e.g. a filter/project/temporal join) in temporal table function join. ## Brief change log - Get the RelOptSchema from the leaf RelNode which holds the non-null RelOptSchema ## Verifying this change - Add a plan test that left input is a filter query - Add a IT case that left input is a temporal join ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-11120) The bug of timestampadd handles time
[
https://issues.apache.org/jira/browse/FLINK-11120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007605#comment-17007605
]
Rong Rong commented on FLINK-11120:
---
Hi [~x1q1j1] yes it will be in 1.10 release and it has already been backported
to 1.9.x as well.
Thanks [~jark] for fixing the JIRA status :-)
> The bug of timestampadd handles time
> -
>
> Key: FLINK-11120
> URL: https://issues.apache.org/jira/browse/FLINK-11120
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: Forward Xu
>Assignee: Forward Xu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.2, 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The error occur when {{timestampadd(MINUTE, 1, time '01:00:00')}} is executed:
> java.lang.ClassCastException: java.lang.Integer cannot be cast to
> java.lang.Long
> at org.apache.calcite.rex.RexBuilder.clean(RexBuilder.java:1520)
> at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:1318)
> at
> org.apache.flink.table.codegen.ExpressionReducer.reduce(ExpressionReducer.scala:135)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:620)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:540)
> at
> org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:288)
> I think it should meet the following conditions:
> ||expression||Expect the result||
> |timestampadd(MINUTE, -1, time '00:00:00')|23:59:00|
> |timestampadd(MINUTE, 1, time '00:00:00')|00:01:00|
> |timestampadd(MINUTE, 1, time '23:59:59')|00:00:59|
> |timestampadd(SECOND, 1, time '23:59:59')|00:00:00|
> |timestampadd(HOUR, 1, time '23:59:59')|00:59:59|
> This problem seems to be a bug in calcite. I have submitted isuse to calcite.
> The following is the link.
> CALCITE-2699
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15462) Support Prestosql JDBC source/sink
[ https://issues.apache.org/jira/browse/FLINK-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007560#comment-17007560 ] li yu edited comment on FLINK-15462 at 1/3/20 4:16 PM: --- [~lzljs3620320] [~jark] Would you help creating a PR and working on this issue ? minor change with non-trivial gain. was (Author: l0yu): [~lzljs3620320] Would you help creating a PR and working on this issue ? minor change with non-trivial gain. > Support Prestosql JDBC source/sink > -- > > Key: FLINK-15462 > URL: https://issues.apache.org/jira/browse/FLINK-15462 > Project: Flink > Issue Type: Wish > Components: Connectors / JDBC >Reporter: li yu >Priority: Major > > flink-jdbc support > Derby, Mysql, Postgre > [https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/dialect/JDBCDialects.java] > Could we add support for prestosql ? > Link to prestosql jdbc [https://prestosql.io/download.html] > Advantage is presto support a variety of data source (i.e we could > ingest/load data to or from those data source just through presto jdbc) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] GJL commented on issue #10116: [FLINK-14655] Change Type of Field jobStatusListeners from CopyOnWriteArrayList to ArrayList
GJL commented on issue #10116: [FLINK-14655] Change Type of Field jobStatusListeners from CopyOnWriteArrayList to ArrayList URL: https://github.com/apache/flink/pull/10116#issuecomment-570619142 Merging. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14765) Remove STATE_UPDATER in Execution
[ https://issues.apache.org/jira/browse/FLINK-14765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao updated FLINK-14765: - Description: After making access to ExecutionGraph single-threaded in FLINK-11417, we can simplify execution state update and get rid of STATE_UPDATER. (was: After making access to ExecutionGraph single-threaded in FLINK-11417, we can simplify execution state update and get rid of STATE_UPDATER while about the volatile at the moment which could be further investigated.) > Remove STATE_UPDATER in Execution > - > > Key: FLINK-14765 > URL: https://issues.apache.org/jira/browse/FLINK-14765 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > After making access to ExecutionGraph single-threaded in FLINK-11417, we can > simplify execution state update and get rid of STATE_UPDATER. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-14765) Remove STATE_UPDATER in Execution
[ https://issues.apache.org/jira/browse/FLINK-14765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Yao closed FLINK-14765. Resolution: Fixed master: f503664da181972f12aae6af2e8e6eb71d356085 > Remove STATE_UPDATER in Execution > - > > Key: FLINK-14765 > URL: https://issues.apache.org/jira/browse/FLINK-14765 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Reporter: vinoyang >Assignee: vinoyang >Priority: Major > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > After making access to ExecutionGraph single-threaded in FLINK-11417, we can > simplify execution state update and get rid of STATE_UPDATER while about the > volatile at the moment which could be further investigated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] GJL closed pull request #10288: [FLINK-14765] Remove STATE_UPDATER in Execution
GJL closed pull request #10288: [FLINK-14765] Remove STATE_UPDATER in Execution URL: https://github.com/apache/flink/pull/10288 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager
wangyang0918 edited a comment on issue #10532: [FLINK-15053][runtime] Escape all dynamical property values for taskmanager URL: https://github.com/apache/flink/pull/10532#issuecomment-570588137 @tillrohrmann I take a serious consideration and tend to remove the windows escape code. Since the windows command does not have a very good compatibility. And in different version, we have to escape different characters. Although, using double quotes could satisfy almost the cases. `TaskManagerLoadingDynamicPropertiesITCase` will be a unstable test in the future. Go back to the changes in the PR, the `getDynamicPropertiesAsString` will only be used for Yarn and Kubernetes. Maybe the flink client is running on windows, the flink `YarnResourceManager`, `KubernetesResourceManager`, `YarnTaskExecutorRunner`, `KubernetesTaskExecutorRunner` are always running on unix-like OS. So the current implementation may go the wrong way and exceed what we want to test. Ignoring the `TaskManagerLoadingDynamicPropertiesITCase` test when running on windows is enough. ---Updated- From the hadoop document, it support windows for long time. So maybe some flink users are using hadoop on windows. Escaping the dynamic properties is necessary and i will try to escape with double quotes. `^ % "` needs to be processed specially. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils
flinkbot edited a comment on issue #10761: [FLINK-15457][yarn] Remove finished TODO in YarnEntrypointUtils URL: https://github.com/apache/flink/pull/10761#issuecomment-570553190 ## CI report: * 544ee0be382ed4beaa34e9c56c59efb6bc36a384 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142988493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4079) * b1fe280cb359d96df845e2eaa7ecfa1f0d03f832 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/143012009) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4083) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9
flinkbot edited a comment on issue #10762: [FLINK-15115][kafka] Drop Kafka 0.8/0.9 URL: https://github.com/apache/flink/pull/10762#issuecomment-570591280 ## CI report: * 6bfb6d18af5fdfbddadf6da53adab78567d64aa9 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143008336) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4082) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#discussion_r362825818 ## File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTest.java ## @@ -0,0 +1,52 @@ +package org.apache.flink.runtime.taskexecutor.slot;/* Review comment: This does not look right. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362852268
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskExecutor.java
##
@@ -376,7 +372,7 @@ private void
handleStartTaskExecutorServicesException(Exception e) throws Except
return FutureUtils
.runAfterwards(
-
taskCompletionTracker.failIncompleteTasksAndGetTerminationFuture(),
+ taskSlotTable.freeAllSlots(new
FlinkException("The task executor is shutting down.")),
Review comment:
The current way of ensuring that all tasks are being terminated before the
returned future is completed is super hard to understand because the code path
is too convoluted and too many components depend on each other to make it
happen. It took me actually quite some time to figure this out and I'm somewhat
familiar with the code. Instead of introducing additional APIs which need to be
called in a very specific manner, I would suggest that we simply call
`taskSlotTable.closeAsync()` which returns a future which is completed once all
slots are closed. A slot is closed if all its task have terminated. The
information about a task being closed does not need to be passed into the
`TaskSlotTable` via the `freeSlot` but we could simple take a look at the
`Task#getTerminationFuture` in order to control when a `TaskSlot` is completely
released.
A good design principle is context independence. If possible a component
should work independent of its context meaning that it tries to decouple it
self as good as possible from other components. In this case, for example, we
require that `TaskExecutor` calls `TaskSlotTable#freeSlot` after we have called
`TaskSlotTable#freeAllSlots` and before we can call `close`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Assigned] (FLINK-14200) Temporal Table Function Joins do not work on Tables (only TableSources) on the query side
[
https://issues.apache.org/jira/browse/FLINK-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reassigned FLINK-14200:
---
Assignee: Jark Wu
> Temporal Table Function Joins do not work on Tables (only TableSources) on
> the query side
> -
>
> Key: FLINK-14200
> URL: https://issues.apache.org/jira/browse/FLINK-14200
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.9.0
> Environment: Java 8, Scala 2.11, Flink 1.9
>Reporter: Benoît Paris
>Assignee: Jark Wu
>Priority: Major
> Fix For: 1.9.2
>
> Attachments:
> temporal-table-function-query-side-as-not-table-source.zip
>
>
> This only affects the Blink planner. The legacy planner works fine.
> With Orders as a TableSource, and Orders2 as a Table with the same content:
> {code:java}
> tEnv.registerTableSource("Orders", new FooSource(new String[] {"o_currency",
> "o_amount", "o_proctime"}));
> Table orders2 = tEnv.sqlQuery("SELECT * FROM Orders");
> tEnv.registerTable("Orders2", orders2);{code}
> This works (TableSource on the query side):
> {code:java}
> SELECT
> o_amount * r_amount AS amount
> FROM Orders
> , LATERAL TABLE (Rates(o_proctime))
> WHERE r_currency = o_currency{code}
> While this does not (Table on the query side):
> {code:java}
> SELECT
> o_amount * r_amount AS amount
> FROM Orders2
> , LATERAL TABLE (Rates(o_proctime))
> WHERE r_currency = o_currency{code}
> Throwing an NPE in FlinkRelBuilder, called from
> LogicalCorrelateToJoinFromTemporalTableFunctionRule. Attached is Java code
> for reproduction, along with the full log and stacktrace, and a pom.xml.
> EDIT: This may not be Table vs TableSource, maybe more a projection or a
> decorrelate issue? Don't know how the planner works well enough to
> characterize it better.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362821873
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -146,12 +149,7 @@ public void start(SlotActions initialSlotActions) {
public void stop() {
started = false;
timerService.stop();
- allocatedSlots
- .values()
- .stream()
- .filter(slot -> !taskSlots.containsKey(slot.getIndex()))
- .forEach(TaskSlot::close);
- allocatedSlots.clear();
+ Preconditions.checkState(allocatedSlots.isEmpty(), "All slots
have to be freed before stopping TaskSlotTable");
Review comment:
This precondition check could be moved to the beginning of this method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362823684
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskManagerServicesBuilder.java
##
@@ -57,6 +61,7 @@ public TaskManagerServicesBuilder() {
broadcastVariableManager = new BroadcastVariableManager();
taskEventDispatcher = new TaskEventDispatcher();
taskSlotTable = mock(TaskSlotTable.class);
+
when(taskSlotTable.freeAllSlots(any(Throwable.class))).thenReturn(CompletableFuture.completedFuture(null));
Review comment:
Instead of adding mockito logic, I would suggest to extract an interface and
provide a proper testing implementation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362841392
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTableTest.java
##
@@ -243,8 +252,181 @@ public void testGenerateSlotReport() throws
SlotNotFoundException {
}
}
+ @Test
+ public void testAllocateSlot() {
+ final JobID jobId = new JobID();
+ final AllocationID allocationId = new AllocationID();
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable = createTaskSlotTableWithAllocatedSlot(
+ jobId,
+ allocationId,
+ new TestingSlotActionsBuilder().build());
+ Iterator allocatedSlots =
taskSlotTable.getAllocatedSlots(jobId);
+ TaskSlot nextSlot = allocatedSlots.next();
+ assertThat(nextSlot.getIndex(), is(0));
+ assertThat(nextSlot.getAllocationId(),
is(allocationId));
+ assertThat(nextSlot.getJobId(), is(jobId));
+ assertThat(allocatedSlots.hasNext(), is(false));
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test
+ public void testAddTask() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ EmptyInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, new
TestingSlotActionsBuilder().build());
+ Iterator tasks = taskSlotTable.getTasks(jobId);
+ Task nextTask = tasks.next();
+ assertThat(nextTask.getExecutionId(),
is(executionAttemptId));
+ assertThat(nextTask.getAllocationId(),
is(allocationId));
+ assertThat(tasks.hasNext(), is(false));
+ taskSlotTable.freeSlot(allocationId);
+ taskSlotTable.removeTask(executionAttemptId);
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test(timeout = 1)
+ public void testRemoveTaskCallsFreeSlotAction() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ CompletableFuture freeSlotFuture = new
CompletableFuture<>();
+ SlotActions slotActions = new
TestingSlotActions(freeSlotFuture::complete, (aid, uid) -> {});
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ EmptyInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, slotActions);
+ taskSlotTable.freeSlot(allocationId);
+ taskSlotTable.removeTask(executionAttemptId);
+ assertThat(freeSlotFuture.get(), is(allocationId));
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test(timeout = 1)
+ public void testFreeSlotInterruptsSubmittedTask() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ TestInterruptableInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, new
TestingSlotActionsBuilder().build());
+ TestInterruptableInvokable.STARTED_FUTURE.get();
+ assertThat(taskSlotTable.freeSlot(allocationId),
is(-1));
+ TestInterruptableInvokable.INTERRUPTED_FUTURE.get();
+
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362840043
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTest.java
##
@@ -0,0 +1,52 @@
+package org.apache.flink.runtime.taskexecutor.slot;/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.clusterframework.types.AllocationID;
+import org.apache.flink.runtime.clusterframework.types.ResourceProfile;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.junit.Test;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.hamcrest.CoreMatchers.is;
+import static org.junit.Assert.assertThat;
+
+public class TaskSlotTest {
Review comment:
We should also test that the `MemoryManager` stays alive until all `Tasks`
have terminated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362840658
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTableTest.java
##
@@ -243,8 +252,181 @@ public void testGenerateSlotReport() throws
SlotNotFoundException {
}
}
+ @Test
+ public void testAllocateSlot() {
+ final JobID jobId = new JobID();
+ final AllocationID allocationId = new AllocationID();
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable = createTaskSlotTableWithAllocatedSlot(
+ jobId,
+ allocationId,
+ new TestingSlotActionsBuilder().build());
+ Iterator allocatedSlots =
taskSlotTable.getAllocatedSlots(jobId);
+ TaskSlot nextSlot = allocatedSlots.next();
+ assertThat(nextSlot.getIndex(), is(0));
+ assertThat(nextSlot.getAllocationId(),
is(allocationId));
+ assertThat(nextSlot.getJobId(), is(jobId));
+ assertThat(allocatedSlots.hasNext(), is(false));
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test
+ public void testAddTask() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ EmptyInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, new
TestingSlotActionsBuilder().build());
+ Iterator tasks = taskSlotTable.getTasks(jobId);
+ Task nextTask = tasks.next();
+ assertThat(nextTask.getExecutionId(),
is(executionAttemptId));
+ assertThat(nextTask.getAllocationId(),
is(allocationId));
+ assertThat(tasks.hasNext(), is(false));
+ taskSlotTable.freeSlot(allocationId);
+ taskSlotTable.removeTask(executionAttemptId);
Review comment:
why are these calls necessary?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#discussion_r362826624 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlot.java ## @@ -77,6 +78,9 @@ /** Allocation id of this slot; null if not allocated. */ Review comment: I guess `allocationId` is never `null`. Could update the comment in a separate commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362835993
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlot.java
##
@@ -274,8 +282,10 @@ public String toString() {
@Override
public void close() {
+ Preconditions.checkState(tasks.isEmpty(), "All tasks have to be
removed before closing the slot");
Review comment:
I think the current API design is cumbersome. Before closing a `TaskSlot`,
we now always have to make sure that all `Tasks` have completely terminated and
are removed from the `TaskSlot`. Why can't we say that we fail all remaining
tasks, remove them from the slot once they are terminated and complete the
`closingFuture` once this has happened?
Please also take a look at `AutoCloseableAsync`. I think it makes sense to
let this class implement this interface.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362826149
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTest.java
##
@@ -0,0 +1,52 @@
+package org.apache.flink.runtime.taskexecutor.slot;/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.clusterframework.types.AllocationID;
+import org.apache.flink.runtime.clusterframework.types.ResourceProfile;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.junit.Test;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.hamcrest.CoreMatchers.is;
+import static org.junit.Assert.assertThat;
+
+public class TaskSlotTest {
+ @Test
Review comment:
I think a line break is missing before the annotation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362836388
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -146,12 +149,7 @@ public void start(SlotActions initialSlotActions) {
public void stop() {
started = false;
timerService.stop();
- allocatedSlots
- .values()
- .stream()
- .filter(slot -> !taskSlots.containsKey(slot.getIndex()))
- .forEach(TaskSlot::close);
- allocatedSlots.clear();
+ Preconditions.checkState(allocatedSlots.isEmpty(), "All slots
have to be freed before stopping TaskSlotTable");
Review comment:
Here again, `stop` requires us to first clean up all `allocatedSlots`. I
think this is not a good contract. Instead we should think about letting
`TaskSlotTable` implement `AutoCloseableAsync` and to trigger the release of
the `TaskSlots` when `stop` is being called.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362847991
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskExecutorTest.java
##
@@ -731,70 +697,6 @@ public void invoke() throws Exception {
}
}
- @Test
- public void testTaskInterruptionAndTerminationOnShutdown() throws
Exception {
Review comment:
By removing this test we lose coverage that we don't stop the `TaskExecutor`
before all tasks have been terminated. I couldn't find another test testing
this scenario. Hence, please add one or revert the removal.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362841198
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTableTest.java
##
@@ -243,8 +252,181 @@ public void testGenerateSlotReport() throws
SlotNotFoundException {
}
}
+ @Test
+ public void testAllocateSlot() {
+ final JobID jobId = new JobID();
+ final AllocationID allocationId = new AllocationID();
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable = createTaskSlotTableWithAllocatedSlot(
+ jobId,
+ allocationId,
+ new TestingSlotActionsBuilder().build());
+ Iterator allocatedSlots =
taskSlotTable.getAllocatedSlots(jobId);
+ TaskSlot nextSlot = allocatedSlots.next();
+ assertThat(nextSlot.getIndex(), is(0));
+ assertThat(nextSlot.getAllocationId(),
is(allocationId));
+ assertThat(nextSlot.getJobId(), is(jobId));
+ assertThat(allocatedSlots.hasNext(), is(false));
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test
+ public void testAddTask() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ EmptyInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, new
TestingSlotActionsBuilder().build());
+ Iterator tasks = taskSlotTable.getTasks(jobId);
+ Task nextTask = tasks.next();
+ assertThat(nextTask.getExecutionId(),
is(executionAttemptId));
+ assertThat(nextTask.getAllocationId(),
is(allocationId));
+ assertThat(tasks.hasNext(), is(false));
+ taskSlotTable.freeSlot(allocationId);
+ taskSlotTable.removeTask(executionAttemptId);
+ } finally {
+ stopTable(taskSlotTable);
+ }
+ }
+
+ @Test(timeout = 1)
+ public void testRemoveTaskCallsFreeSlotAction() throws Exception {
+ final JobID jobId = new JobID();
+ final ExecutionAttemptID executionAttemptId = new
ExecutionAttemptID();
+ final AllocationID allocationId = new AllocationID();
+ CompletableFuture freeSlotFuture = new
CompletableFuture<>();
+ SlotActions slotActions = new
TestingSlotActions(freeSlotFuture::complete, (aid, uid) -> {});
+ Task task = createTask(
+ jobId,
+ executionAttemptId,
+ allocationId,
+ EmptyInvokable.class);
+ TaskSlotTable taskSlotTable = null;
+ try {
+ taskSlotTable =
createTaskSlotTableWithStartedTask(task, slotActions);
+ taskSlotTable.freeSlot(allocationId);
Review comment:
Why do we need to call this method for the test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362839773
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTest.java
##
@@ -0,0 +1,52 @@
+package org.apache.flink.runtime.taskexecutor.slot;/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.clusterframework.types.AllocationID;
+import org.apache.flink.runtime.clusterframework.types.ResourceProfile;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.junit.Test;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.hamcrest.CoreMatchers.is;
+import static org.junit.Assert.assertThat;
+
+public class TaskSlotTest {
Review comment:
I think this test class lacks some important tests: Test that the `TaskSlot`
is not closed before all contained tasks have terminated.
Moreover, tests should be in the commit where the feature is being
introduced.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362825084
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskExecutorSubmissionTest.java
##
@@ -157,6 +159,9 @@ public void testTaskSubmission() throws Exception {
tmGateway.submitTask(tdd, env.getJobMasterId(),
timeout).get();
taskRunningFuture.get();
+ taskFinishedFuture.get();
+
+ taskSlotTable.removeTask(tdd.getExecutionAttemptId());
Review comment:
Why do we have to explicitly call `removeTask`? This seems to be quite
cumbersome.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362836806
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTableTest.java
##
@@ -239,7 +239,12 @@ public void testGenerateSlotReport() throws
SlotNotFoundException {
is(new SlotStatus(new SlotID(resourceId, 2),
TaskSlotUtils.DEFAULT_RESOURCE_PROFILE, null, null)),
is(new
SlotStatus(SlotID.generateDynamicSlotID(resourceId),
TaskSlotUtils.DEFAULT_RESOURCE_PROFILE, jobId, allocationId3;
} finally {
- taskSlotTable.stop();
+ stopTable(taskSlotTable);
}
}
+
+ private static void stopTable(TaskSlotTable taskSlotTable) {
+ taskSlotTable.freeAllSlots(null).join();
+ taskSlotTable.stop();
+ }
Review comment:
This is a good example for why the current API design is not very elegant.
Stopping a `TaskSlotTable` should simply be `taskSlotTable.stop`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362826032
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTest.java
##
@@ -0,0 +1,52 @@
+package org.apache.flink.runtime.taskexecutor.slot;/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.clusterframework.types.AllocationID;
+import org.apache.flink.runtime.clusterframework.types.ResourceProfile;
+import org.apache.flink.runtime.memory.MemoryManager;
+import org.junit.Test;
+
+import java.util.concurrent.CompletableFuture;
+
+import static org.hamcrest.CoreMatchers.is;
+import static org.junit.Assert.assertThat;
+
+public class TaskSlotTest {
Review comment:
`extends TestLogger` is missing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682:
[FLINK-15247][Runtime] Wait for all slots to be free before task executor
services shutdown upon stopping
URL: https://github.com/apache/flink/pull/10682#discussion_r362829356
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/slot/TaskSlotTable.java
##
@@ -146,12 +149,7 @@ public void start(SlotActions initialSlotActions) {
public void stop() {
started = false;
timerService.stop();
- allocatedSlots
- .values()
- .stream()
- .filter(slot -> !taskSlots.containsKey(slot.getIndex()))
- .forEach(TaskSlot::close);
- allocatedSlots.clear();
+ Preconditions.checkState(allocatedSlots.isEmpty(), "All slots
have to be freed before stopping TaskSlotTable");
taskSlots.values().forEach(TaskSlot::close);
Review comment:
This looks a bit odd since we checked the line above that `allocatedSlots`
is empty.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping
tillrohrmann commented on a change in pull request #10682: [FLINK-15247][Runtime] Wait for all slots to be free before task executor services shutdown upon stopping URL: https://github.com/apache/flink/pull/10682#discussion_r362823661 ## File path: flink-runtime/src/test/java/org/apache/flink/runtime/taskexecutor/TaskSubmissionTestEnvironment.java ## @@ -124,6 +124,7 @@ private TaskSubmissionTestEnvironment( this.taskSlotTable = mock(TaskSlotTable.class); when(taskSlotTable.tryMarkSlotActive(eq(jobId), any())).thenReturn(true); when(taskSlotTable.addTask(any(Task.class))).thenReturn(true); + when(taskSlotTable.freeAllSlots(any(Throwable.class))).thenReturn(CompletableFuture.completedFuture(null)); Review comment: Instead of adding mockito logic, I would suggest to extract an interface and provide a proper testing implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
