[jira] [Updated] (FLINK-14452) Keep only one execution topology in scheduler
[ https://issues.apache.org/jira/browse/FLINK-14452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-14452: Description: Currently there are 3 failover topology instances created, 2 permanently kept in JM. 2 scheduling topology instances created permanently kept in JM. It a waste of computation to build the topologies and memory to keep these topologies. Which may be a significant issue when the job scale is large. With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. We can change the scheduler to create and keep only one such an execution topology instance to reduce the cost to build and host execution topologies. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. was: With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. Currently there are 3 failover topology instances created, 2 permanently kept in JM. 2 scheduling topology instances created permanently kept in JM. This can be a waste of We can change the scheduler to create and keep only one such an execution topology instance to reduce the cost to build and host execution topologies. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. > Keep only one execution topology in scheduler > - > > Key: FLINK-14452 > URL: https://issues.apache.org/jira/browse/FLINK-14452 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Major > Fix For: 1.10.0 > > > Currently there are 3 failover topology instances created, 2 permanently kept > in JM. 2 scheduling topology instances created permanently kept in JM. It a > waste of computation to build the topologies and memory to keep these > topologies. Which may be a significant issue when the job scale is large. > With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology > are able to share one default implementation. > We can change the scheduler to create and keep only one such an execution > topology instance to reduce the cost to build and host execution topologies. > More details see FLINK-14330 and the [design > doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14452) Keep only one execution topology in scheduler
[ https://issues.apache.org/jira/browse/FLINK-14452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-14452: Description: With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. Currently there are 3 failover topology instances created, 2 permanently kept in JM. 2 scheduling topology instances created permanently kept in JM. This can be a waste of We can change the scheduler to create and keep only one such an execution topology instance to reduce the cost to build and host execution topologies. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. was: With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. We can change the scheduler to create and keep only one such an execution topology instance to reduce the cost to build and host execution topologies. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. > Keep only one execution topology in scheduler > - > > Key: FLINK-14452 > URL: https://issues.apache.org/jira/browse/FLINK-14452 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Major > Fix For: 1.10.0 > > > With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology > are able to share one default implementation. > Currently there are 3 failover topology instances created, 2 permanently kept > in JM. 2 scheduling topology instances created permanently kept in JM. This > can be a waste of > We can change the scheduler to create and keep only one such an execution > topology instance to reduce the cost to build and host execution topologies. > More details see FLINK-14330 and the [design > doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9
flinkbot edited a comment on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9 URL: https://github.com/apache/flink/pull/9949#issuecomment-544369447 ## CI report: * 4fae019a26fa9b4d8ac4ec62d24527dd892dee66 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132758308) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology
flinkbot edited a comment on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology URL: https://github.com/apache/flink/pull/9948#issuecomment-544357486 ## CI report: * f40f0b1786d0c8b389c3f77cb52d65033753207b : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132755102) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543240888 ## CI report: * a91235b01f24114bd0634f15dfaedc04a280b250 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132381590) * 5c0f625091d4c0f608a0d82c16b0d19daa8f997b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132748070) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132269780) * f8ded75e3aa2efbc305d52fabb3a71d71ad35080 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132514120) * 1097b5e17cca1f18f46f6ddacc7232b02846911d : UNKNOWN * 72827e1766dd28f66b72777fb8ad393cb1c0feef : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132746507) * bd109e2fa708a7de0f5939c8e508b2032c7b97e3 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output
flinkbot edited a comment on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output URL: https://github.com/apache/flink/pull/9905#issuecomment-542185017 ## CI report: * 8b3baa21776bbfa97a36413136fba365ea960703 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/131961879) * 2caa4c446127547d127433d5614b6994cd095330 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14441) Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
[ https://issues.apache.org/jira/browse/FLINK-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-14441: Component/s: (was: Table SQL / Planner) Table SQL / API > Fix ValueLiteralExpression#getValueAs when valueClass is Period.class > - > > Key: FLINK-14441 > URL: https://issues.apache.org/jira/browse/FLINK-14441 > Project: Flink > Issue Type: Bug > Components: Table SQL / API >Affects Versions: 1.10.0 >Reporter: hailong wang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In ValueLiteralExpression#getValueAs, when valueClass is Period.class, the > expect class is inconsistent with the result class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543231644 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit e1aeee937b887092b0570839f100d3da844971ab (Mon Oct 21 06:44:27 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14441).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output
flinkbot edited a comment on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output URL: https://github.com/apache/flink/pull/9905#issuecomment-542178315 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 2caa4c446127547d127433d5614b6994cd095330 (Mon Oct 21 06:43:26 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wangxlong commented on a change in pull request #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wangxlong commented on a change in pull request #9931:
[FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when
valueClass is Period.class
URL: https://github.com/apache/flink/pull/9931#discussion_r336856606
##
File path:
flink-table/flink-table-common/src/test/java/org/apache/flink/table/expressions/ExpressionTest.java
##
@@ -160,6 +161,16 @@ public void testSymbolValueLiteralExtraction() {
.orElseThrow(AssertionError::new));
}
+ @Test
+ public void testPeriodlValueLiteralExtraction() {
Review comment:
Thank you very much @wuchong. I have fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wangxlong commented on a change in pull request #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wangxlong commented on a change in pull request #9931:
[FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when
valueClass is Period.class
URL: https://github.com/apache/flink/pull/9931#discussion_r336856655
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/expressions/ValueLiteralExpression.java
##
@@ -106,7 +106,7 @@ else if (valueClass == Long.class && clazz ==
Duration.class) {
else if (valueClass == Period.class && clazz == Integer.class) {
final Period period = (Period) value;
- convertedValue = period.toTotalMonths();
+ convertedValue = (int)period.toTotalMonths();
Review comment:
Thank you very much @wuchong. I have fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wuchong commented on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-544372194 Thanks @wangxlong for the updating. I will merge this when travis is passed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output
zhijiangW commented on issue #9905: [FLINK-14396][network] Implement rudimentary non-blocking network output URL: https://github.com/apache/flink/pull/9905#issuecomment-544371931 Thanks for the review @pnowojski . I have updated the codes for addressing above concerns. Only left one issue of providing non-blocking request from global `NetworkBufferPool`, and I think it is better to be done separately together with adjusting backpressure monitor later. Because when we provide `NetworkBufferPool#isAvailable` to avoid `availableMemorySegments.wait(2000);` in `LocalBufferPool`, it would destroy the exiting monitor of backpressure. And now this PR could still keep the previous behavior of backpressure monitor. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543231644 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit e1aeee937b887092b0570839f100d3da844971ab (Mon Oct 21 06:41:24 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14441).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9
flinkbot commented on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9 URL: https://github.com/apache/flink/pull/9949#issuecomment-544369447 ## CI report: * 4fae019a26fa9b4d8ac4ec62d24527dd892dee66 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543240888 ## CI report: * a91235b01f24114bd0634f15dfaedc04a280b250 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132381590) * 5c0f625091d4c0f608a0d82c16b0d19daa8f997b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132748070) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments
flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments URL: https://github.com/apache/flink/pull/9927#issuecomment-543180251 ## CI report: * 14b59100e851fa7494838f1b5c68b4dc732da590 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132339165) * 9293184a08efd0c075be4fa67c02170894287099 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132493044) * 995c2d239889a09bffd87f228b222434cb052820 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132753822) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit bd109e2fa708a7de0f5939c8e508b2032c7b97e3 (Mon Oct 21 06:26:06 UTC 2019) **Warnings:** * **3 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543231644 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5c0f625091d4c0f608a0d82c16b0d19daa8f997b (Mon Oct 21 06:19:59 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14441).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wuchong commented on a change in pull request #9931:
[FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when
valueClass is Period.class
URL: https://github.com/apache/flink/pull/9931#discussion_r336850924
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/expressions/ValueLiteralExpression.java
##
@@ -106,7 +106,7 @@ else if (valueClass == Long.class && clazz ==
Duration.class) {
else if (valueClass == Period.class && clazz == Integer.class) {
final Period period = (Period) value;
- convertedValue = period.toTotalMonths();
+ convertedValue = (int)period.toTotalMonths();
Review comment:
```suggestion
convertedValue = (int) period.toTotalMonths();
```
Please add a space after the casting.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wuchong commented on a change in pull request #9931:
[FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when
valueClass is Period.class
URL: https://github.com/apache/flink/pull/9931#discussion_r336850720
##
File path:
flink-table/flink-table-common/src/test/java/org/apache/flink/table/expressions/ExpressionTest.java
##
@@ -160,6 +161,16 @@ public void testSymbolValueLiteralExtraction() {
.orElseThrow(AssertionError::new));
}
+ @Test
+ public void testPeriodlValueLiteralExtraction() {
Review comment:
```suggestion
public void testPeriodValueLiteralExtraction() {
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543231644 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5c0f625091d4c0f608a0d82c16b0d19daa8f997b (Mon Oct 21 06:14:54 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14441).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
wuchong commented on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-544364415 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14441) Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
[ https://issues.apache.org/jira/browse/FLINK-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955768#comment-16955768 ] Jark Wu commented on FLINK-14441: - Thanks [~hailong wang], I will reply you in the PR. > Fix ValueLiteralExpression#getValueAs when valueClass is Period.class > - > > Key: FLINK-14441 > URL: https://issues.apache.org/jira/browse/FLINK-14441 > Project: Flink > Issue Type: Bug > Components: Table SQL / Planner >Affects Versions: 1.10.0 >Reporter: hailong wang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In ValueLiteralExpression#getValueAs, when valueClass is Period.class, the > expect class is inconsistent with the result class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology
flinkbot edited a comment on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology URL: https://github.com/apache/flink/pull/9948#issuecomment-544357486 ## CI report: * f40f0b1786d0c8b389c3f77cb52d65033753207b : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132755102) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6
flinkbot edited a comment on issue #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6 URL: https://github.com/apache/flink/pull/9714#issuecomment-532990781 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9c2f79d83a667cefc8364c4244f183e3798d9602 (Mon Oct 21 05:55:32 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9
flinkbot commented on issue #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9 URL: https://github.com/apache/flink/pull/9949#issuecomment-544360049 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 4fae019a26fa9b4d8ac4ec62d24527dd892dee66 (Mon Oct 21 05:54:26 UTC 2019) **Warnings:** * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-1).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] liupc closed pull request #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6
liupc closed pull request #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6 URL: https://github.com/apache/flink/pull/9714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] liupc commented on issue #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6
liupc commented on issue #9714: [FLINK-14123]Change taskmanager.memory.fraction default value to 0.6 URL: https://github.com/apache/flink/pull/9714#issuecomment-544359887 Close for opening another PR for the docs update https://github.com/apache/flink/pull/9949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] liupc opened a new pull request #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9
liupc opened a new pull request #9949: [FLINK-14123][DOCS]Update release notes for flink-1.9 URL: https://github.com/apache/flink/pull/9949 ## What is the purpose of the change Update the release notes docs for flink-1.9 on the memory management. We suggest users to manually lower the `taskmanager.memory.fraction` conf value to avoid OOMs in some case. ## Brief change log - Update release notes for flink-1.9 on the memory management. ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14452) Keep only one execution topology in scheduler
[ https://issues.apache.org/jira/browse/FLINK-14452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-14452: Description: With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. We can change the scheduler to create and keep only one such an execution topology instance to reduce the cost to build and host execution topologies. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. was: With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology are able to share one default implementation. We can change the scheduler to create and keep only one such an execution topology instance to save computing and memory cost. More details see FLINK-14330 and the [design doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. > Keep only one execution topology in scheduler > - > > Key: FLINK-14452 > URL: https://issues.apache.org/jira/browse/FLINK-14452 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Major > Fix For: 1.10.0 > > > With FLINK-14450 and FLINK-14451, the SchedulingTopology and FailoverTopology > are able to share one default implementation. We can change the scheduler to > create and keep only one such an execution topology instance to reduce the > cost to build and host execution topologies. > More details see FLINK-14330 and the [design > doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology
flinkbot commented on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology URL: https://github.com/apache/flink/pull/9948#issuecomment-544357486 ## CI report: * f40f0b1786d0c8b389c3f77cb52d65033753207b : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support
flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947#issuecomment-544343979 ## CI report: * af2cf1fe9c360487ec0ffa6e01d9bcf982daa8fe : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/132749373) * d76a1e9eecd5b4a197cd30e9bd6195a159398ef4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132752553) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14123) Lower the default value of taskmanager.memory.fraction
[
https://issues.apache.org/jira/browse/FLINK-14123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liupengcheng updated FLINK-14123:
-
Summary: Lower the default value of taskmanager.memory.fraction (was:
Change taskmanager.memory.fraction default value to 0.6)
> Lower the default value of taskmanager.memory.fraction
> --
>
> Key: FLINK-14123
> URL: https://issues.apache.org/jira/browse/FLINK-14123
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration
>Affects Versions: 1.9.0
>Reporter: liupengcheng
>Assignee: liupengcheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, we are testing flink batch task, such as terasort, however, it
> started only awhile then it failed due to OOM.
>
> {code:java}
> org.apache.flink.client.program.ProgramInvocationException: Job failed.
> (JobID: a807e1d635bd4471ceea4282477f8850)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:326)
> at
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
> at
> org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:539)
> at
> com.github.ehiggs.spark.terasort.FlinkTeraSort$.main(FlinkTeraSort.scala:89)
> at
> com.github.ehiggs.spark.terasort.FlinkTeraSort.main(FlinkTeraSort.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:604)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:466)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
> at
> org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1007)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1080)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1080)
> Caused by: org.apache.flink.runtime.client.JobExecutionException: Job
> execution failed.
> at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
> at
> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:259)
> ... 23 more
> Caused by: java.lang.RuntimeException: Error obtaining the sorted input:
> Thread 'SortMerger Reading Thread' terminated due to an exception: GC
> overhead limit exceeded
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:650)
> at
> org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1109)
> at org.apache.flink.runtime.operators.NoOpDriver.run(NoOpDriver.java:82)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504)
> at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Thread 'SortMerger Reading Thread' terminated
> due to an exception: GC overhead limit exceeded
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:831)
> Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
> at
> org.apache.flink.api.common.typeutils.base.array.BytePrimitiveArraySerializer.deserialize(BytePrimitiveArraySerializer.java:84)
> at
> org.apache.flink.api.common.typeutils.base.array.BytePr
[GitHub] [flink] flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments
flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments URL: https://github.com/apache/flink/pull/9927#issuecomment-543180251 ## CI report: * 14b59100e851fa7494838f1b5c68b4dc732da590 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132339165) * 9293184a08efd0c075be4fa67c02170894287099 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132493044) * 995c2d239889a09bffd87f228b222434cb052820 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132753822) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support
flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947#issuecomment-544343979 ## CI report: * af2cf1fe9c360487ec0ffa6e01d9bcf982daa8fe : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/132749373) * d76a1e9eecd5b4a197cd30e9bd6195a159398ef4 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132752553) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments
flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments URL: https://github.com/apache/flink/pull/9927#issuecomment-543180251 ## CI report: * 14b59100e851fa7494838f1b5c68b4dc732da590 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132339165) * 9293184a08efd0c075be4fa67c02170894287099 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132493044) * 995c2d239889a09bffd87f228b222434cb052820 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology
flinkbot commented on issue #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology URL: https://github.com/apache/flink/pull/9948#issuecomment-544351168 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f40f0b1786d0c8b389c3f77cb52d65033753207b (Mon Oct 21 05:09:55 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14451) Refactor FailoverTopology to extend base topology
[ https://issues.apache.org/jira/browse/FLINK-14451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14451: --- Labels: pull-request-available (was: ) > Refactor FailoverTopology to extend base topology > - > > Key: FLINK-14451 > URL: https://issues.apache.org/jira/browse/FLINK-14451 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Coordination >Affects Versions: 1.10.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > > This task is to change FailoverTopology to extend the base topology > introduced in FLINK-14330. ExecutionGraphToSchedulingTopologyAdapter(default > implementation of SchedulingTopology) should also implements and replace > DefaultFailoverTopology. > More details see FLINK-14330 and the [design > doc|https://docs.google.com/document/d/1f88luAOfUQ6Pm4JkxYexLXpfH-crcXJdbubi1pS2Y5A/edit#]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zhuzhurk opened a new pull request #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology
zhuzhurk opened a new pull request #9948: [FLINK-14451][runtime] Refactor FailoverTopology to extend base topology URL: https://github.com/apache/flink/pull/9948 ## What is the purpose of the change This PR is to change FailoverTopology to extend the base topology introduced in FLINK-14330. ExecutionGraphToSchedulingTopologyAdapter(default implementation of SchedulingTopology) is also to implement the FailoverTopology and replace DefaultFailoverTopology. More details see FLINK-14330. ## Brief change log - *Change FailoverTopology to extend the base topology and unifying the interfaces* - *Refactored the implementations of FailoverTopology* - *Refactored the usages of SchedulingTopology* ## Verifying this change This change is already covered by existing tests, such as ExecutionGraphToSchedulingTopologyAdapterTest*. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-9953) Active Kubernetes integration
[
https://issues.apache.org/jira/browse/FLINK-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955721#comment-16955721
]
Yang Wang edited comment on FLINK-9953 at 10/21/19 5:04 AM:
I do not insist to introduce the `kubernetes-job.sh`. If we could make the
`flink run` more flexible, it is reasonable for me.
1. No jar is specified
In this way, before the jm/tm pod, the jar has already in the image. The job
graph is generated in the jm pod. Since the job is submitted on the jm side, we
could only support detach mode.
{code:java}
./bin/flink run -d -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s-windowjoin:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
-kuser-jar /path/in/the/image/WinowJoin.jar
{code}
2. Client local jar is specified
We need to deploy a session cluster to simulate per job mode. Since the jm need
to start first, and then Flink client submit the user jar and job graph to jm
pod through rest. Detach and attach mode are both supported.
{code:java}
./bin/flink run [-d] -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
/path/in/the/client/WinowJoin.jar
{code}
The primary reason to introduce `kubernetes-job.sh` is for #1. And `flink run`
is always going to #2.
[~trohrmann] [~felixzheng] , waiting for your response.
was (Author: fly_in_gis):
[~felixzheng]
I do not insist to introduce the `kubernetes-job.sh`. If we could make the
`flink run` more flexible, it is reasonable for me.
1. No jar is specified
In this way, before the jm/tm pod, the jar has already in the image. The job
graph is generated in the jm pod. Since the job is submitted on the jm side, we
could only support detach mode.
{code:java}
./bin/flink run -d -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s-windowjoin:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
-kuser-jar /path/in/the/image/WinowJoin.jar
{code}
2. Client local jar is specified
We need to deploy a session cluster to simulate per job mode. Since the jm need
to start first, and then Flink client submit the user jar and job graph to jm
pod through rest. Detach and attach mode are both supported.
{code:java}
./bin/flink run [-d] -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
/path/in/the/client/WinowJoin.jar
{code}
The primary reason to introduce `kubernetes-job.sh` is for #1. And `flink run`
is always going to #2.
[~trohrmann], waiting for your response.
> Active Kubernetes integration
> -
>
> Key: FLINK-9953
> URL: https://issues.apache.org/jira/browse/FLINK-9953
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
>Reporter: Till Rohrmann
>Assignee: Yang Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue tracking Flink's active Kubernetes integration.
> Active means in this context that the {{ResourceManager}} can talk to
> Kubernetes to launch new pods similar to Flink's Yarn and Mesos integration.
> Phase1 implementation will have complete functions to make flink running on
> kubernetes. Phrase2 is mainly focused on production optimization, including
> k8s native high-availability, storage, network, log collector and etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments
flinkbot edited a comment on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments URL: https://github.com/apache/flink/pull/9927#issuecomment-543172555 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 995c2d239889a09bffd87f228b222434cb052820 (Mon Oct 21 04:59:32 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lirui-apache commented on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments
lirui-apache commented on issue #9927: [FLINK-14397][hive] Failed to run Hive UDTF with array arguments URL: https://github.com/apache/flink/pull/9927#issuecomment-544349155 PR updated to cover more types in the `ObjectInspector -> Class` mapping. Also noted an issue of reading date from Hive and fixed it by bridging date type to `java.sql.Date` (can be in a separate PR though). @bowenli86 @xuefuz Please have another look. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support
flinkbot edited a comment on issue #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947#issuecomment-544343979 ## CI report: * af2cf1fe9c360487ec0ffa6e01d9bcf982daa8fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132749373) * d76a1e9eecd5b4a197cd30e9bd6195a159398ef4 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer.
flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer. URL: https://github.com/apache/flink/pull/9576#issuecomment-526650772 ## CI report: * 4fdf72f3ec019e4c90c5ab8e0d71f82c7365c26c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125254011) * e04e263ec3c7819ed4fa3049e3584a6f1ec81c54 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129299234) * 9a55dacccf5141e4379af27d708cf3f9850f550b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132520117) * 010706cbbd12d506f23a52e8b71474ab2b06c2b6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132526261) * 588d625109e9ec53209d57a5187017c506134049 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132746489) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9947: [FLINK-14469][python] Drop Python 2 support
flinkbot commented on issue #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947#issuecomment-544343979 ## CI report: * af2cf1fe9c360487ec0ffa6e01d9bcf982daa8fe : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543240888 ## CI report: * a91235b01f24114bd0634f15dfaedc04a280b250 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132381590) * 5c0f625091d4c0f608a0d82c16b0d19daa8f997b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132748070) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132269780) * f8ded75e3aa2efbc305d52fabb3a71d71ad35080 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132514120) * 1097b5e17cca1f18f46f6ddacc7232b02846911d : UNKNOWN * 72827e1766dd28f66b72777fb8ad393cb1c0feef : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132746507) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9947: [FLINK-14469][python] Drop Python 2 support
flinkbot commented on issue #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947#issuecomment-544341297 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit af2cf1fe9c360487ec0ffa6e01d9bcf982daa8fe (Mon Oct 21 04:11:37 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14469).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14469) Drop Python 2 support
[ https://issues.apache.org/jira/browse/FLINK-14469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-14469: --- Labels: pull-request-available (was: ) > Drop Python 2 support > - > > Key: FLINK-14469 > URL: https://issues.apache.org/jira/browse/FLINK-14469 > Project: Flink > Issue Type: Task > Components: API / Python >Affects Versions: 1.10.0 >Reporter: Dian Fu >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0 > > > As discussed in the > [ML|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html], > the aim of this Jira is to drop the Python 2 support. This includes the > following work: > 1. Remove Python 2.7 CI tests > 2. Throw an exception if Python below 3.5 is used > 3. Clean up the code which is specific for Python 2.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] dianfu opened a new pull request #9947: [FLINK-14469][python] Drop Python 2 support
dianfu opened a new pull request #9947: [FLINK-14469][python] Drop Python 2 support URL: https://github.com/apache/flink/pull/9947 ## What is the purpose of the change *This pull request drops Python 2 support.* ## Brief change log - *Remove the CI for Python 2.7* - *Throw an exception if Python below 3.5 is used* - *Clean up code specific for Python 2.x* ## Verifying this change This change is a code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543240888 ## CI report: * a91235b01f24114bd0634f15dfaedc04a280b250 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132381590) * 5c0f625091d4c0f608a0d82c16b0d19daa8f997b : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132269780) * f8ded75e3aa2efbc305d52fabb3a71d71ad35080 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132514120) * 1097b5e17cca1f18f46f6ddacc7232b02846911d : UNKNOWN * 72827e1766dd28f66b72777fb8ad393cb1c0feef : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer.
flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer. URL: https://github.com/apache/flink/pull/9576#issuecomment-526650772 ## CI report: * 4fdf72f3ec019e4c90c5ab8e0d71f82c7365c26c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125254011) * e04e263ec3c7819ed4fa3049e3584a6f1ec81c54 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129299234) * 9a55dacccf5141e4379af27d708cf3f9850f550b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132520117) * 010706cbbd12d506f23a52e8b71474ab2b06c2b6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132526261) * 588d625109e9ec53209d57a5187017c506134049 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132746489) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (FLINK-13485) Translate "Table API Example Walkthrough" page into Chinese
[
https://issues.apache.org/jira/browse/FLINK-13485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu resolved FLINK-13485.
-
Fix Version/s: 1.10.0
Resolution: Fixed
1.10.0: 6dfcbc0bf3769a1a36ef8d19d23260485dbee4c8
> Translate "Table API Example Walkthrough" page into Chinese
> ---
>
> Key: FLINK-13485
> URL: https://issues.apache.org/jira/browse/FLINK-13485
> Project: Flink
> Issue Type: Sub-task
> Components: chinese-translation, Documentation
>Reporter: Jark Wu
>Assignee: AT-Fieldless
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> FLINK-12747 has added a page to walkthrough Table API. We can translate it
> into Chinese now.
> The page is located in {{docs/getting-started/walkthroughs/table_api.zh.md}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
flinkbot edited a comment on issue #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784#issuecomment-535882318 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 096444b219447b1d467fe63ab6278c1c28802eb0 (Mon Oct 21 03:52:21 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong closed pull request #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
wuchong closed pull request #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese URL: https://github.com/apache/flink/pull/9784 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on a change in pull request #9784: [FLINK-13485] [chinese-translation] Translate "Table API Example Walkthrough" page into Chinese
wuchong commented on a change in pull request #9784: [FLINK-13485]
[chinese-translation] Translate "Table API Example Walkthrough" page into
Chinese
URL: https://github.com/apache/flink/pull/9784#discussion_r336824601
##
File path: docs/getting-started/walkthroughs/table_api.zh.md
##
@@ -299,11 +297,11 @@ This query consumes all records from the `transactions`
table, calculates the re
> 2, 2019-01-01 04:00:00.0, $760.76
{% endhighlight %}
-## Adding Windows
+## 添加窗口
-Grouping data based on time is a typical operation in data processing,
especially when working with infinite streams.
-A grouping based on time is called a [window]({{ site.baseurl }}
/dev/stream/operators/windows.html) and Flink offers flexible windowing
semantics.
-The most basic type of window is called a `Tumble` window, which has a fixed
size and whose buckets do not overlap.
+根据时间进行分组在数据处理中是一种很常见的方式，特别是在处理无限的数据流时。
+基于时间的分组称为[窗口]({{ site.baseurl }}/zh/dev/stream/operators/windows.html) ，Flink
提供了灵活的窗口语义。
+其中最基础的是 `滚动窗口`，它具有固定大小且窗口之间不重叠。
Review comment:
```suggestion
其中最基础的是 `Tumble` window （滚动窗口），它具有固定大小且窗口之间不重叠。
```
Tumble 这里是更多的是指 代码中的 `Tumble`，所以不翻译。
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9946: [FLINK-14468][docs] Update Kubernetes docs
flinkbot edited a comment on issue #9946: [FLINK-14468][docs] Update Kubernetes docs URL: https://github.com/apache/flink/pull/9946#issuecomment-544252792 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 43019ed32dab94129ea0a51d645f2dc55d18cd3d (Mon Oct 21 03:41:10 UTC 2019) **Warnings:** * Documentation files were touched, but no `.zh.md` files: Update Chinese documentation or file Jira ticket. * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14468).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wangyang0918 commented on a change in pull request #9946: [FLINK-14468][docs] Update Kubernetes docs
wangyang0918 commented on a change in pull request #9946: [FLINK-14468][docs] Update Kubernetes docs URL: https://github.com/apache/flink/pull/9946#discussion_r336828488 ## File path: docs/ops/deployment/kubernetes.md ## @@ -230,6 +240,8 @@ spec: volumeMounts: - name: flink-config-volume mountPath: /opt/flink/conf/ +securityContext: + runAsUser: Review comment: Why securityContext need to be set? What will happen if the uid() does not exist. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542991773 ## CI report: * a7539d69ef1d149aaf1b6a749a04ca94b16ef6d2 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132269780) * f8ded75e3aa2efbc305d52fabb3a71d71ad35080 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132514120) * 1097b5e17cca1f18f46f6ddacc7232b02846911d : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14441) Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
[ https://issues.apache.org/jira/browse/FLINK-14441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955725#comment-16955725 ] hailong wang commented on FLINK-14441: -- Hi [~jark], Thanks you for your tip. I have fix it by casting the return value to Integer. But travis failed in python module which has be fixed. What should I do. > Fix ValueLiteralExpression#getValueAs when valueClass is Period.class > - > > Key: FLINK-14441 > URL: https://issues.apache.org/jira/browse/FLINK-14441 > Project: Flink > Issue Type: Bug > Components: Table SQL / Planner >Affects Versions: 1.10.0 >Reporter: hailong wang >Priority: Major > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In ValueLiteralExpression#getValueAs, when valueClass is Period.class, the > expect class is inconsistent with the result class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex
flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex URL: https://github.com/apache/flink/pull/9601#issuecomment-527404637 ## CI report: * 13d895349390698404a375fb4362a41b736ab0c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125577621) * 5a1866be2d9765b0e941a531d5e9dda8616fecf4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125842787) * 2db52501630bae1806c93573eb3c88b091b5c951 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126002450) * 8a9c4066216e3bf81a258bc8a8c15c33e67bf1d4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126009476) * 2cbd3756b9d00c9b42d494e487d4950c8bb280e0 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129579795) * b2b94e7516dda963422234b5c19eb09baa278480 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132299831) * fd0cf151970941ad425bfbe749b97852d154efbe : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132499327) * a8d4af6f4318992e43bdeeebf7aec54fcb60d667 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132742145) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer.
flinkbot edited a comment on issue #9576: [FLINK-13915][ml] Add several base classes of summarizer. URL: https://github.com/apache/flink/pull/9576#issuecomment-526650772 ## CI report: * 4fdf72f3ec019e4c90c5ab8e0d71f82c7365c26c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125254011) * e04e263ec3c7819ed4fa3049e3584a6f1ec81c54 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/129299234) * 9a55dacccf5141e4379af27d708cf3f9850f550b : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132520117) * 010706cbbd12d506f23a52e8b71474ab2b06c2b6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132526261) * 588d625109e9ec53209d57a5187017c506134049 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class
flinkbot edited a comment on issue #9931: [FLINK-14441][Table-Common] Fix ValueLiteralExpression#getValueAs when valueClass is Period.class URL: https://github.com/apache/flink/pull/9931#issuecomment-543231644 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 5c0f625091d4c0f608a0d82c16b0d19daa8f997b (Mon Oct 21 03:26:55 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14441).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955718#comment-16955718 ] Jiayi Liao edited comment on FLINK-14470 at 10/21/19 3:22 AM: -- [~thw][~vthinkxie] Actually I think we should check FLINK-14147 first and see how much redundancy we can remove. For example, assuming that we can only list like 4,000 vertices' watermark at most, then we should make watermark board blank, otherwise the browse will become very slow even if we list them all. was (Author: wind_ljy): [~thw][~vthinkxie] Actually I think we should check FLINK-14147 first and see how much redundancy we can remove. For example, if we can only list like 4,000 vertices' watermark at most, then we should make watermark board blank, otherwise the browse will become very slow even if we list them all. > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 72827e1766dd28f66b72777fb8ad393cb1c0feef (Mon Oct 21 03:22:51 UTC 2019) **Warnings:** * **3 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-9953) Active Kubernetes integration
[
https://issues.apache.org/jira/browse/FLINK-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955721#comment-16955721
]
Yang Wang commented on FLINK-9953:
--
[~felixzheng]
I do not insist to introduce the `kubernetes-job.sh`. If we could make the
`flink run` more flexible, it is reasonable for me.
1. No jar is specified
In this way, before the jm/tm pod, the jar has already in the image. The job
graph is generated in the jm pod. Since the job is submitted on the jm side, we
could only support detach mode.
{code:java}
./bin/flink run -d -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s-windowjoin:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
-kuser-jar /path/in/the/image/WinowJoin.jar
{code}
2. Client local jar is specified
We need to deploy a session cluster to simulate per job mode. Since the jm need
to start first, and then Flink client submit the user jar and job graph to jm
pod through rest. Detach and attach mode are both supported.
{code:java}
./bin/flink run [-d] -m kubernetes-cluster -kid flink-perjob-example-2 \
-ki flink-k8s:latest \
-kjob-class org.apache.flink.streaming.examples.join.WindowJoin \
/path/in/the/client/WinowJoin.jar
{code}
The primary reason to introduce `kubernetes-job.sh` is for #1. And `flink run`
is always going to #2.
[~trohrmann], waiting for your response.
> Active Kubernetes integration
> -
>
> Key: FLINK-9953
> URL: https://issues.apache.org/jira/browse/FLINK-9953
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
>Reporter: Till Rohrmann
>Assignee: Yang Wang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue tracking Flink's active Kubernetes integration.
> Active means in this context that the {{ResourceManager}} can talk to
> Kubernetes to launch new pods similar to Flink's Yarn and Mesos integration.
> Phase1 implementation will have complete functions to make flink running on
> kubernetes. Phrase2 is mainly focused on production optimization, including
> k8s native high-availability, storage, network, log collector and etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1097b5e17cca1f18f46f6ddacc7232b02846911d (Mon Oct 21 03:19:48 UTC 2019) **Warnings:** * **3 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955720#comment-16955720 ] Yadong Xie commented on FLINK-14470: [~wind_ljy] We could do something like that, but I think the 4000 limited is a magic number, which may not a very good idea in system design. There are also many REST API meet the same problem, like the subtask list which also will almost sure to break when the parallelism is too high. > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9
flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9 URL: https://github.com/apache/flink/pull/9943#issuecomment-544224545 ## CI report: * e816681d90c2070b6136ddaca1ce11e4d32a21fd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132692257) * 6b765d22f10e7e2d5a37fd05271c9c098721d877 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132743723) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955718#comment-16955718 ] Jiayi Liao commented on FLINK-14470: [~thw][~vthinkxie] Actually I think we should check FLINK-14147 first and see how much redundancy we can remove. For example, if we can only list like 4,000 vertices' watermark at most, then we should make watermark board blank, otherwise the browse will become very slow even if we list them all. > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9907: [FLINK-14202][table][python] Optimize the execution plan for Python Calc when there is a condition
flinkbot edited a comment on issue #9907: [FLINK-14202][table][python] Optimize the execution plan for Python Calc when there is a condition URL: https://github.com/apache/flink/pull/9907#issuecomment-542243031 ## CI report: * dd14191ee919be31148be254070e7a777cf9cb4d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/131984556) * 1559637102832603a0dc0d09ab730e00f2e9d224 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/132628239) * cebed5831381b7bc00f3a56244f8123099f0b5bb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132741010) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955716#comment-16955716 ] Yadong Xie commented on FLINK-14470: Hi [~thw] Do you mean the blank place when watermarks are gone? A screenshot is better if If you are convenient > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955715#comment-16955715 ] Thomas Weise commented on FLINK-14470: -- I'm actually not sure hiding is desirable. As it stands the user will at least notice there is something wrong with the UI whereas before the an issue with the job was the first assumption. > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex
flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex URL: https://github.com/apache/flink/pull/9601#issuecomment-527401073 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit a8d4af6f4318992e43bdeeebf7aec54fcb60d667 (Mon Oct 21 03:00:29 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14147) Reduce REST API Request/Response redundancy
[ https://issues.apache.org/jira/browse/FLINK-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955713#comment-16955713 ] Jiayi Liao commented on FLINK-14147: This should be a great improvement. Curretly we suffer the slow reponding of Flink UI a lot when running high-parallelism jobs. > Reduce REST API Request/Response redundancy > > > Key: FLINK-14147 > URL: https://issues.apache.org/jira/browse/FLINK-14147 > Project: Flink > Issue Type: Improvement > Components: Runtime / Web Frontend >Reporter: Yadong Xie >Priority: Major > Attachments: 16_17_07__09_20_2019.jpg > > > 1. Redundancy Response > In the response of API /jobs/:jobid, the id and name in both plan and the > vertices data are exactly the same, it would waste a lot of network bandwidth > if the vertex graph is very big(1000+ vertex in a job). > !16_17_07__09_20_2019.jpg|width=427,height=279! > 2. Redundancy Request > In the current Web UI design, we have to query vertex number times to display > the low watermarks in the job graph. If the vertex number is very large(1000+ > sometimes), the Web UI will send 1000+ request to the REST API, the max > concurrent HTTP request in a browser is limited, it would bring a long time > delay for users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] jinglining commented on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex
jinglining commented on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex URL: https://github.com/apache/flink/pull/9601#issuecomment-544329926 > Could we not update the `host` column to include the port (like on the taskmanagers page) and link to the TaskManager page? Wouldn't this effectively work like the current iteration, just without an additional column? Like this:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] jinglining removed a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex
jinglining removed a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex URL: https://github.com/apache/flink/pull/9601#issuecomment-544320185 Like this:  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9805: [FLINK-11635][docs-zh] translate dev/stream/state/checkpointing into Chinese
flinkbot edited a comment on issue #9805: [FLINK-11635][docs-zh] translate dev/stream/state/checkpointing into Chinese URL: https://github.com/apache/flink/pull/9805#issuecomment-536292278 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ec65529633886e97035e0cf638d3b21d4b6fffa8 (Mon Oct 21 02:54:23 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9805: [FLINK-11635][docs-zh] translate dev/stream/state/checkpointing into Chinese
wuchong commented on issue #9805: [FLINK-11635][docs-zh] translate dev/stream/state/checkpointing into Chinese URL: https://github.com/apache/flink/pull/9805#issuecomment-544328977 Hi @Henvealf do you have time to update this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955710#comment-16955710 ] Yadong Xie commented on FLINK-14470: Got it, I will hide the error message in the web as a temp solution > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu reassigned FLINK-14470: --- Assignee: Yadong Xie > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955708#comment-16955708 ] Jark Wu commented on FLINK-14470: - Hi [~vthinkxie], I assigned this issue to you. > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Assignee: Yadong Xie >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9
flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9 URL: https://github.com/apache/flink/pull/9943#issuecomment-544224545 ## CI report: * e816681d90c2070b6136ddaca1ce11e4d32a21fd : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132692257) * 6b765d22f10e7e2d5a37fd05271c9c098721d877 : UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14471) Hide error message when metric api failed
Yadong Xie created FLINK-14471: -- Summary: Hide error message when metric api failed Key: FLINK-14471 URL: https://issues.apache.org/jira/browse/FLINK-14471 Project: Flink Issue Type: Sub-task Components: Runtime / Web Frontend Affects Versions: 1.9.1 Reporter: Yadong Xie Fix For: 1.10.0 The error message should hide when metric api failed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test
flinkbot edited a comment on issue #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#issuecomment-542986145 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit f8ded75e3aa2efbc305d52fabb3a71d71ad35080 (Mon Oct 21 02:49:17 UTC 2019) **Warnings:** * **3 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zjuwangg commented on a change in pull request #9919: [FLINK-13303][hive]add hive e2e connector test
zjuwangg commented on a change in pull request #9919: [FLINK-13303][hive]add hive e2e connector test URL: https://github.com/apache/flink/pull/9919#discussion_r336822275 ## File path: flink-end-to-end-tests/test-scripts/docker-hive-hadoop-cluster/config/hive-site.xml ## @@ -0,0 +1,54 @@ + + + + +javax.jdo.option.ConnectionURL + jdbc:mysql://mysql:3306/metastore?createDatabaseIfNotExist=true&useSSL=false Review comment: Since we have configured the MySQL service hostname 'mysql' in docker-compose.yaml. It's also ok to use 'mysql.docker-hadoop-cluster-network' as the hostname. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex
flinkbot edited a comment on issue #9601: [FLINK-13894][web]Web Ui add log url for subtask of vertex URL: https://github.com/apache/flink/pull/9601#issuecomment-527404637 ## CI report: * 13d895349390698404a375fb4362a41b736ab0c6 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125577621) * 5a1866be2d9765b0e941a531d5e9dda8616fecf4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/125842787) * 2db52501630bae1806c93573eb3c88b091b5c951 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126002450) * 8a9c4066216e3bf81a258bc8a8c15c33e67bf1d4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126009476) * 2cbd3756b9d00c9b42d494e487d4950c8bb280e0 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/129579795) * b2b94e7516dda963422234b5c19eb09baa278480 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132299831) * fd0cf151970941ad425bfbe749b97852d154efbe : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/132499327) * a8d4af6f4318992e43bdeeebf7aec54fcb60d667 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/132742145) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14429) Wrong app final status when running batch job on yarn with non-detached mode
[
https://issues.apache.org/jira/browse/FLINK-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955702#comment-16955702
]
liupengcheng commented on FLINK-14429:
--
Hi, [~trohrmann], it will throw exceptions when job execution failed currently,
so just need to tell if the exception is thrown, we can know the status. I've
put an PR for this, you can check more detail in it.
I'm also glad to hear more about the reworking of this client component, can
you give more information. [~aljoscha]
> Wrong app final status when running batch job on yarn with non-detached mode
>
>
> Key: FLINK-14429
> URL: https://issues.apache.org/jira/browse/FLINK-14429
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
>Affects Versions: 1.9.0
>Reporter: liupengcheng
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2019-10-17-16-47-47-038.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Recently, we found that the app final status is not correct when an
> application failed when running batch job on yarn with non-detached mode, It
> reported SUCCEEDED but FAILED is what we expected.
> !image-2019-10-17-16-47-47-038.png!
>
> But the logs and client reported error and job failed(It's caused by OOM):
> {code:java}
> 2019-10-10 14:36:21,797 INFO
> org.apache.flink.runtime.executiongraph.ExecutionGraph- Job TeraSort
> (d82cbfaae905c695597083b1476e51b8) switched from state FAILING to FAILED.
> org.apache.flink.runtime.io.network.partition.consumer.PartitionConnectionException:
> Connection for partition
> a254412fc7464cd4e0fe04ab9e3a6309@8d5afff58c86dd7f5bc78946f0101699 not
> reachable.
> at
> org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.requestSubpartition(RemoteInputChannel.java:168)
> at
> org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.requestPartitions(SingleInputGate.java:237)
> at
> org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.setup(SingleInputGate.java:215)
> at
> org.apache.flink.runtime.taskmanager.InputGateWithMetrics.setup(InputGateWithMetrics.java:65)
> at
> org.apache.flink.runtime.taskmanager.Task.setupPartitionsAndGates(Task.java:866)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:621)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Connecting the channel failed: Connecting to
> remote task manager + 'zjy-hadoop-prc-st164.bj/10.152.47.8:45704' has failed.
> This might indicate that the remote task manager has been lost.
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory$ConnectingChannel.waitForChannel(PartitionRequestClientFactory.java:197)
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory$ConnectingChannel.access$000(PartitionRequestClientFactory.java:134)
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory.createPartitionRequestClient(PartitionRequestClientFactory.java:86)
> at
> org.apache.flink.runtime.io.network.netty.NettyConnectionManager.createPartitionRequestClient(NettyConnectionManager.java:68)
> at
> org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.requestSubpartition(RemoteInputChannel.java:165)
> ... 7 more
> Caused by:
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
> Connecting to remote task manager +
> 'zjy-hadoop-prc-st164.bj/10.152.47.8:45704' has failed. This might indicate
> that the remote task manager has been lost.
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory$ConnectingChannel.operationComplete(PartitionRequestClientFactory.java:220)
> at
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactory$ConnectingChannel.operationComplete(PartitionRequestClientFactory.java:134)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNi
[jira] [Commented] (FLINK-14470) Watermark display not working with high parallelism job
[ https://issues.apache.org/jira/browse/FLINK-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955700#comment-16955700 ] Thomas Weise commented on FLINK-14470: -- Related threads: [https://lists.apache.org/thread.html/c96d86ae47050ae8fbe1e046794323a1bc1089090c6884d531116635@%3Cdev.flink.apache.org%3E] [https://lists.apache.org/thread.html/6205f54fe2c393475f9b4d754844a74160e37abaeeef826847a834d9@%3Cdev.flink.apache.org%3E] > Watermark display not working with high parallelism job > --- > > Key: FLINK-14470 > URL: https://issues.apache.org/jira/browse/FLINK-14470 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.8.2, 1.9.1 >Reporter: Thomas Weise >Priority: Major > > Watermarks don't display in the UI when the job graph has many vertices. The > REST API call to fetch currentInputWatermark fails as it enumerates each > subtask, which obviously fails beyond a limit. With the new UI that can be > noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14470) Watermark display not working with high parallelism job
Thomas Weise created FLINK-14470: Summary: Watermark display not working with high parallelism job Key: FLINK-14470 URL: https://issues.apache.org/jira/browse/FLINK-14470 Project: Flink Issue Type: Bug Components: Runtime / Web Frontend Affects Versions: 1.9.1, 1.8.2 Reporter: Thomas Weise Watermarks don't display in the UI when the job graph has many vertices. The REST API call to fetch currentInputWatermark fails as it enumerates each subtask, which obviously fails beyond a limit. With the new UI that can be noticed through a flickering error message in the upper right corner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] buptljy commented on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9
buptljy commented on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9 URL: https://github.com/apache/flink/pull/9943#issuecomment-544325027 @yanghua Thanks for your reminding. It's fixed now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9
flinkbot edited a comment on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9 URL: https://github.com/apache/flink/pull/9943#issuecomment-544223382 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6b765d22f10e7e2d5a37fd05271c9c098721d877 (Mon Oct 21 02:31:00 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-14463).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] buptljy commented on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9
buptljy commented on issue #9943: [FLINK-14463] [Kafka Consumer] Merge Handover in flink-connector-kafka and flink-connector-kafka-0.9 URL: https://github.com/apache/flink/pull/9943#issuecomment-544325053 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14442) Add time based interval execution to JDBC connectors.
[
https://issues.apache.org/jira/browse/FLINK-14442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955698#comment-16955698
]
Jark Wu commented on FLINK-14442:
-
Hi [~javadevmtl], yes, it will flush no matter the batch size or interval
triggers first.
This is supported from 1.9.0, see {{connector.write.flush.max-rows}} and
{{connector.write.flush.interval}} in the 1.9 docs:
https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/connect.html#jdbc-connector.
Sorry I gave a snapshot version docuementation before.
> Add time based interval execution to JDBC connectors.
> -
>
> Key: FLINK-14442
> URL: https://issues.apache.org/jira/browse/FLINK-14442
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / JDBC
>Affects Versions: 1.8.0, 1.8.1, 1.8.2, 1.9.0
>Reporter: None none
>Priority: Minor
>
> Hi, currently the JDBC sink/output only supports batch interval execution.
> For data to be streamed/committed to the JDBC database we need to wait for
> the batch interval to be filled up.
> For example if you set a batch interval of 100 but only get 99 records then
> no data will be committed to the database.
> The JDBC driver should maybe also have a time based interval so that data is
> eventually pushed to the database.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14443) Enable checkpointing for JDBC sink/output
[
https://issues.apache.org/jira/browse/FLINK-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955695#comment-16955695
]
Jark Wu commented on FLINK-14443:
-
Hi [~javadevmtl], I think you are right.
Btw, JDBC received some updates in 1.9, for example,
{{JDBCUpsertOutputFormat}}, {{JDBCUpsertSinkFunction}},
{{JDBCUpsertTableSink}} for writing records to JDBC using update or insert
statements.
> Enable checkpointing for JDBC sink/output
> -
>
> Key: FLINK-14443
> URL: https://issues.apache.org/jira/browse/FLINK-14443
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / JDBC
>Affects Versions: 1.8.0, 1.8.1, 1.8.2, 1.9.0
>Reporter: None none
>Priority: Minor
>
> Currently the JDBC sink/output does not support checkpointing. It would be
> advantageous to enable checkpointing on JDBC sinks for Dbs that have
> transactional support.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-14469) Drop Python 2 support
[ https://issues.apache.org/jira/browse/FLINK-14469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu updated FLINK-14469: Description: As discussed in the [ML|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html], the aim of this Jira is to drop the Python 2 support. This includes the following work: 1. Remove Python 2.7 CI tests 2. Throw an exception if Python below 3.5 is used 3. Clean up the code which is specific for Python 2.x was: As discussed in the [ML|[http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html]], the aim of this Jira is to drop the Python 2 support. This includes the following work: 1. Remove Python 2.7 CI tests 2. Throw an exception if Python below 3.5 is used 3. Clean up the code which is specific for Python 2.x > Drop Python 2 support > - > > Key: FLINK-14469 > URL: https://issues.apache.org/jira/browse/FLINK-14469 > Project: Flink > Issue Type: Task > Components: API / Python >Affects Versions: 1.10.0 >Reporter: Dian Fu >Priority: Major > Fix For: 1.10.0 > > > As discussed in the > [ML|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html], > the aim of this Jira is to drop the Python 2 support. This includes the > following work: > 1. Remove Python 2.7 CI tests > 2. Throw an exception if Python below 3.5 is used > 3. Clean up the code which is specific for Python 2.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-14469) Drop Python 2 support
Dian Fu created FLINK-14469: --- Summary: Drop Python 2 support Key: FLINK-14469 URL: https://issues.apache.org/jira/browse/FLINK-14469 Project: Flink Issue Type: Task Components: API / Python Affects Versions: 1.10.0 Reporter: Dian Fu Fix For: 1.10.0 As discussed in the [ML|[http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Drop-Python-2-support-for-1-10-tt33962.html]], the aim of this Jira is to drop the Python 2 support. This includes the following work: 1. Remove Python 2.7 CI tests 2. Throw an exception if Python below 3.5 is used 3. Clean up the code which is specific for Python 2.x -- This message was sent by Atlassian Jira (v8.3.4#803005)
