[GitHub] [flink] flinkbot commented on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock
flinkbot commented on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock URL: https://github.com/apache/flink/pull/10868#issuecomment-575029714 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6738a937a595abc0af8e89fe39b29d28bc96d44a (Thu Jan 16 07:58:21 UTC 2020) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] klion26 edited a comment on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock
klion26 edited a comment on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock URL: https://github.com/apache/flink/pull/10868#issuecomment-575028797 cc @carp84 @wuchong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] klion26 commented on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock
klion26 commented on issue #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock URL: https://github.com/apache/flink/pull/10868#issuecomment-575028797 cc @carp84 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] klion26 opened a new pull request #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock
klion26 opened a new pull request #10868: [FLINK-15278][docs-zh] Update Chinese version StreamingFileSink dock URL: https://github.com/apache/flink/pull/10868 ## What is the purpose of the change Sync the Chinese version doc of `StreamingFileSink` ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (yno) - The runtime per-record code paths (performance sensitive): ( no ) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no ) - The S3 file system connector: ( no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #10852: [FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate the original intended behavior
xintongsong commented on a change in pull request #10852: [FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate the original intended behavior URL: https://github.com/apache/flink/pull/10852#discussion_r367270089 ## File path: flink-yarn/src/test/java/org/apache/flink/yarn/YarnClusterDescriptorTest.java ## @@ -104,9 +104,7 @@ public void testFailIfTaskSlotsHigherThanMaxVcores() throws ClusterDeploymentExc clusterDescriptor.setLocalJarPath(new Path(flinkJar.getPath())); ClusterSpecification clusterSpecification = new ClusterSpecification.ClusterSpecificationBuilder() - .setMasterMemoryMB(1) - .setTaskManagerMemoryMB(1) - .setNumberTaskManagers(1) + .setTaskManagerMemoryMB(1024) Review comment: Making `ClusterSpecificationBuilder#taskManagerMemoryMB` default 1024 turns out works well. No test broken due to this change. https://travis-ci.org/xintongsong/flink/builds/637743056 I pushed a fixup commit to this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10822: [FLINK-15081][docs-zh]Translate "Concepts & Common API" page of Table…
flinkbot edited a comment on issue #10822: [FLINK-15081][docs-zh]Translate "Concepts & Common API" page of Table… URL: https://github.com/apache/flink/pull/10822#issuecomment-572897310 ## CI report: * 6a9eb6f6b70dc406a0c2285da4b02b17a18e4c7b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143846846) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4246) * 0951428e2212abfe79d25d03dd8441a0ecd374d8 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144680093) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4389) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-15611) KafkaITCase.testOneToOneSources fails on Travis
Yangze Guo created FLINK-15611:
--
Summary: KafkaITCase.testOneToOneSources fails on Travis
Key: FLINK-15611
URL: https://issues.apache.org/jira/browse/FLINK-15611
Project: Flink
Issue Type: Bug
Reporter: Yangze Guo
Fix For: 1.10.0
{{The test KafkaITCase.testOneToOneSources failed on Travis.}}
{code:java}
03:15:02,019 INFO
org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl -
Deleting topic scale-down-before-first-checkpoint
03:15:02,037 INFO
org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase -
Test
testScaleDownBeforeFirstCheckpoint(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase)
successfully run.
03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- -
03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- Shut down KafkaTestBase
03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- -
03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- -
03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- KafkaTestBase finished
03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
- -
03:15:25.731 [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time
elapsed: 245.845 s - in
org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase
03:15:26.099 [INFO]
03:15:26.099 [INFO] Results:
03:15:26.099 [INFO]
03:15:26.099 [ERROR] Failures:
03:15:26.099 [ERROR]
KafkaITCase.testOneToOneSources:97->KafkaConsumerTestBase.runOneToOneExactlyOnceTest:862
Test failed: Job execution failed.
{code}
https://api.travis-ci.com/v3/job/276124537/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-15608) State Processor api Write new state will get NullPointerException
[ https://issues.apache.org/jira/browse/FLINK-15608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HunterHunter closed FLINK-15608. > State Processor api Write new state will get NullPointerException > --- > > Key: FLINK-15608 > URL: https://issues.apache.org/jira/browse/FLINK-15608 > Project: Flink > Issue Type: Bug >Reporter: HunterHunter >Priority: Major > > When I execute a simple sample program. Sometimes report this mistake > (occasionally): > Assertion failed: (last_ref), function ~ColumnFamilySet, file > db/column_family.cc, line 1238. > And When I Write new State to a new SavepointPath I will get ( v1.9.0 is > fine): > Caused by: java.lang.NullPointerExceptionCaused by: > java.lang.NullPointerException at > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406) > at > org.apache.flink.state.api.output.BoundedOneInputStreamTaskRunner.mapPartition(BoundedOneInputStreamTaskRunner.java:76) > at > org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103) > at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504) at > org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369) at > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at > java.lang.Thread.run(Thread.java:748) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (FLINK-15608) State Processor api Write new state will get NullPointerException
[ https://issues.apache.org/jira/browse/FLINK-15608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HunterHunter resolved FLINK-15608. -- Resolution: Not A Bug flink-state-processor-api 1.9.0 are not compatible flink-runtime 1.9.1 > State Processor api Write new state will get NullPointerException > --- > > Key: FLINK-15608 > URL: https://issues.apache.org/jira/browse/FLINK-15608 > Project: Flink > Issue Type: Bug >Reporter: HunterHunter >Priority: Major > > When I execute a simple sample program. Sometimes report this mistake > (occasionally): > Assertion failed: (last_ref), function ~ColumnFamilySet, file > db/column_family.cc, line 1238. > And When I Write new State to a new SavepointPath I will get ( v1.9.0 is > fine): > Caused by: java.lang.NullPointerExceptionCaused by: > java.lang.NullPointerException at > org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406) > at > org.apache.flink.state.api.output.BoundedOneInputStreamTaskRunner.mapPartition(BoundedOneInputStreamTaskRunner.java:76) > at > org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103) > at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504) at > org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369) at > org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at > java.lang.Thread.run(Thread.java:748) > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-10308) remove docker folder
[
https://issues.apache.org/jira/browse/FLINK-10308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016481#comment-17016481
]
ptmagic edited comment on FLINK-10308 at 1/16/20 7:12 AM:
--
Please assign to me [~nicos.maris]
was (Author: ptmagic):
Please assign to me
> remove docker folder
>
>
> Key: FLINK-10308
> URL: https://issues.apache.org/jira/browse/FLINK-10308
> Project: Flink
> Issue Type: Task
> Components: Deployment / Docker, Documentation
>Reporter: Nicos Maris
>Priority: Minor
>
> At flink-web repo, running ./docker/run.sh returns the following error:
> {code:java}
> W: The repository 'http://security.ubuntu.com/ubuntu yakkety-security
> Release' does not have a Release file.
> {code}
> However, build.sh works fine. Is it worth fixing and maintaining the docker
> folder?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15176) Add '--job-classname' to flink-container 'job-cluster-job.yaml.template'
[ https://issues.apache.org/jira/browse/FLINK-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016478#comment-17016478 ] ptmagic edited comment on FLINK-15176 at 1/16/20 7:12 AM: -- Please assign to me，look like add some comment to job-cluster-job.yaml.template or readme can slove this [~ouyangwuli] was (Author: ptmagic): Please assign to me，look like add some comment to job-cluster-job.yaml.template or readme can slove this > Add '--job-classname' to flink-container 'job-cluster-job.yaml.template' > > > Key: FLINK-15176 > URL: https://issues.apache.org/jira/browse/FLINK-15176 > Project: Flink > Issue Type: Improvement > Components: Deployment / Docker >Affects Versions: 1.11.0 >Reporter: ouyangwulin >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > As from 'user...@flink.apache.org'. When use 'job-cluster-job.yaml.template' > deploy a job, the template don't have a good sense about how to use > '--job-classname' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-10140) Log files are not available via web UI in containerized environment
[
https://issues.apache.org/jira/browse/FLINK-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016495#comment-17016495
]
ptmagic edited comment on FLINK-10140 at 1/16/20 7:12 AM:
--
Start the job manager or task manager in the background, and then direct the
STDOUT to a log file? If so, please assign to me
[~trohrmann]
was (Author: ptmagic):
Start the job manager or task manager in the background, and then direct the
STDOUT to a log file? If so, please assign to me
> Log files are not available via web UI in containerized environment
> ---
>
> Key: FLINK-10140
> URL: https://issues.apache.org/jira/browse/FLINK-10140
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Docker, Deployment / Kubernetes
>Affects Versions: 1.5.2, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Priority: Major
>
> Since we start Flink components in the foreground (see
> {{flink-contrib/docker-flink/docker-entrypoint.sh}} and
> {{flink-container/docker/docker-entrypoint.sh}} we print the log statements
> to STDOUT and don't write them into a file. Consequently, the web UI cannot
> server the log files since they don't exist.
> A simple way to fix the problem is to also create a log file like in daemon
> mode
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15609) Add blink built-in functions from FlinkSqlOperatorTable to BuiltInFunctionDefinitions
[ https://issues.apache.org/jira/browse/FLINK-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016607#comment-17016607 ] Jingsong Lee commented on FLINK-15609: -- Hi [~jark] , these new functions can be used in SQL, just wrong resolution order. > Add blink built-in functions from FlinkSqlOperatorTable to > BuiltInFunctionDefinitions > - > > Key: FLINK-15609 > URL: https://issues.apache.org/jira/browse/FLINK-15609 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jingsong Lee >Priority: Major > Fix For: 1.10.0 > > > In FLINK-15595, CoreModule should contains all functions in > FlinkSqlOperatorTable. Otherwise, resolution Order is chaotic. I think it is > time to align blink built-in functions to BuiltInFunctionDefinitions. > Impact to legacy planner: user can not use the function name directly that he > define function with the same name of blink built-in function in catalog. I > think it is reasonable, since he will migrate his job to blink planner. > What do you think? [~twalthr] [~dwysakowicz] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-573276729 ## CI report: * 5eb1599945f1bee35c342f762cc25684013d2d83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143984465) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4264) * 73579bbc6b556fe42e55e21581c5994f84480843 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144033919) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4267) * e303d84258f1e63edd45867f6c4e89e181307ee5 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144299177) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4325) * 426112c59a86ec127040178efda1085231f1988f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144326556) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4334) * fe731e4a039fba606be83a9535841c157a048d58 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144458002) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4355) * 1278b4488cb8bc05addaf359458ed4bae6c1967b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144670693) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4386) * 3eba7e89df2c0b5ad6ba098aaabbab42be37df2f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144675097) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4388) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10822: [FLINK-15081][docs-zh]Translate "Concepts & Common API" page of Table…
flinkbot edited a comment on issue #10822: [FLINK-15081][docs-zh]Translate "Concepts & Common API" page of Table… URL: https://github.com/apache/flink/pull/10822#issuecomment-572897310 ## CI report: * 6a9eb6f6b70dc406a0c2285da4b02b17a18e4c7b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143846846) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4246) * 0951428e2212abfe79d25d03dd8441a0ecd374d8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] curcur edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
curcur edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-575012464 Thank you so much @zhuzhurk for reviewing the code and providing great suggestions and feedback! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] curcur commented on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
curcur commented on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-575012464 Thank you so much @zhuzhurk for reviewing the code and providing great suggestions and feedbacks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
KarmaGYZ edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-574945954 Thanks a lot to point out this @tillrohrmann . I update the PR. E2E tests passed https://travis-ci.org/KarmaGYZ/flink/builds/637769337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-15609) Add blink built-in functions from FlinkSqlOperatorTable to BuiltInFunctionDefinitions
[ https://issues.apache.org/jira/browse/FLINK-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016596#comment-17016596 ] Jark Wu edited comment on FLINK-15609 at 1/16/20 6:43 AM: -- Should we mark a higher priority for this issue? AFAIK, it lead to the new functions introduced by blink planner can't work (about 36 functions). You can search "blink planner" in this page: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/systemFunctions.html was (Author: jark): Should we mark a higher priority for this issue. AFAIK, it lead to the new functions introduced by blink planner can't work (about 36 functions). You can search "blink planner" in this page: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/systemFunctions.html > Add blink built-in functions from FlinkSqlOperatorTable to > BuiltInFunctionDefinitions > - > > Key: FLINK-15609 > URL: https://issues.apache.org/jira/browse/FLINK-15609 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jingsong Lee >Priority: Major > Fix For: 1.10.0 > > > In FLINK-15595, CoreModule should contains all functions in > FlinkSqlOperatorTable. Otherwise, resolution Order is chaotic. I think it is > time to align blink built-in functions to BuiltInFunctionDefinitions. > Impact to legacy planner: user can not use the function name directly that he > define function with the same name of blink built-in function in catalog. I > think it is reasonable, since he will migrate his job to blink planner. > What do you think? [~twalthr] [~dwysakowicz] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-15609) Add blink built-in functions from FlinkSqlOperatorTable to BuiltInFunctionDefinitions
[ https://issues.apache.org/jira/browse/FLINK-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016596#comment-17016596 ] Jark Wu commented on FLINK-15609: - Should we mark a higher priority for this issue. AFAIK, it lead to the new functions introduced by blink planner can't work (about 36 functions). You can search "blink planner" in this page: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/functions/systemFunctions.html > Add blink built-in functions from FlinkSqlOperatorTable to > BuiltInFunctionDefinitions > - > > Key: FLINK-15609 > URL: https://issues.apache.org/jira/browse/FLINK-15609 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jingsong Lee >Priority: Major > Fix For: 1.10.0 > > > In FLINK-15595, CoreModule should contains all functions in > FlinkSqlOperatorTable. Otherwise, resolution Order is chaotic. I think it is > time to align blink built-in functions to BuiltInFunctionDefinitions. > Impact to legacy planner: user can not use the function name directly that he > define function with the same name of blink built-in function in catalog. I > think it is reasonable, since he will migrate his job to blink planner. > What do you think? [~twalthr] [~dwysakowicz] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children
flinkbot edited a comment on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children URL: https://github.com/apache/flink/pull/10867#issuecomment-574982744 ## CI report: * 3d94fa8f5a675706dcb974b397452d165bcee987 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144670708) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4387) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-573276729 ## CI report: * 5eb1599945f1bee35c342f762cc25684013d2d83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143984465) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4264) * 73579bbc6b556fe42e55e21581c5994f84480843 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144033919) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4267) * e303d84258f1e63edd45867f6c4e89e181307ee5 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144299177) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4325) * 426112c59a86ec127040178efda1085231f1988f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144326556) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4334) * fe731e4a039fba606be83a9535841c157a048d58 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144458002) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4355) * 1278b4488cb8bc05addaf359458ed4bae6c1967b Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144670693) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4386) * 3eba7e89df2c0b5ad6ba098aaabbab42be37df2f Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144675097) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4388) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed
flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed URL: https://github.com/apache/flink/pull/10824#issuecomment-572926550 ## CI report: * 7a61978007a5a02723a561ee145d82e2413e51ae Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143855396) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4250) * e718b2d4d8b3f4c6f846a72ee0cfe28b7a0ce648 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144450734) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4351) * 3b07862f15aaddd2b6d2a0d21cceaba4a1f55f6e Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144670673) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4385) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-15610) How to achieve the udf that the number of return column is uncertain
[
https://issues.apache.org/jira/browse/FLINK-15610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hehuiyuan updated FLINK-15610:
--
Description:
For
example:[https://help.aliyun.com/knowledge_detail/98948.html?spm=a2c4g.11186631.2.3.21b81761QhpBte]
{code:java}
SELECT c1, c2
FROM T1, lateral table(MULTI_KEYVALUE(str, split1, split2, key1, key2))
as T(c1, c2)
SELECT c1, c2, c3
FROM T1, lateral table(MULTI_KEYVALUE(str, split1, split2, key1, key2, key3))
as T(c1, c2, c3)
{code}
For Tablefunction:
{code:java}
public TypeInformation getResultType() {
return Types.ROW(Types.STRING(),Types.STRING());
}
{code}
The retrun type of `getResultType` is `TypeInformation`,i want to achieve
the size of row is not fixed.
> How to achieve the udf that the number of return column is uncertain
> -
>
> Key: FLINK-15610
> URL: https://issues.apache.org/jira/browse/FLINK-15610
> Project: Flink
> Issue Type: Wish
> Components: Table SQL / API
>Reporter: hehuiyuan
>Priority: Major
>
> For
> example:[https://help.aliyun.com/knowledge_detail/98948.html?spm=a2c4g.11186631.2.3.21b81761QhpBte]
>
> {code:java}
> SELECT c1, c2
> FROM T1, lateral table(MULTI_KEYVALUE(str, split1, split2, key1, key2))
> as T(c1, c2)
> SELECT c1, c2, c3
> FROM T1, lateral table(MULTI_KEYVALUE(str, split1, split2, key1, key2, key3))
> as T(c1, c2, c3)
> {code}
> For Tablefunction:
> {code:java}
> public TypeInformation getResultType() {
> return Types.ROW(Types.STRING(),Types.STRING());
> }
> {code}
> The retrun type of `getResultType` is `TypeInformation`,i want to
> achieve the size of row is not fixed.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15610) How to achieve the udf that the number of return column is uncertain
hehuiyuan created FLINK-15610: - Summary: How to achieve the udf that the number of return column is uncertain Key: FLINK-15610 URL: https://issues.apache.org/jira/browse/FLINK-15610 Project: Flink Issue Type: Wish Components: Table SQL / API Reporter: hehuiyuan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) * 62f75bffd7ec4cafde8e9d1796d8f95248c40a83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144668655) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4384) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574997174 Hi @aljoscha Can you please review this PR? I have also updated the NOTICE file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler
flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler URL: https://github.com/apache/flink/pull/10858#issuecomment-574561465 ## CI report: * cf716ecc2f23aa89683be8096f16725b1f1f8d26 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144472558) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4359) * a0f5bacf182ee5fc8711e18d2c47edaffa8d1948 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144666495) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4383) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-573276729 ## CI report: * 5eb1599945f1bee35c342f762cc25684013d2d83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143984465) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4264) * 73579bbc6b556fe42e55e21581c5994f84480843 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144033919) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4267) * e303d84258f1e63edd45867f6c4e89e181307ee5 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144299177) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4325) * 426112c59a86ec127040178efda1085231f1988f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144326556) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4334) * fe731e4a039fba606be83a9535841c157a048d58 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144458002) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4355) * 1278b4488cb8bc05addaf359458ed4bae6c1967b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144670693) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4386) * 3eba7e89df2c0b5ad6ba098aaabbab42be37df2f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed
flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed URL: https://github.com/apache/flink/pull/10824#issuecomment-572926550 ## CI report: * 7a61978007a5a02723a561ee145d82e2413e51ae Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143855396) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4250) * e718b2d4d8b3f4c6f846a72ee0cfe28b7a0ce648 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144450734) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4351) * 3b07862f15aaddd2b6d2a0d21cceaba4a1f55f6e Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144670673) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4385) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) * 62f75bffd7ec4cafde8e9d1796d8f95248c40a83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144668655) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4384) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children
flinkbot edited a comment on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children URL: https://github.com/apache/flink/pull/10867#issuecomment-574982744 ## CI report: * 3d94fa8f5a675706dcb974b397452d165bcee987 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144670708) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4387) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed
flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed URL: https://github.com/apache/flink/pull/10824#issuecomment-572926550 ## CI report: * 7a61978007a5a02723a561ee145d82e2413e51ae Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143855396) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4250) * e718b2d4d8b3f4c6f846a72ee0cfe28b7a0ce648 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144450734) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4351) * 3b07862f15aaddd2b6d2a0d21cceaba4a1f55f6e Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144670673) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4385) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-573276729 ## CI report: * 5eb1599945f1bee35c342f762cc25684013d2d83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143984465) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4264) * 73579bbc6b556fe42e55e21581c5994f84480843 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144033919) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4267) * e303d84258f1e63edd45867f6c4e89e181307ee5 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144299177) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4325) * 426112c59a86ec127040178efda1085231f1988f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144326556) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4334) * fe731e4a039fba606be83a9535841c157a048d58 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144458002) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4355) * 1278b4488cb8bc05addaf359458ed4bae6c1967b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144670693) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4386) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) * 62f75bffd7ec4cafde8e9d1796d8f95248c40a83 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144668655) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4384) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children
flinkbot commented on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children URL: https://github.com/apache/flink/pull/10867#issuecomment-574982744 ## CI report: * 3d94fa8f5a675706dcb974b397452d165bcee987 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler
flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler URL: https://github.com/apache/flink/pull/10858#issuecomment-574561465 ## CI report: * cf716ecc2f23aa89683be8096f16725b1f1f8d26 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144472558) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4359) * a0f5bacf182ee5fc8711e18d2c47edaffa8d1948 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144666495) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4383) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
flinkbot edited a comment on issue #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions URL: https://github.com/apache/flink/pull/10832#issuecomment-573276729 ## CI report: * 5eb1599945f1bee35c342f762cc25684013d2d83 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143984465) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4264) * 73579bbc6b556fe42e55e21581c5994f84480843 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144033919) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4267) * e303d84258f1e63edd45867f6c4e89e181307ee5 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144299177) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4325) * 426112c59a86ec127040178efda1085231f1988f Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144326556) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4334) * fe731e4a039fba606be83a9535841c157a048d58 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144458002) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4355) * 1278b4488cb8bc05addaf359458ed4bae6c1967b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed
flinkbot edited a comment on issue #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed URL: https://github.com/apache/flink/pull/10824#issuecomment-572926550 ## CI report: * 7a61978007a5a02723a561ee145d82e2413e51ae Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143855396) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4250) * e718b2d4d8b3f4c6f846a72ee0cfe28b7a0ce648 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144450734) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4351) * 3b07862f15aaddd2b6d2a0d21cceaba4a1f55f6e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] klion26 commented on a change in pull request #10824: [FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if StopWithSavepoint failed
klion26 commented on a change in pull request #10824:
[FLINK-15152][checkpointing] Restatrt CheckpointCoordinator if
StopWithSavepoint failed
URL: https://github.com/apache/flink/pull/10824#discussion_r367226527
##
File path:
flink-tests/src/test/java/org/apache/flink/runtime/jobmaster/JobMasterStopWithSavepointIT.java
##
@@ -357,4 +391,36 @@ public void finishTask() throws Exception {
finishLatch.trigger();
}
}
+
+ /**
+* A {@link StreamTask} that simply calls {@link
CountDownLatch#countDown()} when
+* invoking {@link #triggerCheckpointAsync}.
+*/
+ public static class CheckpointCountingTask extends NoOpStreamTask {
Review comment:
Ok, will update it. previously did not change to this, because I though in
`CheckpointCountingTask` we do not need to distinguish whether this task is in
subtask 0 (and can tolerate some code dup in test), but distinguish whether
this task is subtask 0 seems has no side effect.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) * 62f75bffd7ec4cafde8e9d1796d8f95248c40a83 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children
flinkbot commented on issue #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children URL: https://github.com/apache/flink/pull/10867#issuecomment-574976242 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 3d94fa8f5a675706dcb974b397452d165bcee987 (Thu Jan 16 04:26:42 UTC 2020) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-15609) Add blink built-in functions from FlinkSqlOperatorTable to BuiltInFunctionDefinitions
Jingsong Lee created FLINK-15609: Summary: Add blink built-in functions from FlinkSqlOperatorTable to BuiltInFunctionDefinitions Key: FLINK-15609 URL: https://issues.apache.org/jira/browse/FLINK-15609 Project: Flink Issue Type: Sub-task Components: Table SQL / API Reporter: Jingsong Lee Fix For: 1.10.0 In FLINK-15595, CoreModule should contains all functions in FlinkSqlOperatorTable. Otherwise, resolution Order is chaotic. I think it is time to align blink built-in functions to BuiltInFunctionDefinitions. Impact to legacy planner: user can not use the function name directly that he define function with the same name of blink built-in function in catalog. I think it is reasonable, since he will migrate his job to blink planner. What do you think? [~twalthr] [~dwysakowicz] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-14701:
---
Labels: pull-request-available (was: )
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.9.2
>
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] zhuzhurk opened a new pull request #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children
zhuzhurk opened a new pull request #10867: [FLINK-14701][runtime] Fix MultiTaskSlot to not remove slots which are not its children URL: https://github.com/apache/flink/pull/10867 ## What is the purpose of the change If a SharedSlotOversubscribedException happens, the MultiTaskSlot will release some of its child SingleTaskSlot. The triggered releasing will trigger a re-allocation of the task slot right inside SingleTaskSlot#release(...). So that a previous allocation in SloSharingManager#allTaskSlots will be replaced by the new allocation because they share the same slotRequestId. However, the SingleTaskSlot#release(...) will then invoke MultiTaskSlot#releaseChild to release the previous allocation with the slotRequestId, which will unexpectedly remove the new allocation from the SloSharingManager. In this way, slot leak happens because the pending slot request is not tracked by the SloSharingManager and cannot be released when its payload terminates. Note that the it's not a problem in 1.10/master now since SharedSlotOversubscribedException is removed in FLINK-14314. However, it's still an issue in 1.9. However, the fix would still help in master to avoid similar issue to happen in the future. A test case testNoSlotLeakOnSharedSlotOversubscribedException which exhibits this issue can be found at https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14. ## Brief change log - *See the code change.* ## Verifying this change The change is verified on 1.9 with the test in https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574591161 ## CI report: * f4506ad0e9302a279940670fa90c305c240b3c12 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144485388) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4364) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler
flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler URL: https://github.com/apache/flink/pull/10858#issuecomment-574561465 ## CI report: * cf716ecc2f23aa89683be8096f16725b1f1f8d26 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144472558) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4359) * a0f5bacf182ee5fc8711e18d2c47edaffa8d1948 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144666495) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4383) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] curcur commented on a change in pull request #10832: [FLINK-14163][runtime]Enforce synchronous registration of Execution#producedPartitions
curcur commented on a change in pull request #10832:
[FLINK-14163][runtime]Enforce synchronous registration of
Execution#producedPartitions
URL: https://github.com/apache/flink/pull/10832#discussion_r367223282
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionTest.java
##
@@ -540,6 +553,46 @@ public void testSlotReleaseAtomicallyReleasesExecution()
throws Exception {
});
}
+ /**
+* Tests that producedPartitions are registered synchronously under an
asynchronous interface.
+*/
+ @Test(expected = IllegalStateException.class)
+ public void testSynchronousRegisterProducedPartitions() throws
Exception {
+ final ScheduledExecutorService executor = new
DirectScheduledExecutorService();
+ final ShuffleMaster shuffleMaster = new
TestingShuffleMaster();
+
+ final ExecutionJobVertex sourceJobVertex =
getExecutionVertex(new JobVertexID(), executor, EAGER, shuffleMaster);
Review comment:
synced up offline and resolved.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-14701:
Fix Version/s: (was: 1.11.0)
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
> Fix For: 1.9.2
>
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-14701:
Fix Version/s: 1.11.0
1.9.2
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
> Fix For: 1.9.2, 1.11.0
>
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu reopened FLINK-14701:
-
Assignee: Zhu Zhu
Reopen this issue since it's still a issue in 1.9 and we should fix it.
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2, 1.10.0
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-14701:
Comment: was deleted
(was: This issue does not exist anymore since SharedSlotOversubscribedException
is removed in FLINK-14314.)
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2, 1.10.0
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-15608) State Processor api Write new state will get NullPointerException
HunterHunter created FLINK-15608: Summary: State Processor api Write new state will get NullPointerException Key: FLINK-15608 URL: https://issues.apache.org/jira/browse/FLINK-15608 Project: Flink Issue Type: Bug Reporter: HunterHunter When I execute a simple sample program. Sometimes report this mistake (occasionally): Assertion failed: (last_ref), function ~ColumnFamilySet, file db/column_family.cc, line 1238. And When I Write new State to a new SavepointPath I will get ( v1.9.0 is fine): Caused by: java.lang.NullPointerExceptionCaused by: java.lang.NullPointerException at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:279) at org.apache.flink.streaming.runtime.tasks.StreamTask.run(StreamTask.java:301) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:406) at org.apache.flink.state.api.output.BoundedOneInputStreamTaskRunner.mapPartition(BoundedOneInputStreamTaskRunner.java:76) at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:504) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:369) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-14701:
Affects Version/s: (was: 1.10.0)
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Minor
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574591161 ## CI report: * f4506ad0e9302a279940670fa90c305c240b3c12 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144485388) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4364) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler
flinkbot edited a comment on issue #10858: [FLINK-15582][tests] Enable batch scheduling tests for both LegacyScheduler and DefaultScheduler URL: https://github.com/apache/flink/pull/10858#issuecomment-574561465 ## CI report: * cf716ecc2f23aa89683be8096f16725b1f1f8d26 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/144472558) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4359) * a0f5bacf182ee5fc8711e18d2c47edaffa8d1948 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49.
flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49. URL: https://github.com/apache/flink/pull/10860#issuecomment-574591065 ## CI report: * 5f9cd2224ef458e5068f3179aa5be025848f8723 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144485357) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4363) * f57ca417ecb8c6206a0f7b8617776d922ba9cd7d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144496504) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4366) * 3ac114c589d40aed1ecd0267174941e50a423fc2 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144659236) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4381) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15599) SQL client requires both legacy and blink planner to be on the classpath
[
https://issues.apache.org/jira/browse/FLINK-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016517#comment-17016517
]
Jingsong Lee commented on FLINK-15599:
--
We can port {{FunctionService}} to table-common. If no one follows up, you can
assign this to me.
> SQL client requires both legacy and blink planner to be on the classpath
>
>
> Key: FLINK-15599
> URL: https://issues.apache.org/jira/browse/FLINK-15599
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.10.0
>Reporter: Dawid Wysakowicz
>Priority: Critical
> Fix For: 1.10.0
>
>
> Sql client uses directly some of the internal classes of the legacy planner,
> thus it does not work with only the blink planner on the classpath.
> The internal class that's being used is
> {{org.apache.flink.table.functions.FunctionService}}
> This dependency was introduced in FLINK-13195
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-14701) Slot leaks if SharedSlotOversubscribedException happens
[
https://issues.apache.org/jira/browse/FLINK-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu closed FLINK-14701.
---
Fix Version/s: (was: 1.9.2)
(was: 1.10.0)
Resolution: Won't Fix
This issue does not exist anymore since SharedSlotOversubscribedException is
removed in FLINK-14314.
> Slot leaks if SharedSlotOversubscribedException happens
> ---
>
> Key: FLINK-14701
> URL: https://issues.apache.org/jira/browse/FLINK-14701
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.2, 1.10.0
>Reporter: Zhu Zhu
>Priority: Minor
>
> If a {{SharedSlotOversubscribedException}} happens, the {{MultiTaskSlot}}
> will release some of its child {{SingleTaskSlot}}. The triggered releasing
> will trigger a re-allocation of the task slot right inside
> {{SingleTaskSlot#release(...)}}. So that a previous allocation in
> {{SloSharingManager#allTaskSlots}} will be replaced by the new allocation
> because they share the same {{slotRequestId}}.
> However, the {{SingleTaskSlot#release(...)}} will then invoke
> {{MultiTaskSlot#releaseChild}} to release the previous allocation with the
> {{slotRequestId}}, which will unexpectedly remove the new allocation from the
> {{SloSharingManager}}.
> In this way, slot leak happens because the pending slot request is not
> tracked by the {{SloSharingManager}} and cannot be released when its payload
> terminates.
> A test case {{testNoSlotLeakOnSharedSlotOversubscribedException}} which
> exhibits this issue can be found in this
> [commit|https://github.com/zhuzhurk/flink/commit/9024e2e9eb4bd17f371896d6dbc745bc9e585e14].
> The slot leak blocks the TPC-DS queries on flink 1.10, see FLINK-14674.
> To solve it, I'd propose to strengthen the {{MultiTaskSlot#releaseChild}} to
> only remove its true child task slot from the {{SloSharingManager}}, i.e. add
> a check {{if (child == allTaskSlots.get(child.getSlotRequestId()))}} before
> invoking {{allTaskSlots.remove(child.getSlotRequestId())}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15602) Blink planner does not respect the precision when casting timestamp to varchar
[
https://issues.apache.org/jira/browse/FLINK-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016512#comment-17016512
]
Zhenghua Gao commented on FLINK-15602:
--
Hi [~dwysakowicz] [~tiwalter] I investigated the behavior of popular DBMS and
found: PostgreSQL DO NOT pad '0' and Oracle/MS SQL pad '0' (MYSQL would pad '0'
for TIMESTAMP type and not pad '0' for DATETIME type). And, hive/spark would
not pad '0' too.
What's your opinion about the padding behavior?
> Blink planner does not respect the precision when casting timestamp to varchar
> --
>
> Key: FLINK-15602
> URL: https://issues.apache.org/jira/browse/FLINK-15602
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: Dawid Wysakowicz
>Priority: Blocker
> Fix For: 1.10.0
>
>
> According to SQL 2011 Part 2 Section 6.13 General Rules 11) d)
> {quote}
> If SD is a datetime data type or an interval data type then let Y be the
> shortest character string that
> conforms to the definition of in Subclause 5.3, “”, and
> such that the interpreted value
> of Y is SV and the interpreted precision of Y is the precision of SD.
> {quote}
> That means:
> {code}
> select cast(cast(TO_TIMESTAMP('2014-07-02 06:14:00', '-MM-DD HH24:mm:SS')
> as TIMESTAMP(0)) as VARCHAR(256)) from ...;
> // should produce
> // 2014-07-02 06:14:00
> select cast(cast(TO_TIMESTAMP('2014-07-02 06:14:00', '-MM-DD HH24:mm:SS')
> as TIMESTAMP(3)) as VARCHAR(256)) from ...;
> // should produce
> // 2014-07-02 06:14:00.000
> select cast(cast(TO_TIMESTAMP('2014-07-02 06:14:00', '-MM-DD HH24:mm:SS')
> as TIMESTAMP(9)) as VARCHAR(256)) from ...;
> // should produce
> // 2014-07-02 06:14:00.0
> {code}
> One possible solution would be to propagate the precision in
> {{org.apache.flink.table.planner.codegen.calls.ScalarOperatorGens#localTimeToStringCode}}.
> If I am not mistaken this problem was introduced in [FLINK-14599]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] qiuxiafei commented on a change in pull request #9413: [FLINK-13676][ml] Add class of Vector to Columns mapper
qiuxiafei commented on a change in pull request #9413: [FLINK-13676][ml] Add class of Vector to Columns mapper URL: https://github.com/apache/flink/pull/9413#discussion_r367216115 ## File path: flink-ml-parent/flink-ml-lib/src/main/java/org/apache/flink/ml/common/dataproc/vector/VectorToColumnsMapper.java ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.flink.ml.common.dataproc.vector; Review comment: yes, but this is a pretty simple one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-15546) Obscure error message from ScalarOperatorGens::generateCast
[
https://issues.apache.org/jira/browse/FLINK-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016491#comment-17016491
]
Jingsong Lee edited comment on FLINK-15546 at 1/16/20 3:27 AM:
---
[~docete] Yes, do you want to fix this and FLINK-15584 ?
was (Author: lzljs3620320):
[~docete] Yes, do you want to fix this and FLIP-15584 ?
> Obscure error message from ScalarOperatorGens::generateCast
> ---
>
> Key: FLINK-15546
> URL: https://issues.apache.org/jira/browse/FLINK-15546
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Rui Li
>Priority: Minor
>
> Consider the following case:
> {noformat}
> Flink SQL> describe foo;
> root
> |-- x: ROW<`f1` DOUBLE, `f2` VARCHAR(10)>
> Flink SQL> insert into foo select row(1.1,'abc');
> [INFO] Submitting SQL update statement to the cluster...
> [ERROR] Could not execute SQL statement. Reason:
> org.apache.flink.table.planner.codegen.CodeGenException: Unsupported cast
> from 'ROW' to 'ROW'.
> {noformat}
> Users are unlikely to figure out what goes wrong from the above error
> message. Something like {{Unsupported cast from 'ROW'
> to 'ROW'}} will be more helpful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible
flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible URL: https://github.com/apache/flink/pull/10685#issuecomment-568872421 ## CI report: * 18d9268b1f2106fceaffe388ff9e4e1babdad452 Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3908) * d2686edf78b4d3afec05ca65486ff26304000aeb Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143353493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4150) * ce56c02026fdf5e8663b6bcd00e9a3bebbe38e65 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144549267) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4377) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144661716) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4382) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15607) throw exception when users trying to use Hive aggregate functions in streaming mode
[ https://issues.apache.org/jira/browse/FLINK-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016504#comment-17016504 ] Jingsong Lee commented on FLINK-15607: -- > add restart process into the test to verify the accumulator state is correct Good point, yes! > throw exception when users trying to use Hive aggregate functions in > streaming mode > --- > > Key: FLINK-15607 > URL: https://issues.apache.org/jira/browse/FLINK-15607 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Table SQL / API >Reporter: Bowen Li >Priority: Major > > Due to that Hive functions are all built for batch processing, aggregate > functions in Hive that implement UDAF and GenericUDAFResolver2 interfaces may > have unpredictable behaviors when used in streaming mode in Flink. We advice > users to only use Hive aggregate functions interfaces in batch mode. > > It will be better to throw exception when users trying to use Hive aggregate > functions in streaming mode. Seems need to distinguish execution mode in > FunctionCatalogOperatorTable, which is not achievable yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (FLINK-15582) Enable batch scheduling tests in LegacySchedulerBatchSchedulingTest for DefaultScheduler as well
[
https://issues.apache.org/jira/browse/FLINK-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-15582:
Comment: was deleted
(was: [~trohrmann] this is a pre-requisite for removing LegacyScheduler but I
think it is also a needed test for DefaultScheduler in 1.10.
To be detailed, I plan to extract most of the codes in
{{LegacySchedulerBatchSchedulingTest}} to a {{BatchSchedulingTestBase}} and
then let the implementations {{LegacySchedulerBatchSchedulingTest}} and
{{DefaultSchedulerBatchSchedulingTest}} implement the {{createScheduler(...)}}
method only. So later when removing the LegacyScheduler we can just remove
{{LegacySchedulerBatchSchedulingTest}} as well.
WDYT?)
> Enable batch scheduling tests in LegacySchedulerBatchSchedulingTest for
> DefaultScheduler as well
>
>
> Key: FLINK-15582
> URL: https://issues.apache.org/jira/browse/FLINK-15582
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0, 1.11.0
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{testSchedulingOfJobWithFewerSlotsThanParallelism}} is a common case but it
> is only tested with legacy scheduler in
> {{LegacySchedulerBatchSchedulingTest}} at the moment.
> We should enable it for DefaultScheduler as well.
> This also allows us to safely remove {{LegacySchedulerBatchSchedulingTest}}
> when we are removing the LegacyScheduler and related components without
> loosing test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-15582) Enable batch scheduling tests in LegacySchedulerBatchSchedulingTest for DefaultScheduler as well
[
https://issues.apache.org/jira/browse/FLINK-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-15582:
Fix Version/s: (was: 1.10.0)
1.11.0
> Enable batch scheduling tests in LegacySchedulerBatchSchedulingTest for
> DefaultScheduler as well
>
>
> Key: FLINK-15582
> URL: https://issues.apache.org/jira/browse/FLINK-15582
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.10.0, 1.11.0
>Reporter: Zhu Zhu
>Assignee: Zhu Zhu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{testSchedulingOfJobWithFewerSlotsThanParallelism}} is a common case but it
> is only tested with legacy scheduler in
> {{LegacySchedulerBatchSchedulingTest}} at the moment.
> We should enable it for DefaultScheduler as well.
> This also allows us to safely remove {{LegacySchedulerBatchSchedulingTest}}
> when we are removing the LegacyScheduler and related components without
> loosing test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-15307) Subclasses of FailoverStrategy are easily confused with implementation classes of RestartStrategy
[ https://issues.apache.org/jira/browse/FLINK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-15307. --- Resolution: Fixed Fixed via: master: 5984e7e363c87ebef4e6a9b501ca36d9d20fe798 > Subclasses of FailoverStrategy are easily confused with implementation > classes of RestartStrategy > - > > Key: FLINK-15307 > URL: https://issues.apache.org/jira/browse/FLINK-15307 > Project: Flink > Issue Type: Improvement > Components: Runtime / Configuration >Reporter: Andrew.D.lin >Assignee: Andrew.D.lin >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.0 > > Attachments: image-2019-12-18-14-59-03-181.png > > Time Spent: 20m > Remaining Estimate: 23h 50m > > Subclasses of RestartStrategy > * FailingRestartStrategy > * FailureRateRestartStrategy > * FixedDelayRestartStrategy > * InfiniteDelayRestartStrategy > Implementation class of FailoverStrategy > * AdaptedRestartPipelinedRegionStrategyNG > * RestartAllStrategy > * RestartIndividualStrategy > * RestartPipelinedRegionStrategy > > FailoverStrategy describes how the job computation recovers from task > failures. > I think the following names may be easier to understand and easier to > distinguish： > Implementation class of FailoverStrategy > * AdaptedPipelinedRegionFailoverStrategyNG > * FailoverAllStrategy > * FailoverIndividualStrategy > * FailoverPipelinedRegionStrategy > FailoverStrategy is currently generated by configuration. If we change the > name of the implementation class, it will not affect compatibility. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zhuzhurk closed pull request #10848: [FLINK-15307][failover]Rename Subclasses of FailoverStrategy
zhuzhurk closed pull request #10848: [FLINK-15307][failover]Rename Subclasses of FailoverStrategy URL: https://github.com/apache/flink/pull/10848 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10140) Log files are not available via web UI in containerized environment
[
https://issues.apache.org/jira/browse/FLINK-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016495#comment-17016495
]
ptmagic commented on FLINK-10140:
-
Start the job manager or task manager in the background, and then direct the
STDOUT to a log file? If so, please assign to me
> Log files are not available via web UI in containerized environment
> ---
>
> Key: FLINK-10140
> URL: https://issues.apache.org/jira/browse/FLINK-10140
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Docker, Deployment / Kubernetes
>Affects Versions: 1.5.2, 1.6.0, 1.7.0
>Reporter: Till Rohrmann
>Priority: Major
>
> Since we start Flink components in the foreground (see
> {{flink-contrib/docker-flink/docker-entrypoint.sh}} and
> {{flink-container/docker/docker-entrypoint.sh}} we print the log statements
> to STDOUT and don't write them into a file. Consequently, the web UI cannot
> server the log files since they don't exist.
> A simple way to fix the problem is to also create a log file like in daemon
> mode
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
flinkbot edited a comment on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574591161 ## CI report: * f4506ad0e9302a279940670fa90c305c240b3c12 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144485388) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4364) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on a change in pull request #10852: [FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate the original intended behavior
xintongsong commented on a change in pull request #10852:
[FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate
the original intended behavior
URL: https://github.com/apache/flink/pull/10852#discussion_r367211936
##
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##
@@ -318,6 +308,20 @@ private void isReadyForDeployment(ClusterSpecification
clusterSpecification) thr
}
}
+ @VisibleForTesting
+ protected int getNumYarnMaxVcores() throws YarnDeploymentException {
Review comment:
I'm not sure about this.
`YarnClient` is an abstract class, and to introduce a `StubYarnClientImpl`
we would need to also implement more than 20 useless methods and mock the
`NodeReport` as well. The complication seems unnecessary to me.
It is true the current approach touches the production class
`YarnClusterDescriptor`, but only with a trivial refactor. IMO, even without
this testability issue, it is not a bad thing to extract the logic getting max
vcores from Yarn into a separate method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-15607) throw exception when users trying to use Hive aggregate functions in streaming mode
[ https://issues.apache.org/jira/browse/FLINK-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016492#comment-17016492 ] Jark Wu commented on FLINK-15607: - [~lzljs3620320] Thanks for pointing it out, however, I think we should add restart process into the test to verify the accumulator state is correct. > throw exception when users trying to use Hive aggregate functions in > streaming mode > --- > > Key: FLINK-15607 > URL: https://issues.apache.org/jira/browse/FLINK-15607 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Table SQL / API >Reporter: Bowen Li >Priority: Major > > Due to that Hive functions are all built for batch processing, aggregate > functions in Hive that implement UDAF and GenericUDAFResolver2 interfaces may > have unpredictable behaviors when used in streaming mode in Flink. We advice > users to only use Hive aggregate functions interfaces in batch mode. > > It will be better to throw exception when users trying to use Hive aggregate > functions in streaming mode. Seems need to distinguish execution mode in > FunctionCatalogOperatorTable, which is not achievable yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-15546) Obscure error message from ScalarOperatorGens::generateCast
[
https://issues.apache.org/jira/browse/FLINK-15546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016491#comment-17016491
]
Jingsong Lee commented on FLINK-15546:
--
[~docete] Yes, do you want to fix this and FLIP-15584 ?
> Obscure error message from ScalarOperatorGens::generateCast
> ---
>
> Key: FLINK-15546
> URL: https://issues.apache.org/jira/browse/FLINK-15546
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
>Reporter: Rui Li
>Priority: Minor
>
> Consider the following case:
> {noformat}
> Flink SQL> describe foo;
> root
> |-- x: ROW<`f1` DOUBLE, `f2` VARCHAR(10)>
> Flink SQL> insert into foo select row(1.1,'abc');
> [INFO] Submitting SQL update statement to the cluster...
> [ERROR] Could not execute SQL statement. Reason:
> org.apache.flink.table.planner.codegen.CodeGenException: Unsupported cast
> from 'ROW' to 'ROW'.
> {noformat}
> Users are unlikely to figure out what goes wrong from the above error
> message. Something like {{Unsupported cast from 'ROW'
> to 'ROW'}} will be more helpful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] xintongsong commented on a change in pull request #10852: [FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate the original intended behavior
xintongsong commented on a change in pull request #10852: [FLINK-15564][yarn][test] Fix YarnClusterDescriptorTest that failed to validate the original intended behavior URL: https://github.com/apache/flink/pull/10852#discussion_r367209409 ## File path: flink-yarn/src/test/java/org/apache/flink/yarn/YarnClusterDescriptorTest.java ## @@ -104,9 +104,7 @@ public void testFailIfTaskSlotsHigherThanMaxVcores() throws ClusterDeploymentExc clusterDescriptor.setLocalJarPath(new Path(flinkJar.getPath())); ClusterSpecification clusterSpecification = new ClusterSpecification.ClusterSpecificationBuilder() - .setMasterMemoryMB(1) - .setTaskManagerMemoryMB(1) - .setNumberTaskManagers(1) + .setTaskManagerMemoryMB(1024) Review comment: I'll give it a try to change the default to 1024, see if it breaks anything. I'm actually thinking about remove `ClusterSpecification` entirely. I've visited all the usages of this class, and it seems to me all its information can be fetched directly from configuration. However, I'd like to scope this out from this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible
flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible URL: https://github.com/apache/flink/pull/10685#issuecomment-568872421 ## CI report: * 18d9268b1f2106fceaffe388ff9e4e1babdad452 Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3908) * d2686edf78b4d3afec05ca65486ff26304000aeb Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143353493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4150) * ce56c02026fdf5e8663b6bcd00e9a3bebbe38e65 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144549267) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4377) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15603) Show "barrier lag" in checkpoint statistics
[
https://issues.apache.org/jira/browse/FLINK-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016484#comment-17016484
]
Congxian Qiu(klion26) commented on FLINK-15603:
---
And we may add some debug log in {{CheckpointBarrierAligner}} when received a
barrier and when barrier align complete such as {{CheckpointBarrierTracker}},
this can be helpful when debugging checkpoint problem.
> Show "barrier lag" in checkpoint statistics
> ---
>
> Key: FLINK-15603
> URL: https://issues.apache.org/jira/browse/FLINK-15603
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Web Frontend
>Reporter: Stephan Ewen
>Priority: Critical
> Labels: usability
> Fix For: 1.11.0
>
>
> One of the most important metrics is missing in the checkpoint stats:
> "barrier lag", meaning the time it between when the checkpoint was triggered
> and when the barriers arrive at a task.
> That time is critical to identify if a checkpoint takes too long because of
> backpressure or other contention.
> You can implicitly calculate this by "end_to_end_time - sync_time -
> async_time", but it is much more obvious for users that something is up when
> this number is explicitly shown.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
flinkbot edited a comment on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-568630874 ## CI report: * 1e6a533950438cf87afae1af481e227f5671688b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/142179966) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3867) * e503911f6872b66cdce77f01bb704038ffca30f4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10308) remove docker folder
[
https://issues.apache.org/jira/browse/FLINK-10308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016481#comment-17016481
]
ptmagic commented on FLINK-10308:
-
Please assign to me
> remove docker folder
>
>
> Key: FLINK-10308
> URL: https://issues.apache.org/jira/browse/FLINK-10308
> Project: Flink
> Issue Type: Task
> Components: Deployment / Docker, Documentation
>Reporter: Nicos Maris
>Priority: Minor
>
> At flink-web repo, running ./docker/run.sh returns the following error:
> {code:java}
> W: The repository 'http://security.ubuntu.com/ubuntu yakkety-security
> Release' does not have a Release file.
> {code}
> However, build.sh works fine. Is it worth fixing and maintaining the docker
> folder?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15176) Add '--job-classname' to flink-container 'job-cluster-job.yaml.template'
[ https://issues.apache.org/jira/browse/FLINK-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016478#comment-17016478 ] ptmagic commented on FLINK-15176: - Please assign to me，look like add some comment to job-cluster-job.yaml.template or readme can slove this > Add '--job-classname' to flink-container 'job-cluster-job.yaml.template' > > > Key: FLINK-15176 > URL: https://issues.apache.org/jira/browse/FLINK-15176 > Project: Flink > Issue Type: Improvement > Components: Deployment / Docker >Affects Versions: 1.11.0 >Reporter: ouyangwulin >Priority: Minor > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > As from 'user...@flink.apache.org'. When use 'job-cluster-job.yaml.template' > deploy a job, the template don't have a good sense about how to use > '--job-classname' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] JingsongLi commented on issue #10857: [FLINK-15593][doc] add doc to remind users not using Hive aggregate f…
JingsongLi commented on issue #10857: [FLINK-15593][doc] add doc to remind users not using Hive aggregate f… URL: https://github.com/apache/flink/pull/10857#issuecomment-574957473 > @JingsongLi This PR is for FLINK-15592 - Batch mode not work too. - ROW_NUMBER is the function related to internal operator, can not be override. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zjffdu commented on issue #10857: [FLINK-15593][doc] add doc to remind users not using Hive aggregate f…
zjffdu commented on issue #10857: [FLINK-15593][doc] add doc to remind users not using Hive aggregate f… URL: https://github.com/apache/flink/pull/10857#issuecomment-574956435 @JingsongLi This PR is for FLINK-15592 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49.
flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49. URL: https://github.com/apache/flink/pull/10860#issuecomment-574591065 ## CI report: * 5f9cd2224ef458e5068f3179aa5be025848f8723 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144485357) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4363) * f57ca417ecb8c6206a0f7b8617776d922ba9cd7d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144496504) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4366) * 3ac114c589d40aed1ecd0267174941e50a423fc2 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144659236) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4381) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574954559 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector
yanghua commented on issue #10861: [FLINK-15558][Connector] Bump Elasticsearch version from 7.3.2 to 7.5.1 for es7 connector URL: https://github.com/apache/flink/pull/10861#issuecomment-574954527 Relevant test error. rebuild now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15247) Closing (Testing)MiniCluster may cause ConcurrentModificationException
[
https://issues.apache.org/jira/browse/FLINK-15247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016473#comment-17016473
]
vinoyang commented on FLINK-15247:
--
Another instance: https://api.travis-ci.com/v3/job/275753985/log.txt
> Closing (Testing)MiniCluster may cause ConcurrentModificationException
> --
>
> Key: FLINK-15247
> URL: https://issues.apache.org/jira/browse/FLINK-15247
> Project: Flink
> Issue Type: Bug
> Components: Tests
>Affects Versions: 1.10.0
>Reporter: Gary Yao
>Assignee: Andrey Zagrebin
>Priority: Blocker
> Labels: pull-request-available, test-stability
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> Test
> operatorsBecomeBackPressured(org.apache.flink.test.streaming.runtime.BackPressureITCase)
> failed with:
> org.apache.flink.util.FlinkException: Could not close resource.
> at
> org.apache.flink.util.AutoCloseableAsync.close(AutoCloseableAsync.java:42)org.apache.flink.test.streaming.runtime.BackPressureITCase.tearDown(BackPressureITCase.java:165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: org.apache.flink.util.FlinkException: Error while shutting the
> TaskExecutor down.
> at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.handleOnStopException(TaskExecutor.java:397)
> at
> org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$onStop$0(TaskExecutor.java:382)
> at
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
> at
>
[jira] [Commented] (FLINK-15607) throw exception when users trying to use Hive aggregate functions in streaming mode
[
https://issues.apache.org/jira/browse/FLINK-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016469#comment-17016469
]
Jingsong Lee commented on FLINK-15607:
--
[~jark] We have tests, though very simple. In
{{HiveCatalogUseBlinkITCase.testBlinkUdf}}.
> throw exception when users trying to use Hive aggregate functions in
> streaming mode
> ---
>
> Key: FLINK-15607
> URL: https://issues.apache.org/jira/browse/FLINK-15607
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Hive, Table SQL / API
>Reporter: Bowen Li
>Priority: Major
>
> Due to that Hive functions are all built for batch processing, aggregate
> functions in Hive that implement UDAF and GenericUDAFResolver2 interfaces may
> have unpredictable behaviors when used in streaming mode in Flink. We advice
> users to only use Hive aggregate functions interfaces in batch mode.
>
> It will be better to throw exception when users trying to use Hive aggregate
> functions in streaming mode. Seems need to distinguish execution mode in
> FunctionCatalogOperatorTable, which is not achievable yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-15599) SQL client requires both legacy and blink planner to be on the classpath
[
https://issues.apache.org/jira/browse/FLINK-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016467#comment-17016467
]
Jingsong Lee commented on FLINK-15599:
--
Thanks [~dwysakowicz] , Good catch! +1, looks like {{FunctionService}} is the
only dependent of sql-cli to legacy planner.
> SQL client requires both legacy and blink planner to be on the classpath
>
>
> Key: FLINK-15599
> URL: https://issues.apache.org/jira/browse/FLINK-15599
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
>Affects Versions: 1.10.0
>Reporter: Dawid Wysakowicz
>Priority: Critical
> Fix For: 1.10.0
>
>
> Sql client uses directly some of the internal classes of the legacy planner,
> thus it does not work with only the blink planner on the classpath.
> The internal class that's being used is
> {{org.apache.flink.table.functions.FunctionService}}
> This dependency was introduced in FLINK-13195
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible
flinkbot edited a comment on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible URL: https://github.com/apache/flink/pull/10685#issuecomment-568872421 ## CI report: * 18d9268b1f2106fceaffe388ff9e4e1babdad452 Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=3908) * d2686edf78b4d3afec05ca65486ff26304000aeb Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/143353493) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4150) * ce56c02026fdf5e8663b6bcd00e9a3bebbe38e65 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144549267) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4377) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-15607) throw exception when users trying to use Hive aggregate functions in streaming mode
[ https://issues.apache.org/jira/browse/FLINK-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016461#comment-17016461 ] Jark Wu commented on FLINK-15607: - I discussed with [~lzljs3620320] about this yesterday and learned that the Hive UDAF is supported in streaming mode (the accumulator will be stored in state). But it seems that we dont' have tests for this. > throw exception when users trying to use Hive aggregate functions in > streaming mode > --- > > Key: FLINK-15607 > URL: https://issues.apache.org/jira/browse/FLINK-15607 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Table SQL / API >Reporter: Bowen Li >Priority: Major > > Due to that Hive functions are all built for batch processing, aggregate > functions in Hive that implement UDAF and GenericUDAFResolver2 interfaces may > have unpredictable behaviors when used in streaming mode in Flink. We advice > users to only use Hive aggregate functions interfaces in batch mode. > > It will be better to throw exception when users trying to use Hive aggregate > functions in streaming mode. Seems need to distinguish execution mode in > FunctionCatalogOperatorTable, which is not achievable yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] sunhaibotb commented on a change in pull request #10151: [FLINK-14231] Handle the pending processing-time timers to make endInput semantics on the operator chain strict
sunhaibotb commented on a change in pull request #10151: [FLINK-14231] Handle
the pending processing-time timers to make endInput semantics on the operator
chain strict
URL: https://github.com/apache/flink/pull/10151#discussion_r367200322
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/OperatorChain.java
##
@@ -140,34 +140,41 @@ public OperatorChain(
}
// we create the chain of operators and grab the
collector that leads into the chain
- List> allOps = new
ArrayList<>(chainedConfigs.size());
+ List> allOpWrappers = new
ArrayList<>(chainedConfigs.size());
this.chainEntryPoint = createOutputCollector(
containingTask,
configuration,
chainedConfigs,
userCodeClassloader,
streamOutputMap,
- allOps,
+ allOpWrappers,
containingTask.getMailboxExecutorFactory());
if (operatorFactory != null) {
WatermarkGaugeExposingOutput>
output = getChainEntryPoint();
- headOperator =
StreamOperatorFactoryUtil.createOperator(
+ OP headOperator =
StreamOperatorFactoryUtil.createOperator(
operatorFactory,
containingTask,
configuration,
output);
headOperator.getMetricGroup().gauge(MetricNames.IO_CURRENT_OUTPUT_WATERMARK,
output.getWatermarkGauge());
+ this.headOperatorWrapper =
createOperatorWrapper(headOperator, containingTask, configuration);
+
+ // add head operator to end of chain
+ allOpWrappers.add(headOperatorWrapper);
+
+ this.tailOperatorWrapper = allOpWrappers.get(0);
} else {
- headOperator = null;
+ checkState(allOpWrappers.size() == 0);
Review comment:
Currently, only in the case of iteration, `StreamIterationHead` and
`StreamIterationTail` used for executing feedback edges do not contain any
operators. In the future, I think we don't want to introduce the case that the
operator chain contains operators but the header operator is `null`.
@rkhachatryan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49.
flinkbot edited a comment on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49. URL: https://github.com/apache/flink/pull/10860#issuecomment-574591065 ## CI report: * 5f9cd2224ef458e5068f3179aa5be025848f8723 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144485357) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4363) * f57ca417ecb8c6206a0f7b8617776d922ba9cd7d Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144496504) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4366) * 3ac114c589d40aed1ecd0267174941e50a423fc2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sunhaibotb commented on a change in pull request #10151: [FLINK-14231] Handle the pending processing-time timers to make endInput semantics on the operator chain strict
sunhaibotb commented on a change in pull request #10151: [FLINK-14231] Handle
the pending processing-time timers to make endInput semantics on the operator
chain strict
URL: https://github.com/apache/flink/pull/10151#discussion_r367200322
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/OperatorChain.java
##
@@ -140,34 +140,41 @@ public OperatorChain(
}
// we create the chain of operators and grab the
collector that leads into the chain
- List> allOps = new
ArrayList<>(chainedConfigs.size());
+ List> allOpWrappers = new
ArrayList<>(chainedConfigs.size());
this.chainEntryPoint = createOutputCollector(
containingTask,
configuration,
chainedConfigs,
userCodeClassloader,
streamOutputMap,
- allOps,
+ allOpWrappers,
containingTask.getMailboxExecutorFactory());
if (operatorFactory != null) {
WatermarkGaugeExposingOutput>
output = getChainEntryPoint();
- headOperator =
StreamOperatorFactoryUtil.createOperator(
+ OP headOperator =
StreamOperatorFactoryUtil.createOperator(
operatorFactory,
containingTask,
configuration,
output);
headOperator.getMetricGroup().gauge(MetricNames.IO_CURRENT_OUTPUT_WATERMARK,
output.getWatermarkGauge());
+ this.headOperatorWrapper =
createOperatorWrapper(headOperator, containingTask, configuration);
+
+ // add head operator to end of chain
+ allOpWrappers.add(headOperatorWrapper);
+
+ this.tailOperatorWrapper = allOpWrappers.get(0);
} else {
- headOperator = null;
+ checkState(allOpWrappers.size() == 0);
Review comment:
Currently, only in the case of iteration, `StreamIterationHead` and
`StreamIterationTail` do not contain any operators. In the future, I think we
don't want to introduce the case that the operator chain contains operators but
the header operator is `null`. @rkhachatryan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_…
KarmaGYZ commented on issue #10668: [hotfix] Align the parameter pattern of retry_times with retry_times_… URL: https://github.com/apache/flink/pull/10668#issuecomment-574945954 Thanks a lot to point out this @tillrohrmann . I update the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] banmoy commented on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible
banmoy commented on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible URL: https://github.com/apache/flink/pull/10685#issuecomment-574942577 @flinkbot run travis This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xintongsong commented on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49.
xintongsong commented on issue #10860: [FLINK-15145][config] Change TM memory configuration default values for FLIP-49. URL: https://github.com/apache/flink/pull/10860#issuecomment-574942430 Thank you @tillrohrmann. I've addressed your comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] banmoy commented on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible
banmoy commented on issue #10685: [FLINK-14843][e2e] Refactor bucketing sink test to make it more stable and comprehensible URL: https://github.com/apache/flink/pull/10685#issuecomment-574942495 travis on ce56c02 failed with `BackPressureITCase` which is tracked in [FLINK-15247](https://issues.apache.org/jira/browse/FLINK-15247), and rerun travis. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14812) Add custom libs to Flink classpath with an environment variable.
[ https://issues.apache.org/jira/browse/FLINK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eui Heo updated FLINK-14812: Description: To use plugin library you need to add it to the flink classpath. The documentation explains to put the jar file in the lib path. https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter However, to deploy metric-enabled Flinks on a kubernetes cluster, we have the burden of creating and managing another container image. It would be more efficient to add the classpath using environment variables inside the constructFlinkClassPath function in the config.sh file. In particular, it seems inconvenient for me to create separate images to use the jars, even though the /opt/ flink/opt of the stock image already contains them. For example, there are metrics libs and file system libs: flink-azure-fs-hadoop-1.9.1.jar flink-s3-fs-hadoop-1.9.1.jar flink-metrics-prometheus-1.9.1.jar flink-metrics-influxdb-1.9.1.jar was: To use plugin library you need to add it to the flink classpath. The documentation explains to put the jar file in the lib path. https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter However, to deploy metric-enabled Flinks on a kubernetes cluster, we have the burden of creating and managing another container image. It would be more efficient to add the classpath using environment variables inside the constructFlinkClassPath function in the config.sh file. In particular, it seems inconvenient for me to create a separate image to use the jars, even though the /opt/ flink/opt of the stock image already contains them. For example, there are metrics libs and file system libs: flink-azure-fs-hadoop-1.9.1.jar flink-s3-fs-hadoop-1.9.1.jar flink-metrics-prometheus-1.9.1.jar flink-metrics-influxdb-1.9.1.jar > Add custom libs to Flink classpath with an environment variable. > > > Key: FLINK-14812 > URL: https://issues.apache.org/jira/browse/FLINK-14812 > Project: Flink > Issue Type: New Feature > Components: Deployment / Kubernetes, Deployment / Scripts >Reporter: Eui Heo >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > To use plugin library you need to add it to the flink classpath. The > documentation explains to put the jar file in the lib path. > https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter > However, to deploy metric-enabled Flinks on a kubernetes cluster, we have the > burden of creating and managing another container image. It would be more > efficient to add the classpath using environment variables inside the > constructFlinkClassPath function in the config.sh file. > In particular, it seems inconvenient for me to create separate images to use > the jars, even though the /opt/ flink/opt of the stock image already contains > them. > For example, there are metrics libs and file system libs: > flink-azure-fs-hadoop-1.9.1.jar > flink-s3-fs-hadoop-1.9.1.jar > flink-metrics-prometheus-1.9.1.jar > flink-metrics-influxdb-1.9.1.jar -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-14812) Add custom libs to Flink classpath with an environment variable.
[ https://issues.apache.org/jira/browse/FLINK-14812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eui Heo updated FLINK-14812: Labels: (was: pull-request-available) > Add custom libs to Flink classpath with an environment variable. > > > Key: FLINK-14812 > URL: https://issues.apache.org/jira/browse/FLINK-14812 > Project: Flink > Issue Type: New Feature > Components: Deployment / Kubernetes, Deployment / Scripts >Reporter: Eui Heo >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > To use plugin library you need to add it to the flink classpath. The > documentation explains to put the jar file in the lib path. > https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter > However, to deploy metric-enabled Flinks on a kubernetes cluster, we have the > burden of creating and managing another container image. It would be more > efficient to add the classpath using environment variables inside the > constructFlinkClassPath function in the config.sh file. > In particular, it seems inconvenient for me to create a separate image to use > the jars, even though the /opt/ flink/opt of the stock image already contains > them. > For example, there are metrics libs and file system libs: > flink-azure-fs-hadoop-1.9.1.jar > flink-s3-fs-hadoop-1.9.1.jar > flink-metrics-prometheus-1.9.1.jar > flink-metrics-influxdb-1.9.1.jar -- This message was sent by Atlassian Jira (v8.3.4#803005)
