[jira] [Created] (FLINK-13794) Remove unused field printStatusDuringExecution in ClusterClient

[TisonKun (Jira)]

TisonKun created FLINK-13794:


 Summary: Remove unused field printStatusDuringExecution in 
ClusterClient
 Key: FLINK-13794
 URL: https://issues.apache.org/jira/browse/FLINK-13794
 Project: Flink
  Issue Type: Improvement
  Components: Command Line Client
Reporter: TisonKun
 Fix For: 1.10.0


{{printStatusDuringExecution}} in {{ClusterClient}} is only set but never used. 
It is itself a legacy field. Remove it as cleanup.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] Johnlon commented on issue #9456: FLINK-13588 flink-streaming-java don't throw away exception info in logging

[GitBox]

Johnlon commented on issue #9456: FLINK-13588 flink-streaming-java don't throw 
away exception info in logging 
URL: https://github.com/apache/flink/pull/9456#issuecomment-522953640
 
 
   Right sorry.
   
   The change passed tests in its own module.
   
   
   
   On Tue, 20 Aug 2019, 10:28 am Till Rohrmann, 
   wrote:
   
   > I'll try to clean the commit history up for you @Johnlon
   > . In the future I would recommend using git
   > rebase instead of git merge.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or mute the thread
   > 

   > .
   >
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9489: [FLINK-13793][docs] build each language in a separate subprocess

[GitBox]

flinkbot edited a comment on issue #9489: [FLINK-13793][docs] build each 
language in a separate subprocess
URL: https://github.com/apache/flink/pull/9489#issuecomment-522916512
 
 
   ## CI report:
   
   * b12cae90257c454f629c7c4482201051c6db5734 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123849622)
   * b2ef69716d1977ade128506a3e6f5d427ac95f1b : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123850835)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9456: FLINK-13588 flink-streaming-java don't throw away exception info in logging

[GitBox]

flinkbot edited a comment on issue #9456: FLINK-13588 flink-streaming-java 
don't throw away exception info in logging 
URL: https://github.com/apache/flink/pull/9456#issuecomment-521825874
 
 
   ## CI report:
   
   * 1242679f7bd5ec3f7c1115006e978267abafc84b : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123441772)
   * c2e57b175b07e9ee854598140676ab428c2b4b8f : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123442281)
   * cd9568ae549b007727edaacb0607c7310b2fd520 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123541232)
   * d954b81d8404c629808eb0c03d8c55e2d849a4e4 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123636613)
   * 9d5ecb78532bbccec41ebc5f4527b376ac9eb2d8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123742676)
   * 0e5d7e9f2a20e885f026790412a2bfba792763e0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123863103)
   * e8fd2de2dc011dd527f5d0fe7ecb8bf9f775d7b0 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123865363)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (FLINK-13750) Separate HA services between client-/ and server-side

[TisonKun (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911184#comment-16911184
 ] 

TisonKun commented on FLINK-13750:
--

Hi Till!

It also comes to me hours ago that ClusterClient should only hold 
WebMonitorRetriever and WebMonitorRetriever should be only held by 
ClusterClient, and then request to ConnectionInfo is forwarded to Dispatcher by 
WebMonitor the same as other requests. I'm really inspired we are in the same 
way.

Given the (Client)HighAvailabilityServices differs between RestClusterClient 
and MiniClusterClient I would prefer remove the field highAvailabilityServices 
and shift down the relevant implementations to subclasses. For 
RestClusterClient, it would be quite similar to your description; while for 
MiniClusterClient we can access proper service with 
miniCluster.getHighAvailabilityServices.(It is an embedded one and the whole 
mini cluster must run in the same process. Thus create a new service looks 
unnecessary and break how embedded service is implemented)

> Separate HA services between client-/ and server-side
> -
>
> Key: FLINK-13750
> URL: https://issues.apache.org/jira/browse/FLINK-13750
> Project: Flink
>  Issue Type: Improvement
>  Components: Command Line Client, Runtime / Coordination
>Reporter: Chesnay Schepler
>Assignee: TisonKun
>Priority: Major
>
> Currently, we use the same {{HighAvailabilityServices}} on the client and 
> server. However, the client does not need several of the features that the 
> services currently provide (access to the blobstore or checkpoint metadata).
> Additionally, due to how these services are setup they also require the 
> client to have access to the blob storage, despite it never actually being 
> used, which can cause issues, like FLINK-13500.
> [~Tison] Would be be interested in this issue?



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] 1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the 
usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315612373
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -166,18 +249,34 @@ kind: Service
 metadata:
   name: flink-jobmanager
 spec:
+  type: ClusterIP
   ports:
   - name: rpc
 port: 6123
   - name: blob
 port: 6124
-  - name: query
-port: 6125
   - name: ui
 port: 8081
   selector:
 app: flink
 component: jobmanager
 {% endhighlight %}
 
+`jobmanager-rest-service.yaml`. Optional service, which is used to add a 
`NodePort` on the jobmanager rest service so that user could submit job via 
`:`.
 
 Review comment:
   My reasoning, that there maybe two types of readers of this docs:
* people like devs, who have a test K8s cluster and want to setup 
test/experimental Flink cluster. For this, I assume they can run `kubectl 
port-forward|proxy` and also create resources by `kubectl`. For them the 
existing means should be already enough to make Flink JM available from a local 
dev machine. Showing them method that exposes `NodePort` would be like giving a 
poor example;
* people like ops, that operate production K8s cluster. In that case they 
are probably knowledgeable enough how to expose a service properly.
   
   There are some blog posts in internet, for example [this 
one](https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0)
 that give some information what are the different ways and in which situations 
to make service available outside of K8s cluster.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9456: FLINK-13588 flink-streaming-java don't throw away exception info in logging

[GitBox]

flinkbot edited a comment on issue #9456: FLINK-13588 flink-streaming-java 
don't throw away exception info in logging 
URL: https://github.com/apache/flink/pull/9456#issuecomment-521825874
 
 
   ## CI report:
   
   * 1242679f7bd5ec3f7c1115006e978267abafc84b : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123441772)
   * c2e57b175b07e9ee854598140676ab428c2b4b8f : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123442281)
   * cd9568ae549b007727edaacb0607c7310b2fd520 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123541232)
   * d954b81d8404c629808eb0c03d8c55e2d849a4e4 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123636613)
   * 9d5ecb78532bbccec41ebc5f4527b376ac9eb2d8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123742676)
   * 0e5d7e9f2a20e885f026790412a2bfba792763e0 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123863103)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

flinkbot edited a comment on issue #9470: [FLINK-13380][k8s] Improve the 
usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#issuecomment-522260447
 
 
   ## CI report:
   
   * 3db70d3f656ed9739d1a8f0b913e33732df36150 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123605754)
   * 075ce69d04e45dbf8534cca172a139d888460019 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123762022)
   * 7b67aa08a32d53d90bbe3756c9441f0522126760 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123848567)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955
 
 
   ## CI report:
   
   * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122292142)
   * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122941321)
   * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123229382)
   * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123235786)
   * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123332630)
   * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123335647)
   * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123336279)
   * c6ee15104ee678c239367670773723920e34c26d : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123371348)
   * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123472764)
   * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123476701)
   * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645059)
   * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123696981)
   * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123698096)
   * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123703814)
   * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123726809)
   * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123758034)
   * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842040)
   * 0507aa67d2f7183c3a7e4556fbf7732414647ac7 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123855539)
   * 9a293ae331986d8fae16f619158f1e59572cd1e9 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123856922)
   * 2bb1bc290a3fc8e4dc843be781063515e86509d8 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123859379)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Comment Edited] (FLINK-13787) PrometheusPushGatewayReporter does not cleanup TM metrics when run on kubernetes

[Kaibo Zhou (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910508#comment-16910508
 ] 

Kaibo Zhou edited comment on FLINK-13787 at 8/20/19 9:44 AM:
-

I did some investigate, the reason is that flink's jobmanager's pod is 
controlled by the *Job* in kubernetes, and the task manager's pod is controlled 
by *Deployment*.

When we run a standalone job on kubernetes, if we cancel the job, the 
jobmanager will shutdown and the pod will turn to a *Completed* state. So the 
jobmanager will call the close method of PrometheusPushGatewayReporter to 
delete the metrics on the pushgateway. However, the pod of the taskmanager is 
still running, and the shutdown logic inside the taskmanager is not called, the 
metrics of the taskmanager on the pushgateay will not be deleted.

In the current implementation, the taskmanager pod cannot be directly 
killed/stopped, otherwise, kubernetes Deployment will restart it to ensure the 
replicas of the pods. 

I think the expected behavior is when the user cancels the job, taskmanager 
should run shutdown logic which calls MetricRegistryImpl.shutdown() to release 
resource. 

 


was (Author: kaibo.zhou):
I did some investigate, the reason is that flink's jobmanager's pod is 
controlled by the *Job* in kubernetes, and the task manager's pod is controlled 
by *Deployment*.

When we run a standalone job on kubernetes, if we cancel the job, the 
jobmanager will shutdown and the pod will turn to a *Completed* state. So the 
jobmanager will call the close method of PrometheusPushGatewayReporter to 
delete the metrics on the pushgateway. However, the pod of the taskmanager is 
still running, and the shutdown logic inside the taskmanager is not called, the 
metrics of the taskmanager on the pushgateay will not be deleted.

In the current implementation, the taskmanager pod cannot be directly 
killed/stopped, otherwise, kubernetes Deployment will restart it to ensure the 
replicas of the pods.

 

I think the expected behavior is users delete taskmanager Deployment, and 
kubernetes send terminate signal to TM pods. TM pod can trigger registered JVM 
shutdown hook to call MetricRegistryImpl.shutdown. Is it right?

 

> PrometheusPushGatewayReporter does not cleanup TM metrics when run on 
> kubernetes
> 
>
> Key: FLINK-13787
> URL: https://issues.apache.org/jira/browse/FLINK-13787
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Metrics
>Affects Versions: 1.7.2, 1.8.1, 1.9.0
>Reporter: Kaibo Zhou
>Priority: Major
>
> I have run a flink job on kubernetes and use PrometheusPushGatewayReporter, I 
> can see the metrics from the flink jobmanager and taskmanager from the push 
> gateway's UI.
> When I cancel the job, I found the jobmanager's metrics disappear, but the 
> taskmanager's metrics still exist, even though I have set the 
> _deleteOnShutdown_ to true_._
> The configuration is:
> {code:java}
> metrics.reporters: "prom"
> metrics.reporter.prom.class: 
> "org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter"
> metrics.reporter.prom.jobName: "WordCount"
> metrics.reporter.prom.host: "localhost"
> metrics.reporter.prom.port: "9091"
> metrics.reporter.prom.randomJobNameSuffix: "true"
> metrics.reporter.prom.filterLabelValueCharacters: "true"
> metrics.reporter.prom.deleteOnShutdown: "true"
> {code}
>  
> Other people have also encountered this problem: 
> [https://stackoverflow.com/questions/54420498/flink-prometheus-push-gateway-reporter-delete-metrics-on-job-shutdown].
>   And another similar issue: FLINK-11457.
>  
> As prometheus is a very import metrics system on kubernetes, if we can solve 
> this problem, it is beneficial for users to monitor their flink jobs.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Assigned] (FLINK-13516) YARNSessionFIFOSecuredITCase fails on Java 11

[Chesnay Schepler (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-13516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler reassigned FLINK-13516:


Assignee: Haibo Sun

> YARNSessionFIFOSecuredITCase fails on Java 11
> -
>
> Key: FLINK-13516
> URL: https://issues.apache.org/jira/browse/FLINK-13516
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / YARN, Tests
>Reporter: Chesnay Schepler
>Assignee: Haibo Sun
>Priority: Major
> Fix For: 1.10.0
>
>
> {{YARNSessionFIFOSecuredITCase#testDetachedMode}} times out when run on Java 
> 11. This may be related to security changes in Java 11.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve 
the usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315598541
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -166,18 +249,34 @@ kind: Service
 metadata:
   name: flink-jobmanager
 spec:
+  type: ClusterIP
   ports:
   - name: rpc
 port: 6123
   - name: blob
 port: 6124
-  - name: query
-port: 6125
   - name: ui
 port: 8081
   selector:
 app: flink
 component: jobmanager
 {% endhighlight %}
 
+`jobmanager-rest-service.yaml`. Optional service, which is used to add a 
`NodePort` on the jobmanager rest service so that user could submit job via 
`:`.
 
 Review comment:
   @1u0  Use `NodePort` to expose an application from K8S cluster is a general 
solution from official doc. Why you think this should be avoided to add?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Assigned] (FLINK-13515) ClassLoaderITCase fails on Java 11

[Chesnay Schepler (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-13515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chesnay Schepler reassigned FLINK-13515:


Assignee: Haibo Sun

> ClassLoaderITCase fails on Java 11
> --
>
> Key: FLINK-13515
> URL: https://issues.apache.org/jira/browse/FLINK-13515
> Project: Flink
>  Issue Type: Sub-task
>  Components: Command Line Client, Tests
>Reporter: Chesnay Schepler
>Assignee: Haibo Sun
>Priority: Major
> Fix For: 1.10.0
>
>
> {{ClassLoaderITCas#testCheckpointedStreamingClassloaderJobWithCustomClassLoader}}
>  fails on Java 11 because the usercode exception can be serialized in the 
> client. This shouldn't be possible since the user-jar isn't on the classpath 
> of the client.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve 
the usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315598541
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -166,18 +249,34 @@ kind: Service
 metadata:
   name: flink-jobmanager
 spec:
+  type: ClusterIP
   ports:
   - name: rpc
 port: 6123
   - name: blob
 port: 6124
-  - name: query
-port: 6125
   - name: ui
 port: 8081
   selector:
 app: flink
 component: jobmanager
 {% endhighlight %}
 
+`jobmanager-rest-service.yaml`. Optional service, which is used to add a 
`NodePort` on the jobmanager rest service so that user could submit job via 
`:`.
 
 Review comment:
   Use `NodePort` to expose an application from K8S cluster is a general 
solution from official doc. Why you think this should be avoided to add?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] JingsongLi commented on a change in pull request #9484: [FLINK-13774][table] Expression DSL use should resolve Expressions in blink

[GitBox]

JingsongLi commented on a change in pull request #9484: [FLINK-13774][table] 
Expression DSL use should resolve Expressions in blink
URL: https://github.com/apache/flink/pull/9484#discussion_r315594066
 
 

 ##
 File path: 
flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/PushFilterIntoTableSourceScanRule.scala
 ##
 @@ -112,8 +112,8 @@ class PushFilterIntoTableSourceScanRule extends RelOptRule(
   call.transformTo(newScan)
 } else {
   relBuilder.push(scan)
-  val converter = new RexNodeConverter(relBuilder)
 
 Review comment:
   One question is whether `UnsolvedExpression` or `ResolvedExpression` is 
passed to `FilterableTableSource`.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve 
the usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315593616
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -54,20 +54,38 @@ A basic Flink session cluster deployment in Kubernetes has 
three components:
 
 Using the resource definitions for a [session 
cluster](#session-cluster-resource-definitions), launch the cluster with the 
`kubectl` command:
 
+kubectl create -f flink-configuration-configmap.yaml
 kubectl create -f jobmanager-service.yaml
 kubectl create -f jobmanager-deployment.yaml
 kubectl create -f taskmanager-deployment.yaml
 
-You can then access the Flink UI via `kubectl proxy`:
+Note that you could define your own customized options of `flink-conf.yaml` 
within `flink-configuration-configmap.yaml`.
 
-1. Run `kubectl proxy` in a terminal
-2. Navigate to 
[http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy](http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy)
 in your browser
+You can then access the Flink UI via different ways:
+*  `kubectl proxy`:
+
+1. Run `kubectl proxy` in a terminal.
+2. Navigate to 
[http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy](http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy)
 in your browser.
+
+*  `kubectl port-forward`:
+1. Run `kubectl port-forward ${flink-jobmanager-pod} 8081:8081` to forward 
your jobmanager's web ui port to local 8081.
+2. Navigate to [http://localhost:8001](http://localhost:8001) in your 
browser.
 
 Review comment:
   good catch.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955
 
 
   ## CI report:
   
   * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122292142)
   * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122941321)
   * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123229382)
   * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123235786)
   * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123332630)
   * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123335647)
   * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123336279)
   * c6ee15104ee678c239367670773723920e34c26d : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123371348)
   * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123472764)
   * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123476701)
   * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645059)
   * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123696981)
   * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123698096)
   * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123703814)
   * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123726809)
   * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123758034)
   * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842040)
   * 0507aa67d2f7183c3a7e4556fbf7732414647ac7 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123855539)
   * 9a293ae331986d8fae16f619158f1e59572cd1e9 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123856922)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on issue #9456: FLINK-13588 flink-streaming-java don't throw away exception info in logging

[GitBox]

tillrohrmann commented on issue #9456: FLINK-13588 flink-streaming-java don't 
throw away exception info in logging 
URL: https://github.com/apache/flink/pull/9456#issuecomment-522933304
 
 
   I'll try to clean the commit history up for you @Johnlon. In the future I 
would recommend using `git rebase` instead of `git merge`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (FLINK-13750) Separate HA services between client-/ and server-side

[Till Rohrmann (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911145#comment-16911145
 ] 

Till Rohrmann commented on FLINK-13750:
---

Hi Tison,

I would try to go the following way: The {{RestClusterClient}} should only need 
the {{webMonitorRetrievalService}}. Hence we should try to get rid of the 
{{dispatcherLeaderRetriever}} and then the {{HighAvailabilityServices}}stored 
in the {{ClusterClient}}. Btw. the {{ClusterClient}} is a legacy class with a 
lot of unneeded code.

Then I would introduce a {{ClientHighAvailabilityServices}} which has a method 
{{LeaderRetrievalService getWebMonitorLeaderRetriever();}}. In order to not 
break backwards compatibility we could only deprecate the same method in 
{{HighAvailabilityServices}}.

Next, we would need to introduce a new method to 
{{HighAvailabilityServicesFactory#createClientHAServices}} which allows us to 
create a {{ClientHighAvailabilityServices}} instance. This method should have a 
default implementation which fails.

For backwards compatibility, we could still create a 
{{HighAvailabilityServices}} if {{#createClientHAServices}} fails and then call 
{{HighAvailabilityServices#getWebMonitorLeaderRetriever()}}.

In order for proper resource clean up, one either needs to pass the 
{{(Client)HighAvailabilitySerivces}} to the {{RestClusterClient}} (ideally as a 
{{AutoCloseable}}) or create a wrapper for the {{LeaderRetrievalService}} which 
also closes the services when closing the {{LeaderRetrievalService}}.

I hope I haven't overlooked too many details here.

> Separate HA services between client-/ and server-side
> -
>
> Key: FLINK-13750
> URL: https://issues.apache.org/jira/browse/FLINK-13750
> Project: Flink
>  Issue Type: Improvement
>  Components: Command Line Client, Runtime / Coordination
>Reporter: Chesnay Schepler
>Assignee: TisonKun
>Priority: Major
>
> Currently, we use the same {{HighAvailabilityServices}} on the client and 
> server. However, the client does not need several of the features that the 
> services currently provide (access to the blobstore or checkpoint metadata).
> Additionally, due to how these services are setup they also require the 
> client to have access to the blob storage, despite it never actually being 
> used, which can cause issues, like FLINK-13500.
> [~Tison] Would be be interested in this issue?



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] 1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the 
usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315586136
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -54,20 +54,38 @@ A basic Flink session cluster deployment in Kubernetes has 
three components:
 
 Using the resource definitions for a [session 
cluster](#session-cluster-resource-definitions), launch the cluster with the 
`kubectl` command:
 
+kubectl create -f flink-configuration-configmap.yaml
 kubectl create -f jobmanager-service.yaml
 kubectl create -f jobmanager-deployment.yaml
 kubectl create -f taskmanager-deployment.yaml
 
-You can then access the Flink UI via `kubectl proxy`:
+Note that you could define your own customized options of `flink-conf.yaml` 
within `flink-configuration-configmap.yaml`.
 
-1. Run `kubectl proxy` in a terminal
-2. Navigate to 
[http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy](http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy)
 in your browser
+You can then access the Flink UI via different ways:
+*  `kubectl proxy`:
+
+1. Run `kubectl proxy` in a terminal.
+2. Navigate to 
[http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy](http://localhost:8001/api/v1/namespaces/default/services/flink-jobmanager:ui/proxy)
 in your browser.
+
+*  `kubectl port-forward`:
+1. Run `kubectl port-forward ${flink-jobmanager-pod} 8081:8081` to forward 
your jobmanager's web ui port to local 8081.
+2. Navigate to [http://localhost:8001](http://localhost:8001) in your 
browser.
 
 Review comment:
   Should be `localhost:8081` in this example.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] 1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

1u0 commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the 
usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315590339
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -166,18 +249,34 @@ kind: Service
 metadata:
   name: flink-jobmanager
 spec:
+  type: ClusterIP
   ports:
   - name: rpc
 port: 6123
   - name: blob
 port: 6124
-  - name: query
-port: 6125
   - name: ui
 port: 8081
   selector:
 app: flink
 component: jobmanager
 {% endhighlight %}
 
+`jobmanager-rest-service.yaml`. Optional service, which is used to add a 
`NodePort` on the jobmanager rest service so that user could submit job via 
`:`.
 
 Review comment:
   Can be rephrased to smt. like:
   ```
   Optional service, that exposes the jobmanager `ui` port as public Kubernetes 
node's port.
   ```
   
   But personally, I'd avoid adding this at all, as poor 
recommendation/guidance for users (as mentioned before).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9422: [FLINK-13697][table-api] Removed deprecated ExternalCatalog API

[GitBox]

flinkbot edited a comment on issue #9422:  [FLINK-13697][table-api] Removed 
deprecated ExternalCatalog API 
URL: https://github.com/apache/flink/pull/9422#issuecomment-520477677
 
 
   ## CI report:
   
   * 355a7706b108371f31efe68f181c3dbeceed06d5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122848893)
   * 9417eebaa0e2fa5831e4cf1c821c97480740f737 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/122932295)
   * b1b7222f53ea38c398f23a5169df6be834ddbf15 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123739835)
   * e764914d0224edcafac5972a4fcaf0fdc8e33558 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123741330)
   * 593df542b3833beeaccd9642b06f96b356e69612 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123855511)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9484: [FLINK-13774][table] Expression DSL use should resolve Expressions in blink

[GitBox]

flinkbot edited a comment on issue #9484: [FLINK-13774][table] Expression DSL 
use should resolve Expressions in blink
URL: https://github.com/apache/flink/pull/9484#issuecomment-522628616
 
 
   ## CI report:
   
   * 9559898f9d03c5aad327febdcd6fcd6a2d3e99f2 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123744310)
   * 5104e3ba53a3fb347b5c065d8dfdfcb10ad959ed : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123746834)
   * 7e28e22701c3f56fd8e1cddfb862d0f48c1d417e : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123753968)
   * 7b4e88c349ddf809576a7c75ac2a9bcc779a157d : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123855476)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] NicoK commented on issue #9444: [FLINK-13726][docs] build docs with jekyll 4.0.0.pre.beta1

[GitBox]

NicoK commented on issue #9444: [FLINK-13726][docs] build docs with jekyll 
4.0.0.pre.beta1
URL: https://github.com/apache/flink/pull/9444#issuecomment-522931163
 
 
   yes, @sjwiesman a complete rewrite of the site navigation would ultimately 
fix the build times. I have a follow-up PR which uses the current sidenav but 
builds it in an optimized way that further helps (probably not as much but 
enough for now).
   
   Regarding the link changes, I postponed that change for now and put it into 
https://issues.apache.org/jira/browse/FLINK-13727 as it is still working as it 
was before


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955
 
 
   ## CI report:
   
   * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122292142)
   * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122941321)
   * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123229382)
   * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123235786)
   * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123332630)
   * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123335647)
   * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123336279)
   * c6ee15104ee678c239367670773723920e34c26d : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123371348)
   * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123472764)
   * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123476701)
   * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645059)
   * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123696981)
   * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123698096)
   * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123703814)
   * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123726809)
   * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123758034)
   * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842040)
   * 0507aa67d2f7183c3a7e4556fbf7732414647ac7 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123855539)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (FLINK-13717) allow to set taskmanager.host and taskmanager.bind-host separately

[Stephan Ewen (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911137#comment-16911137
 ] 

Stephan Ewen commented on FLINK-13717:
--

My suggestion would be to

  - Set the default for "bind" to "0.0.0.0"
  - Set the default for "host" to "{{InetAddress.getLocalHost()}}" (like 
currently)



> allow to set taskmanager.host and taskmanager.bind-host separately
> --
>
> Key: FLINK-13717
> URL: https://issues.apache.org/jira/browse/FLINK-13717
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Configuration, Runtime / Network
>Affects Versions: 1.8.1, 1.9.0
>Reporter: Robert Fiser
>Priority: Major
>
> We trying to use flink in docker container with bridge network.
> Without specifying taskmanager.host taskmanager binds the host/address which 
> is not visible in cluster. It's same behaviour when taskmanager.host is set 
> to 0.0.0.0.
> When it is se to external address or host name then taskmanager cannot bind 
> the address because of bridge network.
> So we need to set taskmanager.host which will be reported to jobmanager and 
> taskmanager.bind-host which can taskmanager bind inside the container
> It similar to https://issues.apache.org/jira/browse/FLINK-2821 but the 
> problem is with taskmanagers.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] flinkbot edited a comment on issue #9471: [FLINK-13754][task] Decouple OperatorChain from StreamStatusMaintainer

[GitBox]

flinkbot edited a comment on issue #9471: [FLINK-13754][task] Decouple 
OperatorChain from StreamStatusMaintainer
URL: https://github.com/apache/flink/pull/9471#issuecomment-522269622
 
 
   ## CI report:
   
   * 46356e9f2ac97632021b3450f2585ea8b6120175 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123609454)
   * 330c8be5df79465a8804b7059c104984c6ac43ad : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123644155)
   * df762cf9c977de44fda9ccd5f805e7784c7dff6f : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123807523)
   * d162b3fa56334555fa2dc8c5489d5656a070f0a4 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842993)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955
 
 
   ## CI report:
   
   * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122292142)
   * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122941321)
   * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123229382)
   * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123235786)
   * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123332630)
   * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123335647)
   * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123336279)
   * c6ee15104ee678c239367670773723920e34c26d : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123371348)
   * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123472764)
   * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123476701)
   * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645059)
   * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123696981)
   * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123698096)
   * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123703814)
   * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123726809)
   * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123758034)
   * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842040)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Comment Edited] (FLINK-13787) PrometheusPushGatewayReporter does not cleanup TM metrics when run on kubernetes

[Kaibo Zhou (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910508#comment-16910508
 ] 

Kaibo Zhou edited comment on FLINK-13787 at 8/20/19 9:03 AM:
-

I did some investigate, the reason is that flink's jobmanager's pod is 
controlled by the *Job* in kubernetes, and the task manager's pod is controlled 
by *Deployment*.

When we run a standalone job on kubernetes, if we cancel the job, the 
jobmanager will shutdown and the pod will turn to a *Completed* state. So the 
jobmanager will call the close method of PrometheusPushGatewayReporter to 
delete the metrics on the pushgateway. However, the pod of the taskmanager is 
still running, and the shutdown logic inside the taskmanager is not called, the 
metrics of the taskmanager on the pushgateay will not be deleted.

In the current implementation, the taskmanager pod cannot be directly 
killed/stopped, otherwise, kubernetes Deployment will restart it to ensure the 
replicas of the pods.

 

I think the expected behavior is users delete taskmanager Deployment, and 
kubernetes send terminate signal to TM pods. TM pod can trigger registered JVM 
shutdown hook to call MetricRegistryImpl.shutdown. Is it right?

 


was (Author: kaibo.zhou):
I did some investigate, the reason is that flink's jobmanager's pod is 
controlled by the *Job* in kubernetes, and the task manager's pod is controlled 
by *Deployment*.

When we run a standalone job on kubernetes, if we cancel the job, the 
jobmanager will shutdown and the pod will turn to a *Completed* state. So the 
jobmanager will call the close method of PrometheusPushGatewayReporter to 
delete the metrics on the pushgateway. However, the pod of the taskmanager is 
still running, and the shutdown logic inside the taskmanager is not called, the 
metrics of the taskmanager on the pushgateay will not be deleted.

In the current implementation, the taskmanager cannot be stopped, otherwise 
kubernetes Deployment will restart it to ensure the replicas of the pod.

 

> PrometheusPushGatewayReporter does not cleanup TM metrics when run on 
> kubernetes
> 
>
> Key: FLINK-13787
> URL: https://issues.apache.org/jira/browse/FLINK-13787
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Metrics
>Affects Versions: 1.7.2, 1.8.1, 1.9.0
>Reporter: Kaibo Zhou
>Priority: Major
>
> I have run a flink job on kubernetes and use PrometheusPushGatewayReporter, I 
> can see the metrics from the flink jobmanager and taskmanager from the push 
> gateway's UI.
> When I cancel the job, I found the jobmanager's metrics disappear, but the 
> taskmanager's metrics still exist, even though I have set the 
> _deleteOnShutdown_ to true_._
> The configuration is:
> {code:java}
> metrics.reporters: "prom"
> metrics.reporter.prom.class: 
> "org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter"
> metrics.reporter.prom.jobName: "WordCount"
> metrics.reporter.prom.host: "localhost"
> metrics.reporter.prom.port: "9091"
> metrics.reporter.prom.randomJobNameSuffix: "true"
> metrics.reporter.prom.filterLabelValueCharacters: "true"
> metrics.reporter.prom.deleteOnShutdown: "true"
> {code}
>  
> Other people have also encountered this problem: 
> [https://stackoverflow.com/questions/54420498/flink-prometheus-push-gateway-reporter-delete-metrics-on-job-shutdown].
>   And another similar issue: FLINK-11457.
>  
> As prometheus is a very import metrics system on kubernetes, if we can solve 
> this problem, it is beneficial for users to monitor their flink jobs.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] flinkbot edited a comment on issue #9438: [WIP] [FLINK-13712] [docs] Add release notes for Flink 1.9.0

[GitBox]

flinkbot edited a comment on issue #9438: [WIP] [FLINK-13712] [docs] Add 
release notes for Flink 1.9.0
URL: https://github.com/apache/flink/pull/9438#issuecomment-521294079
 
 
   ## CI report:
   
   * 4d379770e97ef5333d7816713b79c9b5e9ee936e : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123244674)
   * 1afac18d0d672587db91f365fe9ac3b5e399684b : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123391776)
   * a2746fe782762782adffd6df5fd30e5edfecafb6 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123850865)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9475: [FLINK-13762][task] Implement a unified ForwardingValveOutputHandler for StreamOne/TwoInputProcessor

[GitBox]

flinkbot edited a comment on issue #9475: [FLINK-13762][task] Implement a 
unified ForwardingValveOutputHandler for StreamOne/TwoInputProcessor
URL: https://github.com/apache/flink/pull/9475#issuecomment-522352954
 
 
   ## CI report:
   
   * 7048a7504f200a099b5e4e42844888f51ba604aa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645048)
   * ec6450b37a2986aad216f00be0dbdee639ec2ffb : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123681958)
   * bda36681e3b81deb0b2e48ebc72b230adfb6830b : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123849658)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9489: [FLINK-13793][docs] build each language in a separate subprocess

[GitBox]

flinkbot edited a comment on issue #9489: [FLINK-13793][docs] build each 
language in a separate subprocess
URL: https://github.com/apache/flink/pull/9489#issuecomment-522916512
 
 
   ## CI report:
   
   * b12cae90257c454f629c7c4482201051c6db5734 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123849622)
   * b2ef69716d1977ade128506a3e6f5d427ac95f1b : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123850835)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tillrohrmann commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

tillrohrmann commented on a change in pull request #9470: [FLINK-13380][k8s] 
Improve the usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315575104
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -112,20 +153,38 @@ spec:
   containers:
   - name: jobmanager
 image: flink:latest
-args:
-- jobmanager
+workingDir: /opt/flink
+command: ["/bin/bash", "-c", "$FLINK_HOME/bin/jobmanager.sh start;\
+  while :;
+  do
+if [[ -f $(find log -name '*jobmanager*.log' -print -quit) ]];
+  then tail -f -n +1 log/*jobmanager*.log;
+fi;
+  done"]
 
 Review comment:
   Of course, one would have to use the entrypoint.sh then and not specify 
`command`. But I'm ok with discussing this as a separate issue and keeping it 
out of this PR's scope.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9475: [FLINK-13762][task] Implement a unified ForwardingValveOutputHandler for StreamOne/TwoInputProcessor

[GitBox]

flinkbot edited a comment on issue #9475: [FLINK-13762][task] Implement a 
unified ForwardingValveOutputHandler for StreamOne/TwoInputProcessor
URL: https://github.com/apache/flink/pull/9475#issuecomment-522352954
 
 
   ## CI report:
   
   * 7048a7504f200a099b5e4e42844888f51ba604aa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645048)
   * ec6450b37a2986aad216f00be0dbdee639ec2ffb : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123681958)
   * bda36681e3b81deb0b2e48ebc72b230adfb6830b : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123849658)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #9489: [FLINK-13793][docs] build each language in a separate subprocess

[GitBox]

flinkbot commented on issue #9489: [FLINK-13793][docs] build each language in a 
separate subprocess
URL: https://github.com/apache/flink/pull/9489#issuecomment-522916512
 
 
   ## CI report:
   
   * b12cae90257c454f629c7c4482201051c6db5734 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123849622)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] twalthr commented on a change in pull request #9438: [WIP] [FLINK-13712] [docs] Add release notes for Flink 1.9.0

[GitBox]

twalthr commented on a change in pull request #9438: [WIP] [FLINK-13712] [docs] 
Add release notes for Flink 1.9.0
URL: https://github.com/apache/flink/pull/9438#discussion_r315571352
 
 

 ##
 File path: docs/release-notes/flink-1.9.md
 ##
 @@ -0,0 +1,239 @@
+---
+title: "Release Notes - Flink 1.9"
+---
+
+
+
+These release notes discuss important aspects, such as configuration, behavior,
+or dependencies, that changed between Flink 1.8 and Flink 1.9. It also 
provides an overview
+on known shortcoming or limitations with new experimental features introduced 
in 1.9.
+
+Please read these notes carefully if you are planning to upgrade your Flink 
version to 1.9.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Known shortcomings or limitations for new features
+
+### New Table / SQL Blink planner
+
+Flink 1.9.0 provides support for two planners for the Table API, namely 
Flink's original planner and the new Blink
+planner. The original planner maintains same behaviour as previous releases, 
while the new Blink planner is still
+considered experimental and has the following limitations:
+
+- The Blink planner can not be used with `BatchTableEnvironment`, and 
therefore Table programs ran with the planner can not 
+be transformed to `DataSet` programs. This is by design and will also not be 
supported in the future. Therefore, if
+you want to run a batch job with the Blink planner, please use the new 
`TableEnvironment`. For streaming jobs,
+both `StreamTableEnvironment` and `TableEnvironment` works.
+- Implementations of `StreamTableSink` should implement the 
`consumeDataStream` method instead of `emitDataStream`
+if it is used with the Blink planner. Both methods work with the original 
planner.
+This is by design to make the returned `DataStreamSink` accessible for the 
planner.
+- Due to a bug with how transformations are not being cleared on execution, 
`TableEnvironment` instances should not
+be reused across multiple SQL statements when using the Blink planner.
+- `Table.flatAggregate` is not supported
+- Session and count windows are not supported when running batch jobs.
+- The Blink planner only supports the new `Catalog` API, and does not support 
`ExternalCatalog` which is now deprecated.
+
+Related issues:
+- [FLINK-13708: Transformations should be cleared because a table environment 
could execute multiple job](https://issues.apache.org/jira/browse/FLINK-13708)
+- [FLINK-13473: Add GroupWindowed FlatAggregate support to stream Table API 
(Blink planner), i.e, align with Flink 
planner](https://issues.apache.org/jira/browse/FLINK-13473)
+- [FLINK-13735: Support session window with Blink planner in batch 
mode](https://issues.apache.org/jira/browse/FLINK-13735)
+- [FLINK-13736: Support count window with Blink planner in batch 
mode](https://issues.apache.org/jira/browse/FLINK-13736)
+
+### SQL DDL
+
+In Flink 1.9.0, the community also added a preview feature about SQL DDL, but 
only for batch style DDLs.
+Therefore, all streaming related concepts are not supported yet, for example 
watermarks.
+
+Related issues:
+- [FLINK-13661: Add a stream specific CREATE TABLE SQL 
DDL](https://issues.apache.org/jira/browse/FLINK-13661)
+- [FLINK-13568: DDL create table doesn't allow STRING data 
type](https://issues.apache.org/jira/browse/FLINK-13568)
+
+### Java 9 support
+
+Since Flink 1.9.0, Flink can now be compiled and run on Java 9. Note that 
certain components interacting
+with external systems (connectors, filesystems, metric reporters, etc.) may 
not work since the respective projects may
+have skipped Java 9 support.
+
+Related issues:
+- [FLINK-8033: JDK 9 support](https://issues.apache.org/jira/browse/FLINK-8033)
+
+## Deprecations and breaking changes
+
+### Scala expression DSL for Table API moved to `flink-table-api-scala`
 
 Review comment:
   Looks good to me. Thanks for the update.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] tzulitai commented on a change in pull request #9438: [WIP] [FLINK-13712] [docs] Add release notes for Flink 1.9.0

[GitBox]

tzulitai commented on a change in pull request #9438: [WIP] [FLINK-13712] 
[docs] Add release notes for Flink 1.9.0
URL: https://github.com/apache/flink/pull/9438#discussion_r315569699
 
 

 ##
 File path: docs/release-notes/flink-1.9.md
 ##
 @@ -0,0 +1,239 @@
+---
+title: "Release Notes - Flink 1.9"
+---
+
+
+
+These release notes discuss important aspects, such as configuration, behavior,
+or dependencies, that changed between Flink 1.8 and Flink 1.9. It also 
provides an overview
+on known shortcoming or limitations with new experimental features introduced 
in 1.9.
+
+Please read these notes carefully if you are planning to upgrade your Flink 
version to 1.9.
+
+* This will be replaced by the TOC
+{:toc}
+
+## Known shortcomings or limitations for new features
+
+### New Table / SQL Blink planner
+
+Flink 1.9.0 provides support for two planners for the Table API, namely 
Flink's original planner and the new Blink
+planner. The original planner maintains same behaviour as previous releases, 
while the new Blink planner is still
+considered experimental and has the following limitations:
+
+- The Blink planner can not be used with `BatchTableEnvironment`, and 
therefore Table programs ran with the planner can not 
+be transformed to `DataSet` programs. This is by design and will also not be 
supported in the future. Therefore, if
+you want to run a batch job with the Blink planner, please use the new 
`TableEnvironment`. For streaming jobs,
+both `StreamTableEnvironment` and `TableEnvironment` works.
+- Implementations of `StreamTableSink` should implement the 
`consumeDataStream` method instead of `emitDataStream`
+if it is used with the Blink planner. Both methods work with the original 
planner.
+This is by design to make the returned `DataStreamSink` accessible for the 
planner.
+- Due to a bug with how transformations are not being cleared on execution, 
`TableEnvironment` instances should not
+be reused across multiple SQL statements when using the Blink planner.
+- `Table.flatAggregate` is not supported
+- Session and count windows are not supported when running batch jobs.
+- The Blink planner only supports the new `Catalog` API, and does not support 
`ExternalCatalog` which is now deprecated.
+
+Related issues:
+- [FLINK-13708: Transformations should be cleared because a table environment 
could execute multiple job](https://issues.apache.org/jira/browse/FLINK-13708)
+- [FLINK-13473: Add GroupWindowed FlatAggregate support to stream Table API 
(Blink planner), i.e, align with Flink 
planner](https://issues.apache.org/jira/browse/FLINK-13473)
+- [FLINK-13735: Support session window with Blink planner in batch 
mode](https://issues.apache.org/jira/browse/FLINK-13735)
+- [FLINK-13736: Support count window with Blink planner in batch 
mode](https://issues.apache.org/jira/browse/FLINK-13736)
+
+### SQL DDL
+
+In Flink 1.9.0, the community also added a preview feature about SQL DDL, but 
only for batch style DDLs.
+Therefore, all streaming related concepts are not supported yet, for example 
watermarks.
+
+Related issues:
+- [FLINK-13661: Add a stream specific CREATE TABLE SQL 
DDL](https://issues.apache.org/jira/browse/FLINK-13661)
+- [FLINK-13568: DDL create table doesn't allow STRING data 
type](https://issues.apache.org/jira/browse/FLINK-13568)
+
+### Java 9 support
+
+Since Flink 1.9.0, Flink can now be compiled and run on Java 9. Note that 
certain components interacting
+with external systems (connectors, filesystems, metric reporters, etc.) may 
not work since the respective projects may
+have skipped Java 9 support.
+
+Related issues:
+- [FLINK-8033: JDK 9 support](https://issues.apache.org/jira/browse/FLINK-8033)
+
+## Deprecations and breaking changes
+
+### Scala expression DSL for Table API moved to `flink-table-api-scala`
 
 Review comment:
   @twalthr Could you take a look at this latest addition?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot commented on issue #9489: [FLINK-13793][docs] build each language in a separate subprocess

[GitBox]

flinkbot commented on issue #9489: [FLINK-13793][docs] build each language in a 
separate subprocess
URL: https://github.com/apache/flink/pull/9489#issuecomment-522914882
 
 
   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit b12cae90257c454f629c7c4482201051c6db5734 (Tue Aug 20 
08:37:23 UTC 2019)
   
   **Warnings:**
* Documentation files were touched, but no `.zh.md` files: Update Chinese 
documentation or file Jira ticket.
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

flinkbot edited a comment on issue #9470: [FLINK-13380][k8s] Improve the 
usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#issuecomment-522260447
 
 
   ## CI report:
   
   * 3db70d3f656ed9739d1a8f0b913e33732df36150 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123605754)
   * 075ce69d04e45dbf8534cca172a139d888460019 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123762022)
   * 7b67aa08a32d53d90bbe3756c9441f0522126760 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123848567)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] NicoK opened a new pull request #9489: [FLINK-13793][docs] build each language in a separate subprocess

[GitBox]

NicoK opened a new pull request #9489: [FLINK-13793][docs] build each language 
in a separate subprocess
URL: https://github.com/apache/flink/pull/9489
 
 
   ## What is the purpose of the change
   
   When not serving docs and just building them, e.g. via `./build_docs.sh`, we 
can spawn sub-processes for each language build instead of running them 
single-threaded (jekyll currently does not support parallel builds).
   
   before: ~37s
   after: ~20s
   
   ## Brief change log
   
   Building on top of #9487, this PR adds:
   - change `build_docs.sh` to support parallel builds and merge resulting 
files together
   (admittedly this seems a bit hacky)
   - enhance `build_docs.sh`: show usage if wrong option is given
   - exclude build scripts from generated contents
   
   ## Verifying this change
   
   I verified the changes in the generated HTML pages (nothing changed except 
for the build scripts which I removed from the generated pages via a hotfix).
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (FLINK-13793) Build different language docs in parallel

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/FLINK-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-13793:
---
Labels: pull-request-available  (was: )

> Build different language docs in parallel
> -
>
> Key: FLINK-13793
> URL: https://issues.apache.org/jira/browse/FLINK-13793
> Project: Flink
>  Issue Type: Sub-task
>  Components: Documentation
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> Unfortunately, jekyll is lacking parallel builds and thus not making use of 
> unused resources. In the special case of building the documentation without 
> serving it, we could build each language (en, zh) in a separate sub-process 
> and thus get some parallelization.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] 
Adding processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#discussion_r315558967
 
 

 ##
 File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java
 ##
 @@ -227,11 +233,73 @@ public void setup(StreamTask containingTask, 
StreamConfig config, OutputThis method should be called whenever an operator would need to 
block the task thread to wait for output
 
 Review comment:
   Okay, I will remove it then. But FYI, HTML5 is as strict as XML, so you need 
to have a closing tag.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] 1u0 commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

1u0 commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#discussion_r315556884
 
 

 ##
 File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java
 ##
 @@ -227,11 +233,73 @@ public void setup(StreamTask containingTask, 
StreamConfig config, OutputThis method should be called whenever an operator would need to 
block the task thread to wait for output
 
 Review comment:
   I can't say much about newer Java version transition and if it would involve 
reformatting of all Java docs.
   Afaik, HTML5 also allows non paired `` tags. So the old "convention" 
could be still valid.
   
   I'm not opinionated which style to choose, but I'm for consistence and more 
uniform style.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] 
Adding processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#discussion_r315556403
 
 

 ##
 File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/Emitter.java
 ##
 @@ -97,22 +102,18 @@ public void run() {
}
}
 
-   private void output(AsyncResult asyncResult) throws 
InterruptedException {
+   /**
+* Executed as a mail in the mailbox thread. Output needs to be guarded 
with checkpoint lock (for the time being).
+*
+* @param asyncResult the result to output.
+*/
+   private void output(AsyncResult asyncResult) {
if (asyncResult.isWatermark()) {
-   synchronized (checkpointLock) {
-   AsyncWatermarkResult asyncWatermarkResult = 
asyncResult.asWatermark();
+   AsyncWatermarkResult asyncWatermarkResult = 
asyncResult.asWatermark();
 
 Review comment:
   Moved poll inside checkpoint lock


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9471: [FLINK-13754][task] Decouple OperatorChain from StreamStatusMaintainer

[GitBox]

flinkbot edited a comment on issue #9471: [FLINK-13754][task] Decouple 
OperatorChain from StreamStatusMaintainer
URL: https://github.com/apache/flink/pull/9471#issuecomment-522269622
 
 
   ## CI report:
   
   * 46356e9f2ac97632021b3450f2585ea8b6120175 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123609454)
   * 330c8be5df79465a8804b7059c104984c6ac43ad : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123644155)
   * df762cf9c977de44fda9ccd5f805e7784c7dff6f : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123807523)
   * d162b3fa56334555fa2dc8c5489d5656a070f0a4 : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842993)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (FLINK-13793) Build different language docs in parallel

[Nico Kruber (Jira)]

Nico Kruber created FLINK-13793:
---

 Summary: Build different language docs in parallel
 Key: FLINK-13793
 URL: https://issues.apache.org/jira/browse/FLINK-13793
 Project: Flink
  Issue Type: Sub-task
  Components: Documentation
Reporter: Nico Kruber
Assignee: Nico Kruber


Unfortunately, jekyll is lacking parallel builds and thus not making use of 
unused resources. In the special case of building the documentation without 
serving it, we could build each language (en, zh) in a separate sub-process and 
thus get some parallelization.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] 
Adding processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#discussion_r315548601
 
 

 ##
 File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/AbstractStreamOperator.java
 ##
 @@ -227,11 +233,73 @@ public void setup(StreamTask containingTask, 
StreamConfig config, OutputThis method should be called whenever an operator would need to 
block the task thread to wait for output
 
 Review comment:
   Since we are slowly adopting newer Java versions, it's quite likely that we 
also switch to new javadoc with HTML5. I'd prefer to not revert back to HTML4 
unless you think it is confusing to devs and users.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding 
processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955
 
 
   ## CI report:
   
   * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122292142)
   * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/122941321)
   * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123229382)
   * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123235786)
   * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123332630)
   * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123335647)
   * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123336279)
   * c6ee15104ee678c239367670773723920e34c26d : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123371348)
   * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123472764)
   * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS 
[Build](https://travis-ci.com/flink-ci/flink/builds/123476701)
   * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123645059)
   * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123696981)
   * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED 
[Build](https://travis-ci.com/flink-ci/flink/builds/123698096)
   * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123703814)
   * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123726809)
   * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE 
[Build](https://travis-ci.com/flink-ci/flink/builds/123758034)
   * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : PENDING 
[Build](https://travis-ci.com/flink-ci/flink/builds/123842040)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] xy2953396112 commented on issue #9401: [hotfix] Fix logger arguments in CatalogManager

[GitBox]

xy2953396112 commented on issue #9401: [hotfix] Fix logger arguments in 
CatalogManager
URL: https://github.com/apache/flink/pull/9401#issuecomment-522895773
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [flink] AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops

[GitBox]

AHeise commented on a change in pull request #9383: [FLINK-13248] [runtime] 
Adding processing of downstream messages in AsyncWaitOperator's wait loops
URL: https://github.com/apache/flink/pull/9383#discussion_r315546676
 
 

 ##
 File path: 
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperator.java
 ##
 @@ -387,35 +365,19 @@ private void stopResources(boolean waitForShutdown) 
throws InterruptedException
 * Add the given stream element queue entry to the operator's stream 
element queue. This
 * operation blocks until the element has been added.
 *
-* For that it tries to put the element into the queue and if not 
successful then it waits on
-* the checkpointing lock. The checkpointing lock is also used by the 
{@link Emitter} to output
-* elements. The emitter is also responsible for notifying this method 
if the queue has capacity
-* left again, by calling notifyAll on the checkpointing lock.
-*
 * @param streamElementQueueEntry to add to the operator's queue
 * @param  Type of the stream element queue entry's result
 * @throws InterruptedException if the current thread has been 
interrupted
 */
private  void addAsyncBufferEntry(StreamElementQueueEntry 
streamElementQueueEntry) throws InterruptedException {
-   assert(Thread.holdsLock(checkpointingLock));
-
-   pendingStreamElementQueueEntry = streamElementQueueEntry;
-
while (!queue.tryPut(streamElementQueueEntry)) {
-   // we wait for the emitter to notify us if the queue 
has space left again
-   checkpointingLock.wait();
+   yieldToDownstream();
}
-
-   pendingStreamElementQueueEntry = null;
}
 
-   private void waitInFlightInputsFinished() throws InterruptedException {
-   assert(Thread.holdsLock(checkpointingLock));
-
+   private void waitInFlightInputsFinished() {
while (!queue.isEmpty()) {
-   // wait for the emitter thread to output the remaining 
elements
-   // for that he needs the checkpointing lock and thus we 
have to free it
-   checkpointingLock.wait();
+   yieldToDownstream();
 
 Review comment:
   If we do yield now before migrating processing timers, we can run into 
deadlocks.
   
   Imagine the following: asyncIO is using an external system that is currently 
not available. The implementation relies on timeouts to occur. Thus, the queue 
of the AsyncWaitOperator is full with tasks that require the timeout to happen. 
At that point, we need to release the checkpoint lock instead of yielding 
(since processor timers are not migrated yet). Thus, we need to alternate 
between yieldToDownstream and releasing checkpoint lock.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (FLINK-13792) source and sink support manual rate limit

[zzsmdfj (Jira)]

zzsmdfj created FLINK-13792:
---

 Summary: source and sink support manual rate limit
 Key: FLINK-13792
 URL: https://issues.apache.org/jira/browse/FLINK-13792
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / Common
Affects Versions: 1.8.1
Reporter: zzsmdfj


in current flink implement automatic flow control by back pressure, it is 
efficient for the most scene, but in some special scenario, do we need 
fine-grained flow control to avoid impact on other systems? For example: if i 
have window with days(a lot of datas), then do call ProcessWindowFunction when 
trigger, this will produce a lot of data to sink, if sink to message queue, it 
can have a huge impact to message queue. so if there is sink rate limiter, it 
is friendly to external system. for source rate limiter, it is appropriate for 
having window operator and accumulating a large amount of historical data.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Comment Edited] (FLINK-13717) allow to set taskmanager.host and taskmanager.bind-host separately

[Robert Fiser (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911066#comment-16911066
 ] 

Robert Fiser edited comment on FLINK-13717 at 8/20/19 7:22 AM:
---

I don't think that just default listening on _0.0.0.0_ will fit. By default I 
mean without specifying _taskmanager.host_. We need to register taskmanager 
with external host/address and the process doesn't know it without specifying 
via config so it would be registered with zeroes or container host/address 
which is not visible in cluster.

The only minimum scenario that fits our requirements is to always listening on 
zeroes and register taskmanager with _taskmanager.host_ and system default. But 
as I said before I don't think it is a good practice to listening always on 
zeores(all visible interfaces).

 


was (Author: fiserro):
I don't think that just default listening on 0.0.0.0 will fit. By default I 
mean without specifying /taskmanager.host/. We need to register taskmanager 
with external host/address and the process doesn't know it without specifying 
via config so it would be registered with zeroes or container host/address 
which is not visible in cluster.

The only minimum scenario that fits our requirements is to always listening on 
zeroes and register taskmanager with /taskmanager.host/ and system default. But 
as I said before I don't think it is a good practice to listening always on 
zeores(all visible interfaces).

 

> allow to set taskmanager.host and taskmanager.bind-host separately
> --
>
> Key: FLINK-13717
> URL: https://issues.apache.org/jira/browse/FLINK-13717
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Configuration, Runtime / Network
>Affects Versions: 1.8.1, 1.9.0
>Reporter: Robert Fiser
>Priority: Major
>
> We trying to use flink in docker container with bridge network.
> Without specifying taskmanager.host taskmanager binds the host/address which 
> is not visible in cluster. It's same behaviour when taskmanager.host is set 
> to 0.0.0.0.
> When it is se to external address or host name then taskmanager cannot bind 
> the address because of bridge network.
> So we need to set taskmanager.host which will be reported to jobmanager and 
> taskmanager.bind-host which can taskmanager bind inside the container
> It similar to https://issues.apache.org/jira/browse/FLINK-2821 but the 
> problem is with taskmanagers.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (FLINK-13717) allow to set taskmanager.host and taskmanager.bind-host separately

[Robert Fiser (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911066#comment-16911066
 ] 

Robert Fiser commented on FLINK-13717:
--

I don't think that just default listening on 0.0.0.0 will fit. By default I 
mean without specifying /taskmanager.host/. We need to register taskmanager 
with external host/address and the process doesn't know it without specifying 
via config so it would be registered with zeroes or container host/address 
which is not visible in cluster.

The only minimum scenario that fits our requirements is to always listening on 
zeroes and register taskmanager with /taskmanager.host/ and system default. But 
as I said before I don't think it is a good practice to listening always on 
zeores(all visible interfaces).

 

> allow to set taskmanager.host and taskmanager.bind-host separately
> --
>
> Key: FLINK-13717
> URL: https://issues.apache.org/jira/browse/FLINK-13717
> Project: Flink
>  Issue Type: New Feature
>  Components: Runtime / Configuration, Runtime / Network
>Affects Versions: 1.8.1, 1.9.0
>Reporter: Robert Fiser
>Priority: Major
>
> We trying to use flink in docker container with bridge network.
> Without specifying taskmanager.host taskmanager binds the host/address which 
> is not visible in cluster. It's same behaviour when taskmanager.host is set 
> to 0.0.0.0.
> When it is se to external address or host name then taskmanager cannot bind 
> the address because of bridge network.
> So we need to set taskmanager.host which will be reported to jobmanager and 
> taskmanager.bind-host which can taskmanager bind inside the container
> It similar to https://issues.apache.org/jira/browse/FLINK-2821 but the 
> problem is with taskmanagers.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[GitHub] [flink] Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve the usability of Flink session cluster on Kubernetes

[GitBox]

Myasuka commented on a change in pull request #9470: [FLINK-13380][k8s] Improve 
the usability of Flink session cluster on Kubernetes
URL: https://github.com/apache/flink/pull/9470#discussion_r315535740
 
 

 ##
 File path: docs/ops/deployment/kubernetes.md
 ##
 @@ -166,18 +242,31 @@ kind: Service
 metadata:
   name: flink-jobmanager
 spec:
+  type: ClusterIP
   ports:
   - name: rpc
 port: 6123
   - name: blob
 port: 6124
-  - name: query
-port: 6125
   - name: ui
 port: 8081
   selector:
 app: flink
 component: jobmanager
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: flink-jobmanager-svc
+spec:
+  type: NodePort
 
 Review comment:
   Okay, I think we might add a separate service yaml file in the documentation 
to represent the `rest service` and tell users different ways to submit jobs: 
via `kubectl port-forward` or via the nodeport of this rest service.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (FLINK-13769) BatchFineGrainedRecoveryITCase.testProgram failed on Travis

[Till Rohrmann (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911033#comment-16911033
 ] 

Till Rohrmann commented on FLINK-13769:
---

Another instance: https://api.travis-ci.org/v3/job/573915348/log.txt

> BatchFineGrainedRecoveryITCase.testProgram failed on Travis
> ---
>
> Key: FLINK-13769
> URL: https://issues.apache.org/jira/browse/FLINK-13769
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Andrey Zagrebin
>Assignee: Andrey Zagrebin
>Priority: Critical
>  Labels: test-stability
>
> {{BatchFineGrainedRecoveryITCase.testProgram}} failed on Travis.
> {code}
> 23:14:26.860 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time 
> elapsed: 50.007 s <<< FAILURE! - in 
> org.apache.flink.test.recovery.BatchFineGrainedRecoveryITCase
> 23:14:26.868 [ERROR] 
> testProgram(org.apache.flink.test.recovery.BatchFineGrainedRecoveryITCase)  
> Time elapsed: 49.469 s  <<< ERROR!
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
>   at 
> org.apache.flink.test.recovery.BatchFineGrainedRecoveryITCase.testProgram(BatchFineGrainedRecoveryITCase.java:225)
> Caused by: java.util.concurrent.CompletionException: 
> akka.pattern.AskTimeoutException: Ask timed out on 
> [Actor[akka.tcp://flink@localhost:39333/user/taskmanager_3#-344551647]] after 
> [1 ms]. Message of type 
> [org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation]. A typical reason 
> for `AskTimeoutException` is that the recipient actor didn't send a reply.
> Caused by: akka.pattern.AskTimeoutException: Ask timed out on 
> [Actor[akka.tcp://flink@localhost:39333/user/taskmanager_3#-344551647]] after 
> [1 ms]. Message of type 
> [org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation]. A typical reason 
> for `AskTimeoutException` is that the recipient actor didn't send a reply.
> {code}
> [https://travis-ci.org/apache/flink/jobs/573523669]



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (FLINK-7860) Support YARN proxy user in Flink (impersonation)

[Shengnan YU (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-7860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911025#comment-16911025
 ] 

Shengnan YU commented on FLINK-7860:


Is there any plan for this improvement?

> Support YARN proxy user in Flink (impersonation)
> 
>
> Key: FLINK-7860
> URL: https://issues.apache.org/jira/browse/FLINK-7860
> Project: Flink
>  Issue Type: New Feature
>  Components: Deployment / YARN
>Reporter: Shuyi Chen
>Assignee: Shuyi Chen
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.2#803003)

[jira] [Commented] (FLINK-13790) Support -e option with a sql script file as input

[Zhenghua Gao (Jira)]

[ 
https://issues.apache.org/jira/browse/FLINK-13790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911020#comment-16911020
 ] 

Zhenghua Gao commented on FLINK-13790:
--

OK， I will create a PR for it soon.

> Support -e option with a sql script file as input
> -
>
> Key: FLINK-13790
> URL: https://issues.apache.org/jira/browse/FLINK-13790
> Project: Flink
>  Issue Type: Sub-task
>  Components: Command Line Client
>Reporter: Bowen Li
>Assignee: Zhenghua Gao
>Priority: Major
> Fix For: 1.10.0
>
>
> We expect user to run SQL directly on the command line. Something like: 
> sql-client embedded -f "query in string", which will execute the given file 
> without entering interactive mode
> This is related to FLINK-12828.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)