Great idea Tison! I will create the umbrella issue and post it here so that we are all on the same page!
Cheers, Kostas On Wed, Sep 4, 2019 at 11:36 AM Zili Chen <wander4...@gmail.com> wrote: > > Hi Kostas & Aljoscha, > > I notice that there is a JIRA(FLINK-13946) which could be included > in this refactor thread. Since we agree on most of directions in > big picture, is it reasonable that we create an umbrella issue for > refactor client APIs and also linked subtasks? It would be a better > way that we join forces of our community. > > Best, > tison. > > > Zili Chen <wander4...@gmail.com> 于2019年8月31日周六 下午12:52写道： >> >> Great Kostas! Looking forward to your POC! >> >> Best, >> tison. >> >> >> Jeff Zhang <zjf...@gmail.com> 于2019年8月30日周五 下午11:07写道： >>> >>> Awesome, @Kostas Looking forward your POC. >>> >>> Kostas Kloudas <kklou...@gmail.com> 于2019年8月30日周五 下午8:33写道： >>> >>> > Hi all, >>> > >>> > I am just writing here to let you know that I am working on a POC that >>> > tries to refactor the current state of job submission in Flink. >>> > I want to stress out that it introduces NO CHANGES to the current >>> > behaviour of Flink. It just re-arranges things and introduces the >>> > notion of an Executor, which is the entity responsible for taking the >>> > user-code and submitting it for execution. >>> > >>> > Given this, the discussion about the functionality that the JobClient >>> > will expose to the user can go on independently and the same >>> > holds for all the open questions so far. >>> > >>> > I hope I will have some more new to share soon. >>> > >>> > Thanks, >>> > Kostas >>> > >>> > On Mon, Aug 26, 2019 at 4:20 AM Yang Wang <danrtsey...@gmail.com> wrote: >>> > > >>> > > Hi Zili, >>> > > >>> > > It make sense to me that a dedicated cluster is started for a per-job >>> > > cluster and will not accept more jobs. >>> > > Just have a question about the command line. >>> > > >>> > > Currently we could use the following commands to start different >>> > clusters. >>> > > *per-job cluster* >>> > > ./bin/flink run -d -p 5 -ynm perjob-cluster1 -m yarn-cluster >>> > > examples/streaming/WindowJoin.jar >>> > > *session cluster* >>> > > ./bin/flink run -p 5 -ynm session-cluster1 -m yarn-cluster >>> > > examples/streaming/WindowJoin.jar >>> > > >>> > > What will it look like after client enhancement? >>> > > >>> > > >>> > > Best, >>> > > Yang >>> > > >>> > > Zili Chen <wander4...@gmail.com> 于2019年8月23日周五 下午10:46写道： >>> > > >>> > > > Hi Till, >>> > > > >>> > > > Thanks for your update. Nice to hear :-) >>> > > > >>> > > > Best, >>> > > > tison. >>> > > > >>> > > > >>> > > > Till Rohrmann <trohrm...@apache.org> 于2019年8月23日周五 下午10:39写道： >>> > > > >>> > > > > Hi Tison, >>> > > > > >>> > > > > just a quick comment concerning the class loading issues when using >>> > the >>> > > > per >>> > > > > job mode. The community wants to change it so that the >>> > > > > StandaloneJobClusterEntryPoint actually uses the user code class >>> > loader >>> > > > > with child first class loading [1]. Hence, I hope that this problem >>> > will >>> > > > be >>> > > > > resolved soon. >>> > > > > >>> > > > > [1] https://issues.apache.org/jira/browse/FLINK-13840 >>> > > > > >>> > > > > Cheers, >>> > > > > Till >>> > > > > >>> > > > > On Fri, Aug 23, 2019 at 2:47 PM Kostas Kloudas <kklou...@gmail.com> >>> > > > wrote: >>> > > > > >>> > > > > > Hi all, >>> > > > > > >>> > > > > > On the topic of web submission, I agree with Till that it only >>> > seems >>> > > > > > to complicate things. >>> > > > > > It is bad for security, job isolation (anybody can submit/cancel >>> > jobs), >>> > > > > > and its >>> > > > > > implementation complicates some parts of the code. So, if it were >>> > to >>> > > > > > redesign the >>> > > > > > WebUI, maybe this part could be left out. In addition, I would say >>> > > > > > that the ability to cancel >>> > > > > > jobs could also be left out. >>> > > > > > >>> > > > > > Also I would also be in favour of removing the "detached" mode, >>> > > > > > for >>> > > > > > the reasons mentioned >>> > > > > > above (i.e. because now we will have a future representing the >>> > result >>> > > > > > on which the user >>> > > > > > can choose to wait or not). >>> > > > > > >>> > > > > > Now for the separating job submission and cluster creation, I am >>> > > > > > in >>> > > > > > favour of keeping both. >>> > > > > > Once again, the reasons are mentioned above by Stephan, Till, >>> > Aljoscha >>> > > > > > and also Zili seems >>> > > > > > to agree. They mainly have to do with security, isolation and ease >>> > of >>> > > > > > resource management >>> > > > > > for the user as he knows that "when my job is done, everything >>> > will be >>> > > > > > cleared up". This is >>> > > > > > also the experience you get when launching a process on your local >>> > OS. >>> > > > > > >>> > > > > > On excluding the per-job mode from returning a JobClient or not, I >>> > > > > > believe that eventually >>> > > > > > it would be nice to allow users to get back a jobClient. The >>> > reason is >>> > > > > > that 1) I cannot >>> > > > > > find any objective reason why the user-experience should diverge, >>> > and >>> > > > > > 2) this will be the >>> > > > > > way that the user will be able to interact with his running job. >>> > > > > > Assuming that the necessary >>> > > > > > ports are open for the REST API to work, then I think that the >>> > > > > > JobClient can run against the >>> > > > > > REST API without problems. If the needed ports are not open, then >>> > we >>> > > > > > are safe to not return >>> > > > > > a JobClient, as the user explicitly chose to close all points of >>> > > > > > communication to his running job. >>> > > > > > >>> > > > > > On the topic of not hijacking the "env.execute()" in order to get >>> > the >>> > > > > > Plan, I definitely agree but >>> > > > > > for the proposal of having a "compile()" method in the env, I >>> > > > > > would >>> > > > > > like to have a better look at >>> > > > > > the existing code. >>> > > > > > >>> > > > > > Cheers, >>> > > > > > Kostas >>> > > > > > >>> > > > > > On Fri, Aug 23, 2019 at 5:52 AM Zili Chen <wander4...@gmail.com> >>> > > > wrote: >>> > > > > > > >>> > > > > > > Hi Yang, >>> > > > > > > >>> > > > > > > It would be helpful if you check Stephan's last comment, >>> > > > > > > which states that isolation is important. >>> > > > > > > >>> > > > > > > For per-job mode, we run a dedicated cluster(maybe it >>> > > > > > > should have been a couple of JM and TMs during FLIP-6 >>> > > > > > > design) for a specific job. Thus the process is prevented >>> > > > > > > from other jobs. >>> > > > > > > >>> > > > > > > In our cases there was a time we suffered from multi >>> > > > > > > jobs submitted by different users and they affected >>> > > > > > > each other so that all ran into an error state. Also, >>> > > > > > > run the client inside the cluster could save client >>> > > > > > > resource at some points. >>> > > > > > > >>> > > > > > > However, we also face several issues as you mentioned, >>> > > > > > > that in per-job mode it always uses parent classloader >>> > > > > > > thus classloading issues occur. >>> > > > > > > >>> > > > > > > BTW, one can makes an analogy between session/per-job mode >>> > > > > > > in Flink, and client/cluster mode in Spark. >>> > > > > > > >>> > > > > > > Best, >>> > > > > > > tison. >>> > > > > > > >>> > > > > > > >>> > > > > > > Yang Wang <danrtsey...@gmail.com> 于2019年8月22日周四 上午11:25写道： >>> > > > > > > >>> > > > > > > > From the user's perspective, it is really confused about the >>> > scope >>> > > > of >>> > > > > > > > per-job cluster. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > If it means a flink cluster with single job, so that we could >>> > get >>> > > > > > better >>> > > > > > > > isolation. >>> > > > > > > > >>> > > > > > > > Now it does not matter how we deploy the cluster, directly >>> > > > > > deploy(mode1) >>> > > > > > > > >>> > > > > > > > or start a flink cluster and then submit job through cluster >>> > > > > > client(mode2). >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > Otherwise, if it just means directly deploy, how should we >>> > name the >>> > > > > > mode2, >>> > > > > > > > >>> > > > > > > > session with job or something else? >>> > > > > > > > >>> > > > > > > > We could also benefit from the mode2. Users could get the same >>> > > > > > isolation >>> > > > > > > > with mode1. >>> > > > > > > > >>> > > > > > > > The user code and dependencies will be loaded by user class >>> > loader >>> > > > > > > > >>> > > > > > > > to avoid class conflict with framework. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > Anyway, both of the two submission modes are useful. >>> > > > > > > > >>> > > > > > > > We just need to clarify the concepts. >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > Best, >>> > > > > > > > >>> > > > > > > > Yang >>> > > > > > > > >>> > > > > > > > Zili Chen <wander4...@gmail.com> 于2019年8月20日周二 下午5:58写道： >>> > > > > > > > >>> > > > > > > > > Thanks for the clarification. >>> > > > > > > > > >>> > > > > > > > > The idea JobDeployer ever came into my mind when I was >>> > muddled >>> > > > with >>> > > > > > > > > how to execute per-job mode and session mode with the same >>> > user >>> > > > > code >>> > > > > > > > > and framework codepath. >>> > > > > > > > > >>> > > > > > > > > With the concept JobDeployer we back to the statement that >>> > > > > > environment >>> > > > > > > > > knows every configs of cluster deployment and job >>> > submission. We >>> > > > > > > > > configure or generate from configuration a specific >>> > JobDeployer >>> > > > in >>> > > > > > > > > environment and then code align on >>> > > > > > > > > >>> > > > > > > > > *JobClient client = env.execute().get();* >>> > > > > > > > > >>> > > > > > > > > which in session mode returned by clusterClient.submitJob >>> > and in >>> > > > > > per-job >>> > > > > > > > > mode returned by clusterDescriptor.deployJobCluster. >>> > > > > > > > > >>> > > > > > > > > Here comes a problem that currently we directly run >>> > > > > ClusterEntrypoint >>> > > > > > > > > with extracted job graph. Follow the JobDeployer way we'd >>> > better >>> > > > > > > > > align entry point of per-job deployment at JobDeployer. >>> > Users run >>> > > > > > > > > their main method or by a Cli(finally call main method) to >>> > deploy >>> > > > > the >>> > > > > > > > > job cluster. >>> > > > > > > > > >>> > > > > > > > > Best, >>> > > > > > > > > tison. >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > Stephan Ewen <se...@apache.org> 于2019年8月20日周二 下午4:40写道： >>> > > > > > > > > >>> > > > > > > > > > Till has made some good comments here. >>> > > > > > > > > > >>> > > > > > > > > > Two things to add: >>> > > > > > > > > > >>> > > > > > > > > > - The job mode is very nice in the way that it runs the >>> > > > client >>> > > > > > inside >>> > > > > > > > > the >>> > > > > > > > > > cluster (in the same image/process that is the JM) and >>> > > > > > > > > > thus >>> > > > > unifies >>> > > > > > > > both >>> > > > > > > > > > applications and what the Spark world calls the "driver >>> > mode". >>> > > > > > > > > > >>> > > > > > > > > > - Another thing I would add is that during the FLIP-6 >>> > design, >>> > > > > we >>> > > > > > were >>> > > > > > > > > > thinking about setups where Dispatcher and JobManager are >>> > > > > separate >>> > > > > > > > > > processes. >>> > > > > > > > > > A Yarn or Mesos Dispatcher of a session could run >>> > > > > independently >>> > > > > > > > (even >>> > > > > > > > > > as privileged processes executing no code). >>> > > > > > > > > > Then you the "per-job" mode could still be helpful: >>> > when a >>> > > > > job >>> > > > > > is >>> > > > > > > > > > submitted to the dispatcher, it launches the JM again in a >>> > > > > per-job >>> > > > > > > > mode, >>> > > > > > > > > so >>> > > > > > > > > > that JM and TM processes are bound to teh job only. For >>> > higher >>> > > > > > security >>> > > > > > > > > > setups, it is important that processes are not reused >>> > across >>> > > > > jobs. >>> > > > > > > > > > >>> > > > > > > > > > On Tue, Aug 20, 2019 at 10:27 AM Till Rohrmann < >>> > > > > > trohrm...@apache.org> >>> > > > > > > > > > wrote: >>> > > > > > > > > > >>> > > > > > > > > > > I would not be in favour of getting rid of the per-job >>> > mode >>> > > > > > since it >>> > > > > > > > > > > simplifies the process of running Flink jobs >>> > considerably. >>> > > > > > Moreover, >>> > > > > > > > it >>> > > > > > > > > > is >>> > > > > > > > > > > not only well suited for container deployments but also >>> > for >>> > > > > > > > deployments >>> > > > > > > > > > > where you want to guarantee job isolation. For example, >>> > > > > > > > > > > a >>> > > > user >>> > > > > > could >>> > > > > > > > > use >>> > > > > > > > > > > the per-job mode on Yarn to execute his job on a >>> > > > > > > > > > > separate >>> > > > > > cluster. >>> > > > > > > > > > > >>> > > > > > > > > > > I think that having two notions of cluster deployments >>> > > > (session >>> > > > > > vs. >>> > > > > > > > > > per-job >>> > > > > > > > > > > mode) does not necessarily contradict your ideas for the >>> > > > client >>> > > > > > api >>> > > > > > > > > > > refactoring. For example one could have the following >>> > > > > interfaces: >>> > > > > > > > > > > >>> > > > > > > > > > > - ClusterDeploymentDescriptor: encapsulates the logic >>> > how to >>> > > > > > deploy a >>> > > > > > > > > > > cluster. >>> > > > > > > > > > > - ClusterClient: allows to interact with a cluster >>> > > > > > > > > > > - JobClient: allows to interact with a running job >>> > > > > > > > > > > >>> > > > > > > > > > > Now the ClusterDeploymentDescriptor could have two >>> > methods: >>> > > > > > > > > > > >>> > > > > > > > > > > - ClusterClient deploySessionCluster() >>> > > > > > > > > > > - JobClusterClient/JobClient >>> > deployPerJobCluster(JobGraph) >>> > > > > > > > > > > >>> > > > > > > > > > > where JobClusterClient is either a supertype of >>> > ClusterClient >>> > > > > > which >>> > > > > > > > > does >>> > > > > > > > > > > not give you the functionality to submit jobs or >>> > > > > > deployPerJobCluster >>> > > > > > > > > > > returns directly a JobClient. >>> > > > > > > > > > > >>> > > > > > > > > > > When setting up the ExecutionEnvironment, one would then >>> > not >>> > > > > > provide >>> > > > > > > > a >>> > > > > > > > > > > ClusterClient to submit jobs but a JobDeployer which, >>> > > > depending >>> > > > > > on >>> > > > > > > > the >>> > > > > > > > > > > selected mode, either uses a ClusterClient (session >>> > mode) to >>> > > > > > submit >>> > > > > > > > > jobs >>> > > > > > > > > > or >>> > > > > > > > > > > a ClusterDeploymentDescriptor to deploy per a job mode >>> > > > cluster >>> > > > > > with >>> > > > > > > > the >>> > > > > > > > > > job >>> > > > > > > > > > > to execute. >>> > > > > > > > > > > >>> > > > > > > > > > > These are just some thoughts how one could make it >>> > working >>> > > > > > because I >>> > > > > > > > > > > believe there is some value in using the per job mode >>> > from >>> > > > the >>> > > > > > > > > > > ExecutionEnvironment. >>> > > > > > > > > > > >>> > > > > > > > > > > Concerning the web submission, this is indeed a bit >>> > tricky. >>> > > > > From >>> > > > > > a >>> > > > > > > > > > cluster >>> > > > > > > > > > > management stand point, I would in favour of not >>> > executing >>> > > > user >>> > > > > > code >>> > > > > > > > on >>> > > > > > > > > > the >>> > > > > > > > > > > REST endpoint. Especially when considering security, it >>> > would >>> > > > > be >>> > > > > > good >>> > > > > > > > > to >>> > > > > > > > > > > have a well defined cluster behaviour where it is >>> > explicitly >>> > > > > > stated >>> > > > > > > > > where >>> > > > > > > > > > > user code and, thus, potentially risky code is executed. >>> > > > > Ideally >>> > > > > > we >>> > > > > > > > > limit >>> > > > > > > > > > > it to the TaskExecutor and JobMaster. >>> > > > > > > > > > > >>> > > > > > > > > > > Cheers, >>> > > > > > > > > > > Till >>> > > > > > > > > > > >>> > > > > > > > > > > On Tue, Aug 20, 2019 at 9:40 AM Flavio Pompermaier < >>> > > > > > > > > pomperma...@okkam.it >>> > > > > > > > > > > >>> > > > > > > > > > > wrote: >>> > > > > > > > > > > >>> > > > > > > > > > > > In my opinion the client should not use any >>> > environment to >>> > > > > get >>> > > > > > the >>> > > > > > > > > Job >>> > > > > > > > > > > > graph because the jar should reside ONLY on the >>> > > > > > > > > > > > cluster >>> > > > (and >>> > > > > > not in >>> > > > > > > > > the >>> > > > > > > > > > > > client classpath otherwise there are always >>> > inconsistencies >>> > > > > > between >>> > > > > > > > > > > client >>> > > > > > > > > > > > and Flink Job manager's classpath). >>> > > > > > > > > > > > In the YARN, Mesos and Kubernetes scenarios you have >>> > the >>> > > > jar >>> > > > > > but >>> > > > > > > > you >>> > > > > > > > > > > could >>> > > > > > > > > > > > start a cluster that has the jar on the Job Manager as >>> > well >>> > > > > > (but >>> > > > > > > > this >>> > > > > > > > > > is >>> > > > > > > > > > > > the only case where I think you can assume that the >>> > client >>> > > > > has >>> > > > > > the >>> > > > > > > > > jar >>> > > > > > > > > > on >>> > > > > > > > > > > > the classpath..in the REST job submission you don't >>> > have >>> > > > any >>> > > > > > > > > > classpath). >>> > > > > > > > > > > > >>> > > > > > > > > > > > Thus, always in my opinion, the JobGraph should be >>> > > > generated >>> > > > > > by the >>> > > > > > > > > Job >>> > > > > > > > > > > > Manager REST API. >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > On Tue, Aug 20, 2019 at 9:00 AM Zili Chen < >>> > > > > > wander4...@gmail.com> >>> > > > > > > > > > wrote: >>> > > > > > > > > > > > >>> > > > > > > > > > > >> I would like to involve Till & Stephan here to >>> > > > > > > > > > > >> clarify >>> > > > some >>> > > > > > > > concept >>> > > > > > > > > of >>> > > > > > > > > > > >> per-job mode. >>> > > > > > > > > > > >> >>> > > > > > > > > > > >> The term per-job is one of modes a cluster could run >>> > on. >>> > > > It >>> > > > > is >>> > > > > > > > > mainly >>> > > > > > > > > > > >> aimed >>> > > > > > > > > > > >> at spawn >>> > > > > > > > > > > >> a dedicated cluster for a specific job while the job >>> > could >>> > > > > be >>> > > > > > > > > packaged >>> > > > > > > > > > > >> with >>> > > > > > > > > > > >> Flink >>> > > > > > > > > > > >> itself and thus the cluster initialized with job so >>> > that >>> > > > get >>> > > > > > rid >>> > > > > > > > of >>> > > > > > > > > a >>> > > > > > > > > > > >> separated >>> > > > > > > > > > > >> submission step. >>> > > > > > > > > > > >> >>> > > > > > > > > > > >> This is useful for container deployments where one >>> > create >>> > > > > his >>> > > > > > > > image >>> > > > > > > > > > with >>> > > > > > > > > > > >> the job >>> > > > > > > > > > > >> and then simply deploy the container. >>> > > > > > > > > > > >> >>> > > > > > > > > > > >> However, it is out of client scope since a >>> > > > > > client(ClusterClient >>> > > > > > > > for >>> > > > > > > > > > > >> example) is for >>> > > > > > > > > > > >> communicate with an existing cluster and performance >>> > > > > actions. >>> > > > > > > > > > Currently, >>> > > > > > > > > > > >> in >>> > > > > > > > > > > >> per-job >>> > > > > > > > > > > >> mode, we extract the job graph and bundle it into >>> > cluster >>> > > > > > > > deployment >>> > > > > > > > > > and >>> > > > > > > > > > > >> thus no >>> > > > > > > > > > > >> concept of client get involved. It looks like >>> > reasonable >>> > > > to >>> > > > > > > > exclude >>> > > > > > > > > > the >>> > > > > > > > > > > >> deployment >>> > > > > > > > > > > >> of per-job cluster from client api and use dedicated >>> > > > utility >>> > > > > > > > > > > >> classes(deployers) for >>> > > > > > > > > > > >> deployment. >>> > > > > > > > > > > >> >>> > > > > > > > > > > >> Zili Chen <wander4...@gmail.com> 于2019年8月20日周二 >>> > 下午12:37写道： >>> > > > > > > > > > > >> >>> > > > > > > > > > > >> > Hi Aljoscha, >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > Thanks for your reply and participance. The Google >>> > Doc >>> > > > you >>> > > > > > > > linked >>> > > > > > > > > to >>> > > > > > > > > > > >> > requires >>> > > > > > > > > > > >> > permission and I think you could use a share link >>> > > > instead. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > I agree with that we almost reach a consensus that >>> > > > > > JobClient is >>> > > > > > > > > > > >> necessary >>> > > > > > > > > > > >> > to >>> > > > > > > > > > > >> > interacte with a running Job. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > Let me check your open questions one by one. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > 1. Separate cluster creation and job submission for >>> > > > > per-job >>> > > > > > > > mode. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > As you mentioned here is where the opinions >>> > diverge. In >>> > > > my >>> > > > > > > > > document >>> > > > > > > > > > > >> there >>> > > > > > > > > > > >> > is >>> > > > > > > > > > > >> > an alternative[2] that proposes excluding per-job >>> > > > > deployment >>> > > > > > > > from >>> > > > > > > > > > > client >>> > > > > > > > > > > >> > api >>> > > > > > > > > > > >> > scope and now I find it is more reasonable we do >>> > > > > > > > > > > >> > the >>> > > > > > exclusion. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > When in per-job mode, a dedicated JobCluster is >>> > launched >>> > > > > to >>> > > > > > > > > execute >>> > > > > > > > > > > the >>> > > > > > > > > > > >> > specific job. It is like a Flink Application more >>> > than a >>> > > > > > > > > submission >>> > > > > > > > > > > >> > of Flink Job. Client only takes care of job >>> > submission >>> > > > and >>> > > > > > > > assume >>> > > > > > > > > > > there >>> > > > > > > > > > > >> is >>> > > > > > > > > > > >> > an existing cluster. In this way we are able to >>> > consider >>> > > > > > per-job >>> > > > > > > > > > > issues >>> > > > > > > > > > > >> > individually and JobClusterEntrypoint would be the >>> > > > utility >>> > > > > > class >>> > > > > > > > > for >>> > > > > > > > > > > >> > per-job >>> > > > > > > > > > > >> > deployment. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > Nevertheless, user program works in both session >>> > mode >>> > > > and >>> > > > > > > > per-job >>> > > > > > > > > > mode >>> > > > > > > > > > > >> > without >>> > > > > > > > > > > >> > necessary to change code. JobClient in per-job mode >>> > is >>> > > > > > returned >>> > > > > > > > > from >>> > > > > > > > > > > >> > env.execute as normal. However, it would be no >>> > longer a >>> > > > > > wrapper >>> > > > > > > > of >>> > > > > > > > > > > >> > RestClusterClient but a wrapper of >>> > PerJobClusterClient >>> > > > > which >>> > > > > > > > > > > >> communicates >>> > > > > > > > > > > >> > to Dispatcher locally. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > 2. How to deal with plan preview. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > With env.compile functions users can get JobGraph >>> > > > > > > > > > > >> > or >>> > > > > > FlinkPlan >>> > > > > > > > and >>> > > > > > > > > > > thus >>> > > > > > > > > > > >> > they can preview the plan with programming. >>> > Typically it >>> > > > > > looks >>> > > > > > > > > like >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > if (preview configured) { >>> > > > > > > > > > > >> > FlinkPlan plan = env.compile(); >>> > > > > > > > > > > >> > new JSONDumpGenerator(...).dump(plan); >>> > > > > > > > > > > >> > } else { >>> > > > > > > > > > > >> > env.execute(); >>> > > > > > > > > > > >> > } >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > And `flink info` would be invalid any more. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > 3. How to deal with Jar Submission at the Web >>> > Frontend. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > There is one more thread talked on this topic[1]. >>> > Apart >>> > > > > from >>> > > > > > > > > > removing >>> > > > > > > > > > > >> > the functions there are two alternatives. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > One is to introduce an interface has a method >>> > returns >>> > > > > > > > > > > JobGraph/FilnkPlan >>> > > > > > > > > > > >> > and Jar Submission only support main-class >>> > implements >>> > > > this >>> > > > > > > > > > interface. >>> > > > > > > > > > > >> > And then extract the JobGraph/FlinkPlan just by >>> > calling >>> > > > > the >>> > > > > > > > > method. >>> > > > > > > > > > > >> > In this way, it is even possible to consider a >>> > > > separation >>> > > > > > of job >>> > > > > > > > > > > >> creation >>> > > > > > > > > > > >> > and job submission. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > The other is, as you mentioned, let execute() do >>> > > > > > > > > > > >> > the >>> > > > > actual >>> > > > > > > > > > execution. >>> > > > > > > > > > > >> > We won't execute the main method in the WebFrontend >>> > but >>> > > > > > spawn a >>> > > > > > > > > > > process >>> > > > > > > > > > > >> > at WebMonitor side to execute. For return part we >>> > could >>> > > > > > generate >>> > > > > > > > > the >>> > > > > > > > > > > >> > JobID from WebMonitor and pass it to the execution >>> > > > > > environemnt. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > 4. How to deal with detached mode. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > I think detached mode is a temporary solution for >>> > > > > > non-blocking >>> > > > > > > > > > > >> submission. >>> > > > > > > > > > > >> > In my document both submission and execution return >>> > a >>> > > > > > > > > > > CompletableFuture >>> > > > > > > > > > > >> and >>> > > > > > > > > > > >> > users control whether or not wait for the result. >>> > > > > > > > > > > >> > In >>> > > > this >>> > > > > > point >>> > > > > > > > we >>> > > > > > > > > > > don't >>> > > > > > > > > > > >> > need a detached option but the functionality is >>> > covered. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > 5. How does per-job mode interact with interactive >>> > > > > > programming. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > All of YARN, Mesos and Kubernetes scenarios follow >>> > the >>> > > > > > pattern >>> > > > > > > > > > launch >>> > > > > > > > > > > a >>> > > > > > > > > > > >> > JobCluster now. And I don't think there would be >>> > > > > > inconsistency >>> > > > > > > > > > between >>> > > > > > > > > > > >> > different resource management. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > Best, >>> > > > > > > > > > > >> > tison. >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > [1] >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://lists.apache.org/x/thread.html/6db869c53816f4e2917949a7c6992c2b90856d7d639d7f2e1cd13768@%3Cdev.flink.apache.org%3E >>> > > > > > > > > > > >> > [2] >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://docs.google.com/document/d/1UWJE7eYWiMuZewBKS0YmdVO2LUTqXPd6-pbOCof9ddY/edit?disco=AAAADZaGGfs >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> > Aljoscha Krettek <aljos...@apache.org> >>> > 于2019年8月16日周五 >>> > > > > > 下午9:20写道： >>> > > > > > > > > > > >> > >>> > > > > > > > > > > >> >> Hi, >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> I read both Jeffs initial design document and the >>> > newer >>> > > > > > > > document >>> > > > > > > > > by >>> > > > > > > > > > > >> >> Tison. I also finally found the time to collect >>> > > > > > > > > > > >> >> our >>> > > > > > thoughts on >>> > > > > > > > > the >>> > > > > > > > > > > >> issue, >>> > > > > > > > > > > >> >> I had quite some discussions with Kostas and this >>> > is >>> > > > the >>> > > > > > > > result: >>> > > > > > > > > > [1]. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> I think overall we agree that this part of the >>> > code is >>> > > > in >>> > > > > > dire >>> > > > > > > > > need >>> > > > > > > > > > > of >>> > > > > > > > > > > >> >> some refactoring/improvements but I think there >>> > > > > > > > > > > >> >> are >>> > > > still >>> > > > > > some >>> > > > > > > > > open >>> > > > > > > > > > > >> >> questions and some differences in opinion what >>> > those >>> > > > > > > > refactorings >>> > > > > > > > > > > >> should >>> > > > > > > > > > > >> >> look like. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> I think the API-side is quite clear, i.e. we need >>> > some >>> > > > > > > > JobClient >>> > > > > > > > > > API >>> > > > > > > > > > > >> that >>> > > > > > > > > > > >> >> allows interacting with a running Job. It could be >>> > > > > > worthwhile >>> > > > > > > > to >>> > > > > > > > > > spin >>> > > > > > > > > > > >> that >>> > > > > > > > > > > >> >> off into a separate FLIP because we can probably >>> > find >>> > > > > > consensus >>> > > > > > > > > on >>> > > > > > > > > > > that >>> > > > > > > > > > > >> >> part more easily. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> For the rest, the main open questions from our doc >>> > are >>> > > > > > these: >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> - Do we want to separate cluster creation and >>> > > > > > > > > > > >> >> job >>> > > > > > submission >>> > > > > > > > > for >>> > > > > > > > > > > >> >> per-job mode? In the past, there were conscious >>> > efforts >>> > > > > to >>> > > > > > > > *not* >>> > > > > > > > > > > >> separate >>> > > > > > > > > > > >> >> job submission from cluster creation for per-job >>> > > > clusters >>> > > > > > for >>> > > > > > > > > > Mesos, >>> > > > > > > > > > > >> YARN, >>> > > > > > > > > > > >> >> Kubernets (see StandaloneJobClusterEntryPoint). >>> > Tison >>> > > > > > suggests >>> > > > > > > > in >>> > > > > > > > > > his >>> > > > > > > > > > > >> >> design document to decouple this in order to unify >>> > job >>> > > > > > > > > submission. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> - How to deal with plan preview, which needs to >>> > > > hijack >>> > > > > > > > > execute() >>> > > > > > > > > > > and >>> > > > > > > > > > > >> >> let the outside code catch an exception? >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> - How to deal with Jar Submission at the Web >>> > > > Frontend, >>> > > > > > which >>> > > > > > > > > > needs >>> > > > > > > > > > > to >>> > > > > > > > > > > >> >> hijack execute() and let the outside code catch an >>> > > > > > exception? >>> > > > > > > > > > > >> >> CliFrontend.run() “hijacks” >>> > > > > ExecutionEnvironment.execute() >>> > > > > > to >>> > > > > > > > > get a >>> > > > > > > > > > > >> >> JobGraph and then execute that JobGraph manually. >>> > We >>> > > > > could >>> > > > > > get >>> > > > > > > > > > around >>> > > > > > > > > > > >> that >>> > > > > > > > > > > >> >> by letting execute() do the actual execution. One >>> > > > caveat >>> > > > > > for >>> > > > > > > > this >>> > > > > > > > > > is >>> > > > > > > > > > > >> that >>> > > > > > > > > > > >> >> now the main() method doesn’t return (or is forced >>> > to >>> > > > > > return by >>> > > > > > > > > > > >> throwing an >>> > > > > > > > > > > >> >> exception from execute()) which means that for Jar >>> > > > > > Submission >>> > > > > > > > > from >>> > > > > > > > > > > the >>> > > > > > > > > > > >> >> WebFrontend we have a long-running main() method >>> > > > running >>> > > > > > in the >>> > > > > > > > > > > >> >> WebFrontend. This doesn’t sound very good. We >>> > could get >>> > > > > > around >>> > > > > > > > > this >>> > > > > > > > > > > by >>> > > > > > > > > > > >> >> removing the plan preview feature and by removing >>> > Jar >>> > > > > > > > > > > >> Submission/Running. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> - How to deal with detached mode? Right now, >>> > > > > > > > > DetachedEnvironment >>> > > > > > > > > > > will >>> > > > > > > > > > > >> >> execute the job and return immediately. If users >>> > > > control >>> > > > > > when >>> > > > > > > > > they >>> > > > > > > > > > > >> want to >>> > > > > > > > > > > >> >> return, by waiting on the job completion future, >>> > how do >>> > > > > we >>> > > > > > deal >>> > > > > > > > > > with >>> > > > > > > > > > > >> this? >>> > > > > > > > > > > >> >> Do we simply remove the distinction between >>> > > > > > > > > detached/non-detached? >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> - How does per-job mode interact with >>> > “interactive >>> > > > > > > > programming” >>> > > > > > > > > > > >> >> (FLIP-36). For YARN, each execute() call could >>> > spawn a >>> > > > > new >>> > > > > > > > Flink >>> > > > > > > > > > YARN >>> > > > > > > > > > > >> >> cluster. What about Mesos and Kubernetes? >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> The first open question is where the opinions >>> > diverge, >>> > > > I >>> > > > > > think. >>> > > > > > > > > The >>> > > > > > > > > > > >> rest >>> > > > > > > > > > > >> >> are just open questions and interesting things >>> > that we >>> > > > > > need to >>> > > > > > > > > > > >> consider. >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> Best, >>> > > > > > > > > > > >> >> Aljoscha >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> [1] >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://docs.google.com/document/d/1E-8UjOLz4QPUTxetGWbU23OlsIH9VIdodpTsxwoQTs0/edit#heading=h.na7k0ad88tix >>> > > > > > > > > > > >> >> < >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://docs.google.com/document/d/1E-8UjOLz4QPUTxetGWbU23OlsIH9VIdodpTsxwoQTs0/edit#heading=h.na7k0ad88tix >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> > On 31. Jul 2019, at 15:23, Jeff Zhang < >>> > > > > zjf...@gmail.com> >>> > > > > > > > > wrote: >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > Thanks tison for the effort. I left a few >>> > comments. >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > Zili Chen <wander4...@gmail.com> 于2019年7月31日周三 >>> > > > > 下午8:24写道： >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> >> Hi Flavio, >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> Thanks for your reply. >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> Either current impl and in the design, >>> > ClusterClient >>> > > > > > > > > > > >> >> >> never takes responsibility for generating >>> > JobGraph. >>> > > > > > > > > > > >> >> >> (what you see in current codebase is several >>> > class >>> > > > > > methods) >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> Instead, user describes his program in the main >>> > > > method >>> > > > > > > > > > > >> >> >> with ExecutionEnvironment apis and calls >>> > > > env.compile() >>> > > > > > > > > > > >> >> >> or env.optimize() to get FlinkPlan and JobGraph >>> > > > > > > > respectively. >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> For listing main classes in a jar and choose >>> > one for >>> > > > > > > > > > > >> >> >> submission, you're now able to customize a CLI >>> > to do >>> > > > > it. >>> > > > > > > > > > > >> >> >> Specifically, the path of jar is passed as >>> > arguments >>> > > > > and >>> > > > > > > > > > > >> >> >> in the customized CLI you list main classes, >>> > choose >>> > > > > one >>> > > > > > > > > > > >> >> >> to submit to the cluster. >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> Best, >>> > > > > > > > > > > >> >> >> tison. >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >> Flavio Pompermaier <pomperma...@okkam.it> >>> > > > > 于2019年7月31日周三 >>> > > > > > > > > > 下午8:12写道： >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> Just one note on my side: it is not clear to >>> > > > > > > > > > > >> >> >>> me >>> > > > > > whether the >>> > > > > > > > > > > client >>> > > > > > > > > > > >> >> needs >>> > > > > > > > > > > >> >> >> to >>> > > > > > > > > > > >> >> >>> be able to generate a job graph or not. >>> > > > > > > > > > > >> >> >>> In my opinion, the job jar must resides only >>> > on the >>> > > > > > > > > > > >> server/jobManager >>> > > > > > > > > > > >> >> >> side >>> > > > > > > > > > > >> >> >>> and the client requires a way to get the job >>> > graph. >>> > > > > > > > > > > >> >> >>> If you really want to access to the job graph, >>> > I'd >>> > > > > add >>> > > > > > a >>> > > > > > > > > > > dedicated >>> > > > > > > > > > > >> >> method >>> > > > > > > > > > > >> >> >>> on the ClusterClient. like: >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >>> - getJobGraph(jarId, mainClass): JobGraph >>> > > > > > > > > > > >> >> >>> - listMainClasses(jarId): List<String> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >>> These would require some addition also on the >>> > job >>> > > > > > manager >>> > > > > > > > > > > endpoint >>> > > > > > > > > > > >> as >>> > > > > > > > > > > >> >> >>> well..what do you think? >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >>> On Wed, Jul 31, 2019 at 12:42 PM Zili Chen < >>> > > > > > > > > > wander4...@gmail.com >>> > > > > > > > > > > > >>> > > > > > > > > > > >> >> wrote: >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >>>> Hi all, >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> Here is a document[1] on client api >>> > enhancement >>> > > > from >>> > > > > > our >>> > > > > > > > > > > >> perspective. >>> > > > > > > > > > > >> >> >>>> We have investigated current implementations. >>> > And >>> > > > we >>> > > > > > > > propose >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> 1. Unify the implementation of cluster >>> > deployment >>> > > > > and >>> > > > > > job >>> > > > > > > > > > > >> submission >>> > > > > > > > > > > >> >> in >>> > > > > > > > > > > >> >> >>>> Flink. >>> > > > > > > > > > > >> >> >>>> 2. Provide programmatic interfaces to allow >>> > > > flexible >>> > > > > > job >>> > > > > > > > and >>> > > > > > > > > > > >> cluster >>> > > > > > > > > > > >> >> >>>> management. >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> The first proposal is aimed at reducing code >>> > paths >>> > > > > of >>> > > > > > > > > cluster >>> > > > > > > > > > > >> >> >> deployment >>> > > > > > > > > > > >> >> >>>> and >>> > > > > > > > > > > >> >> >>>> job submission so that one can adopt Flink in >>> > his >>> > > > > > usage >>> > > > > > > > > > easily. >>> > > > > > > > > > > >> The >>> > > > > > > > > > > >> >> >>> second >>> > > > > > > > > > > >> >> >>>> proposal is aimed at providing rich >>> > interfaces for >>> > > > > > > > advanced >>> > > > > > > > > > > users >>> > > > > > > > > > > >> >> >>>> who want to make accurate control of these >>> > stages. >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> Quick reference on open questions: >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> 1. Exclude job cluster deployment from client >>> > side >>> > > > > or >>> > > > > > > > > redefine >>> > > > > > > > > > > the >>> > > > > > > > > > > >> >> >>> semantic >>> > > > > > > > > > > >> >> >>>> of job cluster? Since it fits in a process >>> > quite >>> > > > > > different >>> > > > > > > > > > from >>> > > > > > > > > > > >> >> session >>> > > > > > > > > > > >> >> >>>> cluster deployment and job submission. >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> 2. Maintain the codepaths handling class >>> > > > > > > > > > > o.a.f.api.common.Program >>> > > > > > > > > > > >> or >>> > > > > > > > > > > >> >> >>>> implement customized program handling logic >>> > > > > > > > > > > >> >> >>>> by >>> > > > > > customized >>> > > > > > > > > > > >> >> CliFrontend? >>> > > > > > > > > > > >> >> >>>> See also this thread[2] and the document[1]. >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> 3. Expose ClusterClient as public api or just >>> > > > expose >>> > > > > > api >>> > > > > > > > in >>> > > > > > > > > > > >> >> >>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>> and delegate them to ClusterClient? Further, >>> > in >>> > > > > > either way >>> > > > > > > > > is >>> > > > > > > > > > it >>> > > > > > > > > > > >> >> worth >>> > > > > > > > > > > >> >> >> to >>> > > > > > > > > > > >> >> >>>> introduce a JobClient which is an >>> > encapsulation of >>> > > > > > > > > > ClusterClient >>> > > > > > > > > > > >> that >>> > > > > > > > > > > >> >> >>>> associated to specific job? >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> Best, >>> > > > > > > > > > > >> >> >>>> tison. >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> [1] >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://docs.google.com/document/d/1UWJE7eYWiMuZewBKS0YmdVO2LUTqXPd6-pbOCof9ddY/edit?usp=sharing >>> > > > > > > > > > > >> >> >>>> [2] >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://lists.apache.org/thread.html/7ffc9936a384b891dbcf0a481d26c6d13b2125607c200577780d1e18@%3Cdev.flink.apache.org%3E >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>> Jeff Zhang <zjf...@gmail.com> 于2019年7月24日周三 >>> > > > > 上午9:19写道： >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>>>> Thanks Stephan, I will follow up this issue >>> > in >>> > > > next >>> > > > > > few >>> > > > > > > > > > weeks, >>> > > > > > > > > > > >> and >>> > > > > > > > > > > >> >> >> will >>> > > > > > > > > > > >> >> >>>>> refine the design doc. We could discuss more >>> > > > > details >>> > > > > > > > after >>> > > > > > > > > > 1.9 >>> > > > > > > > > > > >> >> >> release. >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>> Stephan Ewen <se...@apache.org> >>> > 于2019年7月24日周三 >>> > > > > > 上午12:58写道： >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>>> Hi all! >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> This thread has stalled for a bit, which I >>> > > > assume >>> > > > > > ist >>> > > > > > > > > mostly >>> > > > > > > > > > > >> due to >>> > > > > > > > > > > >> >> >>> the >>> > > > > > > > > > > >> >> >>>>>> Flink 1.9 feature freeze and release >>> > > > > > > > > > > >> >> >>>>>> testing >>> > > > > effort. >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> I personally still recognize this issue as >>> > one >>> > > > > > important >>> > > > > > > > > to >>> > > > > > > > > > be >>> > > > > > > > > > > >> >> >>> solved. >>> > > > > > > > > > > >> >> >>>>> I'd >>> > > > > > > > > > > >> >> >>>>>> be happy to help resume this discussion >>> > > > > > > > > > > >> >> >>>>>> soon >>> > > > > (after >>> > > > > > the >>> > > > > > > > > 1.9 >>> > > > > > > > > > > >> >> >> release) >>> > > > > > > > > > > >> >> >>>> and >>> > > > > > > > > > > >> >> >>>>>> see if we can do some step towards this in >>> > Flink >>> > > > > > 1.10. >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> Best, >>> > > > > > > > > > > >> >> >>>>>> Stephan >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>> On Mon, Jun 24, 2019 at 10:41 AM Flavio >>> > > > > Pompermaier >>> > > > > > < >>> > > > > > > > > > > >> >> >>>>> pomperma...@okkam.it> >>> > > > > > > > > > > >> >> >>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>>>> That's exactly what I suggested a long >>> > > > > > > > > > > >> >> >>>>>>> time >>> > > > ago: >>> > > > > > the >>> > > > > > > > > Flink >>> > > > > > > > > > > REST >>> > > > > > > > > > > >> >> >>>> client >>> > > > > > > > > > > >> >> >>>>>>> should not require any Flink dependency, >>> > only >>> > > > > http >>> > > > > > > > > library >>> > > > > > > > > > to >>> > > > > > > > > > > >> >> >> call >>> > > > > > > > > > > >> >> >>>> the >>> > > > > > > > > > > >> >> >>>>>> REST >>> > > > > > > > > > > >> >> >>>>>>> services to submit and monitor a job. >>> > > > > > > > > > > >> >> >>>>>>> What I suggested also in [1] was to have a >>> > way >>> > > > to >>> > > > > > > > > > > automatically >>> > > > > > > > > > > >> >> >>>> suggest >>> > > > > > > > > > > >> >> >>>>>> the >>> > > > > > > > > > > >> >> >>>>>>> user (via a UI) the available main classes >>> > and >>> > > > > > their >>> > > > > > > > > > required >>> > > > > > > > > > > >> >> >>>>>>> parameters[2]. >>> > > > > > > > > > > >> >> >>>>>>> Another problem we have with Flink is that >>> > the >>> > > > > Rest >>> > > > > > > > > client >>> > > > > > > > > > > and >>> > > > > > > > > > > >> >> >> the >>> > > > > > > > > > > >> >> >>>> CLI >>> > > > > > > > > > > >> >> >>>>>> one >>> > > > > > > > > > > >> >> >>>>>>> behaves differently and we use the CLI >>> > client >>> > > > > (via >>> > > > > > ssh) >>> > > > > > > > > > > because >>> > > > > > > > > > > >> >> >> it >>> > > > > > > > > > > >> >> >>>>> allows >>> > > > > > > > > > > >> >> >>>>>>> to call some other method after >>> > env.execute() >>> > > > [3] >>> > > > > > (we >>> > > > > > > > > have >>> > > > > > > > > > to >>> > > > > > > > > > > >> >> >> call >>> > > > > > > > > > > >> >> >>>>>> another >>> > > > > > > > > > > >> >> >>>>>>> REST service to signal the end of the >>> > > > > > > > > > > >> >> >>>>>>> job). >>> > > > > > > > > > > >> >> >>>>>>> Int his regard, a dedicated interface, >>> > like the >>> > > > > > > > > JobListener >>> > > > > > > > > > > >> >> >>> suggested >>> > > > > > > > > > > >> >> >>>>> in >>> > > > > > > > > > > >> >> >>>>>>> the previous emails, would be very helpful >>> > > > > (IMHO). >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> [1] >>> > > > > > https://issues.apache.org/jira/browse/FLINK-10864 >>> > > > > > > > > > > >> >> >>>>>>> [2] >>> > > > > > https://issues.apache.org/jira/browse/FLINK-10862 >>> > > > > > > > > > > >> >> >>>>>>> [3] >>> > > > > > https://issues.apache.org/jira/browse/FLINK-10879 >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> Best, >>> > > > > > > > > > > >> >> >>>>>>> Flavio >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> On Mon, Jun 24, 2019 at 9:54 AM Jeff Zhang >>> > < >>> > > > > > > > > > zjf...@gmail.com >>> > > > > > > > > > > > >>> > > > > > > > > > > >> >> >>> wrote: >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> Hi, Tison, >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> Thanks for your comments. Overall I agree >>> > with >>> > > > > you >>> > > > > > > > that >>> > > > > > > > > it >>> > > > > > > > > > > is >>> > > > > > > > > > > >> >> >>>>> difficult >>> > > > > > > > > > > >> >> >>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>> down stream project to integrate with >>> > flink >>> > > > and >>> > > > > we >>> > > > > > > > need >>> > > > > > > > > to >>> > > > > > > > > > > >> >> >>> refactor >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>>> current flink client api. >>> > > > > > > > > > > >> >> >>>>>>>> And I agree that CliFrontend should only >>> > > > parsing >>> > > > > > > > command >>> > > > > > > > > > > line >>> > > > > > > > > > > >> >> >>>>> arguments >>> > > > > > > > > > > >> >> >>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>> then pass them to ExecutionEnvironment. >>> > It is >>> > > > > > > > > > > >> >> >>>> ExecutionEnvironment's >>> > > > > > > > > > > >> >> >>>>>>>> responsibility to compile job, create >>> > cluster, >>> > > > > and >>> > > > > > > > > submit >>> > > > > > > > > > > job. >>> > > > > > > > > > > >> >> >>>>> Besides >>> > > > > > > > > > > >> >> >>>>>>>> that, Currently flink has many >>> > > > > > ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>> implementations, >>> > > > > > > > > > > >> >> >>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>> flink will use the specific one based on >>> > the >>> > > > > > context. >>> > > > > > > > > > IMHO, >>> > > > > > > > > > > it >>> > > > > > > > > > > >> >> >> is >>> > > > > > > > > > > >> >> >>>> not >>> > > > > > > > > > > >> >> >>>>>>>> necessary, ExecutionEnvironment should be >>> > able >>> > > > > to >>> > > > > > do >>> > > > > > > > the >>> > > > > > > > > > > right >>> > > > > > > > > > > >> >> >>>> thing >>> > > > > > > > > > > >> >> >>>>>>> based >>> > > > > > > > > > > >> >> >>>>>>>> on the FlinkConf it is received. Too many >>> > > > > > > > > > > ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>> implementation is another burden for >>> > > > downstream >>> > > > > > > > project >>> > > > > > > > > > > >> >> >>>> integration. >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> One thing I'd like to mention is flink's >>> > scala >>> > > > > > shell >>> > > > > > > > and >>> > > > > > > > > > sql >>> > > > > > > > > > > >> >> >>>> client, >>> > > > > > > > > > > >> >> >>>>>>>> although they are sub-modules of flink, >>> > they >>> > > > > > could be >>> > > > > > > > > > > treated >>> > > > > > > > > > > >> >> >> as >>> > > > > > > > > > > >> >> >>>>>>> downstream >>> > > > > > > > > > > >> >> >>>>>>>> project which use flink's client api. >>> > > > Currently >>> > > > > > you >>> > > > > > > > will >>> > > > > > > > > > > find >>> > > > > > > > > > > >> >> >> it >>> > > > > > > > > > > >> >> >>> is >>> > > > > > > > > > > >> >> >>>>> not >>> > > > > > > > > > > >> >> >>>>>>>> easy for them to integrate with flink, >>> > they >>> > > > > share >>> > > > > > many >>> > > > > > > > > > > >> >> >> duplicated >>> > > > > > > > > > > >> >> >>>>> code. >>> > > > > > > > > > > >> >> >>>>>>> It >>> > > > > > > > > > > >> >> >>>>>>>> is another sign that we should refactor >>> > flink >>> > > > > > client >>> > > > > > > > > api. >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> I believe it is a large and hard change, >>> > and I >>> > > > > am >>> > > > > > > > afraid >>> > > > > > > > > > we >>> > > > > > > > > > > >> can >>> > > > > > > > > > > >> >> >>> not >>> > > > > > > > > > > >> >> >>>>>> keep >>> > > > > > > > > > > >> >> >>>>>>>> compatibility since many of changes are >>> > user >>> > > > > > facing. >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> Zili Chen <wander4...@gmail.com> >>> > > > 于2019年6月24日周一 >>> > > > > > > > > 下午2:53写道： >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Hi all, >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> After a closer look on our client apis, >>> > I can >>> > > > > see >>> > > > > > > > there >>> > > > > > > > > > are >>> > > > > > > > > > > >> >> >> two >>> > > > > > > > > > > >> >> >>>>> major >>> > > > > > > > > > > >> >> >>>>>>>>> issues to consistency and integration, >>> > namely >>> > > > > > > > different >>> > > > > > > > > > > >> >> >>>> deployment >>> > > > > > > > > > > >> >> >>>>> of >>> > > > > > > > > > > >> >> >>>>>>>>> job cluster which couples job graph >>> > creation >>> > > > > and >>> > > > > > > > > cluster >>> > > > > > > > > > > >> >> >>>>> deployment, >>> > > > > > > > > > > >> >> >>>>>>>>> and submission via CliFrontend confusing >>> > > > > control >>> > > > > > flow >>> > > > > > > > > of >>> > > > > > > > > > > job >>> > > > > > > > > > > >> >> >>>> graph >>> > > > > > > > > > > >> >> >>>>>>>>> compilation and job submission. I'd like >>> > to >>> > > > > > follow >>> > > > > > > > the >>> > > > > > > > > > > >> >> >> discuss >>> > > > > > > > > > > >> >> >>>>> above, >>> > > > > > > > > > > >> >> >>>>>>>>> mainly the process described by Jeff and >>> > > > > > Stephan, and >>> > > > > > > > > > share >>> > > > > > > > > > > >> >> >> my >>> > > > > > > > > > > >> >> >>>>>>>>> ideas on these issues. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> 1) CliFrontend confuses the control flow >>> > of >>> > > > job >>> > > > > > > > > > compilation >>> > > > > > > > > > > >> >> >> and >>> > > > > > > > > > > >> >> >>>>>>>> submission. >>> > > > > > > > > > > >> >> >>>>>>>>> Following the process of job submission >>> > > > Stephan >>> > > > > > and >>> > > > > > > > > Jeff >>> > > > > > > > > > > >> >> >>>> described, >>> > > > > > > > > > > >> >> >>>>>>>>> execution environment knows all configs >>> > of >>> > > > the >>> > > > > > > > cluster >>> > > > > > > > > > and >>> > > > > > > > > > > >> >> >>>>>>> topos/settings >>> > > > > > > > > > > >> >> >>>>>>>>> of the job. Ideally, in the main method >>> > of >>> > > > user >>> > > > > > > > > program, >>> > > > > > > > > > it >>> > > > > > > > > > > >> >> >>> calls >>> > > > > > > > > > > >> >> >>>>>>>> #execute >>> > > > > > > > > > > >> >> >>>>>>>>> (or named #submit) and Flink deploys the >>> > > > > cluster, >>> > > > > > > > > compile >>> > > > > > > > > > > the >>> > > > > > > > > > > >> >> >>> job >>> > > > > > > > > > > >> >> >>>>>> graph >>> > > > > > > > > > > >> >> >>>>>>>>> and submit it to the cluster. However, >>> > > > current >>> > > > > > > > > > CliFrontend >>> > > > > > > > > > > >> >> >> does >>> > > > > > > > > > > >> >> >>>> all >>> > > > > > > > > > > >> >> >>>>>>> these >>> > > > > > > > > > > >> >> >>>>>>>>> things inside its #runProgram method, >>> > which >>> > > > > > > > introduces >>> > > > > > > > > a >>> > > > > > > > > > > lot >>> > > > > > > > > > > >> >> >> of >>> > > > > > > > > > > >> >> >>>>>>>> subclasses >>> > > > > > > > > > > >> >> >>>>>>>>> of (stream) execution environment. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Actually, it sets up an exec env that >>> > hijacks >>> > > > > the >>> > > > > > > > > > > >> >> >>>>>> #execute/executePlan >>> > > > > > > > > > > >> >> >>>>>>>>> method, initializes the job graph and >>> > abort >>> > > > > > > > execution. >>> > > > > > > > > > And >>> > > > > > > > > > > >> >> >> then >>> > > > > > > > > > > >> >> >>>>>>>>> control flow back to CliFrontend, it >>> > deploys >>> > > > > the >>> > > > > > > > > > cluster(or >>> > > > > > > > > > > >> >> >>>>> retrieve >>> > > > > > > > > > > >> >> >>>>>>>>> the client) and submits the job graph. >>> > This >>> > > > is >>> > > > > > quite >>> > > > > > > > a >>> > > > > > > > > > > >> >> >> specific >>> > > > > > > > > > > >> >> >>>>>>> internal >>> > > > > > > > > > > >> >> >>>>>>>>> process inside Flink and none of >>> > consistency >>> > > > to >>> > > > > > > > > anything. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> 2) Deployment of job cluster couples job >>> > > > graph >>> > > > > > > > creation >>> > > > > > > > > > and >>> > > > > > > > > > > >> >> >>>> cluster >>> > > > > > > > > > > >> >> >>>>>>>>> deployment. Abstractly, from user job to >>> > a >>> > > > > > concrete >>> > > > > > > > > > > >> >> >> submission, >>> > > > > > > > > > > >> >> >>>> it >>> > > > > > > > > > > >> >> >>>>>>>> requires >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> create JobGraph --\ >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> create ClusterClient --> submit >>> > > > > > > > > > > >> >> >>>>>>>>> JobGraph >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> such a dependency. ClusterClient was >>> > created >>> > > > by >>> > > > > > > > > deploying >>> > > > > > > > > > > or >>> > > > > > > > > > > >> >> >>>>>>> retrieving. >>> > > > > > > > > > > >> >> >>>>>>>>> JobGraph submission requires a compiled >>> > > > > JobGraph >>> > > > > > and >>> > > > > > > > > > valid >>> > > > > > > > > > > >> >> >>>>>>> ClusterClient, >>> > > > > > > > > > > >> >> >>>>>>>>> but the creation of ClusterClient is >>> > > > abstractly >>> > > > > > > > > > independent >>> > > > > > > > > > > >> >> >> of >>> > > > > > > > > > > >> >> >>>> that >>> > > > > > > > > > > >> >> >>>>>> of >>> > > > > > > > > > > >> >> >>>>>>>>> JobGraph. However, in job cluster mode, >>> > we >>> > > > > > deploy job >>> > > > > > > > > > > cluster >>> > > > > > > > > > > >> >> >>>> with >>> > > > > > > > > > > >> >> >>>>> a >>> > > > > > > > > > > >> >> >>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>> graph, which means we use another >>> > process: >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> create JobGraph --> deploy cluster with >>> > the >>> > > > > > JobGraph >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Here is another inconsistency and >>> > downstream >>> > > > > > > > > > > projects/client >>> > > > > > > > > > > >> >> >>> apis >>> > > > > > > > > > > >> >> >>>>> are >>> > > > > > > > > > > >> >> >>>>>>>>> forced to handle different cases with >>> > rare >>> > > > > > supports >>> > > > > > > > > from >>> > > > > > > > > > > >> >> >> Flink. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Since we likely reached a consensus on >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> 1. all configs gathered by Flink >>> > > > configuration >>> > > > > > and >>> > > > > > > > > passed >>> > > > > > > > > > > >> >> >>>>>>>>> 2. execution environment knows all >>> > configs >>> > > > and >>> > > > > > > > handles >>> > > > > > > > > > > >> >> >>>>> execution(both >>> > > > > > > > > > > >> >> >>>>>>>>> deployment and submission) >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> to the issues above I propose >>> > > > > > > > > > > >> >> >>>>>>>>> eliminating >>> > > > > > > > > inconsistencies >>> > > > > > > > > > > by >>> > > > > > > > > > > >> >> >>>>>> following >>> > > > > > > > > > > >> >> >>>>>>>>> approach: >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> 1) CliFrontend should exactly be a front >>> > end, >>> > > > > at >>> > > > > > > > least >>> > > > > > > > > > for >>> > > > > > > > > > > >> >> >>> "run" >>> > > > > > > > > > > >> >> >>>>>>> command. >>> > > > > > > > > > > >> >> >>>>>>>>> That means it just gathered and passed >>> > all >>> > > > > config >>> > > > > > > > from >>> > > > > > > > > > > >> >> >> command >>> > > > > > > > > > > >> >> >>>> line >>> > > > > > > > > > > >> >> >>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>> the main method of user program. >>> > Execution >>> > > > > > > > environment >>> > > > > > > > > > > knows >>> > > > > > > > > > > >> >> >>> all >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>> info >>> > > > > > > > > > > >> >> >>>>>>>>> and with an addition to utils for >>> > > > > ClusterClient, >>> > > > > > we >>> > > > > > > > > > > >> >> >> gracefully >>> > > > > > > > > > > >> >> >>>> get >>> > > > > > > > > > > >> >> >>>>> a >>> > > > > > > > > > > >> >> >>>>>>>>> ClusterClient by deploying or >>> > retrieving. In >>> > > > > this >>> > > > > > > > way, >>> > > > > > > > > we >>> > > > > > > > > > > >> >> >> don't >>> > > > > > > > > > > >> >> >>>>> need >>> > > > > > > > > > > >> >> >>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>> hijack #execute/executePlan methods and >>> > can >>> > > > > > remove >>> > > > > > > > > > various >>> > > > > > > > > > > >> >> >>>> hacking >>> > > > > > > > > > > >> >> >>>>>>>>> subclasses of exec env, as well as #run >>> > > > methods >>> > > > > > in >>> > > > > > > > > > > >> >> >>>>> ClusterClient(for >>> > > > > > > > > > > >> >> >>>>>> an >>> > > > > > > > > > > >> >> >>>>>>>>> interface-ized ClusterClient). Now the >>> > > > control >>> > > > > > flow >>> > > > > > > > > flows >>> > > > > > > > > > > >> >> >> from >>> > > > > > > > > > > >> >> >>>>>>>> CliFrontend >>> > > > > > > > > > > >> >> >>>>>>>>> to the main method and never returns. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> 2) Job cluster means a cluster for the >>> > > > specific >>> > > > > > job. >>> > > > > > > > > From >>> > > > > > > > > > > >> >> >>> another >>> > > > > > > > > > > >> >> >>>>>>>>> perspective, it is an ephemeral session. >>> > We >>> > > > may >>> > > > > > > > > decouple >>> > > > > > > > > > > the >>> > > > > > > > > > > >> >> >>>>>> deployment >>> > > > > > > > > > > >> >> >>>>>>>>> with a compiled job graph, but start a >>> > > > session >>> > > > > > with >>> > > > > > > > > idle >>> > > > > > > > > > > >> >> >>> timeout >>> > > > > > > > > > > >> >> >>>>>>>>> and submit the job following. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> These topics, before we go into more >>> > details >>> > > > on >>> > > > > > > > design >>> > > > > > > > > or >>> > > > > > > > > > > >> >> >>>>>>> implementation, >>> > > > > > > > > > > >> >> >>>>>>>>> are better to be aware and discussed for >>> > a >>> > > > > > consensus. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Best, >>> > > > > > > > > > > >> >> >>>>>>>>> tison. >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>> Zili Chen <wander4...@gmail.com> >>> > > > 于2019年6月20日周四 >>> > > > > > > > > 上午3:21写道： >>> > > > > > > > > > > >> >> >>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> Hi Jeff, >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> Thanks for raising this thread and the >>> > > > design >>> > > > > > > > > document! >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> As @Thomas Weise mentioned above, >>> > extending >>> > > > > > config >>> > > > > > > > to >>> > > > > > > > > > > flink >>> > > > > > > > > > > >> >> >>>>>>>>>> requires far more effort than it should >>> > be. >>> > > > > > Another >>> > > > > > > > > > > example >>> > > > > > > > > > > >> >> >>>>>>>>>> is we achieve detach mode by introduce >>> > > > another >>> > > > > > > > > execution >>> > > > > > > > > > > >> >> >>>>>>>>>> environment which also hijack #execute >>> > > > method. >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> I agree with your idea that user would >>> > > > > > configure all >>> > > > > > > > > > > things >>> > > > > > > > > > > >> >> >>>>>>>>>> and flink "just" respect it. On this >>> > topic I >>> > > > > > think >>> > > > > > > > the >>> > > > > > > > > > > >> >> >> unusual >>> > > > > > > > > > > >> >> >>>>>>>>>> control flow when CliFrontend handle >>> > "run" >>> > > > > > command >>> > > > > > > > is >>> > > > > > > > > > the >>> > > > > > > > > > > >> >> >>>> problem. >>> > > > > > > > > > > >> >> >>>>>>>>>> It handles several configs, mainly >>> > > > > > > > > > > >> >> >>>>>>>>>> about >>> > > > > cluster >>> > > > > > > > > > settings, >>> > > > > > > > > > > >> >> >> and >>> > > > > > > > > > > >> >> >>>>>>>>>> thus main method of user program is >>> > unaware >>> > > > of >>> > > > > > them. >>> > > > > > > > > > Also >>> > > > > > > > > > > it >>> > > > > > > > > > > >> >> >>>>>> compiles >>> > > > > > > > > > > >> >> >>>>>>>>>> app to job graph by run the main method >>> > > > with a >>> > > > > > > > > hijacked >>> > > > > > > > > > > exec >>> > > > > > > > > > > >> >> >>>> env, >>> > > > > > > > > > > >> >> >>>>>>>>>> which constrain the main method >>> > > > > > > > > > > >> >> >>>>>>>>>> further. >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> I'd like to write down a few of notes >>> > > > > > > > > > > >> >> >>>>>>>>>> on >>> > > > > > > > configs/args >>> > > > > > > > > > pass >>> > > > > > > > > > > >> >> >> and >>> > > > > > > > > > > >> >> >>>>>>> respect, >>> > > > > > > > > > > >> >> >>>>>>>>>> as well as decoupling job compilation >>> > and >>> > > > > > > > submission. >>> > > > > > > > > > > Share >>> > > > > > > > > > > >> >> >> on >>> > > > > > > > > > > >> >> >>>>> this >>> > > > > > > > > > > >> >> >>>>>>>>>> thread later. >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> Best, >>> > > > > > > > > > > >> >> >>>>>>>>>> tison. >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> SHI Xiaogang <shixiaoga...@gmail.com> >>> > > > > > 于2019年6月17日周一 >>> > > > > > > > > > > >> >> >> 下午7:29写道： >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> Hi Jeff and Flavio, >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> Thanks Jeff a lot for proposing the >>> > design >>> > > > > > > > document. >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> We are also working on refactoring >>> > > > > > ClusterClient to >>> > > > > > > > > > allow >>> > > > > > > > > > > >> >> >>>>> flexible >>> > > > > > > > > > > >> >> >>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>> efficient job management in our >>> > real-time >>> > > > > > platform. >>> > > > > > > > > > > >> >> >>>>>>>>>>> We would like to draft a document to >>> > share >>> > > > > our >>> > > > > > > > ideas >>> > > > > > > > > > with >>> > > > > > > > > > > >> >> >>> you. >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> I think it's a good idea to have >>> > something >>> > > > > like >>> > > > > > > > > Apache >>> > > > > > > > > > > Livy >>> > > > > > > > > > > >> >> >>> for >>> > > > > > > > > > > >> >> >>>>>>> Flink, >>> > > > > > > > > > > >> >> >>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>> the efforts discussed here will take a >>> > > > great >>> > > > > > step >>> > > > > > > > > > forward >>> > > > > > > > > > > >> >> >> to >>> > > > > > > > > > > >> >> >>>> it. >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> Regards, >>> > > > > > > > > > > >> >> >>>>>>>>>>> Xiaogang >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> Flavio Pompermaier < >>> > pomperma...@okkam.it> >>> > > > > > > > > > 于2019年6月17日周一 >>> > > > > > > > > > > >> >> >>>>> 下午7:13写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> Is there any possibility to have >>> > something >>> > > > > > like >>> > > > > > > > > Apache >>> > > > > > > > > > > >> >> >> Livy >>> > > > > > > > > > > >> >> >>>> [1] >>> > > > > > > > > > > >> >> >>>>>>> also >>> > > > > > > > > > > >> >> >>>>>>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>> Flink in the future? >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> [1] https://livy.apache.org/ >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> On Tue, Jun 11, 2019 at 5:23 PM Jeff >>> > > > Zhang < >>> > > > > > > > > > > >> >> >>> zjf...@gmail.com >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> Any API we expose should not have >>> > > > > > dependencies >>> > > > > > > > > on >>> > > > > > > > > > > >> >> >>> the >>> > > > > > > > > > > >> >> >>>>>>> runtime >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> (flink-runtime) package or other >>> > > > > > implementation >>> > > > > > > > > > > >> >> >> details. >>> > > > > > > > > > > >> >> >>> To >>> > > > > > > > > > > >> >> >>>>> me, >>> > > > > > > > > > > >> >> >>>>>>>> this >>> > > > > > > > > > > >> >> >>>>>>>>>>>> means >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> that the current ClusterClient >>> > cannot be >>> > > > > > exposed >>> > > > > > > > to >>> > > > > > > > > > > >> >> >> users >>> > > > > > > > > > > >> >> >>>>>> because >>> > > > > > > > > > > >> >> >>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>> uses >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> quite some classes from the >>> > optimiser and >>> > > > > > runtime >>> > > > > > > > > > > >> >> >>> packages. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> We should change ClusterClient from >>> > class >>> > > > > to >>> > > > > > > > > > interface. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> ExecutionEnvironment only use the >>> > > > interface >>> > > > > > > > > > > >> >> >> ClusterClient >>> > > > > > > > > > > >> >> >>>>> which >>> > > > > > > > > > > >> >> >>>>>>>>>>> should be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> in flink-clients while the concrete >>> > > > > > > > implementation >>> > > > > > > > > > > >> >> >> class >>> > > > > > > > > > > >> >> >>>>> could >>> > > > > > > > > > > >> >> >>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> flink-runtime. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> What happens when a >>> > failure/restart in >>> > > > > the >>> > > > > > > > > client >>> > > > > > > > > > > >> >> >>>>> happens? >>> > > > > > > > > > > >> >> >>>>>>>> There >>> > > > > > > > > > > >> >> >>>>>>>>>>> need >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> to be a way of re-establishing the >>> > > > > > connection to >>> > > > > > > > > the >>> > > > > > > > > > > >> >> >> job, >>> > > > > > > > > > > >> >> >>>> set >>> > > > > > > > > > > >> >> >>>>>> up >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> listeners again, etc. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> Good point. First we need to define >>> > what >>> > > > > > does >>> > > > > > > > > > > >> >> >>>>> failure/restart >>> > > > > > > > > > > >> >> >>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> client mean. IIUC, that usually mean >>> > > > > network >>> > > > > > > > > failure >>> > > > > > > > > > > >> >> >>> which >>> > > > > > > > > > > >> >> >>>>> will >>> > > > > > > > > > > >> >> >>>>>>>>>>> happen in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> class RestClient. If my >>> > understanding is >>> > > > > > correct, >>> > > > > > > > > > > >> >> >>>>> restart/retry >>> > > > > > > > > > > >> >> >>>>>>>>>>> mechanism >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> should be done in RestClient. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> Aljoscha Krettek < >>> > aljos...@apache.org> >>> > > > > > > > > 于2019年6月11日周二 >>> > > > > > > > > > > >> >> >>>>>> 下午11:10写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> Some points to consider: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> * Any API we expose should not have >>> > > > > > dependencies >>> > > > > > > > > on >>> > > > > > > > > > > >> >> >> the >>> > > > > > > > > > > >> >> >>>>>> runtime >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> (flink-runtime) package or other >>> > > > > > implementation >>> > > > > > > > > > > >> >> >>> details. >>> > > > > > > > > > > >> >> >>>> To >>> > > > > > > > > > > >> >> >>>>>> me, >>> > > > > > > > > > > >> >> >>>>>>>>>>> this >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> means >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> that the current ClusterClient >>> > cannot be >>> > > > > > exposed >>> > > > > > > > > to >>> > > > > > > > > > > >> >> >>> users >>> > > > > > > > > > > >> >> >>>>>>> because >>> > > > > > > > > > > >> >> >>>>>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> uses >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> quite some classes from the >>> > optimiser >>> > > > and >>> > > > > > > > runtime >>> > > > > > > > > > > >> >> >>>> packages. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> * What happens when a >>> > failure/restart in >>> > > > > the >>> > > > > > > > > client >>> > > > > > > > > > > >> >> >>>>> happens? >>> > > > > > > > > > > >> >> >>>>>>>> There >>> > > > > > > > > > > >> >> >>>>>>>>>>> need >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> be a way of re-establishing the >>> > > > connection >>> > > > > > to >>> > > > > > > > the >>> > > > > > > > > > > >> >> >> job, >>> > > > > > > > > > > >> >> >>>> set >>> > > > > > > > > > > >> >> >>>>> up >>> > > > > > > > > > > >> >> >>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> listeners >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> again, etc. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> Aljoscha >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> On 29. May 2019, at 10:17, Jeff >>> > Zhang < >>> > > > > > > > > > > >> >> >>>> zjf...@gmail.com> >>> > > > > > > > > > > >> >> >>>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> Sorry folks, the design doc is >>> > late as >>> > > > > you >>> > > > > > > > > > > >> >> >> expected. >>> > > > > > > > > > > >> >> >>>>> Here's >>> > > > > > > > > > > >> >> >>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>> design >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> doc >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> I drafted, welcome any comments >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> and >>> > > > > > feedback. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://docs.google.com/document/d/1VavBrYn8vJeZs-Mhu5VzKO6xrWCF40aY0nlQ_UVVTRg/edit?usp=sharing >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> Stephan Ewen <se...@apache.org> >>> > > > > > 于2019年2月14日周四 >>> > > > > > > > > > > >> >> >>>> 下午8:43写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> Nice that this discussion is >>> > > > happening. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> In the FLIP, we could also >>> > revisit the >>> > > > > > entire >>> > > > > > > > > role >>> > > > > > > > > > > >> >> >>> of >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>> environments >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> again. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> Initially, the idea was: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> - the environments take care of >>> > the >>> > > > > > specific >>> > > > > > > > > > > >> >> >> setup >>> > > > > > > > > > > >> >> >>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>> standalone >>> > > > > > > > > > > >> >> >>>>>>>>>>>> (no >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> setup needed), yarn, mesos, etc. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> - the session ones have control >>> > over >>> > > > the >>> > > > > > > > > session. >>> > > > > > > > > > > >> >> >>> The >>> > > > > > > > > > > >> >> >>>>>>>>>>> environment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> holds >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> the session client. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> - running a job gives a "control" >>> > > > object >>> > > > > > for >>> > > > > > > > > that >>> > > > > > > > > > > >> >> >>>> job. >>> > > > > > > > > > > >> >> >>>>>> That >>> > > > > > > > > > > >> >> >>>>>>>>>>>> behavior >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> the same in all environments. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> The actual implementation >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> diverged >>> > > > quite >>> > > > > > a bit >>> > > > > > > > > > > >> >> >> from >>> > > > > > > > > > > >> >> >>>>> that. >>> > > > > > > > > > > >> >> >>>>>>>> Happy >>> > > > > > > > > > > >> >> >>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> see a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> discussion about straitening this >>> > out >>> > > > a >>> > > > > > bit >>> > > > > > > > > more. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> On Tue, Feb 12, 2019 at 4:58 AM >>> > Jeff >>> > > > > > Zhang < >>> > > > > > > > > > > >> >> >>>>>>> zjf...@gmail.com> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> Hi folks, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> Sorry for late response, It >>> > seems we >>> > > > > > reach >>> > > > > > > > > > > >> >> >>> consensus >>> > > > > > > > > > > >> >> >>>> on >>> > > > > > > > > > > >> >> >>>>>>>> this, I >>> > > > > > > > > > > >> >> >>>>>>>>>>>> will >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> create >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> FLIP for this with more detailed >>> > > > design >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> Thomas Weise <t...@apache.org> >>> > > > > > 于2018年12月21日周五 >>> > > > > > > > > > > >> >> >>>>> 上午11:43写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Great to see this discussion >>> > seeded! >>> > > > > The >>> > > > > > > > > > > >> >> >> problems >>> > > > > > > > > > > >> >> >>>> you >>> > > > > > > > > > > >> >> >>>>>> face >>> > > > > > > > > > > >> >> >>>>>>>>>>> with >>> > > > > > > > > > > >> >> >>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Zeppelin integration are also >>> > > > > affecting >>> > > > > > > > other >>> > > > > > > > > > > >> >> >>>>> downstream >>> > > > > > > > > > > >> >> >>>>>>>>>>> projects, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> like >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Beam. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> We just enabled the savepoint >>> > > > restore >>> > > > > > option >>> > > > > > > > > in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> RemoteStreamEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> [1] >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> and that was more difficult >>> > than it >>> > > > > > should >>> > > > > > > > be. >>> > > > > > > > > > > >> >> >> The >>> > > > > > > > > > > >> >> >>>>> main >>> > > > > > > > > > > >> >> >>>>>>>> issue >>> > > > > > > > > > > >> >> >>>>>>>>>>> is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> that >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> environment and cluster client >>> > > > aren't >>> > > > > > > > > decoupled. >>> > > > > > > > > > > >> >> >>>>> Ideally >>> > > > > > > > > > > >> >> >>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>> should >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> possible to just get the >>> > matching >>> > > > > > cluster >>> > > > > > > > > client >>> > > > > > > > > > > >> >> >>>> from >>> > > > > > > > > > > >> >> >>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> environment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> then control the job through it >>> > > > > > (environment >>> > > > > > > > > as >>> > > > > > > > > > > >> >> >>>>> factory >>> > > > > > > > > > > >> >> >>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>> cluster >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> client). But note that the >>> > > > environment >>> > > > > > > > classes >>> > > > > > > > > > > >> >> >> are >>> > > > > > > > > > > >> >> >>>>> part >>> > > > > > > > > > > >> >> >>>>>> of >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> public >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> API, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> and it is not straightforward >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> to >>> > > > make >>> > > > > > larger >>> > > > > > > > > > > >> >> >>> changes >>> > > > > > > > > > > >> >> >>>>>>> without >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> breaking >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> backward compatibility. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> ClusterClient currently exposes >>> > > > > internal >>> > > > > > > > > classes >>> > > > > > > > > > > >> >> >>>> like >>> > > > > > > > > > > >> >> >>>>>>>>>>> JobGraph and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> StreamGraph. But it should be >>> > > > possible >>> > > > > > to >>> > > > > > > > wrap >>> > > > > > > > > > > >> >> >>> this >>> > > > > > > > > > > >> >> >>>>>> with a >>> > > > > > > > > > > >> >> >>>>>>>> new >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> public >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> API >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> that brings the required job >>> > control >>> > > > > > > > > > > >> >> >> capabilities >>> > > > > > > > > > > >> >> >>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>> downstream >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> projects. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Perhaps it is helpful to look >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> at >>> > > > some >>> > > > > > of the >>> > > > > > > > > > > >> >> >>>>> interfaces >>> > > > > > > > > > > >> >> >>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>>>>> Beam >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> while >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> thinking about this: [2] for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> the >>> > > > > > portable >>> > > > > > > > job >>> > > > > > > > > > > >> >> >> API >>> > > > > > > > > > > >> >> >>>> and >>> > > > > > > > > > > >> >> >>>>>> [3] >>> > > > > > > > > > > >> >> >>>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> old >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> asynchronous job control from >>> > the >>> > > > Beam >>> > > > > > Java >>> > > > > > > > > SDK. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> The backward compatibility >>> > > > discussion >>> > > > > > [4] is >>> > > > > > > > > > > >> >> >> also >>> > > > > > > > > > > >> >> >>>>>> relevant >>> > > > > > > > > > > >> >> >>>>>>>>>>> here. A >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> new >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> API >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> should shield downstream >>> > projects >>> > > > from >>> > > > > > > > > internals >>> > > > > > > > > > > >> >> >>> and >>> > > > > > > > > > > >> >> >>>>>> allow >>> > > > > > > > > > > >> >> >>>>>>>>>>> them to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> interoperate with multiple >>> > future >>> > > > > Flink >>> > > > > > > > > versions >>> > > > > > > > > > > >> >> >>> in >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>> same >>> > > > > > > > > > > >> >> >>>>>>>>>>>> release >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> line >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> without forced upgrades. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Thanks, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> Thomas >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> [1] >>> > > > > > > > https://github.com/apache/flink/pull/7249 >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> [2] >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://github.com/apache/beam/blob/master/model/job-management/src/main/proto/beam_job_api.proto >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> [3] >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/PipelineResult.java >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> [4] >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > https://lists.apache.org/thread.html/064c75c5d10f0806095b14f6d76942598917a14429c1acbddd151fe2@%3Cdev.flink.apache.org%3E >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> On Thu, Dec 20, 2018 at 6:15 PM >>> > Jeff >>> > > > > > Zhang < >>> > > > > > > > > > > >> >> >>>>>>>> zjf...@gmail.com> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> I'm not so sure whether the >>> > user >>> > > > > > should >>> > > > > > > > be >>> > > > > > > > > > > >> >> >>> able >>> > > > > > > > > > > >> >> >>>> to >>> > > > > > > > > > > >> >> >>>>>>>> define >>> > > > > > > > > > > >> >> >>>>>>>>>>>> where >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> runs (in your example Yarn). >>> > This >>> > > > is >>> > > > > > > > actually >>> > > > > > > > > > > >> >> >>>>>> independent >>> > > > > > > > > > > >> >> >>>>>>>> of >>> > > > > > > > > > > >> >> >>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> development and is something >>> > which >>> > > > is >>> > > > > > > > decided >>> > > > > > > > > > > >> >> >> at >>> > > > > > > > > > > >> >> >>>>>>> deployment >>> > > > > > > > > > > >> >> >>>>>>>>>>> time. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> User don't need to specify >>> > > > execution >>> > > > > > mode >>> > > > > > > > > > > >> >> >>>>>>> programmatically. >>> > > > > > > > > > > >> >> >>>>>>>>>>> They >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> can >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> also >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> pass the execution mode from >>> > the >>> > > > > > arguments >>> > > > > > > > in >>> > > > > > > > > > > >> >> >>> flink >>> > > > > > > > > > > >> >> >>>>> run >>> > > > > > > > > > > >> >> >>>>>>>>>>> command. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> e.g. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> bin/flink run -m yarn-cluster >>> > .... >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> bin/flink run -m local ... >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> bin/flink run -m host:port ... >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Does this make sense to you ? >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> To me it makes sense that >>> > the >>> > > > > > > > > > > >> >> >>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>> is >>> > > > > > > > > > > >> >> >>>>>>>> not >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> directly >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> initialized by the user and >>> > instead >>> > > > > > context >>> > > > > > > > > > > >> >> >>>> sensitive >>> > > > > > > > > > > >> >> >>>>>> how >>> > > > > > > > > > > >> >> >>>>>>>> you >>> > > > > > > > > > > >> >> >>>>>>>>>>>> want >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> execute your job (Flink CLI >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> vs. >>> > > > IDE, >>> > > > > > for >>> > > > > > > > > > > >> >> >>> example). >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Right, currently I notice >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Flink >>> > > > would >>> > > > > > > > create >>> > > > > > > > > > > >> >> >>>>> different >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> ContextExecutionEnvironment >>> > based >>> > > > on >>> > > > > > > > > different >>> > > > > > > > > > > >> >> >>>>>> submission >>> > > > > > > > > > > >> >> >>>>>>>>>>>> scenarios >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> (Flink >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Cli vs IDE). To me this is >>> > kind of >>> > > > > hack >>> > > > > > > > > > > >> >> >> approach, >>> > > > > > > > > > > >> >> >>>> not >>> > > > > > > > > > > >> >> >>>>>> so >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> straightforward. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> What I suggested above is that >>> > is >>> > > > > that >>> > > > > > > > flink >>> > > > > > > > > > > >> >> >>> should >>> > > > > > > > > > > >> >> >>>>>>> always >>> > > > > > > > > > > >> >> >>>>>>>>>>> create >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> same >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> ExecutionEnvironment but with >>> > > > > different >>> > > > > > > > > > > >> >> >>>>> configuration, >>> > > > > > > > > > > >> >> >>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>> based >>> > > > > > > > > > > >> >> >>>>>>>>>>>> on >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> configuration it would create >>> > the >>> > > > > > proper >>> > > > > > > > > > > >> >> >>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> different >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> behaviors. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Till Rohrmann < >>> > > > trohrm...@apache.org> >>> > > > > > > > > > > >> >> >>>> 于2018年12月20日周四 >>> > > > > > > > > > > >> >> >>>>>>>>>>> 下午11:18写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> You are probably right that >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> we >>> > > > have >>> > > > > > code >>> > > > > > > > > > > >> >> >>>> duplication >>> > > > > > > > > > > >> >> >>>>>>> when >>> > > > > > > > > > > >> >> >>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>> comes >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> creation of the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> ClusterClient. >>> > > > This >>> > > > > > should >>> > > > > > > > > be >>> > > > > > > > > > > >> >> >>>>> reduced >>> > > > > > > > > > > >> >> >>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> future. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> I'm not so sure whether the >>> > user >>> > > > > > should be >>> > > > > > > > > > > >> >> >> able >>> > > > > > > > > > > >> >> >>> to >>> > > > > > > > > > > >> >> >>>>>>> define >>> > > > > > > > > > > >> >> >>>>>>>>>>> where >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> runs (in your example Yarn). >>> > This >>> > > > is >>> > > > > > > > > actually >>> > > > > > > > > > > >> >> >>>>>>> independent >>> > > > > > > > > > > >> >> >>>>>>>>>>> of the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> development and is something >>> > which >>> > > > > is >>> > > > > > > > > decided >>> > > > > > > > > > > >> >> >> at >>> > > > > > > > > > > >> >> >>>>>>>> deployment >>> > > > > > > > > > > >> >> >>>>>>>>>>>> time. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> To >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> me >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> makes sense that the >>> > > > > > ExecutionEnvironment >>> > > > > > > > is >>> > > > > > > > > > > >> >> >> not >>> > > > > > > > > > > >> >> >>>>>>> directly >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> initialized >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> by >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> the user and instead context >>> > > > > > sensitive how >>> > > > > > > > > you >>> > > > > > > > > > > >> >> >>>> want >>> > > > > > > > > > > >> >> >>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>> execute >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> your >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> (Flink CLI vs. IDE, for >>> > example). >>> > > > > > > > However, I >>> > > > > > > > > > > >> >> >>> agree >>> > > > > > > > > > > >> >> >>>>>> that >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> ExecutionEnvironment should >>> > give >>> > > > you >>> > > > > > > > access >>> > > > > > > > > to >>> > > > > > > > > > > >> >> >>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> job (maybe in the form of the >>> > > > > > JobGraph or >>> > > > > > > > a >>> > > > > > > > > > > >> >> >> job >>> > > > > > > > > > > >> >> >>>>> plan). >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> Cheers, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> Till >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> On Thu, Dec 13, 2018 at 4:36 >>> > AM >>> > > > Jeff >>> > > > > > > > Zhang < >>> > > > > > > > > > > >> >> >>>>>>>>>>> zjf...@gmail.com> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> Hi Till, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> Thanks for the feedback. You >>> > are >>> > > > > > right >>> > > > > > > > > that I >>> > > > > > > > > > > >> >> >>>>> expect >>> > > > > > > > > > > >> >> >>>>>>>> better >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> programmatic >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> job submission/control api >>> > which >>> > > > > > could be >>> > > > > > > > > > > >> >> >> used >>> > > > > > > > > > > >> >> >>> by >>> > > > > > > > > > > >> >> >>>>>>>>>>> downstream >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> project. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> And >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> it would benefit for the >>> > flink >>> > > > > > ecosystem. >>> > > > > > > > > > > >> >> >> When >>> > > > > > > > > > > >> >> >>> I >>> > > > > > > > > > > >> >> >>>>> look >>> > > > > > > > > > > >> >> >>>>>>> at >>> > > > > > > > > > > >> >> >>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>> code >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> of >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> flink >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> scala-shell and sql-client >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> (I >>> > > > > believe >>> > > > > > > > they >>> > > > > > > > > > > >> >> >> are >>> > > > > > > > > > > >> >> >>>> not >>> > > > > > > > > > > >> >> >>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> core of >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> flink, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> but >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> belong to the ecosystem of >>> > > > flink), >>> > > > > I >>> > > > > > find >>> > > > > > > > > > > >> >> >> many >>> > > > > > > > > > > >> >> >>>>>>> duplicated >>> > > > > > > > > > > >> >> >>>>>>>>>>> code >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> creating >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> ClusterClient from user >>> > provided >>> > > > > > > > > > > >> >> >> configuration >>> > > > > > > > > > > >> >> >>>>>>>>>>> (configuration >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> format >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> may >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> different from scala-shell >>> > and >>> > > > > > > > sql-client) >>> > > > > > > > > > > >> >> >> and >>> > > > > > > > > > > >> >> >>>> then >>> > > > > > > > > > > >> >> >>>>>> use >>> > > > > > > > > > > >> >> >>>>>>>>>>> that >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> to manipulate jobs. I don't >>> > think >>> > > > > > this is >>> > > > > > > > > > > >> >> >>>>> convenient >>> > > > > > > > > > > >> >> >>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> downstream >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> projects. What I expect is >>> > that >>> > > > > > > > downstream >>> > > > > > > > > > > >> >> >>>> project >>> > > > > > > > > > > >> >> >>>>>> only >>> > > > > > > > > > > >> >> >>>>>>>>>>> needs >>> > > > > > > > > > > >> >> >>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> provide >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> necessary configuration info >>> > > > (maybe >>> > > > > > > > > > > >> >> >> introducing >>> > > > > > > > > > > >> >> >>>>> class >>> > > > > > > > > > > >> >> >>>>>>>>>>>> FlinkConf), >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> build ExecutionEnvironment >>> > based >>> > > > on >>> > > > > > this >>> > > > > > > > > > > >> >> >>>> FlinkConf, >>> > > > > > > > > > > >> >> >>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> ExecutionEnvironment will >>> > create >>> > > > > the >>> > > > > > > > proper >>> > > > > > > > > > > >> >> >>>>>>>> ClusterClient. >>> > > > > > > > > > > >> >> >>>>>>>>>>> It >>> > > > > > > > > > > >> >> >>>>>>>>>>>> not >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> only >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> benefit for the downstream >>> > > > project >>> > > > > > > > > > > >> >> >> development >>> > > > > > > > > > > >> >> >>>> but >>> > > > > > > > > > > >> >> >>>>>> also >>> > > > > > > > > > > >> >> >>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> helpful >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> their integration test with >>> > > > flink. >>> > > > > > Here's >>> > > > > > > > > one >>> > > > > > > > > > > >> >> >>>>> sample >>> > > > > > > > > > > >> >> >>>>>>> code >>> > > > > > > > > > > >> >> >>>>>>>>>>>> snippet >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> that >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> I >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> expect. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> val conf = new >>> > > > > > FlinkConf().mode("yarn") >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> val env = new >>> > > > > > ExecutionEnvironment(conf) >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> val jobId = env.submit(...) >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> val jobStatus = >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > env.getClusterClient().queryJobStatus(jobId) >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > env.getClusterClient().cancelJob(jobId) >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> What do you think ? >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> Till Rohrmann < >>> > > > > trohrm...@apache.org> >>> > > > > > > > > > > >> >> >>>>> 于2018年12月11日周二 >>> > > > > > > > > > > >> >> >>>>>>>>>>> 下午6:28写道： >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> Hi Jeff, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> what you are proposing is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> to >>> > > > > > provide the >>> > > > > > > > > > > >> >> >> user >>> > > > > > > > > > > >> >> >>>> with >>> > > > > > > > > > > >> >> >>>>>>>> better >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> programmatic >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> control. There was actually >>> > an >>> > > > > > effort to >>> > > > > > > > > > > >> >> >>> achieve >>> > > > > > > > > > > >> >> >>>>>> this >>> > > > > > > > > > > >> >> >>>>>>>> but >>> > > > > > > > > > > >> >> >>>>>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> has >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> never >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> been >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> completed [1]. However, >>> > there >>> > > > are >>> > > > > > some >>> > > > > > > > > > > >> >> >>>> improvement >>> > > > > > > > > > > >> >> >>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> code >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> base >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> now. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> Look for example at the >>> > > > > > NewClusterClient >>> > > > > > > > > > > >> >> >>>> interface >>> > > > > > > > > > > >> >> >>>>>>> which >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> offers a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> non-blocking job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> submission. >>> > > > But I >>> > > > > > agree >>> > > > > > > > > > > >> >> >> that >>> > > > > > > > > > > >> >> >>> we >>> > > > > > > > > > > >> >> >>>>>> need >>> > > > > > > > > > > >> >> >>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> improve >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Flink >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> this regard. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> I would not be in favour if >>> > > > > > exposing all >>> > > > > > > > > > > >> >> >>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>> calls >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> via >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> ExecutionEnvironment >>> > because it >>> > > > > > would >>> > > > > > > > > > > >> >> >> clutter >>> > > > > > > > > > > >> >> >>>> the >>> > > > > > > > > > > >> >> >>>>>>> class >>> > > > > > > > > > > >> >> >>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> would >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> not >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> good separation of >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> concerns. >>> > > > > > Instead one >>> > > > > > > > > > > >> >> >> idea >>> > > > > > > > > > > >> >> >>>>> could >>> > > > > > > > > > > >> >> >>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> retrieve >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> current ClusterClient from >>> > the >>> > > > > > > > > > > >> >> >>>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>> which >>> > > > > > > > > > > >> >> >>>>>>>>>>> can >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> used >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> for cluster and job >>> > control. But >>> > > > > > before >>> > > > > > > > we >>> > > > > > > > > > > >> >> >>> start >>> > > > > > > > > > > >> >> >>>>> an >>> > > > > > > > > > > >> >> >>>>>>>> effort >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> here, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> we >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> need >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> agree and capture what >>> > > > > > functionality we >>> > > > > > > > > want >>> > > > > > > > > > > >> >> >>> to >>> > > > > > > > > > > >> >> >>>>>>> provide. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> Initially, the idea was >>> > that we >>> > > > > > have the >>> > > > > > > > > > > >> >> >>>>>>>> ClusterDescriptor >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> describing >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> how >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> to talk to cluster manager >>> > like >>> > > > > > Yarn or >>> > > > > > > > > > > >> >> >> Mesos. >>> > > > > > > > > > > >> >> >>>> The >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> ClusterDescriptor >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> can >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> be >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> used for deploying Flink >>> > > > clusters >>> > > > > > (job >>> > > > > > > > and >>> > > > > > > > > > > >> >> >>>>> session) >>> > > > > > > > > > > >> >> >>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>> gives >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> you a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> ClusterClient. The >>> > ClusterClient >>> > > > > > > > controls >>> > > > > > > > > > > >> >> >> the >>> > > > > > > > > > > >> >> >>>>>> cluster >>> > > > > > > > > > > >> >> >>>>>>>>>>> (e.g. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> submitting >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> jobs, listing all running >>> > jobs). >>> > > > > And >>> > > > > > > > then >>> > > > > > > > > > > >> >> >>> there >>> > > > > > > > > > > >> >> >>>>> was >>> > > > > > > > > > > >> >> >>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> idea >>> > > > > > > > > > > >> >> >>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> introduce a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> JobClient which you obtain >>> > from >>> > > > > the >>> > > > > > > > > > > >> >> >>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>> trigger >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> specific >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> operations (e.g. taking a >>> > > > > savepoint, >>> > > > > > > > > > > >> >> >>> cancelling >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>>> job). >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> [1] >>> > > > > > > > > > > >> >> >>>>>> >>> > > > https://issues.apache.org/jira/browse/FLINK-4272 >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> Cheers, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> Till >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> On Tue, Dec 11, 2018 at >>> > 10:13 AM >>> > > > > > Jeff >>> > > > > > > > > Zhang >>> > > > > > > > > > > >> >> >> < >>> > > > > > > > > > > >> >> >>>>>>>>>>> zjf...@gmail.com >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> wrote: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Hi Folks, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> I am trying to integrate >>> > flink >>> > > > > into >>> > > > > > > > > apache >>> > > > > > > > > > > >> >> >>>>> zeppelin >>> > > > > > > > > > > >> >> >>>>>>>>>>> which is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> an >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> interactive >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> notebook. And I hit >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> several >>> > > > > issues >>> > > > > > that >>> > > > > > > > > is >>> > > > > > > > > > > >> >> >>>> caused >>> > > > > > > > > > > >> >> >>>>>> by >>> > > > > > > > > > > >> >> >>>>>>>>>>> flink >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> client >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> api. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> So >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> I'd like to proposal the >>> > > > > following >>> > > > > > > > > changes >>> > > > > > > > > > > >> >> >>> for >>> > > > > > > > > > > >> >> >>>>>> flink >>> > > > > > > > > > > >> >> >>>>>>>>>>> client >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> api. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 1. Support nonblocking >>> > > > execution. >>> > > > > > > > > > > >> >> >> Currently, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> ExecutionEnvironment#execute >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> is a blocking method which >>> > > > would >>> > > > > > do 2 >>> > > > > > > > > > > >> >> >> things, >>> > > > > > > > > > > >> >> >>>>> first >>> > > > > > > > > > > >> >> >>>>>>>>>>> submit >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> wait for job until it is >>> > > > > finished. >>> > > > > > I'd >>> > > > > > > > > like >>> > > > > > > > > > > >> >> >>>>>>> introduce a >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> nonblocking >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> execution method like >>> > > > > > > > > > > >> >> >>>> ExecutionEnvironment#submit >>> > > > > > > > > > > >> >> >>>>>>> which >>> > > > > > > > > > > >> >> >>>>>>>>>>> only >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> submit >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> then return jobId to >>> > client. >>> > > > And >>> > > > > > allow >>> > > > > > > > > user >>> > > > > > > > > > > >> >> >>> to >>> > > > > > > > > > > >> >> >>>>>> query >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> status >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> via >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> jobId. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 2. Add cancel api in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >> ExecutionEnvironment/StreamExecutionEnvironment, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> currently the only way to >>> > > > cancel >>> > > > > > job is >>> > > > > > > > > via >>> > > > > > > > > > > >> >> >>> cli >>> > > > > > > > > > > >> >> >>>>>>>>>>> (bin/flink), >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> this >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> not >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> convenient for downstream >>> > > > project >>> > > > > > to >>> > > > > > > > use >>> > > > > > > > > > > >> >> >> this >>> > > > > > > > > > > >> >> >>>>>>> feature. >>> > > > > > > > > > > >> >> >>>>>>>>>>> So I'd >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> like >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> add >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> cancel api in >>> > > > > ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 3. Add savepoint api in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>> >>> > ExecutionEnvironment/StreamExecutionEnvironment. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> It >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> is similar as cancel api, >>> > we >>> > > > > > should use >>> > > > > > > > > > > >> >> >>>>>>>>>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> as >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> unified >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> api for third party to >>> > > > integrate >>> > > > > > with >>> > > > > > > > > > > >> >> >> flink. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 4. Add listener for job >>> > > > execution >>> > > > > > > > > > > >> >> >> lifecycle. >>> > > > > > > > > > > >> >> >>>>>>> Something >>> > > > > > > > > > > >> >> >>>>>>>>>>> like >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> following, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> so >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> that downstream project >>> > can do >>> > > > > > custom >>> > > > > > > > > logic >>> > > > > > > > > > > >> >> >>> in >>> > > > > > > > > > > >> >> >>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>> lifecycle >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> of >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> job. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> e.g. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Zeppelin would capture the >>> > > > jobId >>> > > > > > after >>> > > > > > > > > job >>> > > > > > > > > > > >> >> >> is >>> > > > > > > > > > > >> >> >>>>>>> submitted >>> > > > > > > > > > > >> >> >>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> use >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> this >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> jobId to cancel it later >>> > when >>> > > > > > > > necessary. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> public interface >>> > JobListener { >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> void onJobSubmitted(JobID >>> > > > > jobId); >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> void >>> > > > > > onJobExecuted(JobExecutionResult >>> > > > > > > > > > > >> >> >>>>> jobResult); >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> void onJobCanceled(JobID >>> > > > jobId); >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> } >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 5. Enable session in >>> > > > > > > > > ExecutionEnvironment. >>> > > > > > > > > > > >> >> >>>>>> Currently >>> > > > > > > > > > > >> >> >>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>> is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> disabled, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> but >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> session is very convenient >>> > for >>> > > > > > third >>> > > > > > > > > party >>> > > > > > > > > > > >> >> >> to >>> > > > > > > > > > > >> >> >>>>>>>> submitting >>> > > > > > > > > > > >> >> >>>>>>>>>>> jobs >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> continually. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> I hope flink can enable it >>> > > > again. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 6. Unify all flink client >>> > api >>> > > > > into >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>> >>> > ExecutionEnvironment/StreamExecutionEnvironment. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> This is a long term issue >>> > which >>> > > > > > needs >>> > > > > > > > > more >>> > > > > > > > > > > >> >> >>>>> careful >>> > > > > > > > > > > >> >> >>>>>>>>>>> thinking >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> design. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Currently some of features >>> > of >>> > > > > > flink is >>> > > > > > > > > > > >> >> >>> exposed >>> > > > > > > > > > > >> >> >>>> in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>> >>> > ExecutionEnvironment/StreamExecutionEnvironment, >>> > > > > > > > > > > >> >> >>>>>> but >>> > > > > > > > > > > >> >> >>>>>>>>>>> some are >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> exposed >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> in >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> cli instead of api, like >>> > the >>> > > > > > cancel and >>> > > > > > > > > > > >> >> >>>>> savepoint I >>> > > > > > > > > > > >> >> >>>>>>>>>>> mentioned >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> above. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> I >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> think the root cause is >>> > due to >>> > > > > that >>> > > > > > > > flink >>> > > > > > > > > > > >> >> >>>> didn't >>> > > > > > > > > > > >> >> >>>>>>> unify >>> > > > > > > > > > > >> >> >>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> interaction >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> with >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> flink. Here I list 3 >>> > scenarios >>> > > > of >>> > > > > > flink >>> > > > > > > > > > > >> >> >>>> operation >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> - Local job execution. >>> > Flink >>> > > > > will >>> > > > > > > > > create >>> > > > > > > > > > > >> >> >>>>>>>>>>> LocalEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> use >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> this LocalEnvironment to >>> > > > create >>> > > > > > > > > > > >> >> >>> LocalExecutor >>> > > > > > > > > > > >> >> >>>>> for >>> > > > > > > > > > > >> >> >>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> execution. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> - Remote job execution. >>> > Flink >>> > > > > will >>> > > > > > > > > create >>> > > > > > > > > > > >> >> >>>>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> first >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> create ContextEnvironment >>> > > > based >>> > > > > > on the >>> > > > > > > > > > > >> >> >>>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> run >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> job. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> - Job cancelation. Flink >>> > will >>> > > > > > create >>> > > > > > > > > > > >> >> >>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>> first >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> then >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> cancel >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> this job via this >>> > > > ClusterClient. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> As you can see in the >>> > above 3 >>> > > > > > > > scenarios. >>> > > > > > > > > > > >> >> >>> Flink >>> > > > > > > > > > > >> >> >>>>>> didn't >>> > > > > > > > > > > >> >> >>>>>>>>>>> use the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> same >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> approach(code path) to >>> > interact >>> > > > > > with >>> > > > > > > > > flink >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> What I propose is >>> > following: >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Create the proper >>> > > > > > > > > > > >> >> >>>>>> LocalEnvironment/RemoteEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>> (based >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> on >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> user >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> configuration) --> Use >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> this >>> > > > > > Environment >>> > > > > > > > > to >>> > > > > > > > > > > >> >> >>>> create >>> > > > > > > > > > > >> >> >>>>>>>> proper >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> ClusterClient >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> (LocalClusterClient or >>> > > > > > > > RestClusterClient) >>> > > > > > > > > > > >> >> >> to >>> > > > > > > > > > > >> >> >>>>>>>> interactive >>> > > > > > > > > > > >> >> >>>>>>>>>>> with >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Flink ( >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> job >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> execution or cancelation) >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> This way we can unify the >>> > > > process >>> > > > > > of >>> > > > > > > > > local >>> > > > > > > > > > > >> >> >>>>>> execution >>> > > > > > > > > > > >> >> >>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> remote >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> execution. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> And it is much easier for >>> > third >>> > > > > > party >>> > > > > > > > to >>> > > > > > > > > > > >> >> >>>>> integrate >>> > > > > > > > > > > >> >> >>>>>>> with >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> flink, >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> because >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> ExecutionEnvironment is >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> the >>> > > > > unified >>> > > > > > > > entry >>> > > > > > > > > > > >> >> >>> point >>> > > > > > > > > > > >> >> >>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>> flink. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> What >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> third >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> party >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> needs to do is just pass >>> > > > > > configuration >>> > > > > > > > to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> and >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> ExecutionEnvironment will >>> > do >>> > > > the >>> > > > > > right >>> > > > > > > > > > > >> >> >> thing >>> > > > > > > > > > > >> >> >>>>> based >>> > > > > > > > > > > >> >> >>>>>> on >>> > > > > > > > > > > >> >> >>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> configuration. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Flink cli can also be >>> > > > considered >>> > > > > as >>> > > > > > > > flink >>> > > > > > > > > > > >> >> >> api >>> > > > > > > > > > > >> >> >>>>>>> consumer. >>> > > > > > > > > > > >> >> >>>>>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> just >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> pass >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> the >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> configuration to >>> > > > > > ExecutionEnvironment >>> > > > > > > > and >>> > > > > > > > > > > >> >> >> let >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> ExecutionEnvironment >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> create the proper >>> > ClusterClient >>> > > > > > instead >>> > > > > > > > > of >>> > > > > > > > > > > >> >> >>>>> letting >>> > > > > > > > > > > >> >> >>>>>>> cli >>> > > > > > > > > > > >> >> >>>>>>>> to >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> create >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> ClusterClient directly. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> 6 would involve large code >>> > > > > > refactoring, >>> > > > > > > > > so >>> > > > > > > > > > > >> >> >> I >>> > > > > > > > > > > >> >> >>>>> think >>> > > > > > > > > > > >> >> >>>>>> we >>> > > > > > > > > > > >> >> >>>>>>>> can >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> defer >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> it >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> for >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> future release, 1,2,3,4,5 >>> > could >>> > > > > be >>> > > > > > done >>> > > > > > > > > at >>> > > > > > > > > > > >> >> >>>> once I >>> > > > > > > > > > > >> >> >>>>>>>>>>> believe. >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> Let >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> me >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> know >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> your >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> comments and feedback, >>> > thanks >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> -- >>> > > > > > > > > > > >> >> >>>>>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>>>>> >>> > > > > > > > > > > >> >> >>>>>>> >>> > > > > > > > > > > >> >> >>>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>> -- >>> > > > > > > > > > > >> >> >>>>> Best Regards >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>>> Jeff Zhang >>> > > > > > > > > > > >> >> >>>>> >>> > > > > > > > > > > >> >> >>>> >>> > > > > > > > > > > >> >> >>> >>> > > > > > > > > > > >> >> >> >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > -- >>> > > > > > > > > > > >> >> > Best Regards >>> > > > > > > > > > > >> >> > >>> > > > > > > > > > > >> >> > Jeff Zhang >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >> >>> > > > > > > > > > > >> >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > >>> >>> >>> -- >>> Best Regards >>> >>> Jeff Zhang
