Thanks for starting this discussion Jeff. I can see the need for additional hooks for third party integrations.
The thing I'm wondering is whether we really need/want to expose a JobListener via the ExecutionEnvironment. The ExecutionEnvironment is usually used by the user who writes the code and this person (I assume) would not be really interested in these callbacks. If he would, then one should rather think about a better programmatic job control where the `ExecutionEnvironment#execute` call returns a `JobClient` instance. Moreover, we would effectively make this part of the public API and every implementation would need to offer it. In your case, it could be sufficient to offer some hooks for the ClusterClient or being able to provide a custom ClusterClient. The ClusterClient is the component responsible for the job submission and retrieval of the job result and, hence, would be able to signal when a job has been submitted or completed. Cheers, Till On Thu, Apr 18, 2019 at 8:57 AM vino yang <yanghua1...@gmail.com> wrote: > Hi Jeff, > > I personally like this proposal. From the perspective of programmability, > the JobListener can make the third program more appreciable. > > The scene where I need the listener is the Flink cube engine for Apache > Kylin. In the case, the Flink job program is embedded into the Kylin's > executable context. > > If we could have this listener, it would be easier to integrate with Kylin. > > Best, > Vino > > Jeff Zhang <zjf...@gmail.com> 于2019年4月18日周四 下午1:30写道： > >> >> Hi All, >> >> I created FLINK-12214 <https://issues.apache.org/jira/browse/FLINK-12214> for >> adding JobListener (hook) in flink job lifecycle. Since this is a new >> public api for flink, so I'd like to discuss it more widely in community to >> get more feedback. >> >> The background and motivation is that I am integrating flink into apache >> zeppelin <http://zeppelin.apache.org/>(which is a notebook in case you >> don't know). And I'd like to capture some job context (like jobId) in the >> lifecycle of flink job (submission, executed, cancelled) so that I can >> manipulate job in more fined grained control (e.g. I can capture the jobId >> when job is submitted, and then associate it with one paragraph, and when >> user click the cancel button, I can call the flink cancel api to cancel >> this job) >> >> I believe other projects which integrate flink would need similar >> mechanism. I plan to add api addJobListener in >> ExecutionEnvironment/StreamExecutionEnvironment so that user can add >> customized hook in flink job lifecycle. >> >> Here's draft interface JobListener. >> >> public interface JobListener { >> >> void onJobSubmitted(JobID jobId); >> >> void onJobExecuted(JobExecutionResult jobResult); >> >> void onJobCanceled(JobID jobId, String savepointPath); >> } >> >> Let me know your comment and concern, thanks. >> >> >> -- >> Best Regards >> >> Jeff Zhang >> >
