Till or Chesnay (cc'ed), have you thought about adding a hook on the
JobMaster/JobManager to allow external systems to get push notifications
about submitted jobs.
If they are ok with such as future, would you maybe be interested in
implementing it yourself, Wenhao?
Best,
Aljoscha
On 2020/09/28 11:14, 季文昊 wrote:
Hi Aljoscha,
Yes, that is not enough, since the `JobListener`s are called only once when
`excute()` or `executeAsync()` is called. And in order to sync the status,
I also have to call `JobClient#getJobStatus` periodically.
On Fri, Sep 25, 2020 at 8:12 PM Aljoscha Krettek <aljos...@apache.org>
wrote:
Hi,
I understand from your email that
`StreamExecutionEnvironment.registerJobListener()` would not be enought
for you because you want to be notified of changes on the cluster side,
correct? That is when the job status changes on the master.
Best,
Aljoscha
On 23.09.20 14:31, 季文昊 wrote:
> Hi there,
>
> I'm working on a Flink platform in my corp, which provides a service to
> provision and manage multiple dedicated Flink clusters. The problem is
that
> we want to sync a job status without delay after its submission through
our
> platform as long as it has been changed.
>
> Since we want to update this in-time and make our services stateless,
> pulling a job's status periodically is not a good solution. I do not find
> any proper way to achieve this by letting a job manager push changes
> directly to our platform except changing the source code, which registers
> an additional `JobStatusListener` in the method
> `org.apache.flink.runtime.jobmaster.JobMaster#startScheduling`.
>
> I wonder if we can enhance `JobStatusListener` a little bit so that a
Flink
> user can register his custom JobStatusListener at the startup.
>
> To be specific, we can have a `JobStatusListenerFactory` interface and
its
> corresponding `ServiceLoader<JobStatusListenerFactory>`, where
> the JobStatusListenerFactory will have the following method:
> - JobStatusListener createJobStatusListener(Properties properties);
>
> Custom listeners will be created during the JobMaster#startScheduling
> method.
>
> If someone would like to implement his own JobStatusListener, he will
> package all the related classes into a standalone jar with a
>
`META-INF/services/org.apache.flink.runtime.executiongraph.JobStatusListener`
> file and place it under the `lib/` directory.
>
> In addition, I find that there is a Jira ticket similar to what I'm
> asking: FLINK-17104 but I do not see any comment or update yet. Hope
anyone
> could help me move on this feature or give me some suggestions about it.
>
> Thanks,
> Wenhao
>
