Hi Peter, Missed previous email. Sure, I will take a look.
In current design, there is no guarantee that a JobCreationEvent comes before the Job transition event. DefaultExecutionGraphBuilder can emit a transition event before a JobCreationEvent is emitted. If that cannot be changed (or if it was already considered previously) on Flink side, I was wondering if Listeners need to be equipped to handle that. >> Would you please elaborate a little bit more on the question "And also >> handle cases like Job transition events coming before the JobCreationEvent?" On Thu, Apr 23, 2026 at 6:20 AM Peter Huang <[email protected]> wrote: > Hi Swapna, > > A jira is created to track this improvement. Please follow the PR status > attached to https://issues.apache.org/jira/browse/FLINK-39491. > > > Best Regards > Peter Huang > > On Sun, Apr 19, 2026 at 5:00 PM Peter Huang <[email protected]> > wrote: > > > Hi Swapna, > > > > I was away from the keyboard for the last two weeks. Sorry for the late > > reply. > > > > 1. Dual-instantiation in Application Mode > > Yes, currently, the job creation event happens in client side, and any > > follow up status change events propagation happen in > DefaultExecutionGraph. > > In this case, there are two instances of the same job listener created. > We > > probably may consider moving the job creation event to dispatcher. > > In this case, a single job listener instance for each type in JVM needs > to > > be created. I may work on a POC on this change. > > > > 2. If a single job listener instance will be created, then states within > > the instance will be shared within the whole job lifecycle. > > Would you please elaborate a little bit more on the question "And also > > handle cases like Job transition events coming before the > JobCreationEvent?" > > > > Best Regards > > Peter Huang > > > > On Fri, Apr 3, 2026 at 10:48 AM Swapna Marru <[email protected]> > > wrote: > > > >> Hello Flink Devs, > >> > >> I am trying to understand the behavioral contract of > >> JobStatusChangedListener (FLIP-314) in Application Mode, to correctly > fix > >> the OpenLineage Flink integration. > >> > >> In Application Mode, JobStatusChangedListenerFactory.createListener() is > >> invoked independently at two call sites, producing two distinct > instances: > >> > >> - EmbeddedExecutor fires the lineage creation event on Instance A — > >> L138–142 > >> < > >> > https://github.com/apache/flink/blob/master/flink-clients/src/main/java/org/apache/flink/client/deployment/application/executors/EmbeddedExecutor.java#L138-L142 > >> > > >> - DefaultExecutionGraphBuilder fires the job transition event on > >> Instance B — L147–149 > >> < > >> > https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/DefaultExecutionGraphBuilder.java#L147-L149 > >> > > >> > >> The OpenLineage integration assumes a single shared instance, where > >> lineage > >> context captured during graph planning is referenced upon job > transition — > >> L131–135 > >> < > >> > https://github.com/OpenLineage/OpenLineage/blob/main/integration/flink/flink2/src/main/java/io/openlineage/flink/listener/OpenLineageJobStatusChangedListener.java#L131-L135 > >> >. > >> With two independent instances, Instance B has no JobId set on the > >> context. > >> > >> Could you clarify: > >> > >> 1. In the FLIP, I see "In this FLIP the JobStatusChangedListener will > >> be > >> in Client and JobMaster, which will report lineage information and > job > >> status independently." So is dual-instantiation in Application Mode > >> intentional ? > >> 2. Are listener implementations expected to be stateless, > externalizing > >> any shared state themselves? And also handle cases like Job > transition > >> events coming before the JobCreationEvent ? > >> > >> -Thanks > >> > >> M.Swapna > >> > > >
