Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2024-02-22 Thread Yong Fang
Hi Martijn, If there're no more comments, I will start a vote for this, thanks Best, Fang Yong On Tue, Feb 20, 2024 at 4:53 PM Yong Fang wrote: > Hi Martijn, > > Thank you for your attention. Let me first explain the specific situation > of FLIP-314. FLIP-314 is currently in an accepted

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2024-02-20 Thread Yong Fang
Hi Martijn, Thank you for your attention. Let me first explain the specific situation of FLIP-314. FLIP-314 is currently in an accepted state, but actual code development has not yet begun, and interface related PR has not been merged into the master. So it may not be necessary for us to create a

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2024-02-19 Thread Martijn Visser
I'm a bit confused: did we add new interfaces after FLIP-314 was accepted? If so, please move the new interfaces to a new FLIP and start a separate vote. We can't retrospectively change an accepted FLIP with new interfaces and a new vote. On Mon, Feb 19, 2024 at 3:22 AM Yong Fang wrote: > > Hi

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2024-02-18 Thread Yong Fang
Hi all, If there are no more feedbacks, I will start a vote for the new interfaces in the next day, thanks Best, Fang Yong On Thu, Feb 8, 2024 at 1:30 PM Yong Fang wrote: > Hi devs, > > According to the online-discussion in FLINK-3127 [1] and > offline-discussion with Maciej Obuchowski and

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2024-02-07 Thread Yong Fang
Hi devs, According to the online-discussion in FLINK-3127 [1] and offline-discussion with Maciej Obuchowski and Zhenqiu Huang, we would like to update the lineage vertex relevant interfaces in FLIP-314 [2] as follows: 1. Introduce `LineageDataset` which represents source and sink in

Re: FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2023-09-24 Thread Shammon FY
Hi David, Do you want the detailed topology for Flink job? You can get `JobDetailsInfo` in `RestCusterClient` with the submitted job id, it has `String jsonPlan`. You can parse the json plan to get all steps and relations between them in a Flink job. Hope this can help you, thanks! Best, Shammon

FW: RE: [DISCUSS] FLIP-314: Support Customized Job Lineage Listener

2023-09-19 Thread David Radley
Hi there, I am looking at the interfaces. If I am reading it correctly,there is one relationship between the source and sink and this relationship represents the operational lineage. Lineage is usually represented as asset -> process - > asset – see for example