An interesting topic, I noticed that the datahub community has launched the feature request discussion of Flink Integration [1].
@Martijn Visser<mailto:martijnvis...@apache.org> Did the Flink community had created tickets to track this topic? >From my current understanding, Flink lacks rich information on >FlinkJobListener just as Feng mentioned, which has been supported well by >Spark, to send data lineage to external systems. [1] https://feature-requests.datahubproject.io/p/flink-integration Best Yun Tang ________________________________ From: wangqinghuan <1095193...@qq.com> Sent: Monday, January 17, 2022 18:27 To: user@flink.apache.org <user@flink.apache.org> Subject: Re: [FEEDBACK] Metadata Platforms / Catalogs / Lineage integration we are using Datahub to address table-level lineage and column-level lineage for Flink SQL. 在 2022/1/13 23:27, Martijn Visser 写道: Hi everyone, I'm currently checking out different metadata platforms, such as Amundsen [1] and Datahub [2]. In short, these types of tools try to address problems related to topics such as data discovery, data lineage and an overall data catalogue. I'm reaching out to the Dev and User mailing lists to get some feedback. It would really help if you could spend a couple of minutes to let me know if you already use either one of the two mentioned metadata platforms or another one, or are you evaluating such tools? If so, is that for the purpose as a catalogue, for lineage or anything else? Any type of feedback on these types of tools is appreciated. Best regards, Martijn [1] https://github.com/amundsen-io/amundsen/ [2] https://github.com/linkedin/datahub
