charlespnh commented on issue #33981: URL: https://github.com/apache/beam/issues/33981#issuecomment-3529652247
Hey @shnapz, there was a discussion and proposal for this feature - https://lists.apache.org/thread/dodd538lzy5g4obtjrr67bdf9vdlqwc6, and I was planning to take this up although with slow progress. The proposal is to not affect how IOs are emitting lineage info as metrics today, and use these lineage metrics reported by IOs to build out lineage graph for Prism local runner. Looks like your approach is to extend the current Lineage API so that you can have your own future implementation of how lineage will be reported from IOs? Nevertheless I don't think your foundational work will be a conflict with my proposal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
