Hi all, I’ve been working on OpenLineage support in Polaris and wanted to ask for feedback on the current implementation approach and where it should go next.
The current direction is to emit OpenLineage events from Polaris-managed operations and publish them through an event-listener-based design. Before moving further, I’d like to get suggestions on whether this implementation model makes sense and what should be adjusted early. Below is the link for the proposal and PoC branch. - Proposal: https://docs.google.com/document/d/1NEPzcMIcbxKEvBIWd6SJGllRIu4brjV-23xO5Qp3DlM/edit?tab=t.0 - PoC branch: https://github.com/iting0321/polaris/tree/data-lineage I’d appreciate feedback on the implementation direction and any suggestions before continuing further. Best regards, ITing
