Hi Dmitri, Thanks for the comments. Replying point-by-point:
> re: How to use this data I agree, we should list this in the document itself so it does not get lost. Let me put something in the appendix for now and link it to the relevant section so we don't lose the document's flow. Please take a look there. > re: Link to OL Spec? https://openlineage.io/apidocs/openapi/ > re: Does OL persistence have to be limited to JDBC? No, this is the LineagePersistent Contract (from earlier in this thread, linked again for comfort): https://docs.google.com/document/d/1sHFR_rVG9y-35iuy_92GpAPms3RMHS_sVPQqsHxnXHs/edit?tab=t.0. This was made to ensure that different persistence layers can implement a single interface for Lineage information. > re: Does OL data have to be located in the same database as MetaStore data? I don't think there is a hard requirement for this. Happy to discuss if anyone else thinks differently! > I believe Polaris should allow downstream projects to opt in/out of including OL REST APIs and related persistence code. This is the general approach we're following for the Metrics API proposal. Understood. I think this is an implementation detail and you've marked it on the PR as well. Let us take care of this discussion there. Best, Adnan Hemani On Fri, Jun 19, 2026 at 12:34 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi All, > > Commenting in general. > > I see a few PRs have been opened related to OpenLineage. This is fine. > > However, I believe we need to review some project-wide approaches related > to adding this feature (among other features). > > 1) Does OL persistence have to be limited to JDBC? > > 2) Does OL data have to be located in the same database as MetaStore data? > > 3) I believe Polaris should allow downstream projects to opt in/out of > including OL REST APIs and related persistence code. This is the general > approach we're following for the Metrics API proposal [1] [4115] > > These points are not in opposition to supporting OL in Polaris. These are > merely technical concerns, which I think would benefit Polaris as a whole > if designed with flexibility and modularity in mind upfront. > > WDYT? > > [1] https://lists.apache.org/thread/3t79vpmm8bw38q411o36o85fqj6584t0 > > [4115] https://github.com/apache/polaris/pull/4115 > > Thanks, > Dmitri. > > On Thu, May 14, 2026 at 12:59 AM Adnan Hemani via dev < > [email protected]> wrote: > > > Hi all, > > > > I wanted to ensure that the OpenLineage proposal I previously posted in a > > different thread [1] was actually being found, given that it was deep > into > > the thread. I request the community to review this proposal so we can > > potentially start implementation. > > > > Proposal: > > > > > https://docs.google.com/document/d/1iOzIuFW66SFL2wZOADD9knMTG21OwY7VmaWVSvMUqQk/edit?tab=t.0#heading=h.59bmbnsf0gp1 > > > > Best, > > Adnan Hemani > > > > [1] https://lists.apache.org/thread/1fd6hrvx0v0s5wm6gh74cdo3yn4w1zhx > > >
