Hi all, I am not a fan of gating an entire API behind a feature flag.
Another reason not mentioned yet is: if a security vulnerability is detected in the new code, and that code is shipped unconditionally in polaris-runtime-service, then all deployments of that artifact will be flagged by security scans, regardless of whether they opted out of it via the feature flag. If the CVE targets a separate module instead, only users of that module would be affected. That is another reason why I think isolating the API in its module is a better design choice. Thanks, Alex Le ven. 26 juin 2026 à 01:02, Yufei Gu <[email protected]> a écrit : > Hi Dmitri, > > Thanks for the clarification. > > Could you elaborate on why having an empty HTTP layer is a concern for > downstream systems? If the feature is disabled, couldn't we simply return a > 404 or 501, similar to how Quarkus behaves when an endpoint is not > registered? > > Thanks, > > Yufei > > > On Wed, Jun 24, 2026 at 12:38 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Yufei, > > > > As I commented in this thread earlier, storing OSI data as Polaris > entities > > is a reasonable approach. > > > > However, adding hard dependencies from `runtime/service` to the new OSI > > RESP API impl. is not acceptable from my POV, as it forces > > downstream projects into exposing the OSI API without explicit opt-in. > > Feature flags are not relevant here because they work only after REST > > requests are accepted at the HTTP layer. > > > > This is discussed from a more general perspective in [1] > > > > All in all, I do not see any disadvantage to using separate modules for > new > > REST API implementations, but disadvantages in bundling them into > > runtime/serice do exist. > > > > [1] https://lists.apache.org/thread/d9dj3w8ktwdn6w27z7tvvgkljgw3n43b > > > > Cheers, > > Dmitri. > > > > On Tue, Jun 23, 2026 at 8:58 PM Yufei Gu <[email protected]> wrote: > > > > > Hi Dmitri, > > > > > > I see the value of keeping Polaris modular, but I have a slightly > > different > > > view on this particular case. > > > > > > To me, semantic models are closer to tables, views, and policies than > to > > > metrics or events. The proposal introduces a new Polaris entity type > with > > > its own lifecycle, authorization model, and metadata management. In > that > > > sense, semantic model support is part of the core Polaris metadata > model > > > rather than an optional auxiliary capability like metrics. > > > > > > For that reason, I would lean toward treating semantic models similarly > > to > > > other Polaris entities and keeping the API as part of the core Polaris > > > service. We already provide a feature flag to disable the > functionality, > > > which gives operators and downstream distributions the flexibility to > > turn > > > it off when it is not needed. > > > > > > Thanks, > > > Yufei > > > > > > > > > On Mon, Jun 22, 2026 at 2:28 PM Dmitri Bourlatchkov < > > > [email protected]> wrote: > > > > > > > Hi Yufei, > > > > > > > > Persisting OSI data as Polaris entities sounds reasonable to me. > > > > > > > > However, I believe the REST API layer for OSI should be structured > as a > > > > module with opt in/out opportunities for downstream builds (similar > to > > > the > > > > Metric query API). This is not a feature flag concern, but a point > > about > > > > the composition of the Polaris code. A modular approach promotes code > > > > clarity and allows both including the new API into default Polaris > > > > images as well as flexibility downstream projects. I do not see any > > > > downside to the modular approach. > > > > > > > > Feature flags can certainly be supported in the new API modules. > > > > > > > > Cheers, > > > > Dmitri. > > > > > > > > On Mon, Jun 22, 2026 at 5:00 PM Yufei Gu <[email protected]> > wrote: > > > > > > > > > Anand, thanks for chiming in. Looking forward to work together on > it. > > > > > > > > > > Dmitri, Adam, Adnan, thanks for the clarification. I think we can > > > > separate > > > > > a few concerns here. > > > > > > > > > > Apache Ossie specifies the OSI model spec itself, but not the CRUD > > REST > > > > > endpoints for managing OSI documents in Polaris. Polaris has the > > > > > opportunity to define those APIs. As Adam mentioned, the validator > is > > > > > intended for Ossie schema validation. That should definitely be > > version > > > > > based, so Polaris can validate the submitted document against the > > > > > corresponding OSI spec version while keeping the REST API contract > > > under > > > > > Polaris control. > > > > > > > > > > On the "first class" point, I think Adnan's interpretation is > > correct. > > > > The > > > > > intent is that a semantic model is a Polaris entity in the same > sense > > > as > > > > an > > > > > Iceberg table, view, generic table, or policy. It participates in > the > > > > > Polaris metadata model, authorization model, and lifecycle as a > > managed > > > > > entity. In that sense, it is different from metrics or events, > which > > > are > > > > > auxiliary data associated with entities rather than entities > > > themselves. > > > > > > > > > > On the "always active" point, providing a feature flag makes sense, > > > this > > > > is > > > > > already included in PR 4816. We can run the OSI API by default in > the > > > > > Apache Polaris build, but allow downstream admins to turn it off if > > > they > > > > do > > > > > not need it in their deployment. > > > > > > > > > > Thanks, > > > > > Yufei > > > > > > > > > > > > > > > On Mon, Jun 22, 2026 at 1:37 PM Anand Kumar Sankaran via dev < > > > > > [email protected]> wrote: > > > > > > > > > > > JB and Yufei, > > > > > > > > > > > > Thanks for doing this. We have customers asking for this as well. > > > Happy > > > > > to > > > > > > help in any way possible. > > > > > > > > > > > > - > > > > > > Anand > > > > > > > > > > > > From: Adnan Hemani via dev <[email protected]> > > > > > > Date: Monday, June 22, 2026 at 12:18 PM > > > > > > To: [email protected] <[email protected]> > > > > > > Cc: Adnan Hemani <[email protected]> > > > > > > Subject: Re: [DISCUSS] Semantic Layer Support in Apache Polaris > > > > > > > > > > > > This Message Is From an External Sender > > > > > > This message came from outside your organization. > > > > > > Report Suspicious< > > > > > > > > > > > > > > > > > > > > > https://us-phishalarm-ewt.proofpoint.com/EWT/v1/Iz9xO38YGHZK!YhNDZAGomgiHL51L-6FL3QPZjxHXwiq6JCAQHbb6PAE7K6Eqwb--zyy23NolE2-B94Vu6rTO00mQ6c0S3xLY-wGl3G8wkj5qTIJjWF_iK7wIvcJej0eX1hsbj7Uhl7_c$ > > > > > > > > > > > > > > > > > > > > > > > > > Hi Adam, Dmitri, Yufei, > > > > > > > > > > > > Adding in a clarification: I believe "first class" in the context > > of > > > > OSI > > > > > > would mean that it is given the same level of importance as a > > Polaris > > > > > > entity as a Table or View would. Is that generally correct? > > > > > > > > > > > > Best, > > > > > > Adnan Hemani > > > > > > > > > > > > On Mon, Jun 22, 2026 at 10:50 AM Adam Christian < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > This proposal [1] includes a second tab with the detailed > design. > > > It > > > > > > shows > > > > > > > the REST APIs that handle the CRUD operations for OSI Semantic > > > > Models. > > > > > > The > > > > > > > Semantic Model will be validated in the OsiDocumentValidator > > which > > > I > > > > > > assume > > > > > > > will validate against the Apache Ossie version. In my reading, > > > > Polaris > > > > > > does > > > > > > > not control it; we will leverage the upstream spec. > > > > > > > > > > > > > > Regarding OSI functionality if this feature is always active, I > > > > assume > > > > > > > users would benefit from it being active. If an admin user does > > not > > > > > want > > > > > > to > > > > > > > leverage OSI inside their Polaris instance, they simply won't > > grant > > > > the > > > > > > > privileges to the consuming users. > > > > > > > > > > > > > > [1] - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 6:13 PM Dmitri Bourlatchkov < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > Sorry for getting to this proposal late. I postred some > > comments > > > on > > > > > PR > > > > > > > > 4816, recounting the key points here in more detail. > > > > > > > > > > > > > > > > * From the proposal doc: Goal G1: "Store OSI 0.1.x documents > as > > > > > > > first-class > > > > > > > > Polaris entities, scoped under a Namespace" > > > > > > > > > > > > > > > > I believe this needs a bit more discussion before we proceed > to > > > > > > concrete > > > > > > > > code changes. The idea of persisting OSI data is totally > valid. > > > > > > However, > > > > > > > > I'm not sure what "first class" means in this context? Does > it > > > mean > > > > > > that > > > > > > > > OSI functionality has to be active all the time? > > > > > > > > > > > > > > > > My initial perception of this proposal is that as a use case > it > > > is > > > > > > > similar > > > > > > > > to persisting Metrics (or Events) in Polaris. That is, the > > > feature > > > > is > > > > > > > > valuable, but downstream projects may want to have the > > > flexibility > > > > of > > > > > > > > deciding whether to include it or not. > > > > > > > > > > > > > > > > * Another point I'd like to clarity is about the REST API > > > > definition. > > > > > > Are > > > > > > > > API endpoints going be defined and controlled by the Polaris > > > > project? > > > > > > > > > > > > > > > > * Are REST API payload types defined and controlled by > Polaris > > or > > > > by > > > > > > > Apache > > > > > > > > Ossie [1]? > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://www.mail-archive.com/[email protected]/msg86564.html__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfgVHpkQ$ > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > On Fri, May 29, 2026 at 6:34 PM Yufei Gu < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > As AI agents, BI tools, notebooks, and query engines > > > increasingly > > > > > > > consume > > > > > > > > > the same data, semantic definitions such as metrics and > > > > dimensions > > > > > > are > > > > > > > > > often duplicated across multiple systems. This leads to > > > > > inconsistent > > > > > > > > > definitions, duplicated effort, and governance challenges. > > The > > > > rise > > > > > > of > > > > > > > AI > > > > > > > > > agents further amplifies this problem, as agents rely on > > > semantic > > > > > > > context > > > > > > > > > to understand data and reason about business concepts. > > Without > > > a > > > > > > shared > > > > > > > > > semantic layer, organizations often end up maintaining > > multiple > > > > > > > versions > > > > > > > > of > > > > > > > > > the same business definitions across tools and > applications. > > > > > > > > > > > > > > > > > > JB and I would like to start a discussion on adding > semantic > > > > layer > > > > > > > > support > > > > > > > > > to Apache Polaris so semantic models can be defined once, > > > > governed > > > > > > > > > centrally, and consumed consistently across tools. The > > > > proposal[1] > > > > > > > > > introduces semantic models as a first class Polaris entity > > > using > > > > > the > > > > > > > Open > > > > > > > > > Semantic Interchange (OSI)[2] specification[3]. At a high > > > level, > > > > > the > > > > > > > > > proposal adds: > > > > > > > > > > > > > > > > > > - A new SEMANTIC_MODEL entity type > > > > > > > > > - CRUD APIs for semantic models > > > > > > > > > - Schema validation and authorization > > > > > > > > > > > > > > > > > > Polaris remains a metadata service and does not execute > > metrics > > > > or > > > > > > > > semantic > > > > > > > > > queries. > > > > > > > > > Feedback on the overall direction, design, and OSI adoption > > > would > > > > > be > > > > > > > > > greatly appreciated. > > > > > > > > > > > > > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?usp=sharing__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBIq6oELUw$ > > > > > > > > > 2. > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://open-semantic-interchange.org__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBKnqDA0QQ$ > > > > > > > > > 3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/open-semantic-interchange/OSI/blob/main/core-spec/spec.md__;!!Iz9xO38YGHZK!5zRt5PLr106Rj8WbH_RftJ4SqCWP119n37Z77kzoNL-_JhobudorMvD0UdqyXJTi1PCMu0vGL3KGPBLfoGUc7Q$ > > > > > > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Dmitri Bourlatchkov > > > > Senior Staff Software Engineer, Dremio > > > > Dremio.com > > > > < > > > > > > > > > > https://www.dremio.com/?utm_medium=email&utm_source=signature&utm_term=na&utm_content=email-signature&utm_campaign=email-signature > > > > > > > > > / > > > > Follow Us on LinkedIn <https://www.linkedin.com/company/dremio> / > Get > > > > Started <https://www.dremio.com/get-started/> > > > > > > > > > > > > The Agentic Lakehouse > > > > The only lakehouse built for agents, managed by agents > > > > > > > > > >
