Thanks for the discussion.
Dmitri, could you clarify what the envelope would look like?
My current understanding is that it might look roughly like this:
{
"format": "ossie",
"version": "1.0",
"payload": {
"...": "..."
}
}
For a non JSON format, the payload would instead need to be
represented as an encoded string:
{
"format": "okf",
"version": "1.0",
"payload": "encoded document content"
}
If that is the intended design, I am not sure the envelope gives us
much beyond option 1, as clients still need format specific handling.
So the clearest choices seem to be either:
1. Direct JSON for Ossie, with no envelope, that seems to be what JB
proposed. In that case, we'd have to change the spec or introduce a v2
endpoint when we introduce a new format like OKF.
2. a format-agnostic envelope with an opaque string payload.
Am I missing another benefit of the envelope approach?
Thanks,
Yufei
Yufei
On Thu, Jul 16, 2026 at 4:22 AM Jean-Baptiste Onofré <[email protected]> wrote:
>
> Hi everyone,
>
> I prefer to store and expose the semantic API using a direct JSON
> payload format ("raw").
>
> I also suggest that we focus exclusively on the Apache Ossie format
> for now, without any wrapping.
>
> Regarding the evolution of the Ossie specification, I believe it is
> better to store raw JSON and handle the plumbing on our end. This
> approach allows any tool, including AI tools, to directly consume the
> Ossie definition.
>
> Regards,
> JB
>
> On Mon, Jul 13, 2026 at 8:13 PM Yufei Gu <[email protected]> wrote:
> >
> > Hi Dmitri, Thanks! I think option 2 is a reasonable choice. The advantage I
> > see with option 1 is future extensibility. If we later decide to support
> > another semantic model format that isn't JSON, such as OKF, option 2 would
> > require a REST spec change, while option 1 would not. That's why I slightly
> > prefer option 1, although I think either option is much better than
> > modeling the semantic model structure directly in the REST API. What do you
> > think?
> >
> > Thanks,
> > Yufei
> >
> >
> > On Mon, Jul 13, 2026 at 11:10 AM Yufei Gu <[email protected]> wrote:
> >
> > > Hi Robert,
> > >
> > > Thanks for the thoughtful feedback.
> > >
> > > I agree that the current API is narrower than a complete semantic layer.
> > > Today it's primarily about managing the lifecycle of semantic model
> > > documents. Discovery, search, and client consumption are all important
> > > topics, but I think they're broader than the Ossie semantic model itself.
> > > We touched on some of these in the last community sync[1], and I'm happy
> > > to
> > > continue that discussion in a separate thread.
> > >
> > > I also agree that we should be careful not to overstate what this initial
> > > API provides. Calling it a beta semantic model registry or document
> > > hosting
> > > API seems reasonable.
> > >
> > > That said, I think those questions are orthogonal to the payload
> > > representation discussion. My original topic was simply whether the REST
> > > API should treat the semantic model as a raw document, an opaque JSON
> > > payload, or model the schema directly. Since the consumption story is
> > > still
> > > evolving, I think keeping the REST contract loosely coupled to the
> > > underlying semantic model specification gives us the most flexibility.
> > >
> > > [1]
> > > https://drive.google.com/file/d/1hxYkk2t-BcnFOk8eJG9NYCHfjkOXg3Iz/view?usp=sharing
> > >
> > > Thanks,
> > >
> > > Yufei
> > >
> > > On Mon, Jul 13, 2026 at 6:16 AM Robert Stupp <[email protected]> wrote:
> > >
> > >> Hi,
> > >>
> > >> I think the payload representation question depends on the client model
> > >> we
> > >> expect this API to support.
> > >>
> > >> I support Polaris hosting Apache Ossie semantic-model documents as a beta
> > >> foundation. That seems like a useful first step while Ossie itself is
> > >> still
> > >> evolving.
> > >>
> > >> But I do not think we should describe the current API as enabling AI
> > >> tools,
> > >> BI tools, or human semantic-model discovery yet.
> > >>
> > >> The merged API is primarily namespace/name CRUD for a semantic-model
> > >> document.
> > >> That is opaque document storage and exact point-retrieval.
> > >>
> > >> It does not enable clients to discover the right semantic model from a
> > >> query,
> > >> nor to find semantic models by table, metric, domain, user, or
> > >> capability.
> > >>
> > >> It also does not define a standard consuming API or tool contract,
> > >> search/indexing contract, freshness model, or current/trusted/certified
> > >> model
> > >> semantics.
> > >>
> > >> That distinction matters because the REST API is the user-facing
> > >> contract.
> > >>
> > >> If the beta API is intended only as opaque document hosting, I think the
> > >> spec
> > >> and docs should say that clearly, and users/clients should not infer
> > >> broader
> > >> discovery or interoperability semantics from the CRUD API.
> > >>
> > >> I am not asking to solve the full semantic-layer story immediately.
> > >>
> > >> I am asking that durable implementation work does not get ahead of the
> > >> client-consumption story.
> > >>
> > >> The client model should drive the persistent data model, not the other
> > >> way
> > >> around.
> > >>
> > >> Once semantic models are stored as durable Polaris entities, choices
> > >> around
> > >> identity, versioning, validation, indexing, size limits, source-table
> > >> references, and freshness become much harder to change.
> > >>
> > >> For now, I would be comfortable describing this as beta Apache Ossie
> > >> document
> > >> hosting / semantic-model registry work.
> > >>
> > >> I would not yet be comfortable describing it as enabling AI, BI, or human
> > >> semantic workflows until the discovery and client-consumption story
> > >> exists.
> > >>
> > >> Robert
> > >>
> > >>
> > >> On Fri, Jul 10, 2026 at 3:10 PM Dmitri Bourlatchkov <[email protected]>
> > >> wrote:
> > >>
> > >> > Hi Yufei,
> > >> >
> > >> > Thanks for starting this thread!
> > >> >
> > >> > I tend to think it is best to represent Ossie data as direct JSON
> > >> without
> > >> > defining its structure in the Polaris OpenAPI spec (Ossie schemas are
> > >> > controlled by Ossie, not Polaris). I believe this corresponds to Option
> > >> 2
> > >> > from your email.
> > >> >
> > >> > Polaris code that implements the new API will then interpret the Ossie
> > >> > parts according to the declared version of the Ossie spec.
> > >> >
> > >> > With that in mind, the API should clearly state the format (Ossie or
> > >> OKF)
> > >> > and the specification version of the semantic data sub-object
> > >> (apologies if
> > >> > it has that already, I'm behind on the related PR updates). Obviously
> > >> OKF
> > >> > will have a different representation in the payload, but this should
> > >> > not
> > >> > prevent Ossie from leveraging JSON synergies.
> > >> >
> > >> > Anand's work on solving a similar problem in the Metrics API [4115] may
> > >> be
> > >> > reusable here.
> > >> >
> > >> > [4115] https://github.com/apache/polaris/pull/4115
> > >> >
> > >> > Cheers,
> > >> > Dmitri.
> > >> >
> > >> >
> > >> > On Wed, Jul 8, 2026 at 2:20 PM Yufei Gu <[email protected]> wrote:
> > >> >
> > >> > > Hi folks,
> > >> > >
> > >> > > Following JB's suggestion, I'd like to start a dedicated discussion
> > >> > > on
> > >> > the
> > >> > > REST API payload representation for semantic models.
> > >> > >
> > >> > > I think there are three possible approaches:
> > >> > >
> > >> > > 1.
> > >> > >
> > >> > > Represent the semantic model as a raw string.
> > >> > > 2.
> > >> > >
> > >> > > Represent the semantic model as an opaque JSON document.
> > >> > > 3.
> > >> > >
> > >> > > Model the semantic model structure directly in the REST
> > >> specification.
> > >> > >
> > >> > > I think it's helpful to separate the REST API from Polaris' internal
> > >> > > representation. The REST API is the long-term contract with clients,
> > >> > while
> > >> > > the internal representation can evolve independently.
> > >> > >
> > >> > > I'm comfortable with either option 1 or option 2. Both avoid coupling
> > >> the
> > >> > > REST API to the Ossie schema and allow Polaris to validate the
> > >> > > payload
> > >> > > based on the semantic model type and version while preserving the
> > >> > document
> > >> > > through write and read operations.
> > >> > >
> > >> > > My concern is with option 3. Since the Ossie schema is versioned and
> > >> > > expected to evolve, modeling the full semantic model structure
> > >> directly
> > >> > in
> > >> > > the REST specification would tightly couple the Polaris REST API to
> > >> Ossie
> > >> > > versions. Every Ossie schema evolution could require changes to the
> > >> REST
> > >> > > specification, generated clients, and potentially client
> > >> > > applications.
> > >> > >
> > >> > > Between options 1 and 2, I think there is an additional tradeoff.
> > >> > >
> > >> > > An opaque JSON document assumes that semantic models are always
> > >> > represented
> > >> > > as JSON. While that works well for Ossie today, Polaris may support
> > >> other
> > >> > > semantic model formats in the future. For example, OKF[1] is defined
> > >> as
> > >> > > Markdown rather than JSON. Using a raw string keeps the REST API
> > >> > > independent of any particular document format, allowing Polaris to
> > >> > support
> > >> > > JSON, Markdown, or other representations without changing the API
> > >> > contract.
> > >> > >
> > >> > > So my current view is:
> > >> > >
> > >> > > -
> > >> > >
> > >> > > Option 1 provides the greatest flexibility and is format agnostic.
> > >> > > -
> > >> > >
> > >> > > Option 2 is a natural choice if we want to optimize specifically
> > >> for
> > >> > > JSON based semantic models.
> > >> > > -
> > >> > >
> > >> > > Option 3 provides strong typing, but at the cost of coupling the
> > >> REST
> > >> > > API to Ossie schema evolution.
> > >> > >
> > >> > > I'm happy with either option 1 or option 2, but I'd avoid option 3
> > >> > > for
> > >> > the
> > >> > > reasons above.
> > >> > >
> > >> > > Thoughts?
> > >> > >
> > >> > > 1.
> > >> > >
> > >> > >
> > >> >
> > >> https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing
> > >> > >
> > >> > > Thanks,
> > >> > >
> > >> > > Yufei
> > >> > >
> > >> >
> > >>
> > >
