This is an automated email from the ASF dual-hosted git repository.
mchades pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new cb0accec80 [#12614] feat(python): Support Ossie Semantic Model import
and export (#13795)
cb0accec80 is described below
commit cb0accec80cf8e7591ebb2c482029ede2335ddc4
Author: Akshay Thorat <[email protected]>
AuthorDate: Mon Oct 12 20:42:30 2026 -0700
[#12614] feat(python): Support Ossie Semantic Model import and export
(#13795)
### What changes were proposed in this pull request?
Add `import_ossie_document(namespace, document, document_format="yaml")`
and `export_ossie_document(identifier, document_format="yaml")` to the
Python Semantic Model catalog API.
Use the existing HTTP client for raw UTF-8 YAML/JSON text and the
lossless Semantic Model response decoder. Export returns document text
unchanged. Add unit tests, a separate Ossie integration-test module, and
a usage example without adding dependencies.
### Why are the changes needed?
The server supports Ossie import/export, but the Python SDK lacks these
operations. The new real-server round-trip test reproduced the missing
method before implementation.
Part of #12209.
Fix: #12614
### Does this PR introduce _any_ user-facing change?
Yes. Python users can import and export Ossie YAML/JSON documents,
defaulting to YAML. Imports report conflicts without replacing existing
models.
### How was this patch tested?
- Full Python unit suite: 1,221 passed (`python -m unittest discover -s
tests/unittests`).
- All 6 new real-server integration tests passed
(`START_EXTERNAL_GRAVITINO=true python -m unittest -v
tests.integration.test_semantic_model_ossie`).
- Round trips verify both formats, exact decimals/large integers,
additional AI context, properties, and custom extensions. Error cases
verify malformed documents, missing schema/model/source, and
non-destructive conflicts.
- Changed-file Black/Pylint, repository Spotless, and `git diff --check`
passed.
- Tests ran from `clients/client-python` on Python 3.11 against a JDK 17
server built from current main with `./gradlew compileDistribution
-PskipWeb=true -PskipTrinoConnector=true -PskipTests -PskipITs -x test`.
Dependencies: none beyond main at `b837e07a9`, including merged #12860.
This does not stack on #13768 or change metadata/privilege mappings,
tag/owner support, or server authorization.
No new integration tests were skipped. The full repository integration
suite and Python-version matrix were not run.
---
.../api/semantic/semantic_model_catalog.py | 57 +++-
.../client/semantic_model_catalog_operations.py | 39 +++
.../client-python/gravitino/utils/http_client.py | 5 +-
.../tests/integration/test_semantic_model_ossie.py | 300 +++++++++++++++++++++
.../tests/unittests/test_http_client_raw_body.py | 64 +++++
.../tests/unittests/test_semantic_model_ossie.py | 250 +++++++++++++++++
docs/how-to-use-python-client.md | 33 +++
7 files changed, 746 insertions(+), 2 deletions(-)
diff --git
a/clients/client-python/gravitino/api/semantic/semantic_model_catalog.py
b/clients/client-python/gravitino/api/semantic/semantic_model_catalog.py
index 81b472a091..815781a699 100644
--- a/clients/client-python/gravitino/api/semantic/semantic_model_catalog.py
+++ b/clients/client-python/gravitino/api/semantic/semantic_model_catalog.py
@@ -21,7 +21,10 @@ from typing import Optional
from gravitino.api.semantic.semantic_model import SemanticModel
from gravitino.api.semantic.semantic_model_change import SemanticModelChange
from gravitino.api.semantic.semantic_model_definition import
SemanticModelDefinition
-from gravitino.exceptions.base import NoSuchSemanticModelException
+from gravitino.exceptions.base import (
+ NoSuchSemanticModelException,
+ UnsupportedOperationException,
+)
from gravitino.name_identifier import NameIdentifier
from gravitino.namespace import Namespace
@@ -113,6 +116,58 @@ class SemanticModelCatalog(ABC):
If the definition is invalid.
"""
+ def import_ossie_document(
+ self, namespace: Namespace, document: str, document_format: str =
"yaml"
+ ) -> SemanticModel:
+ """Import standalone Apache Ossie document text as a new Semantic
Model.
+
+ The name comes from the document. Import never replaces an existing
model.
+ The server parses the declared format without falling back to another
+ parser, and applies the same validation as
:meth:`create_semantic_model`.
+ A root ``GRAVITINO_PROPERTIES`` custom extension supplies model
properties;
+ other custom extensions and additional AI-context properties are
preserved.
+
+ Args:
+ namespace (Namespace): The destination schema namespace.
+ document (str): Raw document text, not a path or a JSON-encoded
string.
+ document_format (str): ``"yaml"`` (default) or ``"json"``.
+
+ Returns:
+ SemanticModel: The created model, including its definition and
audit info.
+
+ Raises:
+ IllegalArgumentException: If the text is blank or the format is
unsupported.
+ IllegalSemanticModelException: If the document or definition is
invalid.
+ NoSuchSchemaException: If the destination schema does not exist.
+ SemanticModelAlreadyExistsException: If the model already exists.
+ UnsupportedOperationException: If this catalog does not support
import.
+ """
+ raise UnsupportedOperationException("Ossie import is not supported")
+
+ def export_ossie_document(
+ self, identifier: NameIdentifier, document_format: str = "yaml"
+ ) -> str:
+ """Export a Semantic Model as standalone Apache Ossie document text.
+
+ The ``ossie-version`` property supplies the document version. Other
model
+ properties are encoded in a root ``GRAVITINO_PROPERTIES`` custom
extension.
+ The SDK returns the server's text without parsing or reformatting it.
+
+ Args:
+ identifier (NameIdentifier): The Semantic Model identifier.
+ document_format (str): ``"yaml"`` (default) or ``"json"``.
+
+ Returns:
+ str: The serialized document in the requested format.
+
+ Raises:
+ IllegalArgumentException: If the format is unsupported.
+ NoSuchSemanticModelException: If the model does not exist.
+ IllegalSemanticModelException: If the model cannot be exported.
+ UnsupportedOperationException: If this catalog does not support
export.
+ """
+ raise UnsupportedOperationException("Ossie export is not supported")
+
@abstractmethod
def alter_semantic_model(
self, identifier: NameIdentifier, *changes: SemanticModelChange
diff --git
a/clients/client-python/gravitino/client/semantic_model_catalog_operations.py
b/clients/client-python/gravitino/client/semantic_model_catalog_operations.py
index ff3569fd61..5cb820f3e5 100644
---
a/clients/client-python/gravitino/client/semantic_model_catalog_operations.py
+++
b/clients/client-python/gravitino/client/semantic_model_catalog_operations.py
@@ -123,6 +123,37 @@ class SemanticModelCatalogOperations(SemanticModelCatalog):
)
return self._to_semantic_model(resp)
+ def import_ossie_document(
+ self, namespace: Namespace, document: str, document_format: str =
"yaml"
+ ) -> SemanticModel:
+ self._check_semantic_model_namespace(namespace)
+ if not isinstance(document, str) or not document.strip():
+ raise IllegalArgumentException("Ossie document must be non-blank
text")
+ media_type = self._ossie_media_type(document_format)
+ full_namespace = self._get_semantic_model_full_namespace(namespace)
+ resp = self._rest_client.post(
+
f"{self._format_semantic_model_request_path(full_namespace)}/ossie",
+ data=document,
+ headers={"Content-Type": media_type},
+ error_handler=SEMANTIC_MODEL_ERROR_HANDLER,
+ )
+ return self._to_semantic_model(resp)
+
+ def export_ossie_document(
+ self, identifier: NameIdentifier, document_format: str = "yaml"
+ ) -> str:
+ self._check_semantic_model_name_identifier(identifier)
+ media_type = self._ossie_media_type(document_format)
+ full_namespace =
self._get_semantic_model_full_namespace(identifier.namespace())
+ resp = self._rest_client.get(
+ f"{self._format_semantic_model_request_path(full_namespace)}"
+ f"/{encode_string(identifier.name())}/ossie",
+ params={"format": document_format},
+ headers={"Accept": media_type},
+ error_handler=SEMANTIC_MODEL_ERROR_HANDLER,
+ )
+ return resp.body.decode("utf-8")
+
def alter_semantic_model(
self, identifier: NameIdentifier, *changes: SemanticModelChange
) -> SemanticModel:
@@ -155,6 +186,14 @@ class SemanticModelCatalogOperations(SemanticModelCatalog):
drop_resp.validate()
return drop_resp.dropped()
+ @staticmethod
+ def _ossie_media_type(document_format: str) -> str:
+ if document_format not in ("yaml", "json"):
+ raise IllegalArgumentException(
+ "Ossie document format must be 'yaml' or 'json'"
+ )
+ return f"application/{document_format}"
+
@staticmethod
def _to_semantic_model(resp) -> SemanticModel:
semantic_model_resp = SemanticModelResponse.from_json(
diff --git a/clients/client-python/gravitino/utils/http_client.py
b/clients/client-python/gravitino/utils/http_client.py
index c7fece2bd5..1196e17ef9 100644
--- a/clients/client-python/gravitino/utils/http_client.py
+++ b/clients/client-python/gravitino/utils/http_client.py
@@ -168,7 +168,10 @@ class HTTPClient:
method = method.upper()
request_data = None
- if data:
+ if isinstance(data, (str, bytes)):
+ request_data = data.encode("utf-8") if isinstance(data, str) else
data
+ self._update_headers(self.JSON_HEADER)
+ elif data:
request_data = urlencode(data.to_dict()).encode()
self._update_headers(self.FORMDATA_HEADER)
else:
diff --git
a/clients/client-python/tests/integration/test_semantic_model_ossie.py
b/clients/client-python/tests/integration/test_semantic_model_ossie.py
new file mode 100644
index 0000000000..1e623ed756
--- /dev/null
+++ b/clients/client-python/tests/integration/test_semantic_model_ossie.py
@@ -0,0 +1,300 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import json
+from decimal import Decimal
+from uuid import uuid4
+
+import simplejson
+
+from gravitino import Catalog, GravitinoAdminClient, NameIdentifier
+from gravitino.api.rel.types.types import Types
+from gravitino.dto.rel.column_dto import ColumnDTO
+from gravitino.exceptions.base import (
+ IllegalArgumentException,
+ IllegalSemanticModelException,
+ NoSuchSchemaException,
+ NoSuchSemanticModelException,
+ SemanticModelAlreadyExistsException,
+)
+from gravitino.namespace import Namespace
+from tests.integration.integration_test_env import IntegrationTestEnv
+
+
+class TestSemanticModelOssie(IntegrationTestEnv):
+ """Exercise Ossie documents through the public SDK and a real server."""
+
+ def setUp(self):
+ super().setUp()
+ admin = GravitinoAdminClient(uri="http://localhost:8090";)
+ self.addCleanup(admin.close)
+ metalake_name = "ossie_it_" + uuid4().hex
+ client = self.create_metalake_client(admin, metalake_name)
+ self.addCleanup(client.close)
+ self.addCleanup(admin.drop_metalake, metalake_name, force=True)
+ self.catalog = client.create_catalog(
+ "ossie_catalog", Catalog.Type.RELATIONAL, "lakehouse-generic", "",
{}
+ )
+ self.catalog.as_schemas().create_schema("source", "", {})
+ self.catalog.as_table_catalog().create_table(
+ NameIdentifier.of("source", "orders"),
+ [
+ ColumnDTO.builder()
+ .with_name("order_id")
+ .with_data_type(Types.StringType.get())
+ .build()
+ ],
+ properties={
+ "format": "delta",
+ "external": "true",
+ "location": f"file:///tmp/{metalake_name}/orders",
+ },
+ )
+ self.models = self.catalog.as_semantic_model_catalog()
+ self.schema = "models_" + uuid4().hex
+ self.catalog.as_schemas().create_schema(self.schema, "", {})
+ self.addCleanup(self.catalog.as_schemas().drop_schema, self.schema,
True)
+ self.namespace = Namespace.of(self.schema)
+ self.identifier = NameIdentifier.of(self.schema, "sales")
+
+ @staticmethod
+ def _yaml_document():
+ return """version: 0.2.0.dev0
+name: sales
+description: Sales definitions - café 雪
+ai_context:
+ instructions: Use certified definitions
+ nested:
+ precise: 0.12345678901234567890123456789
+ large: 1234567890123456789012345678901234567890
+ values: [true, null, extra]
+datasets:
+ - name: orders
+ source: ossie_catalog.source.orders
+ primary_key: [order_id]
+ fields:
+ - name: order_id
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: orders.order_id
+ datatype: String
+ ai_context: Order identifier
+metrics:
+ - name: order_count
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: COUNT(orders.order_id)
+ datatype: Integer
+custom_extensions:
+ - vendor_name: EXAMPLE
+ data: '{"certified":true}'
+ - vendor_name: GRAVITINO_PROPERTIES
+ data: '{"domain":"sales"}'
+"""
+
+ @staticmethod
+ def _json_document():
+ return simplejson.dumps(
+ {
+ "version": "0.2.0.dev0",
+ "name": "sales",
+ "description": "Sales definitions - café 雪",
+ "ai_context": {
+ "instructions": "Use certified definitions",
+ "nested": {
+ "precise": Decimal("0.12345678901234567890123456789"),
+ "large": 1234567890123456789012345678901234567890,
+ "values": [True, None, "extra"],
+ },
+ },
+ "datasets": [
+ {
+ "name": "orders",
+ "source": "ossie_catalog.source.orders",
+ "primary_key": ["order_id"],
+ "fields": [
+ {
+ "name": "order_id",
+ "datatype": "String",
+ "expression": {
+ "dialects": [
+ {
+ "dialect": "ANSI_SQL",
+ "expression": "orders.order_id",
+ }
+ ]
+ },
+ "ai_context": "Order identifier",
+ }
+ ],
+ }
+ ],
+ "metrics": [
+ {
+ "name": "order_count",
+ "datatype": "Integer",
+ "expression": {
+ "dialects": [
+ {
+ "dialect": "ANSI_SQL",
+ "expression": "COUNT(orders.order_id)",
+ }
+ ]
+ },
+ }
+ ],
+ "custom_extensions": [
+ {"vendor_name": "EXAMPLE", "data": '{"certified":true}'},
+ {
+ "vendor_name": "GRAVITINO_PROPERTIES",
+ "data": '{"domain":"sales"}',
+ },
+ ],
+ },
+ use_decimal=True,
+ ensure_ascii=False,
+ )
+
+ def _assert_model(self, model):
+ self.assertEqual("sales", model.name())
+ self.assertEqual("Sales definitions - café 雪", model.comment())
+ self.assertEqual("sales", model.properties()["domain"])
+ self.assertEqual("0.2.0.dev0", model.properties()["ossie-version"])
+ definition = model.definition()
+ self.assertEqual(
+ "Use certified definitions",
definition.ai_context().object().instructions()
+ )
+ self.assertEqual('{"certified":true}',
definition.custom_extensions()[0].data())
+ self.assertEqual(
+ NameIdentifier.of("ossie_catalog", "source", "orders"),
+ definition.datasets()[0].source(),
+ )
+ self.assertEqual("order_id",
definition.datasets()[0].fields()[0].name())
+ self.assertEqual("order_count", definition.metrics()[0].name())
+ self.assertEqual("EXAMPLE",
definition.custom_extensions()[0].vendor_name())
+ self.assertEqual(
+ {
+ "nested": {
+ "precise": Decimal("0.12345678901234567890123456789"),
+ "large": 1234567890123456789012345678901234567890,
+ "values": [True, None, "extra"],
+ }
+ },
+ definition.ai_context().object().additional_properties(),
+ )
+
+ def _round_trip(self, content, document_format):
+ imported = self.models.import_ossie_document(
+ self.namespace, content, document_format
+ )
+ self._assert_model(imported)
+ self._assert_model(self.models.load_semantic_model(self.identifier))
+ for output_format in ("yaml", "json"):
+ with self.subTest(output_format=output_format):
+ exported = self.models.export_ossie_document(
+ self.identifier, output_format
+ )
+ self.assertIsInstance(exported, str)
+ if output_format == "json":
+ document = json.loads(exported, parse_float=Decimal)
+ self.assertEqual("sales", document["name"])
+ self.assertEqual(
+ "ossie_catalog.source.orders",
document["datasets"][0]["source"]
+ )
+ self.assertEqual(
+ Decimal("0.12345678901234567890123456789"),
+ document["ai_context"]["nested"]["precise"],
+ )
+
self.assertTrue(self.models.drop_semantic_model(self.identifier))
+ restored = self.models.import_ossie_document(
+ self.namespace, exported, output_format
+ )
+ self._assert_model(restored)
+ self.assertEqual(imported.definition(), restored.definition())
+
+ def test_yaml_round_trip(self):
+ self._round_trip(self._yaml_document(), "yaml")
+
+ def test_json_round_trip(self):
+ self._round_trip(self._json_document(), "json")
+
+ def test_default_format_is_yaml(self):
+ imported = self.models.import_ossie_document(
+ self.namespace, self._yaml_document()
+ )
+ self._assert_model(imported)
+ exported = self.models.export_ossie_document(self.identifier)
+ self.assertIn("datasets:", exported)
+ self.models.drop_semantic_model(self.identifier)
+ self._assert_model(self.models.import_ossie_document(self.namespace,
exported))
+
+ def test_conflicting_import_does_not_replace_model(self):
+ original = self.models.import_ossie_document(
+ self.namespace, self._yaml_document()
+ )
+ for content, document_format in (
+ (self._yaml_document(), "yaml"),
+ (self._json_document(), "json"),
+ ):
+ with self.subTest(document_format=document_format):
+ with self.assertRaises(SemanticModelAlreadyExistsException):
+ self.models.import_ossie_document(
+ self.namespace,
+ content.replace("Sales definitions", "Changed"),
+ document_format,
+ )
+ loaded = self.models.load_semantic_model(self.identifier)
+ self.assertEqual(original.definition(), loaded.definition())
+ self.assertEqual(original.comment(), loaded.comment())
+
+ def test_malformed_documents_are_rejected_without_creating_models(self):
+ for content, document_format in (
+ ("name: [", "yaml"),
+ ("{", "json"),
+ (self._yaml_document(), "json"),
+ ('{"name":"sales","datasets":[]}', "json"),
+ ):
+ with self.subTest(document_format=document_format,
content=content):
+ with self.assertRaises(IllegalSemanticModelException):
+ self.models.import_ossie_document(
+ self.namespace, content, document_format
+ )
+ self.assertEqual([],
self.models.list_semantic_models(self.namespace))
+ with self.assertRaises(IllegalArgumentException):
+ self.models.import_ossie_document(self.namespace, " ")
+
+ def test_missing_resources(self):
+ for content, document_format in (
+ (self._yaml_document(), "yaml"),
+ (self._json_document(), "json"),
+ ):
+ with self.subTest(document_format=document_format):
+ with self.assertRaises(NoSuchSchemaException):
+ self.models.import_ossie_document(
+ Namespace.of("missing"), content, document_format
+ )
+ with self.assertRaises(NoSuchSemanticModelException):
+ self.models.export_ossie_document(self.identifier,
document_format)
+ with self.assertRaises(IllegalSemanticModelException):
+ self.models.import_ossie_document(
+ self.namespace,
+ content.replace("source.orders", "source.missing"),
+ document_format,
+ )
+ self.assertEqual([],
self.models.list_semantic_models(self.namespace))
diff --git a/clients/client-python/tests/unittests/test_http_client_raw_body.py
b/clients/client-python/tests/unittests/test_http_client_raw_body.py
new file mode 100644
index 0000000000..7db2052893
--- /dev/null
+++ b/clients/client-python/tests/unittests/test_http_client_raw_body.py
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import unittest
+from unittest.mock import Mock, patch
+
+from gravitino.utils.http_client import HTTPClient
+
+
+class TestHTTPClientRawBody(unittest.TestCase):
+ @patch("gravitino.utils.http_client.build_opener")
+ def test_raw_text_and_bytes_use_existing_auth_and_timeout(self,
build_opener):
+ auth = Mock()
+ auth.get_token_data.return_value = b"Bearer token"
+ client = HTTPClient("http://localhost";, auth_data_provider=auth)
+ for body in ("name: café 雪\n", b"name: sales\n", "", b""):
+ with self.subTest(body=body):
+ client.post(
+ "ossie",
+ data=body,
+ headers={"Content-Type": "application/yaml"},
+ timeout=7,
+ )
+ request = build_opener.return_value.open.call_args.args[0]
+ self.assertEqual(
+ body.encode("utf-8") if isinstance(body, str) else body,
+ request.data,
+ )
+ self.assertEqual("application/yaml",
request.get_header("Content-type"))
+ self.assertEqual("Bearer token",
request.get_header("Authorization"))
+ self.assertEqual(
+ 7,
build_opener.return_value.open.call_args.kwargs["timeout"]
+ )
+
+ @patch("gravitino.utils.http_client.build_opener")
+ def test_existing_form_and_json_bodies(self, build_opener):
+ client = HTTPClient("http://localhost";)
+ body = Mock()
+ body.to_dict.return_value = {"grant_type": "client_credentials",
"scope": "a b"}
+ client.post_form("token", data=body)
+ request = build_opener.return_value.open.call_args.args[0]
+ self.assertEqual(b"grant_type=client_credentials&scope=a+b",
request.data)
+ self.assertEqual(
+ "application/x-www-form-urlencoded",
request.get_header("Content-type")
+ )
+ body.to_json.return_value = '{"name":"café"}'
+ client.post("models", json=body)
+ request = build_opener.return_value.open.call_args.args[0]
+ self.assertEqual('{"name":"café"}'.encode("utf-8"), request.data)
+ self.assertEqual("application/json",
request.get_header("Content-type"))
diff --git a/clients/client-python/tests/unittests/test_semantic_model_ossie.py
b/clients/client-python/tests/unittests/test_semantic_model_ossie.py
new file mode 100644
index 0000000000..b531f29ef2
--- /dev/null
+++ b/clients/client-python/tests/unittests/test_semantic_model_ossie.py
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import io
+import json
+import unittest
+from decimal import Decimal
+from unittest.mock import Mock, patch
+from urllib.error import HTTPError
+
+from gravitino.api.semantic.semantic_model_catalog import SemanticModelCatalog
+from gravitino.client.relational_catalog import RelationalCatalog
+from gravitino.dto.audit_dto import AuditDTO
+from gravitino.exceptions.base import (
+ CatalogNotInUseException,
+ ConnectionFailedException,
+ ForbiddenException,
+ IllegalArgumentException,
+ IllegalNameIdentifierException,
+ IllegalNamespaceException,
+ IllegalSemanticModelException,
+ InternalError,
+ MetalakeNotInUseException,
+ NoSuchCatalogException,
+ NoSuchMetalakeException,
+ NoSuchSchemaException,
+ NoSuchSemanticModelException,
+ SemanticModelAlreadyExistsException,
+ UnauthorizedException,
+ UnsupportedOperationException,
+)
+from gravitino.name_identifier import NameIdentifier
+from gravitino.namespace import Namespace
+from gravitino.utils.http_client import HTTPClient
+
+
+class TestSemanticModelOssie(unittest.TestCase):
+ def setUp(self):
+ self.client = HTTPClient("http://localhost:8090";)
+ self.models = RelationalCatalog(
+ catalog_namespace=Namespace.of("metalake /雪"),
+ name="catalog +%",
+ catalog_type=RelationalCatalog.Type.RELATIONAL,
+ provider="lakehouse-generic",
+ audit=AuditDTO("test"),
+ rest_client=self.client,
+ ).as_semantic_model_catalog()
+ self.namespace = Namespace.of("schema #?")
+ self.identifier = NameIdentifier.of("schema #?", "sales /é+%")
+ self.path = (
+ "http://localhost:8090/api/metalakes/metalake%20%2F%E9%9B%AA";
+
"/catalogs/catalog%20%2B%25/schemas/schema%20%23%3F/semantic-models"
+ )
+ self.response_body = b"""{"code":0,"semanticModel":{
+ "name":"sales", "comment":"comment",
"properties":{"domain":"sales"},
+ "audit":{"creator":"test","createTime":"2026-01-01T00:00:00Z"},
+ "definition":{
+
"datasets":[{"name":"orders","source":{"namespace":["catalog","schema"],"name":"orders"}}],
+
"aiContext":{"nested":{"precise":0.12345678901234567890123456789,
+
"large":1234567890123456789012345678901234567890,"values":[true,null,"extra"]}}
+ }
+ }}"""
+ opener_patch = patch("gravitino.utils.http_client.build_opener")
+ self.opener = opener_patch.start().return_value
+ self.addCleanup(opener_patch.stop)
+ self.opener.open.return_value.getcode.return_value = 200
+ self.opener.open.return_value.read.return_value = self.response_body
+
+ def test_import_raw_text_and_lossless_response(self):
+ self.assertIsInstance(self.models, SemanticModelCatalog)
+ for document_format, content in (
+ ("yaml", "name: café 雪\nnumber: 0.12345678901234567890\n"),
+ ("json", '{"name":"café 雪","number":0.12345678901234567890}'),
+ ):
+ with self.subTest(document_format=document_format):
+ model = self.models.import_ossie_document(
+ self.namespace, content, document_format
+ )
+ request = self.opener.open.call_args.args[0]
+ self.assertEqual(self.path + "/ossie", request.full_url)
+ self.assertEqual("POST", request.get_method())
+ self.assertEqual(content.encode("utf-8"), request.data)
+ self.assertEqual(
+ "application/" + document_format,
request.get_header("Content-type")
+ )
+ self.assertEqual(
+ "application/vnd.gravitino.v1+json",
request.get_header("Accept")
+ )
+ self.assertEqual("sales", model.name())
+ self.assertEqual("comment", model.comment())
+ self.assertEqual({"domain": "sales"}, model.properties())
+ self.assertEqual("test", model.audit_info().creator())
+ self.assertEqual(
+ {
+ "nested": {
+ "precise":
Decimal("0.12345678901234567890123456789"),
+ "large": 1234567890123456789012345678901234567890,
+ "values": [True, None, "extra"],
+ }
+ },
+
model.definition().ai_context().object().additional_properties(),
+ )
+
+ def test_export_returns_document_text_unchanged(self):
+ for document_format, content in (
+ ("yaml", "---\nname: café 雪\nnumber: 0.12345678901234567890\n"),
+ ("json", '{ "name": "café 雪", "number": 0.12345678901234567890
}\n'),
+ ):
+ with self.subTest(document_format=document_format):
+ self.opener.open.return_value.read.return_value =
content.encode(
+ "utf-8"
+ )
+ exported = self.models.export_ossie_document(
+ self.identifier, document_format
+ )
+ request = self.opener.open.call_args.args[0]
+ self.assertEqual(
+ self.path
+ + "/sales%20%2F%C3%A9%2B%25/ossie?format="
+ + document_format,
+ request.full_url,
+ )
+ self.assertEqual("GET", request.get_method())
+ self.assertIsNone(request.data)
+ self.assertEqual(
+ "application/" + document_format,
request.get_header("Accept")
+ )
+ self.assertEqual(content, exported)
+
+ def test_default_format_and_headers_do_not_leak(self):
+ self.models.import_ossie_document(self.namespace, "name: sales")
+ self.assertEqual(
+ "application/yaml",
+ self.opener.open.call_args.args[0].get_header("Content-type"),
+ )
+ self.models.export_ossie_document(self.identifier)
+ self.assertTrue(
+
self.opener.open.call_args.args[0].full_url.endswith("?format=yaml")
+ )
+ self.assertEqual(
+ "application/yaml",
self.opener.open.call_args.args[0].get_header("Accept")
+ )
+ self.models.load_semantic_model(self.identifier)
+ self.assertEqual(
+ "application/vnd.gravitino.v1+json",
+ self.opener.open.call_args.args[0].get_header("Accept"),
+ )
+ self.assertEqual(
+ "application/json",
+ self.opener.open.call_args.args[0].get_header("Content-type"),
+ )
+
+ def test_invalid_local_arguments_do_not_send_requests(self):
+ for content in (None, "", " \t\n", b"name: sales", {"name": "sales"}):
+ with self.subTest(content=content), self.assertRaises(
+ IllegalArgumentException
+ ):
+ self.models.import_ossie_document(self.namespace, content)
+ for document_format in (None, "", "csv", "YAML", 1, []):
+ with self.subTest(document_format=document_format):
+ with self.assertRaises(IllegalArgumentException):
+ self.models.import_ossie_document(
+ self.namespace, "name: sales", document_format
+ )
+ with self.assertRaises(IllegalArgumentException):
+ self.models.export_ossie_document(self.identifier,
document_format)
+ for namespace in (None, Namespace.empty(), Namespace.of("catalog",
"schema")):
+ with self.subTest(namespace=namespace), self.assertRaises(
+ IllegalNamespaceException
+ ):
+ self.models.import_ossie_document(namespace, "name: sales")
+ with self.assertRaises(IllegalNameIdentifierException):
+ self.models.export_ossie_document(None)
+ with self.assertRaises(IllegalNamespaceException):
+ self.models.export_ossie_document(NameIdentifier.of("model"))
+ self.opener.open.assert_not_called()
+
+ def test_import_validates_native_response(self):
+ for body, message in (
+ (b'{"code":0,"semanticModel":null}', "semanticModel must not be
null"),
+ (
+ self.response_body.replace(b'"code":0', b'"code":-1'),
+ "code must be >= 0",
+ ),
+ ):
+ with self.subTest(body=body):
+ self.opener.open.return_value.read.return_value = body
+ with self.assertRaisesRegex(IllegalArgumentException, message):
+ self.models.import_ossie_document(self.namespace, "name:
sales")
+
+ def test_http_error_envelopes_map_to_sdk_exceptions(self):
+ cases = (
+ (400, 1001, IllegalSemanticModelException),
+ (400, 1001, IllegalArgumentException),
+ (404, 1003, NoSuchMetalakeException),
+ (404, 1003, NoSuchCatalogException),
+ (404, 1003, NoSuchSchemaException),
+ (404, 1003, NoSuchSemanticModelException),
+ (409, 1004, SemanticModelAlreadyExistsException),
+ (401, 1011, UnauthorizedException),
+ (403, 1008, ForbiddenException),
+ (409, 1009, CatalogNotInUseException),
+ (409, 1009, MetalakeNotInUseException),
+ (500, 1002, InternalError),
+ (501, 1006, UnsupportedOperationException),
+ (502, 1007, ConnectionFailedException),
+ )
+ for operation in (
+ lambda: self.models.import_ossie_document(self.namespace, "name:
sales"),
+ lambda: self.models.export_ossie_document(self.identifier),
+ ):
+ for status, code, exception in cases:
+ with self.subTest(operation=operation, exception=exception):
+ body = json.dumps(
+ {
+ "code": code,
+ "type": exception.__name__,
+ "message": "server detail",
+ }
+ ).encode()
+ self.opener.open.side_effect = HTTPError(
+ self.path, status, "error", {}, io.BytesIO(body)
+ )
+ with self.assertRaisesRegex(exception, "server detail"):
+ operation()
+
+ def test_base_api_defaults_preserve_existing_implementations(self):
+ for method, args in (
+ (
+ SemanticModelCatalog.import_ossie_document,
+ (self.namespace, "name: sales"),
+ ),
+ (SemanticModelCatalog.export_ossie_document, (self.identifier,)),
+ ):
+ with self.assertRaises(UnsupportedOperationException):
+ method(Mock(spec=SemanticModelCatalog), *args)
diff --git a/docs/how-to-use-python-client.md b/docs/how-to-use-python-client.md
index c6ba115c26..4dd9b1691d 100644
--- a/docs/how-to-use-python-client.md
+++ b/docs/how-to-use-python-client.md
@@ -32,6 +32,39 @@ pip install apache-gravitino
1. [Manage metalake using Gravitino Python
API](./manage-metalake-using-gravitino.md?language=python)
2. [Manage fileset metadata using Gravitino Python
API](./manage-fileset-metadata-using-gravitino.md?language=python)
+### Semantic Model Ossie Import and Export
+
+Import a standalone Apache Ossie YAML or JSON document into an existing schema
of a
+relational catalog. Its dataset sources must refer to existing tables or views
in the
+same metalake. The document supplies the model name; importing an existing
name raises
+`SemanticModelAlreadyExistsException` without replacing it.
+
+```python
+from pathlib import Path
+
+from gravitino import GravitinoClient, NameIdentifier
+from gravitino.namespace import Namespace
+
+client = GravitinoClient(uri="http://localhost:8090";, metalake_name="metalake")
+try:
+ models = client.load_catalog("catalog").as_semantic_model_catalog()
+ model = models.import_ossie_document(
+ Namespace.of("schema"),
Path("sales.ossie.yaml").read_text(encoding="utf-8")
+ )
+ document = models.export_ossie_document(
+ NameIdentifier.of("schema", model.name()), document_format="json"
+ )
+ Path("sales.ossie.json").write_text(document, encoding="utf-8")
+finally:
+ client.close()
+```
+
+Both methods default to `document_format="yaml"`; use `"json"` for JSON input
or output.
+The SDK transfers document text directly and requires no YAML parser. Import
returns a
+`SemanticModel` using the existing lossless JSON decoder, while export returns
the server's
+text unchanged. Model properties are exchanged through the root
`GRAVITINO_PROPERTIES`
+custom extension, and `ossie-version` supplies the document version.
+
### Fileset Example
We offer a playground environment to help you quickly understand how to use
Gravitino Python
