dengc367 opened a new pull request, #14347:
URL: https://github.com/apache/gravitino/pull/14347
### What changes were proposed in this pull request?
First slice of the ontology design in #14310: the public API for a
metalake-level `Ontology`, and the converter between it and a standalone Apache
Ossie ontology document.
API (`org.apache.gravitino.ontology`):
- `Ontology`, a snapshot with name, comment, properties, `DRAFT`/`PUBLISHED`
status and an immutable `OntologyDefinition`.
- `OntologyDefinition` holds the concept graph: `OntologyConcept` components
with a closed `ConceptType`, `OntologyRelationship` with required `verbalizes`
phrasings and an optional `Multiplicity`, and `OntologyRole`.
- `OntologyChange` and `OntologyOperations` for create, list, get, alter,
delete and Ossie import/export.
- `NoSuchOntologyException`, `OntologyAlreadyExistsException`,
`IllegalOntologyException`, `NonEmptyOntologyException`.
- `MetadataObject.Type.ONTOLOGY` as a metalake-level, one-level name, so
owners and the ontology-specific privileges can target it.
Converter (`OssieOntologyConverter` in `core`): validates the closed root,
component, relationship and role property sets of `ontology/ontology.json`,
maps the two concept types and two multiplicities, resolves QName IRIs against
`prefixes`, and rejects embedded `ontology_mappings` with a message pointing at
the standalone mapping document. Export re-imports the generated document
before returning it.
No storage, manager, REST or client changes yet, so behavior of existing
endpoints is unchanged.
### Why are the changes needed?
Semantic Models hold metrics and joins but do not say what a concept means;
Domains group assets but define no concepts. `Ontology` is the machine-readable
half of that gap: an Ossie `ontology.json` concept graph that a glossary term
can later reference and a query planner can resolve against (#14308).
Two rules are enforced beyond the schema's keywords, because the document
has no place for external references:
- a relationship must carry at least one non-blank `verbalizes` phrasing, so
every edge is expressible in language for an agent;
- every concept named by `extends` or by a relationship role must be
declared in the same document, so a dangling name is rejected at the boundary
instead of being resolved later.
The lifecycle is `DRAFT`/`PUBLISHED` and the privileges are
`MANAGE_ONTOLOGY`/`USE_ONTOLOGY`, following Domain rather than the glossary's
`DEFINITION` family, as settled in the updated design.
Fixes #14309
### Does this PR introduce _any_ user-facing change?
Yes, additively:
1. New public API types in `api/` (`Ontology`, `OntologyDefinition`,
`OntologyConcept`, `OntologyRelationship`, `OntologyRole`, `ConceptType`,
`Multiplicity`, `OntologyChange`, `OntologyOperations`) and four new exceptions.
2. `MetadataObject.Type.ONTOLOGY` is added, which makes ontology a valid
metadata object type for ownership and privileges.
No REST endpoint, configuration key or storage table is added yet.
### How was this patch tested?
44 new tests:
- `:api:test --tests "org.apache.gravitino.ontology.*"` — 25 tests over the
value types, defensive copies, equality, and the definition's
duplicate/unresolved-reference rules.
- `:core:test --tests "org.apache.gravitino.ontology.*"` — 19 tests over
JSON and YAML import, export, round trip, the Ossie version property, and 11
rejection paths (unknown root property, missing name, embedded mappings,
missing or blank `verbalizes`, closed `type` and `multiplicity`, unresolved
`extends` and role, undeclared QName prefix, bare IRI, invalid export input).
```
JAVA_HOME=<jdk17> ./gradlew :api:spotlessApply :core:spotlessApply \
:api:test --tests "org.apache.gravitino.ontology.*" \
:core:test --tests "org.apache.gravitino.ontology.*" -PskipITs
BUILD SUCCESSFUL
```
Note for reviewers: `MetadataObject.java` and `MetadataObjects.java` will
conflict trivially with #13881, which appends `DOMAIN` at the same place.
Resolution is a pure append of both enum constants; whoever merges second can
take both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
