This is an automated email from the ASF dual-hosted git repository.
mchades pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new b30e457318 [#11985] fix(openapi): correct the OpenLineage translation
in the lineage schemas (#11989)
b30e457318 is described below
commit b30e45731883cfe37311ec9b6c418ffb2245e3d7
Author: Nevin Zheng <[email protected]>
AuthorDate: Sat Jul 18 22:40:36 2026 -0700
[#11985] fix(openapi): correct the OpenLineage translation in the lineage
schemas (#11989)
### What changes were proposed in this pull request?
A minimal correction to the lineage event schemas in
`docs/open-api/lineage.yaml` — two line edits and two deletions, no
structural change:
- remove the two `not: { required: [...] }` blocks on
`DatasetEvent`/`JobEvent`;
- fix the `eventType` example — the pipe-joined
`START|RUNNING|COMPLETE|ABORT|FAIL|OTHER` is a single string that is not
a member of the enum; replace it with `START`;
- change the `/lineage` request-body union from `oneOf` to `anyOf`.
`BaseEvent`, the `allOf` composition, and the open schemas all stay
exactly as they are.
### Why are the changes needed?
These schemas are translated from the [OpenLineage
spec](https://github.com/OpenLineage/OpenLineage/blob/main/spec/OpenLineage.json),
whose source of truth is **JSON Schema draft 2020-12**. Two constructs
did not survive the translation into this **OpenAPI 3.0.3** document,
and this PR is the faithful re-translation.
**1. The `not` blocks are invalid here, and ineffective everywhere.**
They express OpenLineage's intent that static events carry no run state.
But they reference properties (`job`, `run`) the enclosing schemas never
declare — which is what Redocly's
`no-required-schema-properties-undefined` flags — and `not: {required:
[job, run]}` only rejects a payload carrying *both* fields, so a
`DatasetEvent` carrying just `run` validated anyway. Code generators
ignore `not` entirely.
**2. `oneOf` cannot survive their removal.** The event schemas are
deliberately **open** (OpenLineage's extensibility model), so a valid
`RunEvent` — which carries `job` — also matches `JobEvent`. Under a
strict `oneOf` that is two matches, and the most common event type is
rejected. The `not` blocks were the only thing masking this. Verified
with an ajv payload matrix against the bundled spec:
| payload | old `oneOf` | `not` removed, `oneOf` kept | this PR
(`anyOf`) |
|---|---|---|---|
| valid `RunEvent` | ACCEPT | **REJECT** (also matches `JobEvent`) |
ACCEPT |
| valid `JobEvent` | ACCEPT | ACCEPT | ACCEPT |
| valid `DatasetEvent` | ACCEPT | ACCEPT | ACCEPT |
| garbage (no required fields) | REJECT | REJECT | REJECT |
**Why `anyOf` is the right encoding.** With open, overlapping variants,
"matches exactly one" is unsatisfiable by construction; "matches at
least one well-formed event" is the contract the endpoint actually
offers. The per-event `required` sets keep malformed payloads out, and
exact event discrimination stays where it already lives — the server's
typed deserialization. This mirrors how OpenLineage itself handles the
gap (its OpenAPI file offloads validation to the JSON Schema layer) and
how Marquez, the reference implementation, accepts a single loose
`LineageEvent`.
Clears 4 Redocly `recommended-strict` findings (3 ×
`no-required-schema-properties-undefined`, 1 ×
`no-invalid-schema-examples`).
Part of #11985
### Does this PR introduce _any_ user-facing change?
No. Spec/docs-only. Every payload the old spec accepted is still
accepted; the schemas remain open, so OpenLineage producer extensibility
is preserved. No server behavior change.
### How was this patch tested?
- `redocly lint --extends=recommended-strict` — the 4 lineage findings
are cleared (10 → 6 against `main`), no new findings.
- `redocly bundle` — bundles to valid `openapi.json`.
- ajv payload matrix (table above) against three bundles — the original,
a `not`-removed/`oneOf`-kept variant, and this PR — confirming all valid
events accept and payloads missing every event's required fields reject.
---
docs/open-api/lineage.yaml | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)
diff --git a/docs/open-api/lineage.yaml b/docs/open-api/lineage.yaml
index 30074bf05b..cc6c5f27cf 100644
--- a/docs/open-api/lineage.yaml
+++ b/docs/open-api/lineage.yaml
@@ -29,7 +29,7 @@ paths:
content:
application/json:
schema:
- oneOf:
+ anyOf:
- $ref: '#/components/schemas/RunEvent'
- $ref: '#/components/schemas/DatasetEvent'
- $ref: '#/components/schemas/JobEvent'
@@ -218,7 +218,7 @@ components:
- ABORT
- FAIL
- OTHER
- example: START|RUNNING|COMPLETE|ABORT|FAIL|OTHER
+ example: START
run:
$ref: '#/components/schemas/Run'
job:
@@ -250,10 +250,6 @@ components:
$ref: '#/components/schemas/StaticDataset'
required:
- dataset
- not:
- required:
- - job
- - run
JobEvent:
allOf:
- $ref: '#/components/schemas/BaseEvent'
@@ -273,6 +269,3 @@ components:
$ref: '#/components/schemas/OutputDataset'
required:
- job
- not:
- required:
- - run
