Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
jerryshao merged PR #12850: URL: https://github.com/apache/gravitino/pull/12850 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3930418319
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,16 +56,25 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
@Timed(name = "post-lineage." + MetricNames.HTTP_PROCESS_DURATION, absolute
= true)
@ResponseMetered(name = "post-lineage", absolute = true)
- public Response postLineage(OpenLineage.RunEvent event) {
-LOG.info(
-"Open lineage event, run id:{}, job name:{}",
-org.apache.gravitino.lineage.Utils.getRunID(event),
-org.apache.gravitino.lineage.Utils.getJobName(event));
+ @AuthorizationExpression(expression =
AuthorizationExpressionConstants.CAN_ACCESS_METADATA)
Review Comment:
Addressed in b874aa471. Metadata visibility remains intentional for both
inputs and outputs because this endpoint records producer-reported lineage
rather than attesting that the caller performed the underlying read or write.
The OpenAPI description now states that rationale explicitly, and a regression
test pins that a SELECT-only caller may report an output. A stronger
lineage-attestation privilege would be a separate authorization feature.
##
server/src/main/java/org/apache/gravitino/server/web/filter/authorization/LineageAuthorizationExecutor.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.server.web.filter.authorization;
+
+import static
org.apache.gravitino.server.web.filter.ParameterUtil.extractFromParameters;
+
+import com.google.common.base.Preconditions;
+import io.openlineage.server.OpenLineage.Dataset;
+import io.openlineage.server.OpenLineage.DatasetFacet;
+import io.openlineage.server.OpenLineage.DatasetFacets;
+import io.openlineage.server.OpenLineage.RunEvent;
+import java.lang.reflect.Parameter;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Optional;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.Entity;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.MetadataObjects;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.authorization.AuthorizationRequestContext;
+import org.apache.gravitino.lineage.source.rest.LineageEventValidator;
+import
org.apache.gravitino.server.authorization.expression.AuthorizationExpressionEvaluator;
+import org.apache.gravitino.utils.MetadataObjectUtil;
+import org.apache.gravitino.utils.NameIdentifierUtil;
+
+/** Authorization executor for every input and output dataset in an
OpenLineage event. */
+public class LineageAuthorizationExecutor implements AuthorizationExecutor {
+
+ private static final String DATASET_TYPE_FACET = "datasetType";
+
+ private final Parameter[] parameters;
+ private final Object[] args;
+ private final String expression;
+ private List authorizationTargets = List.of();
+ private boolean authorizationTargetsResolved;
+
+ /**
+ * Creates an authorization executor for an OpenLineage event.
+ *
+ * @param parameters parameters of the intercepted REST method
+ * @param args arguments passed to the intercepted REST method
+ * @param expression authorization expression to evaluate for every dataset
+ */
+ public LineageAuthorizationExecutor(Parameter[] parameters, Object[] args,
String expression) {
+this.parameters = parameters;
+this.args = args;
+this.expression = expression;
+ }
+
+ @Override
+ public Optional getAuthorizationMetalake() {
+RunEvent event = extractRunEvent();
+resolveAuthorizationTargets(event);
+return Optional.of(event.getJob().getNamespace());
+ }
+
+ @Override
+ public boolean execute(AuthorizationRequestContext context) {
+if (!authorizationTargetsResolved) {
+ resolveAuthorizationTargets(extractRunEvent());
+}
+
+AuthorizationExpressionEvaluator evaluator = new
AuthorizationExpressionEvaluator(expression);
+context.setOriginalAuthorizationExpression(expression);
+
+for (AuthorizationTarget
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3930357435
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,16 +56,25 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
Review Comment:
Thanks for the detailed verification. I confirmed that the 406 behavior
exists on `main`. It is caused by the existing content-negotiation setup rather
than by the validation or authorization changes in this PR, so I consider it
outside the scope of this PR.
I have also not been able to reproduce the 201 response from #12840 through
the standard server wiring. The authorization gap applies once the request
reaches the resource and interceptor path.
##
docs/open-api/lineage.yaml:
##
@@ -21,21 +21,34 @@ paths:
/lineage:
post:
summary: Post runEvent
- description: Updates a run state for a job.
+ description: |
+Updates a run state for a job. When authorization is enabled,
`job.namespace`
+identifies the Gravitino metalake (organization), and every input and
output
+dataset namespace must match it. Dataset names must be Gravitino
metadata full
+names. The optional `datasetType` facet defaults to `TABLE`; supported
values are
+`TABLE`, `VIEW`, `FILE`, `FILESET`, `MODEL`, `MODEL_VERSION`, and
`TOPIC`.
+Inputs and outputs both require metadata visibility. Unsupported or
external
+dataset identifiers are rejected when authorization is enabled. When
authorization
+is disabled, generic OpenLineage namespaces remain supported.
Review Comment:
The strict namespace contract is intentional. The lineage endpoint has no
metalake path parameter or configured identifier resolver, while authorization
must resolve every target to exactly one Gravitino metalake and fail closed.
Therefore, when authorization is enabled, `job.namespace` is treated as the
event metalake and every dataset must use that namespace with a
Gravitino-normalized name.
Allowing generic or unresolved namespaces would require skipping dataset
authorization and would reintroduce the issue this PR fixes. A pluggable
identifier resolver could support standard producer namespaces in the future,
but that is a separate feature. The authorization-disabled path continues to
accept generic OpenLineage namespaces, and the strict authorization contract is
documented and tested in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
yuqi1129 commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3924722588
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,16 +56,25 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
Review Comment:
**Blocking, please verify before merge.** With this `@Produces`, I could not
reach this method at all. I ran `POST /api/lineage` against a real server on
this branch and on `main`, with three different `Accept` values, and every
single one came back **406**:
```
[PR] Accept: application/vnd.gravitino.v1+json -> 406
[MAIN] Accept: application/vnd.gravitino.v1+json -> 406
[MAIN] Accept: application/json -> 406 {"message":"Not
Acceptable","url":"/api/lineage"}
[MAIN] Accept: */*-> 406
```
`VersioningFilter` is registered on `/api/*` (`GravitinoServer.java:203`).
When the request carries no Gravitino version it **replaces** `Accept` with
`application/vnd.gravitino.v1+json` (`MutableHttpServletRequest.putHeader`
overwrites, it does not append). This method produces plain `application/json`,
so the two never match. Every other REST resource in the repo declares
`@Produces("application/vnd.gravitino.v1+json")`, this one does not.
The 406 comes from Jersey's method matching, before the resource method
runs, so the hk2 interceptor never fires either, which means the whole
authorization path this PR adds is unreachable as the server is currently
wired. It returns 406 rather than 404, so registration itself is fine, it is
purely content negotiation.
This is pre-existing, not introduced here, but two things follow:
1. It is one line in a file this PR already touches, so I would fix
`@Produces` here and add an integration test, otherwise nothing proves this fix
works.
2. I could not reproduce #12840's stated repro (both requests returning `201
Created`). Could you confirm how you reached 201? If the issue's premise does
not hold, the scope may need revisiting.
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,16 +56,25 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
@Timed(name = "post-lineage." + MetricNames.HTTP_PROCESS_DURATION, absolute
= true)
@ResponseMetered(name = "post-lineage", absolute = true)
- public Response postLineage(OpenLineage.RunEvent event) {
-LOG.info(
-"Open lineage event, run id:{}, job name:{}",
-org.apache.gravitino.lineage.Utils.getRunID(event),
-org.apache.gravitino.lineage.Utils.getJobName(event));
+ @AuthorizationExpression(expression =
AuthorizationExpressionConstants.CAN_ACCESS_METADATA)
Review Comment:
`CAN_ACCESS_METADATA` resolves to `LOAD_TABLE` for a table, i.e. SELECT or
MODIFY or owner. So a user holding only SELECT can list a table under
`outputs`, that is, assert "this job wrote this table".
The description calls this out as intentional ("metadata-visibility
permissions"), but it does not explain the asymmetry. Lineage feeds impact
analysis and governance, so a write-claim is arguably not the same as a read.
Worth either requiring MODIFY-level rights for `outputs`, or writing down why
visibility is enough.
##
server/src/main/java/org/apache/gravitino/server/web/filter/authorization/LineageAuthorizationExecutor.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.server.web.filter.authorization;
+
+import static
org.apache.gravitino.server.web.filter.ParameterUtil.extractFromParameters;
+
+import com.google.common.base.Preconditions;
+import io.openlineage.server.OpenLineage.Dataset;
+import io.openlineage.server.OpenLineage.DatasetFacet;
+import io.openlineage.server.OpenLineage.DatasetFacets;
+import io.openlineage.server.OpenLineage.RunEvent;
+import java.lang.reflect.Parameter;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on PR #12850: URL: https://github.com/apache/gravitino/pull/12850#issuecomment-5523066285 Addressed the overall review in e7f6ccfa2. The contract is now explicit: job.namespace is the single event metalake (organization) when authorization is enabled; every dataset namespace must match it; missing datasetType defaults to TABLE; supported types and external-target behavior are documented. Inputs and outputs intentionally use the same CAN_ACCESS_METADATA check because this issue is scoped to metadata visibility, not read/write lineage attestation. OpenAPI now documents 201, 400, and 403. Tests cover every input/output position, exact identifiers for all supported types, malformed/denied/internal status mapping, authorization-disabled generic namespaces, authorization-enabled interception, and no dispatch for rejected events. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3922716665
##
server/src/main/java/org/apache/gravitino/server/web/filter/GravitinoInterceptionService.java:
##
@@ -238,6 +194,24 @@ public Object invoke(MethodInvocation methodInvocation)
throws Throwable {
secondaryExpression,
secondaryExpressionCondition,
expressionAnnotation.allowCheckExistence());
+
authorizationMetalakes.addAll(executor.getAuthorizationMetalakes());
+ }
+
+ for (String metalake : authorizationMetalakes) {
+Optional validationFailure =
+validateCurrentUserAndActiveRoles(
+NameIdentifier.of(metalake),
+authorizationRequestContext,
Review Comment:
Fixed in e7f6ccfa2. A lineage RunEvent now has exactly one organization:
job.namespace is the authorization metalake, and every input/output namespace
must match it. Cross-metalake events return 400 before user or active-role
validation, so one AuthorizationRequestContext is never shared across
metalakes. Added regression coverage for mismatches at every input and output
position.
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,22 +56,28 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
@Timed(name = "post-lineage." + MetricNames.HTTP_PROCESS_DURATION, absolute
= true)
@ResponseMetered(name = "post-lineage", absolute = true)
- public Response postLineage(OpenLineage.RunEvent event) {
-LOG.info(
-"Open lineage event, run id:{}, job name:{}",
-org.apache.gravitino.lineage.Utils.getRunID(event),
-org.apache.gravitino.lineage.Utils.getJobName(event));
-
+ @AuthorizationExpression(expression =
AuthorizationExpressionConstants.CAN_ACCESS_METADATA)
+ public Response postLineage(
+ @AuthorizationRequest(type = AuthorizationRequest.RequestType.LINEAGE)
+ OpenLineage.RunEvent event) {
try {
return Utils.doAs(
httpRequest,
() -> {
+LineageEventValidator.validate(event);
+LOG.info(
+"Open lineage event, run id:{}, job name:{}",
+org.apache.gravitino.lineage.Utils.getRunID(event),
+org.apache.gravitino.lineage.Utils.getJobName(event));
if (lineageDispatcher.dispatchLineageEvent(event)) {
return Utils.created();
} else {
return Utils.tooManyRequests();
}
});
+} catch (IllegalArgumentException e) {
Review Comment:
Fixed in e7f6ccfa2. RunEvent validation now has its own try/catch before
dispatcher execution. An IllegalArgumentException from the dispatcher is
treated as an internal error (500), with a regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3922718968
##
server/src/main/java/org/apache/gravitino/server/web/filter/authorization/LineageAuthorizationExecutor.java:
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.server.web.filter.authorization;
+
+import static
org.apache.gravitino.server.web.filter.ParameterUtil.extractFromParameters;
+
+import com.google.common.base.Preconditions;
+import io.openlineage.server.OpenLineage.Dataset;
+import io.openlineage.server.OpenLineage.DatasetFacet;
+import io.openlineage.server.OpenLineage.DatasetFacets;
+import io.openlineage.server.OpenLineage.RunEvent;
+import java.lang.reflect.Parameter;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.gravitino.Entity;
+import org.apache.gravitino.MetadataObject;
+import org.apache.gravitino.MetadataObjects;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.authorization.AuthorizationRequestContext;
+import
org.apache.gravitino.server.authorization.expression.AuthorizationExpressionEvaluator;
+import org.apache.gravitino.utils.MetadataObjectUtil;
+import org.apache.gravitino.utils.NameIdentifierUtil;
+
+/** Authorization executor for every input and output dataset in an
OpenLineage event. */
+public class LineageAuthorizationExecutor implements AuthorizationExecutor {
+
+ private static final String DATASET_TYPE_FACET = "datasetType";
+
+ private final Parameter[] parameters;
+ private final Object[] args;
+ private final String expression;
+
+ /**
+ * Creates an authorization executor for an OpenLineage event.
+ *
+ * @param parameters parameters of the intercepted REST method
+ * @param args arguments passed to the intercepted REST method
+ * @param expression authorization expression to evaluate for every dataset
+ */
+ public LineageAuthorizationExecutor(Parameter[] parameters, Object[] args,
String expression) {
+this.parameters = parameters;
+this.args = args;
+this.expression = expression;
+ }
+
+ @Override
+ public Set getAuthorizationMetalakes() {
+Object request = extractFromParameters(parameters, args);
+if (!(request instanceof RunEvent)) {
+ return Set.of();
+}
+
+RunEvent event = (RunEvent) request;
+if (!hasValidDatasetIdentifiers(event.getInputs())
+|| !hasValidDatasetIdentifiers(event.getOutputs())) {
+ return Set.of();
+}
+
+Set metalakes = new LinkedHashSet<>();
+addDatasetMetalakes(event.getInputs(), metalakes);
+addDatasetMetalakes(event.getOutputs(), metalakes);
+return metalakes;
+ }
+
+ @Override
+ public boolean execute(AuthorizationRequestContext context) {
+Object request = extractFromParameters(parameters, args);
+if (!(request instanceof RunEvent)) {
+ // Request validation is owned by LineageOperations and must still run
when authorization is
+ // disabled. Let it return HTTP 400 for a null or malformed event.
+ return true;
+}
+
+RunEvent event = (RunEvent) request;
+if (!hasValidDatasetIdentifiers(event.getInputs())
+|| !hasValidDatasetIdentifiers(event.getOutputs())) {
+ return true;
+}
+
+AuthorizationExpressionEvaluator evaluator = new
AuthorizationExpressionEvaluator(expression);
+context.setOriginalAuthorizationExpression(expression);
+
+return authorizeDatasets(event.getInputs(), context, evaluator)
+&& authorizeDatasets(event.getOutputs(), context, evaluator);
+ }
+
+ static MetadataObject.Type getMetadataType(Dataset dataset) {
+DatasetFacets facets = dataset.getFacets();
+if (facets == null) {
+ return MetadataObject.Type.TABLE;
+}
+
+DatasetFacet datasetTypeFacet =
facets.getAdditionalProperties().get(DATASET_TYPE_FACET);
+if (datasetTypeFacet == null) {
+ return MetadataObject.Type.TABLE;
+}
+
+Object datasetType =
datasetTypeFacet.getAdditionalProperties().get(DATASET_TYPE_FACET);
+
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
yuqi1129 commented on code in PR #12850:
URL: https://github.com/apache/gravitino/pull/12850#discussion_r3922166383
##
server/src/main/java/org/apache/gravitino/server/web/filter/GravitinoInterceptionService.java:
##
@@ -238,6 +194,24 @@ public Object invoke(MethodInvocation methodInvocation)
throws Throwable {
secondaryExpression,
secondaryExpressionCondition,
expressionAnnotation.allowCheckExistence());
+
authorizationMetalakes.addAll(executor.getAuthorizationMetalakes());
+ }
+
+ for (String metalake : authorizationMetalakes) {
+Optional validationFailure =
+validateCurrentUserAndActiveRoles(
+NameIdentifier.of(metalake),
+authorizationRequestContext,
Review Comment:
This request context cannot be shared across different metalakes.
AuthorizationRequestContext keeps a single hasLoadRole flag and a single
prefetchedRoleVersions map, so after mlA is validated or authorized, role
loading for mlB can be skipped or the prefetched roles from mlA can be reused.
That makes cross-metalake lineage role authorization order-dependent, and
named-role validation can inspect the wrong metalake state. Please use one
AuthorizationRequestContext per metalake and pass the matching context when
authorizing each dataset. Add a role-based mlA + mlB regression test in both
dataset orders; the current isOwner(any()) test does not exercise this path.
##
lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java:
##
@@ -53,22 +56,28 @@ public LineageOperations(LineageDispatcher
lineageDispatcher) {
@Produces(MediaType.APPLICATION_JSON)
@Timed(name = "post-lineage." + MetricNames.HTTP_PROCESS_DURATION, absolute
= true)
@ResponseMetered(name = "post-lineage", absolute = true)
- public Response postLineage(OpenLineage.RunEvent event) {
-LOG.info(
-"Open lineage event, run id:{}, job name:{}",
-org.apache.gravitino.lineage.Utils.getRunID(event),
-org.apache.gravitino.lineage.Utils.getJobName(event));
-
+ @AuthorizationExpression(expression =
AuthorizationExpressionConstants.CAN_ACCESS_METADATA)
+ public Response postLineage(
+ @AuthorizationRequest(type = AuthorizationRequest.RequestType.LINEAGE)
+ OpenLineage.RunEvent event) {
try {
return Utils.doAs(
httpRequest,
() -> {
+LineageEventValidator.validate(event);
+LOG.info(
+"Open lineage event, run id:{}, job name:{}",
+org.apache.gravitino.lineage.Utils.getRunID(event),
+org.apache.gravitino.lineage.Utils.getJobName(event));
if (lineageDispatcher.dispatchLineageEvent(event)) {
return Utils.created();
} else {
return Utils.tooManyRequests();
}
});
+} catch (IllegalArgumentException e) {
Review Comment:
The IllegalArgumentException catch also covers
lineageDispatcher.dispatchLineageEvent. If a processor or dispatcher has an
internal bug or configuration error and throws IllegalArgumentException, this
endpoint now reports it as an invalid client event (400) instead of 500. Please
isolate the validation try/catch, or use a dedicated validation exception, and
add a regression test where the dispatcher throws IllegalArgumentException.
##
server/src/main/java/org/apache/gravitino/server/web/filter/authorization/LineageAuthorizationExecutor.java:
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.server.web.filter.authorization;
+
+import static
org.apache.gravitino.server.web.filter.ParameterUtil.extractFromParameters;
+
+import com.google.common.base.Preconditions;
+import io.openlineage.server.OpenLineage.Dataset;
+import io.openlineage.server.OpenLineage.DatasetFacet;
+import io.openlineage.server.OpenLineage.DatasetFacets;
+import io.openlineage.server.OpenLineage.RunEvent;
+import java.lang.reflect.Parameter;
+import java.util.LinkedHashSet;
+import java.util.List;
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
yuqi1129 commented on PR #12850: URL: https://github.com/apache/gravitino/pull/12850#issuecomment-5522243005 Overall, the validation and fail-closed per-dataset authorization direction is sound, but I do not think this is ready to merge until the multi-metalake context issue is fixed. Two contracts also need to be explicit: - Inputs and outputs currently use the same CAN_ACCESS_METADATA expression. That is reasonable if the requirement is only metadata visibility; if lineage integrity requires the reporter to represent reads and writes, use direction-specific privileges or a dedicated lineage-ingest privilege. - OpenLineage namespace is generic, while this code treats it as a Gravitino metalake and defaults a missing datasetType facet to TABLE. Please document that only Gravitino-normalized identifiers are authorizable and define the behavior for unsupported or external dataset types. Please also update docs/open-api/lineage.yaml with the new 201, 400, and 403 behavior. Test coverage should include role-based authorization across two metalakes, including reversed input and output order; failure at every input and output position; exact resolved identifiers for every supported type; malformed target versus denial versus internal-error status mapping; authorization-enabled and disabled HTTP paths; and verification that no rejected event is dispatched. The current cross-metalake success test stubs isOwner(any()) and therefore misses the role-context bug. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
roryqi commented on PR #12850: URL: https://github.com/apache/gravitino/pull/12850#issuecomment-5510036195 > please fix the PR description. Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
jerryshao commented on PR #12850: URL: https://github.com/apache/gravitino/pull/12850#issuecomment-5509511064 please fix the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#12840] fix(lineage): Validate and authorize lineage events [gravitino]
github-actions[bot] commented on PR #12850: URL: https://github.com/apache/gravitino/pull/12850#issuecomment-5508493097 ### Code Coverage Report |Overall Project|69.02% **`+0.07%`**|:green_circle:| |:-|:-|:-:| |Files changed|92.67%|:green_circle:| |Module|Coverage|| |:-|:-|:-:| |aliyun|19.74%|:red_circle:| |api|51.72%|:green_circle:| |authorization-common|85.96%|:green_circle:| |authorization-ranger|4.38%|:red_circle:| |aws|53.54%|:green_circle:| |azure|32.1%|:red_circle:| |catalog-common|19.1%|:red_circle:| |catalog-fileset|80.3%|:green_circle:| |catalog-glue|69.24%|:green_circle:| |catalog-hive|82.96%|:green_circle:| |catalog-jdbc-common|45.69%|:green_circle:| |catalog-jdbc-doris|82.69%|:green_circle:| |catalog-jdbc-mysql|79.33%|:green_circle:| |catalog-jdbc-postgresql|83.39%|:green_circle:| |catalog-jdbc-starrocks|79.16%|:green_circle:| |catalog-kafka|76.99%|:green_circle:| |catalog-lakehouse-generic|60.55%|:green_circle:| |catalog-lakehouse-hudi|79.1%|:green_circle:| |catalog-lakehouse-iceberg|85.86%|:green_circle:| |catalog-lakehouse-paimon|84.26%|:green_circle:| |catalog-model|77.99%|:green_circle:| |cli|44.48%|:green_circle:| |client-java|77.49%|:green_circle:| |common|56.15%|:green_circle:| |core|83.8%|:green_circle:| |filesystem-hadoop3|76.45%|:green_circle:| |flink|0.0%|:red_circle:| |flink-common|52.1%|:green_circle:| |flink-runtime|0.0%|:red_circle:| |gcp|32.2%|:red_circle:| |hadoop-auth|68.0%|:green_circle:| |hadoop-common|17.84%|:red_circle:| |hive-metastore-common|53.4%|:green_circle:| |iceberg-aliyun-bundle|0.0%|:red_circle:| |iceberg-common|64.75%|:green_circle:| |iceberg-rest-server|75.89%|:green_circle:| |idp-basic|86.42%|:green_circle:| |integration-test-common|0.0%|:red_circle:| |jobs|62.92%|:green_circle:| |lance-common|32.63%|:red_circle:| |lance-rest-server|65.46%|:green_circle:| |lineage|58.08% **`+6.7%`**|:green_circle:| |optimizer|83.24%|:green_circle:| |optimizer-api|21.95%|:red_circle:| |server|89.35% **`+0.22%`**|:green_circle:| |server-common|80.67%|:green_circle:| |spark|28.57%|:red_circle:| |spark-common|48.92%|:green_circle:| |tencent|81.78%|:green_circle:| |trino-connector|51.26%|:green_circle:| Files |Module|File|Coverage|| |:-|:-|:-|:-:| |lineage|[LineageEventValidator.java](https://github.com/qqqttt123/gravitino/blob/45fb1904db5b9ae85b58140a33e0a9591949e419/lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageEventValidator.java)|100.0%|:green_circle:| ||[LineageOperations.java](https://github.com/qqqttt123/gravitino/blob/45fb1904db5b9ae85b58140a33e0a9591949e419/lineage/src/main/java/org/apache/gravitino/lineage/source/rest/LineageOperations.java)|83.33%|:green_circle:| |server|[GravitinoInterceptionService.java](https://github.com/qqqttt123/gravitino/blob/45fb1904db5b9ae85b58140a33e0a9591949e419/server/src/main/java/org/apache/gravitino/server/web/filter/GravitinoInterceptionService.java)|92.64%|:green_circle:| -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
