jiangxt2 opened a new issue, #13186: URL: https://github.com/apache/gravitino/issues/13186
### Version main branch ### Describe what's wrong The Lance REST service can address a table stored in a format-agnostic Generic Catalog without validating that the entity's stored `format` property is `lance`. As a result, `describeTable` can return non-Lance metadata as a Lance response, `tableExists` can report a non-Lance table as present, and Lance REST drop, deregister, and alter requests may be handled by another format's delegator instead of being rejected at the Lance boundary. The highest-risk case is create/register with `OVERWRITE`. The Lance request selects the Lance table delegator based on the request properties, while the existing entity is loaded from the catalog. Before checking the existing entity's format, the create-overwrite path can purge the existing non-Lance entity and pass its stored location to Lance dataset deletion; the register-overwrite path can drop the existing metadata through the Lance delegator. Together, these paths can remove non-Lance metadata, and the create-overwrite path can invoke a destructive Lance dataset operation against the non-Lance table location. The same overwrite path is reachable through direct Gravitino `TableCatalog` calls without using the Lance REST server. ### Error message and/or stacktrace The overwrite path does not need to raise an exception: it may incorrectly succeed after replacing an existing non-Lance entity. Other operations expose format-dependent errors; for example, dropping an external Delta entity through the Lance REST namespace can surface as HTTP `406 NOT_ACCEPTABLE` from the Delta operation instead of a clear Lance format-boundary error. The more serious failure is the possible metadata and data-path mutation before the format mismatch is detected. ### How to reproduce 1. Start Gravitino with the Lance REST service and create a relational `lakehouse-generic` catalog. 2. Create an external Delta table in a schema through the Gravitino TableCatalog API, for example with `format=delta`, `external=true`, and a location containing a sentinel file. 3. Address the same three-level identifier through the Lance REST namespace API and call `describeTable` or `tableExists`. 4. Call Lance REST `createTable` with `mode=overwrite`, or `registerTable` with `mode=overwrite`, using the same identifier and a Lance location. The equivalent direct API call sets `format=lance` and `lance.creation-mode=OVERWRITE` on `TableCatalog.createTable`. 5. Check the Delta metadata and sentinel file after each operation. ### Additional context Expected behavior is to enforce the Lance namespace boundary after authorization and before any metadata or storage mutation. For a known non-Lance entity, direct Lance operations should return HTTP `400 INVALID_INPUT`, `tableExists` should present it as absent, and create `EXIST_OK` or `OVERWRITE` (including register overwrite) should be rejected without changing the existing entity. Plain `CREATE` should retain the normal existing-name conflict behavior. Generic Catalog `ListTables` should remain format-agnostic. The issue is reproducible with mixed-format metadata without requiring a Delta data engine. The same boundary has to hold for both entry points: the Lance REST namespace and direct calls through the Gravitino `TableCatalog` API. I hit this while using the Lance REST namespace over a Generic Catalog that also holds non-Lance tables, and I plan to submit a fix for the boundary. Feedback on the expected status codes and on anything I may have missed is welcome. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
