youngyjd opened a new pull request, #9181:
URL: https://github.com/apache/gravitino/pull/9181
### What changes were proposed in this pull request?
This PR implements create, load, and list table operations for Lance format
tables in the generic lakehouse catalog:
1. Added LanceCatalogOperations and LanceDataTypeConverter for
Lance-specific operations
2. Introduced GenericTable API and GenericTableEntity for generic format
table support
3. Implemented table versioning system with TableVersionMapper and related
SQL providers
4. Extended GenericLakehouseCatalogOperations to delegate to format-specific
implementations
5. Updated database schema to store table storage details (format, location,
version)
### Why are the changes needed?
Lance is a columnar format optimized for ML workloads. This enables users to
manage Lance tables through Gravitino's unified catalog API alongside other
formats like Iceberg and Hudi in a multi-format lakehouse architecture.
Fix: #8838 #8837
### Does this PR introduce _any_ user-facing change?
Yes:
1. New Lance table support: Users can create/load Lance tables via REST API
with "format": "lance" property
2. New table properties: format (table storage format) and location (storage
path)
Example:
`POST /api/metalakes/test/catalogs/lance_catalog/schemas/schema/tables{
"name": "my_table", "columns": [...], "properties": {"format": "lance",
"location": "/path/to/table"}}`
### How was this patch tested?
Tested locally with manual REST API calls:
1. Created Lance tables with primary keys and default values ✓
2. Loaded existing tables and verified metadata ✓
3. Listed tables in schema ✓
4. Verified on-disk storage structure (_transactions/, _versions/) ✓
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
