youngyjd commented on code in PR #9659:
URL: https://github.com/apache/gravitino/pull/9659#discussion_r2684309785
##########
catalogs/catalog-lakehouse-generic/src/main/java/org/apache/gravitino/catalog/lakehouse/lance/LanceTableOperations.java:
##########
@@ -259,7 +262,29 @@ Table createTableInternal(
String location)
throws NoSuchSchemaException, TableAlreadyExistsException {
+ Map<String, String> storageProps =
LancePropertiesUtils.getLanceStorageOptions(properties);
if (register) {
+ // Try to check if the Lance dataset exists at the location and then
extract the schema.
+ // Note: We will ignore the column parameter in this case.
+ LOG.warn(
+ "Registering existing Lance table at location {}. The provided
schema will be ignored. columns: {}",
+ location,
+ Arrays.toString(columns));
+
+ try (Dataset readDataset =
+ Dataset.open(
+ location, new
ReadOptions.Builder().setStorageOptions(storageProps).build())) {
Review Comment:
This requires server having the capability of reading lance dataset. Is
there any option to make it configurable? In our initial phase roll out, the
server does not have the permission as all it does is to provide the lance
dataset location.
My second question - lance dataset schema can change overtime, how will
column changes be captured?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
