laserninja commented on issue #9866:
URL: https://github.com/apache/gravitino/issues/9866#issuecomment-4129473625
By the time registerGravitinoCatalogs runs, the plugin has already loaded
all catalogs and knows their providers, right?
can we Auto-detect and auto-set in `registerGravitinoCatalogs`?
will it help if in `GravitinoDriverPlugin.registerGravitinoCatalogs()` we do:
```private void registerGravitinoCatalogs(
SparkConf sparkConf, Map<String, Catalog> gravitinoCatalogs) {
// Auto-set spark.sql.catalogImplementation=hive if any catalog uses the
hive provider,
// unless the user has already explicitly configured it.
boolean hasHiveCatalog = gravitinoCatalogs.values().stream()
.anyMatch(c -> "hive".equalsIgnoreCase(c.provider()));
if (hasHiveCatalog &&
!sparkConf.contains("spark.sql.catalogImplementation")) {
sparkConf.set("spark.sql.catalogImplementation", "hive");
LOG.info("Detected hive catalog in Gravitino metalake; "
+ "automatically set spark.sql.catalogImplementation=hive.");
}
gravitinoCatalogs.entrySet().forEach(entry -> {
// ... existing registration logic unchanged ...
});
}```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
