skywalker0618 commented on code in PR #20219:
URL: https://github.com/apache/hudi/pull/20219#discussion_r4233238625
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HoodieHiveSyncClient.java:
##########
@@ -235,7 +235,14 @@ private IMetaStoreClient
createMetaStoreClient(HiveSyncConfig config) {
new Class<?>[] {HiveSyncConfig.class},
config);
}
- return IMetaStoreClientUtil.getMSC(config.getHiveConf());
+ IMetaStoreClient metaStoreClient =
IMetaStoreClientUtil.getMSC(config.getHiveConf());
+ // Hive caches the instance that owns this client for the thread, and
replaces it, closing
+ // this client, whenever a Hive session on the thread brings a different
metastore conf. A
+ // HiveQL-mode Driver's session does, by setting the metastore filter
hook for authorization.
+ // Taken out of the cache, the client is left to this class, which
closes it in close().
+ // Hive.closeCurrent() would close the client too, hence the set.
+ Hive.set(null);
Review Comment:
Changed the implementation. The client now stays in Hive's cache until just
before the first HiveQL statement, so a sync that runs none lets the session
teardown reuse it instead of opening a new connection.
This exposed another bug: a second extra connection when the tool is closed
twice (syncHoodieTable() plus try-with-resources), which I fixed too;
testHiveQLSyncThatRunsNoStatementOpensOneMetastoreConnection checks only one
connection is opened.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
