vikramahuja1001 commented on code in PR #6012:
URL: https://github.com/apache/hive/pull/6012#discussion_r2262011408
##########
standalone-metastore/metastore-client/src/main/java/org/apache/hadoop/hive/metastore/client/ThriftHiveMetaStoreClient.java:
##########
@@ -2162,6 +2162,16 @@ public List<String> listTableNamesByFilter(String
catName, String dbName, String
isClientFilterEnabled, filterHook, catName, dbName, tableNames);
}
+ @Override
+ public List<Table> listTablesByFilter(String catName, String dbName, String
filter,
+ int maxTables) throws TException {
+ List<Table> tables =
+ client.get_tables_by_filter(prependCatalogToDbName(catName, dbName,
conf), filter,
Review Comment:
By reducing it to just one HMS call we risk thrift memory issues in case the
number of tables are in 1000s. Instead of 1 call we can break it down into
batches so that overall calls to HMS are reduced from 1000s to say 40 or 50.
That is what batching is meant to achieve.
Names can be achieved through already implemented listTableNamesByFilter API
and can be sent to TableIterable object
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
