dengzhhu653 commented on code in PR #5843:
URL: https://github.com/apache/hive/pull/5843#discussion_r2128179602
##########
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:
##########
@@ -1905,6 +1905,8 @@ public enum ConfVars {
"Only collect the non-native table's summary that has been
updated/changed in configured recent days."),
METADATA_SUMMARY_MAX_NONNATIVE_TABLES("hive.metatool.summary.maxNonNativeTables",
"hive.metatool.summary.maxNonNativeTables", "",
"The maximum non-native tables allowed per table type during
collecting the summary."),
+
METADATA_SUMMARY_NONNATIVE_THREADS("hive.metatool.summary.nonnative.threads",
"hive.metatool.summary.nonnative.threads", 20,
+ "Number of threads to be allocated for MetaToolTaskMetadataSummary for
collecting the non-native table's summary."),
Review Comment:
I was inspired by `hive.metastore.fshandler.threads`, which default value is
15. The thread is used for loading the Iceberg table: reading the metadata
files, where it might take time over thousands of Iceberg tables, leading to
timeout. The actual threads is min(number of Iceberg tables,
hive.metatool.summary.nonnative.threads)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
