BsoBird commented on code in PR #5934:
URL: https://github.com/apache/hive/pull/5934#discussion_r2193987257
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java:
##########
@@ -71,8 +70,8 @@ public class SplitGrouper {
// TODO This needs to be looked at. Map of Map to Map... Made concurrent for
now since split generation
// can happen in parallel.
- private static final Map<Map<Path, PartitionDesc>, Map<Path, PartitionDesc>>
cache =
- new ConcurrentHashMap<>();
+ private final Map<Map<Path, PartitionDesc>, Map<Path, PartitionDesc>> cache =
Review Comment:
> @BsoBird, that setup uses `HadoopTables` rather than `HadoopCatalog` !
>
> * **HadoopTables**: Implementation of Iceberg tables that uses the Hadoop
FileSystem to store metadata and manifests
> * **HadoopCatalog**: provides a way to use table names like db.table to
work with path-based tables under a common location. It uses a specified
directory under a specified filesystem as the warehouse directory, and
organizes multiple levels directories that mapped to the database, namespace
and the table respectively.
Sorry,Sir.
Because I previously reviewed the code and found that the submission and
management logic of location_based_table is basically the same as that of
hadoopCatalog. Therefore, I previously defaulted to treating
location_based_table as HadoopCatalogTable. I confused the concepts, and this
is my mistake.
Additionally, in the production environment, we currently only use
HadoopTables (location_based_table) in HIVE.
(In Spark, I use hadoopCatalog to create a table. After Spark maintains this
table, in HIVE I use HadoopTables to query the hadoopCatalog table maintained
by Spark.)
Therefore, I only set the properties of IcebergHiveCatalog in HIVE (But I
did not use HMS Catalog in the production environment, it was only used for
some testing) , and did not set the related properties of other types of
iceberg (including HadoopCatalog,Because HadoopTables can be used without any
setup,so no need setup/use hadoopCatalog).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
