Hello Peter Rozsa, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24712
to look at the new patch set (#5).
Change subject: IMPALA-13314: Cache HadoopCatalog instances and use holder
pattern for singletons
......................................................................
IMPALA-13314: Cache HadoopCatalog instances and use holder pattern for
singletons
Previously, a new HadoopCatalog (and its FileIO/filesystem resources)
was created on every table operation via getIcebergCatalog(). This
caused excessive resource consumption when many Iceberg tables shared
the same catalog location.
Changes:
- Introduce a Guava Cache with soft values in IcebergHadoopCatalog
keyed by catalog location string. Tables sharing the same
iceberg.catalog_location now reuse a single HadoopCatalog instance.
Entries are evicted under memory pressure via soft references.
- Replace synchronized lazy initialization with the
initialization-on-demand holder idiom for IcebergHiveCatalog,
IcebergHadoopTables, and IcebergCatalogUtil. The JVM guarantees
thread-safe lazy initialization without synchronization overhead on
every access.
- Convert getIcebergCatalog() to a switch expression, removing the
throws ImpalaRuntimeException declaration. Simplify loadTable()
by dropping the now-unnecessary try-catch.
- Remove unused getLastComputeStatsTimeMs() and
collectFieldIdsWithNdvStats() from IcebergTable.
- Fix logger in IcebergHadoopCatalog (was incorrectly using
IcebergHadoopTables.class).
- Make hadoopTables field final in IcebergHadoopTables.
Testing: Ran existing Iceberg catalog tests.
Assisted-by: Claude Opus 4.6 (Claude Code)
Change-Id: Ibb3a6c8e4f1d2a9e7c5b0f8d3e6a4c2b1d0e9f7a
---
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCatalogUtil.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHadoopCatalog.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHadoopTables.java
M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergHiveCatalog.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
6 files changed, 56 insertions(+), 70 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/12/24712/5
--
To view, visit http://gerrit.cloudera.org:8080/24712
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb3a6c8e4f1d2a9e7c5b0f8d3e6a4c2b1d0e9f7a
Gerrit-Change-Number: 24712
Gerrit-PatchSet: 5
Gerrit-Owner: Nandor Kollar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Nandor Kollar <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
