terrytlu commented on code in PR #17317:
URL: https://github.com/apache/iceberg/pull/17317#discussion_r3642762171
##########
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##########
@@ -354,6 +370,48 @@ private List<TableIdentifier> listIcebergTables(
.collect(Collectors.toList());
}
+ /**
+ * Lists the {@link TableIdentifier} of all Iceberg tables under the given
database.
+ *
+ * <p>Implementation uses the HMS {@code get_table_names_by_filter} API,
filtering on {@code
+ * parameters.table_type} on the server side so that only matching table
names (rather than full
+ * Table objects) are returned.
+ *
+ * @param namespace the namespace corresponding to the database
+ * @param tableTypeValue the value of {@code parameters.table_type} to
filter on, e.g. {@link
+ * BaseMetastoreTableOperations#ICEBERG_TABLE_TYPE_VALUE} for tables.
HMS normalizes this
+ * value to upper case when persisting it, so the filter value is
upper-cased to match.
+ */
+ private List<TableIdentifier> listIcebergTablesByFilter(
+ Namespace namespace, String tableTypeValue) throws TException,
InterruptedException {
+ String database = namespace.level(0);
+ // HMS normalizes the `table_type` parameter value to upper case when
persisting it
+ // (e.g. "iceberg" is stored as "ICEBERG"), so the filter value must be
upper-cased to match.
+ String filter =
+ hive_metastoreConstants.HIVE_FILTER_FIELD_PARAMS
+ + BaseMetastoreTableOperations.TABLE_TYPE_PROP
+ + " like \""
Review Comment:
@pvary Thanks for the question! I dug into this (with some help from AI to
trace through the HMS code paths), and here is what I found — happy to be
corrected if I missed anything.
You're right that = is generally cheaper than like, but in this specific
case the difference seems to be effectively zero, for two reasons:
1. **No index benefit either way**: the filter is applied on
TABLE_PARAMS.PARAM_VALUE, which has no index in the HMS schema (the index is on
(TBL_ID, PARAM_KEY)). So both = and like end up scanning the same rows, and
since we don't use any %/_ wildcards in the pattern, like "ICEBERG" should be
semantically equivalent to = "ICEBERG" here.
2. **= can be problematic on some backends**: PARAM_VALUE is a CLOB on Derby
and Oracle, where direct equality comparison fails — see HIVE-21614
("Derby/Oracle does not support CLOB comparisons"). That seems to be exactly
why newer HMS versions rewrite EQUALS into LIKE internally (see
ExpressionTree.visit(LeafNode),
L346-L350)：https://github.com/apache/hive/blob/abf94a5993763523f5249df69dc7ef5ca35bff89/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java#L346-L350
— but older versions like HMS 2.3.x don't have this rewrite, so sending =
directly might break against those metastores.
This also seems consistent with what Hive itself does in IcebergTableUtil
(L925), which uses like for the same filter.
Given all this, I can see two reasonable ways forward — I'd love your call
on which you prefer:
**Option A (current): keep like**. Same performance in practice, and it
works consistently across HMS versions and DB backends without any extra config.
**Option B: switch to =, but bring back the list-iceberg-tables-legacy
flag**. This keeps the exact-match semantics, while giving users an escape
hatch to fall back to the old client-side behavior if they hit issues on
older/CLOB-based metastores.
My mild preference leans towards Option A (simpler, one less config to
maintain), but I'm genuinely fine either way. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
