Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24871 )
Change subject: IMPALA-15203: Add aggregate function support for UUID columns ...................................................................... IMPALA-15203: Add aggregate function support for UUID columns This patch adds support for common aggregate functions on Iceberg UUID columns, including COUNT, MIN, MAX, COUNT(DISTINCT), NDV, and sampled_ndv. MIN/MAX comparisons use RFC 4122 byte ordering on the raw 16-byte representation. For approximate distinct counting (NDV and sampled_ndv), UUID reuses the existing STRING HLL update functions since both types cross the UDF boundary as StringVal and require the same hashing behavior. Additionally, this patch registers planner helper builtins (aggif, case, decode, murmur_hash) needed for grouping sets and multi-class DISTINCT aggregates over UUID columns. COMPUTE STATS skips UUID columns via ignoreColumn() until stats gathering is fully validated. Key changes: - BE: Implement MinUuid and MaxUuid aggregate update functions that compare raw 16-byte UUID values. - BE: Add AggIfUpdateUuid/AggIfMergeUuid using inline intermediate slots. - BE: Fix SetAnyVal in agg-fn-evaluator to always bind inline intermediate StringVal ptr to the current tuple slot, even when NULL. This applies to TYPE_CHAR, TYPE_FIXED_UDA_INTERMEDIATE, and TYPE_UUID, preventing stale pointer writes in the interpreted aggregate path. - BE: Exclude UUID from codegen optimizations that assume numeric MIN/MAX semantics (default-value initialization and null-bit shortcuts). - BE: Enable hash-table codegen for TYPE_UUID. - FE: Enable aggif, case, decode builtins for UUID. - FE: Skip UUID columns in COMPUTE STATS via ignoreColumn() and allow column whitelists that exclude UUID columns. Testing: - Testdata: Add iceberg_uuid_test to functional_schema_template.sql so the table is available for ad-hoc testing. - Add positive test cases for MIN, MAX, COUNT, COUNT(DISTINCT), GROUP BY, NDV with precision, sampled_ndv, and HAVING clauses. - Add tests for aggif/case/decode with grouping sets and multi-distinct aggregates. - Test COMPUTE STATS rejection for table-level and UUID column, and positive case for non-UUID column whitelist. - Retain negative tests confirming that SUM and AVG are correctly rejected at analysis time for UUID. Change-Id: I2187495c8be5b95ecff3f458b6549916466e9c83 Assisted-by: Composer 2.5 (Cursor) Reviewed-on: http://gerrit.cloudera.org:8080/24871 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/codegen/codegen-anyval.cc M be/src/exec/aggregator.cc M be/src/exec/hash-table.cc M be/src/exprs/agg-fn-evaluator.cc M be/src/exprs/aggregate-functions-ir.cc M be/src/exprs/aggregate-functions.h M be/src/exprs/case-expr.cc M common/function-registry/impala_functions.py M fe/src/main/java/org/apache/impala/analysis/CaseExpr.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/iceberg-uuid-aggregates.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-uuid-type.test M tests/query_test/test_iceberg.py 16 files changed, 537 insertions(+), 110 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2187495c8be5b95ecff3f458b6549916466e9c83 Gerrit-Change-Number: 24871 Gerrit-PatchSet: 9 Gerrit-Owner: Arnab Karmakar <[email protected]> Gerrit-Reviewer: Arnab Karmakar <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
