Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24557 )
Change subject: IMPALA-15057: Add variant_get() builtin and first-class VARIANT expressions ...................................................................... IMPALA-15057: Add variant_get() builtin and first-class VARIANT expressions The patch adds variant_get(VARIANT, STRING) -> VARIANT variant_get(VARIANT, STRING, STRING) -> <type> try_variant_get() (2-args and 3-args as well) variant_to_json(VARIANT). There are two main parts: an infrastructure part that makes VARIANT a first-class expression type, and a feature part that implements the functions on top of it. The infrastructure part makes VARIANT a first-class expression type. It introduces the VariantVal UDF ABI (two StringVals: metadata + value) and unblocks the mechanisms that previously restricted VARIANT to a scan-only pass-through SlotRef: AnyVal sizing/alignment/marshalling, symbol mangling, the Get*Val eval dispatch and interpreted overrides, RawValue::WriteVariant materialization, and the FE type-resolution and guard checks. Result rendering (hs2-util, query-result-set) now consumes a VariantVal instead of a raw slot. Because VARIANT codegen is unimplemented, any expression tree that involves a VARIANT type is forced onto the interpreted path. The feature part implements the functions in variant-functions-ir.cc: * the 2-arg form returns a zero-copy slice (shared metadata, value sub-slice) * the 3-arg form coerces to primitive type with widening, checked narrowing, and string parsing. STRING results are rendered through the JSON serializer * DATE is only produced from a DATE value, not silently reinterpreted from an integer The typed forms are registered under internal names because the catalog cannot hold overloads that differ only by return type. FunctionCallExpr maps the type tag to the internal overload. Testing: * backend tests added * e2e tests added Change-Id: I55bfed394ac2fb57135fadedd489f59e4cc10de4 Assisted-by: Claude Opus 5 (Claude Code) Reviewed-on: http://gerrit.cloudera.org:8080/24557 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exprs/CMakeLists.txt M be/src/exprs/agg-fn-evaluator.cc M be/src/exprs/anyval-util.h M be/src/exprs/expr-value.h M be/src/exprs/null-literal.cc M be/src/exprs/null-literal.h M be/src/exprs/scalar-expr-evaluator-ir.cc M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/exprs/scalar-expr.cc M be/src/exprs/scalar-expr.h M be/src/exprs/scalar-expr.inline.h M be/src/exprs/scalar-fn-call.cc M be/src/exprs/scalar-fn-call.h M be/src/exprs/slot-ref.cc M be/src/exprs/slot-ref.h M be/src/exprs/variant-functions-ir.cc A be/src/exprs/variant-functions-test.cc M be/src/exprs/variant-functions.h M be/src/runtime/raw-value.cc M be/src/runtime/variant-value.cc M be/src/runtime/variant-value.h M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M be/src/udf/udf-internal.h M be/src/util/symbols-util.cc M be/src/util/variant-util-test.cc M be/src/util/variant-util.cc M be/src/util/variant-util.h M common/function-registry/impala_functions.py M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java M fe/src/main/java/org/apache/impala/catalog/Function.java M fe/src/main/java/org/apache/impala/catalog/VariantType.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeExprsTest.java A testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-variant-get.test M tests/query_test/test_iceberg.py 37 files changed, 2,158 insertions(+), 80 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I55bfed394ac2fb57135fadedd489f59e4cc10de4 Gerrit-Change-Number: 24557 Gerrit-PatchSet: 17 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Vanko <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
