Hello Quanlong Huang, Qifan Chen, Daniel Becker, Gabor Kaszab, Attila Jeges, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/17811 to look at the new patch set (#28). Change subject: IMPALA-9498: Allow returning arrays in select list ...................................................................... IMPALA-9498: Allow returning arrays in select list Until now ARRAYs had to be unnested in queries. This patch adds support to return ARRAYs as STRINGs (JSON arrays) in select list, for example: select id, int_array from functional_parquet.complextypestbl where id = 1; returns: 1, [1,2,3] Returning ARRAYs from inline or HMS views is also supported - these arrays can be used both in the select list or as relative table references. Using them as non-relative table reference is not supported (IMPALA-11052). Though STRUCTs are already supported, ARRAYs and STRUCTs nested in each other are not supported yet. Things intentionally postponed for later commits: - Add MAP suppport too - this shouldn't be too tricky after ARRAY support, but I don't want to make this patch even more complex. - Unify HS2 / Beeswax logic with the way STRUCTs are handled. This could be done in a "final" logic that can handle STRUCTS/ARRAYS nested to each other - Implement "deep copy" and "deep serialize" for ARRAYs in BE. This would enable all operators, e.g. ORDER BY and UNION. Testing: - FE tests were added for analyses and authorization - EE tests were added - core tests were ran Change-Id: Ibb1e42ffb21c7ddc033aba0f754b0108e46f34d0 --- M be/src/codegen/codegen-anyval.cc M be/src/exec/blocking-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/parquet/parquet-collection-column-reader.cc M be/src/exec/plan-root-sink.cc M be/src/exec/plan-root-sink.h M be/src/exprs/expr.h M be/src/exprs/slot-ref.cc M be/src/exprs/slot-ref.h M be/src/runtime/collection-value.h M be/src/runtime/raw-value.cc M be/src/runtime/raw-value.h M be/src/runtime/types.cc M be/src/runtime/types.h M be/src/service/hs2-util.cc M be/src/service/impala-beeswax-server.cc M be/src/service/impala-server.h M be/src/service/query-result-set.cc M be/src/service/query-result-set.h M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/main/java/org/apache/impala/analysis/SetOperationStmt.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/authorization/TableMask.java M fe/src/main/java/org/apache/impala/catalog/Type.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/main/java/org/apache/impala/planner/UnionNode.java M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeUpsertStmtTest.java M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-planner/queries/PlannerTest/constant-folding.test M testdata/workloads/functional-planner/queries/PlannerTest/constant-propagation.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test M testdata/workloads/functional-planner/queries/PlannerTest/parquet-filtering.test M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test A testdata/workloads/functional-query/queries/QueryTest/nested-array-in-select-list.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_struct_in_select_list.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M testdata/workloads/functional-query/queries/QueryTest/zipping-unnest-in-from-clause.test M testdata/workloads/functional-query/queries/QueryTest/zipping-unnest-in-select-list.test M tests/authorization/test_ranger.py M tests/query_test/test_nested_types.py 55 files changed, 1,119 insertions(+), 232 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/17811/28 -- To view, visit http://gerrit.cloudera.org:8080/17811 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibb1e42ffb21c7ddc033aba0f754b0108e46f34d0 Gerrit-Change-Number: 17811 Gerrit-PatchSet: 28 Gerrit-Owner: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Attila Jeges <atti...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>