Hello Tim Armstrong, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14135 to look at the new patch set (#2). Change subject: IMPALA-8718: project out collection slots in analytic's sort tuple ...................................................................... IMPALA-8718: project out collection slots in analytic's sort tuple Subplan node is mainly used to extract collection values. It evaluates its right plan tree (usually a nested loop join) for every row from its left child (usually a scan producing tuples with collection values), and returns those rows produced by the right child. Each row (TupleRow) produced by the join node consists of several tuples from the join operands. So the scan node tuple that contains collection values will be part of the output of the join node, then become part of the output of the subplan node. When generating analytic plan, a TupleDescriptor for sort is created based on the materialized slots of the input. If the input comes from a subplan node, there are collection slots in it. These collection slots will be picked out into the sort tuple, and occur in the smap of it. Then the output smap of the analytic plan will contain the collection slot consequently. This causes IllegalStateException if the analytic plan is the nullable side of an outer join. The exception is thrown when we are checking the necessary of adding a TupleIsNullPredicate for each output slot. We should project out the collection slots in creating the sort tuple of analytic plan to avoid causing such an exception. Projecting out them is safe since outputs of the analytic node must be in the select list of the block with the analytic, and we don't allow collection types to be returned from a select block, and also don't support any builtin or UDF functions that take collection types as an argument. Tests - Add Planner test in analytic-fns.test with VALIDATE_CARDINALITY enabled. Also fix some incorrect row-sizes of existing tests. - Add e2e test in nested-types-runtime.test to verify that collection slots are projected out. Change-Id: I7edf74ff0f603dfd33ff546e61545bc724990655 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test M testdata/workloads/functional-query/queries/QueryTest/nested-types-runtime.test 5 files changed, 137 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/14135/2 -- To view, visit http://gerrit.cloudera.org:8080/14135 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7edf74ff0f603dfd33ff546e61545bc724990655 Gerrit-Change-Number: 14135 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>