Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23107 )
Change subject: IMPALA-14185: Error unnesting nested array from Iceberg with DELETE files ...................................................................... IMPALA-14185: Error unnesting nested array from Iceberg with DELETE files When trying to doubly unnest a 2D array from an Iceberg table that has delete files but not for every data file, we run into an error: Filtering an unnested collection that comes from a UNION [ALL] is not supported yet. This is because there is a UNION node because of the Iceberg delete files, and there is an added "not-empty" conjunct on the collections. IMPALA-12753 describes a bug where a conjunct on an unnested collection coming from a UNION ALL is only applied to the first UNION operand. To avoid incorrectness, we disabled this case in the commit for IMPALA-12695, but its unintended consequence is that it leads to this error with Iceberg tables. However, in this case with Iceberg deletes, the bug described in IMPALA-12753 is not present because both sides of the UNION have the same tuple id, so conjuncts are naturally applied to both sides. This commit relaxes the check, which now does not fire if all UNION operands have the same tuple ids. Testing: - existing tests related to IMPALA-12753 pass - added a regression test with an Iceberg table with DELETE files Change-Id: Ifbc6f580586d4b337f33a2f32052aa07f6fca828 Reviewed-on: http://gerrit.cloudera.org:8080/23107 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/planner/UnnestNode.java M tests/query_test/test_nested_types.py 2 files changed, 37 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/23107 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifbc6f580586d4b337f33a2f32052aa07f6fca828 Gerrit-Change-Number: 23107 Gerrit-PatchSet: 5 Gerrit-Owner: Daniel Becker <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
