Daniel Becker has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/19167 )
Change subject: IMPALA-11685: Slot memory sharing between struct and field not working if the field is also a struct ...................................................................... IMPALA-11685: Slot memory sharing between struct and field not working if the field is also a struct IMPALA-10838 introduced that if a struct and one of its fields are both present in the select list, no extra slot is generated in the row for the struct field but the memory of the struct is reused, i.e. the row size is the same as when only the struct is queried. It works when the struct field is a primitive type: explain select id, outer_struct from functional_orc_def.complextypes_nested_structs; row-size=64B explain select id, outer_struct, outer_struct.str from functional_orc_def.complextypes_nested_structs; row-size=64B However, it does not if the child is itself a struct: explain select id, outer_struct, outer_struct.inner_struct3 from functional_orc_def.complextypes_nested_structs; row-size=80B This is because struct slot descriptors are registered before others so that it is easier to reuse the slot memory of the struct fields, but struct slot descriptors among themselves are sorted in the wrong order (see https://github.com/apache/impala/blob/c12ac6c27b2df1eae693b44c157d65499f491d21/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java#L340). Also, because struct children are also inserted into the Analyzer's slotPathMap_ when the struct is registered, there is no need for the mechanism that tries to find ancestors of fields to be able to share their slot memory - they can be retrieved from the slotPathMap_. This change deletes the code that dealt with that. Testing: - PlannerTest#testStructFieldSlotSharedWithStruct has been updated to include queries where the struct field is also a struct; the elements in the select list are permutated to make sure the order in which they are listed does not matter. Change-Id: I6d4dee3941fb2d285fbd3836ea5712c859db8848 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/Path.java M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java 4 files changed, 25 insertions(+), 159 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/67/19167/3 -- To view, visit http://gerrit.cloudera.org:8080/19167 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6d4dee3941fb2d285fbd3836ea5712c859db8848 Gerrit-Change-Number: 19167 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
