Hello Quanlong Huang, Gabor Kaszab, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17039
to look at the new patch set (#2).
Change subject: IMPALA-10485: part(1): make ORC column reader creation
independent of schema resolution
......................................................................
IMPALA-10485: part(1): make ORC column reader creation independent of schema
resolution
Currently ORC schema resolution is position-based only. Then the
positional information is reused during ORC column reader creation.
This prevents adding other column resolution strategies, e.g.
column resolution by name or Iceberg field id. It also prevents
schema evolution, as table metadata and file metadata should be
in sync.
This patch makes column reader creation independent of schema
resolution. It does this by creating a mapping between slot/tuple
descriptors and ORC type ids during schema resolution and use
this mapping during column reader creation.
Now further patches just need to add support for other column
resolution strategies, the column readers will be created
accordingly.
Testing:
* no additional tests as it is neither a bug fix nor a new feature
Change-Id: I0f7d521f9397c5188fadc7996cee0bd1650d363e
---
M be/src/exec/hdfs-orc-scanner.cc
M be/src/exec/hdfs-orc-scanner.h
M be/src/exec/orc-column-readers.cc
M be/src/exec/orc-column-readers.h
M be/src/exec/orc-metadata-utils.cc
M be/src/exec/orc-metadata-utils.h
M tests/query_test/test_scanners.py
7 files changed, 69 insertions(+), 137 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/17039/2
--
To view, visit http://gerrit.cloudera.org:8080/17039
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0f7d521f9397c5188fadc7996cee0bd1650d363e
Gerrit-Change-Number: 17039
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
