Hello Zoltan Borok-Nagy, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11949
to look at the new patch set (#10).
Change subject: IMPALA-7869: break up parquet-column-readers.cc
......................................................................
IMPALA-7869: break up parquet-column-readers.cc
Move parquet classes into exec/parquet.
Move CollectionColumnReader and ParquetLevelDecoder into separate files.
Remove unnecessary 'encoding_' field from ParquetLevelDecoder.
Switch BOOLEAN decoding to use composition instead of inheritance. This
lets the boolean decoding use the faster batched implementations in
ScalarColumnReader and avoids some confusing aspects of the class
hierarchy, like the ReadValueBatch() implementation on the base class
that was shared between BoolColumnReader and CollectionColumnReader.
Improve compile times by instantiating BitPacking templates in a
separate file (this looks to give a 30s+ speedup for
compiling parquet-column-readers.cc).
Testing:
Ran exhaustive tests.
Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8
---
M be/CMakeLists.txt
M be/src/benchmarks/bit-packing-benchmark.cc
M be/src/benchmarks/bswap-benchmark.cc
M be/src/codegen/impala-ir.cc
M be/src/exec/CMakeLists.txt
M be/src/exec/data-source-scan-node.cc
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-table-sink.cc
A be/src/exec/parquet/CMakeLists.txt
R be/src/exec/parquet/hdfs-parquet-scanner-ir.cc
R be/src/exec/parquet/hdfs-parquet-scanner-test.cc
R be/src/exec/parquet/hdfs-parquet-scanner.cc
R be/src/exec/parquet/hdfs-parquet-scanner.h
R be/src/exec/parquet/hdfs-parquet-table-writer.cc
R be/src/exec/parquet/hdfs-parquet-table-writer.h
A be/src/exec/parquet/parquet-bool-decoder.cc
A be/src/exec/parquet/parquet-bool-decoder.h
A be/src/exec/parquet/parquet-collection-column-reader.cc
A be/src/exec/parquet/parquet-collection-column-reader.h
R be/src/exec/parquet/parquet-column-readers.cc
R be/src/exec/parquet/parquet-column-readers.h
R be/src/exec/parquet/parquet-column-stats.cc
R be/src/exec/parquet/parquet-column-stats.h
R be/src/exec/parquet/parquet-column-stats.inline.h
R be/src/exec/parquet/parquet-common.cc
R be/src/exec/parquet/parquet-common.h
A be/src/exec/parquet/parquet-level-decoder.cc
A be/src/exec/parquet/parquet-level-decoder.h
R be/src/exec/parquet/parquet-metadata-utils.cc
R be/src/exec/parquet/parquet-metadata-utils.h
R be/src/exec/parquet/parquet-plain-test.cc
R be/src/exec/parquet/parquet-scratch-tuple-batch.h
R be/src/exec/parquet/parquet-version-test.cc
M be/src/util/CMakeLists.txt
M be/src/util/bit-packing-test.cc
A be/src/util/bit-packing.cc
M be/src/util/bit-packing.h
M be/src/util/bit-packing.inline.h
M be/src/util/bit-stream-utils.inline.h
M be/src/util/dict-encoding.h
M be/src/util/dict-test.cc
M be/src/util/parquet-reader.cc
M be/src/util/rle-test.cc
M common/thrift/generate_error_codes.py
M
testdata/workloads/functional-query/queries/QueryTest/parquet-num-values-def-levels-mismatch.test
45 files changed, 1,171 insertions(+), 785 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/11949/10
--
To view, visit http://gerrit.cloudera.org:8080/11949
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0efd5c50b781fe9e3c022b33c66c06cfb529c0b8
Gerrit-Change-Number: 11949
Gerrit-PatchSet: 10
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
