Hello Pranay Singh, Lars Volker, Csaba Ringhofer,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9556
to look at the new patch set (#3).
Change subject: IMPALA-6589: remove invalid DCHECK in parquet reader
......................................................................
IMPALA-6589: remove invalid DCHECK in parquet reader
The DCHECK was only valid if the Parquet file metadata is internally
consistent, with the number of values reported by the metadata
matching the number of encoded levels.
The DCHECK was intended to directly detect misuse of the RleBatchDecoder
interface, which would lead to incorrect results. However, our other
test coverage for reading Parquet files is sufficient to test the
correctness of level decoding.
Testing:
Added a minimal corrupt test file that reproduces the issue.
Change-Id: Idd6e09f8c8cca8991be5b5b379f6420adaa97daa
---
M be/src/exec/parquet-column-readers.cc
M be/src/exec/parquet-column-readers.h
M testdata/data/README
A testdata/data/num_values_def_levels_mismatch.parquet
A
testdata/workloads/functional-query/queries/QueryTest/parquet-num-values-def-levels-mismatch.test
M tests/query_test/test_scanners.py
6 files changed, 50 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/9556/3
--
To view, visit http://gerrit.cloudera.org:8080/9556
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd6e09f8c8cca8991be5b5b379f6420adaa97daa
Gerrit-Change-Number: 9556
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Pranay Singh
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
