Hello Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17748
to look at the new patch set (#2).
Change subject: IMPALA-10808: (addendum) Abort on illegal decimal parquet
schemas
......................................................................
IMPALA-10808: (addendum) Abort on illegal decimal parquet schemas
The previous patch added checks on illegal decimal schemas of parquet
files. However, it doesn't return a non-ok status in
ParquetMetadataUtils::ValidateColumn if abort_on_error is set to false.
So we continue to use the illegal file schema and hit the DCHECK.
This patch fixes this and adding test coverage for illegal decimal
schemas.
Tests:
- Add a bad parquet file with illegal decimal schemas.
- Add e2e tests on the file.
- Ran test_fuzz_decimal_tbl 100 times. Saw the errors are caught as
expected.
Change-Id: I623f255a7f40be57bfa4ade98827842cee6f1fee
---
M be/src/exec/parquet/parquet-metadata-utils.cc
M common/thrift/generate_error_codes.py
M testdata/bad_parquet_data/README
A testdata/bad_parquet_data/illegal_decimals.parq
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
M
testdata/workloads/functional-query/queries/QueryTest/parquet-abort-on-error.test
M
testdata/workloads/functional-query/queries/QueryTest/parquet-continue-on-error.test
8 files changed, 144 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/17748/2
--
To view, visit http://gerrit.cloudera.org:8080/17748
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I623f255a7f40be57bfa4ade98827842cee6f1fee
Gerrit-Change-Number: 17748
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
