Hello Riza Suminto, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17831
to look at the new patch set (#2).
Change subject: IMPALA-10900: Add Iceberg tests that write many files
......................................................................
IMPALA-10900: Add Iceberg tests that write many files
In earlier versions of Impala we had a bug that affected
insertions to Iceberg tables. When Impala wrote multiple
files during a single INSERT statement it could crash, or
even worse, it could silently omit data files from the
Iceberg metadata.
The current master doesn't have this bug, but we don't
really have tests for this case.
This patch adds tests that write many files during inserts
to an Iceberg table. Both non-partitioned and partitioned
Iceberg tables are tested.
We achieve writing lots of files by setting 'parquet_file_size'
to 8 megabytes.
Testing:
* added e2e test that write many data files
* added exhaustive e2e test that writes even more data files
Change-Id: Ia2dbc2c5f9574153842af308a61f9d91994d067b
---
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-write-many-files-stress.test
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-write-many-files.test
M tests/query_test/test_iceberg.py
3 files changed, 193 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/17831/2
--
To view, visit http://gerrit.cloudera.org:8080/17831
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia2dbc2c5f9574153842af308a61f9d91994d067b
Gerrit-Change-Number: 17831
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
