Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17831 )
Change subject: IMPALA-10900: Add Iceberg tests that write many files ...................................................................... IMPALA-10900: Add Iceberg tests that write many files In earlier versions of Impala we had a bug that affected insertions to Iceberg tables. When Impala wrote multiple files during a single INSERT statement it could crash, or even worse, it could silently omit data files from the Iceberg metadata. The current master doesn't have this bug, but we don't really have tests for this case. This patch adds tests that write many files during inserts to an Iceberg table. Both non-partitioned and partitioned Iceberg tables are tested. We achieve writing lots of files by setting 'parquet_file_size' to 8 megabytes. Testing: * added e2e test that write many data files * added exhaustive e2e test that writes even more data files Change-Id: Ia2dbc2c5f9574153842af308a61f9d91994d067b Reviewed-on: http://gerrit.cloudera.org:8080/17831 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- A testdata/workloads/functional-query/queries/QueryTest/iceberg-write-many-files-stress.test A testdata/workloads/functional-query/queries/QueryTest/iceberg-write-many-files.test M tests/query_test/test_iceberg.py 3 files changed, 193 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17831 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia2dbc2c5f9574153842af308a61f9d91994d067b Gerrit-Change-Number: 17831 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
