Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17234 )
Change subject: IMPALA-10607: Fixed test_ctas_exprs failure for S3 build ...................................................................... IMPALA-10607: Fixed test_ctas_exprs failure for S3 build New test case TestDecimalOverflowExprs::test_ctas_exprs was added in the patch for IMPALA-10564. But it failed in S3 build with Parquet format and complained the Parquet file had an invalid file length when accessing a table. The table was created by CTAS which finished with error "decimal expression overflowed". Verified this issue does not happen if query option s3_skip_insert_staging is set as false. When s3_skip_insert_staging is set true by default, INSERT writing to S3 goes directly to their final location rather than being copied there by the coordinator. If CTAS finishs with error during INSERT, the parquet partition file is left in un-finalized without file footer. This causes subsequent query failed with error like "have an invalid file length on S3" when the query attemps to access the same table. This patch fixed the issue by deleting the un-finalized file in its final location when AppendRows() return error and staging has been skipped. Testing: - Reproduced the test failure in local box with defaultFS as s3. Verified the fixing by running test_ctas_exprs with defaultFS as s3. - Passed core tests. Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Reviewed-on: http://gerrit.cloudera.org:8080/17234 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exec/hdfs-table-sink.cc M tests/query_test/test_decimal_queries.py 2 files changed, 47 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/17234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic2f64ab987aeada2cda41502e8c5dbbc229daefd Gerrit-Change-Number: 17234 Gerrit-PatchSet: 5 Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
