Hello Zoltan Borok-Nagy, Impala Public Jenkins, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18372 to look at the new patch set (#4). Change subject: IMPALA-11039: Fix incorrect page jumping in late materialization of Parquet ...................................................................... IMPALA-11039: Fix incorrect page jumping in late materialization of Parquet The current calculation of LastRowIdxInCurrentPage() is incorrect. It uses the first row index of the next candidate page instead of the next valid page. The next candidate page could be far away from the current page. Thus giving a number larger than the current page size. Skipping rows in the current page could overflow the boundary due to this. This patch fixes LastRowIdxInCurrentPage() to use the next valid page. When skip_row_id is set (>0), the current approach of SkipRowsInternal<false>() expects jumping to a page containing this row and then skipping rows in that page. However, the expected row might not be in the candidate pages. When we jump to the next candidate page, the target row could already be skipped. In this case, we don't need to skip rows in the current page. Tests: - Add a test on alltypes_empty_pages to reveal the bug. - Add more batch_size values in test_page_index. - Pass tests/query_test/test_parquet_stats.py locally. Change-Id: I3a783115ba8faf1a276e51087f3a70f79402c21d --- M be/src/exec/parquet/hdfs-parquet-scanner.cc M be/src/exec/parquet/parquet-column-readers.cc M be/src/exec/parquet/parquet-column-readers.h M be/src/exec/parquet/parquet-common.cc M be/src/exec/parquet/parquet-common.h M testdata/workloads/functional-query/queries/QueryTest/parquet-page-index.test M tests/query_test/test_parquet_stats.py 7 files changed, 97 insertions(+), 30 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/18372/4 -- To view, visit http://gerrit.cloudera.org:8080/18372 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3a783115ba8faf1a276e51087f3a70f79402c21d Gerrit-Change-Number: 18372 Gerrit-PatchSet: 4 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>