Alex Behm has posted comments on this change. ( http://gerrit.cloudera.org:8080/8085 )
Change subject: IMPALA-5307: part 1: don't transfer disk I/O buffers out of parquet ...................................................................... Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/8085/6//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8085/6//COMMIT_MSG@44 PS6, Line 44: There is a significant regression (50% increase in runtime) in > I reran with mt_dop=8. Time spent in the scan on STRING-Q1 went from 3.35s Thanks for the experiment. I'm in favor of moving forward, but still a little worried. Maybe we can investigate later whether we can recoup some of the perf losses. The main issue is that affected users have no way to get the performance back, except downgrading. I'm curious to hear what other think of the perf regression. http://gerrit.cloudera.org:8080/#/c/8085/6/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/8085/6/be/src/exec/parquet-column-readers.cc@1071 PS6, Line 1071: if (PageContainsTupleData(current_page_header_.data_page_header.encoding)) { Instead of eagerly copying the entire page, have you considered only coping the var-len data of surviving tuples in TransferScratchTuples() or FinalizeTupleTransfer()? That approach might be more efficient for selective scans, but worse for full scans. -- To view, visit http://gerrit.cloudera.org:8080/8085 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I767c1e2dabde7d5bd7a4d5c1ec6d14801b8260d2 Gerrit-Change-Number: 8085 Gerrit-PatchSet: 6 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Thu, 28 Sep 2017 23:44:50 +0000 Gerrit-HasComments: Yes
