Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/8085 )
Change subject: IMPALA-5307: part 1: don't transfer disk I/O buffers out of parquet ...................................................................... Patch Set 5: (7 comments) http://gerrit.cloudera.org:8080/#/c/8085/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/8085/5//COMMIT_MSG@7 PS5, Line 7: IMPALA-5307 Can you add a line at the bottom what the other part(s) would look like? http://gerrit.cloudera.org:8080/#/c/8085/5//COMMIT_MSG@56 PS5, Line 56: +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ Nit: You could make the second column smaller to make this more readable, and add a bottom delimiter line to indicate it was truncated on purposed and not by mistake. http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/hdfs-parquet-scanner.cc File be/src/exec/hdfs-parquet-scanner.cc: http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/hdfs-parquet-scanner.cc@245 PS5, Line 245: context_->ReleaseCompletedResources(nullptr, true); I think it's best to change the whole file at once, or only change occurrences where necessary. This looks like it may be left from a previous patchset. http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.h File be/src/exec/parquet-column-readers.h: http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.h@476 PS5, Line 476: Status AllocateUncompressedDataPage( Should we call this "AllocateUncompressedDataBuffer"? Otherwise it sounds to me like it'll only be needed for uncompressed pages. http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.h@477 PS5, Line 477: int64_t size, const std::string& desc, uint8_t** buffer); Maybe err_desc, err_detail, or detail? "desc" reminds me of descriptors. http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.h@485 PS5, Line 485: IsStringType This does not say "VarLenStringType" but above in a comment you refer to var-len data. Can you clarify one of them? http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.cc File be/src/exec/parquet-column-readers.cc: http://gerrit.cloudera.org:8080/#/c/8085/5/be/src/exec/parquet-column-readers.cc@1075 PS5, Line 1075: uncompressed_size, "uncompressed variable-length data", ©_buffer)); DCHECK(copy_buffer != nullptr); And maybe initialize it to nullptr, so that it's explicit what the allocation will do. -- To view, visit http://gerrit.cloudera.org:8080/8085 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I767c1e2dabde7d5bd7a4d5c1ec6d14801b8260d2 Gerrit-Change-Number: 8085 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Comment-Date: Mon, 25 Sep 2017 23:30:35 +0000 Gerrit-HasComments: Yes
