Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19869 )
Change subject: IMPALA-12123: Fix crash triggered by incomplete HDFS cache reads ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/19869/1/be/src/runtime/io/scan-range.cc File be/src/runtime/io/scan-range.cc: http://gerrit.cloudera.org:8080/#/c/19869/1/be/src/runtime/io/scan-range.cc@344 PS1, Line 344: if (cache_.data != nullptr) { : DCHECK_LE(offset + bytes_to_read, cache_.len); : memcpy(buffer_desc->buffer_ + buffer_desc->len(), : cache_.data + offset, bytes_to_read); : } else { My assumption here is that we already did checks up in the Parquet level so even if it was reading a corrupted / invalid file, we won't have subranges that go past the edge of this buffer. The other option would be to return Status rather than DCHECKing. It is complicated to exercise this line in the end-to-end minicluster with real HDFS caching, so the DCHECK isn't likely to get exercised or provide much protection. -- To view, visit http://gerrit.cloudera.org:8080/19869 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51d8be6c03716badee81675447ed94ae6249b21b Gerrit-Change-Number: 19869 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Wed, 10 May 2023 23:20:54 +0000 Gerrit-HasComments: Yes
