Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/12097 )
Change subject: IMPALA-7980: Fix spinning threads because of buggy handling of num_unqueued_files_. ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/12097/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12097/1//COMMIT_MSG@48 PS1, Line 48: Michael Ho. I didn't do much so please feel free to leave me out. http://gerrit.cloudera.org:8080/#/c/12097/1/be/src/exec/hdfs-scan-node-base.h File be/src/exec/hdfs-scan-node-base.h: http://gerrit.cloudera.org:8080/#/c/12097/1/be/src/exec/hdfs-scan-node-base.h@313 PS1, Line 313: /// Calls RangeComplete() with skipped=true for all the splits of the file nit: May also want to update the side effect of decrementing num_unqueued_files_; http://gerrit.cloudera.org:8080/#/c/12097/1/be/src/exec/hdfs-scan-node.cc File be/src/exec/hdfs-scan-node.cc: http://gerrit.cloudera.org:8080/#/c/12097/1/be/src/exec/hdfs-scan-node.cc@469 PS1, Line 469: // File ranges haven't been issued yet, skip entire file : SkipFile(partition->file_format(), desc); I guess this is slightly subtle here as we never decrement "num_unqueued_files_" for reading the sequence file's header in BaseSequenceScanner::IssueInitialRanges(), unlike other file formats which usually decrement that counter in their IssueInitialRanges(). -- To view, visit http://gerrit.cloudera.org:8080/12097 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I133de13238d3d05c510e2ff771d48979125735b1 Gerrit-Change-Number: 12097 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Comment-Date: Mon, 17 Dec 2018 22:51:23 +0000 Gerrit-HasComments: Yes
