Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10810 )
Change subject: IMPALA-7095: clean up scan node profiles ...................................................................... IMPALA-7095: clean up scan node profiles Add counters to scan node implementations where they make sense but were missing (e.g. row batch queue counters for multithread Kudu scans) and remove them where they don't make sense (e.g. scanner thread counters for non-multithreaded scans). Refactors the multithreaded Kudu and HDFS scans to share logic via composition (single inheritance doesn't work for this case), which enables the same set of counters to be maintained with shared code. The row batch queueing and thread tracking is now shared. I looked at combining the logic around 'status_', 'lock_' and 'done_' between the two but the details were different enough that it didn't seem worth abstracting. Adds a PeakScannerThreadConcurrency counter - this answers a common question. Fixes RowsRead for data source scans. Fix some of the comments to be more accurate/useful. Testing: Ran exhaustive tests. Ran various types of scans (HDFS, Kudu, HBase, Data source) and inspected the profile output manually. Change-Id: I77286282d42e7764bfdf94c7ec47cec9d743f787 Reviewed-on: http://gerrit.cloudera.org:8080/10810 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exec/data-source-scan-node.cc M be/src/exec/data-source-scan-node.h M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/exec/hbase-table-scanner.cc M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/hdfs-scan-node-mt.h M be/src/exec/hdfs-scan-node.cc M be/src/exec/hdfs-scan-node.h M be/src/exec/hdfs-text-scanner.cc M be/src/exec/kudu-scan-node-base.h M be/src/exec/kudu-scan-node-mt.cc M be/src/exec/kudu-scan-node.cc M be/src/exec/kudu-scan-node.h M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/fragment-instance-state.cc M be/src/util/blocking-queue.h M be/src/util/thread.h 20 files changed, 496 insertions(+), 357 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10810 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I77286282d42e7764bfdf94c7ec47cec9d743f787 Gerrit-Change-Number: 10810 Gerrit-PatchSet: 11 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
