Jiawei Wang has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/14776 )
Change subject: IMPALA-7550: Add documentation to profile counters ...................................................................... IMPALA-7550: Add documentation to profile counters This is the work based on Lars' experiment on https://gerrit.cloudera.org/#/c/12116/ This commit did the following refactors on profile counters. 1. Add a singleton registry for runtime profile counters prototypes, similiar to what Kudu does for metrics. This allows us to generate profile documentation for all counters from the code. We add /profile_docs and a correspoding UI for the documentation of profile counters. 2. Profile counters are annotated with their stability: * Stable_HIGH counters - High level and stable counters. Always useful on measuring query performance and status. Counters that everyone is interested. Should rarely change and if it does we will make some effort to notify users. * Stable_LOW - Low level and sable counters - interesting counters to monitor and analyze by machine. It will probably be interesting under some circumstance for users. Lots of developers are interested. * Unstable but useful - useful to understand query performance, but subject to change, particularly if the implementation changes. E.g. RowBatchQueuePutWaitTime, MaterializeTupleTimer * Debugging counters - generally not useful to users of Impala, the main use case is low-level debugging. Can be hidden to reduce noise for most consumers of profiles. 3. We have around 250 counters. This commit did the replacement in scan-node and hdfs-scan-node-base and coordinator. This is still a WIP work and all the advices are welcomed! The downside is that we'd reduce the comments that currently explain some of the counters in header files by moving them to the .cc files. Additionally a (arguably good) limitation is that profile counter names need to be unique. Change-Id: Idc03faddb27754001290bb6d899840e2cbe7ccb7 --- M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/exec/hdfs-scan-node-base.cc M be/src/exec/hdfs-scan-node-base.h M be/src/exec/kudu-scan-node-base.cc M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator.cc D be/src/util/debug-counters.h M be/src/util/default-path-handlers.cc M be/src/util/runtime-profile-counters.h M be/src/util/runtime-profile.cc A www/profile_docs.tmpl M www/query_profile.tmpl 15 files changed, 620 insertions(+), 292 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/14776/2 -- To view, visit http://gerrit.cloudera.org:8080/14776 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idc03faddb27754001290bb6d899840e2cbe7ccb7 Gerrit-Change-Number: 14776 Gerrit-PatchSet: 2 Gerrit-Owner: Jiawei Wang <jiawei.w...@cloudera.com> Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com> Gerrit-Reviewer: Balazs Jeszenszky <jes...@gmail.com> Gerrit-Reviewer: David Rorke <dro...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Jiawei Wang <jiawei.w...@cloudera.com> Gerrit-Reviewer: Lars Volker <l...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
