Arnab Karmakar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24992
Change subject: IMPALA-15418: Add Bottleneck Summary to query profiles ...................................................................... IMPALA-15418: Add Bottleneck Summary to query profiles Adds a concise "Bottleneck Summary" to the top of the query profile that ranks plan nodes by the fraction of execution wall time they consumed. Execution wall time is computed as the longest fragment instance CompletionTime across all backends. Each plan node's query_time_frac is set to the max latency_ns across its instances divided by that wall time, clamped to [0, 1]. The existing per-node "% non-child" value does not help, because it is relative to the node's own total time rather than to the query. This patch adds a query-relative measure of operator time and uses it to list the busiest operators at the top of the execution profile: ``` Bottleneck Summary: 01:SCAN HDFS busy 18%, 02:HASH JOIN busy 9%, 00:SCAN HDFS busy 9% of execution time ``` Key changes: - The coordinator records the query's execution wall time, i.e. the longest completion time of any fragment instance, in the exec summary. - Each plan node gets a query time fraction: its local time on its slowest instance divided by the execution wall time. Local time excludes time spent waiting on other operators, so the value shows how long the operator itself was busy. Fractions do not add up to 1, because fragments run concurrently. - The Bottleneck Summary lists up to 3 nodes that were busy for at least 5% of the execution time. The fraction is also exposed in the query plan JSON of the debug web UI. Testing: - Added BE unit tests for computing the fractions and building the summary, covering ranking, thresholds and ties. - Added e2e tests that check the fractions against per-instance times, and that an operator with an injected delay ranks first in the summary, with MT_DOP=0 and MT_DOP=2. Change-Id: I6d63cdb9dca24981054bf7c524c1bc3cd3167452 Assisted-by: Claude Opus 5.5 (Cursor) --- M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/coordinator-backend-state.h M be/src/runtime/coordinator.cc M be/src/service/impala-http-handler.cc M be/src/util/CMakeLists.txt M be/src/util/summary-util-test.cc M be/src/util/summary-util.cc M be/src/util/summary-util.h M common/thrift/ExecStats.thrift M tests/query_test/test_observability.py 10 files changed, 245 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/24992/1 -- To view, visit http://gerrit.cloudera.org:8080/24992 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6d63cdb9dca24981054bf7c524c1bc3cd3167452 Gerrit-Change-Number: 24992 Gerrit-PatchSet: 1 Gerrit-Owner: Arnab Karmakar <[email protected]>
