Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24353 )
Change subject: IMPALA-12956: Add profile summary output ...................................................................... IMPALA-12956: Add profile summary output Add a summary profile format that emits one TSV row per archived profile with query-level fields useful for batches of profile logs. The summary output extracts metadata, timing, memory, progress, IO, spill, query status, and statement fields from archived profile trees. Missing fields are emitted as N/A instead of failing the whole export or using synthetic zeroes. The output uses a shared progress formatter and reports duration from Start Time and End Time when possible. It falls back to the query timeline when End Time is absent. It handles both the default V1 profile layout and the experimental V2 profile layout. V2 scan progress uses aggregated counters, while spilled bytes are summed from host-level counters under Per Node Profiles. Refactor shared helpers used by the summary path: move ParseUtil::ParseMemSpec() to parse-mem-spec.cc, extract ProgressToString() from ImpalaHttpHandler into progress-util, and split profile archive encoding/decoding into reusable Python helpers for test fixture generation. Add regression coverage for V1 and V2 summary output, query id filtering, TSV column shape, unlimited text length, failed/cancelled queries, non-zero spilled bytes, and WebUI thrift-profile downloads. Assisted-by: gpt-5.6-sol (OpenAI Codex) Change-Id: I35da958106ffb35b14ef867da50ed23ffda0a45b Reviewed-on: http://gerrit.cloudera.org:8080/24353 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/util/CMakeLists.txt M be/src/util/impala-profile-tool.cc A be/src/util/parse-mem-spec.cc M be/src/util/parse-util.cc A be/src/util/progress-util.cc A be/src/util/progress-util.h M be/src/util/summary-util-test.cc M bin/rat_exclude_files.txt M lib/python/impala_py_lib/profiles.py M testdata/impala-profiles/README A testdata/impala-profiles/impala_profile_log_cancelled_query A testdata/impala-profiles/impala_profile_log_failed_query A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.summary A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2.expected.summary M tests/observability/test_profile_tool.py M tests/webserver/test_web_pages.py 18 files changed, 925 insertions(+), 116 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24353 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I35da958106ffb35b14ef867da50ed23ffda0a45b Gerrit-Change-Number: 24353 Gerrit-PatchSet: 16 Gerrit-Owner: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Aleksandr Efimov <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
