Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24968
to look at the new patch set (#4).
Change subject: IMPALA-15447: Iceberg scan telemetry (Stream A + B)
......................................................................
IMPALA-15447: Iceberg scan telemetry (Stream A + B)
Adds two-stream Iceberg query telemetry to the Impala frontend:
Stream A (scan reports): Wire a catalog-configured side MetricsReporter
alongside the existing InMemoryMetricsReporter in IcebergScanPlanner.
The side reporter is loaded once per JVM via CatalogUtil.loadMetricsReporter
using the 'metrics-reporter-impl' JVM system property. For the fast path
where planFiles() is skipped, synthesize a faithful ScanReport from the
already-computed file descriptors and post it only to the side reporter.
Stamp a query-id and engine-name onto every scan report via
IcebergUtil.planFiles overload that passes scan options as report metadata,
enabling cross-stream correlation downstream.
Stream B (query shape): Add IcebergQueryShapeCollector, which runs
once per query after Planner.createPlans. It walks the plan tree,
extracts equi-join keys and ORDER BY columns that resolve to Iceberg
tables, and POSTs a per-table query-shape event via the same telemetry
library (called reflectively to avoid a compile-time dep), tagged with
the identical query id. The two streams correlate on source.query_id.
Both streams are entirely best-effort: neither throws nor blocks query
planning, and both no-op when no side reporter is configured (default =
today's behavior).
Change-Id: I2a290d79ec467a663af32efcce4bcdfebd7d9287
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
---
A fe/src/main/java/org/apache/impala/planner/IcebergQueryShapeCollector.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
4 files changed, 564 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/68/24968/4
--
To view, visit http://gerrit.cloudera.org:8080/24968
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2a290d79ec467a663af32efcce4bcdfebd7d9287
Gerrit-Change-Number: 24968
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
