tanmayrauth opened a new pull request, #1501:
URL: https://github.com/apache/iceberg-go/pull/1501
PR 4: Instruments local scan planning to gather scan metrics and emit a
ScanReport through the scan's configured reporter.
Builds on the reporter plumbing already merged in #1430 (Scan.Reporter(),
catalog/table/scan wiring). No new public API — metrics-reporter-impl /
WithMetricsReporter remain the entry points.
Changes
- PlanFiles now times local planning and, on success, emits a ScanReport
via scan.Reporter().Report(...). Split into PlanFiles (mode dispatch +
emission) and planFilesLocal (the local-planning body).
- Manifest counts (total / scanned / skipped, split by data vs delete
content) gathered in the single-goroutine manifest-filter loop.
- Result-scoped metrics (result-data-files, result-delete-files,
positional/equality/DV counts, total-file-size-in-bytes,
total-delete-file-size-in-bytes) derived from the planned tasks — delete files
counted once by path, and
DV-suppressed positional deletes excluded, so they stay consistent with
result-data-files.
- Schema/filter/projected-fields resolved best-effort; report building
never fails a scan.
Behavior / scope
- Opt-in, zero-overhead default: with the no-op reporter (the default) no
report is built — the instrumented path costs only a few integer increments.
- Remote planning reports its own metrics server-side and does not emit a
local report.
- Race-safe: all counters written from a single goroutine (manifest counts
before the read barrier, result/delete counts after); verified under -race.
- Omit, don't fabricate: scanned/skipped reflect partition-spec pruning;
per-entry skipped-data-files / skipped-delete-files and indexed-delete-files
are left unset (deferred) rather than reported as zero.
Testing
go test ./table/... ./metrics/... and -race on the scan/delete/DV planning
suites. New tests: report assembly, projected-field resolution, result-scoped
delete dedup (TestApplyResultDeleteMetrics), emit-on-local, no-emit-on-remote,
and no-op-reporter safety.
Related: #1236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
