andygrove opened a new pull request, #6557: URL: https://github.com/apache/datafusion-comet/pull/6557
## Which issue does this PR close? Related to #5703. This PR does not close it. ## Rationale for this change The user guide lists `peak native aggregate memory` as a working `CometHashAggregate` metric, but it never gets a value. #5423 added the metric. Since the DataFusion 55 upgrade (#5262), DataFusion picks new aggregate streams by default (`datafusion.execution.enable_migration_aggregate = true`). Those streams register the spill metrics but not `peak_mem_used`. Only the older `GroupedHashAggregateStream` does. Comet doesn't change that setting, so the native aggregate never reports the metric. #5262 ignored the two `CometAggregateSuite` tests that check it, and #5703 tracks re-enabling them. The three aggregate spill metrics in the same table are not affected. ## What changes are included in this PR? One row in `docs/source/user-guide/latest/metrics.md` now says the metric isn't currently reported and links #5703. When #5703 is fixed, the row can go back to the plain description. ## How are these changes tested? This is a docs-only change. `npx prettier@latest --check` passes on the file. To confirm the cause, I checked the DataFusion 55.1.0 sources that Comet builds against: `peak_mem_used` is registered only in `aggregates/grouped_hash_stream.rs`. DataFusion main is the same as of 2026-10-01. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
