andygrove opened a new issue, #4964: URL: https://github.com/apache/datafusion-comet/issues/4964
Spark 4.2 adds a segment-tree window-aggregation optimization to `WindowExec`, along with metrics such as "number of segment-tree frames prepared" and "number of segment-tree fallback frames prepared", plus an eligibility allowlist. Comet replaces `WindowExec` with `CometWindowExec`, which has no segment-tree implementation and does not expose those metrics. The new test suites read the metric from a Spark `Window` node in the executed plan and fail with "No Window node in plan" when Comet is enabled. Affected Spark SQL test suites (currently marked `IgnoreCometSuite` in `dev/diffs/4.2.0.diff`): - `SegmentTreeWindowMetricsSuite` - `WindowSegmentTreeAllowlistSuite` Part of #4142 (Track Spark 4.2 test failures). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
