NEUpanning commented on issue #10367: URL: https://github.com/apache/incubator-gluten/issues/10367#issuecomment-3162318344
I have figured out the root cause. The aggregation op containing sum for floating type leads to use velox single aggregation rather than partial aggregation. Therefore Velox cannot enable "adaptive disabling of partial aggregation" feature for this mostly unique keys case and triggers many spill operations. If we enable flushable agg for this case, the performance is significantly better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
