sam-1112 commented on PR #5835: URL: https://github.com/apache/datafusion-comet/pull/5835#issuecomment-6363873965
@sunchao @andygrove Now that #6548 has merged, I’ve narrowed this PR to non-foldable `sha2(payload, numBits)`. `hash` and `xxhash64` no longer use `CodegenDispatchFallback`. The decimal routing tests now require native execution with no dispatcher activity, including nested decimals, and the documentation reflects that. I also reran `CometHashExpressionSuite`: 29 tests passed on Spark 4.1, and 28 passed on Spark 3.5, with only the Spark 4.1+ TimeType test canceled as expected. The focused SHA-2 benchmark compares dispatch enabled, disabled, and Spark. On my M4 with Spark 4.1.3, the 1M-row mixed projection averaged 159/291/334 ms respectively. Isolated SHA-2 was about 8% slower with dispatch, so the measured benefit is retaining the surrounding projection in Comet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
