dwsmith1983 commented on PR #5874: URL: https://github.com/apache/datafusion-comet/pull/5874#issuecomment-5678612068
> Could you add a representative `length(binary)` microbenchmark and report Spark-versus-Comet results before enabling this path? ... please include a base-versus-head string comparison. Added `CometBinaryLengthBenchmark`: 1M rows, five shapes, best time over the harness's iterations, Spark 3.5 on an M-series laptop. Base is `main` at 1d0ce5fe8 with its own native build, head is this branch. On base the binary cases fall back to Spark, so the base Comet column there is the fallback path including the columnar-to-row conversion. | case | Spark | Comet base | Comet head | head vs Spark | | --- | --- | --- | --- | --- | | length(string), 10-char varied | 600 ms | 73 ms | 71 ms | 8.4x | | length(binary), 8 B varied | 45 ms | 43 ms | 36 ms | 1.2x | | length(binary), 1 KiB varied | 530 ms | 713 ms | 502 ms | 1.1x | | length(binary), 8 B, 20% null | 43 ms | 41 ms | 32 ms | 1.3x | | length(binary), 5 repeated values | 28 ms | 20 ms | 17 ms | 1.7x | The string path is unchanged within noise, 73 ms base against 71 ms head. The binary path goes from a fallback that is slower than Spark on wide payloads (713 ms against 530 ms for 1 KiB values, where the conversion dominates) to native and ahead of Spark on every shape; the 1 KiB case is scan-bound on both sides, which is why the ratio is small there. The benchmark file is on the branch so the numbers can be reproduced with `make benchmark-org.apache.spark.sql.benchmark.CometBinaryLengthBenchmark`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
