andygrove opened a new pull request, #4876: URL: https://github.com/apache/datafusion-comet/pull/4876
> ⚠️ **Autonomously generated by an LLM.** This PR was created end-to-end by an automated agent with no human author. It has passed automated correctness (unit tests + differential fuzz) and micro-benchmark gates, but **requires full human review**. Do not merge without expert scrutiny. ## What changed Optimizes `spark_unhex` in the native `spark-expr` crate. Replace unhex's per-digit range-match with a compile-time 256-entry hex lookup table and preallocate the BinaryBuilder to its known output size, cutting per-byte branching and repeated buffer reallocations. ## Evidence - Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs `main`). - Benchmark (criterion, Rust-only): see numbers below; gate required ≥ 5.0% mean improvement with non-overlapping confidence intervals and no regressions. ``` spark_unhex_ valid mixed hex: 19.964% faster (base 125189ns -> cand 100197ns) spark_unhex_ mixed hex column: 13.986% faster (base 109637ns -> cand 94303ns) spark_unhex_ all valid: 42.128% faster (base 201069ns -> cand 116362ns) spark_unhex_ with nulls: 36.465% faster (base 179663ns -> cand 114150ns) spark_unhex_ valid hex: 22.177% faster (base 152680ns -> cand 118821ns) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
