andygrove opened a new issue, #5102: URL: https://github.com/apache/datafusion-comet/issues/5102
### What is the problem the feature request solves? `RLike::is_match` in `native/spark-expr/src/predicate_funcs/rlike.rs` loops over a `StringArray` calling `regex::Regex::is_match` per row into a `BooleanBuilder`. Arrow's `regexp_is_match` kernel does the same thing: it is built on the same Rust `regex` crate, so the documented divergence from Java regex semantics is unchanged, and null propagation matches. The kernel would also make the array path more robust: it supports Utf8/LargeUtf8/Utf8View, while the current code hard-`expect`s a `StringArray` and would panic on other string layouts. ### Describe the potential solution Replace the row loop with `regexp_is_match(array, &StringArray::new_scalar(pattern), None)`. Trade-offs to resolve: - The struct currently pre-compiles the `Regex` once at plan build; the kernel takes a pattern string and compiles per batch. For complex patterns this is a measurable regression; there is no arrow API accepting a pre-built `Regex`. Either accept per-batch compilation or keep the loop and only fix the downcast robustness. - The dictionary fast path (run on dictionary values then `take`) and the scalar-input branch stay as wrappers. ### Additional context Found during an audit of native code that replicates existing arrow-rs kernels. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
