lgbo-ustc opened a new pull request, #13608:
URL: https://github.com/apache/gluten/pull/13608

   ## What changes are proposed in this pull request?
   
   Fixes #13607.
   
   The ClickHouse backend currently applies Spark/Java-to-RE2 adjustments only 
to `regexp_extract`; other regex functions can interpret the same pattern 
differently. Introduce a shared `spark_regexp::re2::translatePattern()` with 
typed top-level nodes and recursively parsed character classes, and use it for 
constant patterns in `regexp_extract`, `regexp_extract_all`, `regexp_replace`, 
`rlike`, and `split`. Preserve capture indices, translate the supported nested 
class/anchor/dot subset and scoped DOTALL flags, validate the result with RE2, 
and add CH/Spark comparison and C++ tests. Fix lack of forward progress on 
zero-width matches in `regexp_extract_all`.
   
   The replacement-string `$n` conversion remains separate in the existing 
Scala transformer. Unsupported constructs are conservatively filtered at the 
Scala level where recognized and rejected by the native translator otherwise.
   
   **Draft: known limitations / review focus**
   - Full ClickHouse native build and integration suite have not been run 
locally (required CH build/test environment unavailable).
   - Java `$` before a final CRLF has a zero-width boundary that the current 
RE2 lowering does not represent exactly; this and matching semantics for 
zero-width `regexp_extract_all` need CH/Spark differential tests before this is 
ready for review.
   - The Scala prefilter is heuristic, not a parser, so unsupported Java 
patterns can still pass planning and be rejected by the native translator 
instead of falling back to Spark. A robust fallback strategy is needed before 
marking ready.
   - Java constructs such as lookaround, backreferences, and character-class 
intersection are not generally lowered.
   
   ## How was this patch tested?
   
   - Added C++ translation tests and Spark/CH SQL comparison tests.
   - Compiled and ran the translation assertions against a locally available 
RE2 library with lightweight CH exception and gtest shims; **this is not a 
ClickHouse native build**.
   - `./dev/format-scala-code.sh check` (with a writable local Maven 
repository) passed, as did `git diff --check`.
   - Native CH compilation and the Spark/ClickHouse integration tests remain 
pending. Header checker could not run locally because Python `regex` is missing.
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: DeepSeek Harness gpt-6-sol
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to