exmy opened a new issue, #13593:
URL: https://github.com/apache/gluten/issues/13593
### Backend
CH (ClickHouse)
### Bug description
### Backend
CH (ClickHouse)
### Bug description
**[Expected behavior]** `regexp_extract` applies the Spark regex as written.
When the pattern contains an escaped bracket inside a character class, valid
input rows return the captured group:
```sql
CREATE TABLE t USING parquet AS SELECT * FROM VALUES
('[com.example.demo.AlphaController viewDidAppear:]'),
('[com.example.demo.BetaViewController]') AS v(top_vc);
SELECT regexp_extract(top_vc, '\\[(?:.*\\.)?([^\\.\\s\\]]+)', 1) FROM t;
-- expected
-- AlphaController
-- BetaViewController
```
**[Actual behavior]** Gluten returns an empty string for every row:
```text
-- actual
--
--
```
There is no error and no fallback, so the wrong value propagates silently.
In our workload the extracted column is then used by `WHERE top_vc IS NOT NULL
AND top_vc != ''`, which filtered out **all** rows of one join input and the
query wrote an empty table instead of ~1.8k rows. It also corrupts projections:
a `CASE WHEN top_vc LIKE ...` expression falls through to its `ELSE` branch for
every row.
The same happens with a simpler pattern:
```sql
SELECT regexp_extract('[123-456]', '\\[([\\d\\-]+)\\]', 1);
-- expected: 123-456
-- actual: (empty)
```
**Cause.** `FunctionParserRegexpExtract::adjustSparkRegexpRule()` (added by
#5908 for #5897) walks the pattern to normalize Java's nested character class
syntax, but it treats every `[` / `]` as a class boundary and does not skip
`\X` escapes. For `\[(?:.*\.)?([^\.\s\]]+)` it takes the `[` of `\[` as a class
start and the `[` of `[^\.\s\]]` as a nested class, so the `]` of `\]` closes
the inner "class" and the real class end `]` closes the outer one:
| stage | pattern |
|---|---|
| from Spark | `\[(?:.*\.)?([^\.\s\]]+)` |
| after `adaptPatternForRE2()` (`(?-s)` prefix) |
`(?-s)\[(?:.*\.)?([^\.\s\]]+)` |
| after `adjustSparkRegexpRule()` (what actually reaches RE2) |
`(?-s)\[(?:.*\.)?(^\.\s\]+)` |
The character class disappears (`[^\.\s\]]` becomes `^` + `\.` + `\s` + `\]`
+ `+`) and `^` turns into a line anchor, so the group can never match and RE2's
`extract()` returns an empty string. This is data independent: every input
returns empty for such patterns.
**Suggested fix.** Skip `\X` as an atomic unit while scanning brackets:
```diff
while (!buf.eof())
{
+ // Keep "\x" as an atomic unit, otherwise escaped brackets such
as "\[" and "\]"
+ // (especially "\]" inside a character class) are mistaken for
class boundaries.
+ if (*buf.position() == '\\')
+ {
+ strs.top() += *buf.position();
+ ++buf.position();
+ if (!buf.eof())
+ {
+ strs.top() += *buf.position();
+ ++buf.position();
+ }
+ continue;
+ }
+
if (*buf.position() == '[')
```
This leaves the existing GLUTEN-5897 regression cases untouched —
`regexp_extract(a, '([0-9][[\\.][0-9]]*)', 1)` and `regexp_extract(a,
'([0-9][[]]]*)', 1)` produce exactly the same rewritten pattern before and
after — while restoring the escaped-bracket patterns. I have verified the fix
on our side (UT + a local-mode SQL run); I can send a PR with the change and a
UT in `GlutenClickhouseStringFunctionsSuite` if this direction looks right.
### Gluten version
main branch
### Spark version
Spark-3.3.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
### Gluten version
_No response_
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]