surawut-jirasaktavee opened a new pull request, #59422:
URL: https://github.com/apache/spark/pull/59422
### What changes were proposed in this pull request?
`regexp_instr(str, regexp, idx)` ignored its `idx` argument and always
returned the position of the entire match. This PR makes it return the position
of the regex group `idx` within the first match, consistent with
`regexp_extract`:
- Add `RegExpExtractBase.instr`, shared by the interpreted and codegen paths
of `RegExpInStr` (the same pattern as `extract` / `extractAll`). It uses
`Matcher.start(idx)` instead of `Matcher.start()`.
- Validate `idx` with the existing `RegExpExtractBase.checkGroupIndex`, so a
negative `idx` or an `idx` greater than the number of groups raises
`INVALID_PARAMETER_VALUE.REGEX_GROUP_INDEX`, as `regexp_extract` and
`regexp_extract_all` do.
- Return 0 when the selected group does not take part in the match (for
example an optional group), the same value as "no match".
- Update the `regexp_instr` docs (SQL usage and example, Scala
`functions.scala`, the PySpark docstring example) and add a migration guide
entry.
### Why are the changes needed?
The Scala and Python APIs document `idx` as the "matched group id", but the
value was never used:
```sql
SELECT regexp_instr('abc123', '([a-z]+)([0-9]+)', 2);
-- before: 1 (start of the entire match)
-- after: 4 (start of group 2, '123')
```
### Does this PR introduce _any_ user-facing change?
Yes, this changes released behavior.
- `regexp_instr(str, regexp, idx)` with `idx >= 1` now returns the position
of group `idx`. Previously it returned the position of the entire match. For
example, `regexp_instr('1a 2b 14m', '\\d+(a|b|m)', 1)` returns 2 instead of 1.
- An `idx` that is negative or greater than the number of groups now raises
`INVALID_PARAMETER_VALUE.REGEX_GROUP_INDEX` instead of returning the position
of the entire match.
- `idx = 0` and the two-argument form are unchanged.
The change is documented in `docs/sql-migration-guide.md`. I did not add a
legacy config because the new behavior matches the documented contract and
`regexp_extract`; I can add one if reviewers prefer.
### How was this patch tested?
- New test `SPARK-60125: regexp_instr returns the position of the selected
group` in `RegexpExpressionsSuite`. It covers groups 0, 1 and 2, an optional
group that does not take part in the match, an out-of-range `idx` when there is
no match, supplementary characters, and the error for a negative, too-large, or
zero-group `idx`. `checkEvaluation` runs both the interpreted and the codegen
paths. The test failed before the fix (`actual: 1, expected: 4`).
- Updated `StringFunctionsSuite` ("string regexp_instr"), which asserted the
old behavior.
- New queries in `regexp-functions.sql`, with regenerated golden files.
- Updated the PySpark doctest for `regexp_instr` (Example 2).
- Ran: `RegexpExpressionsSuite`, `CollationRegexpExpressionsSuite`,
`StringFunctionsSuite`, `ExpressionInfoSuite`, `ExpressionsSchemaSuite`, and
`SQLQueryTestSuite -z regexp-functions.sql`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
