raams95 opened a new pull request, #40563:
URL: https://github.com/apache/beam/pull/40563
Fixes #40562
`Regex.all_matches` built each output from `range(m.lastindex + 1)`.
`lastindex` is the index of the last group that matched, and with nested groups
it is the outer group that closed last. It is `None` when the regex has no
groups. Because of this:
```python
p | beam.Create(['ab']) | beam.Regex.all_matches(r'((a)(b))') # ['ab',
'ab'], groups 2 and 3 dropped
p | beam.Create(['a']) | beam.Regex.all_matches(r'(a)(b)?') # ['a',
'a'], group 2 dropped
p | beam.Create(['aaa']) | beam.Regex.all_matches(r'a+') # TypeError:
unsupported operand type(s) for +: 'NoneType' and 'int'
```
Each output is now `[m.group(), *m.groups()]`: the entire match, then one
entry per group of the regex, with `None` for a group that did not take part in
the match. This is what Java's `Regex.AllMatches` returns (it adds `null` for
such groups), and it is the same change #40399 made for `Regex.find_all(...,
Regex.ALL)`. Output for a regex whose groups all match, like the existing tests
and the `regex_all_matches` snippet, does not change.
With this change the pipelines above print `['ab', 'ab', 'a', 'b']`, `['a',
'a', None]` and `['aaa']`.
I added `test_all_matched_returns_every_group` to `util_test.py`. It runs
the three cases in a `TestPipeline`. It fails with the `TypeError` before the
change and passes after it. I also updated the docstring to describe the output.
Testing:
- `pytest apache_beam/transforms/util_test.py` (157 passed, 3 skipped)
- `pytest apache_beam/examples/snippets/transforms/elementwise/ -k regex`
- `yapf --diff` and `ruff check` on the changed files
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [x] Mention the appropriate issue in your description (for example:
`addresses #123` or `fixes #456`).
- [x] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). (Not
needed, small change.)
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
