airborne12 opened a new pull request, #66861:
URL: https://github.com/apache/doris/pull/66861
### What problem does this PR solve?
Issue Number: None
Related PR: #66052
Problem Summary:
SNII evaluated wildcard question marks byte by byte, so `a?b` could not
match `a猫b`, while V3 converts `?` to a RE2 dot and matches one Unicode code
point. The SNII matcher now validates UTF-8 and advances its existing reusable
dynamic-programming rows only at code-point boundaries. This preserves the
bounded two-buffer scratch design, makes invalid patterns return
`INVALID_ARGUMENT`, and aligns SNII query results with V3 without changing the
writer or stored bytes.
### Release note
SNII wildcard question marks now match one UTF-8 code point, consistent with
V3.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. `?` now consumes exactly one UTF-8 code point. This is
query-only; the storage format and write path are unchanged, so existing SNII
indexes are upgrade-compatible.
- Does this need documentation?
- [x] No.
- [ ] Yes. (add document PR link here)
- Release notes must contain the following three elements:
- [x] Problem Summary
- [x] User-visible Changes
- [x] Compatibility (storage format, upgrade, downgrade, rolling
upgrade, etc.)
### Tests
- ASAN BE unit tests: `SniiWildcardQueryTest.*:SniiPatternQuery.*` (22/22
passed)
- Regression:
`inverted_index_p0/storage_format/test_storage_format_snii_utf8_wildcard`
- Full BE build: `./build.sh --be -j 192`
### Reviewer Checklist
- [ ] The title is of the form `type(scope): [Issue #xxx] summary`
- [ ] The PR has a detailed problem description
- [ ] The PR has a detailed solution description
- [ ] The PR has a detailed test plan
- [ ] The PR has a detailed compatibility description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
