david-mollitor-db opened a new pull request, #58354:
URL: https://github.com/apache/spark/pull/58354
### What changes were proposed in this pull request?
`UTF8String.contains` scans for the needle byte-by-byte. For a **single-byte
needle** --
the common case produced by `LIKE '%x%'` (which the optimizer rewrites to
`Contains`) with
an ASCII character -- this PR adds a word-at-a-time (SWAR / memchr-style)
scan.
- New `ByteArrayMethods.containsByte(Object base, long offset, long length,
byte target)`:
broadcasts the target byte to all 8 lanes of a word, then for each 8-byte
word XORs and
applies the classic exact "a word contains a zero byte" test
`(w - 0x0101010101010101L) & ~w & 0x8080808080808080L`. It reports only
existence, not
position, so it is endianness independent. Alignment handling mirrors
`arrayEquals`.
- `UTF8String.contains` takes a `numBytes == 1` fast path delegating to it,
which also drops
the redundant per-position `matchAt` call.
A JDK SIMD intrinsic (`String.indexOf` / `ArraysSupport.vectorizedMismatch`)
would be faster
still, but those require an on-heap `byte[]`; `UTF8String`'s backing memory
may be off-heap or
a slice of a larger buffer, so a word-at-a-time scan over `(base, offset)`
is the portable
improvement (the same reason `arrayEquals` is hand-rolled rather than using
`Arrays.equals`).
### Why are the changes needed?
`makeInPredicate`/`Contains` single-character predicates are common
(`'%,%'`, `'%@%'`,
`'% %'`, `'%/%'`). The current byte-at-a-time scan reads through
`Platform.getByte` (Unsafe),
which the JIT cannot auto-vectorize, so it processes one byte per iteration.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- `UTF8StringSuite.contains` gains single-byte cases (first byte, past the
first word, last
byte, absent, sub-word length); the property-based
`UTF8StringPropertyCheckSuite` also
passes.
- A standalone Unsafe-based microbenchmark (faithfully modeling `Platform`
reads) over the
"needle absent" full-scan case shows ~2-3x over the byte-at-a-time scan
across 16 B - 64 KB,
and 500k randomized inputs match a brute-force reference. Official
`UTF8String` benchmark
results on the consistent runners to be added.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
