Arawoof06 opened a new pull request, #50505:
URL: https://github.com/apache/arrow/pull/50505
### Rationale for this change
`upper()`, `lower()` and `initcap()` decode a multibyte glyph with
`arrow::util::UTF8Decode` in `gdv_string_function_stubs.cc` without checking
that the whole glyph fits inside `data_len`. When a string ends in a truncated
multibyte sequence (a trailing `0xC3` or `0xE0` lead byte), `UTF8Decode` walks
continuation bytes past `data[data_len]`. The input is a string array value
buffer, so a packed, exactly-sized buffer makes that read land past the
allocation. The three functions share the same decode loop, so all three are
affected.
ASAN on an exact-sized input buffer:
```
heap-buffer-overflow READ of size 1 ... 0 bytes after 4-byte region
in arrow::util::UTF8Decode (utf8_internal.h:353)
in gdv_fn_upper_utf8 (gdv_string_function_stubs.cc:361)
```
### What changes are included in this PR?
Reject a glyph whose declared length runs past `data_len` before decoding
it, in `gdv_fn_lower_utf8`, `gdv_fn_upper_utf8` and `gdv_fn_initcap_utf8`. This
takes the same invalid-utf8 error path the functions already use for a bad lead
byte, so valid input is handled exactly as before.
One existing initcap test also passed length 19 for a 20-byte literal, which
cut off the final `ɦ` glyph. That only stayed hidden because a string literal
keeps a spare trailing byte; the length is corrected to 20.
### Are these changes tested?
Yes. New cases in `TestUpper`, `TestLower` and `TestInitCap` feed an
exact-sized `new char[]` buffer ending in a truncated lead byte. They fail
under ASAN on the current code (heap-buffer-overflow in `UTF8Decode`) and pass
with the fix.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
