ETS2K7 opened a new pull request, #27291:
URL: https://github.com/apache/datafusion/pull/27291
## Which issue does this PR close?
- Closes #27277.
## Rationale for this change
Calling `ScalarValue::to_array_of_size` on `Utf8`, `LargeUtf8`, `Binary`, or
`LargeBinary` scalars panics when `value.len() * size` overflows the array
offset type (`i32::MAX` for `Utf8`/`Binary`, `i64::MAX` for
`LargeUtf8`/`LargeBinary`) or `usize`, because `GenericByteArray::new_repeated`
invokes `OffsetBuffer::from_repeated_length`, which panics on overflow. Queries
such as `SELECT repeat('a', 1048576) AS s FROM generate_series(1, 2048)` crash
instead of returning an error.
## What changes are included in this PR?
- Extract `try_repeated_byte_array<T: ByteArrayType>` in
`datafusion/common/src/scalar/mod.rs` and route the `Utf8`, `LargeUtf8`,
`Binary`, and `LargeBinary` arms in `ScalarValue::to_array_of_size` through it.
- Guard against `usize` multiplication overflow, `T::Offset` overflow
(returning `ArrowError::OffsetOverflowError`), and `isize::MAX` address-space
limits (for `Large*` on 32-bit targets) before calling
`GenericByteArray::new_repeated`.
- Document the new overflow error condition in
`ScalarValue::to_array_of_size`'s `# Errors` section.
## What is the testing strategy for this PR?
- Added unit tests in `datafusion/common/src/scalar/mod.rs`
(`test_to_array_of_size_byte_offset_overflow`) covering `i32::MAX` overflow
(`Utf8`, `Binary`), `usize` multiplication overflow, `i64::MAX` overflow
(`LargeUtf8`, `LargeBinary` on 64-bit), and valid empty/zero-size/non-overflow
cases.
- Added an end-to-end `sqllogictest` query in
`datafusion/sqllogictest/test_files/string/string_literal.slt` executing the
SQL reproduction (`SELECT repeat('a', 1048576) AS s FROM generate_series(1,
2048);`).
- Verified pre-fix behavior on `main` (`9e0a08fd8`): confirmed it panics at
`arrow-buffer`'s `OffsetBuffer::from_repeated_length` with `offset overflow:
2147483648 does not fit in i32`, while with this patch it returns `DataFusion
error: Arrow error: Offset overflow error: 2147483648`.
## Are there any user-facing changes?
Queries that previously panicked on offset overflow in
`ScalarValue::to_array_of_size` now fail gracefully with an
`ArrowError::OffsetOverflowError` or `DataFusionError::Execution` error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
