Jefffrey commented on PR #10324:
URL: https://github.com/apache/arrow-rs/pull/10324#issuecomment-4983790430
the only other ways i can think of is either a reflection based approach,
e.g.
```rust
use std::any::TypeId;
if TypeId::of::<E>() == TypeId::of::<base64::engine::GeneralPurpose>() {
// Safety: Base64 is valid UTF-8
unsafe {
GenericStringArray::new_unchecked(
offsets,
Buffer::from_vec(buffer),
array.nulls().cloned(),
)
}
} else {
GenericStringArray::try_new(offsets, Buffer::from_vec(buffer),
array.nulls().cloned())
}
```
or for something more flexible, a trait based approach like
```rust
/// Implement this on an engine that guarantees it produces valid utf8
pub unsafe trait SafeBase64 {}
unsafe impl SafeBase64 for base64::engine::GeneralPurpose {}
/// Bas64 encode each element of `array` with the provided [`Engine`]
pub fn b64_encode<E: Engine + SafeBase64, O: OffsetSizeTrait>(
```
frankly i dont know how common this base64 code is used downstream anyway (i
cant find any usages in datafusion) so maybe we dont need to be overkill and
can just force validation 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
