This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new 558c5fec9f perf(arrow-array): Reduce binary size 0.5%: do
`PrimitiveArray` → `ArrayData` conversion once, not per type (#10893)
558c5fec9f is described below
commit 558c5fec9ffa2ac813cb1311ee2fc72ed3d16a97
Author: Andrew Lamb <[email protected]>
AuthorDate: Sat Aug 29 09:22:50 2026 -0400
perf(arrow-array): Reduce binary size 0.5%: do `PrimitiveArray` →
`ArrayData` conversion once, not per type (#10893)
# Which issue does this PR close?
- Part of the binary-size / LLVM IR reduction effort started in #10889.
# Rationale for this change
`From<PrimitiveArray<T>> for ArrayData` (reached via `Array::to_data` /
`into_data`) is monomorphized for all ~32 primitive types in every crate
that calls it
# What changes are included in this PR?
Move the `ArrayDataBuilder` construction into a private non-generic
`primitive_to_array_data(data_type, len, values, nulls)` helper,
compiled once in `arrow-array`.
The same pattern applies to the byte/list array `From` impls (far fewer
instantiations each); those can be follow-ups if this approach is
agreeable.
## Measurements
I measured the size of the `cast_kernels` binary with
```shell
cargo bench --bench cast_kernels --no-run
```
To see the actual number of bytes saved (60K / 0.45%):
| before | after | delta |
|---:|---:|---:|
| 13,213,184 bytes | 13,153,152 bytes | **−60,032 bytes (−0.45%)** |
I also measured the generated LLVM IR in a representative consumer crate
with
```shell
cargo llvm-lines --release -p arrow-row --lib
```
| | before | after | delta |
|---|---:|---:|---:|
| `From<PrimitiveArray<T>> for ArrayData` IR in arrow-row | 4,992 lines
/ 32 copies | 1,952 lines / 32 thin shims | **−61%** |
The other consumer crates (arrow-cast, arrow-ord, arrow-select, parquet,
...) have near-identical numbers, so the ~3,000-line saving repeats in
each of them.
# Are these changes tested?
Covered by existing tests (`to_data`/`into_data` round-trips are
exercised throughout the test suite).
# Are there any user-facing changes?
No. The helper is private and the produced `ArrayData` is identical.
---
arrow-array/src/array/primitive_array.rs | 29 +++++++++++++++++++++++------
1 file changed, 23 insertions(+), 6 deletions(-)
diff --git a/arrow-array/src/array/primitive_array.rs
b/arrow-array/src/array/primitive_array.rs
index c0d000e7b7..30d8f95f80 100644
--- a/arrow-array/src/array/primitive_array.rs
+++ b/arrow-array/src/array/primitive_array.rs
@@ -1207,12 +1207,29 @@ impl<T: ArrowPrimitiveType> PrimitiveArray<T> {
impl<T: ArrowPrimitiveType> From<PrimitiveArray<T>> for ArrayData {
fn from(array: PrimitiveArray<T>) -> Self {
- let builder = ArrayDataBuilder::new(array.data_type)
- .len(array.values.len())
- .nulls(array.nulls)
- .buffers(vec![array.values.into_inner()]);
-
- unsafe { builder.build_unchecked() }
+ /// Converts the parts of a `PrimitiveArray` into [`ArrayData`].
+ /// Use an inner function to avoid code duplication over all
+ /// generic callsites as the body is the same.
+ ///
+ /// # Safety
+ /// The parts must come from a valid `PrimitiveArray`
+ unsafe fn inner(
+ data_type: DataType,
+ len: usize,
+ values: Buffer,
+ nulls: Option<NullBuffer>,
+ ) -> ArrayData {
+ let builder = ArrayDataBuilder::new(data_type)
+ .len(len)
+ .nulls(nulls)
+ .buffers(vec![values]);
+
+ // SAFETY: arguments are valid, per contract
+ unsafe { builder.build_unchecked() }
+ }
+ let len = array.values.len();
+ // SAFETY: the parts come from a valid PrimitiveArray
+ unsafe { inner(array.data_type, len, array.values.into_inner(),
array.nulls) }
}
}
