This is an automated email from the ASF dual-hosted git repository.
Jefffrey pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-rs.git
The following commit(s) were added to refs/heads/main by this push:
new 433e0617a9 Expose builder buffer capacity accessors (#10342)
433e0617a9 is described below
commit 433e0617a9886eb85fb28afdc45527e051e26bc3
Author: Alex Huang <[email protected]>
AuthorDate: Fri Jul 17 14:38:33 2026 +0800
Expose builder buffer capacity accessors (#10342)
# Which issue does this PR close?
- Closes #10341.
# Rationale for this change
Downstream memory accounting sometimes needs to measure the allocated
capacity currently retained by Arrow array builders, including
allocation slack after buffer growth. Existing slice accessors expose
initialized data, but not the underlying values, offsets, or
validity-buffer capacity.
# What changes are included in this PR?
This adds read-only capacity accessors for existing builder internals:
- `GenericByteBuilder::{values_capacity, offsets_capacity,
validity_capacity}`
- `GenericListBuilder::{offsets_capacity, validity_capacity}`
- `PrimitiveBuilder::validity_capacity`
- `StructBuilder::validity_capacity`
The methods only read existing builder state and do not change append,
reserve, finish, or resize behavior.
# Are these changes tested?
No behavior-specific tests are added because these methods only expose
existing capacity fields through read-only forwarding accessors.
Existing arrow-array builder tests were run unchanged, and the new
public API is compile-checked by the crate.
Validated with:
- `cargo fmt --all -- --check`
- `cargo test -p arrow-array builder --lib`
- `cargo test -p arrow-array --lib`
- `cargo +1.96.0 clippy -p arrow-array --all-targets --no-deps -- -D
warnings`
# Are there any user-facing changes?
Yes. This adds public read-only capacity accessors on Arrow array
builders.
Co-authored-by: flarion-weijun <[email protected]>
---
arrow-array/src/builder/generic_bytes_builder.rs | 15 +++++++++++++++
arrow-array/src/builder/generic_list_builder.rs | 10 ++++++++++
arrow-array/src/builder/primitive_builder.rs | 5 +++++
arrow-array/src/builder/struct_builder.rs | 5 +++++
4 files changed, 35 insertions(+)
diff --git a/arrow-array/src/builder/generic_bytes_builder.rs
b/arrow-array/src/builder/generic_bytes_builder.rs
index 0a83ff989d..a2d044e808 100644
--- a/arrow-array/src/builder/generic_bytes_builder.rs
+++ b/arrow-array/src/builder/generic_bytes_builder.rs
@@ -231,16 +231,31 @@ impl<T: ByteArrayType> GenericByteBuilder<T> {
self.value_builder.as_slice()
}
+ /// Returns the current values buffer capacity, in bytes.
+ pub fn values_capacity(&self) -> usize {
+ self.value_builder.capacity()
+ }
+
/// Returns the current offsets buffer as a slice
pub fn offsets_slice(&self) -> &[T::Offset] {
self.offsets_builder.as_slice()
}
+ /// Returns the current offsets buffer capacity, in offsets.
+ pub fn offsets_capacity(&self) -> usize {
+ self.offsets_builder.capacity()
+ }
+
/// Returns the current null buffer as a slice
pub fn validity_slice(&self) -> Option<&[u8]> {
self.null_buffer_builder.as_slice()
}
+ /// Returns the current null buffer allocated capacity, in bytes.
+ pub fn validity_capacity(&self) -> usize {
+ self.null_buffer_builder.allocated_size()
+ }
+
/// Returns the current null buffer as a mutable slice
pub fn validity_slice_mut(&mut self) -> Option<&mut [u8]> {
self.null_buffer_builder.as_slice_mut()
diff --git a/arrow-array/src/builder/generic_list_builder.rs
b/arrow-array/src/builder/generic_list_builder.rs
index 2f130baae9..f93625e8eb 100644
--- a/arrow-array/src/builder/generic_list_builder.rs
+++ b/arrow-array/src/builder/generic_list_builder.rs
@@ -355,10 +355,20 @@ where
self.offsets_builder.as_slice()
}
+ /// Returns the current offsets buffer capacity, in offsets.
+ pub fn offsets_capacity(&self) -> usize {
+ self.offsets_builder.capacity()
+ }
+
/// Returns the current null buffer as a slice
pub fn validity_slice(&self) -> Option<&[u8]> {
self.null_buffer_builder.as_slice()
}
+
+ /// Returns the current null buffer allocated capacity, in bytes.
+ pub fn validity_capacity(&self) -> usize {
+ self.null_buffer_builder.allocated_size()
+ }
}
impl<O, B, V, E> Extend<Option<V>> for GenericListBuilder<O, B>
diff --git a/arrow-array/src/builder/primitive_builder.rs
b/arrow-array/src/builder/primitive_builder.rs
index dc96486bca..311ad8b100 100644
--- a/arrow-array/src/builder/primitive_builder.rs
+++ b/arrow-array/src/builder/primitive_builder.rs
@@ -361,6 +361,11 @@ impl<T: ArrowPrimitiveType> PrimitiveBuilder<T> {
self.null_buffer_builder.as_slice()
}
+ /// Returns the current null buffer allocated capacity, in bytes.
+ pub fn validity_capacity(&self) -> usize {
+ self.null_buffer_builder.allocated_size()
+ }
+
/// Returns the current null buffer as a mutable slice
pub fn validity_slice_mut(&mut self) -> Option<&mut [u8]> {
self.null_buffer_builder.as_slice_mut()
diff --git a/arrow-array/src/builder/struct_builder.rs
b/arrow-array/src/builder/struct_builder.rs
index d299f7bf9d..3d773864f5 100644
--- a/arrow-array/src/builder/struct_builder.rs
+++ b/arrow-array/src/builder/struct_builder.rs
@@ -314,6 +314,11 @@ impl StructBuilder {
pub fn validity_slice(&self) -> Option<&[u8]> {
self.null_buffer_builder.as_slice()
}
+
+ /// Returns the current null buffer allocated capacity, in bytes.
+ pub fn validity_capacity(&self) -> usize {
+ self.null_buffer_builder.allocated_size()
+ }
}
#[cfg(test)]
