mbutrovich commented on code in PR #12228:
URL: https://github.com/apache/arrow-rs/pull/12228#discussion_r4565263214


##########
arrow-array/src/builder/primitive_builder.rs:
##########
@@ -350,31 +349,16 @@ impl<T: ArrowPrimitiveType> PrimitiveBuilder<T> {
 
     /// Builds the [`PrimitiveArray`] and reset this builder.
     pub fn finish(&mut self) -> PrimitiveArray<T> {
-        let len = self.len();
         let nulls = self.null_buffer_builder.finish();

Review Comment:
   Could we drop a null buffer that has no nulls before building the array? 
That means two changes:
   
   1. Add `.filter(|n| n.null_count() > 0)` to each 
`null_buffer_builder.finish()` and `finish_cloned()` call this PR adds. Those 
are `primitive_builder.rs` lines 352 and 359, `boolean_builder.rs` lines 149 
and 156, `generic_bytes_builder.rs` lines 211 and 229, and 
`fixed_size_binary_builder.rs` lines 121 and 134. The suggestion below is for 
this line. `StructArray` does the same 
([struct_array.rs](https://github.com/apache/arrow-rs/blob/9c9486188744e80ec86c3b2bcdd649b89945ffe5/arrow-array/src/array/struct_array.rs#L173)).
   2. Add a test like the one below to each of the four builders' tests.
   
   ```suggestion
           let nulls = self.null_buffer_builder.finish().filter(|n| 
n.null_count() > 0);
   ```
   
   ```rust
   #[test]
   fn test_finish_materialized_without_nulls() {
       let mut builder = Int32Builder::new();
       builder.append_nulls(0);
       builder.append_value(1);
       assert!(builder.finish_cloned().nulls().is_none());
       assert!(builder.finish().nulls().is_none());
   }
   ```
   
   Without the filter, `nulls()` returns `Some` for an array with no nulls 
whenever the builder's null bitmap was allocated, for example by 
`append_nulls(0)` or by clearing a null through `validity_slice_mut`. 
`ArrayDataBuilder::build` used to drop that buffer 
([data.rs](https://github.com/apache/arrow-rs/blob/9c9486188744e80ec86c3b2bcdd649b89945ffe5/arrow-data/src/data.rs#L2465)),
 and the new constructors keep it. I ran the snippet above against all four 
builders, through both `finish` and `finish_cloned`. The merge base gives 
`None` every time and this PR gives `Some` with a null count of 0. The array is 
still valid, but kernels check `nulls()` to pick their no-null paths, so they'd 
lose them.
   
   `NullBufferBuilder::finish` should return `None` here itself, which is 
#12238. Once that's fixed, these filters can go.



##########
arrow-array/src/builder/generic_bytes_builder.rs:
##########
@@ -201,33 +202,34 @@ impl<T: ByteArrayType> GenericByteBuilder<T> {
 
     /// Builds the [`GenericByteArray`] and reset this builder.
     pub fn finish(&mut self) -> GenericByteArray<T> {
-        let array_type = T::DATA_TYPE;
-        let array_builder = ArrayDataBuilder::new(array_type)
-            .len(self.len())
-            .add_buffer(std::mem::take(&mut self.offsets_builder).into())
-            .add_buffer(std::mem::take(&mut self.value_builder).into())
-            .nulls(self.null_buffer_builder.finish());
+        // SAFETY: the builder maintains non-empty, monotonically increasing
+        // offsets and matching value/null lengths.

Review Comment:
   Could this `SAFETY` comment state the contract of 
`OffsetBuffer::new_unchecked`? That contract is non-empty, monotonically 
increasing offsets that are at least 0. Value and null lengths aren't part of 
it. The same text applies at lines 221-222 in `finish_cloned`.
   
   ```suggestion
           // SAFETY: offsets start at 0 and every append pushes the current 
end of
           // `value_builder`, so they are non-empty and monotonically 
increasing.
   ```



##########
arrow-array/src/builder/generic_bytes_builder.rs:
##########
@@ -201,33 +202,34 @@ impl<T: ByteArrayType> GenericByteBuilder<T> {
 
     /// Builds the [`GenericByteArray`] and reset this builder.
     pub fn finish(&mut self) -> GenericByteArray<T> {
-        let array_type = T::DATA_TYPE;
-        let array_builder = ArrayDataBuilder::new(array_type)
-            .len(self.len())
-            .add_buffer(std::mem::take(&mut self.offsets_builder).into())
-            .add_buffer(std::mem::take(&mut self.value_builder).into())
-            .nulls(self.null_buffer_builder.finish());
+        // SAFETY: the builder maintains non-empty, monotonically increasing
+        // offsets and matching value/null lengths.
+        let offsets = unsafe {
+            OffsetBuffer::new_unchecked(std::mem::take(&mut 
self.offsets_builder).into())
+        };
+        let values = std::mem::take(&mut self.value_builder).into();
+        let nulls = self.null_buffer_builder.finish();
 
-        self.offsets_builder.push(self.next_offset());
-        // SAFETY: builder is constructed from valid offset and value buffers 
maintained by the builder
-        let array_data = unsafe { array_builder.build_unchecked() };
-        GenericByteArray::from(array_data)
+        self.offsets_builder.push(T::Offset::usize_as(0));
+
+        // SAFETY: offsets and values were produced by this builder
+        unsafe { GenericByteArray::new_unchecked(offsets, values, nulls) }

Review Comment:
   Could this `SAFETY` comment say why `try_new` would not error, which is what 
`GenericByteArray::new_unchecked` requires? "Produced by this builder" doesn't 
cover UTF-8 for `StringBuilder`. The same text applies at line 231 in 
`finish_cloned`.
   
   ```suggestion
           // SAFETY: `try_new` would not error. The last offset is 
`values.len()`,
           // `nulls` has one entry per value, and for strings every value was
           // appended from a `&str`, so each offset pair bounds valid UTF-8.
           unsafe { GenericByteArray::new_unchecked(offsets, values, nulls) }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to