Re: [PR] [Parquet] perf: Create PrimitiveArrays directly rather than via `ArrayData` [arrow-rs]

2026-01-09 Thread via GitHub


alamb-ghbot commented on PR #9122:
URL: https://github.com/apache/arrow-rs/pull/9122#issuecomment-3730447632

   🤖 `./gh_compare_arrow.sh` 
[gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh)
 Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 
2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/less_primitive_allocations 
(4148a3d53f0f18a760fd573aae65a9b104ef9da1) to 
96637fc8b928a94de53bbec3501337c0ecfbf936 
[diff](https://github.com/apache/arrow-rs/compare/96637fc8b928a94de53bbec3501337c0ecfbf936..4148a3d53f0f18a760fd573aae65a9b104ef9da1)
   BENCH_NAME=arrow_reader
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental 
--bench arrow_reader 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_less_primitive_allocations
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] [Parquet] perf: Create PrimitiveArrays directly rather than via `ArrayData` [arrow-rs]

2026-01-09 Thread via GitHub


alamb commented on PR #9122:
URL: https://github.com/apache/arrow-rs/pull/9122#issuecomment-3729412464

   
   run benchmark arrow_reader arrow_reader_clickbench
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



Re: [PR] [Parquet] perf: Create PrimitiveArrays directly rather than via `ArrayData` [arrow-rs]

2026-01-09 Thread via GitHub


alamb commented on code in PR #9122:
URL: https://github.com/apache/arrow-rs/pull/9122#discussion_r2676635354


##
parquet/src/arrow/array_reader/primitive_array.rs:
##
@@ -212,41 +211,69 @@ where
 .consume_record_data()
 .into_buffer(target_type);
 
-let array_data = ArrayDataBuilder::new(arrow_data_type)
-.len(self.record_reader.num_values())
-.add_buffer(record_data)
-.null_bit_buffer(self.record_reader.consume_bitmap_buffer());
+let len = self.record_reader.num_values();
+let nulls = self
+.record_reader
+.consume_bitmap_buffer()
+.map(|b| NullBuffer::new(BooleanBuffer::new(b, 0, len)));
 
-let array_data = unsafe { array_data.build_unchecked() };

Review Comment:
   the point is to avoid this ArrayData as we know here what types of arrays 
will be built



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]