zhuqi-lucas commented on code in PR #7873:
URL: https://github.com/apache/arrow-rs/pull/7873#discussion_r2189757800


##########
arrow-array/src/array/byte_view_array.rs:
##########
@@ -473,13 +473,81 @@ impl<T: ByteViewType + ?Sized> GenericByteViewArray<T> {
     /// Note: this function does not attempt to canonicalize / deduplicate 
values. For this
     /// feature see  [`GenericByteViewBuilder::with_deduplicate_strings`].
     pub fn gc(&self) -> Self {
-        let mut builder = 
GenericByteViewBuilder::<T>::with_capacity(self.len());
+        // 1) Read basic properties once
+        let len = self.len(); // number of elements
+        let views = self.views(); // raw u128 “view” values per slot
+        let nulls = self.nulls().cloned(); // reuse & clone existing null 
bitmap
+
+        // 2) Pre-scan to determine how many out‑of‑line bytes we must store
+        let total_large: usize = (0..len)
+            .filter_map(|i| {
+                // skip null entries
+                if self.is_null(i) {
+                    None
+                } else {
+                    // extract the ByteView metadata from the u128
+                    let raw_view: u128 = views[i];
+                    let bv = ByteView::from(raw_view);
+                    // if length exceeds inline limit, count it
+                    (bv.length > MAX_INLINE_VIEW_LEN).then_some(bv.length as 
usize)
+                }
+            })
+            .sum();
+        // allocate exactly the capacity needed for all non‑inline data
+        let mut data_buf = Vec::with_capacity(total_large);
+
+        // 3) Build new view array in one pass
+        //    map each index to a new u128, collecting into a Vec<u128>
+        let views_buf: Vec<u128> = (0..len)
+            .map(|i| {
+                // if null, represent as 0
+                if self.is_null(i) {

Review Comment:
   Thank you @Dandandan, i try to use set_indices for nulls, but the benchmark 
not got improvement, i checked the benchmark, i found that our benchmark will 
always has nulls, so we should add not null benchmark first, i will submit a PR 
for not null benchmark, thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to