mbrubeck edited a comment on pull request #9235: URL: https://github.com/apache/arrow/pull/9235#issuecomment-763846523
I was able to speed up the fast path of `extend_from_iter` by a few percent, by splitting the loop into two parts. This version is only about 8% slower than `extend_from_trusted_len_iter`. (Previously it was about 12% slower.) Both are still much slower than `extend_from_slice`. <details> ```rust #[inline] fn extend_from_iter<T: ArrowNativeType, I: Iterator<Item = T>>( &mut self, mut iterator: I, ) { let size = std::mem::size_of::<T>(); let (lower, _) = iterator.size_hint(); let additional = lower * size; self.reserve(additional); // this is necessary because of https://github.com/rust-lang/rust/issues/32155 let mut len = SetLenOnDrop::new(&mut self.len); let mut dst = unsafe { self.data.as_ptr().add(len.local_len) as *mut T }; let capacity = self.capacity; while len.local_len + size <= capacity { if let Some(item) = iterator.next() { unsafe { std::ptr::write(dst, item); dst = dst.add(1); } len.local_len += size; } else { break; } } drop(len); iterator.for_each(|item| self.push(item)); } ``` </details> I haven't benchmarked the slow path (the second loop), which I implemented using safe code. It's possible that it could be made faster with unsafe code, but I don't expect large gains there. My complete code including benchmarks is on this branch: https://github.com/apache/arrow/compare/master...mbrubeck:length_faster ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org