tustvold commented on code in PR #2740:
URL: https://github.com/apache/arrow-rs/pull/2740#discussion_r973608338
##########
arrow/src/compute/kernels/arithmetic.rs:
##########
@@ -78,6 +80,24 @@ where
Ok(binary(left, right, op))
}
+/// This is similar to `math_op` as it performs given operation between two
input primitive arrays.
+/// But the given operation can return `Err` if overflow is detected. For the
case, this function
+/// returns an `Err`.
+fn math_checked_op<LT, RT, F>(
Review Comment:
Is it not just a matter of providing the necessary type hint?
##########
arrow/src/compute/kernels/arithmetic.rs:
##########
@@ -522,67 +542,78 @@ macro_rules! typed_dict_math_op {
}};
}
-/// Helper function to perform math lambda function on values from two
dictionary arrays, this
-/// version does not attempt to use SIMD explicitly (though the compiler may
auto vectorize)
-macro_rules! math_dict_op {
- ($left: expr, $right:expr, $op:expr, $value_ty:ty) => {{
- if $left.len() != $right.len() {
- return Err(ArrowError::ComputeError(format!(
- "Cannot perform operation on arrays of different length ({},
{})",
- $left.len(),
- $right.len()
- )));
- }
+/// Perform given operation on two `DictionaryArray`s.
+/// Returns an error if the two arrays have different value type
Review Comment:
Fair enough, the compile times are currently despair inducing and I have a
chip on my shoulder about them 😅
##########
arrow/src/compute/kernels/arity.rs:
##########
@@ -298,36 +297,41 @@ where
let len = a.len();
if a.null_count() == 0 && b.null_count() == 0 {
- let values = a.values().iter().zip(b.values()).map(|(l, r)| op(*l,
*r));
- let buffer = unsafe { Buffer::try_from_trusted_len_iter(values) }?;
- // JUSTIFICATION
- // Benefit
- // ~75% speedup
- // Soundness
- // `values` is an iterator with a known size from a PrimitiveArray
- return Ok(unsafe { build_primitive_array(len, buffer, 0, None) });
+ let mut buffer = BufferBuilder::<O::Native>::new(len);
+ buffer.append_n_zeroed(len);
Review Comment:
It would be faster to reserve the correct capacity and use push_unchecked.
Avoids zero allocating.
As written I suspect this is a performance regression
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
