thorfour opened a new issue, #10891:
URL: https://github.com/apache/arrow-rs/issues/10891

   ### Is your feature request related to a problem or challenge?
   
   **Is your feature request related to a problem or challenge? Please describe 
what you are trying to do.**
   
   `arrow-arith`'s elementwise numeric kernels don't handle 
`DataType::RunEndEncoded`. The type dispatch lives in 
[`arithmetic_op`](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L220),
 called from the public 
[`add`](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L34)/[`add_wrapping`](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L39)/[`sub`](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L44)/[`sub_wrapping`](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L49)
 entry points. It has no `RunEndEncoded` match arm, so a 
`RunEndEncoded`-wrapped operand falls through to the generic [fallback error 
arm](https://github.com/apache/arrow-rs/blob/58.4.0/arrow-arith/src/numeric.rs#L259)
 and returns `ArrowError::InvalidArgumentError`, e.g.:
   
       Invalid arithmetic operation: RunEndEncoded(...) + Int64
   
   We hit this  in [Apache 
DataFusion](https://github.com/apache/datafusion/pull/24565): a `RANGE <offset> 
PRECEDING/FOLLOWING` window frame ordered by an REE-encoded column needs to 
add/subtract the frame offset from each row's ordering value. 
   
   **Describe the solution you'd like**
   
   Add `RunEndEncoded` support to arithmetic_op
   
   **Describe alternatives you've considered**
   
   - Decode REE to a flat array before arithmetic: works today as a caller-side 
workaround, but throws away REE's compression benefit for exactly the workloads 
(e.g. windowed aggregates over repetitive columns) that make REE worth using in 
the first place.
   - Leave callers to detect and reject REE before reaching arithmetic: what 
DataFusion does today (see linked PR) — safe, but means REE columns can't be 
used in RANGE window frames or similar arithmetic-dependent operations at all 
until this lands.
   
   **Additional context**
   
   Related prior art: #3520 (REE epic — still has an open "Support REE in 
compute kernels" checklist item), #9620 (REE support for `arrow-ord` comparison 
kernels, landed).
   
   ### Describe the solution you'd like
   
   _No response_
   
   ### Describe alternatives you've considered
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to