alexeykudinkin commented on issue #25822: URL: https://github.com/apache/arrow/issues/25822#issuecomment-2846064600
> Solutions that don't concatenate tend to be slower and are considered unacceptable. @felipecrv i totally get that. My point is twofold however: 1. Doing take albeit slower still better than doing operation (concat) that's bound to fail (for arrays of cumulative size > 2 Gb) 2. We don't need fall for either extreme of either concatenating all arrays or concatenating none -- we can do concatenations at int32 boundary drastically reducing number of arrays if we've been passed in a lot of small ones while still being able to handle the taking w/o blowing up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
