2010YOUY01 commented on issue #24072: URL: https://github.com/apache/datafusion/issues/24072#issuecomment-5174102468
To solve this issue in aggregate, we need https://github.com/apache/datafusion/issues/7065 However (at least in the near term), we can't guarantee to prevent similar issue in other operators that buffer and concat input batches: `BianryArray` will overflow if it's > 2GB, there are other places like hash join might trigger the similar bug. I think we could let it return a better error message, and suggest user to use `LargeBinary` type instead. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
