FelixYBW commented on issue #6823:
URL:
https://github.com/apache/incubator-gluten/issues/6823#issuecomment-2308495039
vector_size_t is int32_t but numInputRows_, numOutputRows_ and maxOutputRows
is uint32_t. So when (numInputRows_ - numOutputRows_) is bigger than
0x7fffffff, error occurs.
```
const vector_size_t batchSize =
std::min<vector_size_t>(numInputRows_ - numOutputRows_, maxOutputRows);
```
https://github.com/facebookincubator/velox/blob/d33cdb2512052b1bc131908be0a7d26ad240d89d/velox/exec/SortBuffer.cpp#L291
Meanwhile we should also note that the totall row number in a partition
shouldn't exceeds UINT_MAX. Not sure if any other operator has the same issue
or not. We can control the batch size but can't control the partition size. So
we should use uint_64t data type for numInputRows_ and numOutputRows_. Not sure
any other operators have the limitation or not.
@jinchengchenghh can you take a look and fix this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
