shivbhatia10 opened a new issue, #18822:
URL: https://github.com/apache/datafusion/issues/18822
### Describe the bug
When `ideal_batch_size` is set to a value that doesn't evenly divide the
total number of rows in some data for an `AsyncScalarUDFImpl`, we get this
error:
`Error: Internal("Arguments has mixed length. Expected length: 2, found
length: 1")`
for example, the numbers vary depending on both parameters.
### To Reproduce
I wrote a test in the fix PR which fails today:
https://github.com/apache/datafusion/pull/18819/files
### Expected behavior
Batch size shouldn't necessarily need to divide the number of rows evenly.
### Additional context
I have a fix [PR](https://github.com/apache/datafusion/pull/18819), the
issue was in the `AsyncFuncExpr` physical expression. We were calling
`ColumnarValue::values_to_arrays` on all the batches returned asynchronously,
but this method enforces that all batches need to be the same length, which
isn't necessary here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
