jorisvandenbossche commented on pull request #6979: URL: https://github.com/apache/arrow/pull/6979#issuecomment-754691714
> The strange thing is that this seems to have another logic of how many rows are included in each batch when crossing row groups, while in the end it is also using `GetRecordBatchReader` with `batch_size` set in the reader properties: Ah, that's because the RecordBatchReader gets constructed *for each* row group, so that way never crossing row group boundaries ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
