adamreeve commented on issue #47027: URL: https://github.com/apache/arrow/issues/47027#issuecomment-3047462967
This seems to be triggered by writing batches aligned with whole lists, which isn't handled by the logic for finding a record boundary in `DoInBatches`: https://github.com/apache/arrow/blob/b712c1e2ca3ff0eec328cdd7a7db11238baefb7e/cpp/src/parquet/column_writer.cc#L1175-L1209 It looks like this will only look for record boundaries beyond `batch_size` levels. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
