thisisnic commented on issue #28990: URL: https://github.com/apache/arrow/issues/28990#issuecomment-6711396901
Closing. The "straddling object" error is expected: a single JSON object can span at most one block boundary, so records larger than roughly twice `block_size` need a larger `block_size` (the request to grow it automatically was #25674). The crash and hang reported alongside that error no longer reproduce on pyarrow 26.0.1: reading the original `wiki_04.jsonl` with `block_size=2**20` using the script from the report raised the straddling `ArrowInvalid` error and exited normally in 60 out of 60 runs, with no abort or hang. If you can still trigger a crash on a current release, please open a new issue with a reprex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
