taepper opened a new issue, #51470: URL: https://github.com/apache/arrow/issues/51470
### Describe the enhancement requested `simdjson` has a multi-stage parser that first finds structural elements in the entire buffer (including document boundaries). Only the second stage, which is lazy, actually need to parse the documents. Similarily `parse_many` already computes all document boundaries and we can iterate its output to consume the documents without parsing any documents: https://github.com/simdjson/simdjson/blob/6913ee1a17a57d23a2e9f458d8e04e96cd43ecab/doc/parse_many.md?plain=1#L85-L88 ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
