AdityRajGupta opened a new pull request, #54059: URL: https://github.com/apache/arrow/pull/54059
### Rationale for this change With newlines_in_values=true, a \r\n inside a quoted field can be split across two read blocks. When \r is the last byte of one block and \n is the first byte of the next, the reader drops the \n and corrupts the field value. See #51368. ### What changes are included in this PR? CSVBufferIterator now keeps a trailing \r and adds it to the next buffer instead of dropping the following \n. This keeps CRLF together across block boundaries. A pending \r at EOF is also preserved. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. **This PR contains a "Critical Fix"**: it fixes (b) a bug that caused incorrect data to be produced. ### Was AI used for this PR? Yes AI was used for assistance during the development process mainly in understanding the issue. <img width="980" height="488" alt="gh51368-new-tests" src="https://github.com/user-attachments/assets/2f2b8c4c-a8f3-4cfb-9d4d-dc0327df67da"; /> <img width="980" height="664" alt="gh51368-full-suite" src="https://github.com/user-attachments/assets/43fcdd67-3a73-4828-9c2b-e691261201e9"; /> Added regression tests for the serial, async, and streaming readers. These particular test cases were failing before the fix. All 3 now pass with the fix. **PR code and description written by:** - [x] Human - [x] AI **Reviewed before submission by:** - [x] Human - [x] AI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
