Zouxxyy opened a new pull request, #10474:
URL: https://github.com/apache/paimon/pull/10474
### Purpose
CSV parsing currently interprets quotes and backslashes inside unquoted
fields as CSV syntax. This corrupts JSON embedded in TSV, such as
`{"text":"a\nb","empty":""}`, and drops literal backslashes from paths.
Restrict CSV quote/escape handling to fields that start with a quote, and
preserve unrecognized escape sequences in quoted fields. Reuse the existing
per-column quoted state so quoted null literals and empty strings retain their
current behavior.
### Tests
Added 21 direct parser regression cases covering unquoted JSON and other
text in each column, leading whitespace, quoted escapes, custom/disabled quote
and escape characters, empty fields, projection, and parser reuse.
Validated with Java 8:
```sh
mvn -pl paimon-format -am -Pspark3 \
-DfailIfNoTests=false -DwildcardSuites=none \
-Dtest=CsvParserTest,CsvFileFormatTest,CsvCompressionTest test
```
- `CsvParserTest`: 21 passed.
- `CsvFileFormatTest`: 31 passed, 1 existing skipped case.
- `CsvCompressionTest`: 2 passed, 2 existing skipped cases.
Spotless formatting and the Maven checks also passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
