[
https://issues.apache.org/jira/browse/FLINK-14266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940767#comment-16940767
]
Fabian Hueske commented on FLINK-14266:
---------------------------------------
This is a lot more tricky than it seems.
The difference between reading CSV records from a file and a Kafka topic is
that a Kafka topic provides already individual records that just need to be
parsed.
When reading from a file, you need to split the records first. The standard
record delimiter is a new line, however, new lines can also be included in
escaped text fields. Something that makes it even more complicated is that
splitting large files into multiple splits cannot be easily done, because you
do not know whether the next new-line character is a record delimiter or
contained in a string field.
> Introduce RowCsvInputFormat to new CSV module
> ---------------------------------------------
>
> Key: FLINK-14266
> URL: https://issues.apache.org/jira/browse/FLINK-14266
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / FileSystem
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
> Fix For: 1.10.0
>
>
> Now, we have an old CSV, but that is not standard CSV support. we should
> support the RFC-compliant CSV format for table/sql.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
