GitHub user justinuang opened a pull request:
https://github.com/apache/spark/pull/22503
[SPARK-25493] [SQL] Fix multiline crlf
## What changes were proposed in this pull request?
CSVs with windows style crlf (carriage return line feed) don't work in
multiline mode. They work fine in single line mode because the line separation
is done by Hadoop, which can handle all the different types of line separators.
This fixes it by enabling Univocity's line separator detection.
## How was this patch tested?
Unit test with a file with crlf line endings.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/justinuang/spark fix-clrf-multiline
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22503.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22503
----
commit 5ce9de9f789ce108f6afb65e38bab44acc77a4e8
Author: Justin Uang <juang@...>
Date: 2018-09-20T20:41:35Z
Fix multiline crlf
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
