Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 @HyukjinKwon I made code changes based on your suggestions. I also changed the tests to use the data mentioned in the ticket. However, you're right, the tests no longer pass. But that is because the Univocity `CsvParser`, when it encounters an empty string while parsing the data, replaces it with the `nullValue` we set (see [setNullValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/common/CommonSettings.html#setNullValue(java.lang.String))). And the `emptyValue` is only effective when the _empty string_ being read has quotes around it (see [setEmptyValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/csv/CsvParserSettings.html#setEmptyValue(java.lang.String))). So I believe, at this point, the issue needs to be fixed in the underlying library being used.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org