Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22234#discussion_r214572625
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -91,9 +91,10 @@ abstract class CSVDataSource extends Serializable {
}
row.zipWithIndex.map { case (value, index) =>
- if (value == null || value.isEmpty || value == options.nullValue) {
- // When there are empty strings or the values set in
`nullValue`, put the
- // index as the suffix.
+ if (value == null || value.isEmpty || value == options.nullValue ||
+ value == options.emptyValueInRead) {
--- End diff --
ditto for excluding.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
