Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/21631
> So you mean it's a bug in Univocity?
No, I mean we don't handle `null` from Univocity's `parseLine` at all (in
another situations), and we just propagate `NullPointerException` to an user
instead of producing of `BadRecordException`. The `BadRecordException` can be
handled differently there
https://github.com/apache/spark/blob/1a527bde49753535e6b86c18751f50c19a55f0d0/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FailureSafeParser.scala#L59-L73
depend on current mode. In `DropMalformedMode` mode, for example, we could
just drop the line instead of propagating NPE to user space.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
