I think it is users responsibility to validate the input before feeding.
https://databricks.gitbooks.io/databricks-spark-knowledge-base/best_practices/dealing_with_bad_data.html
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-How
gt; 2.apply(SparkPlan.scala:225)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$
> 1$$anonfun$apply$25.apply(RDD.scala:826)
> at
> org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$
> 1$$anonfun$apply$25.apply(RDD.scala:826)
> at org.ap
he-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-How-to-handle-bad-input-tp28420.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org