Re: Structured Streaming: How to handle bad input

2017-02-27 Thread sasubillis
I think it is users responsibility to validate the input before feeding. https://databricks.gitbooks.io/databricks-spark-knowledge-base/best_practices/dealing_with_bad_data.html -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-How

Re: Structured Streaming: How to handle bad input

2017-02-23 Thread Sam Elamin
gt; 2.apply(SparkPlan.scala:225) > at > org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$ > 1$$anonfun$apply$25.apply(RDD.scala:826) > at > org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$ > 1$$anonfun$apply$25.apply(RDD.scala:826) > at org.ap

Structured Streaming: How to handle bad input

2017-02-23 Thread JayeshLalwani
he-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-How-to-handle-bad-input-tp28420.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org