Can you give us the more detailed exception + stack trace in the log? It should be in the driver log. If not, please take a look at the executor logs, through the web ui to find the stack trace.
TD On Tue, Mar 25, 2014 at 10:43 PM, gaganbm <gagan.mis...@gmail.com> wrote: > Hi Folks, > > Is this issue resolved ? If yes, could you please throw some light on how > to > fix this ? > > I am facing the same problem during writing to text files. > > When I do > > stream.foreachRDD(rdd =>{ > rdd.saveAsTextFile(<"Some path">) > }) > > This works fine for me. But it creates multiple text files for each > partition within an RDD. > > So I tried with coalesce option to merge my results in a single file for > each RDD as : > > stream.foreachRDD(rdd =>{ > rdd.coalesce(1, > true).saveAsTextFile(<"Some path">) > }) > > This fails with : > org.apache.spark.SparkException: Job aborted: Task 75.0:0 failed 1 times > (most recent failure: Exception failure: java.lang.IllegalStateException: > unread block data) > > I am using Spark Streaming 0.9.0 > > Any clue what's going wrong when using coalesce ? > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/rdd-saveAsTextFile-problem-tp176p3238.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >