I have the same problem! I start the same job 3 or 4 times again, it depends how big the data and the cluster are. The runtime goes down in the following jobs. And at the end I get the Fetch failure error and at this point I must restart the spark shell and everything works well again. And I don't use the caching option!
By the way, I have the same behavior with different jobs! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-run-slow-after-unexpected-repartition-tp14542p15416.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
