Hi, I have a big dataframe 100giga that I need to join with 3 others dataframes.
For the first join, it's ok For the second, it's ok But for the third, just after the big shuffle, before the execution of the stage, I have an exception org.apache.spark.shuffle.FetchFailedException: java.io.FileNotFoundException: /DEVD/data/fs1/hadoop/yarn/log/usercache/xxxx/appcache/application_1459412930417_0309/blockmgr-ba9d4936-1de0-4205-9913-a579b520ab1f/1a/shuffle_5_65_0.index (No such file or directory) Any idea ?? Thank you -- M'BAREK Med Nihed, Fedora Ambassador, TUNISIA, Northern Africa http://www.nihed.com <http://tn.linkedin.com/in/nihed>
