: Connection refused: slave2/...:54845
Could you look in the executor logs (stderr on slave2) and see what made
it shut down? Since you are doing a join there's a high possibility of OOM
etc.
Thanks
Best Regards
On Wed, Jul 1, 2015 at 10:20 AM, Pooja Jain pooja.ja...@gmail.com wrote:
Hi,
We
Hi,
We are using Spark 1.4.0 on hadoop using yarn-cluster mode via
spark-submit. We are facing parquet write issue after doing dataframe joins
We have a full data set and then an incremental data. We are reading them
as dataframes, joining them, and then writing the data to the hdfs system
in
