Seems like you are having memory issues. Examine your settings.
1. It appears that your driver memory setting is too high. It should
be a fraction of total memy provided by YARN
2. Use the Spark UI to monitor the job's memory consumption. Check the
Storage tab to see how memory is
Hi I'm new to Spark and I'm running into a lot of OOM issues while trying
to scale up my first Spark application. I am running into these issues with
only 1% of the final expected data size. Can anyone help me understand how
to properly configure Spark to use limited memory or how to debug which
