On Wed, 11 Mar 2015 11:19:56 +0100 Marcin Cylke <[email protected]> wrote:
> Hi > > I'm trying to do a join of two datasets: 800GB with ~50MB. The job finishes if I set spark.yarn.executor.memoryOverhead to 2048MB. If it is around 1000MB it fails with "executor lost" errors. My spark settings are: - executor cores - 8 - num executors - 32 - executor memory - 4g Regards Marcin --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
