Hi Patrick,
The fix you need is SPARK-6954: https://github.com/apache/spark/pull/5704.
If possible, you may cherry-pick the following commit into your Spark
deployment and it should resolve the issue:
https://github.com/apache/spark/commit/98ac39d2f5828fbdad8c9a4e563ad1169e3b9948
Note that this
Hi Patrick,
I'm noticing that you're using Spark 1.3.1. We fixed a bug in dynamic
allocation in 1.4 that permitted requesting negative numbers of executors.
Any chance you'd be able to try with the newer version and see if the
problem persists?
-Sandy
On Fri, Jun 12, 2015 at 7:42 PM, Patrick
Hey Sandy,
I'll test it out on 1.4. Do you have a bug number or PR that I could reference
as well?
Thanks!
-Pat
Sent from my iPhone
On Jun 13, 2015, at 11:38 AM, Sandy Ryza sandy.r...@cloudera.com wrote:
Hi Patrick,
I'm noticing that you're using Spark 1.3.1. We fixed a bug in dynamic
Hey all,
I've recently run into an issue where spark dynamicAllocation has asked for
-1 executors from YARN. Unfortunately, this raises an exception that kills
the executor-allocation thread and the application can't request more
resources.
Has anyone seen this before? It is spurious and the
