On Mon, Mar 23, 2015 at 2:15 PM, Manoj Samel manojsamelt...@gmail.com wrote:
Found the issue above error - the setting for spark_shuffle was incomplete.
Now it is able to ask and get additional executors. The issue is once they
are released, it is not able to proceed with next query.
That
Log shows stack traces that seem to match the assert in JIRA so it seems I
am hitting the issue. Thanks for the heads up ...
15/03/23 20:29:50 ERROR actor.OneForOneStrategy: assertion failed:
Allocator killed more executors than are allocated!
java.lang.AssertionError: assertion failed: Allocator
Found the issue above error - the setting for spark_shuffle was incomplete.
Now it is able to ask and get additional executors. The issue is once they
are released, it is not able to proceed with next query.
The environment is CDH 5.3.2 (Hadoop 2.5) with Kerberos Spark 1.3
After idle time, the
bq. Requesting 1 new executor(s) because tasks are backlogged
1 executor was requested.
Which hadoop release are you using ?
Can you check resource manager log to see if there is some clue ?
Thanks
On Fri, Mar 20, 2015 at 4:17 PM, Manoj Samel manojsamelt...@gmail.com
wrote:
Forgot to add -
Hi,
Running Spark 1.3 with secured Hadoop.
Spark-shell with Yarn client mode runs without issue when not using Dynamic
Allocation.
When Dynamic allocation is turned on, the shell comes up but same SQL etc.
causes it to loop.
spark.dynamicAllocation.enabled=true
Forgot to add - the cluster is idle otherwise so there should be no
resource issues. Also the configuration works when not using Dynamic
allocation.
On Fri, Mar 20, 2015 at 4:15 PM, Manoj Samel manojsamelt...@gmail.com
wrote:
Hi,
Running Spark 1.3 with secured Hadoop.
Spark-shell with Yarn
