Try adding the following configurations also, might work.
spark.rdd.compress true
spark.storage.memoryFraction 1
spark.core.connection.ack.wait.timeout 600
spark.akka.frameSize 50
Thanks
Best Regards
On Mon, Nov 10, 2014 at 6:51 PM, Ritesh Kumar Singh <
riteshoneinamill...@gmail.com> wrote:
> Hi,
>
> I am trying to submit my application using spark-submit, using following
> spark-default.conf params:
>
> spark.master spark://<master-ip>:7077
> spark.eventLog.enabled true
> spark.serializer org.apache.spark.serializer.KryoSerializer
> spark.executor.extraJavaOptions -XX:+PrintGCDetails -Dkey=value
> -Dnumbers="one two three"
>
> ===============================================================
> But every time I am getting this error:
>
> 14/11/10 18:39:17 ERROR TaskSchedulerImpl: Lost executor 1 on aa.local:
> remote Akka client disassociated
> 14/11/10 18:39:17 WARN TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:17 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:20 ERROR TaskSchedulerImpl: Lost executor 2 on aa.local:
> remote Akka client disassociated
> 14/11/10 18:39:20 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:20 WARN TaskSetManager: Lost task 1.1 in stage 0.0 (TID 3,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:26 ERROR TaskSchedulerImpl: Lost executor 4 on aa.local:
> remote Akka client disassociated
> 14/11/10 18:39:26 WARN TaskSetManager: Lost task 0.2 in stage 0.0 (TID 5,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:26 WARN TaskSetManager: Lost task 1.2 in stage 0.0 (TID 4,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:29 ERROR TaskSchedulerImpl: Lost executor 5 on aa.local:
> remote Akka client disassociated
> 14/11/10 18:39:29 WARN TaskSetManager: Lost task 0.3 in stage 0.0 (TID 7,
> aa.local): ExecutorLostFailure (executor lost)
> 14/11/10 18:39:29 ERROR TaskSetManager: Task 0 in stage 0.0 failed 4
> times; aborting job
> 14/11/10 18:39:29 WARN TaskSetManager: Lost task 1.3 in stage 0.0 (TID 6,
> aa.local): ExecutorLostFailure (executor lost)
> Exception in thread "main" org.apache.spark.SparkException: Job aborted
> due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent
> failure: Lost task 0.3 in stage 0.0 (TID 7, gonephishing.local):
> ExecutorLostFailure (executor lost)
> Driver stacktrace:
> at org.apache.spark.scheduler.DAGScheduler.org
> $apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
> at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
> at scala.Option.foreach(Option.scala:236)
> at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1391)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
> at akka.actor.ActorCell.invoke(ActorCell.scala:456)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
> at akka.dispatch.Mailbox.run(Mailbox.scala:219)
> at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>
> =================================================================
> Any fixes?
>
