Does anybody have any idea about the error I m having.. I am really clueless... And appreciate any idea :)
Thanks in advance Amjad On Jul 17, 2015 5:37 PM, "Amjad ALSHABANI" <ashshab...@gmail.com> wrote: > Hello, > > First of all I m a newbie in Spark , > > I m trying to start the spark-shell with yarn cluster by running: > > $ spark-shell --master yarn-client > > Sometimes it goes well, but most of the time I got an error: > > Container exited with a non-zero exit code 10 > Failing this attempt. Failing the application. > ApplicationMaster host: N/A > ApplicationMaster RPC port: -1 > queue: default > start time: 1437145851944 > final status: FAILED > tracking URL: > http://My-HadoopServer:50080/cluster/app/application_1437033338028_0030 > user: hadoop > org.apache.spark.SparkException: Yarn application has already ended! It > might have been killed or unable to launch application master. > at > org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:115) > .... > .... > .... > > searching in the yarn logs I got this log > > $ yarn logs -applicationId application_1437033338028_0030 > 2015-07-17 17:11:03,961 - INFO > [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$3@74] > - Starting remoting > 2015-07-17 17:11:04,200 - ERROR > [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$1@65] > - failed to bind to My-HadoopServer/10.98.105.11:0, shutting down Netty > transport > 2015-07-17 17:11:04,210 - WARN [main:Logging$class@71] - Service > 'sparkYarnAM' could not bind on port 0. Attempting port 1. > ... > ... > ... > 2015-07-17 17:11:05,123 - ERROR [main:Logging$class@96] - Uncaught > exception: > java.net.BindException: Failed to bind to: > My-HadoopServer/HadoopServerIP:0: Service 'sparkYarnAM' failed after 16 > retries! > at > org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272) > at > akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393) > at > akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389) > at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) > ... > .... > .... > > > > I m using Spark 1.3, Hadoop 2.6 , > > and in spark-env.sh it points to my hadoop configuration: > > export HADOOP_CONF_DIR=/usr/hdp/2.2.4.4-16/hadoop/conf > > > Is this probleme coming from spark configuration or yarn configuration (or > spark with yarn confs)???? > > Any Ideas?? > > > > Amjad > >
