it might be a network issue. The error states failed to bind the server IP address
Chester Sent from my iPhone > On Jul 18, 2015, at 11:46 AM, Amjad ALSHABANI <ashshab...@gmail.com> wrote: > > Does anybody have any idea about the error I m having.. I am really > clueless... And appreciate any idea :) > > Thanks in advance > > Amjad > >> On Jul 17, 2015 5:37 PM, "Amjad ALSHABANI" <ashshab...@gmail.com> wrote: >> Hello, >> >> First of all I m a newbie in Spark , >> >> I m trying to start the spark-shell with yarn cluster by running: >> >> $ spark-shell --master yarn-client >> >> Sometimes it goes well, but most of the time I got an error: >> >> Container exited with a non-zero exit code 10 >> Failing this attempt. Failing the application. >> ApplicationMaster host: N/A >> ApplicationMaster RPC port: -1 >> queue: default >> start time: 1437145851944 >> final status: FAILED >> tracking URL: >> http://My-HadoopServer:50080/cluster/app/application_1437033338028_0030 >> user: hadoop >> org.apache.spark.SparkException: Yarn application has already ended! It >> might have been killed or unable to launch application master. >> at >> org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:115) >> .... >> .... >> .... >> >> searching in the yarn logs I got this log >> >> $ yarn logs -applicationId application_1437033338028_0030 >> 2015-07-17 17:11:03,961 - INFO >> [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$3@74] >> - Starting remoting >> 2015-07-17 17:11:04,200 - ERROR >> [sparkYarnAM-akka.actor.default-dispatcher-4:Slf4jLogger$$anonfun$receive$1$$anonfun$applyOrElse$1@65] >> - failed to bind to My-HadoopServer/10.98.105.11:0, shutting down Netty >> transport >> 2015-07-17 17:11:04,210 - WARN [main:Logging$class@71] - Service >> 'sparkYarnAM' could not bind on port 0. Attempting port 1. >> ... >> ... >> ... >> 2015-07-17 17:11:05,123 - ERROR [main:Logging$class@96] - Uncaught exception: >> java.net.BindException: Failed to bind to: My-HadoopServer/HadoopServerIP:0: >> Service 'sparkYarnAM' failed after 16 retries! >> at >> org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272) >> at >> akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393) >> at >> akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389) >> at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) >> ... >> .... >> .... >> >> >> >> I m using Spark 1.3, Hadoop 2.6 , >> >> and in spark-env.sh it points to my hadoop configuration: >> >> export HADOOP_CONF_DIR=/usr/hdp/2.2.4.4-16/hadoop/conf >> >> >> Is this probleme coming from spark configuration or yarn configuration (or >> spark with yarn confs)???? >> >> Any Ideas?? >> >> >> >> Amjad
