Hi there, I an trying to configure Spark for running on top of Mesos. But every time I send a job, it fails. I can see mesos downloading correctly the spark.tgz but I have this errors at the end :
Any idea ? I did not find anything for solving my issue.. Is it my cluster ? Spark ? both ? Thanking you in advance. Thibaut I0606 15:06:35.628329 16520 fetcher.cpp:456] Fetched 'http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz' to '/tmp/mesos/slaves/c58064f7-88b6-438d-b76f-fc28c6cc51a1-S0/frameworks/c58064f7-88b6-438d-b76f-fc28c6cc51a1-0079/executors/3/runs/23913146-d87f-445c-9f6b-f412ad2cbbd7/spark-1.5.1-bin-hadoop2.6.tgz' I0606 15:06:35.687414 16527 exec.cpp:143] Version: 0.28.1 I0606 15:06:35.691270 16540 exec.cpp:217] Executor registered on slave c58064f7-88b6-438d-b76f-fc28c6cc51a1-S0 Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 16/06/06 15:06:36 INFO CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT] 16/06/06 15:06:36 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/06/06 15:06:37 INFO SecurityManager: Changing view acls to: thibautg 16/06/06 15:06:37 INFO SecurityManager: Changing modify acls to: thibautg 16/06/06 15:06:37 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(thibautg); users with modify permissions: Set(thibautg) 16/06/06 15:06:37 INFO Slf4jLogger: Slf4jLogger started 16/06/06 15:06:38 INFO Remoting: Starting remoting 16/06/06 15:06:38 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher@141.213.4.119:56419] 16/06/06 15:06:38 INFO Utils: Successfully started service 'driverPropsFetcher' on port 56419. Exception in thread "main" akka.remote.RemoteTransportException: No transport is loaded for protocol: [spark], available protocols: [akka.tcp] at akka.remote.Remoting$.localAddressForRemote(Remoting.scala:87) at akka.remote.Remoting.localAddressForRemote(Remoting.scala:129) at akka.remote.RemoteActorRefProvider.rootGuardianAt(RemoteActorRefProvider.scala:338) at akka.actor.ActorRefFactory$class.actorSelection(ActorRefProvider.scala:318) at akka.actor.ActorSystem.actorSelection(ActorSystem.scala:272) at org.apache.spark.rpc.akka.AkkaRpcEnv.asyncSetupEndpointRefByURI(AkkaRpcEnv.scala:216) at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:98) at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:162) at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:69) at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:68) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:68) at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:149) at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:250) at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
