Curious about this too... I'll be moving Zeppelin off to its own box in the near future. If you figure this out, post your resolution here.
-- Chris Miller On Sat, Mar 26, 2016 at 12:54 AM, Marcin Pilarczyk < marcin.pilarc...@interia.pl> wrote: > Guys, > > I'm trying to switch my zeppelin instance (0.6 snapshot) from the spark > instance installed on very same machine onto ec2 created spark. Both > versions of spark are 1.5.2. > > I've just created a test instance in EC2, I can submit jobs or use spark > shell. I have revised logs each and every worker up and running, master > alive. So far so good. > > The next step is to switch the zeppelin on the newly created spark. I'm > changing two places: zeppelin-env.sh and URL in the interpreter settings. > I'm SURE these settings are pointing the new instance. > > Next step. I'm stopping the spark instance installed together with > zeppelin. > > Final step, zeppelin is restarted, settings are checked. Somehow any > paragraph that requires computations can't be completed. Master logs are > ok, in the slave log however I can find a following error: > > 16/03/25 12:42:25 INFO Remoting: Starting remoting > 16/03/25 12:42:25 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://driverPropsFetcher@172.31.40.27:36098] > 16/03/25 12:42:25 INFO util.Utils: Successfully started service > 'driverPropsFetcher' on port 36098. > 16/03/25 12:43:28 WARN Remoting: Tried to associate with unreachable > remote address [akka.tcp://sparkDriver@172.31.41.186:46358]. Address is > now gated for 5000 ms, all messages to this address will be delivered to > dead letters.$ > Exception in thread "main" akka.actor.ActorNotFound: Actor not found for: > ActorSelection[Anchor(akka.tcp://sparkDriver@172.31.41.186:46358/), > Path(/user/CoarseGrainedScheduler)] > at > akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65) > at > akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63) > at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) > at > akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) > at > akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74) > at > akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110) > at > akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73) > at > scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40) > at > scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248) > at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267) > at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:508) > at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:541) > at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:531) > at > akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87) > at akka.remote.EndpointWriter.postStop(Endpoint.scala:561) > at akka.actor.Actor$class.aroundPostStop(Actor.scala:475) > at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:415) > at > akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210) > at > akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172) > at akka.actor.ActorCell.terminate(ActorCell.scala:369) > at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462) > at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478) > at > akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263) > at akka.dispatch.Mailbox.run(Mailbox.scala:219) > at > akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393) > at > scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > > 172.31.41.186 -> that's the address where zeppelin is running and previous > spark WAS running, In the zeppelin configuration there is no trace of this > IP. Please note again, spark shell and submit on the new master node are > executing their jobs. > > Is there yet another place I need to change? Or generally, is there any > problem with running zeppelin with remote/external spark? > > Regards, > Marcin >
