Hi Aaron,Would really appreciate your help if you can point me to the documentation. Is this something that I need to do with /etc/hosts on each of the worker machines ? Or do I set SPARK_PUBLIC_DNS (if yes, what is the format?) or something else? I have the following set up: master node: pzxnvm2018.x.y.orgworker nodes: pzxnvm2022.x.y.org pzxnvm2023.x.y.org pzxnvm2024.x.y.org
From: ilike...@gmail.com Date: Tue, 8 Jul 2014 11:59:54 -0700 Subject: Re: CoarseGrainedExecutorBackend: Driver Disassociated To: user@spark.apache.org Hmm, looks like the Executor is trying to connect to the driver on localhost, from this line:14/07/08 11:07:13 INFO CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://spark@localhost:39701/user/CoarseGrainedScheduler What is your setup? Standalone mode with 4 separate machines? Are you configuring the driver public dns name somewhere? On Tue, Jul 8, 2014 at 11:52 AM, Sameer Tilak <ssti...@live.com> wrote: Dear All, When I look inside the following directory on my worker node:$SPARK_HOME/work/app-20140708110707-0001/3 I see the following error message: log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration).log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. 14/07/08 11:07:11 INFO SparkHadoopUtil: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties14/07/08 11:07:11 INFO SecurityManager: Changing view acls to: p529444 14/07/08 11:07:11 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(p529444) 14/07/08 11:07:12 INFO Slf4jLogger: Slf4jLogger started14/07/08 11:07:12 INFO Remoting: Starting remoting14/07/08 11:07:13 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkexecu...@pzxnvm2022.dcld.pldc.kp.org:34679] 14/07/08 11:07:13 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkexecu...@pzxnvm2022.x.y.name.org:34679] 14/07/08 11:07:13 INFO CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://spark@localhost:39701/user/CoarseGrainedScheduler14/07/08 11:07:13 INFO WorkerWatcher: Connecting to worker akka.tcp://sparkwor...@pzxnvm2022.x.y.name.org:37054/user/Worker 14/07/08 11:07:13 ERROR CoarseGrainedExecutorBackend: Driver Disassociated [akka.tcp://sparkexecu...@pzxnvm2022.dcld.pldc.kp.org:34679] -> [akka I am not sure what the problem is but it is preventing me to get the 4 node test cluster up and running.
