You actually should avoid setting SPARK_PUBLIC_DNS unless necessary, I thought you might have preemptively done so. I think the issue is actually related to your network configuration, as Spark probably failed to find your driver's ip address. Do you see a warning on the driver that looks something like "Your hostname, localhost resolves to a loopback address, but we couldn't find any external IP address"?
Either way, let's try to set SPARK_LOCAL_IP (see http://spark.apache.org/docs/latest/configuration.html) inside ~/spark/conf/spark-env.sh on your driver machine to an IP address that's reachable by your executors. Something like export SPARK_LOCAL_IP=194.168.1.105 You can make sure it was set correctly by running sc.getConf.get("spark.driver.host"), which should return the driver hostname, and NOT "localhost". (Note that it's also possible that your /etc/hosts file contains a mapping from the driver's ip address to localhost, which it should not.) On Tue, Jul 8, 2014 at 2:23 PM, Sameer Tilak <[email protected]> wrote: > Hi Aaron, > Would really appreciate your help if you can point me to the > documentation. Is this something that I need to do with /etc/hosts on each > of the worker machines ? Or do I set SPARK_PUBLIC_DNS (if yes, what is the > format?) or something else? > > I have the following set up: > > master node: pzxnvm2018.x.y.org > worker nodes: pzxnvm2022.x.y.org pzxnvm2023.x.y.org pzxnvm2024.x.y.org > > > From: [email protected] > Date: Tue, 8 Jul 2014 11:59:54 -0700 > > Subject: Re: CoarseGrainedExecutorBackend: Driver Disassociated > To: [email protected] > > > Hmm, looks like the Executor is trying to connect to the driver on > localhost, from this line: > 14/07/08 11:07:13 INFO CoarseGrainedExecutorBackend: Connecting to driver: > akka.tcp://spark@localhost:39701/user/CoarseGrainedScheduler > > What is your setup? Standalone mode with 4 separate machines? Are you > configuring the driver public dns name somewhere? > > > On Tue, Jul 8, 2014 at 11:52 AM, Sameer Tilak <[email protected]> wrote: > > Dear All, > > When I look inside the following directory on my worker node: > $SPARK_HOME/work/app-20140708110707-0001/3 > > I see the following error message: > > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.conf.Configuration). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > more info. > 14/07/08 11:07:11 INFO SparkHadoopUtil: Using Spark's default log4j > profile: org/apache/spark/log4j-defaults.properties > 14/07/08 11:07:11 INFO SecurityManager: Changing view acls to: p529444 > 14/07/08 11:07:11 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(p529444) > 14/07/08 11:07:12 INFO Slf4jLogger: Slf4jLogger started > 14/07/08 11:07:12 INFO Remoting: Starting remoting > 14/07/08 11:07:13 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://[email protected]:34679 > <http://pzxnvm2022.dcld.pldc.kp.org:34679>] > 14/07/08 11:07:13 INFO Remoting: Remoting now listens on addresses: > [akka.tcp://[email protected]:34679 > <http://pzxnvm2022.x.y.name.org:34679>] > 14/07/08 11:07:13 INFO CoarseGrainedExecutorBackend: Connecting to driver: > akka.tcp://spark@localhost:39701/user/CoarseGrainedScheduler > 14/07/08 11:07:13 INFO WorkerWatcher: Connecting to worker akka.tcp:// > [email protected]:37054/user/Worker > <http://pzxnvm2022.x.y.name.org:37054/user/Worker> > 14/07/08 11:07:13 ERROR CoarseGrainedExecutorBackend: Driver Disassociated > [akka.tcp://[email protected]:34679 > <http://pzxnvm2022.dcld.pldc.kp.org:34679>] -> [akka > > > I am not sure what the problem is but it is preventing me to get the 4 > node test cluster up and running. > > >
