Re: Pre-build Spark for Windows 8.1

2013-12-06 Thread Matei Zaharia 
 
 o.e.j.s.h.ContextHandler{/json,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{*,null}
 13/12/05 08:18:46 INFO server.AbstractConnector: Started 
 SelectChannelConnector@0.0.0.0:8081
 13/12/05 08:18:46 INFO ui.WorkerWebUI: Started Worker web UI at 
 http://ADRIBONA-DEV-1:8081
 13/12/05 08:18:46 INFO worker.Worker: Connecting to master 
 spark://adribona-dev-1:7077
  
  
 --Adrian
  
 From: Adrian Bonar [mailto:adrian.bo...@microsoft.com] 
 Sent: Thursday, December 5, 2013 7:49 AM
 To: user@spark.incubator.apache.org
 Subject: RE: Pre-build Spark for Windows 8.1
  
 Excellent! Thank you, Matei.
  
 From: Matei Zaharia [mailto:matei.zaha...@gmail.com] 
 Sent: Wednesday, December 4, 2013 4:26 PM
 To: user@spark.incubator.apache.org
 Subject: Re: Pre-build Spark for Windows 8.1
  
 Hey Adrian,
  
 Ideally you shouldn’t use Cygwin to run on Windows — use the .cmd scripts we 
 provide instead. Cygwin might be made to work but we haven’t tried to do 
 this so far so it’s not supported. If you can fix it, that would of course 
 be welcome.
  
 Also, the deploy scripts don’t work on Windows — we assumed that people 
 would mostly use Windows for local development and testing. However, if you 
 do want to launch a cluster, you can use spark-class.cmd manually to launch 
 a Master and then Workers. Use the commands in the first part of 
 http://spark.incubator.apache.org/docs/latest/spark-standalone.html.
  
 Matei
  
 On Dec 4, 2013, at 2:31 PM, Adrian Bonar adrian.bo...@microsoft.com wrote:
  
 
 Separate from my previous thread about building a distribution of Spark on 
 Win8, I am also trying to run the pre-build binaries with little success. I 
 downloaded and extract “spark-0.8.0-incubating-bin-hadoop1” to d:\spark and 
 attempted to start a master with the following error:
  
 $ sh bin/start-master.sh
 starting org.apache.spark.deploy.master.Master, logging to 
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
 failed to launch org.apache.spark.deploy.master.Master:
  
   Error: Could not find or load main class 
 org.apache.spark.deploy.master.Master
 full log in 
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
  
 --Adrian (again. J)

RE: Pre-build Spark for Windows 8.1

2013-12-05 Thread Adrian Bonar 
Excellent! Thank you, Matei.

From: Matei Zaharia [mailto:matei.zaha...@gmail.com]
Sent: Wednesday, December 4, 2013 4:26 PM
To: user@spark.incubator.apache.org
Subject: Re: Pre-build Spark for Windows 8.1

Hey Adrian,

Ideally you shouldn't use Cygwin to run on Windows - use the .cmd scripts we 
provide instead. Cygwin might be made to work but we haven't tried to do this 
so far so it's not supported. If you can fix it, that would of course be 
welcome.

Also, the deploy scripts don't work on Windows - we assumed that people would 
mostly use Windows for local development and testing. However, if you do want 
to launch a cluster, you can use spark-class.cmd manually to launch a Master 
and then Workers. Use the commands in the first part of 
http://spark.incubator.apache.org/docs/latest/spark-standalone.html.

Matei

On Dec 4, 2013, at 2:31 PM, Adrian Bonar 
adrian.bo...@microsoft.commailto:adrian.bo...@microsoft.com wrote:


Separate from my previous thread about building a distribution of Spark on 
Win8, I am also trying to run the pre-build binaries with little success. I 
downloaded and extract spark-0.8.0-incubating-bin-hadoop1 to d:\spark and 
attempted to start a master with the following error:

$ sh bin/start-master.sh
starting org.apache.spark.deploy.master.Master, logging to 
/cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
failed to launch org.apache.spark.deploy.master.Master:

  Error: Could not find or load main class org.apache.spark.deploy.master.Master
full log in 
/cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out

--Adrian (again. :))

RE: Pre-build Spark for Windows 8.1

2013-12-05 Thread Adrian Bonar 
The master starts up now as expected but the workers are unable to connect to 
the master. It looks like the master is refusing the connection messages but 
I'm not sure why. The first two error lines below are from trying to connect a 
worker from a separate machine and the last two error lines are from trying to 
connect a worker on the same machine as the master. I verified that the workers 
do not show up in the master's web ui.

MASTER:
D:\sparkspark-class org.apache.spark.deploy.master.Master
13/12/05 08:08:34 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/12/05 08:08:34 INFO master.Master: Starting Spark master at 
spark://ADRIBONA-DEV-1:7077
13/12/05 08:08:34 INFO server.Server: jetty-7.x.y-SNAPSHOT
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/master/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/applications/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/static,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/app/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/app,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{*,null}
13/12/05 08:08:34 INFO server.AbstractConnector: Started 
SelectChannelConnector@0.0.0.0:8088
13/12/05 08:08:34 INFO ui.MasterWebUI: Started Master web UI at 
http://ADRIBONA-DEV-1:8088
13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
RegisterWorker(worker-20131205080912-ADRIBONA-SFC-1-19280,ADRIBONA-SFC-1,19280,2,512,8081,ADRIBONA-SFC-1)
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-SFC-1:19280/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Master])
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
RegisterWorker(worker-20131205081846-ADRIBONA-DEV-1-13521,ADRIBONA-DEV-1,13521,8,31670,8081,ADRIBONA-DEV-1)
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-DEV-1:13521/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Master])
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077

WORKER:
D:\sparkspark-class.cmd org.apache.spark.deploy.worker.Worker 
spark://adribona-dev-1:7077
13/12/05 08:18:46 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/12/05 08:18:46 INFO worker.Worker: Starting Spark worker 
ADRIBONA-DEV-1:13521 with 8 cores, 30.9 GB RAM
13/12/05 08:18:46 INFO worker.Worker: Spark home: D:\spark
13/12/05 08:18:46 INFO server.Server: jetty-7.x.y-SNAPSHOT
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/json,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/static,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/log,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/logPage,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/json,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{*,null}
13/12/05 08:18:46 INFO server.AbstractConnector: Started 
SelectChannelConnector@0.0.0.0:8081
13/12/05 08:18:46 INFO ui.WorkerWebUI: Started Worker web UI at 
http://ADRIBONA-DEV-1:8081
13/12/05 08:18:46 INFO worker.Worker: Connecting to master 
spark://adribona-dev-1:7077


--Adrian

From: Adrian Bonar [mailto:adrian.bo...@microsoft.com]
Sent: Thursday, December 5, 2013 7:49 AM
To: user@spark.incubator.apache.org
Subject: RE: Pre-build Spark for Windows 8.1

Excellent! Thank you, Matei.

From: Matei Zaharia [mailto:matei.zaha...@gmail.com]
Sent: Wednesday, December 4, 2013 4:26 PM
To: user@spark.incubator.apache.orgmailto:user@spark.incubator.apache.org
Subject: Re: Pre-build Spark for Windows 8.1

Hey Adrian,

Ideally you shouldn't use Cygwin to run on Windows - use the .cmd scripts we 
provide instead. Cygwin might be made to work but we haven't tried to do this 
so far so it's not supported. If you can fix it, that would of course be 
welcome.

Also, the deploy scripts don't work

Re: Pre-build Spark for Windows 8.1

2013-12-05 Thread Matei Zaharia 
Hi,

When you launch the worker, try using spark://ADRIBONA-DEV-1:7077 as the URL 
(uppercase instead of lowercase). Unfortunately Akka is very specific about 
seeing hostnames written in the same way on each node, or else it thinks the 
message is for another machine!

Matei

On Dec 5, 2013, at 8:27 AM, Adrian Bonar adrian.bo...@microsoft.com wrote:

 The master starts up now as expected but the workers are unable to connect to 
 the master. It looks like the master is refusing the connection messages but 
 I’m not sure why. The first two error lines below are from trying to connect 
 a worker from a separate machine and the last two error lines are from trying 
 to connect a worker on the same machine as the master. I verified that the 
 workers do not show up in the master’s web ui.
  
 MASTER:
 D:\sparkspark-class org.apache.spark.deploy.master.Master
 13/12/05 08:08:34 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
 13/12/05 08:08:34 INFO master.Master: Starting Spark master at 
 spark://ADRIBONA-DEV-1:7077
 13/12/05 08:08:34 INFO server.Server: jetty-7.x.y-SNAPSHOT
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/metrics/master/json,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/metrics/applications/json,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/static,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/app/json,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/app,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/json,null}
 13/12/05 08:08:34 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{*,null}
 13/12/05 08:08:34 INFO server.AbstractConnector: Started 
 SelectChannelConnector@0.0.0.0:8088
 13/12/05 08:08:34 INFO ui.MasterWebUI: Started Master web UI at 
 http://ADRIBONA-DEV-1:8088
 13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
 RegisterWorker(worker-20131205080912-ADRIBONA-SFC-1-19280,ADRIBONA-SFC-1,19280,2,512,8081,ADRIBONA-SFC-1)
  for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master 
 at akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
 akka://sparkMaster@ADRIBONA-DEV-1:7077
 13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
 DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-SFC-1:19280/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Master])
  for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
 akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
 akka://sparkMaster@ADRIBONA-DEV-1:7077
 13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
 RegisterWorker(worker-20131205081846-ADRIBONA-DEV-1-13521,ADRIBONA-DEV-1,13521,8,31670,8081,ADRIBONA-DEV-1)
  for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master 
 atakka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
 akka://sparkMaster@ADRIBONA-DEV-1:7077
 13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
 DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-DEV-1:13521/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Master])
  for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
 akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
 akka://sparkMaster@ADRIBONA-DEV-1:7077
  
 WORKER:
 D:\sparkspark-class.cmd org.apache.spark.deploy.worker.Worker 
 spark://adribona-dev-1:7077
 13/12/05 08:18:46 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
 13/12/05 08:18:46 INFO worker.Worker: Starting Spark worker 
 ADRIBONA-DEV-1:13521 with 8 cores, 30.9 GB RAM
 13/12/05 08:18:46 INFO worker.Worker: Spark home: D:\spark
 13/12/05 08:18:46 INFO server.Server: jetty-7.x.y-SNAPSHOT
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/metrics/json,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/static,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/log,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/logPage,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{/json,null}
 13/12/05 08:18:46 INFO handler.ContextHandler: started 
 o.e.j.s.h.ContextHandler{*,null}
 13/12/05 08:18:46 INFO server.AbstractConnector: Started 
 SelectChannelConnector@0.0.0.0:8081
 13/12/05 08:18:46 INFO ui.WorkerWebUI: Started Worker web UI at 
 http://ADRIBONA-DEV-1:8081
 13/12/05 08:18:46 INFO worker.Worker: Connecting to master 
 spark://adribona-dev-1:7077
  
  
 --Adrian
  
 From: Adrian Bonar [mailto:adrian.bo...@microsoft.com] 
 Sent: Thursday, December 5, 2013 7:49 AM
 To: user@spark.incubator.apache.org
 Subject: RE: Pre-build Spark for Windows 8.1
  
 Excellent! Thank you, Matei.
  
 From: Matei Zaharia [mailto:matei.zaha...@gmail.com] 
 Sent

Re: Pre-build Spark for Windows 8.1

2013-12-05 Thread Andrew Ash 
://adribona-dev-1:8081/
 13/12/05 08:18:46 INFO worker.Worker: Connecting to master
 spark://adribona-dev-1:7077


 --Adrian

 *From:* Adrian Bonar 
 [mailto:adrian.bo...@microsoft.comadrian.bo...@microsoft.com
 ]
 *Sent:* Thursday, December 5, 2013 7:49 AM
 *To:* user@spark.incubator.apache.org
 *Subject:* RE: Pre-build Spark for Windows 8.1

 Excellent! Thank you, Matei.

 *From:* Matei Zaharia [mailto:matei.zaha...@gmail.commatei.zaha...@gmail.com
 ]
 *Sent:* Wednesday, December 4, 2013 4:26 PM
 *To:* user@spark.incubator.apache.org
 *Subject:* Re: Pre-build Spark for Windows 8.1

 Hey Adrian,

 Ideally you shouldn’t use Cygwin to run on Windows — use the .cmd scripts
 we provide instead. Cygwin might be made to work but we haven’t tried to do
 this so far so it’s not supported. If you can fix it, that would of course
 be welcome.

 Also, the deploy scripts don’t work on Windows — we assumed that people
 would mostly use Windows for local development and testing. However, if you
 do want to launch a cluster, you can use spark-class.cmd manually to launch
 a Master and then Workers. Use the commands in the first part of
 http://spark.incubator.apache.org/docs/latest/spark-standalone.html.

 Matei

 On Dec 4, 2013, at 2:31 PM, Adrian Bonar adrian.bo...@microsoft.com
 wrote:



 Separate from my previous thread about *building* a distribution of Spark
 on Win8, I am also trying to run the pre-build binaries with little
 success. I downloaded and extract “spark-0.8.0-incubating-bin-hadoop1” to
 d:\spark and attempted to start a master with the following error:

 $ sh bin/start-master.sh
 starting org.apache.spark.deploy.master.Master, logging to
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
 failed to launch org.apache.spark.deploy.master.Master:

   Error: Could not find or load main class
 org.apache.spark.deploy.master.Master
 full log in
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out

 --Adrian (again. J)

RE: Pre-build Spark for Windows 8.1

2013-12-05 Thread Adrian Bonar 
Strange, but that definitely did the trick. Thanks again!

From: Matei Zaharia [mailto:matei.zaha...@gmail.com]
Sent: Thursday, December 5, 2013 2:44 PM
To: user@spark.incubator.apache.org
Subject: Re: Pre-build Spark for Windows 8.1

Hi,

When you launch the worker, try using spark://ADRIBONA-DEV-1:7077 as the URL 
(uppercase instead of lowercase). Unfortunately Akka is very specific about 
seeing hostnames written in the same way on each node, or else it thinks the 
message is for another machine!

Matei

On Dec 5, 2013, at 8:27 AM, Adrian Bonar 
adrian.bo...@microsoft.commailto:adrian.bo...@microsoft.com wrote:


The master starts up now as expected but the workers are unable to connect to 
the master. It looks like the master is refusing the connection messages but 
I'm not sure why. The first two error lines below are from trying to connect a 
worker from a separate machine and the last two error lines are from trying to 
connect a worker on the same machine as the master. I verified that the workers 
do not show up in the master's web ui.

MASTER:
D:\sparkspark-class org.apache.spark.deploy.master.Master
13/12/05 08:08:34 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/12/05 08:08:34 INFO master.Master: Starting Spark master at 
spark://ADRIBONA-DEV-1:7077
13/12/05 08:08:34 INFO server.Server: jetty-7.x.y-SNAPSHOT
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/master/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/applications/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/static,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/app/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/app,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/json,null}
13/12/05 08:08:34 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{*,null}
13/12/05 08:08:34 INFO server.AbstractConnector: Started 
SelectChannelConnector@0.0.0.0mailto:SelectChannelConnector@0.0.0.0:8088
13/12/05 08:08:34 INFO ui.MasterWebUI: Started Master web UI at 
http://ADRIBONA-DEV-1:8088http://adribona-dev-1:8088/
13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
RegisterWorker(worker-20131205080912-ADRIBONA-SFC-1-19280,ADRIBONA-SFC-1,19280,2,512,8081,ADRIBONA-SFC-1)
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:09:15 ERROR NettyRemoteTransport(null): dropping message 
DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-SFC-1:19280/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Masterakka://sparkWorker@ADRIBONA-SFC-1:19280/user/Worker%5d,Actor%5bakka:/sparkMaster@adribona-dev-1:7077/user/Master])
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
RegisterWorker(worker-20131205081846-ADRIBONA-DEV-1-13521,ADRIBONA-DEV-1,13521,8,31670,8081,ADRIBONA-DEV-1)
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/user/Master 
atakka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077
13/12/05 08:18:46 ERROR NettyRemoteTransport(null): dropping message 
DaemonMsgWatch(Actor[akka://sparkWorker@ADRIBONA-DEV-1:13521/user/Worker],Actor[akka://sparkMaster@adribona-dev-1:7077/user/Masterakka://sparkWorker@ADRIBONA-DEV-1:13521/user/Worker%5d,Actor%5bakka:/sparkMaster@adribona-dev-1:7077/user/Master])
 for non-local recipient akka://sparkMaster@adribona-dev-1:7077/remote at 
akka://sparkMaster@ADRIBONA-DEV-1:7077 local is 
akka://sparkMaster@ADRIBONA-DEV-1:7077

WORKER:
D:\sparkspark-class.cmd org.apache.spark.deploy.worker.Worker 
spark://adribona-dev-1:7077
13/12/05 08:18:46 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/12/05 08:18:46 INFO worker.Worker: Starting Spark worker 
ADRIBONA-DEV-1:13521 with 8 cores, 30.9 GB RAM
13/12/05 08:18:46 INFO worker.Worker: Spark home: D:\spark
13/12/05 08:18:46 INFO server.Server: jetty-7.x.y-SNAPSHOT
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/metrics/json,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/static,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/log,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/logPage,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{/json,null}
13/12/05 08:18:46 INFO handler.ContextHandler: started 
o.e.j.s.h.ContextHandler{*,null}
13/12/05 08:18:46 INFO server.AbstractConnector: Started 
SelectChannelConnector

Re: Pre-build Spark for Windows 8.1

2013-12-04 Thread Matei Zaharia 
Hey Adrian,

Ideally you shouldn’t use Cygwin to run on Windows — use the .cmd scripts we 
provide instead. Cygwin might be made to work but we haven’t tried to do this 
so far so it’s not supported. If you can fix it, that would of course be 
welcome.

Also, the deploy scripts don’t work on Windows — we assumed that people would 
mostly use Windows for local development and testing. However, if you do want 
to launch a cluster, you can use spark-class.cmd manually to launch a Master 
and then Workers. Use the commands in the first part of 
http://spark.incubator.apache.org/docs/latest/spark-standalone.html.

Matei

On Dec 4, 2013, at 2:31 PM, Adrian Bonar adrian.bo...@microsoft.com wrote:

 Separate from my previous thread about building a distribution of Spark on 
 Win8, I am also trying to run the pre-build binaries with little success. I 
 downloaded and extract “spark-0.8.0-incubating-bin-hadoop1” to d:\spark and 
 attempted to start a master with the following error:
  
 $ sh bin/start-master.sh
 starting org.apache.spark.deploy.master.Master, logging to 
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
 failed to launch org.apache.spark.deploy.master.Master:
  
   Error: Could not find or load main class 
 org.apache.spark.deploy.master.Master
 full log in 
 /cygdrive/d/spark/bin/../logs/spark-adribona-org.apache.spark.deploy.master.Master-1-ADRIBONA-DEV-1.out
  
 --Adrian (again. J)