I have made a little progress - by downloading a prebuilt version of Spark
I can call spark-shell.cmd and bring up a spark shell.
In the shell things run.
Next I go to my development environment and try to run JavaWordCount
i try -Dspark.master=spark://local[*]:55519
OK I tried your build -
First you need to put spt in C:\sbt
Then you get
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.
e:\which java
/cygdrive/c/Program Files/Java/jdk1.6.0_25/bin/java
e:\java -version
java version 1.6.0_25
Java(TM) SE Runtime
I want to look at porting a Hadoop problem to Spark - eventually I want to
run on a Hadoop 2.0 cluster but while I am learning and porting I want to
run small problems in my windows box.
I installed scala and sbt.
I download Spark and in the spark directory can say
mvn -Phadoop-0.23
Hi,
I have built spark-1.0.0 on Windows using Java 7/8 and I have been able to
run several examples - here are my notes -
http://ml-nlp-ir.blogspot.com/2014/04/building-spark-on-windows-and-cloudera.html
on how to build from source and run examples in spark shell.
Regards,
Manu
On Sat, Aug
I am also trying to run on Windows and will post once I am able to launch.
My guess is that by hand it probably means manually forming the java
command I.e. class path and java options and then appending right class
name for worker or master.
Spark script follow hierarchy : start-master or
