Re: Unable to build spark from source

2015-01-03 Thread Manoj Kumar
Hi, I compiled using sbt and it takes lesser time. Thanks for the tip. I'm able to run these examples ( https://spark.apache.org/docs/latest/mllib-linear-methods.html related to MLib in the pyspark shell. However I got some errors related to Spark SQL while compiling. Is that a reason to worry? I

Re: Unable to build spark from source

2015-01-03 Thread Michael Armbrust
I'll add that most of the spark developers I know use sbt for day to day development as it can be much faster for incremental compilation and it has several nice features. On Sat, Jan 3, 2015 at 9:59 AM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > You can use the same build comman

Re: Unable to build spark from source

2015-01-03 Thread Simon Elliston Ball
You can use the same build commands, but it's well worth setting up a zinc server if you're doing a lot of builds. That will allow incremental scala builds, which speeds up the process significantly. SPARK-4501 might be of interest too. Simon > On 3 Jan 2015, at 17:27, Manoj Kumar wrote: > >

Re: Unable to build spark from source

2015-01-03 Thread Manoj Kumar
My question was that if once I make changes in the source code to a file, do I rebuild it using any other command, such that it takes in only the changes (because it takes a lot of time)? On Sat, Jan 3, 2015 at 10:40 PM, Manoj Kumar wrote: > Yes, I've built spark successfully, using the same co

Re: Unable to build spark from source

2015-01-03 Thread Manoj Kumar
Yes, I've built spark successfully, using the same command mvn -DskipTests clean package but it built because now I do not work behind a proxy. Thanks.

Re: Unable to build spark from source

2015-01-03 Thread Sean Owen
from repo1.maven.org. Should some repositoy urls be added to the Maven's settings file? -- bit1...@163.com *From:* Manoj Kumar *Date:* 2015-01-03 18:46 *To:* user *Subject:* Unable to build spark from source Hello, I tried to build Spark from source using th

Re: Unable to build spark from source

2015-01-03 Thread Manoj Kumar
> bit1...@163.com > > > *From:* Manoj Kumar > *Date:* 2015-01-03 18:46 > *To:* user > *Subject:* Unable to build spark from source > Hello, > > I tried to build Spark from source using this command (all dependencies > installed) > but it fails th

Re: Unable to build spark from source

2015-01-03 Thread bit1...@163.com
The error hints that the maven module scala-compiler can't be fetched from repo1.maven.org. Should some repositoy urls be added to the Maven's settings file? bit1...@163.com From: Manoj Kumar Date: 2015-01-03 18:46 To: user Subject: Unable to build spark from source Hello,

Re: Unable to build spark from source

2015-01-03 Thread Sean Owen
This indicates a network problem in getting third party artifacts. Is there a proxy you need to go through? On Jan 3, 2015 11:17 AM, "Manoj Kumar" wrote: > Hello, > > I tried to build Spark from source using this command (all dependencies > installed) > but it fails this error. Any help would be

Unable to build spark from source

2015-01-03 Thread Manoj Kumar
Hello, I tried to build Spark from source using this command (all dependencies installed) but it fails this error. Any help would be appreciated. mvn -DskipTests clean package [INFO] Spark Project Parent POM .. FAILURE [28:14.408s] [INFO] Spark Project Networking ...