Re: spark pre-built binaries for 0.8.0
It takes a while to download all the dependencies from Maven the first time you build. Just let it run, it won’t need to do that next time. Or see if you can build it on a machine with better Internet access and copy the binaries (you can even get an EC2 machine for a few cents if you want). Matei On Dec 17, 2013, at 11:54 PM, phoenix bai wrote: > yeah I know. > I can`t do nothing to improve my network, so, all i manage to do is: > if it looks hanging, i kill it and restart. so far so good, but looks > long way to go. > > > On Wed, Dec 18, 2013 at 3:35 PM, Azuryy Yu wrote: > Hi Phoenix, > This is not Spark releated. It was your local net work limited. > > Thanks. > > > On Wed, Dec 18, 2013 at 3:17 PM, phoenix bai wrote: > I am compiling against hadoop 2.2.0, it really takes time, especially with > network connection is not that stable and all. > > SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true ./sbt/sbt assembly > > > On Wed, Dec 18, 2013 at 10:39 AM, Patrick Wendell wrote: > Hey Philip, > > No - those are compiled against the "mr1" version. You'll need to > build yourself for YARN. > > - Patrick > > On Tue, Dec 17, 2013 at 10:32 AM, Philip Ogren > wrote: > > I have a question about the pre-built binary for 0.8.0 for CDH 4 listed > > here: > > > > http://spark.incubator.apache.org/downloads.html > > > > and linked here: > > > > http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz > > > > Should I expect that this release is compiled with YARN enabled (i.e. the > > SPARK_YARN property is set to true)? > > > > Thanks, > > Philip > > >
Re: spark pre-built binaries for 0.8.0
yeah I know. I can`t do nothing to improve my network, so, all i manage to do is: if it looks hanging, i kill it and restart. so far so good, but looks long way to go. On Wed, Dec 18, 2013 at 3:35 PM, Azuryy Yu wrote: > Hi Phoenix, > This is not Spark releated. It was your local net work limited. > > Thanks. > > > On Wed, Dec 18, 2013 at 3:17 PM, phoenix bai wrote: > >> I am compiling against hadoop 2.2.0, it really takes time, especially >> with network connection is not that stable and all. >> >> SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true ./sbt/sbt assembly >> >> >> On Wed, Dec 18, 2013 at 10:39 AM, Patrick Wendell wrote: >> >>> Hey Philip, >>> >>> No - those are compiled against the "mr1" version. You'll need to >>> build yourself for YARN. >>> >>> - Patrick >>> >>> On Tue, Dec 17, 2013 at 10:32 AM, Philip Ogren >>> wrote: >>> > I have a question about the pre-built binary for 0.8.0 for CDH 4 listed >>> > here: >>> > >>> > http://spark.incubator.apache.org/downloads.html >>> > >>> > and linked here: >>> > >>> > http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz >>> > >>> > Should I expect that this release is compiled with YARN enabled (i.e. >>> the >>> > SPARK_YARN property is set to true)? >>> > >>> > Thanks, >>> > Philip >>> >> >> >
Re: spark pre-built binaries for 0.8.0
Hi Phoenix, This is not Spark releated. It was your local net work limited. Thanks. On Wed, Dec 18, 2013 at 3:17 PM, phoenix bai wrote: > I am compiling against hadoop 2.2.0, it really takes time, especially with > network connection is not that stable and all. > > SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true ./sbt/sbt assembly > > > On Wed, Dec 18, 2013 at 10:39 AM, Patrick Wendell wrote: > >> Hey Philip, >> >> No - those are compiled against the "mr1" version. You'll need to >> build yourself for YARN. >> >> - Patrick >> >> On Tue, Dec 17, 2013 at 10:32 AM, Philip Ogren >> wrote: >> > I have a question about the pre-built binary for 0.8.0 for CDH 4 listed >> > here: >> > >> > http://spark.incubator.apache.org/downloads.html >> > >> > and linked here: >> > >> > http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz >> > >> > Should I expect that this release is compiled with YARN enabled (i.e. >> the >> > SPARK_YARN property is set to true)? >> > >> > Thanks, >> > Philip >> > >
Re: spark pre-built binaries for 0.8.0
I am compiling against hadoop 2.2.0, it really takes time, especially with network connection is not that stable and all. SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true ./sbt/sbt assembly On Wed, Dec 18, 2013 at 10:39 AM, Patrick Wendell wrote: > Hey Philip, > > No - those are compiled against the "mr1" version. You'll need to > build yourself for YARN. > > - Patrick > > On Tue, Dec 17, 2013 at 10:32 AM, Philip Ogren > wrote: > > I have a question about the pre-built binary for 0.8.0 for CDH 4 listed > > here: > > > > http://spark.incubator.apache.org/downloads.html > > > > and linked here: > > > > http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz > > > > Should I expect that this release is compiled with YARN enabled (i.e. the > > SPARK_YARN property is set to true)? > > > > Thanks, > > Philip >
Re: spark pre-built binaries for 0.8.0
Hey Philip, No - those are compiled against the "mr1" version. You'll need to build yourself for YARN. - Patrick On Tue, Dec 17, 2013 at 10:32 AM, Philip Ogren wrote: > I have a question about the pre-built binary for 0.8.0 for CDH 4 listed > here: > > http://spark.incubator.apache.org/downloads.html > > and linked here: > > http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz > > Should I expect that this release is compiled with YARN enabled (i.e. the > SPARK_YARN property is set to true)? > > Thanks, > Philip
spark pre-built binaries for 0.8.0
I have a question about the pre-built binary for 0.8.0 for CDH 4 listed here: http://spark.incubator.apache.org/downloads.html and linked here: http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz Should I expect that this release is compiled with YARN enabled (i.e. the SPARK_YARN property is set to true)? Thanks, Philip
