Actually you'll just want to clone the 1.0 branch then use the spark-ec2 script in there to launch your cluster. The --spark-git-repo flag is if you want to launch with a different version of Spark on the cluster. In your case you just need a different version of the launch script itself, which will be present in the 1.0 branch of Spark.
- Patrick On Tue, Jun 17, 2014 at 9:29 PM, Jeremy Lee <unorthodox.engine...@gmail.com> wrote: > I am about to spin up some new clusters, so I may give that a go... any > special instructions for making them work? I assume I use the " > --spark-git-repo=" option on the spark-ec2 command. Is it as easy as > concatenating your string as the value? > > On cluster management GUIs... I've been looking around at Amabari, Datastax, > Cloudera, OpsCenter etc. Not totally convinced by any of them yet. Anyone > using a good one I should know about? I'm really beginning to lean in the > direction of Cassandra as the distributed data store... > > > On Wed, Jun 18, 2014 at 1:46 PM, Patrick Wendell <pwend...@gmail.com> wrote: >> >> By the way, in case it's not clear, I mean our maintenance branches: >> >> https://github.com/apache/spark/tree/branch-1.0 >> >> On Tue, Jun 17, 2014 at 8:35 PM, Patrick Wendell <pwend...@gmail.com> >> wrote: >> > Hey Jeremy, >> > >> > This is patched in the 1.0 and 0.9 branches of Spark. We're likely to >> > make a 1.0.1 release soon (this patch being one of the main reasons), >> > but if you are itching for this sooner, you can just checkout the head >> > of branch-1.0 and you will be able to use r3.XXX instances. >> > >> > - Patrick >> > >> > On Tue, Jun 17, 2014 at 4:17 PM, Jeremy Lee >> > <unorthodox.engine...@gmail.com> wrote: >> >> Some people (me included) might have wondered why all our m1.large spot >> >> instances (in us-west-1) shut down a few hours ago... >> >> >> >> Simple reason: The EC2 spot price for Spark's default "m1.large" >> >> instances >> >> just jumped from 0.016 per hour, to about 0.750. Yes, Fifty times. >> >> Probably >> >> something to do with world cup. >> >> >> >> So far this is just us-west-1, but prices have a tendency to equalize >> >> across >> >> centers as the days pass. Time to make backups and plans. >> >> >> >> "m3" spot prices are still down at $0.02 (and being new, will be >> >> bypassed by >> >> older systems), so it would be REAAALLYY nice if there had been some >> >> progress on that issue. Let me know if I can help with testing and >> >> whatnot. >> >> >> >> >> >> -- >> >> Jeremy Lee BCompSci(Hons) >> >> The Unorthodox Engineers > > > > > -- > Jeremy Lee BCompSci(Hons) > The Unorthodox Engineers
