I am about to spin up some new clusters, so I may give that a go... any special instructions for making them work? I assume I use the " --spark-git-repo=" option on the spark-ec2 command. Is it as easy as concatenating your string as the value?
On cluster management GUIs... I've been looking around at Amabari, Datastax, Cloudera, OpsCenter etc. Not totally convinced by any of them yet. Anyone using a good one I should know about? I'm really beginning to lean in the direction of Cassandra as the distributed data store... On Wed, Jun 18, 2014 at 1:46 PM, Patrick Wendell <pwend...@gmail.com> wrote: > By the way, in case it's not clear, I mean our maintenance branches: > > https://github.com/apache/spark/tree/branch-1.0 > > On Tue, Jun 17, 2014 at 8:35 PM, Patrick Wendell <pwend...@gmail.com> > wrote: > > Hey Jeremy, > > > > This is patched in the 1.0 and 0.9 branches of Spark. We're likely to > > make a 1.0.1 release soon (this patch being one of the main reasons), > > but if you are itching for this sooner, you can just checkout the head > > of branch-1.0 and you will be able to use r3.XXX instances. > > > > - Patrick > > > > On Tue, Jun 17, 2014 at 4:17 PM, Jeremy Lee > > <unorthodox.engine...@gmail.com> wrote: > >> Some people (me included) might have wondered why all our m1.large spot > >> instances (in us-west-1) shut down a few hours ago... > >> > >> Simple reason: The EC2 spot price for Spark's default "m1.large" > instances > >> just jumped from 0.016 per hour, to about 0.750. Yes, Fifty times. > Probably > >> something to do with world cup. > >> > >> So far this is just us-west-1, but prices have a tendency to equalize > across > >> centers as the days pass. Time to make backups and plans. > >> > >> "m3" spot prices are still down at $0.02 (and being new, will be > bypassed by > >> older systems), so it would be REAAALLYY nice if there had been some > >> progress on that issue. Let me know if I can help with testing and > whatnot. > >> > >> > >> -- > >> Jeremy Lee BCompSci(Hons) > >> The Unorthodox Engineers > -- Jeremy Lee BCompSci(Hons) The Unorthodox Engineers
