Thanks for the info and the tip! I'll look into writing our own script
based on the spark-ec2 scripts.
Best,
Augustus
On Thu, Nov 12, 2015 at 10:01 AM, Jason Rubenstein <
jasondrubenst...@gmail.com> wrote:
> Hi,
>
> With some minor changes to spark-ec2/spark/init.sh and writing your own
>
spark-ec2 does not offer a way to upgrade an existing cluster, and from
what I gather, it wasn't intended to be used to manage long-lasting
infrastructure. The recommended approach really is to just destroy your
existing cluster and launch a new one with the desired configuration.
If you want to
Hi,
With some minor changes to spark-ec2/spark/init.sh and writing your own
"upgrade-spark.sh" script, you can upgrade spark in place.
(Make sure to call not only spark/init.sh but also spark/setup.sh, because
the latter uses copy-dir to get your ner version of spark to the slaves)
I wrote one
Hey All,
I have a Spark cluster(running version 1.5.0) on EC2 launched with the
provided spark-ec2 scripts. If I want to upgrade Spark to 1.5.2 in the same
cluster, what's the safest / recommended way to do that?
I know I can spin up a new cluster running 1.5.2, but it doesn't seem
efficient to
