Hi, As the ec2 launch script provided by spark uses https://github.com/mesos/spark-ec2 to download and configure all the tools in the cluster (spark, hadoop etc). You can create your own git repository to achieve your goal. More precisely:
1. Upload your own version of spark in s3 at address <path to your spark> 2. Fork https://github.com/mesos/spark-ec2 and make a change in ./spark/init.sh (add wget <path to your spark>) 3. Change line 638 in ec2 launch script: git clone <your repository in github> Hope this can be helpful. Cheers Gen On Tue, Jan 6, 2015 at 11:51 PM, Ganon Pierce <ganon.pie...@me.com> wrote: > Is there a way to use the ec2 launch script with a locally built version > of spark? I launch and destroy clusters pretty frequently and would like to > not have to wait each time for the master instance to compile the source as > happens when I set the -v tag with the latest git commit. To be clear, I > would like to launch a non-release version of spark compiled locally as > quickly as I can launch a release version (e.g. -v 1.2.0) which does not > have to be compiled upon launch. > > Up to this point, I have just used the launch script included with the > latest release to set up the cluster and then manually replaced the > assembly file on the master and slaves with the version I built locally and > then stored on s3. Is there anything wrong with doing it this way? Further, > is there a better or more standard way of accomplishing this? > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
