How to create spark AMI in AWS

2015-02-09 Thread Guodong Wang
Hi guys, I want to launch spark cluster in AWS. And I know there is a spark_ec2.py script. I am using the AWS service in China. But I can not find the AMI in the region of China. So, I have to build one. My question is 1. Where is the bootstrap script to create the Spark AMI? Is it here( https:/

Re: How to create spark AMI in AWS

2015-02-09 Thread Franc Carter
Hi, I'm very new to Spark, but experienced with AWS - so take that in to account with my suggestions. I started with an AWS base image and then added the pre-built Spark-1.2. I then added made a 'Master' version and a 'Worker' versions and then made AMIs for them. The Master comes up with a sta

Re: How to create spark AMI in AWS

2015-02-09 Thread Nicholas Chammas
Guodong, spark-ec2 does not currently support the cn-north-1 region, but you can follow [SPARK-4241](https://issues.apache.org/jira/browse/SPARK-4241) to find out when it does. The base AMI used to generate the current Spark AMIs is very old. I'm not sure anyone knows what it is anymore. What I k

Re: How to create spark AMI in AWS

2015-02-09 Thread Guodong Wang
Hi Nicholas, Thanks for your quick reply. I'd like to try to build a image with create_image.sh. Then let's see how we can launch spark cluster in region cn-north-1. Guodong On Tue, Feb 10, 2015 at 3:59 AM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Guodong, > > spark-ec2 does n

Re: How to create spark AMI in AWS

2015-02-09 Thread Nicholas Chammas
OK, good luck! On Mon Feb 09 2015 at 6:41:14 PM Guodong Wang wrote: > Hi Nicholas, > > Thanks for your quick reply. > > I'd like to try to build a image with create_image.sh. Then let's see how > we can launch spark cluster in region cn-north-1. > > > > Guodong > > On Tue, Feb 10, 2015 at 3:59 A