RE: Spark cluster set up on EC2 customization

2015-02-26 Thread Sameer Tilak
Thanks! Date: Thu, 26 Feb 2015 12:51:21 +0530 Subject: Re: Spark cluster set up on EC2 customization From: ak...@sigmoidanalytics.com To: ssti...@live.com CC: user@spark.apache.org You can easily add a function (say setup_pig) inside the function setup_cluster in this scriptThanksBest Regards

Re: Spark cluster set up on EC2 customization

2015-02-25 Thread Akhil Das
You can easily add a function (say setup_pig) inside the function setup_cluster in this script https://github.com/apache/spark/blob/master/ec2/spark_ec2.py#L649 Thanks Best Regards On Thu, Feb 26, 2015 at 7:08 AM, Sameer Tilak ssti...@live.com wrote: Hi, I was looking at the documentation

Spark cluster set up on EC2 customization

2015-02-25 Thread Sameer Tilak
Hi, I was looking at the documentation for deploying Spark cluster on EC2. http://spark.apache.org/docs/latest/ec2-scripts.html We are using Pig to build the data pipeline and then use MLLib for analytics. I was wondering if someone has any experience to include additional tools/services