Re: What is the best way to build my developing Spark for testing on EC2?

2014-10-06 Thread Yu Ishikawa
Hi Evan, Sorry for my replay late. And Thank you for your comment. As far as cluster set up goes, I usually launch spot instances with the spark-ec2 scripts, and then check out a repo which contains a simple driver application for my code. Then I have something crude like bash scripts

What is the best way to build my developing Spark for testing on EC2?

2014-10-02 Thread Yu Ishikawa
Hi all, I am trying to contribute some machine learning algorithms to MLlib. I must evaluate their performance on a cluster, changing input data size, the number of CPU cores and any their parameters. I would like to build my develoipng Spark on EC2 automatically. Is there already a building