Hyy all, I have been using the EC2 script to launch R&D pyspark clusters for a while now. As we use alot of packages such as numpy and scipy with openblas, scikit-learn, bokeh, vowpal wabbit, pystan and etc... All this time, we have been building AMIs on top of the standard spark-AMIs at https://github.com/amplab/spark-ec2/tree/branch-1.6/ami-list/us-east-1
Mainly, I have done the following: - updated yum - Changed the standard python to python 2.7 - changed pip to 2.7 and installed alot of libararies on top of the existing AMIs and created my own AMIs to avoid having to boostrap. But the ec-2 standard AMIs are from *Early February , 2014* and now have become extremely fragile. For example, when I update a certain library, ipython would break, or pip would break and so forth. Can someone please direct me to a more upto date AMI that I can use with more confidence. And I am also interested to know what things need to be in the AMI, if I wanted to build an AMI from scratch (Last resort :( ) And isn't it time to have a ticket in the spark project to build a new suite of AMIs for the EC2 script? https://issues.apache.org/jira/browse/SPARK-922 Many thanks in4maniac -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/New-Amazon-AMIs-for-EC2-script-tp28419.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
