Hi, some update... 1) StarCluster stable release 0.92rc2 has new features that make running larger clusters (100+ instances) real easy. The new additions like dynamic cluster size grow & shrink would reduce the cost of operating semi-permanent clusters (where jobs arrive unpredictably during the lifetime of the cluster), and change in the bootstrap process makes starting larger clusters faster and more reliable. Also, support for Spot, Cluster-Compute, and GPU Instances were added (late last year, I believe).
See: http://web.mit.edu/stardev/cluster/features.html One thing that LSF and Project Hedeby have but missing in StarCluster is cloud bursting, which sounds useful in theory but I am have not seen a lot of adaptions in the real world. I have not used or testdriven cloud bursting before, but I believe with a few lines (200 lines max) of shell scripting to setup VPN connections to the remote EC2 nodes, and not provision a qmaster on EC2, then cloud bursting can be implemented with StarCluster. (Contact me offline if you are interested in this - I started hacking the StarCluster code a bit already.) 2) And the BioTeam integrates StarCluster with Opscode Chef, so you can automate many of the administrative tasks (create users, package management, service setup, etc) of EC2 SGE clusters: http://bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/ While I have more experience with IBM Tivoli & Puppet, I am really impressed with the Chef EC2 module. And Chef is gaining quite a lot of momentum lately. E.g. Dell recently open sourced Crowbar, which is an OpenStack installer based on Chef. I will wait for Puppet Enterprise 2.0, which is supposed to have new EC2 & VMware provisioning & orchestration capabilities, and see how Puppet compares with Chef before I decide if I am switching to Chef. But configuration management is real and it can cut down a lot of IT infrastructure maintenance. Rayson On Mon, Oct 10, 2011 at 7:01 PM, Kristen Eisenberg <kristen.eisenb...@yahoo.com> wrote: > Chris Dagdigian <dag at sonsorol.org> writes: > >> By FAR the best way to run standalone Grid Engine clusters on the Amazon >> Cloud today is to simply use MIT Starcluster : >> >> http://web.mit.edu/stardev/cluster/index.html > > I didn't mention it as I got the impression that that wasn't the OP's > case, but probably it should be mentioned in the same place on > gridengine.info, assuming that's still the best place for such things. > Kristen Eisenberg > Billige Flüge > Marketing GmbH > Emanuelstr. 3, > 10317 Berlin > Deutschland > Telefon: +49 (33) > 5310967 > Email: > utebachmeier at > gmail.com > Site: > http://flug.airego.de - Billige Flüge vergleichen > _______________________________________________ > users mailing list > users@gridengine.org > https://gridengine.org/mailman/listinfo/users > > _______________________________________________ users mailing list users@gridengine.org https://gridengine.org/mailman/listinfo/users
