I have Chef recipes for Grid Engine that will auto install SGE master
and execd onto a single node. We also have recipes that built full-blown
SGE clusters but largely stopped doing that in favor of StarCluster - we
ended up adding modules to StarCluster to do the customizations we
needed when running on AWS.
I can't clean up those recipes in time to share them but keep an eye on
(or an RSS reader) bioteam.net as we eventually get to a point where we
have our act together enough to publish that github repo of our useful
Chef stuff.
And to be honest there is nothing about Chef + SGE that you can't learn
on your own by looking at other Recipes and Cookbooks - that's how I
learned Chef. Grab a coobook or two that you know you'll need from
http://community.opscode.com/cookbooks and start looking at the code.
If I can find the time to clean up my "build SGE onto a single node"
Recipe I'll post it to this list and on bioteam.net. Since the cookbook
creates an SGE autoinstall template file that is compatible with
"./inst_sge -x ..." it should be pretty easy to extend it to install
multiple compute nodes.
-Chris
Chi Chan wrote:
So no one has Puppet or Chef Recipes for SGE?
--Chi
----- 原始信件 ----
寄件者： Chi Chan<chichan.grideng...@yahoo.com.tw>
Anyone has Opscode Chef recipes for SGE Grid Engine? I want to setup a simple
test cluster and try IT automation and see if it is really useful.
--Chi
----- 原始信件 ----
寄件者： Rayson Ho<rayray...@gmail.com>
收件者： Jesse Becker<becker...@mail.nih.gov>
副本： Chi Chan<chichan.grideng...@yahoo.com.tw>; Kristen Eisenberg<kristen.eisenb...@yahoo.com>;
"users@gridengine.org"<users@gridengine.org>
寄件日期： 2011/10/18 (二) 5:08 PM
主旨： Re: [gridengine users] Re??? `cloud' nodes
On Tue, Oct 11, 2011 at 4:39 PM, Rayson Ho<rayray...@gmail.com> wrote:
On Tue, Oct 11, 2011 at 3:58 PM, Jesse Becker<becker...@mail.nih.gov> wrote:
We're getting a bit off topic here, but CFEngine fits the one of your
requirements (but probably not the other). 狢t is written in C, is quite
fast, and has a much lower resource footprint than anything based on
Ruby.
Like many other things, some people who are against configuration
management and some don't...
Re-replying to Jesse's message...
I attended an IBM training on BladeCenter recently, and talked to an
IBMer during lunchtime.
He mentioned U of Toronto's Scinet, which is a TOP500 supercomputer
(highest ranking: #16 in Jun 09). Scinet uses IBM's xCAT, which is
developed for HPC clusters, for provisioning.
In fact some other people use xCAT as a replacement for Platform's
Scali Cluster Manager:
http://www.nodeofcrash.com/?p=353
xCAT is written in Perl, so it also is another package to install& maintain.
Rayson
It all comes down to, is it cheaper to manually manage a cluster by
hand, or should we use tools like Chef, Puppet, or Tivoli and hire 1
less person.
(But for some HPC sites, running tools in the background is not
possible or acceptable. For example, the original Catamount OS in
DoE's Red Store could only run 1 single-threaded process at a time on
the compute PEs.)
I brought up the StarCluster Chef integration because doing things by
hand is not possible in the EC2 scale - in the pre-cloud days, you can
setup machines 1 at a time, but when you can launch hundreds of
machines in minutes, doing things by hand is way too slow& expensive.
BTW, some tutorial videos by Justin& Chris:
- StarCluster 0.91 Demo: http://www.youtube.com/watch?v=vC3lJcPq1FY
- Launching a Cluster on Amazon EC2 Spot Instances Using StarCluster:
http://www.youtube.com/watch?v=2Ym7epCYnSk
Rayson
贌owever, it is probably not "simpler to learn" by a long shot.
Configuration management is deceptively complex once you get beyond the
"golden master" view of the world.
On Tue, Oct 11, 2011 at 12:45 PM, Rayson Ho<rayray...@gmail.com> wrote:
2) And the BioTeam integrates StarCluster with Opscode Chef, so you
can automate many of the administrative tasks (create users, package
management, service setup, etc) of EC2 SGE clusters:
http://bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/
While I have more experience with IBM Tivoli& Puppet, I am really
impressed with the Chef EC2 module. And Chef is gaining quite a lot of
momentum lately. E.g. Dell recently open sourced Crowbar, which is an
OpenStack installer based on Chef.
I will wait for Puppet Enterprise 2.0, which is supposed to have new
EC2& VMware provisioning& orchestration capabilities, and see how
Puppet compares with Chef before I decide if I am switching to Chef.
But configuration management is real and it can cut down a lot of IT
infrastructure maintenance.
Rayson
On Mon, Oct 10, 2011 at 7:01 PM, Kristen Eisenberg
<kristen.eisenb...@yahoo.com> wrote:
Chris Dagdigian<dag at sonsorol.org> writes:
By FAR the best way to run standalone Grid Engine clusters on the Amazon
Cloud today is to simply use MIT Starcluster :
http://web.mit.edu/stardev/cluster/index.html
I didn't mention it as I got the impression that that wasn't the OP's
case, but probably it should be mentioned in the same place on
gridengine.info, assuming that's still the best place for such things.
Kristen Eisenberg
Billige Fl???e
Marketing GmbH
Emanuelstr. 3,
10317 Berlin
Deutschland
Telefon: +49 (33)
5310967
Email:
utebachmeier at
gmail.com
Site:
http://flug.airego.de - Billige Fl???e vergleichen
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
--
Jesse Becker
NHGRI Linux support (Digicon Contractor)
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
_______________________________________________
users mailing list
users@gridengine.org
https://gridengine.org/mailman/listinfo/users
