On Tue, Mar 10, 2009 at 08:32, Tim Slighter <[email protected]> wrote: > I am looking for a brief tour or even unclassified documentation that > would show me how these grids and/or clusters are installed, configured > and managed, including how HPC applications are deployed into these type > of environments.
there are myriad ways to do all of these things--it's a science and an art, like writing perl =) many clusters i've been involved with have grown organically from other roles (desktops, old servers, etc) which informed the route they took from their prior life to their current. so for instance, the last large-scale cluster (~500 nodes) i worked on used a queueing system that ran on every machine in the place and soaked up idle cycles, and supported process migration so that your desktop was part of the cluster while you were at lunch, and apps were "deployed" by running them from your home directory on any machine. longer-running jobs eventually ended up on the dedicated cluster machines as they tried to allocate more resources than desktops were configured to share. in a previous life, we had a system setup wherein nobody could log in to any of the dedicated cluster nodes, and everything had to be run via MPI from a bastion host, and we had a homebrew app to inform you of which machines you should set MPI up to run on based on load of each cluster machine. low-tech but functional and free in the days when you needed a lot of free time to babysit or learn most queueing systems available. these days, queueing systems have improved and i help manage a cluster using SGE to handle queueing that works without hassle. if you want to drink from the firehose, the beowulf mailing list (http://www.beowulf.org/pipermail/beowulf/) is where i learned a lot about what i was doing when i built the first cluster on my campus in 1998. it can also be really informative to set up your own minicluster and run the example pvm and mpi programs (for a feel for parallel computing frameworks) as well as some of the niftier user-accessible projects out there like distcc (a distributed compiler) _______________________________________________ PLUG mailing list [email protected] http://lists.pdxlinux.org/mailman/listinfo/plug
