I just recently finished working on a project called WATERS (Workflow for the Alignment, Taxonomy, and Ecology of Ribosomal Sequences), and it also runs jobs on a Rocks cluster. It uses the Comad framework (so data is managed as a stream that passes sequentially through all the actors, each actor choosing what to add or remove), so it might be a bit unfamiliar, but it could be worth a look.
I handled the Rocks cluster submission and monitoring through qsub etc., but I do it in the Java code rather than at the actor level, so that it can be abstracted away and hidden from the user. A zip intended for end users can be found at daks.ucdavis.edu/ waters.zip. I believe the source is included. Hope it helps. - Sean Sent from my iPhone On Dec 2, 2009, at 9:26 AM, Jianwu Wang <jianwu at sdsc.edu> wrote: > Arthur Goldberg wrote: >> >> Hello >> >> I've a bio-informatics workflow, that I'm considering running in >> Kepler. The workflow analyzes about a dozen genomes, and will take >> a few hundred hours of computing, which I'll run on our 64 node >> Rocks cluster. >> >> Several Perl programs retrieve the data and analyze it, inputting >> and outputting results from and to both files and a MySQL database. >> Each program needs to be instantiated multiple times with different >> input. >> >> It appears that Kepler might be able to execute these computations >> with its PN scheduler and ExternalExecution Actor, but 1) we >> typically manage jobs on the cluster via the SGE (qsub) and 2) >> Kepler's documentation says that "to use the ExternalExecution >> actor, the invoked application must be on the local computer", >> which implies that Kepler must be installed on all nodes of the >> cluster. >> >> How would one approach this problem with Kepler? >> >> If Kepler isn't the right tool for this problem, what would you >> recommend? >> >> Thanks >> Arthur >> >> >> Arthur P. Goldberg, PhD >> >> Research Scientist in Bioinformatics Group >> Plant Systems Biology Laboratory >> www.virtualplant.org >> >> Visiting Academic >> Computer Science Department >> Courant Institute of Mathematical Sciences >> www.cs.nyu.edu/artg >> >> artg at cs.nyu.edu >> New York University >> 212 995-4918 >> 100 Washington Sq East >> 8th Floor Silver Building >> >> _______________________________________________ >> Kepler-users mailing list >> Kepler-users at kepler-project.org >> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users >> > > _______________________________________________ > Kepler-users mailing list > Kepler-users at kepler-project.org > http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mercury.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20091202/2d93a244/attachment.html>
