On 10/12/2012 15:26, Ümit Seren wrote: > Apologies if these question was already answered: > I did some research but couldn't come up with a definite answer/solution > for this question/problem. > > Let's say I have a simple workflow consisting of 5 steps: > > 1.) retrieve data from a database using a REST service > 2.) storing retrieved data as a csv and staging/copying it to the local > HPC cluster > 3.) dispatching first job (SGE) on the cluster for some primary analysis > 4.) dispatching second job (SGE) on the cluster for secondary analysis > 5.) staging out the files from cluster > > Step 1, 2 are quite easy however I haven an issue with step 3-5. > So step 4 should only be started once step 3 is finished. However I > didn't find any native solution in taverna to check if the job is > finished before I start step 4. > Furthermore these jobs can run quickly or take some time (depending on > cluster usage). > Is there an easy solution to this asynchronous problem?
What do you get back when you submit the job in (3) ? > I can think of 2 ways how to solve it: > > 1.) create a service/nested workflow that checks the output files on the > cluster > 2.) create a service/nested workflow that checks the SGE job status (see > http://www.myexperiment.org/workflows/174.html) (2) is the normal way to do this. If you look at http://www.myexperiment.org/workflows/1768.html it has an example of how to do asynchronous calls. > I know there are also some plugins which might abstract away these > things (PBS, Unicore). > But I would be interested in best practices in solving the HPC/Taverna > interaction. The looping nested workflow is the normal mechanism. Do you know the type of cluster you are running on? > thanks in advance > Uemit Alan ------------------------------------------------------------------------------ LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d _______________________________________________ taverna-users mailing list [email protected] [email protected] Web site: http://www.taverna.org.uk Mailing lists: http://www.taverna.org.uk/about/contact-us/
