Somehow I didn't receive Alan's reply via email so here is my inline reply:
> What do you get back when you submit the job in (3) ?
So I haven't really created the workflow yet I was just evaluating my
options
> (2) is the normal way to do this. If you look at
> http://www.myexperiment.org/workflows/1768.html it has an example of how
> to do asynchronous calls.
Thanks for the link. I will check it out.
> The looping nested workflow is the normal mechanism. Do you know the
> type of cluster you are running on?
Currently we are using a HP cluster (SGE) but we are switching to an SGI
cluster (PBS)
Thanks
Uemit
On Mon, Dec 10, 2012 at 4:26 PM, Ümit Seren <[email protected]> wrote:
> Apologies if these question was already answered:
> I did some research but couldn't come up with a definite answer/solution
> for this question/problem.
>
> Let's say I have a simple workflow consisting of 5 steps:
>
> 1.) retrieve data from a database using a REST service
> 2.) storing retrieved data as a csv and staging/copying it to the local
> HPC cluster
> 3.) dispatching first job (SGE) on the cluster for some primary analysis
> 4.) dispatching second job (SGE) on the cluster for secondary analysis
> 5.) staging out the files from cluster
>
> Step 1, 2 are quite easy however I haven an issue with step 3-5.
> So step 4 should only be started once step 3 is finished. However I didn't
> find any native solution in taverna to check if the job is finished before
> I start step 4.
> Furthermore these jobs can run quickly or take some time (depending on
> cluster usage).
> Is there an easy solution to this asynchronous problem?
> I can think of 2 ways how to solve it:
>
> 1.) create a service/nested workflow that checks the output files on the
> cluster
> 2.) create a service/nested workflow that checks the SGE job status (see
> http://www.myexperiment.org/workflows/174.html)
>
> I know there are also some plugins which might abstract away these things
> (PBS, Unicore).
> But I would be interested in best practices in solving the HPC/Taverna
> interaction.
>
> thanks in advance
> Uemit
>
>
>
>
>
>
>
------------------------------------------------------------------------------
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
_______________________________________________
taverna-users mailing list
[email protected]
[email protected]
Web site: http://www.taverna.org.uk
Mailing lists: http://www.taverna.org.uk/about/contact-us/
