Hello again, I am afraid the problem is not resolved yet. After switching of in-memory storage, Taverna creates a lot of directories in /tmp. After the first 10000 iterations workflow execution, there were 100000 subdirectories in /tmp: susana@dae53:/tmp$ ls -l usecase* |wc 100004 380021 3261102 Before to run the 10000 iterations workflow a second time, I closed Taverna, because I thought closing taverna the provenance data is deleted, so this 100000 subdirectories will be removed, but not, this subdirectories were not removed. Thinking that maybe this 100000 subdirectories were not a problem, I run a second time my 10000 iterations workflow, but when it finished I got a lot of error messages in my output values like that: java.io.FileNotFoundException: /tmp/usecase5980252654117511742dir/filetmp.txt (No such file or directory) So I guess the huge amount of subdirectories was a problem, so I deleted them and I re-run the 10000 iterations workflow ant it worked! Is there a way to say to Taverna that delete this subdirectories?
Susana 2011/10/25 Susana Sanchez Exposito <[email protected]>: > Thanks Alan! > > First, as Alan said, I tried to run in-memory with provenance capture > disabled, but it did not work, so I switched off in-memory storage and > It is work! > After three hours the workflow with 10000 iterations finished succesfully > > Susana. > > > 2011/10/24 Alan R Williams <[email protected]>: >> On 24/10/2011 19:22, Susana Sanchez Exposito wrote: >>> Hello all, >> >> Hello >> >>> I am building a workflow that basically does the following: >>> >> [snip of problem description] >> >>> So my questions are: >>> Is there any limit on the number of iterations? >> >> There shouldn't be, and I have run workflows with lots more iterations >> than this. >> >>> Is there any way to split the number of iterations? I mean: if it is >>> possible to give the 40000 lines file as input and say to Tavena >>> "process it 2000 lines by 2000 lines" >> >> You could use a beanshell to split it into temporary files and then pass >> the list of temporary file names to something that reads each file. I'm >> not sure that is necessary, see below. >> >>> Or maybe, there is a way to keep in a file (in disk and not in >>> memory), the results of each iteration ... ? >> >> Yes. If you go to the preferences (under File or Taverna depending on >> your operating system). Select "Data and provenance". First I would try >> to run in-memory with provenance capture disabled. If that does not >> work, then you can switch off in-memory storage and Taverna will keep >> the data in a database. >> >> David Withers has a draft service for streaming data from a file. I can >> check what the state of that is. >> >>> Thanks >> >> Alan >> >> >> ------------------------------------------------------------------------------ >> The demand for IT networking professionals continues to grow, and the >> demand for specialized networking skills is growing even more rapidly. >> Take a complimentary Learning@Cisco Self-Assessment and learn >> about Cisco certifications, training, and career opportunities. >> http://p.sf.net/sfu/cisco-dev2dev >> _______________________________________________ >> taverna-users mailing list >> [email protected] >> [email protected] >> Web site: http://www.taverna.org.uk >> Mailing lists: http://www.taverna.org.uk/about/contact-us/ >> > -- Susana Sánchez Expósito Instituto de Astrofísica de Andalucía IAA (CSIC) Camino Bajo de Huétor, 50. Granada E-18008 Tel:(+34) 958 121 311 / (+34) 958 230 618 Fax:(+34) 958 814 530 e-mail: [email protected] ------------------------------------------------------------------------------ The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev _______________________________________________ taverna-users mailing list [email protected] [email protected] Web site: http://www.taverna.org.uk Mailing lists: http://www.taverna.org.uk/about/contact-us/
