Hi Jeremy: Can you send the part oft he workflow that creates the problem. Seems to be a couple of actors only. Maybe looking at that will help us, help you.
best Bertram PS Re. COMAD release plans, I'm cc-ing Tim who should have more info. On Mon, Jul 13, 2009 at 9:39 AM, Jeremy Douglass <jeremydouglass at gmail.com>wrote: > Bertram, thanks for the suggestion, and for your paper on COMAD. Yes, > "SDC" was my typo of SDF. I've since tried changing the director to > DDF and PN for all my tests in the situation above, although that had > no effect on the type mismatch problem -- Directory Listing produces > an array, and I can't get that into a form consumable by an operation > such as Image Converter. > > I read the paper on COMAD with interest. COMAD isn't part of 1.0 right > now, but I look forward to a version that includes it. > > -- Jeremy > > > On Sat, Jul 11, 2009 at 1:09 PM, Bertram Ludaescher<ludaesch at ucdavis.edu> > wrote: > > > > > > On Sat, Jul 11, 2009 at 12:51 PM, Jeremy Douglass < > jeremydouglass at gmail.com> > > wrote: > >> > >> I'm a new Kepler and struggling to apply a single actor (e.g. Image > >> Converter) to a batch of objects (e.g. a directory of images, as > >> represented by the array generated by Directory Listing). > >> > >> I've read the documentation and searched the forums, and tried > >> modifying examples, but I'm not understanding something paradigmatic, > >> and would appreciate a tip: how do you pipe an array of files through > >> single-file actors? > >> > >> EXAMPLE: > >> > >> In Kepler 1.0.0, 03-ImageDisplay.xml uses an SDC director, and > specifies: > >> Constant Image Filename --> Image Converter --> ImageJ > > > > I think you mean SDF here.. > > > >> > >> In an attempt to converter a *directory* of images, I tried adding a > >> Directory Listing actor: > >> Constant Dirname --> Directory Listing --> Image Converter --> ImageJ > >> > >> ...however this generates an error, as Image Converter will only act > >> on a string, not an array. So I tried > >> Constant Dirname --> Directory Listing --> Array to Sequence --> > >> Image Converter --> ImageJ > >> > >> ...which almost works if I hand-specified the number of Array elements > >> in the directory ahead of time), but still only processed the first > >> file listed in the string (Picture 1.png) -- I'm assuming it just > >> throws the rest of the string away. > > > > > > SDF has the restriction that the director needs to know a-priori (ie, > before > > running the wf) how many tokens an actor consumes and produces on each > port > > for each firing. For cases where this is known, SDF is usually a good > > choice. > > However, when processing arrays / lists of varying lengths, you usually > > don't have this information. > > Try DDF instead (or PN if you want actors to execute as independent > > threads). > >> > >> I'm not understanding something about how one processes a list in a > >> Kepler workflow -- I've looked through Array Operation and Iterative > >> Operation actors, and none of them seem to be analogous to an > >> imperative programming loop. Am I using the wrong kind of director? > >> Something else? > > > > See above: DDF might solve your problem. > > There is also a different way to work with nested data collections (of > which > > lists are a simple example): using the COMAD director. I'll let the COMAD > > experts and developes say more about how to use this feature (might not > be > > part of Kepler 1.0, but in the current developers' version; a new COMAD > > version is also under active development) > >> > >> > >> Thanks for your help. > > > > Also, since you mentioned the paradigma / how-to deal with collections, > you > > might want to check out this overview paper: > > > > Scientific workflow design for mere mortals. Timothy McPhillips and Shawn > > Bowers and Daniel Zinn and Bertram Ludaescher (2008) Future Generation > > Computer Systems. Volume 25. Pages: 541-551. (Download) > > > > from the Kepler pubs page: > > https://kepler-project.org/publications?tags=keplerworkflow > > > > Bertram > > > >> > >> BACKGROUND: My larger goal is to adapt a frame based image processing > >> workflow into Kepler (it is currently a combination of python, > >> javascript, and bash scripts). To illustrate, one part of the workflow > >> takes a heterogeneous collection of images (all file types and sizes) > >> and copies them to a new directory as a collection of standardized > >> collection jpegs. Another part of the workflow dumps a movie to a > >> frame directory of jpegs, then runs multiple image measurement and > >> image processing operations (e.g. ImageJ, MATLAB) on each frame and > >> saves the results to a text file. > >> _______________________________________________ > >> Kepler-users mailing list > >> Kepler-users at kepler-project.org > >> http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users > > > > > _______________________________________________ > Kepler-users mailing list > Kepler-users at kepler-project.org > http://mercury.nceas.ucsb.edu/kepler/mailman/listinfo/kepler-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mercury.nceas.ucsb.edu/kepler/pipermail/kepler-users/attachments/20090713/b4a57b04/attachment-0001.html>
