Oh Ok. So, it adds workflow metadata to the existing metadata. As you suggest, I will continue querying from the PGE Config for pulling specific files.
I'll be glad to contribute to the merging of the PGETask Workflow Pre-condition to the trunk 0.4. Thanks Chris! On Fri, Sep 9, 2011 at 8:12 PM, Mattmann, Chris A (388J) < [email protected]> wrote: > Hey Sheryl, > > On Sep 9, 2011, at 6:10 PM, Sheryl John wrote: > > > Hi, > > > > I have questions regarding the Workflow Manager, particularly the Met > File writers and querying data from the File Manager. > > > > 1) For files that are required for a PGE task workflow, do I specify the > metadata key-value pairs of the file ( e.g Key="TableName" Val="Chartevents" > ) in the metout-config.xml ? > > Basically metout-config.xml is for the specific > MetadataListPcsMetFileWriter [1] instance configured in your CAS-PGE > pge-config.xml file. > This file defines metadata to pull out of the workflow context metadata, > and to write (and merge) with the rest of the file metadata for the product > you are about to ingest. So, putting a key in metout-config.xml is like > saying "I'd like to copy this workflow context metadata to the file product > metadata". > > > And, how does the Workflow mgr use the values for the next step/task in > the pipeline? > > See above. > > > > > > > 2) To query the File Manager, should I use the Query building option > available in the PGETask Workflow Pre-Condition? > > I have previously used the SQL-like query in a Pge Config file to pull > ingested files, but after reading the Workflow 2 Guide, I was wondering if > the File Mgr querying should be done in the pre-conditon of a task. So, this > would be a pre-condition for checking if input files are available before a > task begins. Is this right? > > To use the condition based querying from PGETaskWorkflowCondition, you'd > need to use wengine-branch. > > Rather than do so, I'd recommend just wiring the SQL(... query into your > CAS PGE config and doing the querying there. It'll be > simpler, and I have plans to merge in PGETask Workflow Pre-Condition later > into the trunk for 0.4. If that's something you'd like > to help with, I'd love to see a JIRA issue and a patch and I'll happily > shepherd it in. > > Thanks Sheryl! > > Cheers, > Chris > > [1] http://s.apache.org/bW4 > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > -- -Sheryl
