I tried out for a couple of files, and the metdump alias is not dumping out any metadata for the respective ProductIds.
I also tried dumping out metadata for other files for which I've not specified metout-configs, and they seem to be empty too. The fmquery works fine and it dumps out all the ProductIds. Thanks, On Thu, Sep 15, 2011 at 9:08 PM, Mattmann, Chris A (388J) < [email protected]> wrote: > Hey Sheryl, > > Are you sure that RecordID is being recorded for the particular products > that you are trying to look up? > > One way to find this out: > > 1. get the product IDs back for the products you are interested in. One way > to do this would be > to remove RecordID from your query as you mention below and then add > CAS.ProductId to the > SELECT list, and to change FORMAT='$FileLocation...' to > FORMAT='$CAS.ProductId'. > > 2. Then, for each of the product IDs, check out OODT-306 [1] and use the > metdump alias to dump > out the metadata for each product ID to the current working directory. > Then, inspect that .met file. > Is RecordID recorded? > > Let's start there. > > Cheers, > Chris > > [1] http://issues.apache.org/jira/browse/OODT-306 > > On Sep 15, 2011, at 8:53 PM, Sheryl John wrote: > > > Hi, > > > > I am not clear about the metadata in .met files generated by the > met-config.xml files. Particularly, where do they fall in the product-type > and element mappings? > > > > Because in my query, SQL(FORMAT='$FileLocation/$Filename'){ SELECT > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = > 'Chartevents' AND RecordID = '[PID]' , I'm selecting elements that are > mapped to the product-type, ISMRawData and elements that are mapped to the > GenericFile product-type. > > > > Except RecordID, which was defined as a key in the metout-config file > and I'm assuming that the parsing error is because the RecordID is not > mapped to any product-type. This query works and pulls all the files I need > when I remove RecordID from the query. Again, since I'm not clear about > metadata mapping , it's highly likely that I'm wrong about this. > > > > Thanks, > > On Thu, Sep 15, 2011 at 11:17 AM, holenoter <[email protected]> wrote: > > hey Sheryl, > > > > I think there is a misunderstanding on how the met-config.xml files are > used... the metadata in these files are not added to your workflow context > metadata... they are only used to create the *.met files for the data file > for filemgr ingest... about your query I'm not quite clear on how you are > wanting go use it... it kinda seems like you want to dynamically replace out > RecordIO... in which case you're gonna need it to look more like RecordIO = > '[PID]' > > > > -brian > > > > > > On Sep 14, 2011, at 09:18 PM, Sheryl John <[email protected]> wrote: > > > >> Hi Chris, > >> > >> Thanks for pointing that out. I was working on a different query and > similar to the one I described. Though I've used only single quotes now, I'm > still getting the same parsing error. > >> > >> But, I think what I intended to do with context metadata ( for my > workflow) doesn't make sense for a single file product and rather, would > apply for a group of files. > >> I guess I have revise my workflow logic and metadata use before getting > back to this query. > >> > >> Thanks! > >> > >> > >> > >> On Wed, Sep 14, 2011 at 8:42 PM, Mattmann, Chris A (388J) < > [email protected]> wrote: > >> Hi Sheryl, > >> > >> I notice you're using double quotes on the RecordID = "PID", yet you are > using single quotes on ISMTable = 'Chartevents'. > >> > >> Try using single quotes on both and see if that fixes it. > >> > >> Thanks! > >> > >> Cheers, > >> Chris > >> > >> > >> On Sep 14, 2011, at 7:03 PM, Sheryl John wrote: > >> > >> > Hi, > >> > > >> > I have defined some key-val pairs for a file( say Output.csv) in a > metout-config.xml for my PGETask Workflow. However, after executing the > workflow, the met-config.xml is not creating a Output.csv.cas file. > >> > > >> > I want to be able to use the above keys/metadata later on in an > SQL-like query from the pgeconfig file. For example, if I've defined > 'RecordID' as a key in the metout-config.xml, I would want to use this > metadata in the following query: > >> > > >> > SQL(FORMAT='$FileLocation/$Filename'){ SELECT > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = > 'Chartevents' AND RecordID = "PID"} The others keys included in the query > above are elements and product-types that were defined during ingestion in > the File Manager. > >> > > >> > At present, the task fails to parse the above query when I run the > workflow. Is this because metout-config is not creating the Output.cas file? > And, what is the best way to specify metadata files for a group of files or > for a folder? > >> > > >> > Thanks, > >> > Sheryl > >> > >> > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> Chris Mattmann, Ph.D. > >> Senior Computer Scientist > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >> Office: 171-266B, Mailstop: 171-246 > >> Email: chris.a.mattmann@nasagov > >> > >> WWW: http://sunset.usc.edu/~mattmann/ > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> Adjunct Assistant Professor, Computer Science Department > >> University of Southern California, Los Angeles, CA 90089 USA > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> > >> > >> > >> > >> -- > >> -Sheryl > > > > > > > > -- > > -Sheryl > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > -- -Sheryl
