No problem Val. The fileStaging feature was sort of half baked, however it does work and here is the basic concept behind it. If you have files in your CAS-PGE job that need to be copied from controlled access storage and staged into the job directory for the running CAS-PGE job, the fileStager is the way to do it. You give it a FM query that defines what files to select, and then it uses the File Manager configured data transferer to copy the file(s) from controlled access storage to the PGE job working dir.
Another way to accomplish this is simply in your CAS-PGE exe block and cmd blocks is to add a cp or remote staging command (rsync, scp, etc.) that accomplishes the same thing. Cheers, Chris — Chris Mattmann chris.mattm...@gmail.com -----Original Message----- From: "Mallder, Valerie" <valerie.mall...@jhuapl.edu> Reply-To: <dev@oodt.apache.org> Date: Tuesday, September 29, 2015 at 9:51 AM To: "dev@oodt.apache.org" <dev@oodt.apache.org> Subject: file staging question >Hi All, > >I have a question related to the file staging feature in a PGE Task. >Basically, how are you using this feature and what exactly does it do for >you? > >Does it just move some files from one place to another? If so, does the >destination directory have to exist or does it get created? And, does >the destination directory structure need to be the same as the source >directory structure? > >When does the 'staging' take place? Does it execute before the <exe> >section of the PGE? > > >Thanks, >Val > >P.S. I have read every line of ever file in OODT that contains the word >'staging' and it is still not clear to me what this feature actually does >or how I can configure my PGE config file to use it. But, I did find >some totally awesome documentation in the curator J But, it made it >sound like the directory you set up as your 'staging' directory is the >directory that files get ingested from. Whereas, I was under the >impression that the directory you specify as your <output> directory is >the directory that files get ingested from. So, it was confusing. L >Wonderfully thorough documentation, just confusing for my case. > > >Valerie A. Mallder > >New Horizons Deputy Mission System Engineer >The Johns Hopkins University/Applied Physics Laboratory >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 >240-228-7846 (Office) 410-504-2233 (Blackberry) >
