Great job, Mike! Would you be willing to write up a quick wiki FAQ on this, here?
https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Documentation Cheers, Chris On Feb 8, 2012, at 8:34 AM, Cayanan, Michael D (388J) wrote: > Hey Brian and Chris, > > So, I first re-tried it using Chris' suggestion of using spaces and I still > got an error. Here is what I'm seeing in the log: > > Feb 8, 2012 7:55:33 AM org.apache.oodt.cas.pge.PGETaskInstance > runIngestCrawler > INFO: Executing StdProductCrawler in productPath: > [/Users/mcayanan/smap/staging/L1A_RADAR_SPS/20120208T155531/output] > Feb 8, 2012 7:55:33 AM org.apache.oodt.cas.pge.PGETaskInstance > runIngestCrawler > WARNING: Failed while attempting to ingest products while crawling directory > '/Users/mcayanan/smap/staging/L1A_RADAR_SPS/20120208T155531/output' (all > products may not have been ingested) : No bean named > 'KickoffL1BS0LoResWorkflow MoveMetadataToBackupDir' is defined > org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean > named 'KickoffL1BS0LoResWorkflow MoveMetadataToBackupDir' is defined > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:360) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBeanDefinition(AbstractBeanFactory.java:914) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:241) > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164) > at > org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:761) > at > org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFactory(CrawlerActionRepo.java:68) > at > org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:87) > at > org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance.java:405) > at > org.apache.oodt.cas.pge.PGETaskInstance.ingestProducts(PGETaskInstance.java:349) > at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:454) > at > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > at > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.run(XmlRpcBatchStub.java:229) > at java.lang.Thread.run(Thread.java:680) > > > I then modified my tasks.xml file where I specified the 'PCS_ActionsIds' > property with the action IDs: > > <property name="PCS_ActionsIds" > value="KickoffL1BS0LoResWorkflow,MoveMetadataToBackupDir" /> > > and added a line in the PgeConfig.xml as Brian suggested: > > <customMetadata> > <metadata key="PCS_ActionsIds" val="[PCS_ActionsIds]"/> > > and still got an error in the log as shown above. > > So, I ended up re-naming the property name in the tasks.xml to some other > name (so as to not reference itself essentially) and that did the trick! > > tasks.xml: > > <property name="Actions" > value="KickoffL1BS0LoResWorkflow,MoveMetadataToBackupDir" /> > > PgeConfig.xml: > > <customMetadata> > <metadata key="PCS_ActionsIds" val="[Actions]"/> > > > Also, I found out that you can't have spaces in between the commas like so, > > <property name="Actions" value="KickoffL1BS0LoResWorkflow, > MoveMetadataToBackupDir" /> > > > Thanks for all your help guys! > > > -Mike > > > From: holenoter <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Tue, 7 Feb 2012 22:06:27 -0800 > To: "[email protected]" <[email protected]> > Subject: Re: CAS PGE Crawler Question > > workflow manager properties (static metadata) are passed to CAS-PGE via > WorkflowTaskConfiguration which only supports scalar metadata... CAS-PGE > doesn't re-parse out these metadata value because you have to allow for > workflow metadata to contain values with commas without always being split > out by CAS-PGE.. so you have to explicitly repoint the metadata field at > itself so it becomes custom (or local) metadata for CAS-PGE and then the > value will be parsed on comma... context: CAS-PGE initially did the opposite > (i.e. always re-parse workflow metadata into multi-values) but chris later > changed this to the way it is now because this broke OCO since OCO had static > workflow metadata which contained metadata with commas that was not supposed > to be parsed into multi-values. > > -brian > > On Feb 07, 2012, at 07:36 PM, "Mattmann, Chris A (388J)" > <[email protected]> wrote: > >> Hi Mike, >> >> Space should work in the property value below. Can you show me >> the log output from CAS-PGE in the workflow log file (or the resource manager >> batch stubs?) It should show you what Actions are being executed >> and that can help to debug. >> >> Thanks! >> >> Cheers, >> Chris >> >> On Feb 7, 2012, at 4:06 PM, Cayanan, Michael D (388J) wrote: >> >> > If I wanted to specify more than one PCS_ActionsIds values, how would I do >> > this? >> > >> > I've tried the following with no success: >> > >> > <property name="PCS_ActionIds" value="Action1, Action2" /> >> > >> > >> > and >> > >> > <property name="PCS_ActionIds" value="Action1,Action2" /> >> > >> > >> > and >> > >> > <property name="PCS_ActionIds" value="Action1 Action2" /> >> > >> > >> > >> > -Mike >> > >> > >> > On 2/7/12 3:22 PM, "Mattmann, Chris A (388J)" >> > <[email protected]> wrote: >> > >> >> Hi Mike, >> >> >> >> Glad to hear! >> >> >> >> Cheers, >> >> Chris >> >> >> >> On Feb 7, 2012, at 3:14 PM, Cayanan, Michael D (388J) wrote: >> >> >> >>> Hi all, >> >>> >> >>> Disregard my previous message. Turns out I had a typo. I had to set >> >>> 'PCS_ActionsIds' instead of 'PCS_ActionIds'. >> >>> >> >>> -Mike >> >>> >> >>> From: "Cayanan, Michael D (388J)" <[email protected]> >> >>> Reply-To: "[email protected]" <[email protected]> >> >>> Date: Tue, 7 Feb 2012 13:51:12 -0800 >> >>> To: "[email protected]" <[email protected]> >> >>> Cc: "Wong, Cynthia L (388J)" <[email protected]> >> >>> Subject: CAS PGE Crawler Question >> >>> >> >>> Hi all, >> >>> >> >>> I would like CAS PGE to run a post ingest success action that I created >> >>> upon a successful ingestion of the output file(s). How do you configure >> >>> CAS PGE for this type of scenario? >> >>> >> >>> I have a tasks.xml file in the workflow, where I define my post ingest >> >>> action under the 'PCS_ActionIds' property: >> >>> >> >>> <task id="urn:my:Task" name="Task" >> >>> ... >> >>> <property name="PCS_ActionIds" >> >>> value="MyPostIngestSuccessAction" /> >> >>> <property name="PCS_ActionRepoFile" >> >>> value="crawler-config.xml" /> >> >>> </configuration> >> >>> </task> >> >>> >> >>> My crawler-config.xml has the following: >> >>> >> >>> <beans xmlns="http://www.springframework.org/schema/beans"; >> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> >>> xmlns:p="http://www.springframework.org/schema/p"; >> >>> xsi:schemaLocation="http://www.springframework.org/schema/beans >> >>> http://www.springframework.org/schema/beans/spring-beans-2.5.xsd";> >> >>> >> >>> <import >> >>> resource="classpath:/org/apache/oodt/cas/crawl/crawler-beans.xml"/> >> >>> <import resource="./action-beans.xml"/> >> >>> <import >> >>> resource="classpath:/org/apache/oodt/cas/crawl/comparator/precondition-be >> >>> ans.xml"/> >> >>> <import >> >>> resource="classpath:/org/apache/oodt/cas/crawl/option/cmd-line-option-bea >> >>> ns.xml"/> >> >>> </beans> >> >>> >> >>> Within my action-beans.xml, I added the post ingest success action: >> >>> >> >>> <beans xmlns="http://www.springframework.org/schema/beans"; >> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> >>> xsi:schemaLocation="http://www.springframework.org/schema/beans >> >>> http://www.springframework.org/schema/beans/spring-beans-2.5.xsd";> >> >>> >> >>> <bean >> >>> class="org.apache.oodt.commons.spring.postprocessor.SetIdBeanPostProcesso >> >>> r"/> >> >>> >> >>> <!-- Crawler Actions --> >> >>> >> >>> <bean id="MyPostIngestSuccessAction" lazy-init="true" >> >>> class="gov.nasa.jpl.crawler.action.MyPostIngestSuccessAction"> >> >>> <property name="description" value="Kickoff Another Workflow >> >>> Event"/> >> >>> <property name="executeCommand" value="sh >> >>> [WORKFLOWMGR_HOME]/bin/run_L1B_S0_LoRes.sh"/> >> >>> <property name="phases"> >> >>> <list> >> >>> <value type="java.lang.String">postIngestSuccess</value> >> >>> </list> >> >>> </property> >> >>> </bean> >> >>> >> >>> Not sure if I'm missing anything else. >> >>> >> >>> Thanks in advance, >> >>> Mike >> >>> >> >> >> >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> Chris Mattmann, Ph.D. >> >> Senior Computer Scientist >> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> >> Office: 171-266B, Mailstop: 171-246 >> >> Email: [email protected] >> >> WWW: http://sunset.usc.edu/~mattmann/ >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> Adjunct Assistant Professor, Computer Science Department >> >> University of Southern California, Los Angeles, CA 90089 USA >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> > >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Chris Mattmann, Ph.D. >> Senior Computer Scientist >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 171-266B, Mailstop: 171-246 >> Email: [email protected] >> WWW: http://sunset.usc.edu/~mattmann/ >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Adjunct Assistant Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
