No problemo Mr Mattmann. I've added a FAQ section to the wiki on how to specify multiple crawler actions.
Thanks to you and Brian for your help on this, Mike On 2/8/12 7:44 PM, "Mattmann, Chris A (388J)" <[email protected]> wrote: >Great job, Mike! Would you be willing to write up a quick wiki FAQ on >this, >here? > >https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Document >ation > >Cheers, >Chris > >On Feb 8, 2012, at 8:34 AM, Cayanan, Michael D (388J) wrote: > >> Hey Brian and Chris, >> >> So, I first re-tried it using Chris' suggestion of using spaces and I >>still got an error. Here is what I'm seeing in the log: >> >> Feb 8, 2012 7:55:33 AM org.apache.oodt.cas.pge.PGETaskInstance >>runIngestCrawler >> INFO: Executing StdProductCrawler in productPath: >>[/Users/mcayanan/smap/staging/L1A_RADAR_SPS/20120208T155531/output] >> Feb 8, 2012 7:55:33 AM org.apache.oodt.cas.pge.PGETaskInstance >>runIngestCrawler >> WARNING: Failed while attempting to ingest products while crawling >>directory >>'/Users/mcayanan/smap/staging/L1A_RADAR_SPS/20120208T155531/output' (all >>products may not have been ingested) : No bean named >>'KickoffL1BS0LoResWorkflow MoveMetadataToBackupDir' is defined >> org.springframework.beans.factory.NoSuchBeanDefinitionException: No >>bean named 'KickoffL1BS0LoResWorkflow MoveMetadataToBackupDir' is defined >> at >>org.springframework.beans.factory.support.DefaultListableBeanFactory.getB >>eanDefinition(DefaultListableBeanFactory.java:360) >> at >>org.springframework.beans.factory.support.AbstractBeanFactory.getMergedBe >>anDefinition(AbstractBeanFactory.java:914) >> at >>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abs >>tractBeanFactory.java:241) >> at >>org.springframework.beans.factory.support.AbstractBeanFactory.getBean(Abs >>tractBeanFactory.java:164) >> at >>org.springframework.context.support.AbstractApplicationContext.getBean(Ab >>stractApplicationContext.java:761) >> at >>org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanFac >>tory(CrawlerActionRepo.java:68) >> at >>org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:87) >> at >>org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstance. >>java:405) >> at >>org.apache.oodt.cas.pge.PGETaskInstance.ingestProducts(PGETaskInstance.ja >>va:349) >> at >>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:454) >> at >>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) >> at >>org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob.ru >>n(XmlRpcBatchStub.java:229) >> at java.lang.Thread.run(Thread.java:680) >> >> >> I then modified my tasks.xml file where I specified the >>'PCS_ActionsIds' property with the action IDs: >> >> <property name="PCS_ActionsIds" >>value="KickoffL1BS0LoResWorkflow,MoveMetadataToBackupDir" /> >> >> and added a line in the PgeConfig.xml as Brian suggested: >> >> <customMetadata> >> <metadata key="PCS_ActionsIds" val="[PCS_ActionsIds]"/> >> >> and still got an error in the log as shown above. >> >> So, I ended up re-naming the property name in the tasks.xml to some >>other name (so as to not reference itself essentially) and that did the >>trick! >> >> tasks.xml: >> >> <property name="Actions" >>value="KickoffL1BS0LoResWorkflow,MoveMetadataToBackupDir" /> >> >> PgeConfig.xml: >> >> <customMetadata> >> <metadata key="PCS_ActionsIds" val="[Actions]"/> >> >> >> Also, I found out that you can't have spaces in between the commas like >>so, >> >> <property name="Actions" value="KickoffL1BS0LoResWorkflow, >>MoveMetadataToBackupDir" /> >> >> >> Thanks for all your help guys! >> >> >> -Mike >> >> >> From: holenoter <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Tue, 7 Feb 2012 22:06:27 -0800 >> To: "[email protected]" <[email protected]> >> Subject: Re: CAS PGE Crawler Question >> >> workflow manager properties (static metadata) are passed to CAS-PGE via >>WorkflowTaskConfiguration which only supports scalar metadata... CAS-PGE >>doesn't re-parse out these metadata value because you have to allow for >>workflow metadata to contain values with commas without always being >>split out by CAS-PGE.. so you have to explicitly repoint the metadata >>field at itself so it becomes custom (or local) metadata for CAS-PGE and >>then the value will be parsed on comma... context: CAS-PGE initially did >>the opposite (i.e. always re-parse workflow metadata into multi-values) >>but chris later changed this to the way it is now because this broke OCO >>since OCO had static workflow metadata which contained metadata with >>commas that was not supposed to be parsed into multi-values. >> >> -brian >> >> On Feb 07, 2012, at 07:36 PM, "Mattmann, Chris A (388J)" >><[email protected]> wrote: >> >>> Hi Mike, >>> >>> Space should work in the property value below. Can you show me >>> the log output from CAS-PGE in the workflow log file (or the resource >>>manager >>> batch stubs?) It should show you what Actions are being executed >>> and that can help to debug. >>> >>> Thanks! >>> >>> Cheers, >>> Chris >>> >>> On Feb 7, 2012, at 4:06 PM, Cayanan, Michael D (388J) wrote: >>> >>> > If I wanted to specify more than one PCS_ActionsIds values, how >>>would I do >>> > this? >>> > >>> > I've tried the following with no success: >>> > >>> > <property name="PCS_ActionIds" value="Action1, Action2" /> >>> > >>> > >>> > and >>> > >>> > <property name="PCS_ActionIds" value="Action1,Action2" /> >>> > >>> > >>> > and >>> > >>> > <property name="PCS_ActionIds" value="Action1 Action2" /> >>> > >>> > >>> > >>> > -Mike >>> > >>> > >>> > On 2/7/12 3:22 PM, "Mattmann, Chris A (388J)" >>> > <[email protected]> wrote: >>> > >>> >> Hi Mike, >>> >> >>> >> Glad to hear! >>> >> >>> >> Cheers, >>> >> Chris >>> >> >>> >> On Feb 7, 2012, at 3:14 PM, Cayanan, Michael D (388J) wrote: >>> >> >>> >>> Hi all, >>> >>> >>> >>> Disregard my previous message. Turns out I had a typo. I had to set >>> >>> 'PCS_ActionsIds' instead of 'PCS_ActionIds'. >>> >>> >>> >>> -Mike >>> >>> >>> >>> From: "Cayanan, Michael D (388J)" <[email protected]> >>> >>> Reply-To: "[email protected]" <[email protected]> >>> >>> Date: Tue, 7 Feb 2012 13:51:12 -0800 >>> >>> To: "[email protected]" <[email protected]> >>> >>> Cc: "Wong, Cynthia L (388J)" <[email protected]> >>> >>> Subject: CAS PGE Crawler Question >>> >>> >>> >>> Hi all, >>> >>> >>> >>> I would like CAS PGE to run a post ingest success action that I >>>created >>> >>> upon a successful ingestion of the output file(s). How do you >>>configure >>> >>> CAS PGE for this type of scenario? >>> >>> >>> >>> I have a tasks.xml file in the workflow, where I define my post >>>ingest >>> >>> action under the 'PCS_ActionIds' property: >>> >>> >>> >>> <task id="urn:my:Task" name="Task" >>> >>> ... >>> >>> <property name="PCS_ActionIds" >>> >>> value="MyPostIngestSuccessAction" /> >>> >>> <property name="PCS_ActionRepoFile" >>> >>> value="crawler-config.xml" /> >>> >>> </configuration> >>> >>> </task> >>> >>> >>> >>> My crawler-config.xml has the following: >>> >>> >>> >>> <beans xmlns="http://www.springframework.org/schema/beans"; >>> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>> >>> xmlns:p="http://www.springframework.org/schema/p"; >>> >>> xsi:schemaLocation="http://www.springframework.org/schema/beans >>> >>> http://www.springframework.org/schema/beans/spring-beans-2.5.xsd";> >>> >>> >>> >>> <import >>> >>> resource="classpath:/org/apache/oodt/cas/crawl/crawler-beans.xml"/> >>> >>> <import resource="./action-beans.xml"/> >>> >>> <import >>> >>> >>>resource="classpath:/org/apache/oodt/cas/crawl/comparator/precondition-b >>>e >>> >>> ans.xml"/> >>> >>> <import >>> >>> >>>resource="classpath:/org/apache/oodt/cas/crawl/option/cmd-line-option-be >>>a >>> >>> ns.xml"/> >>> >>> </beans> >>> >>> >>> >>> Within my action-beans.xml, I added the post ingest success action: >>> >>> >>> >>> <beans xmlns="http://www.springframework.org/schema/beans"; >>> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>> >>> xsi:schemaLocation="http://www.springframework.org/schema/beans >>> >>> http://www.springframework.org/schema/beans/spring-beans-2.5.xsd";> >>> >>> >>> >>> <bean >>> >>> >>>class="org.apache.oodt.commons.spring.postprocessor.SetIdBeanPostProcess >>>o >>> >>> r"/> >>> >>> >>> >>> <!-- Crawler Actions --> >>> >>> >>> >>> <bean id="MyPostIngestSuccessAction" lazy-init="true" >>> >>> class="gov.nasa.jpl.crawler.action.MyPostIngestSuccessAction"> >>> >>> <property name="description" value="Kickoff Another Workflow >>> >>> Event"/> >>> >>> <property name="executeCommand" value="sh >>> >>> [WORKFLOWMGR_HOME]/bin/run_L1B_S0_LoRes.sh"/> >>> >>> <property name="phases"> >>> >>> <list> >>> >>> <value type="java.lang.String">postIngestSuccess</value> >>> >>> </list> >>> >>> </property> >>> >>> </bean> >>> >>> >>> >>> Not sure if I'm missing anything else. >>> >>> >>> >>> Thanks in advance, >>> >>> Mike >>> >>> >>> >> >>> >> >>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >> Chris Mattmann, Ph.D. >>> >> Senior Computer Scientist >>> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> >> Office: 171-266B, Mailstop: 171-246 >>> >> Email: [email protected] >>> >> WWW: http://sunset.usc.edu/~mattmann/ >>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >> Adjunct Assistant Professor, Computer Science Department >>> >> University of Southern California, Los Angeles, CA 90089 USA >>> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >> >>> > >>> >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Chris Mattmann, Ph.D. >>> Senior Computer Scientist >>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> Office: 171-266B, Mailstop: 171-246 >>> Email: [email protected] >>> WWW: http://sunset.usc.edu/~mattmann/ >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Adjunct Assistant Professor, Computer Science Department >>> University of Southern California, Los Angeles, CA 90089 USA >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >
