Yay Chris! Thanks, adding the "split" attribute and moving the list of actions to the pgeConfig (without referencing the list from the workflow task properties) did the trick. Thanks so much. The email thread you pointed me to is the same email thread I referenced in my previous email. The concept of why you would need to put the actions in the pgeConfig is discussed there, but the "split" attribute is not mentioned anywhere. Hence my confusion. I will be happy to update the wiki page with this special little tidbit of information. :)
Ok, so now, I am back to where I was 2-3 weeks ago. I'm crawling for new files, ingesting them to the archive and copying some of the files to a new directory for further processing. The only difference is back then I was using just the crawler and filemgr, and now I'm using CAS-PGE with crawler, filemgr, workflow, and resmgr. Whew! It's been quite an adventure. I really appreciate everyone's help! Now, I have to start working on the more complicated aspects of this pipeline. Val Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied Physics Laboratory > -----Original Message----- > From: Mattmann, Chris A (3980) [mailto:[email protected]] > Sent: Monday, October 20, 2014 11:38 PM > To: [email protected] > Subject: Re: multiple post ingest actions with PGE > > Hey Val, > > I think the answer is here: > > 1. First even though it?s documented on the wiki (incorrectly) that you can > put the > met key split ?,? in the tasks.xml file, you can?t: > http://markmail.org/message/gxnbptf4zamq2qdp > > > 2. The parser for workflow task config doesn?t process the ?split? > attribute. However the CAS-PGE XML config file reader *does*, so if you simply > move PCS_ActionsIds into your PGE config xml file, and you add the > split=?true? > attribute, you can use the comma delimiter to have multiple actions. > > <metadata key="PCS_ActionsIds" > val="TriggerPostIngestWorkflow,DeleteDataFile" split="true"/> > > > Special kudos if you can update the wiki above to have that correct info! :) > > Cheers, > Chris > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) NASA Jet > Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > Adjunct Associate Professor, Computer Science Department University of > Southern California, Los Angeles, CA 90089 USA > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > > > > > > > -----Original Message----- > From: <Mallder>, Valerie <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Monday, October 20, 2014 at 1:05 PM > To: "[email protected]" <[email protected]> > Subject: multiple post ingest actions with PGE > > >Hi All, > >I am trying to run two post ingest success actions after PGE runs the > >crawler and I have come across something that Cameron also came across > >back in February. It thinks my two action names separated by a comma is > >one long bean name. And, despite my best efforts I haven't been able to > >solve this by following the instructions provided by an email thread in > >the mailing list archives here: > > > >http://markmail.org/search/?q=oodt+ACTION_IDS+order%3Adate- > backward#query: > >oodt%20ACTION_IDS%20order%3Adate- > backward+page:2+mid:gxnbptf4zamq2qdp+s > >tat > >e:results > > > >or by following Cameron's subsequent documentation on the wiki here: > > > >https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Help+and+Docum > >ent > >ation > > > >I am sure I am missing something subtle, but I would appreciate your > >help. Here are the lines from my tasks.xml and PgeConfig.xml files. > >And the error I am getting (which is similar to the error Cameron > >received back in February) is below that. > > > >Thanks in advance! > >Val > > > >In tasks.xml > > <property name="PostIngestSuccessActions" > >value="MoveFileToLevel0Dir,MoveMetFileToBackupDir"/> > > > >In PgeConfig.xml > > <customMetadata> > > <metadata key="PCS_ActionsIds" val="[PostIngestSuccessActions]" /> > > </customMetadata> > > > > > >INFO: Updating status to workflow as [CRAWLING] Oct 20, 2014 3:56:52 PM > >org.apache.oodt.cas.pge.PGETaskInstance > >runIngestCrawler > >INFO: Crawling for products in > >[/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/stag > >ing > >] > >Oct 20, 2014 3:56:52 PM org.apache.oodt.cas.pge.PGETaskInstance run > >SEVERE: PGETask FAILED!!! : No bean named > >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined > >org.springframework.beans.factory.NoSuchBeanDefinitionException: No > >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined > > at > >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge > >tBe > >anDefinition(DefaultListableBeanFactory.java:360) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged > >Bea > >nDefinition(AbstractBeanFactory.java:914) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A > >bst > >ractBeanFactory.java:241) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A > >bst > >ractBeanFactory.java:164) > > at > >org.springframework.context.support.AbstractApplicationContext.getBean( > >Abs > >tractApplicationContext.java:761) > > at > >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF > >act > >ory(CrawlerActionRepo.java:71) > > at > >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product > >Cra > >wler.java:228) > > at > >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83) > > at > >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc > >e.j > >ava:575) > > at > >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144) > > at > >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex > >ecu > >teTaskLocally(IterativeWorkflowProcessorThread.java:574) > > at > >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru > >n(I > >terativeWorkflowProcessorThread.java:321) > > at > >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) > > at java.lang.Thread.run(Thread.java:745) > > > >org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc > >ept > >ion: PGETask FAILED!!! : No bean named > >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined > > at > >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) > > at > >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ex > >ecu > >teTaskLocally(IterativeWorkflowProcessorThread.java:574) > > at > >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.ru > >n(I > >terativeWorkflowProcessorThread.java:321) > > at > >EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) > > at java.lang.Thread.run(Thread.java:745) > >Caused by: > >org.springframework.beans.factory.NoSuchBeanDefinitionException: No > >bean named 'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined > > at > >org.springframework.beans.factory.support.DefaultListableBeanFactory.ge > >tBe > >anDefinition(DefaultListableBeanFactory.java:360) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getMerged > >Bea > >nDefinition(AbstractBeanFactory.java:914) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A > >bst > >ractBeanFactory.java:241) > > at > >org.springframework.beans.factory.support.AbstractBeanFactory.getBean(A > >bst > >ractBeanFactory.java:164) > > at > >org.springframework.context.support.AbstractApplicationContext.getBean( > >Abs > >tractApplicationContext.java:761) > > at > >org.apache.oodt.cas.crawl.action.CrawlerActionRepo.loadActionsFromBeanF > >act > >ory(CrawlerActionRepo.java:71) > > at > >org.apache.oodt.cas.crawl.ProductCrawler.loadAndValidateActions(Product > >Cra > >wler.java:228) > > at > >org.apache.oodt.cas.crawl.ProductCrawler.crawl(ProductCrawler.java:83) > > at > >org.apache.oodt.cas.pge.PGETaskInstance.runIngestCrawler(PGETaskInstanc > >e.j > >ava:575) > > at > >org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:144) > > ... 4 more > >Oct 20, 2014 3:56:52 PM > >org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > >executeTaskLocally > >WARNING: Exception executing task: [feiCrawlerTask] locally: Message: > >PGETask FAILED!!! : No bean named > >'MoveFileToLevel0Dir,MoveMetFileToBackupDir' is defined > > > >Valerie A. Mallder > > > >New Horizons Deputy Mission System Engineer The Johns Hopkins > >University/Applied Physics Laboratory > >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 > >240-228-7846 (Office) 410-504-2233 (Blackberry) > >
