Can you add lines: -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \ -Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \
Starting at line 92 in your batch_stub script, restart the batch stub and see if that fixes it? Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Renu Kanakamedala <[email protected]> Date: Wednesday, November 5, 2014 at 10:51 PM To: Chris Mattmann <[email protected]> Cc: Meghamsh Utkur <[email protected]>, "[email protected]" <[email protected]>, Preethi Ramesh <[email protected]>, Gouthami Kondakindi <[email protected]> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error. >wmgr script > >http://pastebin.com/zE7PWnXL > > >resmgr batch_stub > > >http://pastebin.com/gPUvbE9e > > >- Renu > > > > > >On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann ><[email protected]> wrote: > >Great. One last question. > >Can you show me the contents (paste.bin) of your wmgr >script and your batch_stub script? > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Adjunct Associate Professor, Computer Science Department >University of Southern California >Los Angeles, CA 90089 USA >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > >-----Original Message----- >From: Renu Kanakamedala <[email protected]> >Date: Wednesday, November 5, 2014 at 10:20 PM >To: Chris Mattmann <[email protected]> >Cc: Meghamsh Utkur <[email protected]>, "[email protected]" ><[email protected]>, Preethi Ramesh <[email protected]>, Gouthami >Kondakindi <[email protected]> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >parse null Error. > >>renu@ubuntu:~$ echo $WORKFLOW_HOME >>/home/renu/oodt/workflow/ >>renu@ubuntu:~$ echo $FILEMGR_HOME >>/home/renu/oodt/filemgr >>renu@ubuntu:~$ echo $WORKFLOW_URL >>http://localhost:9001 >>renu@ubuntu:~$ echo $FILEMGR_URL >>http://localhost:9000 >>renu@ubuntu:~$ echo $PGE_ROOT >>/home/renu/oodt/pge/ >> >>renu@ubuntu:~$ echo $CRAWLER_HOME >>/home/renu/oodt/crawler/ >> >> >> >> >> >> >> >>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann >><[email protected]> wrote: >> >>Right, which is why I asked to specifically echo the value >>of all environment variables so we would be able to see the >>values. Can Renu please show the echo values? >> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>Chris Mattmann, Ph.D. >>Adjunct Associate Professor, Computer Science Department >>University of Southern California >>Los Angeles, CA 90089 USA >>Email: [email protected] >>WWW: http://sunset.usc.edu/~mattmann/ >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >>-----Original Message----- >>From: Meghamsh Utkur <[email protected]> >>Date: Wednesday, November 5, 2014 at 9:35 PM >>To: Renu Kanakamedala <[email protected]> >>Cc: Chris Mattmann <[email protected]>, "[email protected]" >><[email protected]>, Preethi Ramesh <[email protected]>, Gouthami >>Kondakindi <[email protected]> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >>parse null Error. >> >>>Professor, >>> >>> >>>To add to what Renu has said, I went through the source code and I >>>suspect that the value here in this method is passed as null. >>> >>>XmlFilePgeConfigBuilder.java >>> >>>1. private String fillIn(String value,Metadata inputMetadata,boolean >>>envReplaceRecur) >>>2. throwsException { >>>3. try{ >>>4. while((value =PathUtils.doDynamicReplacement(value, >>>inputMetadata)).contains("[")&& envReplaceRecur); >>>5. if(value.toUpperCase() >>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$")) >>>7. value >>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new >>>URL(inputMetadata >>>8. >>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser. >>>p >>>a >>>rseSqlQueryMethod(value))); >>>9. return value; >>>10. }catch(Exception e){ >>>11. thrownew Exception("Failed to parse value: "+ value, e); >>>12. } >>>13. } >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala >>><[email protected]> wrote: >>> >>>Hello Professor, >>> >>> >>>All the environment variables present in tasks.xml i.e. >>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined. >>> >>> >>>Please let us know how to proceed. >>> >>> >>>Regards, >>>Renu >>> >>> >>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann >>><[email protected]> wrote: >>> >>>Can you please confirm that all of the bracketed environment >>>variables below do in fact exist in the shell that you are >>>running ./oodt start in? >>> >>>E.g., please echo $ENV_VAR below and show their values. >>> >>>Cheers, >>>Chris >>> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>Chris Mattmann, Ph.D. >>>Adjunct Associate Professor, Computer Science Department >>>University of Southern California >>>Los Angeles, CA 90089 USA >>>Email: [email protected] >>>WWW: http://sunset.usc.edu/~mattmann/ >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> >>> >>> >>>-----Original Message----- >>>From: Renu Kanakamedala <[email protected]> >>>Date: Wednesday, November 5, 2014 at 8:18 PM >>>To: Chris Mattmann <[email protected]> >>>Cc: "[email protected]" <[email protected]>, Preethi Ramesh >>><[email protected]>, Gouthami Kondakindi <[email protected]> >>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >>>parse null Error. >>> >>>>Please find below tasks.xml file. >>>> >>>> >>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";> >>>><!-- >>>> TODO: Add some examples >>>>--> >>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator" >>>> >>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPG >>>>E >>>>T >>>>a >>>>sk"> >>>> >>>> <conditions/> >>>> >>>> <configuration> >>>> <property name="PGETask_Name" value="FileConcatenator"/> >>>> <property name="PGETask_ConfigFilePath" >>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml" >>>>envReplace="true"/> >>>> <property name="PGETask_DumpMetadata" value="true"/> >>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]" >>>>envReplace="true" /> >>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]" >>>>envReplace="true"/> >>>> <property name="PCS_MetFileExtension" value="met"/> >>>> <property name="PCS_ClientTransferServiceFactory" >>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactor >>>>y >>>>" >>>>/ >>>>> >>>> <property name="PCS_ActionRepoFile" >>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" >>>>envReplace="true"/> >>>> </configuration> >>>> >>>> <requiredMetFields> >>>> <metfield name="RunID"/> >>>> </requiredMetFields> >>>> >>>></task> >>>></cas:tasks> >>>> >>>> >>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann >>>><[email protected]> wrote: >>>> >>>>Hi Renu, >>>> >>>>Can you please show a paste.bin of your tasks.xml? >>>> >>>>Cheers, >>>>Chris >>>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>Chris Mattmann, Ph.D. >>>>Adjunct Associate Professor, Computer Science Department >>>>University of Southern California >>>>Los Angeles, CA 90089 USA >>>>Email: [email protected] >>>>WWW: http://sunset.usc.edu/~mattmann/ >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>> >>>> >>>> >>>> >>>>-----Original Message----- >>>>From: Renu Kanakamedala <[email protected]> >>>>Date: Wednesday, November 5, 2014 at 7:55 PM >>>>To: Chris Mattmann <[email protected]> >>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >>>>parse >>>>null Error. >>>> >>>>>Hello Professor, >>> >>>>> >>>>> >>>>>When i am trying to run file concatenator CAS PGE example using the >>>>>following command. >>>>> >>>>> >>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent >>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1 >>>>> >>>>> >>>>> >>>>>The message shown after executing this command is "Sending Event >>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and >>>>>batch >>>>>stub. >>>>> >>>>> >>>>>Batch Stub: >>>>>Caused by: >>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx >>>>>c >>>>>e >>>>>p >>>>>t >>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse >>>>>value: null >>>>>at >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) >>>>>at >>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) >>>>>... 2 more >>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to >>>>>parse value: null >>>>>at >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >>>>>e >>>>>C >>>>>o >>>>>n >>>>>figBuilder.java:87) >>>>>at >>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc >>>>>e >>>>>. >>>>>j >>>>>a >>>>>va:230) >>>>>at >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) >>>>>... 3 more >>>>>Caused by: java.lang.Exception: Failed to parse value: null >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) >>>>>at >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >>>>>e >>>>>C >>>>>o >>>>>n >>>>>figBuilder.java:77) >>>>>... 5 more >>>>>Caused by: java.lang.NullPointerException >>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234) >>>>>at java.util.regex.Matcher.reset(Matcher.java:308) >>>>>at java.util.regex.Matcher.<init>(Matcher.java:228) >>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088) >>>>>at >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla >>>>>c >>>>>e >>>>>m >>>>>e >>>>>nt(PathUtils.java:321) >>>>>at >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU >>>>>t >>>>>i >>>>>l >>>>>s >>>>>.java:96) >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488) >>>>>... 7 more >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>RES mngr: >>>>> >>>>> >>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack >>>>>addJob >>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue >>>>>Nov 05, 2014 5:26:30 PM >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run >>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue: >>>>>Scheduling for execution >>>>>Nov 05, 2014 5:26:30 PM >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule >>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost] >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>plainConnect >>>>>FINEST: ProxySelector Request for >>>> >>>> >>> >>> >>>>>http://localhost:2001/ <http://localhost:2001/> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>plainConnect >>>>>FINEST: Proxy used: DIRECT >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>writeRequests >>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1: >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*; >>>>> q=.2}{Connection: keep-alive}{Content-Length: 105} >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>getInputStream >>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200 >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: >>>>>text/xml}{Content-Length: 121} >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>plainConnect >>>>>FINEST: ProxySelector Request for >>>>>http://localhost:2001/ <http://localhost:2001/> >>> >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>>plainConnect >>>>>FINEST: Proxy used: DIRECT >>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection >>>>>writeRequests >>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1: >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*; >>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536} >>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection >>>>>getInputStream >>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200 >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: >>>>>text/xml}{Content-Length: 121} >>>>>Nov 05, 2014 5:26:32 PM >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run >>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' >>>>>: >>>>>batchstub.executeJob returned false >>>>>java.lang.Exception: batchstub.executeJob returned false >>>>>at >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBa >>>>>t >>>>>c >>>>>h >>>>>M >>>>>grProxy.java:125) >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>Workflow: >>>>> >>>>> >>>>>Nov 05, 2014 5:26:27 PM >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >>>>>checkTaskRequiredMetadata >>>>>INFO: All required metadata fields present for task: >>>>>[FileConcatenator] >>>>>Nov 05, 2014 5:26:28 PM >>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient >>>>>submitJob >>>>>FINEST: [{job.queueName=high, >>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, >>>> >>>> >>> >>> >>> >>> >>>>>job.name <http://job.name> <http://job.name> <http://job.name> >>>>><http://job.name> >>>>><http://job.name>=urn:oodt:FileConcatenator, >> >> >>>>>job.id <http://job.id> <http://job.id> <http://job.id> <http://job.id> >>>>><http://job.id>=, job.status=, >>>>>job.load=2, >>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, >>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil >>>>>e >>>>>C >>>>>o >>>>>n >>>>>catenatorPGETask, >>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs >>>>>/ >>>>>P >>>>>G >>>>>E >>>>>Config.xml, >>>>> >>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans >>>>>f >>>>>e >>>>>r >>>>>. >>>>>LocalDataTransferFactory, >>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config. >>>>>x >>>>>m >>>>>l >>>>>, >>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true, >>>>>PCS_WorkflowManagerUrl=http://localhost:9001, >>>>> PCS_FileManagerUrl=http://localhost:9000, >>>>>PGETask_Name=FileConcatenator}, >>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator], >>>>>WorkflowManagerUrl=[http://ubuntu:9001], >>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95], >>>>> ProcessingNode=[ubuntu], RunID=[testNumber1], >>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}] >>>>>Nov 05, 2014 5:26:39 PM >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >>>>>stop >>>>>WARNING: Attempt to kill current resmgr job: >>>>>[2014-11-05T17:26:28.785-08:00]: failed >>>>> >>>>> >>>>>When i searched online i found one of your mail archives where you had >>>>>mentioned to change tasks.xml >>>>>https://www.mail-archive.com/[email protected]/msg04403.html >>>>> >>>>> >>>>> >>>>>It did not work. Please let us know how to proceed. >>>>> >>>>> >>>>> >>>>>Regards, >>>>>Renu >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>-- >> >> >>>Meghamsh UtkurGraduate Student,University of Southern California >>>[email protected] >>> > > >>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791> >>> >>> >>> >> >> >> >> >> >> > > > > > >
