Sorry Professor, the file I had sent previously is resmgr script. But that seems like the right file to add the two lines you mentioned. I restarted the res mgr after adding the lines, still getting the same exception.
Here is the link for batch_stub http://pastebin.com/tfRCtJj0 Thanks! - Renu On Wed, Nov 5, 2014 at 10:01 PM, Christian Alan Mattmann <mattm...@usc.edu> wrote: > Can you add lines: > > -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \ > -Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \ > > Starting at line 92 in your batch_stub script, restart > the batch stub and see if that fixes it? > > Cheers, > Chris > > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Adjunct Associate Professor, Computer Science Department > University of Southern California > Los Angeles, CA 90089 USA > Email: mattm...@usc.edu > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > -----Original Message----- > From: Renu Kanakamedala <kanak...@usc.edu> > Date: Wednesday, November 5, 2014 at 10:51 PM > To: Chris Mattmann <mattm...@usc.edu> > Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org" > <dev@oodt.apache.org>, Preethi Ramesh <pram...@usc.edu>, Gouthami > Kondakindi <konda...@usc.edu> > Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > parse null Error. > > >wmgr script > > > >http://pastebin.com/zE7PWnXL > > > > > >resmgr batch_stub > > > > > >http://pastebin.com/gPUvbE9e > > > > > >- Renu > > > > > > > > > > > >On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann > ><mattm...@usc.edu> wrote: > > > >Great. One last question. > > > >Can you show me the contents (paste.bin) of your wmgr > >script and your batch_stub script? > > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >Chris Mattmann, Ph.D. > >Adjunct Associate Professor, Computer Science Department > >University of Southern California > >Los Angeles, CA 90089 USA > >Email: mattm...@usc.edu > >WWW: http://sunset.usc.edu/~mattmann/ > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > > > >-----Original Message----- > >From: Renu Kanakamedala <kanak...@usc.edu> > >Date: Wednesday, November 5, 2014 at 10:20 PM > >To: Chris Mattmann <mattm...@usc.edu> > >Cc: Meghamsh Utkur <ut...@usc.edu>, "dev@oodt.apache.org" > ><dev@oodt.apache.org>, Preethi Ramesh <pram...@usc.edu>, Gouthami > >Kondakindi <konda...@usc.edu> > >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >parse null Error. > > > >>renu@ubuntu:~$ echo $WORKFLOW_HOME > >>/home/renu/oodt/workflow/ > >>renu@ubuntu:~$ echo $FILEMGR_HOME > >>/home/renu/oodt/filemgr > >>renu@ubuntu:~$ echo $WORKFLOW_URL > >>http://localhost:9001 > >>renu@ubuntu:~$ echo $FILEMGR_URL > >>http://localhost:9000 > >>renu@ubuntu:~$ echo $PGE_ROOT > >>/home/renu/oodt/pge/ > >> > >>renu@ubuntu:~$ echo $CRAWLER_HOME > >>/home/renu/oodt/crawler/ > >> > >> > >> > >> > >> > >> > >> > >>On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann > >><mattm...@usc.edu> wrote: > >> > >>Right, which is why I asked to specifically echo the value > >>of all environment variables so we would be able to see the > >>values. Can Renu please show the echo values? > >> > >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>Chris Mattmann, Ph.D. > >>Adjunct Associate Professor, Computer Science Department > >>University of Southern California > >>Los Angeles, CA 90089 USA > >>Email: mattm...@usc.edu > >>WWW: http://sunset.usc.edu/~mattmann/ > >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> > >> > >> > >> > >>-----Original Message----- > >>From: Meghamsh Utkur <ut...@usc.edu> > >>Date: Wednesday, November 5, 2014 at 9:35 PM > >>To: Renu Kanakamedala <kanak...@usc.edu> > >>Cc: Chris Mattmann <mattm...@usc.edu>, "dev@oodt.apache.org" > >><dev@oodt.apache.org>, Preethi Ramesh <pram...@usc.edu>, Gouthami > >>Kondakindi <konda...@usc.edu> > >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >>parse null Error. > >> > >>>Professor, > >>> > >>> > >>>To add to what Renu has said, I went through the source code and I > >>>suspect that the value here in this method is passed as null. > >>> > >>>XmlFilePgeConfigBuilder.java > >>> > >>>1. private String fillIn(String value,Metadata inputMetadata,boolean > >>>envReplaceRecur) > >>>2. throwsException { > >>>3. try{ > >>>4. while((value =PathUtils.doDynamicReplacement(value, > >>>inputMetadata)).contains("[")&& envReplaceRecur); > >>>5. if(value.toUpperCase() > >>>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$")) > >>>7. value > >>>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new > >>>URL(inputMetadata > >>>8. > >>>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser. > >>>p > >>>a > >>>rseSqlQueryMethod(value))); > >>>9. return value; > >>>10. }catch(Exception e){ > >>>11. thrownew Exception("Failed to parse value: "+ value, e); > >>>12. } > >>>13. } > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala > >>><kanak...@usc.edu> wrote: > >>> > >>>Hello Professor, > >>> > >>> > >>>All the environment variables present in tasks.xml i.e. > >>>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined. > >>> > >>> > >>>Please let us know how to proceed. > >>> > >>> > >>>Regards, > >>>Renu > >>> > >>> > >>>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann > >>><mattm...@usc.edu> wrote: > >>> > >>>Can you please confirm that all of the bracketed environment > >>>variables below do in fact exist in the shell that you are > >>>running ./oodt start in? > >>> > >>>E.g., please echo $ENV_VAR below and show their values. > >>> > >>>Cheers, > >>>Chris > >>> > >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>Chris Mattmann, Ph.D. > >>>Adjunct Associate Professor, Computer Science Department > >>>University of Southern California > >>>Los Angeles, CA 90089 USA > >>>Email: mattm...@usc.edu > >>>WWW: http://sunset.usc.edu/~mattmann/ > >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>> > >>> > >>> > >>> > >>>-----Original Message----- > >>>From: Renu Kanakamedala <kanak...@usc.edu> > >>>Date: Wednesday, November 5, 2014 at 8:18 PM > >>>To: Chris Mattmann <mattm...@usc.edu> > >>>Cc: "dev@oodt.apache.org" <dev@oodt.apache.org>, Preethi Ramesh > >>><pram...@usc.edu>, Gouthami Kondakindi <konda...@usc.edu> > >>>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >>>parse null Error. > >>> > >>>>Please find below tasks.xml file. > >>>> > >>>> > >>>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";> > >>>><!-- > >>>> TODO: Add some examples > >>>>--> > >>>><task id="urn:oodt:FileConcatenator" name="FileConcatenator" > >>>> > >>>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPG > >>>>E > >>>>T > >>>>a > >>>>sk"> > >>>> > >>>> <conditions/> > >>>> > >>>> <configuration> > >>>> <property name="PGETask_Name" value="FileConcatenator"/> > >>>> <property name="PGETask_ConfigFilePath" > >>>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml" > >>>>envReplace="true"/> > >>>> <property name="PGETask_DumpMetadata" value="true"/> > >>>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]" > >>>>envReplace="true" /> > >>>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]" > >>>>envReplace="true"/> > >>>> <property name="PCS_MetFileExtension" value="met"/> > >>>> <property name="PCS_ClientTransferServiceFactory" > >>>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactor > >>>>y > >>>>" > >>>>/ > >>>>> > >>>> <property name="PCS_ActionRepoFile" > >>>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" > >>>>envReplace="true"/> > >>>> </configuration> > >>>> > >>>> <requiredMetFields> > >>>> <metfield name="RunID"/> > >>>> </requiredMetFields> > >>>> > >>>></task> > >>>></cas:tasks> > >>>> > >>>> > >>>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann > >>>><mattm...@usc.edu> wrote: > >>>> > >>>>Hi Renu, > >>>> > >>>>Can you please show a paste.bin of your tasks.xml? > >>>> > >>>>Cheers, > >>>>Chris > >>>> > >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>>Chris Mattmann, Ph.D. > >>>>Adjunct Associate Professor, Computer Science Department > >>>>University of Southern California > >>>>Los Angeles, CA 90089 USA > >>>>Email: mattm...@usc.edu > >>>>WWW: http://sunset.usc.edu/~mattmann/ > >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>> > >>>> > >>>> > >>>> > >>>>-----Original Message----- > >>>>From: Renu Kanakamedala <kanak...@usc.edu> > >>>>Date: Wednesday, November 5, 2014 at 7:55 PM > >>>>To: Chris Mattmann <mattm...@usc.edu> > >>>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >>>>parse > >>>>null Error. > >>>> > >>>>>Hello Professor, > >>> > >>>>> > >>>>> > >>>>>When i am trying to run file concatenator CAS PGE example using the > >>>>>following command. > >>>>> > >>>>> > >>>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent > >>>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1 > >>>>> > >>>>> > >>>>> > >>>>>The message shown after executing this command is "Sending Event > >>>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and > >>>>>batch > >>>>>stub. > >>>>> > >>>>> > >>>>>Batch Stub: > >>>>>Caused by: > >>>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceEx > >>>>>c > >>>>>e > >>>>>p > >>>>>t > >>>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse > >>>>>value: null > >>>>>at > >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) > >>>>>at > >>>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > >>>>>... 2 more > >>>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to > >>>>>parse value: null > >>>>>at > >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > >>>>>e > >>>>>C > >>>>>o > >>>>>n > >>>>>figBuilder.java:87) > >>>>>at > >>>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc > >>>>>e > >>>>>. > >>>>>j > >>>>>a > >>>>>va:230) > >>>>>at > >>>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) > >>>>>... 3 more > >>>>>Caused by: java.lang.Exception: Failed to parse value: null > >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) > >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) > >>>>>at > >>>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > >>>>>e > >>>>>C > >>>>>o > >>>>>n > >>>>>figBuilder.java:77) > >>>>>... 5 more > >>>>>Caused by: java.lang.NullPointerException > >>>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234) > >>>>>at java.util.regex.Matcher.reset(Matcher.java:308) > >>>>>at java.util.regex.Matcher.<init>(Matcher.java:228) > >>>>>at java.util.regex.Pattern.matcher(Pattern.java:1088) > >>>>>at > >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla > >>>>>c > >>>>>e > >>>>>m > >>>>>e > >>>>>nt(PathUtils.java:321) > >>>>>at > >>>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU > >>>>>t > >>>>>i > >>>>>l > >>>>>s > >>>>>.java:96) > >>>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488) > >>>>>... 7 more > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>RES mngr: > >>>>> > >>>>> > >>>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack > >>>>>addJob > >>>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue > >>>>>Nov 05, 2014 5:26:30 PM > >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run > >>>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue: > >>>>>Scheduling for execution > >>>>>Nov 05, 2014 5:26:30 PM > >>>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule > >>>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost] > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>plainConnect > >>>>>FINEST: ProxySelector Request for > >>>> > >>>> > >>> > >>> > >>>>>http://localhost:2001/ <http://localhost:2001/> > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>plainConnect > >>>>>FINEST: Proxy used: DIRECT > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>writeRequests > >>>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1: > >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: > >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: > >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*; > >>>>> q=.2}{Connection: keep-alive}{Content-Length: 105} > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>getInputStream > >>>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200 > >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: > >>>>>text/xml}{Content-Length: 121} > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>plainConnect > >>>>>FINEST: ProxySelector Request for > >>>>>http://localhost:2001/ <http://localhost:2001/> > >>> > >>>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>plainConnect > >>>>>FINEST: Proxy used: DIRECT > >>>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>writeRequests > >>>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1: > >>>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: > >>>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: > >>>>>text/html, image/gif, image/jpeg, *; q=.2, */*; > >>>>> q=.2}{Connection: keep-alive}{Content-Length: 2536} > >>>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection > >>>>>getInputStream > >>>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200 > >>>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: > >>>>>text/xml}{Content-Length: 121} > >>>>>Nov 05, 2014 5:26:32 PM > >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run > >>>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' > >>>>>: > >>>>>batchstub.executeJob returned false > >>>>>java.lang.Exception: batchstub.executeJob returned false > >>>>>at > >>>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBa > >>>>>t > >>>>>c > >>>>>h > >>>>>M > >>>>>grProxy.java:125) > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>Workflow: > >>>>> > >>>>> > >>>>>Nov 05, 2014 5:26:27 PM > >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > >>>>>checkTaskRequiredMetadata > >>>>>INFO: All required metadata fields present for task: > >>>>>[FileConcatenator] > >>>>>Nov 05, 2014 5:26:28 PM > >>>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient > >>>>>submitJob > >>>>>FINEST: [{job.queueName=high, > >>>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, > >>>> > >>>> > >>> > >>> > >>> > >>> > >>>>>job.name <http://job.name> <http://job.name> <http://job.name> > >>>>><http://job.name> > >>>>><http://job.name>=urn:oodt:FileConcatenator, > >> > >> > >>>>>job.id <http://job.id> <http://job.id> <http://job.id> <http://job.id > > > >>>>><http://job.id>=, job.status=, > >>>>>job.load=2, > >>>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, > >>>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.Fil > >>>>>e > >>>>>C > >>>>>o > >>>>>n > >>>>>catenatorPGETask, > >>>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs > >>>>>/ > >>>>>P > >>>>>G > >>>>>E > >>>>>Config.xml, > >>>>> > >>>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatrans > >>>>>f > >>>>>e > >>>>>r > >>>>>. > >>>>>LocalDataTransferFactory, > >>>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config. > >>>>>x > >>>>>m > >>>>>l > >>>>>, > >>>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true, > >>>>>PCS_WorkflowManagerUrl=http://localhost:9001, > >>>>> PCS_FileManagerUrl=http://localhost:9000, > >>>>>PGETask_Name=FileConcatenator}, > >>>>>task.metadata={TaskId=[urn:oodt:FileConcatenator], > >>>>>WorkflowManagerUrl=[http://ubuntu:9001], > >>>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95], > >>>>> ProcessingNode=[ubuntu], RunID=[testNumber1], > >>>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}] > >>>>>Nov 05, 2014 5:26:39 PM > >>>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > >>>>>stop > >>>>>WARNING: Attempt to kill current resmgr job: > >>>>>[2014-11-05T17:26:28.785-08:00]: failed > >>>>> > >>>>> > >>>>>When i searched online i found one of your mail archives where you had > >>>>>mentioned to change tasks.xml > >>>>>https://www.mail-archive.com/dev@oodt.apache.org/msg04403.html > >>>>> > >>>>> > >>>>> > >>>>>It did not work. Please let us know how to proceed. > >>>>> > >>>>> > >>>>> > >>>>>Regards, > >>>>>Renu > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>>-- > >> > >> > >>>Meghamsh UtkurGraduate Student,University of Southern California > >>>ut...@usc.edu > >>> > > > > > >>>(503)881-4791 <tel:%28503%29881-4791> <tel:%28503%29881-4791> > >>> > >>> > >>> > >> > >> > >> > >> > >> > >> > > > > > > > > > > > > > >
