*wmgr script * http://pastebin.com/zE7PWnXL
*resmgr batch_stub* http://pastebin.com/gPUvbE9e - Renu On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann <[email protected]> wrote: > Great. One last question. > > Can you show me the contents (paste.bin) of your wmgr > script and your batch_stub script? > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Adjunct Associate Professor, Computer Science Department > University of Southern California > Los Angeles, CA 90089 USA > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > -----Original Message----- > From: Renu Kanakamedala <[email protected]> > Date: Wednesday, November 5, 2014 at 10:20 PM > To: Chris Mattmann <[email protected]> > Cc: Meghamsh Utkur <[email protected]>, "[email protected]" > <[email protected]>, Preethi Ramesh <[email protected]>, Gouthami > Kondakindi <[email protected]> > Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > parse null Error. > > >renu@ubuntu:~$ echo $WORKFLOW_HOME > >/home/renu/oodt/workflow/ > >renu@ubuntu:~$ echo $FILEMGR_HOME > >/home/renu/oodt/filemgr > >renu@ubuntu:~$ echo $WORKFLOW_URL > >http://localhost:9001 > >renu@ubuntu:~$ echo $FILEMGR_URL > >http://localhost:9000 > >renu@ubuntu:~$ echo $PGE_ROOT > >/home/renu/oodt/pge/ > > > >renu@ubuntu:~$ echo $CRAWLER_HOME > >/home/renu/oodt/crawler/ > > > > > > > > > > > > > > > >On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann > ><[email protected]> wrote: > > > >Right, which is why I asked to specifically echo the value > >of all environment variables so we would be able to see the > >values. Can Renu please show the echo values? > > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >Chris Mattmann, Ph.D. > >Adjunct Associate Professor, Computer Science Department > >University of Southern California > >Los Angeles, CA 90089 USA > >Email: [email protected] > >WWW: http://sunset.usc.edu/~mattmann/ > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > > > >-----Original Message----- > >From: Meghamsh Utkur <[email protected]> > >Date: Wednesday, November 5, 2014 at 9:35 PM > >To: Renu Kanakamedala <[email protected]> > >Cc: Chris Mattmann <[email protected]>, "[email protected]" > ><[email protected]>, Preethi Ramesh <[email protected]>, Gouthami > >Kondakindi <[email protected]> > >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >parse null Error. > > > >>Professor, > >> > >> > >>To add to what Renu has said, I went through the source code and I > >>suspect that the value here in this method is passed as null. > >> > >>XmlFilePgeConfigBuilder.java > >> > >>1. private String fillIn(String value,Metadata inputMetadata,boolean > >>envReplaceRecur) > >>2. throwsException { > >>3. try{ > >>4. while((value =PathUtils.doDynamicReplacement(value, > >>inputMetadata)).contains("[")&& envReplaceRecur); > >>5. if(value.toUpperCase() > >>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$")) > >>7. value > >>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new > >>URL(inputMetadata > >>8. > >>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.p > >>a > >>rseSqlQueryMethod(value))); > >>9. return value; > >>10. }catch(Exception e){ > >>11. thrownew Exception("Failed to parse value: "+ value, e); > >>12. } > >>13. } > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala > >><[email protected]> wrote: > >> > >>Hello Professor, > >> > >> > >>All the environment variables present in tasks.xml i.e. > >>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined. > >> > >> > >>Please let us know how to proceed. > >> > >> > >>Regards, > >>Renu > >> > >> > >>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann > >><[email protected]> wrote: > >> > >>Can you please confirm that all of the bracketed environment > >>variables below do in fact exist in the shell that you are > >>running ./oodt start in? > >> > >>E.g., please echo $ENV_VAR below and show their values. > >> > >>Cheers, > >>Chris > >> > >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>Chris Mattmann, Ph.D. > >>Adjunct Associate Professor, Computer Science Department > >>University of Southern California > >>Los Angeles, CA 90089 USA > >>Email: [email protected] > >>WWW: http://sunset.usc.edu/~mattmann/ > >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> > >> > >> > >> > >>-----Original Message----- > >>From: Renu Kanakamedala <[email protected]> > >>Date: Wednesday, November 5, 2014 at 8:18 PM > >>To: Chris Mattmann <[email protected]> > >>Cc: "[email protected]" <[email protected]>, Preethi Ramesh > >><[email protected]>, Gouthami Kondakindi <[email protected]> > >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >>parse null Error. > >> > >>>Please find below tasks.xml file. > >>> > >>> > >>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";> > >>><!-- > >>> TODO: Add some examples > >>>--> > >>><task id="urn:oodt:FileConcatenator" name="FileConcatenator" > >>> > >>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE > >>>T > >>>a > >>>sk"> > >>> > >>> <conditions/> > >>> > >>> <configuration> > >>> <property name="PGETask_Name" value="FileConcatenator"/> > >>> <property name="PGETask_ConfigFilePath" > >>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml" > >>>envReplace="true"/> > >>> <property name="PGETask_DumpMetadata" value="true"/> > >>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]" > >>>envReplace="true" /> > >>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]" > >>>envReplace="true"/> > >>> <property name="PCS_MetFileExtension" value="met"/> > >>> <property name="PCS_ClientTransferServiceFactory" > >>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory > >>>" > >>>/ > >>>> > >>> <property name="PCS_ActionRepoFile" > >>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" > >>>envReplace="true"/> > >>> </configuration> > >>> > >>> <requiredMetFields> > >>> <metfield name="RunID"/> > >>> </requiredMetFields> > >>> > >>></task> > >>></cas:tasks> > >>> > >>> > >>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann > >>><[email protected]> wrote: > >>> > >>>Hi Renu, > >>> > >>>Can you please show a paste.bin of your tasks.xml? > >>> > >>>Cheers, > >>>Chris > >>> > >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>Chris Mattmann, Ph.D. > >>>Adjunct Associate Professor, Computer Science Department > >>>University of Southern California > >>>Los Angeles, CA 90089 USA > >>>Email: [email protected] > >>>WWW: http://sunset.usc.edu/~mattmann/ > >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>> > >>> > >>> > >>> > >>>-----Original Message----- > >>>From: Renu Kanakamedala <[email protected]> > >>>Date: Wednesday, November 5, 2014 at 7:55 PM > >>>To: Chris Mattmann <[email protected]> > >>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to > >>>parse > >>>null Error. > >>> > >>>>Hello Professor, > >> > >>>> > >>>> > >>>>When i am trying to run file concatenator CAS PGE example using the > >>>>following command. > >>>> > >>>> > >>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent > >>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1 > >>>> > >>>> > >>>> > >>>>The message shown after executing this command is "Sending Event > >>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch > >>>>stub. > >>>> > >>>> > >>>>Batch Stub: > >>>>Caused by: > >>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc > >>>>e > >>>>p > >>>>t > >>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse > >>>>value: null > >>>>at > >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) > >>>>at > >>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > >>>>... 2 more > >>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to > >>>>parse value: null > >>>>at > >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge > >>>>C > >>>>o > >>>>n > >>>>figBuilder.java:87) > >>>>at > >>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance > >>>>. > >>>>j > >>>>a > >>>>va:230) > >>>>at > >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) > >>>>... 3 more > >>>>Caused by: java.lang.Exception: Failed to parse value: null > >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) > >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) > >>>>at > >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge > >>>>C > >>>>o > >>>>n > >>>>figBuilder.java:77) > >>>>... 5 more > >>>>Caused by: java.lang.NullPointerException > >>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234) > >>>>at java.util.regex.Matcher.reset(Matcher.java:308) > >>>>at java.util.regex.Matcher.<init>(Matcher.java:228) > >>>>at java.util.regex.Pattern.matcher(Pattern.java:1088) > >>>>at > >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac > >>>>e > >>>>m > >>>>e > >>>>nt(PathUtils.java:321) > >>>>at > >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt > >>>>i > >>>>l > >>>>s > >>>>.java:96) > >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488) > >>>>... 7 more > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>RES mngr: > >>>> > >>>> > >>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack > >>>>addJob > >>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue > >>>>Nov 05, 2014 5:26:30 PM > >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run > >>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue: > >>>>Scheduling for execution > >>>>Nov 05, 2014 5:26:30 PM > >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule > >>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost] > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>plainConnect > >>>>FINEST: ProxySelector Request for > >>> > >>> > >> > >> > >>>>http://localhost:2001/ <http://localhost:2001/> > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>plainConnect > >>>>FINEST: Proxy used: DIRECT > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>writeRequests > >>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1: > >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: > >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: > >>>>text/html, image/gif, image/jpeg, *; q=.2, */*; > >>>> q=.2}{Connection: keep-alive}{Content-Length: 105} > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>getInputStream > >>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200 > >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: > >>>>text/xml}{Content-Length: 121} > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>plainConnect > >>>>FINEST: ProxySelector Request for > >>>>http://localhost:2001/ <http://localhost:2001/> > >> > >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection > >>>>plainConnect > >>>>FINEST: Proxy used: DIRECT > >>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection > >>>>writeRequests > >>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1: > >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: > >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: > >>>>text/html, image/gif, image/jpeg, *; q=.2, */*; > >>>> q=.2}{Connection: keep-alive}{Content-Length: 2536} > >>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection > >>>>getInputStream > >>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200 > >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: > >>>>text/xml}{Content-Length: 121} > >>>>Nov 05, 2014 5:26:32 PM > >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run > >>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' > >>>>: > >>>>batchstub.executeJob returned false > >>>>java.lang.Exception: batchstub.executeJob returned false > >>>>at > >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat > >>>>c > >>>>h > >>>>M > >>>>grProxy.java:125) > >>>> > >>>> > >>>> > >>>> > >>>> > >>>>Workflow: > >>>> > >>>> > >>>>Nov 05, 2014 5:26:27 PM > >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > >>>>checkTaskRequiredMetadata > >>>>INFO: All required metadata fields present for task: [FileConcatenator] > >>>>Nov 05, 2014 5:26:28 PM > >>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient > >>>>submitJob > >>>>FINEST: [{job.queueName=high, > >>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, > >>> > >>> > >> > >> > >> > >> > >>>>job.name <http://job.name> <http://job.name> <http://job.name> > >>>><http://job.name>=urn:oodt:FileConcatenator, > > > > > >>>>job.id <http://job.id> <http://job.id> <http://job.id> > >>>><http://job.id>=, job.status=, > >>>>job.load=2, > >>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, > >>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File > >>>>C > >>>>o > >>>>n > >>>>catenatorPGETask, > >>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/ > >>>>P > >>>>G > >>>>E > >>>>Config.xml, > >>>> > >>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf > >>>>e > >>>>r > >>>>. > >>>>LocalDataTransferFactory, > >>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x > >>>>m > >>>>l > >>>>, > >>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true, > >>>>PCS_WorkflowManagerUrl=http://localhost:9001, > >>>> PCS_FileManagerUrl=http://localhost:9000, > >>>>PGETask_Name=FileConcatenator}, > >>>>task.metadata={TaskId=[urn:oodt:FileConcatenator], > >>>>WorkflowManagerUrl=[http://ubuntu:9001], > >>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95], > >>>> ProcessingNode=[ubuntu], RunID=[testNumber1], > >>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}] > >>>>Nov 05, 2014 5:26:39 PM > >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > >>>>stop > >>>>WARNING: Attempt to kill current resmgr job: > >>>>[2014-11-05T17:26:28.785-08:00]: failed > >>>> > >>>> > >>>>When i searched online i found one of your mail archives where you had > >>>>mentioned to change tasks.xml > >>>>https://www.mail-archive.com/[email protected]/msg04403.html > >>>> > >>>> > >>>> > >>>>It did not work. Please let us know how to proceed. > >>>> > >>>> > >>>> > >>>>Regards, > >>>>Renu > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >>-- > > > > > >>Meghamsh UtkurGraduate Student,University of Southern California > >>[email protected] > >> > >>(503)881-4791 <tel:%28503%29881-4791> > >> > >> > >> > > > > > > > > > > > > > >
