Professor,
To add to what Renu has said, I went through the source code and I suspect
that the value here in this method is passed as null.
XmlFilePgeConfigBuilder.java
1. private String fillIn(String value, Metadata inputMetadata, boolean
envReplaceRecur)
2. throws Exception {
3. try {
4. while ((value = PathUtils.doDynamicReplacement(value, inputMetadata)).
contains("[") && envReplaceRecur);
5. if (value.toUpperCase()
6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$"))
7. value = QueryUtils.getQueryResultsAsString(new XmlRpcFileManagerClient
(new URL(inputMetadata
8. .getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(
SqlParser.parseSqlQueryMethod(value)));
9. return value;
10. }catch (Exception e) {
11. throw new Exception("Failed to parse value: " + value, e);
12. }
13. }
On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala <[email protected]> wrote:
> Hello Professor,
>
> All the environment variables present in tasks.xml i.e.
> $WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined.
>
> Please let us know how to proceed.
>
> Regards,
> Renu
>
> On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann <[email protected]>
> wrote:
>
>> Can you please confirm that all of the bracketed environment
>> variables below do in fact exist in the shell that you are
>> running ./oodt start in?
>>
>> E.g., please echo $ENV_VAR below and show their values.
>>
>> Cheers,
>> Chris
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Adjunct Associate Professor, Computer Science Department
>> University of Southern California
>> Los Angeles, CA 90089 USA
>> Email: [email protected]
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>
>>
>>
>>
>> -----Original Message-----
>> From: Renu Kanakamedala <[email protected]>
>> Date: Wednesday, November 5, 2014 at 8:18 PM
>> To: Chris Mattmann <[email protected]>
>> Cc: "[email protected]" <[email protected]>, Preethi Ramesh
>> <[email protected]>, Gouthami Kondakindi <[email protected]>
>> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to
>> parse null Error.
>>
>> >Please find below tasks.xml file.
>> >
>> >
>> ><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";>
>> ><!--
>> > TODO: Add some examples
>> >-->
>> ><task id="urn:oodt:FileConcatenator" name="FileConcatenator"
>> >
>>
>> >class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGETa
>> >sk">
>> >
>> > <conditions/>
>> >
>> > <configuration>
>> > <property name="PGETask_Name" value="FileConcatenator"/>
>> > <property name="PGETask_ConfigFilePath"
>> >value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml"
>> >envReplace="true"/>
>> > <property name="PGETask_DumpMetadata" value="true"/>
>> > <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]"
>> >envReplace="true" />
>> > <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]"
>> >envReplace="true"/>
>> > <property name="PCS_MetFileExtension" value="met"/>
>> > <property name="PCS_ClientTransferServiceFactory"
>>
>> >value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory"/
>> >>
>> > <property name="PCS_ActionRepoFile"
>> >value="file:[CRAWLER_HOME]/policy/crawler-config.xml" envReplace="true"/>
>> > </configuration>
>> >
>> > <requiredMetFields>
>> > <metfield name="RunID"/>
>> > </requiredMetFields>
>> >
>> ></task>
>> ></cas:tasks>
>> >
>> >
>> >On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann
>> ><[email protected]> wrote:
>> >
>> >Hi Renu,
>> >
>> >Can you please show a paste.bin of your tasks.xml?
>> >
>> >Cheers,
>> >Chris
>> >
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >Chris Mattmann, Ph.D.
>> >Adjunct Associate Professor, Computer Science Department
>> >University of Southern California
>> >Los Angeles, CA 90089 USA
>> >Email: [email protected]
>> >WWW: http://sunset.usc.edu/~mattmann/
>> >++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> >-----Original Message-----
>> >From: Renu Kanakamedala <[email protected]>
>> >Date: Wednesday, November 5, 2014 at 7:55 PM
>> >To: Chris Mattmann <[email protected]>
>> >Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse
>> >null Error.
>> >
>> >>Hello Professor,
>>
>> >>
>> >>
>> >>When i am trying to run file concatenator CAS PGE example using the
>> >>following command.
>> >>
>> >>
>> >>./wmgr-client --url http://localhost:9001 --operation --sendEvent
>> >>--eventName fileconcatenator-pge --metaData --key RunID testNumber1
>> >>
>> >>
>> >>
>> >>The message shown after executing this command is "Sending Event
>> >>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch
>> >>stub.
>> >>
>> >>
>> >>Batch Stub:
>> >>Caused by:
>>
>> >>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep
>> >>t
>> >>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse
>> >>value: null
>> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150)
>> >>at
>> org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102)
>> >>... 2 more
>> >>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to
>> >>parse value: null
>> >>at
>>
>> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>> >>n
>> >>figBuilder.java:87)
>> >>at
>>
>> >>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.j
>> >>a
>> >>va:230)
>> >>at org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123)
>> >>... 3 more
>> >>Caused by: java.lang.Exception: Failed to parse value: null
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480)
>> >>at
>>
>> >>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeCo
>> >>n
>> >>figBuilder.java:77)
>> >>... 5 more
>> >>Caused by: java.lang.NullPointerException
>> >>at java.util.regex.Matcher.getTextLength(Matcher.java:1234)
>> >>at java.util.regex.Matcher.reset(Matcher.java:308)
>> >>at java.util.regex.Matcher.<init>(Matcher.java:228)
>> >>at java.util.regex.Pattern.matcher(Pattern.java:1088)
>> >>at
>>
>> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplacem
>> >>e
>> >>nt(PathUtils.java:321)
>> >>at
>>
>> >>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUtil
>> >>s
>> >>.java:96)
>> >>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488)
>> >>... 7 more
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>RES mngr:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack
>> >>addJob
>> >>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler run
>> >>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue:
>> >>Scheduling for execution
>> >>Nov 05, 2014 5:26:30 PM
>> >>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule
>> >>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost]
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >
>> >
>> >>http://localhost:2001/ <http://localhost:2001/>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 105}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: ProxySelector Request for
>> >>http://localhost:2001/ <http://localhost:2001/>
>>
>> >>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection
>> >>plainConnect
>> >>FINEST: Proxy used: DIRECT
>> >>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection
>> >>writeRequests
>> >>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1:
>> >>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma:
>> >>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept:
>> >>text/html, image/gif, image/jpeg, *; q=.2, */*;
>> >> q=.2}{Connection: keep-alive}{Content-Length: 2536}
>> >>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection
>> >>getInputStream
>> >>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200
>> >>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type:
>> >>text/xml}{Content-Length: 121}
>> >>Nov 05, 2014 5:26:32 PM
>> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run
>> >>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' :
>> >>batchstub.executeJob returned false
>> >>java.lang.Exception: batchstub.executeJob returned false
>> >>at
>>
>> >>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatch
>> >>M
>> >>grProxy.java:125)
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>Workflow:
>> >>
>> >>
>> >>Nov 05, 2014 5:26:27 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> >>checkTaskRequiredMetadata
>> >>INFO: All required metadata fields present for task: [FileConcatenator]
>> >>Nov 05, 2014 5:26:28 PM
>> >>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient
>> submitJob
>> >>FINEST: [{job.queueName=high,
>> >>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
>> >
>> >
>> >>job.name <http://job.name> <http://job.name>=urn:oodt:FileConcatenator,
>> >>job.id <http://job.id> <http://job.id>=, job.status=, job.load=2,
>> >>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput},
>>
>> >>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo
>> >>n
>> >>catenatorPGETask,
>>
>> >>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/PG
>> >>E
>> >>Config.xml,
>> >>
>>
>> >>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer
>> >>.
>> >>LocalDataTransferFactory,
>>
>> >>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.xml
>> >>,
>> >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true,
>> >>PCS_WorkflowManagerUrl=http://localhost:9001,
>> >> PCS_FileManagerUrl=http://localhost:9000,
>> >>PGETask_Name=FileConcatenator},
>> >>task.metadata={TaskId=[urn:oodt:FileConcatenator],
>> >>WorkflowManagerUrl=[http://ubuntu:9001],
>> >>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95],
>> >> ProcessingNode=[ubuntu], RunID=[testNumber1],
>> >>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}]
>> >>Nov 05, 2014 5:26:39 PM
>> >>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
>> stop
>> >>WARNING: Attempt to kill current resmgr job:
>> >>[2014-11-05T17:26:28.785-08:00]: failed
>> >>
>> >>
>> >>When i searched online i found one of your mail archives where you had
>> >>mentioned to change tasks.xml
>> >>https://www.mail-archive.com/[email protected]/msg04403.html
>> >>
>> >>
>> >>
>> >>It did not work. Please let us know how to proceed.
>> >>
>> >>
>> >>
>> >>Regards,
>> >>Renu
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>
--
*Meghamsh Utkur*
*Graduate Student,University of Southern California*
*[email protected] <[email protected]>*
*(503)881-4791*
