Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Can you please confirm via paste.bin the contents of your wmgr and batch_stub scripts? ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 911

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Hello Professor, All the environment variables present in tasks.xml i.e. $WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined. Please let us know how to proceed. Regards, Renu On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann wrote: > Can you please confirm that all of t

Re: RADiX CAS-Crawler cannot trigger workflow event

Dear Prof. Mattmann, Yes, you are right. To solve the problem, we have to change the [WORKFLOWMGR_URL] to [WORKFLOW_URL] in the action-beans.xml file. The default CAS-crawler has been changed to [WORKFLOW_URL] in https://issues.apache.org/jira/browse/OODT-724. I have created a new JIRA ticket (

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Dear Prof. Mattmann, Ok. Then this problem has been solved a few days ago by OODT-763 ( https://issues.apache.org/jira/browse/OODT-763). Also I guess the open issue OODT-768 ( https://issues.apache.org/jira/browse/OODT-768) may be saying the same problem. Thanks! Best, Mengying (Angela) Wang On

Re: re： Question about OODT file manager

Got it. Can you increase the heap space on your batch stub? That should take care of it. Cheers, Chris P.S. Great work! ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Pro

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Please find below tasks.xml file. http://oodt.jpl.nasa.gov/1.0/cas";> On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann wrote: > Hi Renu, > > Can you please show a paste.bin of your tasks.xml? > > Cheers, > Chris

Re: RADiX CAS-Crawler cannot trigger workflow event

Yep merged! :) Great job ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm

Re: re： Question about OODT file manager

Dear Professor, I finally figured out how to trigger a post ingest event. However when I try to crawl the whole dataset, I got an OutOfMemory Error. Could you please take a look and maybe give some suggestions? ➜  bin  ./crawler_launcher \ --operation --launchAutoCrawler \ --filemgrUrl

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Hi Renu, Can you please show a paste.bin of your tasks.xml? Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: mattm...@usc.edu WW

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Can you add lines: -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode="true" \ -Dorg.apache.oodt.cas.pge.task.status.legacyMode="true" \ Starting at line 92 in your batch_stub script, restart the batch stub and see if that fixes it? Cheers, Chris +++

Jenkins build is back to stable : oodt-trunk #1566

See

Jenkins build is back to stable : oodt-trunk » CAS Protocol IMAPS Implementation #1566

See

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

*wmgr script * http://pastebin.com/zE7PWnXL *resmgr batch_stub* http://pastebin.com/gPUvbE9e - Renu On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann wrote: > Great. One last question. > > Can you show me the contents (paste.bin) of your wmgr > script and your batch_stub script? > > +

RE: re： Question about OODT file manager

I just signed up on the wiki(i.e. https://cwiki.apache.org ) with the following account detail: Account name: luke Full Name: Shuai Liu (Luke) Email: hanson311...@gmail.com Password: *** But I am not sure where I can add my notes to the following web article wit

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

renu@ubuntu:~$ echo $WORKFLOW_HOME /home/renu/oodt/workflow/ renu@ubuntu:~$ echo $FILEMGR_HOME /home/renu/oodt/filemgr renu@ubuntu:~$ echo $WORKFLOW_URL http://localhost:9001 renu@ubuntu:~$ echo $FILEMGR_URL http://localhost:9000 renu@ubuntu:~$ echo $PGE_ROOT /home/renu/oodt/pge/ renu@ubuntu:~$ ech

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Great. One last question. Can you show me the contents (paste.bin) of your wmgr script and your batch_stub script? Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles,

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Professor, To add to what Renu has said, I went through the source code and I suspect that the value here in this method is passed as null. XmlFilePgeConfigBuilder.java 1. private String fillIn(String value, Metadata inputMetadata, boolean envReplaceRecur) 2. throws Exception { 3. tr

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Right, which is why I asked to specifically echo the value of all environment variables so we would be able to see the values. Can Renu please show the echo values? Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Departme

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Can you please confirm that all of the bracketed environment variables below do in fact exist in the shell that you are running ./oodt start in? E.g., please echo $ENV_VAR below and show their values. Cheers, Chris Chris Mattmann, Ph.D. Ad

Re: Regarding workflow manager

Professor, My apologies. I do not have the log for that error right now. My set up was as expected yesterday and I made the changes in the tasks.xml specified in my previous mail. I thought instead of dealing with this problem, I better download the new set up and redo the steps. So I do not have

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Sorry I missed paste.bin word in your email.Here is the pastebin link for tasks.xml http://pastebin.com/MAYDeYNH Regards, Renu On Wed, Nov 5, 2014 at 7:18 PM, Renu Kanakamedala wrote: > Please find below tasks.xml file. > > http://oodt.jpl.nasa.gov/1.0/cas";> > > > class="org.apache.oodt.pge

Re: Regarding workflow manager

Thanks Meghamsh, ok. Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: mattm...@usc.edu WWW: http://sunset.usc.edu/~mattmann/ +++

Re: Extra Compiler Tools

OK, let me think about this tonight. Maybe we can figure this out tomorrow, I won’t hold this up longer than that. Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Michael Starch Reply-To: Date: Wednesday, November 5, 2014 at 8:23 PM To: Subj

Jenkins build became unstable: oodt-trunk #1565

See

Jenkins build became unstable: oodt-trunk » CAS Protocol IMAPS Implementation #1565

See

Re: Extra Compiler Tools

According to the specs, any subclass that implements Serializable must manually implement the serialization of the parents' members. I tested this and it fails exactly as expected. The parent's members aren't serialized. Also, JobInput is an interface so I would have no way of catching all o

Re: Regarding workflow manager

Yep, got ya. OK and so using that, you get an Array Index out of bounds exception? Can you please paste the entire exception here? Can you also please stop dropping the CC¹s to the dev list and TAs? Cheers, Chris Chris Mattmann, Ph.D. Adju

Re: FileConcatenator not generating output

Hi Professor, It works now.Looks like the wiki page was recently updated and when I followed that steps (which includes downloading .7 version of oodt ) it works fine . Thanks for checking with us. We were wondering how the overall picture looks like ? As per our current u

Re: Regarding workflow manager

Thanks Mehghamsh, can you suggest what specific changes you made to tasks.xml? Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: m

Re: Extra Compiler Tools

Got it, Mike. Hmm, how about simply creating SerializableJobSpec and SerializableJob and SerializableJobInput and then making them sub-class their parents and implement Serializable. Then, use these classes in your Mesos implementation. That seems self-contained, doesn’t change core classes, and p

Re: Extra Compiler Tools

What is the disadvantage of using Java's built in Serialization? Tyler On Wed, Nov 5, 2014 at 10:00 PM, Michael Starch wrote: > I need to serialize a JobSpec and children (Job and JobInput) to a byte[]. > Java can do this automatically by marking all three as Serializable. > > The work around i

Re: Extra Compiler Tools

I need to serialize a JobSpec and children (Job and JobInput) to a byte[]. Java can do this automatically by marking all three as Serializable. The work around is to manually serialize to a private inner struct and back out again. The inner class will have members for each member in the JobSpec a

Re: Pull Request

Dear Imran, Thank you! Here¹s a way to do it: 0. Download and install hub.github.com 1. File JIRA issue for your fix at https://issues.apache.org/jira/browse/OODT (you will get issue id OODT-xxx where xxx is the issue ID) 2. git clone http://github.com/apache/oodt.git 3. cd oodt 4. git checkout

Re: FileConcatenator not generating output

Great to hear that this is working now Ravi. Note that yes the File Manager (and Workflow Manager) will have the metrics on ingestion since CAS-PGE will help to capture this metadata for you while running ETLLib. Cheers! Chris Chris Matt

Re: Extra Compiler Tools

Hey Mike, Hmm, what’s the work around just so I know what we’re trading against? Cheers, Chris Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Michael Starch Reply-To: Date: Wednesday, November 5, 2014 at 6:31 PM To: Subject: Re: Extra Com

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Dear Angela, Yes that¹s the correct location. See my instructions on Github now for submitting a Pull Request to OODT: http://github.com/apache/oodt You can also submit a patch the traditional way (via SVN) as well. Cheers, Chris Chris Mattmann chris.mattm...@gmail.co

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Dear Prof. Mattmann, OODT-781 is about the filemgr-client script, while this one is about the query-tool script. They may be different issues. Also I would like to submit a patch file, but I am not so sure about where to get the RADiX OODT 0.7 source code? This one http://svn.apache.org/repos/as

Re: Extra Compiler Tools

That is basically what I did. Regardless, protobuff proves to be overkill. If I mark those classes as serializable, the correct solution is 2 lines of code. (protobuff was like 20). Wrote a test case, and it works perfectly. If I cannot make JobSpec Job and JonInput implement Serializable then

[RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Hey everyone, After a text file was ingested into my file manager, I run the command: ./query-tool --url http://localhost:9000 --sql -query "SELECT * FROM GenericFile" to verify the new file. However, it raised such an error: $ ./query-tool --url http://localhost:9000 --sql -query "SELECT * FRO

Re: FileConcatenator not generating output

Hi Ravi, are you still having trouble here? Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Ma

Re: Extra Compiler Tools

Mike, have you looked at this yet? http://techtraits.com/build%20management/maven/2011/09/09/compiling-protoco l-buffers-from-maven/ I’m going to play with it tonight and see if I can help here. Do you have some files I can test with? Can you attach them to JIRA or dropbox them to me so I can sc

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Hi Angela, Awesome, can you please submit a patch to the issue you just created (OODT-781) and can you create a Review Board: http://reviews.apache.org/ Thanks, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and

Re: Extra Compiler Tools

Oktime for an audible. Protoc needs to be built from source, no binary distributions available. Thus I am going to purge proto-buffers from the new code and be done with it. Any problem making the following classes/interfaces implement java.io.Serializable: JobSpec Job JobInput Doing so wo

Re: Made OODT workflow manager take the event

Hi Meghamsh, The cas-pge-0.x.jar is in the PGE_ROOT/lib directory, where 0.x represents your CAS-PGE version. Thanks! Mengying (Angela) Wang On Wed, Nov 5, 2014 at 10:58 AM, Christian Alan Mattmann wrote: > Thank you would really appreciate it. Do you have access? > If not, what¹s your wiki us

Re: RADiX filemgr-client script cannot delete product by id or name

Ugh, OK I will take a look at it tonight! Chris Mattmann chris.mattm...@gmail.com -Original Message- From: MengYing Wang Reply-To: Date: Wednesday, November 5, 2014 at 5:15 PM To: "Ramirez, Paul M (398M)" Cc: "" , Chris Mattmann Subject: Re: RADiX filemgr-c

Re: Could not mvn install OODT from the trunk

Thanks Angela keep us posted! Chris Mattmann chris.mattm...@gmail.com -Original Message- From: MengYing Wang Reply-To: Date: Wednesday, November 5, 2014 at 5:22 PM To: "dev@oodt.apache.org" Cc: , Tyler Palsulich Subject: Re: Could not mvn install OODT from

Re: Could not mvn install OODT from the trunk

Dear Michael and Tyler, No OODT related environment variables are set. *-* Ok, maybe it is time to upgrade my maven version. Thanks again. Best, Mengying (Angela) Wang On Sun, Nov 2, 2014 at 10:10 AM, Tyler Palsulich wrote: > > I use maven 3.2 to build OODT. This might help. > +1 > -- Be

Re: RADiX filemgr-client script cannot delete product by id or name

Dear Prof. Mattmann and Paul, @Mattmann, Retried, still the same error. -_-! @Paul, the jira issue has been created ( https://issues.apache.org/jira/browse/OODT-781). Thanks. Best, Mengying (Angela) Wang On Sun, Nov 2, 2014 at 11:47 AM, Ramirez, Paul M (398M) < paul.m.rami...@jpl.nasa.gov> wr

Re: A Note on Software Design to Make Maintenance Easier

Thanks Mucho! :) Astronomy FTW! Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Bruce Barkstrom Reply-To: Date: Wednesday, November 5, 2014 at 3:51 PM To: Subject: Re: A Note on Software Design to Make Maintenance Easier >Finally got around

Re: A Note on Software Design to Make Maintenance Easier

Finally got around to the Sept. issue of IEEE Computer. Noticed your editing work. Nice job. Bruce B. On Wed, Nov 5, 2014 at 3:38 PM, Mattmann, Chris A (3980) < chris.a.mattm...@jpl.nasa.gov> wrote: > Thanks Bruce, this is great. > > +

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/#review60028 --- Awesome @Varun! - Chris Mattmann On Nov. 4, 2014, 11:12 p.m., Var

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/#review60026 --- Ship it! Ship It! - Chris Mattmann On Nov. 4, 2014, 11:12 p.m.,

Re: A Note on Software Design to Make Maintenance Easier

Thanks Bruce, this is great. ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.m

Re: OODT Source Code

> I`m a PhD studying the modularity of different ecosystems. I intend to > analyze the OODT code through code metrics, however I need the java > projects for each version (release). Where can I download these projects to > be analyzed by a metric tool? You can find archived, older releases of Apac

Re: RADiX CAS-Crawler cannot trigger workflow event

That’s the problem - the policy is wrong in the RADIX crawler policy. Can you open up a JIRA ticket to fix this? It’s probably wrong too in the default CAS-crawler policy. The value inside of that file should be [WORKFLOW_URL]. Please open up a JIRA and submit a patch. Thanks Angela! Cheers, Chri

Re: OODT Source Code

Check out our SVN repository: https://svn.apache.org/repos/asf/oodt/trunk This points to "trunk", our active-development area. -Michael On Tue, Nov 4, 2014 at 5:08 AM, Paulo Silveira wrote: > Dear all, > > I`m a PhD studying the modularity of different ecosystems. I intend to > analyze the OOD

Re: OODT Source Code

Dear Paul, You can grab the code from here: http://svn.apache.org/repos/asf/oodt/ You can also find it on github at: http://github.com/apache/oodt/ Cheers! Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and S

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/ --- (Updated Nov. 4, 2014, 11:12 p.m.) Review request for oodt and Chris Mattmann.

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/ --- (Updated Nov. 4, 2014, 10:40 p.m.) Review request for oodt and Chris Mattmann.

Re: RADiX CAS-Crawler cannot trigger workflow event

Dear Prof. Mattmann, In the default property, there is a line "[WORKFLOWMGR_URL]" while no definition of the [WORKFLOW_URL]. [FAILURE_DIR] [BACKUP_DIR] [WORKFLOWMGR_URL] [FILEMGR_URL] or

Re: Like to update the wiki with some diagnostic information when starting OODT

permissions granted! :) Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: mattm...@usc.edu WWW: http://sunset.usc.edu/~mattmann/

Re: RADiX CAS-Crawler cannot trigger workflow event

Dear Prof. Mattmann, I have set all the required environment variables. export OODT_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7" export CRAWLER_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/crawler" export FILEMGR_HOME="/Users/AngelaWang/Documents/programs/oodt-0.7/filemgr" export W

OODT Source Code

Dear all, I`m a PhD studying the modularity of different ecosystems. I intend to analyze the OODT code through code metrics, however I need the java projects for each version (release). Where can I download these projects to be analyzed by a metric tool? Thanks in advance, -- Paulo Silveira Ph.D

RE: Passing WorkflowInstId to extern extractor

Hey Chris, No worries on the delayed reply. Looks like you've been very busy answering lots of questions. Your email below was very helpful and very much appreciated! Thanks so much! Val Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied Physics

Re: Extra Compiler Tools

Thanks Mike +1 Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Michael Starch Reply-To: Date: Wednesday, November 5, 2014 at 12:31 PM To: Subject: Re: Extra Compiler Tools >Looks like you followed the same reasoning chain that I did. Yes,

Re: Like to update the wiki with some diagnostic information when starting OODT

Professor, Thank you. Here is my username; kothamas On Wed, Nov 5, 2014 at 10:53 AM, Christian Alan Mattmann wrote: > Thank you Naveen! Can you please share your Apache CWIKI > username and I¹ll grant you permissions? > > Thanks! > > Cheers, > Chris > > +

Re: Extra Compiler Tools

Looks like you followed the same reasoning chain that I did. Yes, I came to the same conclusion that ant-build was best. I wasn't sure how to download protoc, but you just answered thatso I think this is a great solution! Thanks, Michael On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann wr

Re: Extra Compiler Tools

Hi Mike, Thanks for flushing this out. My thoughts on the below: -Original Message- From: Michael Starch Reply-To: Date: Wednesday, November 5, 2014 at 12:12 PM To: Subject: Re: Extra Compiler Tools >I tried this approach. The plugin requires a path to the "protoc" tool and >thus a

Re: Extra Compiler Tools

I tried this approach. The plugin requires a path to the "protoc" tool and thus a working installation. This is what prompted the discussion. Running the plugin under a profile works. However, not running the plugin causes compile errors in dependant code. Excluding this code except within the

Re: Extra Compiler Tools

I tried this approach. The plugin requires a path to the "protoc" tool and thus a working installation. This is what prompted the discussion. Running the plugin under a profile works. However, not running the plugin causes compile errors in dependant code. Excluding this code except within the

Re: Like to update the wiki with some diagnostic information when starting OODT

Thank you Naveen! Can you please share your Apache CWIKI username and I¹ll grant you permissions? Thanks! Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los An

Re: Made OODT workflow manager take the event

Thank you would really appreciate it. Do you have access? If not, what¹s your wiki username and I can grant you permissions. Thanks, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern

Re: Placeholder in PGE configuration xml file

Please use system environment variables - these should be defined in the env.sh part of the OODT_HOME/bin dir that comes with RADIX. Thanks, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of

Re: Regarding workflow manager

Can you please check logs/oodt.out - what is showing there? Does the workflow manager receive the event? Does the resource manager receive the job? Any output? Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Scienc

Re: Extra Compiler Tools

Hi Mike, Great discussion. It would be nice if there was a protoc Maven plugin: http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html Looks like there is. My suggestion: 1. use a Profile, something like -Pwith-mesos and then when activated; 2. call the above plugin if -Pwith-mesos is a

Extra Compiler Tools

All, I am trying to integrate apache-mesos with our resource manager. However, mesos uses a technology called "protobuff" from Google for marshaling/unmarshaling data. This requires running a tool called "protoc" to generate a source file in java. What is the best way to integrate this step into