Extra Compiler Tools

All, I am trying to integrate apache-mesos with our resource manager. However, mesos uses a technology called protobuff from Google for marshaling/unmarshaling data. This requires running a tool called protoc to generate a source file in java. What is the best way to integrate this step into

Re: Extra Compiler Tools

Hi Mike, Great discussion. It would be nice if there was a protoc Maven plugin: http://sergei-ivanov.github.io/maven-protoc-plugin/usage.html Looks like there is. My suggestion: 1. use a Profile, something like -Pwith-mesos and then when activated; 2. call the above plugin if -Pwith-mesos is

Re: Regarding workflow manager

Can you please check logs/oodt.out - what is showing there? Does the workflow manager receive the event? Does the resource manager receive the job? Any output? Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer

Re: Placeholder in PGE configuration xml file

Please use system environment variables - these should be defined in the env.sh part of the OODT_HOME/bin dir that comes with RADIX. Thanks, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University

Re: Made OODT workflow manager take the event

Thank you would really appreciate it. Do you have access? If not, what¹s your wiki username and I can grant you permissions. Thanks, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of

Re: Extra Compiler Tools

I tried this approach. The plugin requires a path to the protoc tool and thus a working installation. This is what prompted the discussion. Running the plugin under a profile works. However, not running the plugin causes compile errors in dependant code. Excluding this code except within the

Re: Extra Compiler Tools

I tried this approach. The plugin requires a path to the protoc tool and thus a working installation. This is what prompted the discussion. Running the plugin under a profile works. However, not running the plugin causes compile errors in dependant code. Excluding this code except within the

Re: Extra Compiler Tools

Hi Mike, Thanks for flushing this out. My thoughts on the below: -Original Message- From: Michael Starch starc...@umich.edu Reply-To: dev@oodt.apache.org Date: Wednesday, November 5, 2014 at 12:12 PM To: dev@oodt.apache.org Subject: Re: Extra Compiler Tools I tried this approach. The

Re: Extra Compiler Tools

Looks like you followed the same reasoning chain that I did. Yes, I came to the same conclusion that ant-build was best. I wasn't sure how to download protoc, but you just answered thatso I think this is a great solution! Thanks, Michael On Wed, Nov 5, 2014 at 10:23 AM, Chris Mattmann

RE: Passing WorkflowInstId to extern extractor

Hey Chris, No worries on the delayed reply. Looks like you've been very busy answering lots of questions. Your email below was very helpful and very much appreciated! Thanks so much! Val Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied

OODT Source Code

Dear all, I`m a PhD studying the modularity of different ecosystems. I intend to analyze the OODT code through code metrics, however I need the java projects for each version (release). Where can I download these projects to be analyzed by a metric tool? Thanks in advance, -- Paulo Silveira

Re: Like to update the wiki with some diagnostic information when starting OODT

permissions granted! :) Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: mattm...@usc.edu WWW: http://sunset.usc.edu/~mattmann/

Re: RADiX CAS-Crawler cannot trigger workflow event

Dear Prof. Mattmann, In the default property, there is a line prop key=crawler.workflowmgr.url[WORKFLOWMGR_URL]/prop while no definition of the [WORKFLOW_URL]. !-- Default Properties -- property name=properties props prop

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/ --- (Updated Nov. 4, 2014, 10:40 p.m.) Review request for oodt and Chris Mattmann.

Re: Review Request 15936: OODT 563 - a) Modifying Workflow Manager to allow adding new workflows

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15936/ --- (Updated Nov. 4, 2014, 11:12 p.m.) Review request for oodt and Chris Mattmann.

Re: OODT Source Code

Dear Paul, You can grab the code from here: http://svn.apache.org/repos/asf/oodt/ You can also find it on github at: http://github.com/apache/oodt/ Cheers! Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and

Re: OODT Source Code

Check out our SVN repository: https://svn.apache.org/repos/asf/oodt/trunk This points to trunk, our active-development area. -Michael On Tue, Nov 4, 2014 at 5:08 AM, Paulo Silveira pauload...@gmail.com wrote: Dear all, I`m a PhD studying the modularity of different ecosystems. I intend to

Re: RADiX CAS-Crawler cannot trigger workflow event

That’s the problem - the policy is wrong in the RADIX crawler policy. Can you open up a JIRA ticket to fix this? It’s probably wrong too in the default CAS-crawler policy. The value inside of that file should be [WORKFLOW_URL]. Please open up a JIRA and submit a patch. Thanks Angela! Cheers,

Re: A Note on Software Design to Make Maintenance Easier

Finally got around to the Sept. issue of IEEE Computer. Noticed your editing work. Nice job. Bruce B. On Wed, Nov 5, 2014 at 3:38 PM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Thanks Bruce, this is great.

Re: A Note on Software Design to Make Maintenance Easier

Thanks Mucho! :) Astronomy FTW! Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Bruce Barkstrom brbarkst...@gmail.com Reply-To: dev@oodt.apache.org Date: Wednesday, November 5, 2014 at 3:51 PM To: dev@oodt.apache.org Subject: Re: A Note on

Re: Could not mvn install OODT from the trunk

Dear Michael and Tyler, No OODT related environment variables are set. *-* Ok, maybe it is time to upgrade my maven version. Thanks again. Best, Mengying (Angela) Wang On Sun, Nov 2, 2014 at 10:10 AM, Tyler Palsulich tpalsul...@gmail.com wrote: I use maven 3.2 to build OODT. This might

Re: Extra Compiler Tools

Oktime for an audible. Protoc needs to be built from source, no binary distributions available. Thus I am going to purge proto-buffers from the new code and be done with it. Any problem making the following classes/interfaces implement java.io.Serializable: JobSpec Job JobInput Doing so

Re: FileConcatenator not generating output

Hi Ravi, are you still having trouble here? Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519,

[RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Hey everyone, After a text file was ingested into my file manager, I run the command: ./query-tool --url http://localhost:9000 --sql -query SELECT * FROM GenericFile to verify the new file. However, it raised such an error: $ ./query-tool --url http://localhost:9000 --sql -query SELECT * FROM

Re: Extra Compiler Tools

That is basically what I did. Regardless, protobuff proves to be overkill. If I mark those classes as serializable, the correct solution is 2 lines of code. (protobuff was like 20). Wrote a test case, and it works perfectly. If I cannot make JobSpec Job and JonInput implement Serializable then

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Dear Angela, Yes that¹s the correct location. See my instructions on Github now for submitting a Pull Request to OODT: http://github.com/apache/oodt You can also submit a patch the traditional way (via SVN) as well. Cheers, Chris Chris Mattmann

Re: Extra Compiler Tools

Hey Mike, Hmm, what’s the work around just so I know what we’re trading against? Cheers, Chris Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Michael Starch starc...@umich.edu Reply-To: dev@oodt.apache.org Date: Wednesday, November 5, 2014

Re: FileConcatenator not generating output

Great to hear that this is working now Ravi. Note that yes the File Manager (and Workflow Manager) will have the metrics on ingestion since CAS-PGE will help to capture this metadata for you while running ETLLib. Cheers! Chris Chris

Re: Extra Compiler Tools

What is the disadvantage of using Java's built in Serialization? Tyler On Wed, Nov 5, 2014 at 10:00 PM, Michael Starch starc...@umich.edu wrote: I need to serialize a JobSpec and children (Job and JobInput) to a byte[]. Java can do this automatically by marking all three as Serializable.

Re: Regarding workflow manager

Thanks Mehghamsh, can you suggest what specific changes you made to tasks.xml? Cheers, Chris Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email:

Re: Regarding workflow manager

Yep, got ya. OK and so using that, you get an Array Index out of bounds exception? Can you please paste the entire exception here? Can you also please stop dropping the CC¹s to the dev list and TAs? Cheers, Chris Chris Mattmann, Ph.D.

Re: Extra Compiler Tools

According to the specs, any subclass that implements Serializable must manually implement the serialization of the parents' members. I tested this and it fails exactly as expected. The parent's members aren't serialized. Also, JobInput is an interface so I would have no way of catching all

Jenkins build became unstable: oodt-trunk » CAS Protocol IMAPS Implementation #1565

See https://builds.apache.org/job/oodt-trunk/org.apache.oodt$cas-protocol-imaps/1565/

Jenkins build became unstable: oodt-trunk #1565

See https://builds.apache.org/job/oodt-trunk/1565/changes

Re: Extra Compiler Tools

OK, let me think about this tonight. Maybe we can figure this out tomorrow, I won’t hold this up longer than that. Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Michael Starch starc...@umich.edu Reply-To: dev@oodt.apache.org Date: Wednesday,

Re: Regarding workflow manager

Thanks Meghamsh, ok. Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: mattm...@usc.edu WWW: http://sunset.usc.edu/~mattmann/

Re: Regarding workflow manager

Professor, My apologies. I do not have the log for that error right now. My set up was as expected yesterday and I made the changes in the tasks.xml specified in my previous mail. I thought instead of dealing with this problem, I better download the new set up and redo the steps. So I do not

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Can you please confirm that all of the bracketed environment variables below do in fact exist in the shell that you are running ./oodt start in? E.g., please echo $ENV_VAR below and show their values. Cheers, Chris Chris Mattmann, Ph.D.

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Right, which is why I asked to specifically echo the value of all environment variables so we would be able to see the values. Can Renu please show the echo values? Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Professor, To add to what Renu has said, I went through the source code and I suspect that the value here in this method is passed as null. XmlFilePgeConfigBuilder.java 1. private String fillIn(String value, Metadata inputMetadata, boolean envReplaceRecur) 2. throws Exception { 3.

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Great. One last question. Can you show me the contents (paste.bin) of your wmgr script and your batch_stub script? Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

renu@ubuntu:~$ echo $WORKFLOW_HOME /home/renu/oodt/workflow/ renu@ubuntu:~$ echo $FILEMGR_HOME /home/renu/oodt/filemgr renu@ubuntu:~$ echo $WORKFLOW_URL http://localhost:9001 renu@ubuntu:~$ echo $FILEMGR_URL http://localhost:9000 renu@ubuntu:~$ echo $PGE_ROOT /home/renu/oodt/pge/ renu@ubuntu:~$

RE: re： Question about OODT file manager

I just signed up on the wiki(i.e. https://cwiki.apache.org ) with the following account detail: Account name: luke Full Name: Shuai Liu (Luke) Email: hanson311...@gmail.com Password: *** But I am not sure where I can add my notes to the following web article

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

*wmgr script * http://pastebin.com/zE7PWnXL *resmgr batch_stub* http://pastebin.com/gPUvbE9e - Renu On Wed, Nov 5, 2014 at 9:43 PM, Christian Alan Mattmann mattm...@usc.edu wrote: Great. One last question. Can you show me the contents (paste.bin) of your wmgr script and your batch_stub

Jenkins build is back to stable : oodt-trunk » CAS Protocol IMAPS Implementation #1566

See https://builds.apache.org/job/oodt-trunk/org.apache.oodt$cas-protocol-imaps/1566/

Jenkins build is back to stable : oodt-trunk #1566

See https://builds.apache.org/job/oodt-trunk/1566/changes

Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error.

Can you add lines: -Dorg.apache.oodt.cas.pge.task.metkeys.legacyMode=true \ -Dorg.apache.oodt.cas.pge.task.status.legacyMode=true \ Starting at line 92 in your batch_stub script, restart the batch stub and see if that fixes it? Cheers, Chris

Re: RADiX CAS-Crawler cannot trigger workflow event

Yep merged! :) Great job ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email:

Re: [RADiX File Manager] query-tool script sets the wrong values to the OODT_HOME and FILEMGR_HOME variables

Dear Prof. Mattmann, Ok. Then this problem has been solved a few days ago by OODT-763 ( https://issues.apache.org/jira/browse/OODT-763). Also I guess the open issue OODT-768 ( https://issues.apache.org/jira/browse/OODT-768) may be saying the same problem. Thanks! Best, Mengying (Angela) Wang

Re: RADiX CAS-Crawler cannot trigger workflow event

Dear Prof. Mattmann, Yes, you are right. To solve the problem, we have to change the [WORKFLOWMGR_URL] to [WORKFLOW_URL] in the action-beans.xml file. The default CAS-crawler has been changed to [WORKFLOW_URL] in https://issues.apache.org/jira/browse/OODT-724. I have created a new JIRA ticket (