Re: Resource Manager 0.7-SNAPSHOT and Ganglia Monitor

[Mattmann, Chris A (398J)]

Dear Rajth we need a way to disable it it should be optional 

Sent from my iPhone

> On Dec 9, 2013, at 6:57 PM, "Rajith Siriwardana" 
>  wrote:
> 
> hi Cameron,
> 
> AFAIK there is no way to disable the Ganglia plugin, so you'll have to
> setup and run a Ganglia metadata daemon (Gmetad). I'll check that again.
> 
> Cheers,
> Rajith
> 
> 
>> On Tue, Dec 10, 2013 at 2:53 AM, Cameron Goodale  wrote:
>> 
>> Hey Rajith and Devs,
>> 
>> I am upgrading my resource manager 0.3 to the latest version (currently
>> 0.7-SNAPSHOT) and I have run into the Ganglia Monitor Plugin with the
>> following errors:
>> 
>> SEVERE: The required nodeId is not available: sneffels-11 :null
>> java.lang.NullPointerException
>> at
>> 
>> org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
>> at
>> 
>> org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
>> at java.lang.Thread.run(Thread.java:662)
>> Dec 9, 2013 1:17:05 PM
>> org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor getLoad
>> SEVERE: The required nodeId is not available: kush-11 :null
>> java.lang.NullPointerException
>> at
>> 
>> org.apache.oodt.cas.resource.monitor.ganglia.GangliaResourceMonitor.getLoad(GangliaResourceMonitor.java:73)
>> at
>> 
>> org.apache.oodt.cas.resource.monitor.AssignmentMonitor.getLoad(AssignmentMonitor.java:105)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.nodeAvailable(LRUScheduler.java:232)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.schedule(LRUScheduler.java:134)
>> at
>> 
>> org.apache.oodt.cas.resource.scheduler.LRUScheduler.run(LRUScheduler.java:108)
>> at java.lang.Thread.run(Thread.java:662)
>> 
>> 
>> I get this error message for all 44 of the nodes I have defined in my
>> nodes.xml config.
>> 
>> 
>> A couple questions:
>> 
>> 1.  Can I simply run resource manager without the Plugin via a change in
>> config?  I tried to comment out the config lines, but it didn't work.
>> 
>> 2.  If I cannot simply disconnect the Ganglia Dependency, then do I have to
>> setup and run a Ganglia metadata daemon?
>> 
>> 
>> I look forward to your reply,
>> 
>> 
>> Cameron
>> 
>> --
>> 
>> Sent from a Tin Can attached to a String
>> 

Re: new issue in wmgr-client (0.6 vs 0.3)

[Mattmann, Chris A (398J)]

You didn't bother me at all and I am CC'ing the dev@oodt list so
they benefit from your good work.

Thanks Lindsey!

Cheers,
Chris



-Original Message-
From: Lindsey Davis 
Date: Wednesday, November 13, 2013 9:14 AM
To: jpluser 
Cc: Jeff Kern 
Subject: Re: new issue in wmgr-client (0.6 vs 0.3)

>Hi Chris,
>
>Forget my previous mail. It is working fine. I did not
>notice at first that I need to install the command lines
>options and actions policy files. I also found out that
>I needed to prefix the paths to these files with file://.
>
>Sorry to have bothered you.
>
> Lindsey
>
>> Hi Chris,
>>
>> I have come across what looks like an annoying change in
>> XmlRpcWorkflowManagerClient that you may be able to help
>> me with.
>>
>> I am porting the ALMA pipeline code from 0.3 to 0.6. I
>> downloaded the tarball of jar files and replaced my current
>> 0.3 jars with 0.6 jars.
>>
>> The pipeline code built fine and the workflow manager
>> started up without any problems. However when I ran my test
>> workflow I got a error about a missing class cli.CmdLineUtility.
>>
>> I looked in the workflow distribution code and determined that
>> there is a new jar cas-cli-0.6.jar. I added that to my build system.
>> The missing class definition disappears but now I get what looks
>> like a very annoying parsing error
>>
>> The command issued on the terminal is
>>
>> skye<151>$ wmgr-client --url http://localhost:9001 --operation
>>--sendEvent 
>> --eventName modelPreCasaEvent
>>
>> Assuming that modelPreCalsEvent exists is there anything wrong with
>>this ?
>>
>> This gets translated into a line like the following
>>
>> acsStartJava -endorsed \
>> 
>>-Djava.util.logging.FileHandler.pattern=${SCIPIPE_WFMANAGER_LOGDIR}/cas_w
>>orkflow%g.log 
>> \
>> 
>>-Djava.util.logging.config.file=${SCIPIPE_CAS_PROPERTIES}/logging.propert
>>ies 
>> \
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient $*
>>
>> where acsStartjava with an argument -endorsed is used to start java
>>
>> Unfortunately is looks like the code in cli mangles these commands (see
>> below). Is there
>> anyway I can bypass all this and get back the original behavior which
>>would 
>> be much
>> easier to work with.
>>
>> Lindsey
>>
>>
>>
>>
>> skye<151>$ wmgr-client --url http://localhost:9001 --operation
>>--sendEvent 
>> --eventName modelPreCasaEvent
>> --endorsed -D 
>> 
>>java.util.logging.FileHandler.pattern=/home/skye/wflogdir/cas_workflow%g.
>>log 
>> -D java.util.logging.config.file
>> 
>>=/export/home/skye/working/acs121/HEAD121X/introot121/Sources/Science/src
>>/logging.properties
>> -- org.apache.oodt.cas.workflow
>> .system.XmlRpcWorkflowManagerClient --url http://localhost:9001
>>--operation 
>> --sendEvent --eventName modelPreCasaEvent
>> 2013-11-12T22:02:20.024 INFO [acsStartJava] Starting Java application:
>> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowMan
>> agerClient --url http://localhost:9001 --operation --sendEvent
>>--eventName 
>> modelPreCasaEvent
>> 2013-11-12T22:02:20.066 INFO [acsStartJava] Using endorsed jar files
>>in: 
>> -Djava.endorsed.dirs=/export/home/skye/working/acs1
>> 21/HEAD121X/introot121/lib/endorsed:/alma/ACS-12.1/ACSSW/lib/endorsed:
>> Exception in thread "main" java.lang.IllegalArgumentException: The
>>validated 
>> object is null
>> at org.apache.commons.lang.Validate.notNull(Validate.java:192)
>> at org.apache.commons.lang.Validate.notNull(Validate.java:178)
>> at
>> 
>>org.apache.oodt.cas.cli.option.store.spring.SpringCmdLineOptionStoreFacto
>>ry.createStore(SpringCmdLineOptionStoreF
>> actory.java:40)
>> at
>> 
>>org.apache.oodt.cas.cli.CmdLineUtility.(CmdLineUtility.java:94)
>> at
>> 
>>org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManagerClient.main(XmlR
>>pcWorkflowManagerClient.java:677)
>>

Google Summer of Code Blogs

[Mattmann, Chris A (398J)]

Hey Guys,

GSoC released blog posts that I and Rishi wrote on GSoC:

http://google-opensource.blogspot.com/2013/10/google-summer-of-code-veteran
-orgs.html


Congrats Rajith and Ross!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

SolrCatalog should be in package o.a.oodt.cas.filemgr.catalog

[Mattmann, Chris A (398J)]

Hey Luca,

I noticed that the SolrCatalog is in package
o.a.oodt.cas.filemgr.catalog.solr
instead of o.a.oodt.cas.filemgr.catalog, matching all other catalog
implementations.
I realize this is because you added specific classes to support the Solr
Catalog
and put them in that package as well. I think these classes should be
dispersed
either to util packages in the File Manager, or in some other support
structure
(e.g., as private internal classes to the SolrCatalog or
SolrCatalogFactory) so
as to keep consistent with the rest of the components. I'll make this
change in
the next few weeks (and file a JIRA issue) should there be no objections.

Thanks!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: Git(hub) Mirrors are oudated?

[Mattmann, Chris A (398J)]

Lewis was on it, perfect (contacted infra@)

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Michael Joyce 
Reply-To: "dev@oodt.apache.org" 
Date: Thursday, August 29, 2013 5:29 PM
To: "dev@oodt.apache.org" 
Subject: Re: Git(hub) Mirrors are oudated?

>You're right Sherl I wasn't looking at trunk (but it is still very far
>behind). I wonder who we could bother about why this is so far behind.
>Maybe Chris will know when he finds this thread...
>
>
>-- Joyce
>
>
>On Thu, Aug 29, 2013 at 5:09 PM, Lewis John Mcgibbney <
>lewis.mcgibb...@gmail.com> wrote:
>
>> Yes it is certainly behind.
>>
>>
>> On Thu, Aug 29, 2013 at 4:47 PM, Sheryl John  wrote:
>>
>> > Hi,
>> >
>> > The other branches have later commits [1]. The 'trunk' has commits
>>from 4
>> > months ago..But, it doesn't have a 0.6 branch or tag so it still seems
>> > behind SVN.
>> >
>> > [1] https://github.com/apache/oodt/branches
>> >
>> >
>> > On Thu, Aug 29, 2013 at 4:33 PM, Michael Joyce 
>>wrote:
>> >
>> > > I noticed that the "Last Updated" dates of the files in the OODT
>>Github
>> > > Mirror [1] all read "3 Years Ago". Is the github/git mirror this far
>> > behind
>> > > our SVN?
>> > >
>> > > [1] https://github.com/apache/oodt
>> > >
>> > > -- Joyce
>> > >
>> >
>> >
>> >
>> > --
>> > -Sheryl
>> >
>>
>>
>>
>> --
>> *Lewis*
>>

Re: Fix for OODT-473

[Mattmann, Chris A (398J)]

Hi Tom,

I granted you JIRA karma so you should be able to assign OODT-460
to you :)

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Wednesday, August 28, 2013 8:52 AM
To: "dev@oodt.apache.org" 
Subject: Re: Fix for OODT-473

>Thanks Chris,
>
>A few other things, can you assign OODT-460 to me unless someone else
>wants it, as I've got experience with that type of stuff with my day job
>so I may as well do it.
>
>Also I saw that cas-curator doesn't have any unit tests in the webapp,
>is that because it doesn't require any or they just didn't get written?
>
>Slowly getting my head wrapped around at least the main ingestion
>components :)
>
>Also anyone got an up to date component diagram, I'd like to extend
>something and put something on the wiki so newbs like myself understand
>how stuff hangs togther and their purpose (I still dont know what most
>of the modules do :) )
>
>Tom
>
>On 28/08/13 16:42, Mattmann, Chris A (398J) wrote:
>> Thanks Tom.
>>
>> Don't worry about the etiquette, if it's been sitting for a while,
>> patches *and* commits, welcome! :) Also if you'd like feel free to
>> use Review Board and then just say if there are no comments e.g.,
>> within 24, 48, 72 hours, etc., I'm going to go ahead and commit.
>>
>> Review Board is here: http://reviews.apache.org/
>>
>> Cheers,
>> Chris
>>
>> ++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Tom Barber 
>> Reply-To: "dev@oodt.apache.org" 
>> Date: Wednesday, August 28, 2013 8:35 AM
>> To: "dev@oodt.apache.org" 
>> Subject: Fix for OODT-473
>>
>>> Hey guys
>>>
>>> Looking through the stupidly easy Jira section whilst still getting my
>>> head around the components and setup. Attached a patch to
>>> https://issues.apache.org/jira/browse/OODT-473 to remove the delete
>>> method, I hope its right, the test suite looks okay. I'm not sure about
>>> SVN etiquette and generally if I'm doing the right thing, so I'll
>>> probably just continue adding patches to Jira for now :)
>

Re: Fix for OODT-473

[Mattmann, Chris A (398J)]

Thanks Tom.

Don't worry about the etiquette, if it's been sitting for a while,
patches *and* commits, welcome! :) Also if you'd like feel free to
use Review Board and then just say if there are no comments e.g.,
within 24, 48, 72 hours, etc., I'm going to go ahead and commit.

Review Board is here: http://reviews.apache.org/

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Wednesday, August 28, 2013 8:35 AM
To: "dev@oodt.apache.org" 
Subject: Fix for OODT-473

>Hey guys
>
>Looking through the stupidly easy Jira section whilst still getting my
>head around the components and setup. Attached a patch to
>https://issues.apache.org/jira/browse/OODT-473 to remove the delete
>method, I hope its right, the test suite looks okay. I'm not sure about
>SVN etiquette and generally if I'm doing the right thing, so I'll
>probably just continue adding patches to Jira for now :)

Re: [Ganglia plugin] Dev. documentation added to Wiki

[Mattmann, Chris A (398J)]

Great work Rajith!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Rajith Siriwardana 
Date: Sunday, August 11, 2013 9:53 AM
To: dev 
Cc: jpluser 
Subject: [Ganglia plugin] Dev. documentation added to Wiki

>Hi all,
>
>
>Ganglia plugin developer documentation added to Wiki [1]. please find the
>provided link and your feedback is highly appreciated to improve the
>document's quality.
>
>
>
>
>[1] 
>https://cwiki.apache.org/confluence/display/OODT/Ganglia+plugin+for+CAS+Re
>source+Manager
>
>
>Cheers,
>Rajith
>

Re: Generic MetaData Column

[Mattmann, Chris A (398J)]

Hi Sanjaya,

I would strongly recommend against a single field with key val pairs
delimited by comma.

Use multiple met fields. If your point is that you won't know the met
upfront, I'd ask you the following things:

1. Why can't you model your met up front? Your workflows having a good
idea of what input/output met fields they will generate is one of the
keys to provenance.

2. If you still can't satisfy #1, then there is always the
LenientDataSourceCatalog
that you can use that skips met field and element validation and lets you
store
any met. You can use this but since you were thinking about provenance I'd
urge you
to consider #1 first.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Sanjaya Medonsa 
Reply-To: "dev@oodt.apache.org" 
Date: Monday, July 22, 2013 10:33 PM
To: "dev@oodt.apache.org" 
Subject: Generic MetaData Column

>Hi,
>In OODT it is possible to store new Metadata items by either introducing
>new product type or adding new items to existing product type. Basically
>you need to update the three policy files with required configuration
>(elements.xml, product-type-element-map.xml and product-types.xml). With
>this approach, Metadata items that needs to be stored should be set up
>prior to metadata ingestion. I need to understand best way to support
>following behavior.
>
>Ex: As I discussed in other email thread, I am planning to store input as
>metadata together with output for Airavata integartion. Depending on task
>number of inputs could be vary. So it is not possible to setup metadata
>column such as Input1 and Input2. I guess best way to support this
>requirement is to have one metadata field called Input and store all
>inputs
>as key value pair. Is there any other standard mechanism available to
>handle these sort of situation ?
>
>Best Regards,
>Sanjaya

Re: OODT Documentation Organisation

[Mattmann, Chris A (398J)]

Hey Tom,

This makes great sense to me and I'm glad you're offering your valuable
time and cycles to get the wiki organized. I think a multi-pronged approach
of:

* The website for releases -- static content, updated infrequently
* Pointers to wiki for *active* documents, and links from the website
displayed
more prominently
* Encouraging users to start at the wiki -- and really only head to the
website
for downloads, and for long standing information about the project and team

Will make a huge difference. I'll try and help where I can and thanks. I
plan on putting a bow on the Apache OODT 0.6 RC and pushing the release
tomorrow since the VOTE has already passed.

Cheers!

Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, July 12, 2013 1:18 PM
To: "dev@oodt.apache.org" 
Subject: OODT Documentation Organisation

>Hi guys
>
>Whilst working on getting to know OODT better I walked through the CAS
>Curation tutorial on the website and found it both hard to read and
>buggy in places. I also (as a newbie) found getting started and tutorial
>stuff tricky to locate with bits being on the main website and some on
>the wiki. I guess this is left over from the Apache ingestion.
>
>I took the liberty of migrating the original Curation tutorial over to
>the wiki and made a few tweaks although I would like to improve this
>further over the next week or so:
>https://cwiki.apache.org/confluence/display/OODT/Deploying+and+using+CAS+C
>urator
>
>My question is really this, is there a documentation policy with regards
>to creating stuff and guiding new users. Personally I feel that as much
>of the documentation as possible should be moved across to the wiki to
>allow users to develop and maintain it as OODT grows and changes (and
>allows people to fix errors). Also finding the wiki isn't very obvious
>from the main OODT website, I would suggest that whoever maintains the
>website puts the wiki link into the header or something allowing people
>easy access to it, otherwise its buried in various page menus.
>
>Having worked closely with a number of open source tools over the years,
>especially as a newbie, I find the wiki documentation invaluable in
>getting started and guiding new users, if no one has any objections I'd
>like to put some time and effort into organising the wiki and landing
>pages on their in a way that helps guide people in a more effective
>manner. This will clearly be extended as I learn more myself :)
>
>Cheers
>
>Tom

Thoughts on using JCC to "Python-ize" OODT

[Mattmann, Chris A (398J)]

Hey Guys,

I just found one of the coolest things in the world: Aptivate Tika [1],
which is for all intents and purposes, a full port of Tika from Java
to Python using JCC [2], which was written by the PyLucene community
to automatically generate C++ code that can then be linked to Python
and callable directly in Python to expose the Java lib functionality.

This got me thinking: why not use JCC to do the same thing with OODT?
I'm sure we can do this and I'll do some initial benchmarking and
testing to try it out.

The next question is where precisely does it make sense to use JCC?
I could see JCC creating a Python File Manager, Workflow Manager,
and Resource Manager. This would basically allow folks to write some
of the core server extensions in Python if they so desire. The flip
side of this is that all of those servers are easily interacted
with in Python using xmlrpclib [3].

Anyways, just wanted to share my thoughts. I'll keep plugging away
at this. And, I'm working on figuring out how to do something like
Aptivate Tika in Apache Tika and ship a PyTika :)

Cheers,
Chris

[1] https://github.com/aptivate/python-tika
[2] http://lucene.apache.org/pylucene/jcc/
[3] http://docs.python.org/2/library/xmlrpclib.html


++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

[VOTE] Apache OODT 0.6 Release Candidate #1

[Mattmann, Chris A (398J)]

Hi Guys,

A first candidate for the Apache OODT 0.6 release is available at:

http://people.apache.org/~mattmann/apache-oodt-0.6/rc1/


The release candidate is a zip archive of the sources in:

http://svn.apache.org/repos/asf/oodt/tags/0.6/


The SHA1 checksum of the archive is
32a343a2b23799d39de4d37894b8184b3875b7fd

A staged M2 repository can also be found on repository.apache.org here:

https://repository.apache.org/content/repositories/orgapacheoodt-114/


Please vote on releasing this package as Apache OODT 0.6.
The vote is open for the next 72 hours and passes if a majority of at
least three +1 OODT PMC votes are cast.

[ ] +1 Release this package as Apache OODT 0.6
[ ] -1 Do not release this package because...

Here is my +1 for the release.

Cheers,
Chris


++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

Hi Sanjaya,

-Original Message-

From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Monday, July 8, 2013 12:09 AM
To: Airavata Dev 
Cc: "dev@oodt.apache.org" 
Subject: Re: Apache Airavata-OODT Integration

>Hi Chris,
> I have started looking at changing the current implementation to use
>file Name instead of product id. As per the current PGETask wrapper
>implementation, it takes two inputs (Product ID or file path at the remote
>location. If filePath is used force staging should be set. But I am not
>quite sure what it means by force staging).

Force staging I believe controls whether or not the staged files are
overwritten.

> If I am to use the current
>provisions in PGETaskWrapper, then remote file path (Not the file  name)
>has to be given as input. I am not quite sure whether it is ideal to use
>file path instead of file name.

You can easily generate the file path (which does not have to be remote,
in fact, if you think about it, it could easily be local and in Apache
OODT,
we typically ensure it's local by using distributed filesystems like HDFS
or NFS or Gluster to make remote files appear local by pushing that portion
down into the distributed filesystem which we think does a better job of
data movement :) ). To generate the file path you can use CAS-PGE SQLQuery
facility that will allow you to look up e.g., $FileLocation/$Filename based
on met fields, which in turn you can then feed into the path.


>If filename to use as input, then
>FilesStager needs to be customized to  retrieve product references from
>file name. 

See above for an alternative.

>File manager client doesn't have a mechanism to retrieve product
>by file name. But it has mechanism to retrieve product by product name. I
>guess typically both are the same.

Yeah, or the other easy mechanism is simply to issue a query, e.g., build
yourself a Filename query and then query the FM Catalog.

>One drawback of this approach is that it
>doesn't support list of product names. The method getProductReferences
>which returns list of products is based on back end implementation that is
>based on product id, through actual input is product (Product with just
>product name set is not possible to as input). Please let me know your
>thoughts.

See above.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++



>
>
>
>
>On Mon, Jun 17, 2013 at 5:52 PM, Sanjaya Medonsa
>wrote:
>
>> Thanks Chris. I'll update the implementation to use file name instead of
>> OODT product id.
>>
>> Cheers,
>> Sanjaya
>>
>>
>> On Sun, Jun 16, 2013 at 12:51 AM, Mattmann, Chris A (398J) <
>> chris.a.mattm...@jpl.nasa.gov> wrote:
>>
>>> Hey Sanjaya, sure +1 use the Filename. It's not guaranteed to be
>>>unique,
>>> but you can easily just pop the first one off the top (latest) and take
>>> that (since it's sorted by product received time). You may check out
>>>the
>>> pcs-core module and some of its internal classes like FileManagerUtils
>>> to see some cool helper functions that could aid in this regard.
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattm...@nasa.gov
>>> WWW:  http://sunset.usc.edu/~mattmann/
>>> ++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++
>>>
>>>
>>>
>>>
>>>
>>>
>>> -Original Message-
>>> From: Sanjaya Medonsa 
>>> Reply-To: "d...@airavata.apache.org" 
>>> Date: Saturday, June 15, 2013 4:04 AM
>>> To: Airavata Dev 
>>> Subject: Re: Apache Airavata-OODT Integration
>>>
>>> >Thanks Chris for your help! Working directory is available in
>>> >JobExecuti

Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product Type Metadata

[Mattmann, Chris A (398J)]

Thanks Bruce -- attachment didn't come through (I think the mailing list
strips them) so if you want to send me the attachment,
chris.a.mattm...@nasa.gov
works for that.

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Bruce Barkstrom 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, June 28, 2013 12:49 PM
To: "dev@oodt.apache.org" 
Subject: Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product
Type Metadata

>
>
>
>It's a reasonable start.  I had done something similar
>for CERES.  However, I think there are some kinds
>of collections (like Rock Cores and the NOAA Emergency
>Response Imagery collection) where the number of layers
>is different.
>
>Also, since I don't think you'd seen my examples of the
>FCA web site approach, I've attached my presentation
>for the WMSCI meeting the week after next.  Unzip
>the file (it's got the paper in pdf form).  The presentation
>is the .ppt file.  There are two example web sites included.
>You should be able to link from the appropriate pages to
>the web sites.  The first example has the navigation from
>no attributes to enough to select an object.  The second
>is a reasonable facsimile to the way the NOAA storm damage
>site works in the top level or two.  Note particularly the lack
>of having to input a keyword to navigate through the site.
>
>Thanks for the response.
>
>Bruce B.
>
>On Fri, Jun 28, 2013 at 2:01 PM, Mattmann, Chris A (398J)
> wrote:
>
>Hey Bruce,
>
>Totally agree. In Apache OODT, we have the layered, hierarchical attribute
>approach, already, wherein which the first layer is the "Product Type"
>concept,
>capturing aggregate information bout a set of products (name, id,
>classification,
>aggregate free form metadata, sets of metadata extractors, and
>"versioning" or
>file placement on disk policy, etc.). The next level is the Product level,
>where
>we capture product (or frequently changing) level metadata; apply the
>versioning
>scheme for file placement, and extract metadata using the per product type
>specified
>metadata extractors. Attributes at the product level are hierarchical in
>the sense
>that they are defined by per product type policy (though we can accept
>others) and
>are specified in the OODT File Manager server through the use of a
>particular
>ValidationLayer, and RepositoryManager selected for the running system.
>
>HTH!
>
>Cheers,
>Chris
>
>++
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattm...@nasa.gov
>WWW:  http://sunset.usc.edu/~mattmann/
>++
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA
>++
>
>
>
>
>
>
>-Original Message-
>From: Bruce Barkstrom 
>Reply-To: "dev@oodt.apache.org" 
>Date: Wednesday, June 26, 2013 2:50 PM
>To: "dev@oodt.apache.org" 
>Subject: Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product
>Type Metadata
>
>
>>I'll at least mention an issue we've discovered when examining existing
>>Earth science data collections: the collections have a layered or
>>hierarchical
>>structure, with each layer having different kinds of metadata (or, if I
>>did
>>it
>>a bit more formally, different types of metadata).  In Formal Concept
>>Analysis, it shows up as hierarchical attributes.
>>
>>The easiest example is perhaps the NOAA Emergency Response Imagery
>>collection, where the top layer divides about 250,000 digital photos into
>>about twenty collections - one for each disastrous storm that needed an
>>emergency response.  Examples include Hurricane Sandy, Hurricane
>>Katrina, and the Tuscaloosa Tornado.  The top level metadata fields
>>are (Storm Name, Storm Date, and Storm Location).  One level below
>>that, the storm collections bifurcate into image collections organized
>

Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product Type Metadata

[Mattmann, Chris A (398J)]

Hey Bruce,

Totally agree. In Apache OODT, we have the layered, hierarchical attribute
approach, already, wherein which the first layer is the "Product Type"
concept,
capturing aggregate information bout a set of products (name, id,
classification,
aggregate free form metadata, sets of metadata extractors, and
"versioning" or
file placement on disk policy, etc.). The next level is the Product level,
where
we capture product (or frequently changing) level metadata; apply the
versioning
scheme for file placement, and extract metadata using the per product type
specified
metadata extractors. Attributes at the product level are hierarchical in
the sense
that they are defined by per product type policy (though we can accept
others) and
are specified in the OODT File Manager server through the use of a
particular 
ValidationLayer, and RepositoryManager selected for the running system.

HTH!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Bruce Barkstrom 
Reply-To: "dev@oodt.apache.org" 
Date: Wednesday, June 26, 2013 2:50 PM
To: "dev@oodt.apache.org" 
Subject: Re: [jira] [Resolved] (OODT-639) Add a versioner based on Product
Type Metadata

>I'll at least mention an issue we've discovered when examining existing
>Earth science data collections: the collections have a layered or
>hierarchical
>structure, with each layer having different kinds of metadata (or, if I
>did
>it
>a bit more formally, different types of metadata).  In Formal Concept
>Analysis, it shows up as hierarchical attributes.
>
>The easiest example is perhaps the NOAA Emergency Response Imagery
>collection, where the top layer divides about 250,000 digital photos into
>about twenty collections - one for each disastrous storm that needed an
>emergency response.  Examples include Hurricane Sandy, Hurricane
>Katrina, and the Tuscaloosa Tornado.  The top level metadata fields
>are (Storm Name, Storm Date, and Storm Location).  One level below
>that, the storm collections bifurcate into image collections organized
>by airplane flight path or organized by coarse geographic boxes.
>Each storm collection has this bifurcation - and so once a user
>has distinguished which storm collection he or she is interested in,
>Storm Name (or Date or Location) is not helpful in distinguishing
>a flight path collection from a geographic collection.
>
>The level below that for the flight path requires distinguishing which
>flight path is the one you want.  Once you decide that, you can get
>a zipped file with several hundred jpg images.  On the other hand,
>if you go the geographic boxes, you can get individual images.
>The metadata types for the boxes are quire different from the metadata
>for the flight paths.
>
>The same kind of structure crops up for rock core archives (wells, boxes
>in a well, core fragments in a box), as well as for many of the satellite
>data collections.
>
>The usual library science approach (e.g. Functional Requirements
>for Bibliographic Records, and its relatives) assumes all the inventoried
>objects in an archive should have the same types of metadata.  With
>the Emergency Response Imagery collection, the "Responsible Party"
>field is sort of "Dept. of Commerce, NOAA, National Ocean Survey,
>National Geodetic Survey, Emergency Response Imagery Project"
>and its the same for every image or zipped image file.  At least in
>this case, an "Author" metadata field (assuming that Dept. of Commerce
>is a Responsible Party and that a Responsible Party is equivalent
>to an Author) is of NO help at distinguishing which of the quarter
>million files you might want to get.  In FCA, the algorithms would
>ignore the field if applied to the whole collection.
>
>Some care would appear to be appropriate.
>
>Bruce b.
>
>On Sat, Jun 22, 2013 at 1:31 PM, Chris A. Mattmann (JIRA)
>wrote:
>
>>
>>  [
>> 
>>https://issues.apache.org/jira/browse/OODT-639?page=com.atlassian.jira.pl
>>ugin.system.issuetabpanels:all-tabpanel]
>>
>> Chris A. Mattmann resolved OODT-639.
>> 
>>
>> Resolution: Fixed
>>
>> - fixed in r1495761. Added unit test and doc updates to all product
>>types
>> suggesting how to use the new versioner.
>>
>> > Add a versioner based on Product Type Metadata
>> > --
>> >
>> > Key: OODT-639
>> > URL: https://issues.apache.org/jira/browse/OODT-639
>> > Project: OODT
>> >  Issue Type: New Featur

Re: PEAR packages fixed on pear.apache.org/oodt channel

[Mattmann, Chris A (398J)]

Thank you Gav, this is great news!

I'll try and roll the 0.6 RC #1 this weekend.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Thursday, June 27, 2013 4:43 AM
To: "dev@oodt.apache.org" 
Subject: PEAR packages fixed on pear.apache.org/oodt channel

>As per https://issues.apache.org/jira/browse/INFRA-5910 the PEAR release
>packages were faulty.
>
>I rebuilt them for 0.4 and also created a 0.5 PEAR release from the 0.5
>tag
>in svn. I had to alter the package.xml
>slightly in order to fix both but there was also a problem with the
>packaging system on the pear.a.o host so I rebuilt them
>locally, uploaded and then registered them to the channel from there.
>
>I tested all the rebuilt 0.4 and 0.5 releases on pear.a.o with ''pear
>install oodt/Apache_OODT/$package'' and all installed without
>any problems for me.
>
>Please check and test and see if the same is true for you too.
>
>I'll prepare the trunk package.xml files for the next release iteration
>tomorrow.
>
>Thanks
>
>Gav...
>
>

Re: OODT build issues (Fwd: APPS Third-party (TPS) software component list)

[Mattmann, Chris A (398J)]

Hi Greg,

Thanks for your questions (copying the dev@ list per our discussion).
My replies inline below:

-Original Message-

From: Greg Earle 
Date: Wednesday, June 19, 2013 10:55 PM
To: jpluser 
Cc: Greg Earle , "Lee, Hyun H (398J)"

Subject: OODT build issues (Fwd: APPS Third-party (TPS) software component
list)

>Hi Chris (and Hyun),
>
>I was asked by Costin Radulescu to install Apache OODT on one of our
>Dev systems.  It's a Linux box running 64-bit Red Hat RHEL 6.3.
>
>He had sent me a TPS request (formalized form we use in MIPL) asking
>for the "Latest" version, but later on sent an Excel spreadsheet that
>specifically mentioned version 0.3.  (But I think that was only because
>he made up the spreadsheet some months back.)

Yep latest version is 0.6-SNAPSHOT (trunk). Should be released in the
next week or so, but it's fairly stable right now. I'm going to RM it.

>
>I first tried using Maven 3.0.1 but it splatted all over the place.

0.3 will do that to you since we didn't support Maven3 until OODT-369 [1]
which will ship in 0.6 (and is available now in trunk/0.6-SNAPSHOT).


>I fell back to Maven 2.2.1 which seemed to make OODT a lot happier.

Heh, yep.

>
>Anyway, I was able to build 0.3 OK; but then I thought I should try to
>build the latest & greatest 0.5.  The following message chronicles the
>2 major problems I ran into (in copious detail - sorry) with both 0.5
>and 0.4 (which I tried after 0.5 failed).  Executive summary:
>
>(1) Catalog and Archive Service Generic Multi-valued Metadata Container
>
>A couple of the tests seem to have a race condition where they try to
>clean up their mess and remove a temporary metadata .tests directory
>before all the files are completely removed (NFS issue; possibly due
>to caching/sync), provoking 2 ENOTEMPTY errors.

Yep this is an NFS issue -- you can always change this with a
java.io.tmpDir
property (passed to Maven with mvn
-Djava.io.tmpDir=/some/where/not/on/nfs).
Issue described in detail here:

https://issues.apache.org/jira/browse/OODT-110


>
>(2) CAS Protocol SFTP Implementation
>
>The "TestJschSftpProtocol" test is failing because it can't contact
>whatever server it is trying to contact ("Connection refused").

Odd, it should be trying to connect locally.

>
>If you could give a quick glance-over to this (overly verbose, sorry)
>transcript and suggest anything (like resources to consult) that would
>help me potentially get past these 2 issues, I'd appreciate it.

No worries -- if you get failing tests like the above and they are blocking
progress you can always type mvn -Dmaven.skip.tests=true install and then
that should get you past.

>
>In the meantime I'm going to ask Costin if he can get along with using
>version 0.3 in the interim.

Try 0.6-SNAPSHOT -- should work better :)

Also happy to answer any further questions here on list. Cheers!

Chris

[1] https://issues.apache.org/jira/browse/OODT-369

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++



>
>Begin forwarded message:
>
>> From: Greg Earle 
>> Date: June 19, 2013 10:28:57 PM PDT
>> To: Costin Radulescu , "Levoe, Steven R (398D)"
>>
>> Cc: Alex Leung , Greg Earle
>>
>> Subject: Re: APPS Third-party (TPS) software component list
>> content-type: text/plain; charset=us-ascii
>> 
>> On Jun 19, 2013, at 5:17 PM, Greg Earle wrote:
>> 
>>> I installed the "apache-oodt-0.3-src.tar.gz" file into "external"
>>> because I think you said it had been donated(?) to The Apache
>>>Foundation
>>> and thus was no longer JPL/NASA Proprietary ... right?
>>> 
>>> I compared the Apache version to the old JPL one and the JPL one
>>> just has a "oodt.jar" file (with some docs) - but the Apache one
>>> looks like it's a zillion ".java" files that need to be built with
>>> Maven, which isn't installed yet.
>> 
>> [Removed Janet from the long boring technical discussion.  Sorry Janet.]
>> 
>> Update:
>> 
>> OK - so I found a Maven RPM
>>(apache-maven-3.0.3-1.el6.sroycode.noarch.rpm)
>> and installed it.
>> 
>> The README.txt file in the OODT 0.3 "src" directory says to just run
>> "mvn install" to build it.  I tried that but it croaked.
>> 
>> Then I looked in the README.txt file again and it says it uses Maven2.
>> 
>> So I installed Maven 2.2.1 instead (into externals, alongside
>>apache-oodt -
>> no RPM for it that I could find which didn't have a ton of dependencies)
>> and this time it built all the way through; albeit with a bunch of
>> "WARNING" messages.
>> 
>> It looks like it built a bunch of .j

Re: [DISCUSS] Apache OODT 0.6 release

[Mattmann, Chris A (398J)]

Hi Shakeh,

Thanks. I don't think this is a blocker, but it would be nice
to get fixed. I would say this: I intend to roll and RC in the
next week. If you can get this fixed, working with Gav, by then,
that would be great -- your thoughts?

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: , Shakeh Khudikyan 
Reply-To: "dev@oodt.apache.org" 
Date: Wednesday, June 19, 2013 11:03 AM
To: "dev@oodt.apache.org" 
Subject: Re: [DISCUSS] Apache OODT 0.6 release

>Hi Chris,
>
>We need to make sure we update all the package.xml for PEAR. I know
>filemgr, sso, and balance each have one.
>
>Just made OODT-633 - Update PEAR package.xml
>.
>
>-Shakeh 
>
>
>
>
>On 6/16/13 8:25 AM, "Chris Mattmann"  wrote:
>
>>Hey Guys,
>>
>>I've got some free time and would like to roll the 0.6 release. If
>>there's something that needs to get in that I missed let me know. We've
>>got 60 issues for 0.4 that are complete and I think that's pretty good
>>and it's time for a release.
>>
>>If I don't hear any objections I'll spin the RC by mid-week.
>>
>>Cheers,
>>Chris
>>
>>
>

Re: [SECURITY] Frame injection vulnerability in published Javadoc

[Mattmann, Chris A (398J)]

Thanks Sean -- got it.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: "ke...@apache.org" 
Reply-To: "dev@oodt.apache.org" 
Date: Thursday, June 20, 2013 6:18 AM
To: "dev@oodt.apache.org" 
Subject: Fwd: [SECURITY] Frame injection vulnerability in published Javadoc

>See the forwarded message below. Yuck.
>
>I used the patching tool mentioned to fix OODT's Javadocs online at
>http://oodt.apache.org/
>
>Whoever cuts the next OODT website MUST use Java 1.7.0_23 or higher.
>
>--k
>
>Begin forwarded message:
>
>> From: Mark Thomas 
>> Subject: [SECURITY] Frame injection vulnerability in published Javadoc
>> Date: 2013 June 20 3.29.23a CDT
>> To: committ...@apache.org
>> Cc: r...@apache.org
>> Reply-To: "infrastruct...@apache.org" 
>> 
>> Hi All,
>> 
>> Oracle has announced [1], [2] a frame injection vulnerability in Javadoc
>> generated by Java 5, Java 6 and Java 7 before update 22.
>> 
>> The infrastructure team has completed a scan of our current project
>> websites and identified over 6000 instances of vulnerable Javadoc
>> distributed across most TLPs. The chances are the project(s) you
>> contribute to is(are) affected. A list of projects and the number of
>> affected Javadoc instances per project is provided at the end of this
>> e-mail.
>> 
>> Please take the necessary steps to fix any currently published Javadoc
>> and to ensure that any future Javadoc published by your project does not
>> contain the vulnerability. The announcement by Oracle includes a link to
>> a tool that can be used to fix Javadoc without regeneration.
>> 
>> The infrastructure team is investigating options for preventing the
>> publication of vulnerable Javadoc.
>> 
>> The issue is public and may be discussed freely on your project's dev
>>list.
>> 
>> Thanks,
>> 
>> Mark (ASF Infra)
>> 
>> 
>> 
>> [1]
>> 
>>http://www.oracle.com/technetwork/topics/security/javacpujun2013-1899847.
>>html
>> [2] http://www.kb.cert.org/vuls/id/225657
>> 
>> Project  Instances
>> abdera.apache.org1
>> accumulo.apache.org  2
>> activemq.apache.org  105
>> any23.apache.org 13
>> archiva.apache.org   4
>> archive.apache.org   13
>> aries.apache.org 7
>> avro.apache.org  23
>> axis.apache.org  5
>> beehive.apache.org   16
>> bval.apache.org  12
>> camel.apache.org 786
>> cayenne.apache.org   4
>> chemistry.apache.org 6
>> click.apache.org 3
>> cocoon.apache.org6
>> commons.apache.org   34
>> continuum.apache.org 9
>> creadur.apache.org   19
>> crunch.apache.org4
>> ctakes.apache.org2
>> curator.apache.org   4
>> cxf.apache.org   6
>> db.apache.org39
>> directory.apache.org 4
>> empire-db.apache.org 1
>> felix.apache.org 5
>> flume.apache.org 5
>> geronimo.apache.org  241
>> giraph.apache.org6
>> gora.apache.org  3
>> hadoop.apache.org21
>> hbase.apache.org 2
>> hive.apache.org  4
>> hivemind.apache.org  10
>> incubator.apache.org 355
>> jackrabbit.apache.org9
>> jakarta.apache.org   39
>> james.apache.org 53
>> jena.apache.org  5
>> juddi.apache.org 3
>> lenya.apache.org 46
>> logging.apache.org   111
>> lucene.apache.org713
>> manifoldcf.apache.org112
>> marmotta.apache.org  1
>> maven.apache.org 1623
>> maventest.apache.org 1178
>> mina.apache.org  2
>> mrunit.apache.org3
>> myfaces.apache.org   348
>> nutch.apache.org 8
>> oltu.apache.org  11
>> oodt.apache.org  1
>> ooo-site.apache.org  1
>> oozie.apache.org 10
>> openjpa.apache.org   20
>> opennlp.apache.org   9
>> pdfbox.apache.org1
>> pig.apache.org   7
>> pivot.apache.org 1
>> poi.apache.org   1
>> portals.apache.org   35
>> river.apache.org 2
>> santuario.apache.org 1
>> shale.apache.org 55
>> shiro.apache.org 3
>> sling.apache.org 2
>> sqoop.apache.org 4
>> struts.apache.org190
>> subversion.apache.org3
>> synapse.apache.org   1
>> syncope.apache.org   2
>> tapestry.apache.org  6
>> tika.apache.org  9
>> tiles.apache.org 12
>> turbine.apache.org   100
>> tuscany.apache.org   4
>> uima.apache.org  12
>> velocity.apache.org  41
>> whirr.apache.org 2
>> wicket.apache.org3
>> wink.apache.org  13
>> ws.apache.org22
>> xalan.apache.org 1
>> xerces.apache.org5
>> xml.apache.org  

Re: Help with OODT-123

[Mattmann, Chris A (398J)]

Hey Tom,

Thanks for scoping this out. My feeling behind the intent of OODT-123
is to not replace XML-RPC (which as a low level library still maintains
lots of backwards compat dependencies e.g., on PHP and Python libraries
that client connect to FM, WM and RM services). I think the intent would
be to take the approach that we have taken in other places e.g., with
Curator and the work that Ross Laidlaw is doing in cas-product, and to
look at a REST-ful API based on JAX-RS and Apache CXF. Now there are
already a set of useful features in cas-product, and curator, that expose
many of the underlying FM ingestion, and curation functionalities, and
data delivery.

My recommendation would be:

1. Familiarize yourself with cas-product
2. Familiarize yourself with the curator
3. Decide what services they are exposing in JAX-RS (the cas-product ones
are being developed as a GSoC 2013 project by Ross Laidlaw so you may want
to coordinate with him; curator ones have existed for a while)
4. Suggest some concrete updates and new services to be placed in e.g.,
cas-product or curator based on looking at FM XML-RPC and its API and
looking
at gaps there.

HTH.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Tuesday, June 18, 2013 7:41 AM
To: "dev@oodt.apache.org" 
Subject: Help with OODT-123

>Alright guys, after chatting with Chris, I thought I'd kick off my help
>with looking at OODT-123 and OODT-124.
>
>Now clearly I'm new to how all the components hang together but OODT-123
>seems obvious enough but I'm curious to know quite what "Port any
>filemanger specific services currently in the curator over to the rest
>filemanager services area"
>
>Now my way of working out what is what would be to look for filemgr
>dependencies and acertain whether they can be removed. I assume that if
>you are aiming to go rest based for communications then things like
>direct connections to the XMLRepositoryManager etc would need to be
>replaced with a restful representation.
>
>Am I on the right path or way off?
>
>Cheers
>
>Tom

Re: [Ganglia plugin OODT-219] Command line presentation (draft)

[Mattmann, Chris A (398J)]

Thanks Rajith,

What command line are you talking about -- are you building a CLI
for the Ganglia extensions? And would this be part of the Resmgr command
line?

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Rajith Siriwardana 
Date: Sunday, June 16, 2013 11:17 AM
To: jpluser , jpluser
, dev 
Subject: [Ganglia plugin OODT-219] Command line presentation (draft)

>Following draft categorization, shows how the CLI commands and responses
>should be implemented.Grid info:
>
>* args: gmetad host, port, ³--gridinfo² /³--summary²
>
>
>
>
>* response :  
>
>
>summarized clusters: per cluster (name, owner, latitude & longitude,
>online Nodes, offline Nodes)
>
>
>Cluster info:
>
>
>* args : gmetad host, port, ³--clusterinfo -name  name>²
>
>
>(multiple clusters may be there by the same name, in that case all the
>same name clusters)
>
>
>* response: 
>Cluster name, owner, latitude & longitude, url,
> local time
>Total CPUs, Total Disk, Total Memory
>summarized Nodes : per node (name, IP,location, offline/online,
> load, memory_free, disk_free)
>
>
>(We can say a particular Node is offline by TN
> and TMAX value. gmetad, a host is considered offline and is ignored if
>TN > 4 * TMAX.
>source:
>http://entropy.gforge.inria.fr/ganglia.html)
>TN :
> TN
> value is the number of seconds since the metric was last updated
>TMAX:
>The maximum time in seconds between gmetric calls
>
>
>
>Node info:
>From Node ID and ip we should be able to get status of a
> node.  I guess OODT ³Node Id²  and the ganglia "host name" are the same.
>
>* args:gmetad host, port, ³-nodeId ²
>* args:gmetad host, port, ³-nodeurl ²
>* response: Node
> info: Name, IP,(online/offline), reported, location, gmond started, all
>32 default metrics + added metrics
>
>
>
>Suggestions and your advices are warmly welcome.
>
>
>Thank you,
>Rajith
>

Re: [DISCUSS] Apache OODT 0.6 release

[Mattmann, Chris A (398J)]

Hey Lewis,

Great question - no clue on the intermittent failures. I think it may
be due more to some unit tests not cleaning up than anything else. But
worth investigating, but we've had them for a while so more of a good
to figure out, but I don't think blocker to RC.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Lewis John Mcgibbney 
Reply-To: "dev@oodt.apache.org" 
Date: Sunday, June 16, 2013 10:24 AM
To: "dev@oodt.apache.org" 
Subject: Re: [DISCUSS] Apache OODT 0.6 release

>Nice one Chris.
>Out of interest, is there any reason for intermittent failures for the
>Jenkins builds? I see them every now and then and wondered.
>+1 to RC.
>
>On Sunday, June 16, 2013, Chris Mattmann  wrote:
>> Hey Guys,
>>
>> I've got some free time and would like to roll the 0.6 release. If
>> there's something that needs to get in that I missed let me know. We've
>> got 60 issues for 0.4 that are complete and I think that's pretty good
>> and it's time for a release.
>>
>> If I don't hear any objections I'll spin the RC by mid-week.
>>
>> Cheers,
>> Chris
>>
>>
>>
>
>-- 
>*Lewis*

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

Hey Sanjaya, sure +1 use the Filename. It's not guaranteed to be unique,
but you can easily just pop the first one off the top (latest) and take
that (since it's sorted by product received time). You may check out the
pcs-core module and some of its internal classes like FileManagerUtils
to see some cool helper functions that could aid in this regard.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Saturday, June 15, 2013 4:04 AM
To: Airavata Dev 
Subject: Re: Apache Airavata-OODT Integration

>Thanks Chris for your help! Working directory is available in
>JobExecutionContext in Airavata and directory can easily be retrieved.
>Issue in my case is that, from XBaya GUI I take product id as input not
>the
>file name. Internally file stager query the file manager using product id
>to retrieve product reference and corresponding file name to stage the
>file
>into input dir. Since this product id to file name mapping happens
>internally during the file staging, my implementation don't have access to
>filename unless I query the file manager to retrieve the corresponding
>file
>name using product id.
>
>One of the major issue in my implementation seems that I use OODT product
>id as input, not the file name. Should I change my implementation to use
>file name instead of product id ?
>
>Best Regards,
>Sanjaya
>
>
>On Fri, Jun 14, 2013 at 8:51 PM, Mattmann, Chris A (398J) <
>chris.a.mattm...@jpl.nasa.gov> wrote:
>
>> Hey Sanjaya,
>>
>> Easy, see the attached PGEConfig.xml here:
>>
>> http://paste.apache.org/6OGW
>>
>> In that file:
>>
>> 1. We compute the staged file path by computing JobDir
>> 2. We create in the exe block a staged input dir
>> 3. We stage the files just using cps in the exeBlock (could have
>> just as easily used fileStager)
>> 4. We know that the file is [JobInputDir]/[Filename]
>>
>> HTH.
>>
>> Cheers,
>> Chris
>>
>> ++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Sanjaya Medonsa 
>> Reply-To: "d...@airavata.apache.org" 
>> Date: Friday, June 14, 2013 5:02 AM
>> To: Airavata Dev 
>> Subject: Re: Apache Airavata-OODT Integration
>>
>> >Thanks Chris for your input. I actually use the PGETaskInstance for
>>file
>> >staging with minimal additional code. But my issue issue not with the
>>file
>> >staging. As per my current implementation, application inputs product
>>id.
>> >Then using the capabilities in PGETaskInstance class, it does the file
>> >staging. But my issue is that during the file staging product is
>>mapped to
>> >a file in specified working directory. I don't have a way to retrieve
>>the
>> >staged file name, as it is not recorded in Metadata (For this purpose,
>>I
>> >query the FileManager again to get the corresponding reference name
>>for a
>> >given product id). I need the staged file path, since I modify the
>>input
>> >product id into staged file path prior to actual workflow invocation.
>> >Basically I am looking for some implementation where I can easily
>> >retrieve,
>> >staged file path for a given product id.
>> >
>> >Cheers,
>> >Sanjaya
>> >
>> >
>> >On Wed, Jun 12, 2013 at 10:04 PM, Mattmann, Chris A (398J) <
>> >chris.a.mattm...@jpl.nasa.gov> wrote:
>> >
>> >> Hi Sanjaya,
>> >>
>&

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

+5000 great idea, as usual my friend.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Suresh Marru 
Reply-To: "d...@airavata.apache.org" 
Date: Wednesday, June 12, 2013 9:51 AM
To: "d...@airavata.apache.org" 
Cc: "dev@oodt.apache.org" 
Subject: Re: Apache Airavata-OODT Integration

>On Jun 12, 2013, at 12:34 PM, "Mattmann, Chris A (398J)"
> wrote:
>
>> See above for info on file staging. I would strongly encourage you not
>> to reimplement CAS-PGE in Airavata -- it's pretty functional and
>>expressive
>> anyways and I would work to figure out how to make Airavata leverage
>> CAS-PGE.
>
>+ 1. 
>
>Sanjaya, Airavata and OODT communities,
>
>Any volunteers to write a paper on "A tale of two apache workflow
>systems: Airavata and OODT"?
>
>Given the page limit and to keep in scope, I suggest to leave out the use
>cases of the systems and focus on software architectures. A detailed
>technical paper comparing and contrasting the features and identifying
>potential collaborative components.
>
>If you want a deadline, how about August 15th to WORKS workshop -
>http://works.cs.cardiff.ac.uk/
>
>Suresh

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

Hi Sanjaya,

-Original Message-

From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Monday, June 10, 2013 5:20 PM
To: "d...@airavata.apache.org" 
Cc: "dev@oodt.apache.org" 
Subject: Re: Apache Airavata-OODT Integration

>Hi Chris,
>   On configuration, I have get rid of all the configuration files,
>including pge-config.xml. All the required configurations are
>programmatically set.  Configurations such FileManagerServer URL are
>configured in the airavata-server.properties file. I'll update the review
>request with modified details.

Great work!


>   Still I am not quite clear on how to retrieve staged file path
>properly. Currently I am using getStagedFilePath method
>in ApacheAiravataWorkFlowInstanceImpl to regenerate the staged file path.
>While I am going through the OODT code that I have seen method in
>DataTransferer to notify FileManagerServer once transfer is completed. But
>I couldn't see the same for product retrieval.

Example:
http://svn.apache.org/repos/asf/oodt/trunk/pge/src/test/resources/pge-confi
g.xml


Review Board tickets:
https://reviews.apache.org/r/4746/

https://reviews.apache.org/r/5382/


JIRA issue source (in OODT since 0.4):
  https://issues.apache.org/jira/browse/OODT-443


>   As you suggested I'll improve my workflow using Apache Tika. I'd
>like to continue this as an Parallal task. While modifying staging
>implementation based on community feedback, currently I am looking at
>ingesting output back to OODT.

See above for info on file staging. I would strongly encourage you not
to reimplement CAS-PGE in Airavata -- it's pretty functional and expressive
anyways and I would work to figure out how to make Airavata leverage
CAS-PGE.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++



>
>
>
>On Wed, Jun 5, 2013 at 12:11 AM, Mattmann, Chris A (398J) <
>chris.a.mattm...@jpl.nasa.gov> wrote:
>
>> Hi Sanjaya,
>>
>> I think starting out with /bin/ls would be good, maybe like a /bin/ls
>> workflow, and then for each file returned, maybe run Apache Tika and
>> extract its metadata and then pipe that to a file?
>>
>> How about that?
>>
>> Cheers,
>> Chris
>>
>> ++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Sanjaya Medonsa 
>> Reply-To: "d...@airavata.apache.org" 
>> Date: Tuesday, June 4, 2013 5:31 AM
>> To: "d...@airavata.apache.org" 
>> Cc: "dev@oodt.apache.org" 
>> Subject: Re: Apache Airavata-OODT Integration
>>
>> >Hi Chris,
>> > Please see my comments below on the two items.
>> >
>> >Configuration : It should be possible to set them programmatically.
>> >Actually I have implemented partly it for file staging information.
>>I'll
>> >work to get rid of the other configuration files.
>> >
>> >Staged File Path : I'll work on the suggested approach, though I am not
>> >fully understand it at the moment. I guess I need to go through bit
>>more
>> >on
>> >CAS-PGE and come back to you on the proposed approach.
>> >
>> >Currently I am testing this by wrapping /bin/ls command as GFac
>>service. I
>> >may need to test this with real workflow. Could you please provide me
>>know
>> >some guidance on better scenario to test this.
>> >
>> >Cheers,
>> >Sanjaya
>> >
>> >
>> >
>> >
>> >On Mon, Jun 3, 2013 at 8:17 PM, Mattmann, Chris A (398J) <
>> >chris.a.mattm...@jpl.nasa.gov> wrote:
>&g

Re: Getting started with issues

[Mattmann, Chris A (398J)]

Thanks Tom, both have been committed and resolved by me :)

Thanks for your contributions!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, June 7, 2013 11:04 AM
To: "dev@oodt.apache.org" 
Subject: Re: Getting started with issues

>Dunno if someone gets notified or what, so I've provided 2 minor patches
>
>OODT-572
>OODT-559
>
>Cheers
>
>Tom
>
>On 07/06/13 02:17, Mattmann, Chris A (398J) wrote:
>> Hey Tom,
>>
>> Great to have you working issues on the list! We talked about
>>classifying
>> them along a scale (will dig up thread from Mike Joyce and Gav from
>> earlier)
>> but I don't think we've done too much yet in regards to curating the
>>data
>> for them.
>>
>> However if you find some you are interested in, let me or anyone on
>> the list know and we can provide more information about them.
>>
>> Cheers,
>> Chris
>>
>> ++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Tom Barber 
>> Reply-To: "dev@oodt.apache.org" 
>> Date: Wednesday, June 5, 2013 1:03 PM
>> To: "dev@oodt.apache.org" 
>> Subject: Getting started with issues
>>
>>> Hi guys,
>>>
>>> I had a chat with Chris the other week and said I'd be interested in
>>> helping out fixing bugs and stuff as a part time Java & Scala dev.
>>>
>>> Obviously I don't want to work on stuff that other people are doing, so
>>> what's the best way to get started picking (easy) tasks.
>>>
>>> Thanks
>>>
>>> Tom
>

Re: Getting started with issues

[Mattmann, Chris A (398J)]

Hey Tom,

Great to have you working issues on the list! We talked about classifying
them along a scale (will dig up thread from Mike Joyce and Gav from
earlier)
but I don't think we've done too much yet in regards to curating the data
for them.

However if you find some you are interested in, let me or anyone on
the list know and we can provide more information about them.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Tom Barber 
Reply-To: "dev@oodt.apache.org" 
Date: Wednesday, June 5, 2013 1:03 PM
To: "dev@oodt.apache.org" 
Subject: Getting started with issues

>Hi guys,
>
>I had a chat with Chris the other week and said I'd be interested in
>helping out fixing bugs and stuff as a part time Java & Scala dev.
>
>Obviously I don't want to work on stuff that other people are doing, so
>what's the best way to get started picking (easy) tasks.
>
>Thanks
>
>Tom

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

Hi Sanjaya,

I think starting out with /bin/ls would be good, maybe like a /bin/ls
workflow, and then for each file returned, maybe run Apache Tika and
extract its metadata and then pipe that to a file?

How about that?

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Tuesday, June 4, 2013 5:31 AM
To: "d...@airavata.apache.org" 
Cc: "dev@oodt.apache.org" 
Subject: Re: Apache Airavata-OODT Integration

>Hi Chris,
> Please see my comments below on the two items.
>
>Configuration : It should be possible to set them programmatically.
>Actually I have implemented partly it for file staging information. I'll
>work to get rid of the other configuration files.
>
>Staged File Path : I'll work on the suggested approach, though I am not
>fully understand it at the moment. I guess I need to go through bit more
>on
>CAS-PGE and come back to you on the proposed approach.
>
>Currently I am testing this by wrapping /bin/ls command as GFac service. I
>may need to test this with real workflow. Could you please provide me know
>some guidance on better scenario to test this.
>
>Cheers,
>Sanjaya
>
>
>
>
>On Mon, Jun 3, 2013 at 8:17 PM, Mattmann, Chris A (398J) <
>chris.a.mattm...@jpl.nasa.gov> wrote:
>
>> Hi Sanjaya,
>>
>> -Original Message-
>>
>> From: Sanjaya Medonsa 
>> Reply-To: "d...@airavata.apache.org" 
>> Date: Thursday, May 30, 2013 5:12 AM
>> To: "dev@oodt.apache.org" ,
>>"d...@airavata.apache.org"
>> 
>> Subject: Apache Airavata-OODT Integration
>>
>> >Hi,
>> > I have worked on the Apache Airavata integration with Apache
>>OODT. As
>> >a first step, I have implemented integration with Apache OODT file
>> >manager component.
>>
>> Great work!!
>>
>> Comments below:
>>
>> >  1. Introduce a new GFac Schema type called OODTProduct which
>>takes
>> >APache OODT product IDs as input.
>> >  2. Implemented new pre GFac Handler by extending Apache OODT
>> >PgeTaskInstance to stage the corresponding file into the working
>> >directory.
>> >  3. Once file is staged, input parameter with OODT product id is
>> >replaced with path of the staged file for downstream processing
>> >
>> >I have tested the implementation with Gfac application which wraps
>>/bin/ls
>> >command. Application takes product id as input and stage corresponding
>> >file
>> >into the working directory and /bin/ls is executed against the staged
>> >file.
>> >Hope this is a valid testing scenario.
>> >
>> >Concerns
>> >- Configurations : I have added new configuration file named and
>> >oodt-integration.properties in addition to dynamic_metadata.met and
>> >pge-config.xml files used by OODT. But at the moment there is no item
>> >configured with the oodt-integration.properties.
>>
>> You probably only need the pge-config.xml file. Dynamic metadata, and
>>the
>> task configuration properties can be specified programmatically, right?
>>
>> >- Staged File Name - With the current implementation of
>>PgeTaskInstance it
>> >is not possible to retrieve path of the staged file. Due to this
>> >limitation, I have query the FileManagerServer with product id and
>> >retrieve
>> >the file name and computed the file path using information of working
>> >directory.
>>
>> I'm not sure I understand this? If you store and record the Filename,
>>and
>> FileLocation
>> metadata files, then you can easily retrieve the staged file path via a
>> SQLquery
>> via CAS-PGE by simply setting the FORMAT=('$FileLocation/$Filename') in
>> the response.
>> Can you comment on this?
>>
>> >- Currently it is not possible to execute the workflow using Xbaya due
>>to
>> >validation failure due to new schema type. I have commented out the
>> >relevant validation code for testing purpose.
>>
>> O

Re: [ANNOUNCE] Welcome Arni Sumarlidason as an OODT PMC member!

[Mattmann, Chris A (398J)]

Welcome Arni!!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: "ke...@apache.org" 
Reply-To: "dev@oodt.apache.org" 
Date: Tuesday, June 4, 2013 7:01 AM
To: "dev@oodt.apache.org" 
Cc: "sumarlida...@apache.org" 
Subject: [ANNOUNCE] Welcome Arni Sumarlidason as an OODT PMC member!

>Folks:
>
>I'm happy to report that Arni Sumarlidason has accepted our invitation to
>join the Apache OODT Project Management Committee. WOOT!
>
>Welcome, Arni. Though many of us are already familiar with your work,
>please feel free to make an introduction for yourself.
>
>--k
>

[ANNOUNCE] Apache OODT nets 2 GSoC projects for 2013 (was Re: Interested in participating GSoC 2013 with OODT)

[Mattmann, Chris A (398J)]

Congrats to you both!

Really looking forward to working with our 2 GSoC projects
this summer:

Ross Laidlaw
Rajith Siriwardana


Cheers guys!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Ross Laidlaw 
Reply-To: "dev@oodt.apache.org" 
Date: Tuesday, May 28, 2013 4:57 AM
To: "dev@oodt.apache.org" 
Subject: Re: Interested in participating GSoC 2013 with OODT

>Congratulations Rajith!
>
>Nice work on your project so far and well done for gaining a place on
>GSoC.
>
>I've also been accepted this year and I'd like to echo your comments to
>say
>a massive thank you to Chris and Rishi for giving me this opportunity.
>
>Looking forward to a great summer!
>
>Ross
>
>
>
>On 28 May 2013 01:55, Rajith Siriwardana 
>wrote:
>
>> Hi all,
>>
>> I have been notified that my proposal was accepted for GSOC 2013. Thank
>>you
>> so much to everybody in the OODT community for your encouragements, kind
>> words and welcoming me to your community. Most importantly thank you
>>very
>> much Chris for deciding my proposal is worth sharing your valuable time
>> during the summer. I'm very happy to have this opportunity to work with
>> you.
>>
>> I'm hoping to have a very productive summer and make this project a
>> success.
>>
>> Thanks
>> Rajith
>>
>>
>> T.W.R.O. Siriwardana
>> StMIEEE
>> Undergraduate
>> Department of Computer Science & Engineering
>> Faculty of Engineering
>> University of Moratuwa
>> Sri Lanka.
>>
>>
>> On Tue, Apr 23, 2013 at 9:23 PM, Mattmann, Chris A (398J) <
>> chris.a.mattm...@jpl.nasa.gov> wrote:
>>
>> > Awesome work!
>> >
>> > ++
>> > Chris Mattmann, Ph.D.
>> > Senior Computer Scientist
>> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > Office: 171-266B, Mailstop: 171-246
>> > Email: chris.a.mattm...@nasa.gov
>> > WWW:  http://sunset.usc.edu/~mattmann/
>> > ++
>> > Adjunct Assistant Professor, Computer Science Department
>> > University of Southern California, Los Angeles, CA 90089 USA
>> > ++
>> >
>> >
>> >
>> >
>> >
>> >
>> > -Original Message-
>> > From: Rajith Siriwardana 
>> > Date: Tuesday, April 23, 2013 8:38 AM
>> > To: dev , jpluser 
>> > Subject: Re: Interested in participating GSoC 2013 with OODT
>> >
>> > >Hi Chris,
>> > >
>> > >I've submitted a first draft of my GSoC 2013 proposal. Here's a link
>>to
>> > >the proposal:[1]
>> > >If you'd like me to make any amendments or additions,
>> > >please let me know and I'll update it.
>> > >Many thanks for your valuable time spent on reviewing my proposal in
>>the
>> > >first place.[1]
>> > >
>> >
>> 
>>https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/r
>>a
>> > >jithsiriwardana/1Cheers,
>> > >Rajith
>> > >---
>> > >T.W.R.O. Siriwardana
>> > >StMIEEE
>> > >Undergraduate
>> > >Department of Computer Science & Engineering
>> > >Faculty of Engineering
>> > >University of Moratuwa
>> > >Sri Lanka.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >On Sat, Apr 20, 2013 at 11:59 AM, Mattmann, Chris A (398J)
>> > > wrote:
>> > >
>> > >Thanks Rajith!
>> > >
>> > >++
>> > >Chris Mattmann, Ph.D.
>> > >Senior Computer Scientist
>> > >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > >Office: 171-266B, Mailstop: 171-246
>> > >Email: chris.a.mattm...

Re: Apache Airavata-OODT Integration

[Mattmann, Chris A (398J)]

Hi Sanjaya,

-Original Message-

From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Thursday, May 30, 2013 5:12 AM
To: "dev@oodt.apache.org" , "d...@airavata.apache.org"

Subject: Apache Airavata-OODT Integration

>Hi,
> I have worked on the Apache Airavata integration with Apache OODT. As
>a first step, I have implemented integration with Apache OODT file
>manager component.

Great work!! 

Comments below:

>  1. Introduce a new GFac Schema type called OODTProduct which takes
>APache OODT product IDs as input.
>  2. Implemented new pre GFac Handler by extending Apache OODT
>PgeTaskInstance to stage the corresponding file into the working
>directory.
>  3. Once file is staged, input parameter with OODT product id is
>replaced with path of the staged file for downstream processing
>
>I have tested the implementation with Gfac application which wraps /bin/ls
>command. Application takes product id as input and stage corresponding
>file
>into the working directory and /bin/ls is executed against the staged
>file.
>Hope this is a valid testing scenario.
>
>Concerns
>- Configurations : I have added new configuration file named and
>oodt-integration.properties in addition to dynamic_metadata.met and
>pge-config.xml files used by OODT. But at the moment there is no item
>configured with the oodt-integration.properties.

You probably only need the pge-config.xml file. Dynamic metadata, and the
task configuration properties can be specified programmatically, right?

>- Staged File Name - With the current implementation of PgeTaskInstance it
>is not possible to retrieve path of the staged file. Due to this
>limitation, I have query the FileManagerServer with product id and
>retrieve
>the file name and computed the file path using information of working
>directory.

I'm not sure I understand this? If you store and record the Filename, and
FileLocation
metadata files, then you can easily retrieve the staged file path via a
SQLquery 
via CAS-PGE by simply setting the FORMAT=('$FileLocation/$Filename') in
the response.
Can you comment on this?

>- Currently it is not possible to execute the workflow using Xbaya due to
>validation failure due to new schema type. I have commented out the
>relevant validation code for testing purpose.

OK, will probably need to work on this.

>
>Currently I am having an issue with review board client tool and need to
>resolve it to upload the code for review.

I see later that you got this working, so will head over and review that
now.

Thanks!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

[ANNOUNCE] Open Source Summit 3.0: Communities Meeting: June 25,26, Washington DC USA

[Mattmann, Chris A (398J)]

(apologies for cross-post)

http://ossummit.org/

Registration for the Open Source Summit v3.0: Communities Meeting
is now open! This event grows out of the past two years of success.
The first Open Source Summit  was
held at NASA's Ames Research Center in Mountain View, CA and focused
specifically on NASA's open source policies. Last year, OSS
 [full schedule]

moved to the University of Maryland in College Park and broadened
the discussion to include all agencies, including NASA, the State
Dept, and the VA on the planning team.

This year's Open Source Summit will explain how to build, engage
with, and maintain open source communities -- and when we say open
source, we don't just mean software, we also mean hardware and data.

If you are a federal civil servant that needs to build or engage
with an open source community, you should plan on attending.

Be warned however: this is not your average event! The multi-agency
planning team is tasked with ensuring that the event provides
substantive benefit to federal agency personnel, and the format is
uniquely designed to deliver not just abstract content from subject
matter experts (of course we have those), but also the opportunity
to see this knowledge applied to a specific case study, and then
to learn how to apply it to your specific situation.  In addition,
we will collate the results of the discussions during the event and
making them available afterwards so that other may learn from the
shared experiences and wisdom of their peers.

Also, fyi - the registration numbers are unlimited for government
employees, but limited t o 70 for non-govies. We have a total
registration limit of 200. Please use the #ossdc hashtag
if you want to have conversations on Twitter about the meeting.

Thanks and I look forward to your registrations and interest! If
you have any questions,

Please don't hesitate to ask either by replying to this email,
contacting me or anyone else on the Planning Team directly, or by
sending us a Tweet!

Thanks!

Cheers, 
Chris Mattmann (on behalf of the Planning Team)


++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: JIRA label for new users

[Mattmann, Chris A (398J)]

Thanks Gav!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Tuesday, May 21, 2013 5:34 PM
To: "dev@oodt.apache.org" 
Subject: RE: JIRA label for new users

>Ok so I've implemented this using the labels suggested.
>
>Someone should change the default skill level to any issues they are
>familiar with to what is appropriate.
>
>Gav...
>
>
>> -Original Message-
>> From: mltjo...@gmail.com [mailto:mltjo...@gmail.com] On Behalf Of
>> Michael Joyce
>> Sent: Wednesday, 22 May 2013 1:14 AM
>> To: dev@oodt.apache.org
>> Subject: Re: JIRA label for new users
>> 
>> +1 on this Gavin.
>> 
>> Re. actual label names, how about a name followed by an (explanation) of
>> the difficulty level:
>> 
>> Don't Know
>> Newcomer (Easy) - Everyone can do this level Committer (Medium) - This
>>is
>> for regular contributors/committers. Where the vast majority of issues
>would
>> fall.
>> Expert (Hard) or Guru or ChrisMattmann or whatever everyone likes for
>> "hard" - Used sparingly when an issue requires expert component
>> knowledge
>> 
>> We could default the field to "Don't Know" so every issue always has
>>some
>> level?
>> 
>> I like the CouchDB approach but I feel they have too many options.
>> Easy
>> Easy - Medium
>> Medium - Hard
>> Really Really Hard (Their Guru level)
>> 
>> So, when something is "Medium," which category does it fall into? I
>>think
>the
>> most important part of this system is identifying the difficulty for
>> newcomers. The other levels are just nice to have in my opinion. That
>being
>> said, I think simpler is better here.
>> 
>> What do you guys think?
>> 
>> 
>> -- Joyce
>> 
>> 
>> On Sun, May 19, 2013 at 8:30 AM, Rajith Siriwardana <
>> rajithsiriward...@gmail.com> wrote:
>> 
>> > +1 for having a label for issues that new users can help to resolve.
>> >
>> >
>> > On Sun, May 19, 2013 at 8:51 PM, Mattmann, Chris A (398J) <
>> > chris.a.mattm...@jpl.nasa.gov> wrote:
>> >
>> > > Thanks Gav. +1 to using a controlled value field to make the labels
>> > > meaningful.
>> > >
>> > > Let's see what others think but I am for sure +1.
>> > >
>> > > Cheers,
>> > > Chris
>> > >
>> > >
>> ++
>> 
>> > > Chris Mattmann, Ph.D.
>> > > Senior Computer Scientist
>> > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > > Office: 171-266B, Mailstop: 171-246
>> > > Email: chris.a.mattm...@nasa.gov
>> > > WWW:  http://sunset.usc.edu/~mattmann/
>> > >
>> ++
>> ++++
>> > > Adjunct Assistant Professor, Computer Science Department University
>> > > of Southern California, Los Angeles, CA 90089 USA
>> > >
>> ++
>> 
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > -Original Message-
>> > > From: Gavin McDonald 
>> > > Organization: 16 degrees complete web solutions
>> > > Reply-To: "dev@oodt.apache.org" ,
>> > > "ga...@16degrees.com.au" 
>> > > Date: Sunday, May 19, 2013 3:37 AM
>> > > To: "dev@oodt.apache.org" 
>> > > Subject: RE: JIRA label for new users
>> > >
>> > > >
>> > > >
>> > > >> -Original Message-
>> > > >> From: Mattmann, Chris A (398J)
>> > > >> [mailto:chris.a.mattm...@jpl.nasa.gov]
>> > > >> Sent: Friday, 17 May 2013 1:37 AM
>> > &

Re: JIRA label for new users

[Mattmann, Chris A (398J)]

Thanks Gav. +1 to using a controlled value field to make the labels
meaningful.

Let's see what others think but I am for sure +1.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Sunday, May 19, 2013 3:37 AM
To: "dev@oodt.apache.org" 
Subject: RE: JIRA label for new users

>
>
>> -Original Message-
>> From: Mattmann, Chris A (398J) [mailto:chris.a.mattm...@jpl.nasa.gov]
>> Sent: Friday, 17 May 2013 1:37 AM
>> To: d...@climate.incubator.apache.org
>> Cc: dev@oodt.apache.org
>> Subject: Re: JIRA label for new users
>> 
>> Great idea, please implement on OODT too! (cc to dev@oodt for peeps) :)
>
>We can start to use labels now for jira tickets, we just need an agreed
>and
>documented set
>of labels. Why? Too many labels and we run the risk of there being too
>many
>and they sort of
>Lose the impact and meaning. So a few well-chosen labels for attention
>grabbing is needed.
>
>Every Jira ticket has a Labels section so this is available now.
>
>However, my main point of this mail, is that we can add custom fields to
>our
>issues. Some projects
>have done such a thing, some, like the couchdb project, have implemented a
>drop down field in
>their jiras so that anyone at a glance can determine if the ticket is
>newcomer, contributor, committer,
>hardcore project  guru or Greg Stein level. (OK so I made that last level
>up, but really, the choice of drop down
>field name is ours to decide)
>
>Pop on over to the couchdb jira and take a look.
>
>I am happy to implement this in the OODT Jira if you agree it a good
>idea. 
>
>I'm a big +1 ftr
>
>Gav...
>
>> 
>> Cheers,
>> Chris
>> 
>> ++
>> 
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> 
>> Adjunct Assistant Professor, Computer Science Department University of
>> Southern California, Los Angeles, CA 90089 USA
>> ++
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> -Original Message-
>> From: Michael Joyce 
>> Reply-To: "d...@climate.incubator.apache.org"
>> 
>> Date: Wednesday, May 15, 2013 8:34 PM
>> To: dev 
>> Subject: JIRA label for new users
>> 
>> >I sent this link around to a few people via twitter but I figured it
>> >warranted being sent here as well!
>> >
>> >http://nikcodes.com/2013/05/10/new-contributor-jump-in/
>> >
>> >TLDR: Every OSS project should have a label for issues that new users
>> >could help resolve. Basically a "this is simple enough that a new user
>> >could figure it out and contribute back to the project" tag.
>> >
>> >This shouldn't be a tag for features/issues that the core team doesn't
>> >want to deal with. Mostly it should be for fairly simple auxiliary
>> >issues that won't take forever to debug/implement but that the core
>> >team doesn't necessarily have time to deal with.
>> >
>> >I think this is a great idea. When I first started on OODT this would
>> >have been the coolest thing ever. I think it would be great to have a
>> >section devoted to "getting started" on the website that mentions this
>>as
>> well.
>> >Then it would be really easy for new developers to jump in and help. I
>> >would be more than happy to help get this up and running as well.
>> >
>> >Thoughts? Terrible/great idea? Ideas for a label name?
>> >
>> >-- Joyce
>
>

CFP: First Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE)

[Mattmann, Chris A (398J)]

(apologies for cross post)
  
First Workshop on Sustainable Software for Science:
Practice and Experiences (WSSSPE)
  
 http://wssspe.researchcomputing.org.uk/
   (in conjunction with SC13)
  Sunday, November 17, 2013, Denver, CO

Progress in scientific research is dependent on the quality and
accessibility of
software at all levels and it is now critical to address many new
challenges
related to the development, deployment, and maintenance of reusable
software. In
addition, it is essential that scientists, researchers, and students are
able to
learn and adopt a new set of software-related skills and methodologies.
Established researchers are already acquiring some of these skills, and in
particular a specialized class of software developers is emerging in
academic
environments who are an integral and embedded part of successful research
teams.
This workshop will provide a forum for discussion of the challenges,
including
both positions and experiences. The short papers and discussion will be
archived
as a basis for continued discussion, and we intend the workshop to feed
into the
collaborative writing of one or more journal publications.

In practice, scientific software activities are part of an ecosystem where
key
roles are held by developers, users, and funders.  All three groups supply
resources to the ecosystem, as well as requirements that bound it.  Roughly
following the example of NSF's Vision and Strategy for Software
(http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf12113), the
ecosystem
may be viewed as having challenges related to:

* the development process that leads to new software
- how fundamental research in computer science or science/engineering
  domains is turned  into reusable software
- software created as a by-product of research
- impact of computer science research on the development of scientific
  software
* the support and maintenance of existing software, including
- software engineering
- governance, business, and sustainability models
- the role of community software repositories, their operation and
sustainability
* the role of open source communities or industry
* use of the software
- growing communities
- reproducibility, transparency needs that may be unique to science
* policy issues, such as
- measuring usage and impact
- software credit, attribution, incentive, and reward
- career paths for developers and institutional roles
- issues related to multiple organizations and multiple countries,
  such as intellectual property, licensing, etc.
- mechanisms and venues for publishing software, and the role of
publishers
* education and training
This workshop is interested in all of the above topics.  We invite short
(4-page) position/experience reports that will be used to organize panel
and
discussion sessions.  These papers will be archived by a third-party
service,
and provided DOIs. We encourage submitters to license their papers under a
Creative Commons license that encourages sharing and remixing, as we will
combine ideas (with attribution) into the outcomes of the workshop.  An
interactive site will be created to link these papers and the workshop
discussion, with options for later comments and contributions.
Contributions
will be peer-reviewed for relevance and originality before the links are
added to the workshop site; contributions will also be used to determine
discussion topics and panelists. We will also plan one or more papers to
be collaboratively developed by the contributors, based on the panels and
discussions.

Deadline:

6 September 2013 (any time of day, no extensions)

Submissions:

Submissions of up to four pages should be formatted to be easily readable
and submitted to an open access repository that provides unique identifiers
that can be cited, for example http://arXiv.org 
,
http://figshare.com  , etc.
The submitter should then email the URL and identifier (or any questions)
to
wss...@gmail.com

Organizers:

* Daniel S. Katz, d.k...@ieee.org, National Science Foundation, USA
* Gabrielle Allen, al...@skoltech.ru, Skolkovo Institute of Science and
   Technology, Russian Federation
* Neil Chue Hong, n.chueh...@software.ac.uk, Software Sustainability
   Institute, University of Edinburgh, UK
* Manish Parashar, paras...@rutgers.edu, Rutgers University, USA
* David Proctor, djproc...@gmail.com, National Science Foundation, USA

Program Committee:
* David Abramson, University of Queensland, Australia
* Aron Ahmadia, King Abdullah University of Science & Technology, Saudi
Arabia
* Samuel Arbesman, Ewing Marion Kauffman Foundation, USA
* Lorena A. Barba, Boston University, USA
* Phil Bourne, University of California, San Diego, USA
* Karen Cranston, National Evolutionary Synthesis Center, USA
* Ewa Deelman, Universit

Re: [ANNOUNCE] Welcome Varun Ratnakar as an OODT PMC member!

[Mattmann, Chris A (398J)]

Woot, welcome Varun!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Varun Ratnakar 
Reply-To: "dev@oodt.apache.org" 
Date: Thursday, May 16, 2013 3:52 PM
To: "dev@oodt.apache.org" 
Cc: "ke...@apache.org" 
Subject: Re: [ANNOUNCE] Welcome Varun Ratnakar as an OODT PMC member!

>Thanks Sean !
>
>Quick info about me: Area of specialisation: Workflow systems. Working
>with USC/ISI, though based in London.
>
>I will initially be concentrating my efforts on better integration of
>OODT with the Wings workflow system (www.wings-workflows.org).
>Looking forward to contributing to this excellent community !
>
>Regards,
>Varun
>
>
>On 16 May 2013, at 21:30, ke...@apache.org wrote:
>
>> Folks:
>> 
>> I'm happy to report that Varun Ratnakar has accepted our invitation to
>>become not just a committer on Apache OODT but to also join the OODT
>>Project Management Committee. WOOT!
>> 
>> Welcome, Varun. Feel free to introduce yourself.
>> 
>> --k
>> 
>> 
>> 
>> 
>

Re: New list ?

[Mattmann, Chris A (398J)]

Sounds good Gav. I saw ambivalence, so happy to check it again in 6 months
and see what's up.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Thursday, May 16, 2013 4:51 PM
To: "dev@oodt.apache.org" 
Subject: RE: New list ?

>Ok, so there was not really enough positivity for this to happen at this
>time. (i.e most don't care too much just yet.)
>
> 
>
>I'm happy to revisit this in 6 months and see what the status is then.
>
> 
>
>Gav.
>
> 
>
> 
>
>From: Brian Foster [mailto:holeno...@me.com]
>Sent: Thursday, 2 May 2013 10:40 AM
>To: dev@oodt.apache.org
>Cc: dev@oodt.apache.org; ga...@16degrees.com.au
>Subject: Re: New list ?
>
> 
>
>I think this is solvable with a filter... however i could see a case where
>someone might only cared about following JIRA issues and didn't want all
>the
>svn commits flooding their inbox... so i guess that makes me a +1
>
> 
>
>-brian
>
>On May 01, 2013, at 07:25 AM, "Mattmann, Chris A (398J)"
>mailto:chris.a.mattm...@jpl.nasa.gov> >
>wrote:
>
>I'm +0 on this Gav mainly b/c JIRA dev happens in spurts, but with
>a filter can be dealt with.
>
>I like the clean separation of auto generated emails though, so I
>get that.
>
>Happy either way.
>
>Cheers,
>Chris
>
>++
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattm...@nasa.gov <mailto:chris.a.mattm...@nasa.gov>
>WWW: http://sunset.usc.edu/~mattmann/
>++
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA
>++
>
>
>
>
>
>
>-Original Message-
>From: Gavin McDonald <mailto:ga...@16degrees.com.au>
>>
>Organization: 16 degrees complete web solutions
>Reply-To: "dev@oodt.apache.org <mailto:dev@oodt.apache.org> "
>mailto:dev@oodt.apache.org> >,
>"ga...@16degrees.com.au <mailto:ga...@16degrees.com.au> "
>mailto:ga...@16degrees.com.au> >
>Date: Tuesday, April 30, 2013 3:04 AM
>To: "dev@oodt.apache.org <mailto:dev@oodt.apache.org> "
><mailto:dev@oodt.apache.org> >
>Subject: New list ?
>
>>A personal peeve of mine, all the jira notifications going to the commits
>>list, makes it hard to see whats going on.
>>
>>(Yes I could filter but..)
>>
>>How about an issues@ list where we can send the jira stuff ?
>>
>>Gav...
>>
>>
>

Re: JIRA label for new users

[Mattmann, Chris A (398J)]

Great idea, please implement on OODT too! (cc to dev@oodt for peeps) :)

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Michael Joyce 
Reply-To: "d...@climate.incubator.apache.org"

Date: Wednesday, May 15, 2013 8:34 PM
To: dev 
Subject: JIRA label for new users

>I sent this link around to a few people via twitter but I figured it
>warranted being sent here as well!
>
>http://nikcodes.com/2013/05/10/new-contributor-jump-in/
>
>TLDR: Every OSS project should have a label for issues that new users
>could
>help resolve. Basically a "this is simple enough that a new user could
>figure it out and contribute back to the project" tag.
>
>This shouldn't be a tag for features/issues that the core team doesn't
>want
>to deal with. Mostly it should be for fairly simple auxiliary issues that
>won't take forever to debug/implement but that the core team doesn't
>necessarily have time to deal with.
>
>I think this is a great idea. When I first started on OODT this would have
>been the coolest thing ever. I think it would be great to have a section
>devoted to "getting started" on the website that mentions this as well.
>Then it would be really easy for new developers to jump in and help. I
>would be more than happy to help get this up and running as well.
>
>Thoughts? Terrible/great idea? Ideas for a label name?
>
>-- Joyce

Re: [ANNOUNCE] Welcome Lewis John McGibbney as an OODT PMC member and committer!

[Mattmann, Chris A (398J)]

Lewis!! Welcome buddy!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: "ke...@apache.org" 
Reply-To: "u...@oodt.apache.org" 
Date: Tuesday, May 14, 2013 9:13 AM
To: "u...@oodt.apache.org" 
Cc: lewis john mcgibbney 
Subject: [ANNOUNCE] Welcome Lewis John McGibbney as an OODT PMC member and
committer!

>Folks:
>
>I'm happy to report that Lewis John McGibbney has accepted our invitation
>to become a committer on Apache OODT and to join the OODT Project
>Management Committee. WOOT!
>
>Welcome, Lewis. Though I suspect many of us know you by reputation
>already, please feel free to make an introduction for yourself.
>
>--k
>

Re: Maven dynamic libraries?

[Mattmann, Chris A (398J)]

Hey Viv,

This is possible but you need to bake somewhere into your code
where you load the .so shared lib from. See
http://oodt.jpl.nasa.gov/repo/projects/co2/
for Rishi Verma's example of doing this.

My honest suggestion: use Apache Tika, which in turn wraps the NetCDF4-Java
library which requires no *.SOs like the HDF5 Java lib does. If Tika
doesn't
extra all the info for you, improve Tika and gain merit in the community
there :)

Backup solution: use NetCDF4-Java directly. I published it for UCAR/NCAR
to Maven
Central since 4.2-min, so it's available for use there.

Let me know if you need more help.

Cheers,
Chris

P.S. CC'ed Apache OODT Dev list so others can contribute to the convo.

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: , "Vivian   (398K)" 
Date: Saturday, May 11, 2013 12:08 PM
To: jpluser 
Cc: "Tang, Vivian (398K)" 
Subject: Maven dynamic libraries?

>Hi Chris,
>
>I'm writing a SMAP metadata extractor that needs to extract ISO metadata
>from HDF5. Following your OODT approach of using Maven, the code
>therefore needs to load HDF5 library (hdf-java or jhdf5) and requires to
>load via JNI a shared library (*.so) during run-time.
>
>I have the code working fine when compiled and run manually, but when
>integrated into the Maven environment of SMAP FileManager package with
>Junit test, I get the following error when running the unittest (so it
>compiles fine):
>
>testH5(gov.nasa.smap.spdm.filemgr.metadata.extractors.ISOHDF5MetExtractorT
>est)  Time elapsed: 1.211 sec  <<< ERROR!
>java.lang.NoClassDefFoundError: Could not initialize class
>ncsa.hdf.hdf5lib.HDF5Constants
>at ncsa.hdf.object.h5.H5File.(H5File.java:169)
>at ncsa.hdf.object.h5.H5File.(H5File.java:122)
>at 
>gov.nasa.smap.spdm.filemgr.metadata.extractors.ISOHDF5MetExtractorTest.tes
>tH5(ISOHDF5MetExtractorTest.java:60)
>
>I know it loads the classes fine but keep getting this
>NoClassDefFoundError exception, which indicates something wrong with the
>class loading for the JVM. I've almost exhausted trying many things to no
>avail. Do you have any thoughts on how Maven handles shared library
>loading for JNI native libraries that may cause NoClassDefFoundError
>exceptions during runtime but not compiled time?
>
>Attached is my pom file for FM, and the command line I use to run is the
>following:
>mvn clean 
>-Dncsa.hdf.hdf5lib.H5.hdf5lib=/project/spdm/viviant/hdf-java/lib/linux/lib
>jhdf.so install
>
>
>In the Junit test, I'm just doing one-liner:
>
>ncsa.hdf.object.h5.H5File h5file = new
>ncsa.hdf.object.h5.H5File("/project/spdm/viviant/testdata/SMAP_L1A_RADAR_0
>4047_A_20160301T072816_R00301_002.h5");
>
>Any thoughts on why I keep getting NoClassDefFoundError only when using
>Maven?
>
>Thanks,
>Vivian

Re: Configuration management for Airavata integration with OODT file Manager server

[Mattmann, Chris A (398J)]

I think I'll know better once I see some examples, Sanjaya,
so looking forward to them, please proceed.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Sanjaya Medonsa 
Reply-To: "d...@airavata.apache.org" 
Date: Saturday, May 4, 2013 6:52 AM
To: "d...@airavata.apache.org" ,
"dev@oodt.apache.org" 
Subject: Configuration management for Airavata integration with OODT file
Manager server

>Hi,
>Airavata integration with FileManager to stage file prior to excute
>workflow task is based on the class PGETaskInstance of the OODT CAS-PGE
>module. PGETaskInstance takes both dynamic metadata and workflow
>configuration as an input and it combines them to create PgeMetaData
>object. Main purpose of the Airavata integration with OODT is to stage
>input files and ingest output data back to file manager server and
>Metadata
>catalog. Since workflow configuration is managed by Airavata itself, it is
>possible to configure only dynamic Metadata which should configure items
>such as QUERY_CLIENT_TRANSFER_SERVICE_FACTORY and
>QUERY_FILE_MANAGER_URL(Then static Metadata is represented by Metadata
>object which doesn't have any properties. Presence of both dynamic and
>static Metadata is mandatory in PgeMetadata implementation). Please let me
>know if you see any drawback with this approach of allowing to configure
>only dynamic Metadata for Airavata integration with FileManagerServer to
>stage files and making static Metadata empty.
>I would like to see some sample configuration files for dynamic and static
>Metadata.
>
>Best Regards,
>Sanjaya

Re: [jira] [Commented] (OODT-605) Upgrade the CAS-Product web application to use JAX-RS

[Mattmann, Chris A (398J)]

Thanks Ross. To make sure at least one ASF mentor was willing to mentor
your proposal,
I clicked "wish to mentor" on it.

Rishi please ASAP log onto melange and do the same so you can be a
potential mentor for
the project too.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: "Ross Laidlaw   (JIRA)" 
Date: Friday, May 3, 2013 11:20 AM
To: jpluser 
Subject: [jira] [Commented] (OODT-605) Upgrade the CAS-Product web
application to use JAX-RS

>
>[ 
>https://issues.apache.org/jira/browse/OODT-605?page=com.atlassian.jira.plu
>gin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648644#comme
>nt-13648644 ] 
>
>Ross Laidlaw commented on OODT-605:
>---
>
>Thanks Rishi & Chris!
>
>Today I submitted a [GSoC
>proposal|https://google-melange.appspot.com/gsoc/proposal/review/google/gs
>oc2013/rlaidlaw/15001].  It should be visible to everyone but please let
>me know if you can't access the link.
>
>Ross
>
>
>
>> Upgrade the CAS-Product web application to use JAX-RS
>> -
>>
>> Key: OODT-605
>> URL: https://issues.apache.org/jira/browse/OODT-605
>> Project: OODT
>>  Issue Type: New Feature
>>  Components: product server
>>Affects Versions: 0.6
>>Reporter: Ross Laidlaw
>>Assignee: Ross Laidlaw
>>  Labels: gsoc2013
>> Fix For: 0.6
>>
>>   Original Estimate: 3,360h
>>  Remaining Estimate: 3,360h
>>
>> This is the parent JIRA issue for adding JAX-RS support to the
>>CAS-Product web application (webapp/fmprod).
>> The primary goal is to use Apache CXF's JAX-RS implementation to
>>provide all of the functionality of the existing RSS, RDF and data
>>servlets in the CAS-Product web application, i.e. to provide various
>>ways of accessing File Manager products and product sets using JAX-RS.
>> After completing the primary goal, a secondary goal would be to extend
>>the service to provide additional ways of accessing File Manager
>>products (e.g. providing different output formats for metadata, and
>>using additional parameters to find and filter lists of products and
>>product groups).
>> I'd like to volunteer to work on this and I'm hoping to do it as a
>>Google Summer of Code project.
>> I'll use this JIRA issue to record the schedule and plans for the
>>project.  I'm hoping that if other members of the OODT community are
>>interested they can find information here and make comments,
>>suggestions, etc. and see how everything's progressing.
>
>--
>This message is automatically generated by JIRA.
>If you think it was sent incorrectly, please contact your JIRA
>administrators
>For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: New list ?

[Mattmann, Chris A (398J)]

I'm +0 on this Gav mainly b/c JIRA dev happens in spurts, but with
a filter can be dealt with.

I like the clean separation of auto generated emails though, so I
get that.

Happy either way.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Tuesday, April 30, 2013 3:04 AM
To: "dev@oodt.apache.org" 
Subject: New list ?

>A personal peeve of mine, all the jira notifications going to the commits
>list, makes it hard to see whats going on.
>
>(Yes I could filter but..)
>
>How about an issues@ list where we can send the jira stuff ?
>
>Gav...
>
>

Re: Problem using OpsUI FM Browser

[Mattmann, Chris A (398J)]

Hi Gabe,

Is your FM up and running?

Besides that, check if your FM catalog (lucene?)
has a segments file in it, or if it was manually created.
It should not exist before you start FM.

Also does the command line API for FM work?
CC'ing dev@oodt.a.o.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: , "Gabriel M   (398J)" 
Date: Friday, April 26, 2013 10:39 AM
To: jpluser , "Verma, Rishi (398J)"

Subject: Problem using OpsUI FM Browser

>Hi, guys.
>I recently deployed a new version of our SMAP PCS and now our File
>Catalog Browse function doesn't work.  You can see the error for yourself
>at: http://smap-dev:9080/pcs-opsui/types/
>
>
>Have you guys seen this before?
>
>
>Thanks!
>
>
>WicketMessage: Error attaching this container for rendering:
>[MarkupContainer [Component id = types_component]]
>Root cause:
>java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 1
> at java.util.Vector.get(Vector.java:694)
> at 
>org.apache.oodt.cas.webcomponents.filemgr.browser.types.Types$2.populateIt
>em(Types.java:171)
> at 
>org.apache.wicket.markup.html.list.ListView.onPopulate(ListView.java:562)
> at 
>org.apache.wicket.markup.repeater.AbstractRepeater.onBeforeRender(Abstract
>Repeater.java:131)
> at 
>org.apache.wicket.Component.internalBeforeRender(Component.java:1066)
> at org.apache.wicket.Component.beforeRender(Component.java:1100)
> at 
>org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.j
>ava:1757)
> at org.apache.wicket.Component.onBeforeRender(Component.java:3966)
> at 
>org.apache.wicket.Component.internalBeforeRender(Component.java:1066)
> at org.apache.wicket.Component.beforeRender(Component.java:1100)
> at 
>org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.j
>ava:1757)
> at org.apache.wicket.Component.onBeforeRender(Component.java:3966)
> at org.apache.wicket.Page.onBeforeRender(Page.java:1550)
> at 
>org.apache.wicket.Component.internalBeforeRender(Component.java:1066)
> at org.apache.wicket.Component.beforeRender(Component.java:1100)
> at org.apache.wicket.Component.prepareForRender(Component.java:2292)
> at org.apache.wicket.Page.prepareForRender(Page.java:1540)
> at org.apache.wicket.Component.prepareForRender(Component.java:2329)
> at org.apache.wicket.Page.renderPage(Page.java:911)
> at 
>org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.r
>espond(BookmarkablePageRequestTarget.java:261)
> at 
>org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRe
>questCycleProcessor.java:105)
> at 
>org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1
>258)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
> at 
>org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:486)
> at 
>org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:31
>9)
> at 
>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicati
>onFilterChain.java:243)
> at 
>org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilter
>Chain.java:210)
> at 
>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.
>java:222)
> at 
>org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.
>java:123)
> at 
>org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBa
>se.java:472)
> at 
>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:1
>71)
> at 
>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:9
>9)
> at 
>org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
> at 
>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.ja
>va:118)
> at 
>org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408
>)
> at 
>org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Pro
>cessor.java:1009)
> at 
>org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstr
>actProtocol.java:589)
> at 
>org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.jav
>a:312)
> at 
>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.
>java:886)
> a

FW: GSoC Student Application Window Now Open

[Mattmann, Chris A (398J)]

Great post from Lewis, FYI

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Lewis John Mcgibbney 
Reply-To: "d...@nutch.apache.org" 
Date: Wednesday, April 24, 2013 9:46 PM
To: "d...@giraph.apache.org" ,
"d...@nutch.apache.org" , ""

Subject: GSoC Student Application Window Now Open

>
>
>
>Hi All,
>Apology for cross post...
>
>I thought I would relay this message to students who are now monitoring
>the above lists.
>
>Google are now accepting applications from students to participate in
>Google Summer of Code 2013. The deadline to apply is May 3rd at 19:00
>UTC. Late proposals will not be accepted for any reason.
>
>I would encourage applicants to get on to the project wiki's and begin
>getting ideas down which will act as a base for individual projects.
>Additionally, the community lists are the place to voice any queries and
>to obtain advice about project direction, scope,
> deliverables, etc.
>
>Good luck, I am really looking forward to working with some of you this
>year as there are some great proposals so far.
>
>Lewis 
>
>-- 
>Lewis
>

Re: Interested in participating GSoC 2013 with OODT

[Mattmann, Chris A (398J)]

Awesome work!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Rajith Siriwardana 
Date: Tuesday, April 23, 2013 8:38 AM
To: dev , jpluser 
Subject: Re: Interested in participating GSoC 2013 with OODT

>Hi Chris,
>
>I've submitted a first draft of my GSoC 2013 proposal. Here's a link to
>the proposal:[1]
>If you'd like me to make any amendments or additions,
>please let me know and I'll update it.
>Many thanks for your valuable time spent on reviewing my proposal in the
>first place.[1] 
>https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/ra
>jithsiriwardana/1Cheers,
>Rajith
>---
>T.W.R.O. Siriwardana
>StMIEEE
>Undergraduate 
>Department of Computer Science & Engineering
>Faculty of Engineering
>University of Moratuwa
>Sri Lanka.
>
>
>
>
>
>
>On Sat, Apr 20, 2013 at 11:59 AM, Mattmann, Chris A (398J)
> wrote:
>
>Thanks Rajith!
>
>++
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattm...@nasa.gov
>WWW:  http://sunset.usc.edu/~mattmann/
>++
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA
>++
>
>
>
>
>
>
>
>-Original Message-
>From: Rajith Siriwardana 
>Reply-To: "dev@oodt.apache.org" 
>
>Date: Friday, April 12, 2013 10:38 AM
>To: dev 
>Subject: Re: Interested in participating GSoC 2013 with OODT
>
>>On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) <
>>chris.a.mattm...@jpl.nasa.gov> wrote:
>>
>>> Hi Rajith,
>>>
>>> Thanks, replies below!
>>>
>>>
>>>
>>> -Original Message-
>>> From: Rajith Siriwardana 
>>> Reply-To: "dev@oodt.apache.org" 
>>> Date: Friday, April 12, 2013 10:17 AM
>>> To: dev 
>>> Subject: Re: Interested in participating GSoC 2013 with OODT
>>>
>>> >Hi Chris,
>>> >please find the comments inline.
>>> >[..snip..]
>>> >>
>>> >
>>> >Do you know of one? I looked way back in 2005 but there wasn't one
>>>back
>>> >> then.
>>> >
>>> >
>>> >Still I haven't found a parser compatible with the Apache license
>>>v2.0,
>>> >but
>>> >I found some parsers to get the idea of how it should be. I guess it
>>>won't
>>> >be hard.
>>>
>>> +1, great.
>>>
>>> >
>>> >>
>>> >
>>> >
>>> >> >   - In job status monitoring it will involve jvm monitoring (that
>>>is
>>> >>not
>>> >> >   related to the plugin as it just polling gmetad) correct me if
>>>i¹m
>>> >> >   misunderstood the job concept.
>>> >>
>>> >> Job status is resource manager job status polling. Have you seen the
>>> >> Job class in resource manager?
>>> >
>>> >
>>> >yep I did that, gmetad has nothing to do with polling job status
>>> >information. Should GangliaAssignmentMonitor worry about obtaining job
>>> >status information then? am I missing something here?
>>>
>>> I'm not sure what you mean about it not having anything to do with job
>>> polling?
>>> gmetad collects metrics published by gmond. Some of those metrics are
>>> related to
>>> CPU load, disk space, etc. I guess that's what I meant, more than job
>>> status.
>>>
>>
>>No Chris I guess the confusion caused by this
>>
>>  In the issue OODT-219 it says "Write a GangliaAssignmentMonitor and
>
>
>>GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
>>status information*". hence I thought it should also poll Job status
>>information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
>>resource node monitoring.
>>
>>Thanks for the clarification Chris.
>>
>>Thanks for clarifying.
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattm...@nasa.gov
>>> WWW:  http://sunset.usc.edu/~mattmann/
>>> ++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++
>>>
>>>
>>>
>>>
>
>
>
>
>
>
>
>
>

Re: Interested in participating GSoC 2013 with OODT

[Mattmann, Chris A (398J)]

Thanks Rajith!

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Rajith Siriwardana 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, April 12, 2013 10:38 AM
To: dev 
Subject: Re: Interested in participating GSoC 2013 with OODT

>On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) <
>chris.a.mattm...@jpl.nasa.gov> wrote:
>
>> Hi Rajith,
>>
>> Thanks, replies below!
>>
>>
>>
>> -Original Message-
>> From: Rajith Siriwardana 
>> Reply-To: "dev@oodt.apache.org" 
>> Date: Friday, April 12, 2013 10:17 AM
>> To: dev 
>> Subject: Re: Interested in participating GSoC 2013 with OODT
>>
>> >Hi Chris,
>> >please find the comments inline.
>> >[..snip..]
>> >>
>> >
>> >Do you know of one? I looked way back in 2005 but there wasn't one back
>> >> then.
>> >
>> >
>> >Still I haven't found a parser compatible with the Apache license v2.0,
>> >but
>> >I found some parsers to get the idea of how it should be. I guess it
>>won't
>> >be hard.
>>
>> +1, great.
>>
>> >
>> >>
>> >
>> >
>> >> >   - In job status monitoring it will involve jvm monitoring (that
>>is
>> >>not
>> >> >   related to the plugin as it just polling gmetad) correct me if
>>i¹m
>> >> >   misunderstood the job concept.
>> >>
>> >> Job status is resource manager job status polling. Have you seen the
>> >> Job class in resource manager?
>> >
>> >
>> >yep I did that, gmetad has nothing to do with polling job status
>> >information. Should GangliaAssignmentMonitor worry about obtaining job
>> >status information then? am I missing something here?
>>
>> I'm not sure what you mean about it not having anything to do with job
>> polling?
>> gmetad collects metrics published by gmond. Some of those metrics are
>> related to
>> CPU load, disk space, etc. I guess that's what I meant, more than job
>> status.
>>
>
>No Chris I guess the confusion caused by this
>
>  In the issue OODT-219 it says "Write a GangliaAssignmentMonitor and
>GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
>status information*". hence I thought it should also poll Job status
>information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
>resource node monitoring.
>
>Thanks for the clarification Chris.
>
>Thanks for clarifying.
>>
>> Cheers,
>> Chris
>>
>> ++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattm...@nasa.gov
>> WWW:  http://sunset.usc.edu/~mattmann/
>> ++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++
>>
>>
>>
>>

Re: [jira] [Created] (OODT-602) Load not settable for WorkflowTaskJobs in IterativeWorkflowProcessThread

[Mattmann, Chris A (398J)]

Agreed, would be an improvement to the Resource Manager to compute this..

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Bruce Barkstrom 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, April 19, 2013 6:00 PM
To: "dev@oodt.apache.org" 
Subject: Re: [jira] [Created] (OODT-602) Load not settable for
WorkflowTaskJobs in IterativeWorkflowProcessThread

>Calculating contingency would seem to be a sensible idea - although
>probably
>not often done since I'm pretty sure it would require Monte Carlo
>calculations
>of uncertainties in schedule and budget, particularly if the schedule is
>"structurally
>stochastic" [see D. Rumsfield on "Unknown Unknowns"].
>
>Bruce B.
>
>On Fri, Apr 19, 2013 at 8:55 PM, Chris A. Mattmann (JIRA)
>wrote:
>
>> Chris A. Mattmann created OODT-602:
>> --
>>
>>  Summary: Load not settable for WorkflowTaskJobs in
>> IterativeWorkflowProcessThread
>>  Key: OODT-602
>>  URL: https://issues.apache.org/jira/browse/OODT-602
>>  Project: OODT
>>   Issue Type: Bug
>>   Components: workflow manager
>> Reporter: Chris A. Mattmann
>> Assignee: Chris A. Mattmann
>> Priority: Critical
>>  Fix For: 0.6
>>
>>
>> Currently in IterativeWorkflowProcessorThread, this code block
>> automatically sets Load Value to 2 and doesn't allow override:
>>
>> {code:java}
>>if (rClient != null) {
>> // build up the Job
>> // and the Job Input
>> Job taskJob = new Job();
>> taskJob.setName(task.getTaskId());
>> taskJob
>>
>> 
>>.setJobInstanceClassName("org.apache.oodt.cas.workflow.structs.TaskJob");
>> taskJob
>>
>> 
>>.setJobInputClassName("org.apache.oodt.cas.workflow.structs.TaskJobInput"
>>);
>> taskJob.setLoadValue(new Integer(2));
>> taskJob
>> .setQueueName(task.getTaskConfig().getProperty(QUEUE_NAME)
>>!=
>> null ? task
>> .getTaskConfig().getProperty(QUEUE_NAME) :
>> DEFAULT_QUEUE_NAME);
>> {code}
>>
>> I could have sworn this was configurable before, but I guess it wasn't.
>> I'm going to add a property called "Load" that can be set in tasks.xml
>>on a
>> per WorkflowTask basis, that will propagate and allow override of the
>> resource manager job load when workflow manager is connecting to the
>> resource manager.
>>
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA
>> administrators
>> For more information on JIRA, see:
>>http://www.atlassian.com/software/jira
>>

Re: Review Request: OODT-597: Improve theme/styling of OPSUI

[Mattmann, Chris A (398J)]

Awesome Mike, if you get a chance would love for you to contribute the
Look/Feel back..

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: , "Michael C  [NOAA-JPSS]  (GSFC-4700)"

Reply-To: "dev@oodt.apache.org" 
Date: Thursday, April 18, 2013 1:38 PM
To: "dev@oodt.apache.org" , "Resneck, Gabriel M
(398J)" , "Ramirez, Paul M (398J)"
, Andrew Hart , Cameron
Goodale , Chris Mattmann 
Cc: Rishi Verma 
Subject: Re: Review Request: OODT-597: Improve theme/styling of OPSUI

>I am currently working on a look and feel for my team. I gave a demo a few
>weeks ago on the original and I got feedback on changing the color, fonts
>and a few other things. I fixed a couple things, it looks a little
>different now. However, this is something I am doing for my group. In
>addition, it looks like we need more functionality in this component as
>well, for example the components under the configuration tab and etcŠ
>
>
>
>
>
>On 4/18/13 2:59 PM, "Resneck, Gabriel M" 
>wrote:
>
>>
>>---
>>This is an automatically generated e-mail. To reply, visit:
>>https://reviews.apache.org/r/10535/#review19402
>>---
>>
>>Ship it!
>>
>>
>>Ship It!
>>
>>- Gabe  Resneck
>>
>>
>>On April 17, 2013, 3:40 a.m., Rishi Verma wrote:
>>> 
>>> ---
>>> This is an automatically generated e-mail. To reply, visit:
>>> https://reviews.apache.org/r/10535/
>>> ---
>>> 
>>> (Updated April 17, 2013, 3:40 a.m.)
>>> 
>>> 
>>> Review request for oodt, Chris Mattmann, Andrew Hart, Paul Ramirez, and
>>>Cameron Goodale.
>>> 
>>> 
>>> Description
>>> ---
>>> 
>>> Hey all,
>>> 
>>> Per JIRA Issue: https://issues.apache.org/jira/browse/OODT-597, I
>>>proposed we update some of OPSUI's look and feel to make it more
>>>readable.
>>> 
>>> The attached diff resolves all points addressed in OODT-597, so please
>>>take a look at that for more details. I've attached some screenshots of
>>>what the new OPSUI looks like - for your convenience.
>>> 
>>> I know A LOT of work must have gone into the original OPSUI look and
>>>feel. So I want to make sure my improvements to it (and in some cases
>>>changes) are transparent and reviewed before we push. Like I said in
>>>OODT-597, the old OPSUI was very slick. I feel it was a little bit to
>>>read though on low-contrast screens, and that's why I proposed an
>>>improvement to the theme. Also, I think its good if we keep some
>>>consistency between how we do OODT logos, coloring, and themeing to make
>>>sure we are generating a consistent product.
>>> 
>>> Anyways, let me know what you think! I'll push the commits when I get a
>>>'ship it'! :D
>>> 
>>> rishi
>>> 
>>> 
>>> This addresses bug OODT-597.
>>> https://issues.apache.org/jira/browse/OODT-597
>>> 
>>> 
>>> Diffs
>>> -
>>> 
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/BasePage.css 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/BasePage.html 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/BasePage.java 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/DefaultSkin.css 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/HomePage.css 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/HomePage.java 1468173
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/OPSUIWebPage.java PRE-CREATION
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/project_logo.png UNKNOWN
>>>   
>>>http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/java/org/a
>>>p
>>>ache/oodt/pcs/opsui/splash.png UNKNOWN
>>> 
>>> Diff: https://reviews.apache.org/r/10535/diff/
>>> 
>>> 
>>> Testing
>>> ---
>>> 
>>> Local testing. See screenshots.
>>> 
>>> 
>>> Screenshots
>>> ---
>>> 
>>> Splash page
>>>   https://reviews.apache.org/r/10535/s/20/
>>> Workflow monitor page
>>>   https://reviews.apache.org/r/10535/s/21/
>>> Workflow 

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

Hey Amila,

Check out:

http://sunset.usc.edu/~mattmann/pubs/SEN10.pdf

C. Mattmann, A. Braverman, D. Crichton. Understanding Architectural
Tradeoffs Necessary
to Increase Climate Model Intercomparison Efficiency. ACM SIGSOFT Software
Engineering
Notes, vol. 35, no. 3, pp. 1-6, May 2010.

Also you may find these useful:

http://xldb.org/use



I've been a long time participant of XLDB so happy to connect
you to that community as well too if you are interested.

Cheers,
Chris






++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: AMILA RANATUNGA 
Date: Thursday, April 18, 2013 11:47 AM
To: jpluser 
Cc: "d...@airavata.apache.org" , Martin
Desruisseaux , "d...@sis.apache.org"
, Harsha Kumara , Shahani Markus
Weerawarana , "dev@oodt.apache.org"

Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hi,
>
>
>Thank you very much for the replies so far here and in SIS dev list as
>well. They were really helpful. We are currently writing a research paper
>regarding what kind of model that should be used by when building a
>geoscience gateway. We intend to discuss
> issues that a geoscientist face during his/her research and features
>that should be inserted to such a gateway to overcome them. Then we can
>consider them during our main project as well. It is appreciated if you
>can point any research papers or resources
> where we can see this domain from an eye of a geoscientist will be
>really useful. Even a good case study will be really helpful.
>
>
>
>
>Thank You !
>
>
>
>
>
> 
>
>
>
>On Thu, Apr 18, 2013 at 11:15 PM, Mattmann, Chris A (398J)
> wrote:
>
>Sounding good, very interesting Amila..
>
>Cheers,
>Chris
>
>++
>Chris Mattmann, Ph.D.
>Senior Computer Scientist
>NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>Office: 171-266B, Mailstop: 171-246
>Email: chris.a.mattm...@nasa.gov
>WWW:  http://sunset.usc.edu/~mattmann/
>++
>Adjunct Assistant Professor, Computer Science Department
>University of Southern California, Los Angeles, CA 90089 USA
>++
>
>
>
>
>
>
>-Original Message-
>
>From: Nipuni Perera 
>Reply-To: "d...@airavata.apache.org" 
>Date: Wednesday, April 10, 2013 12:52 AM
>To: Martin Desruisseaux 
>
>Cc: AMILA RANATUNGA , "d...@sis.apache.org"
>, "d...@airavata.apache.org" ,
>Harsha Kumara , Shahani Markus Weerawarana
>
>, "dev@oodt.apache.org" 
>Subject: Re: Research project on integrating geoservices with Apache
>Airavata
>
>
>>Hi,
>>
>>Thanks for the guidance so far and we will study on the above mentioned
>>resources.
>>
>>According to our understanding so far we feel that SIS can be used to
>>implement a server which exposes OGC web services such as WPS.
>>We feel that in order to enable geoscientists to create workflows
>>accessing
>>OGC compliant web services like WPS services, the main focus will be to
>>integrated client side of OGC compliant web services with Airavata.
>>
>>If  SIS can be used as a library for underlying geospatial processes what
>>makes  it different from currently exposed services. Because we have seen
>>that Taverna also has the ability to create geoscience workflows using
>>some
>>web processing services.
>>
>>Any comments on our thoughts are highly appreciated.
>>
>>Thanks,
>>Nipuni
>>
>>
>>On Sun, Apr 7, 2013 at 9:27 PM, Martin Desruisseaux <
>>martin.desruisse...@geomatys.fr> wrote:
>>
>>>  Hello Amila
>>>
>>> Le 06/04/13 06:25, AMILA RANATUNGA a écrit :
>>>
>>> Constellation-SDI consisted of WPS, WMS, WFS as server modules. So will
>>> that integration make Apache SIS be considered as those services
>>>enabled?
>>>
>>> Yes, after the port o

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

Sounding good, very interesting Amila..

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Nipuni Perera 
Reply-To: "d...@airavata.apache.org" 
Date: Wednesday, April 10, 2013 12:52 AM
To: Martin Desruisseaux 
Cc: AMILA RANATUNGA , "d...@sis.apache.org"
, "d...@airavata.apache.org" ,
Harsha Kumara , Shahani Markus Weerawarana
, "dev@oodt.apache.org" 
Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hi,
>
>Thanks for the guidance so far and we will study on the above mentioned
>resources.
>
>According to our understanding so far we feel that SIS can be used to
>implement a server which exposes OGC web services such as WPS.
>We feel that in order to enable geoscientists to create workflows
>accessing
>OGC compliant web services like WPS services, the main focus will be to
>integrated client side of OGC compliant web services with Airavata.
>
>If  SIS can be used as a library for underlying geospatial processes what
>makes  it different from currently exposed services. Because we have seen
>that Taverna also has the ability to create geoscience workflows using
>some
>web processing services.
>
>Any comments on our thoughts are highly appreciated.
>
>Thanks,
>Nipuni
>
>
>On Sun, Apr 7, 2013 at 9:27 PM, Martin Desruisseaux <
>martin.desruisse...@geomatys.fr> wrote:
>
>>  Hello Amila
>>
>> Le 06/04/13 06:25, AMILA RANATUNGA a écrit :
>>
>> Constellation-SDI consisted of WPS, WMS, WFS as server modules. So will
>> that integration make Apache SIS be considered as those services
>>enabled?
>>
>> Yes, after the port of Constellation-SDI as Apache SIS modules
>>(including
>> the test suites that execute CITE tests), SIS should be WPS, WMS and WFS
>> (among other services) compliant. Frédéric or Vincent could hopefully
>> complete this list and tell more about compliance.
>>
>>
>>  And why you said " Having SIS to implement WMS, WMTS, WCS and WFS is a
>> must"? What about WPS. Will that make SIS out of the scope. Because we
>> feel that since Airavata using Science gateway concept, really essential
>> to implement WPS too.
>>
>> I cited WMS, WMTS, WCS and WFS as the minimal services that SIS must
>> support because I perceive those services as the ones that every GIS
>>users
>> take for granted. But they are only the minimum. WPS and SensorML are
>> supplemental services also implemented in Constellation-SDI and to be
>> proposed to Apache SIS. So WPS is not out of SIS scope, quite the
>>contrary
>> :-).
>>
>> Martin
>>
>>
>
>
>-- 
>Nipuni Piyabasi Perera
>Undergraduate
>Department of Computer Science And Engineering
>University of Moratuwa
>Sri Lanka

Re: [ANNOUNCE] Welcome Gavin McDonald as an OODT PMC member and committer!

[Mattmann, Chris A (398J)]

-Original Message-

From: Gavin McDonald 
Organization: 16 degrees complete web solutions
Reply-To: "dev@oodt.apache.org" ,
"ga...@16degrees.com.au" 
Date: Monday, April 15, 2013 5:55 PM
To: "dev@oodt.apache.org" 
Subject: RE: [ANNOUNCE] Welcome Gavin McDonald as an OODT PMC member and
committer!

>
>
>> -Original Message-
>> From: Sean Kelly [mailto:ke...@apache.org]
>> Sent: Wednesday, 10 April 2013 7:56 AM
>> To: dev@oodt.apache.org
>> Cc: gmcdon...@apache.org
>> Subject: [ANNOUNCE] Welcome Gavin McDonald as an OODT PMC member
>> and committer!
>> 
>> Folks:
>> 
>> I'm happy to report that Gavin McDonald has accepted our invitation to
>> become a committer on Apache OODT and to join the OODT Project
>> Management Committee. WOOT!
>> 
>> Welcome, Gav. Please feel free to introduce yourself.
>
>Thanks Sean,
>
>Hi All,
>
>I hope to help out more in the near future, starting with making the PEAR
>stuff more reliable/compatible/working.
>Then onto other stuff.

Dude you rock! Welcome Gav!

>
>I also do some ASF sysadmin type jobs around the place. :)
>I look forward to working with you all .

Me too dude!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: Interested in participating GSoC 2013 with OODT

[Mattmann, Chris A (398J)]

Hey Rajith,

Thanks. Can you start working on a proposal maybe a Google doc,
and then share with me?

I'll look at it and try to help out and we can go from there.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Rajith Siriwardana 
Reply-To: "dev@oodt.apache.org" 
Date: Tuesday, April 16, 2013 11:16 AM
To: dev 
Subject: Re: Interested in participating GSoC 2013 with OODT

>Hi guys,
>
>Ganglia plugin initial architecture diagram is attached with OODT-219. You
>can find it in [1]. Your advises are warmly welcome to modify it. I guess
>Chris [2] will be mentoring this project. And I wanna know what are
>specifications you guys are expecting in the project proposal except the
>things specified in [3].
>
>[1]
>https://issues.apache.org/jira/secure/attachment/12578972/architecture_dia
>gram.jpg
>[2]
>https://issues.apache.org/jira/secure/ViewProfile.jspa?name=chrismattmann
>[3] http://community.apache.org/gsoc.html
>
>
>
>Thank you,
>
>T.W.R.O. Siriwardana
>StMIEEE
>Undergraduate
>Department of Computer Science & Engineering
>Faculty of Engineering
>University of Moratuwa
>Sri Lanka.
>
>
>On Fri, Apr 12, 2013 at 11:08 PM, Rajith Siriwardana <
>rajithsiriward...@gmail.com> wrote:
>
>>
>>
>>
>> On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) <
>> chris.a.mattm...@jpl.nasa.gov> wrote:
>>
>>> Hi Rajith,
>>>
>>> Thanks, replies below!
>>>
>>>
>>>
>>> -Original Message-
>>> From: Rajith Siriwardana 
>>> Reply-To: "dev@oodt.apache.org" 
>>> Date: Friday, April 12, 2013 10:17 AM
>>> To: dev 
>>> Subject: Re: Interested in participating GSoC 2013 with OODT
>>>
>>> >Hi Chris,
>>> >please find the comments inline.
>>> >[..snip..]
>>> >>
>>> >
>>> >Do you know of one? I looked way back in 2005 but there wasn't one
>>>back
>>> >> then.
>>> >
>>> >
>>> >Still I haven't found a parser compatible with the Apache license
>>>v2.0,
>>> >but
>>> >I found some parsers to get the idea of how it should be. I guess it
>>> won't
>>> >be hard.
>>>
>>> +1, great.
>>>
>>> >
>>> >>
>>> >
>>> >
>>> >> >   - In job status monitoring it will involve jvm monitoring (that
>>>is
>>> >>not
>>> >> >   related to the plugin as it just polling gmetad) correct me if
>>>i¹m
>>> >> >   misunderstood the job concept.
>>> >>
>>> >> Job status is resource manager job status polling. Have you seen the
>>> >> Job class in resource manager?
>>> >
>>> >
>>> >yep I did that, gmetad has nothing to do with polling job status
>>> >information. Should GangliaAssignmentMonitor worry about obtaining job
>>> >status information then? am I missing something here?
>>>
>>> I'm not sure what you mean about it not having anything to do with job
>>> polling?
>>> gmetad collects metrics published by gmond. Some of those metrics are
>>> related to
>>> CPU load, disk space, etc. I guess that's what I meant, more than job
>>> status.
>>>
>>
>> No Chris I guess the confusion caused by this
>>
>>   In the issue OODT-219 it says "Write a GangliaAssignmentMonitor
>>and
>> GangliaAssignmentMonitorFactory that plug in to Ganglia to   obtain *Job
>> status information*". hence I thought it should also poll Job status
>> information (JobStatus.SUCCESS/JobStatus.FAILURE/..etc). but it is only
>> resource node monitoring.
>>
>> Thanks for the clarification Chris.
>>
>> Thanks for clarifying.
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattm...@nasa.gov
>>> WWW:  http://sunset.usc.edu/~mattmann/
>>> ++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++
>>>
>>>
>>>
>>>
>>

Re: Interested in participating GSoC 2013 with OODT

[Mattmann, Chris A (398J)]

Hi Rajith,

Thanks, replies below!



-Original Message-
From: Rajith Siriwardana 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, April 12, 2013 10:17 AM
To: dev 
Subject: Re: Interested in participating GSoC 2013 with OODT

>Hi Chris,
>please find the comments inline.
>[..snip..]
>>
>
>Do you know of one? I looked way back in 2005 but there wasn't one back
>> then.
>
>
>Still I haven't found a parser compatible with the Apache license v2.0,
>but
>I found some parsers to get the idea of how it should be. I guess it won't
>be hard.

+1, great.

>
>>
>
>
>> >   - In job status monitoring it will involve jvm monitoring (that is
>>not
>> >   related to the plugin as it just polling gmetad) correct me if i¹m
>> >   misunderstood the job concept.
>>
>> Job status is resource manager job status polling. Have you seen the
>> Job class in resource manager?
>
>
>yep I did that, gmetad has nothing to do with polling job status
>information. Should GangliaAssignmentMonitor worry about obtaining job
>status information then? am I missing something here?

I'm not sure what you mean about it not having anything to do with job
polling?
gmetad collects metrics published by gmond. Some of those metrics are
related to
CPU load, disk space, etc. I guess that's what I meant, more than job
status.
Thanks for clarifying.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++

Re: Interested in participating GSoC 2013 with OODT

[Mattmann, Chris A (398J)]

Hi Rajith,


-Original Message-
From: Rajith Siriwardana 
Reply-To: "dev@oodt.apache.org" 
Date: Sunday, April 7, 2013 9:24 AM
To: dev 
Subject: Re: Interested in participating GSoC 2013 with OODT

>hi all,
>
>*
>
>As I understood there will be resource node monitoring and Job status
>monitoring happen using ganglia. Resource management components plugin
>will
>poll gmetad periodically by xml over tcp and receive an xml file of the
>last status monitored.

+1.

>
>I need some clarification on following matters
>
>
>   -
>
>   By polling gmetad you will get aggregated monitoring data of a cluster.
>   by listening gmond multicast/unicast channel you can get xml from a
>gmond
>   running in a node. I guess it should be polling gmetad.

Yep -- we can create a Java socket connection to port 8649 I believe to
get the XML document. So we'll need to write a Java parser for its schema
if there isn't one already that we can use.

Do you know of one? I looked way back in 2005 but there wasn't one back
then.

>   - In job status monitoring it will involve jvm monitoring (that is not
>   related to the plugin as it just polling gmetad) correct me if i¹m
>   misunderstood the job concept.

Job status is resource manager job status polling. Have you seen the
Job class in resource manager?

>   -
>
>   Is it only be able to poll default gmond metrics. or should it be
>   pluggable with gmetric functionality which is able to introduce
>additional
>   metrics.

Yep it should plug into the Resource Manager AssignmentMonitor interface.

Does that make sense?

Cheers,
Chris

>
>Anything you need to clarify or I have misunderstood?*
>
>Thank you,
>
>T.W.R.O. Siriwardana
>StMIEEE
>Undergraduate
>Department of Computer Science & Engineering
>Faculty of Engineering
>University of Moratuwa
>Sri Lanka.
>
>
>On Sun, Mar 31, 2013 at 7:05 AM, Rajith Siriwardana <
>rajithsiriward...@ieee.org> wrote:
>
>> Awesome! thanks Chris.
>>
>> Thank you,
>>
>> T.W.R.O. Siriwardana
>> StMIEEE
>> Undergraduate
>> Department of Computer Science & Engineering
>> Faculty of Engineering
>>  University of Moratuwa
>> Sri Lanka.
>>
>> On Sun, Mar 31, 2013 at 2:27 AM, Mattmann, Chris A (388J) <
>> chris.a.mattm...@jpl.nasa.gov> wrote:
>>
>>> Hi Rajith,
>>>
>>> We're looking forward to you participating.
>>>
>>> I just outlined the process to someone else who is trying to
>>>participate
>>> in Apache Mesos through GSOC 2013, so I'll repaste here:
>>>
>>>
>>>   1.  ASF is accepted/declined as GSoC 2013 org (let's hope it's
>>>accepted
>>> :) )
>>>   2.  Rajith talks with OODT mentor and OODT mentor agrees to mentor
>>>   3. Rajith comes up with proposal, referencing the GSoC project and
>>>JIRA
>>> issue, inputs into
>>> Google Melange and inputs into Google Melange for GSOC 2013
>>> http://www.google-melange.com/gsoc/homepage/google/gsoc2013
>>>   4.  OODT mentor signs up for Melange
>>> http://www.google-melange.com/gsoc/homepage/google/gsoc2013
>>> and lists ASF as organization
>>>   5. Rajith requests OODT mentor as mentor (ASF GSoC 2013 admins
>>> approve/decline proposed project)
>>>   6.  Rajith and OODT mentor find out if the project was approved
>>>   7.  Project happens
>>>   8. OODT mentor and Rajith must coordinate, and then work together
>>> through the process.
>>>
>>>
>>> HTH!
>>>
>>> Cheers,
>>> Chris
>>>
>>> ++
>>> Chris Mattmann, Ph.D.
>>> Senior Computer Scientist
>>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>>> Office: 171-266B, Mailstop: 171-246
>>> Email: chris.a.mattm...@nasa.gov
>>> WWW:  http://sunset.usc.edu/~mattmann/
>>> ++
>>> Adjunct Assistant Professor, Computer Science Department
>>> University of Southern California, Los Angeles, CA 90089 USA
>>> ++
>>>
>>>
>>>
>>>
>>>
>>>
>>> -Original Message-
>>> From: Rajith Siriwardana 
>>> Reply-To: "dev@oodt.apache.org" 
>>> Date: Wednesday, March 27, 2013 9:08 AM
>>> To: "dev@oodt.apache.org" 
>>> Subject: Interested in participating GSoC 2013 with OODT
>>>
>>> >hi,
>>> >I'm a 4th year (final year) undergraduate of Faculty of Engineering
>>> >University of Moratuwa majoring in Computer Science and Engineering
>>>and I
>>> >would like contribute to OODT.
>>> >I did my internship last year which spanned for 9 months in WSO2[1].
>>> which
>>> >is a middle-ware company and I worked with their WSO2 Stratos
>>>product[2].
>>> >Basically I was assigned for developing WSO2 Stratos system level
>>>health
>>> >monitoring component and WSO2 Stratos heartbeat monitoring package.
>>>For
>>> >WSO2 Stratos health monitoring component[3] before finalizing
>>>Nagios[4]
>>> >for
>>> >monitoring with mk_livestatus[5] replacing NagiosXML engine for query
>>> >stats, which is shown in the architecture image, I did some work with
>>> >Ganglia (how to use it for WSO2 Stratos system level h

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

Hi Amila,

I think that WPS can potentially be something that Airavata and/or OODT
help to layer on top of SIS as a core library.

Thanks!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: AMILA RANATUNGA 
Reply-To: "dev@oodt.apache.org" 
Date: Friday, April 5, 2013 9:25 PM
To: Martin Desruisseaux 
Cc: "d...@sis.apache.org" , "d...@airavata.apache.org"
, Harsha Kumara , Shahani
Markus Weerawarana , Nipuni Perera
, "dev@oodt.apache.org" 
Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hi,
>
>Thank you for the long reply. As you suggest "GeoTk" is the core part
>which
>suits to scientists. And "Constellation-SDI" is intended to
>provide web-services using maximum use of those tools. Constellation-SDI
>consisted of WPS, WMS, WFS as server modules. So will that integration
>make
>Apache SIS be considered as those services enabled?
>
>And why you said " Having SIS to implement WMS, WMTS, WCS and WFS is a
>must"?
>What about WPS. Will that make SIS out of the scope. Because we feel that
>since Airavata using Science gateway concept, really essential to
>implement
>WPS too.
>
>Thank You !
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>On Fri, Apr 5, 2013 at 7:04 PM, Martin Desruisseaux <
>martin.desruisse...@geomatys.fr> wrote:
>
>> Hello Amila
>>
>> Le 05/04/13 12:57, AMILA RANATUNGA a écrit :
>>
>>  the slide 21 describes remaining code to move as WMS, WCS, WCTS, WPS
>>and
>>> more. Is that mean Apache SIS does not support them?
>>>
>>
>> Yes. SIS is still in an early stage and does not support WMS, WCS and
>> similar services yet.
>>
>>
>>
>>  And GeoTk code was moved to SIS and claims that reference
>>implementation
>>> of
>>> GEOAPI.
>>>
>>
>> Geotk code is in process of being moved to SIS. But only metadata port
>>is
>> close to completion. The next module to port will be referencing
>>(hopefully
>> completed before FOSS4G in September).
>>
>>
>>  geotoolkit.org (...snip...) Mapfaces (...snip...) constellation-sdi
>>> (...snip...) puzzle-gis (...snip...)
>>>
>>>
>>> Will integrating those into sis make one step ahead to "SIS
>>>well-suited to
>>> some communities (*scientists, but also non-scientists* wanting to
>>>explore
>>>
>>> data in more dimensions than the usual x,y)."?
>>>
>>
>> Maybe I should said that those projects will not be automatically added
>>to
>> SIS. They will be offered, but by the time we reach them, the
>>technologies
>> may have evolved to a point where peoples may want to explore other
>> approaches. For example MapFaces is built on top of JSF. But maybe some
>> peoples will want to explore the Play framework instead. An other
>>example
>> is Swing-based technologies, which are going to be phased out in favour
>>of
>> JavaFX. However we may still use the existing code as a starting point
>>and
>> try to port them to the new technologies. We will revisit this issue
>>when
>> we will be there.
>>
>> The core part aiming to make SIS "well suited to scientists" is Geotk.
>> First by its focus on ISO 19115 metadata for describing the data. Those
>> metadata include a package for describing data quality, an aspect
>>usually
>> neglected by mass-market projects but important for scientists. The
>>GeoTk
>> (future SIS) referencing module takes its information directly from the
>> EPSG database, which provides us information about transformation
>>accuracy
>> and CRS (Coordinate Reference System) area of validity. Many popular
>> projects use simplified version of EPSG database without those
>>information,
>> since not anyone see them as useful. GeoTk paid high attention to
>> correctness through our current effort of expanding 'geoapi-conformance'
>> test suite with the GIGS tests (provided by the EPSG authors). GeoTk
>>also
>> have support for n-dimensional CRS. Those CRS may be more than
>>(x,y,z,t),
>> for example meteorologists use 2 time axes and oceanographers often use
>> pressure instead of z. On the coverages (rasters) side, GeoTk provides a
>> way to describe the meaning of pixel values (by contrast with some
>>projects
>> handling rasters basically as RGB images), which allow for example to
>> compute "gradient of sea surface temperature" without confusing a
>> temperature value with a pixel covered by a cloud (without such
>>knowledge,
>> calculations like "gradient" produce strong artefacts). Large dataset
>>can
>> be organized in a database schema designed for making easier the
>> statistical analysis over time series.
>>
>> Constellation-

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

Hi Suresh,

Just to add to the below. I *do* see Apache SIS as a reference
implementation of OGC standards that Martin and others are working
on. I'm also a GeoAPI WG member at OGC mainly through Martin's
encouragement
and my own desire to get better at geospatial software.

We're starting small at the library level, with plans to grow big
and support all the way up the stack. There are a ton of interested
geo people here on the list, but Martin is doing the bulk of the
work on the code base at the moment, so we're trying to keep up
and follow his lead.

We welcome any contribution you guys have and connections to Airavata
and to OODT.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Suresh Marru 
Reply-To: "d...@sis.apache.org" 
Date: Wednesday, April 3, 2013 8:01 PM
To: "d...@airavata.apache.org" 
Cc: "d...@sis.apache.org" , Harsha Kumara
, Shahani Markus Weerawarana ,
Nipuni Perera , "dev@oodt.apache.org"

Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hi Martin,
>
>Thank you for taking time to respond in detail. Please see comments
>inline:
>On Apr 3, 2013, at 6:00 PM, Martin Desruisseaux
> wrote:
>
>> Le 03/04/13 23:30, Suresh Marru a écrit :
>>> Hello Martin, It is really pleasing to hear such a commitment from
>>>some one deeply engaged in OGC. While I agree with you on the influence
>>>on a younger project and also the impact an open community process like
>>>Apache can have. I personally respect OGC as a governing organization
>>>and as a standards defining body. But we all could not deny the fact
>>>that community rallied behind OGC and produced some good software. I am
>>>curious to learn how will community respond to Apache SIS vs any
>>>software endorsed by OGC? Do you see SIS positioning itself as a
>>>reference implementation for the OGC standard?
>> 
>> I think that SIS will probably be a reference implementation of GeoAPI
>>[1]. But I think that being a reference implementation of other
>>standards implies a strong participation in the standard working group,
>>which may be done on a case-by-case basis depending on volunteer energy.
>
>That makes sense. 
>
>> However maybe your question was rather if SIS would be officially OGC
>>compliant? This is a different question. Being OGC compliant means
>>passing the CITE tests [2]. Actually, executing the CITE tests will be
>>part of SIS Maven build after we ported the relevant part (I mean, CITE
>>tests can be executed every time the project is built).
>
>I am more wondering on the OGC Compliance from an interoperability stand
>point and not so much on official stamp. CITE tests as part of maven
>builds sounds very interesting.
>
>> Companies can also paid OGC for testing their software and get the
>>official "OGC compliant" logo. This is something that Geomatys plans to
>>do, but it would be on top of SIS rather than directly in the SIS
>>project. With the above-mentioned CITE tests executed at build time, I
>>think that anyone would be able to do that on their side.
>> 
>> Note that CITE tests are essentially about Web Services. An other
>>significant source of tests is GIGS [3]. Those tests are being
>>implemented in GeoAPI, and SIS will also execute them.
>> 
>> On the question about how community will respond to Apache SIS, I think
>>that OGC standards are so large that no single software in the world
>>implement all of them. Different softwares may focus on different needs.
>>We can probably not please to every communities. My hope is rather to
>>have SIS well-suited to some communities (scientists, but also
>>non-scientists wanting to explore data in more dimensions than the usual
>>x,y).
>
>I agree with this. Looking forward to see SIS gets widely adopted.
>
>Thanks again for taking time to elaborate in detail, I kind of got of a
>feel for it. 
>Suresh
>
>>Martin
>> 
>> 
>> [1] http://www.opengeospatial.org/standards/geoapi
>> [2] http://cite.opengeospatial.org/teamengine/
>> [3] http://www.epsg.org/gigs.html
>> 
>

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

What Martin said. x50 :)

Great job Martin.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Martin Desruisseaux 
Organization: Geomatys
Reply-To: "d...@sis.apache.org" 
Date: Wednesday, April 3, 2013 2:13 PM
To: "d...@sis.apache.org" 
Cc: Harsha Kumara , "d...@airavata.apache.org"
, Shahani Markus Weerawarana
, Nipuni Perera ,
"dev@oodt.apache.org" 
Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hello Kumara
>
>Le 03/04/13 15:43, Harsha Kumara a écrit :
>> Those resources are really helpful for us to get a good background about
>> science workflows and gateways.Now we  focus more on to geoscience
>>related
>> aspects.
>> As you suggested we are looking into Apache SIS to some extent. But we
>>need
>> to know whether that integration will completely expose OGC and how far
>> convenient it will be for scientists.
>
>To give some context, it may be worth to present some of the SIS
>contributors.
>
>I'm myself an oceanographer by formation (I'm not supposed to be a
>developer) and started to work on GIS as a side effect of a Ph.D thesis
>in oceanography, with a strong need for statistical analysis work. For
>this reason, my contribution to SIS is strongly focused on scientist
>needs. While I admit that ease of use is important, my personal approach
>is to try to be accurate first, then provide convenience methods next.
>Incidentally, I work in a small company which is hosted by a public
>research institutes specialized in remote sensing images.
>
>We are also OGC members. I'm the current chair of the GeoAPI working
>group. Frédéric Houbie is a member of the OGC Architecture Board (OAB)
>and the editor of some OGC specifications. Johann Sorel has done 2 or 3
>years ago for OGC Met-Ocean group a proof of concept of SLD usability to
>meteorological maps.
>
>We are in the process of porting approximatively one million lines of
>code to Apache SIS. While only ISO 19115 metadata is emerging right now,
>the stack to port comprise referencing services, coverages, features,
>processing, rendering, catalog, sensors, web services (WMS, WCS, WPS,
>etc.), desktop application, and more. They are currently available as
>various open source projects (Geotoolkit.org, MapFaces,
>Constellation-SDI, Puzzle-GIS, MD-Web) that we would like to consolidate
>in Apache SIS. In this process, we are putting effort in consolidation,
>documenting, adding more tests, and resolving some known limitations.
>
>We are in an interesting situation where the project is both relatively
>young, and disposes of a relatively large code base. Peoples can
>evaluate the existing code in the above-mentioned projects and express
>their concerns before we port them to Apache SIS. It may be a nice
>opportunity, because it is easier to influence a young project than an
>old one in order to meet your needs.
>
> Regards,
>
> Martin
>

Re: Research project on integrating geoservices with Apache Airavata

[Mattmann, Chris A (398J)]

Hi Harsha,

Thanks for reaching out!

Comments below:



-Original Message-
From: Harsha Kumara 
Date: Wednesday, April 3, 2013 6:43 AM
To: jpluser 
Cc: "d...@airavata.apache.org" , Shahani Markus
Weerawarana , Nipuni Perera ,
"dev@oodt.apache.org" , "d...@sis.apache.org"

Subject: Re: Research project on integrating geoservices with Apache
Airavata

>Hi Chris,
>(This is the same mail that we sent as a private mail.Since this
>discussions are important for others as well,we moved it to here)
>
>
>Thanks a lot for the resources and the support given so far.
>
>
>Those resources are really helpful for us to get a good background about
>science workflows and gateways.Now we  focus more on to geoscience
>related aspects.
>As you suggested we are looking into Apache SIS to some extent. But we
>need to know whether that integration will completely expose OGC and how
>far convenient it will be for scientists.

I think that the focus of SIS at the moment is as a small library, getting
bigger. Depends on what
you mean specifically by OGC? Here are the OGC that SIS has focused on to
date, and where it's going:

ISO 19103 Conceptual schema language
ISO 19115 Metadata
ISO 19111 Spatial referencing by coordinates

(see slide 11 in 
http://svn.apache.org/repos/asf/sis/presentations/NOAA-Meetup-Feb21-2013/Ap
acheSIS-NOAAMeetup-Final.fodp)

The metadata module is now becoming very fully featured, and ready for
benchmarking

and testing.

> 
>
>
>We also look at some domain related products such as Grass, GeoSever,
>MapServer, GeoNode, qgis, PyWPS,ZOO Project and etc..It would be nicer
>for us to sense some experience from the point of view of a
> geoscientist and OGC Standard user. We appreciate any helpful resources
>and guidance.

Great! Watch for LGPL and GPL dependencies on these services. Service
dependencies
(whether static or dynamic) that provide great functionality and will be
such that
users want to leverage it for using GIS and Airavata violate the "no
surprises" 
mantra if they ref LGPL and GPL components.

I'm a big fan of GeoServer -- however it's not Category-A or ALv2
compatible at the moment.
That was one of the big reasons for starting SIS -- but it's taken a big
undertaking to 
even get to the point that we are at.

HTH!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++