Re: Request for advice on workflows
Hi Chris,
Apologies for my delay in replying. I'm still seeing this issue. I have a
sequential workflow that's working fine, but when I add a parallel section
I'm seeing the error that I pasted into my previous email. I might be
missing some config steps. Is my understanding correct - that OODT is
trying to generate a special task (branch redirection task) that sets up a
new workflow on a batch node? What should happen after that? Should this
workflow task then try to run one of the parallel tasks? I've configured a
batch stub running on the same machine as the master workflow to handle
these requests... I'm not sure if it can access the same workflow manager
configs though... I checked the workflow manager URL and it seems fine for
the branch redirection task.
Ross
On 15 May 2015 at 18:11, Mattmann, Chris A (3980)
chris.a.mattm...@jpl.nasa.gov wrote:
Ross you still having this issue?
++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattm...@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++
-Original Message-
From: Ross Laidlaw rlaidlaw.o...@gmail.com
Reply-To: dev@oodt.apache.org dev@oodt.apache.org
Date: Wednesday, May 13, 2015 at 2:37 AM
To: dev@oodt.apache.org dev@oodt.apache.org
Subject: Re: Request for advice on workflows
Hi Chris,
As a first step I got an XMLWorkflowRepository workflow running. I then
tried using the PackagedWorkflowRepository to create a workflow with the
same tasks. I got a sequential workflow up and running, similar to the
following:
cas:workflows ...
sequential id=urn:test:MainWorkflow name=MainWorkflow
task id-ref=urn:test:TaskA/
task id-ref=urn:test:TaskB/
task id-ref=urn:test:TaskC/
task id-ref=urn:test:TaskD/
task id-ref=urn:test:TaskE/
task id-ref=urn:test:TaskF/
task id-ref=urn:test:TaskG/
/sequential
!-- task definitions --
task.../task
/cas:workflows
I found that I could run the workflow using ./wmgr-client --url
[WorkflowManagerUrl] --sendEvent --eventName [WorkflowId]
e.g. ./wmgr-client --url http://localhost:9001 --sendEvent --eventName
urn:test:MainWorkflow
After getting the sequential workflow running, I wanted to run tasks C and
D in parallel so I then tried adding a parallel section as follows:
cas:workflows ...
sequential id=urn:test:MainWorkflow name=MainWorkflow
task id-ref=urn:test:TaskA/
task id-ref=urn:test:TaskB/
parallel id=urn:test:ParallelWorkflow name=ParallelWorkflow
task id-ref=urn:test:TaskC/
task id-ref=urn:test:TaskD/
/parallel
task id-ref=urn:test:TaskE/
task id-ref=urn:test:TaskF/
task id-ref=urn:test:TaskG/
/sequential
...
/cas:workflows
The parallel section generates an additional task to be submitted called
'Redirector Task' that wants to be executed in the 'high' queue using the
BranchRedirector class:
INFO: Task: [Redirector Task] has no required metadata fields
May 13, 2015 1:27:29 AM
org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob
FINEST:
[
{
job.queueName=high,
job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob,
job.name=redirector-322db9d2-4663-4b89-9af0-f8f97ad74c80,
job.id=, job.status=, job.load=2,
job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput
},
{
task.instance.class=org.apache.oodt.cas.workflow.examples.BranchRedirector
,
task.config={},
task.metadata={...}
}
]
I wasn't sure how to configure my setup to deal with this task. From
reading OODT-70 I gather that it might be trying to start up other
wmgr-clients on batch stubs to run these tasks in separate workflows. I
tried setting up a local batch stub that accepts jobs from the 'high'
queue. When running the full workflow I get the following exception from
the batch stub when it tries to execute the redirector task:
INFO: Assigning job: [redirector-8561de94-2681-477b-8bcb-331d83224d5b] to
node: [local-redirector]
May 13, 2015 1:36:51 AM
org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub
genericExecuteJob
INFO: stub attempting to execute class:
[org.apache.oodt.cas.workflow.structs.TaskJob]
org.apache.xmlrpc.XmlRpcClientException: Failure writing request
at
org.apache.xmlrpc.XmlRpcClientRequestProcessor.encodeRequest(XmlRpcClientR
equestProcessor.java:76)
at
org.apache.xmlrpc.XmlRpcClientRequestProcessor.encodeRequestBytes(XmlRpcCl
ientRequestProcessor.java:102
Re: Request for advice on workflows
at org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBatchMgrProxy.java:125) It looks like my sequential and parallel workflows have the required names and IDs. Do you know what could be going wrong? Maybe I'm missing a few vital configuration steps. Many thanks for your help, Ross On 8 May 2015 at 19:31, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Hi Chris, Many thanks for your advice. I'll check out the PackagedWorkflowRepository and will let you know how it goes. Ross On May 8, 2015, at 12:58 PM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Hey Ross: Great work! Try the PackagedWorkflowRepository. The best documentation that exists of it is in OODT-70 [1]. I would use that to support the type of workflows that you’re doing. If this works we should make it the default since it’s supposed to be back compat with the XMLWorkflowRepositoryManager. Cheers, Chris P.S. see some of the ASO scripts I wrote to check on specific task/ PGE status, etc. [1]https://issues.apache.org/jira/browse/OODT-70 ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Ross Laidlaw rlaidlaw.o...@gmail.com Reply-To: dev@oodt.apache.org dev@oodt.apache.org Date: Friday, May 8, 2015 at 8:14 AM To: dev@oodt.apache.org dev@oodt.apache.org Subject: Request for advice on workflows Hi All, I'm working on an OODT setup (v0.8.1) with several workflow tasks defined and working well individually. I can use the wmgr-client to fire off jobs separately and all appears to be working as planned. I'd like to combine these tasks into a workflow so that I don't have to run each set of jobs. For example, in the workflow I'd like to run several 'task A' jobs and when all these have finished I'd like the workflow to automatically start several 'task B' jobs, etc. I found some example workflows in the distribution, plus several guides up on the wiki. Is there a current recommended way of setting up a simple workflow? Should I use workflow v2 or is it best to stick with v1? Once I've set up a workflow, how do I start it off? It looks like it might be possible to nest workflows, is that functionality available yet? Additionally, what's the best way to check on the status of currently running jobs? I've used the pcs_stat tool a lot and it's great for checking on the overall status, but is there a recommended way to check e.g. if any 'task A' jobs are still running for a certain workflow and what state they're currently in? Many thanks in advance for your advice, Ross
Request for advice on workflows
Hi All, I'm working on an OODT setup (v0.8.1) with several workflow tasks defined and working well individually. I can use the wmgr-client to fire off jobs separately and all appears to be working as planned. I'd like to combine these tasks into a workflow so that I don't have to run each set of jobs. For example, in the workflow I'd like to run several 'task A' jobs and when all these have finished I'd like the workflow to automatically start several 'task B' jobs, etc. I found some example workflows in the distribution, plus several guides up on the wiki. Is there a current recommended way of setting up a simple workflow? Should I use workflow v2 or is it best to stick with v1? Once I've set up a workflow, how do I start it off? It looks like it might be possible to nest workflows, is that functionality available yet? Additionally, what's the best way to check on the status of currently running jobs? I've used the pcs_stat tool a lot and it's great for checking on the overall status, but is there a recommended way to check e.g. if any 'task A' jobs are still running for a certain workflow and what state they're currently in? Many thanks in advance for your advice, Ross
Re: [ANNOUNCE] Apache OODT 0.7 Released
Superb! Nice work everyone! :) On 19 Sep 2014, at 23:28, Tom Barber tom.bar...@meteorite.bi wrote: Good job, chaps. Tom On 20/09/14 05:45, Lewis John Mcgibbney wrote: Boom On Saturday, September 20, 2014, Chris Mattmann mattm...@apache.org wrote: The Apache OODT community is proud to announce the release of Apache OODT 0.7. Apache OODT is a software framework, and an architectural style for the rapid construction of scientific data systems. It provides components for data capture, curation, metadata extraction, workflow management, resource management and data processing. The 0.7 release addresses 79 issues from our JIRA management system: http://issues.apache.org/jira/browse/OODT This release is a core stability release with numerous bug fixes, and a solid baseline for upgrading. It fixes 79 JIRA issues and provides a backwards compatible CAS-PGE upgrade allowing the wrapper to be used with existing config files. 0.7 fixes some bugs in Ganglia support and allows the Resource Manager to work without Ganglia being present. This version also makes significant fixes to File Manager tests, provides a Vagrant version of OODT RADIX and upgrades RADIX to include CAS-PGE support. This version also includes improved support for a new JAX-RS CAS product service, and this version eliminates several bugs in the workflow manager in its support for dynamic workflow instances. The release is available from: http://www.apache.org/dyn/closer.cgi/oodt/ In the first 48 hours, the release will be propagating to the mirrors, so allow time for it to show up in a mirror near you. The release is also made available via Maven Central: http://repo1.maven.org/maven2/org/apache/oodt/ And from PyPi: https://pypi.python.org/pypi/oodt And from PEAR: http://pear.apache.org/oodt/ The Apache OODT website will be updated shortly to reflect the releases. As always if you find anything that you'd like to report (including praise!) please do so on our u...@oodt.apache.org javascript:; (use of OODT) or dev@oodt.apache.org javascript:; (architecture/design of OODT) mailing lists. Also please visit our website: http://oodt.apache.org And our wiki: https://cwiki.apache.org/confluence/display/OODT/Home Thanks! Chris Mattmann (on behalf of the Apache OODT PMC) -- *Tom Barber* | Technical Director meteorite bi *T:* +44 20 8133 3730 *W:* www.meteorite.bi | *Skype:* meteorite.consulting *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK
Review Request 24429: OODT-741: Make three fields in XMLValidationLayer non-static (elementMap, subToSuperMap and productTypeElementMap)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24429/ --- Review request for oodt, brian Foster, Lewis McGibbney, Chris Mattmann, Paul Ramirez, and Rishi Verma. Bugs: OODT-741 https://issues.apache.org/jira/browse/OODT-741 Repository: oodt Description --- The XMLValidationLayer class has a HashMapString, Element called elementMap. This is used to store details about elements from the File Manager's elements.xml configuration file. Currently this field is static, i.e. a class variable, meaning that only one element map is created for the class and this is shared by all XMLValidationLayer objects. This characteristic (being a static/class variable) is causing one of the File Manager unit tests to fail, as detailed below: * The TestXMLValidationLayer#testGetElements test currently fails when doing a full build/test of the File Manager component, with the following result: Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 * This happens because the elementMap field in the XMLValidationLayer class is static (i.e. a class variable), so any data assigned to it persists between objects. When new data is written to the elementMap it is added or overwritten depending on the value of the String key, but data isn't removed from it when new XMLValidationLayer objects are created and all XMLValidationLayer objects currently access the same elementMap. Earlier tests in the test sequence write to elementMap from other elements.xml files (e.g. src/test/resources/examples/core/elements.xml), and this data hangs around in the class variable, so that when testGetElements is executed, there are already 11 elements present in elementMap. I've experimented by removing the 'static' modifier from the elementMap, subToSuperMap and productTypeElementMap fields. This causes each XMLValidationLayer object to have its own elementMap. All TestXMLValidationLayer tests now pass. Additionally, this change does not seem to affect any other File Manager tests (all other tests pass). Diffs - trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/validation/XMLValidationLayer.java 1616362 Diff: https://reviews.apache.org/r/24429/diff/ Testing --- The individual test TestXMLValidationLayer#testGetElements was run and passed. The full set of tests from TestXMLValidationLayer were run and passed. The full set of tests from File Manager were run and passed. All of these steps were carried out both at module level and top-level of the project. Finally, a full build of OODT was carried out and completed successfully (all tests passed). Thanks, Ross Laidlaw
Re: OODT Officially Stable - Hats off to Ross Laidlaw
Thanks all! Glad to contribute! There are some remaining issues to discuss/resolve with file manager tests (e.g. see OODT-736 and OODT-741), but we're getting there! There are also some issues with the protocol tests that Chris highlighted in OODT-644. So Jenkins builds will probably still fail sometimes, but hopefully we can fix these soon. Ross On 8 August 2014 13:47, Verma, Rishi (398J) rishi.ve...@jpl.nasa.gov wrote: Way to go Ross, way to go everyone! Rishi Sent from my iPhone On Aug 8, 2014, at 1:29 PM, Chris Mattmann chris.mattm...@gmail.com wrote: Fscking awesome!! Thank you Ross and everyone! Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Tom Barber tom.bar...@meteorite.bi Reply-To: u...@oodt.apache.org Date: Friday, August 8, 2014 1:07 PM To: u...@oodt.apache.org Subject: Re: OODT Officially Stable - Hats off to Ross Laidlaw Wp On 8 Aug 2014 21:05, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Folks, The OODT trunk (0.7-SNAPSHOT) codebase is stable [0] now for the first time in a while which is extremely good news. I personally (and I'm sure I speak on behalf of many of us) am extremely appreciative of the work Ross has recently put in to ensuring the codebase is stabalized. Hats off Ross. The drinks are on me... and make sure to remind me about it. Have a great weekend folks. Lewis [0] https://builds.apache.org/job/oodt-trunk/1438/ -- Lewis
Review Request 24140: OODT-723: Fix file manager unit tests in the 'validation' package
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24140/ --- Review request for oodt, Lewis McGibbney, Chris Mattmann, Paul Ramirez, and Rishi Verma. Bugs: OODT-723 https://issues.apache.org/jira/browse/OODT-723 Repository: oodt Description --- Fixes tests in the 'validation' package for the file manager. There's a single test class in the package called TestXMLValidationLayer. This patch makes a trivial change to the setUp method so that it can locate the 'vallayer' directory in the src/test/resources area. There aren't any System.setProperty() calls in this test class so this is a straightforward fix. Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/validation/TestXMLValidationLayer.java 1614810 Diff: https://reviews.apache.org/r/24140/diff/ Testing --- As described in OODT-722: The following commands were used to run the tests: At top-level (trunk directory): mvn clean -Dtest=TestXMLValidationLayer -DfailIfNoTests=false test -pl filemgr -am At module level (filemgr directory): mvn clean -Dtest=TestXMLValidationLayer test Test results before applying the patch: --- Running org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 0.038 sec FAILURE! Results : Tests in error: testModifyElement(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testReadProperDescriptionTrimImplicitTrue(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testReadBadFormattedDescriptionTrimImplicitTrue(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testReadDescriptionTrimExplicitFalse(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testReadDescriptionTrimExplicitTrue(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) testGetElementsForProductType(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer) Tests run: 7, Failures: 0, Errors: 7, Skipped: 0 Test results after applying the patch: -- Running org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec Results : Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 Thanks, Ross Laidlaw
Re: Review Request 23870: OODT-685: update the file manager unit tests to reflect changes in the project directory structure
Hi Lewis, Sounds like a smart plan to do things step by step. I'll start experimenting with java.util.Properties and see if I can get a patch together for one of the test packages in file manager. Ross On 26 July 2014 10:37, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Ross, Can you PLEASE make public any changes you have to filemgr? If we can get a test-by-test patch-fix then we are #winning. BTW I also have another local branch with various fixes. We also SHOULD NOT post calls to System.setProperties, as it makes it ultimately more difficult to debug when we wish to add/rremove/edit test cases for OODT. We should instead create a java.util.Properties() object, pump properties into there, then pass this to System.setProperties(properties). Ross, can I please warn you not to touch to many test cases at the same time as this has been my downfall with regards to fixing this issue. We cannot fix everything at once. I have made this mistake over the last few months and do not want you to make it as well. If you fix ANY ONE test then please submit patch (as you are already doing perfectly) then we can merge. BTW apart from anything else a HUGE thank you for tackling this. It is a reasonable undertaking and OODT 0.7 will result directly from the stable tests we will achieve with this issue. Thank you SO much Ross. On Fri, Jul 25, 2014 at 11:04 AM, Rishi Verma rive...@apache.org wrote: This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/ Hi Ross, So for example, for src/test/resources/filemgr.properties, the first line above would be: URL url = this.getClass().getResource(/filemgr.properties); I've tried the above in a few unit tests and it seems to work so far. I'll see if I can get a patch together. Yes, that looks great. Getting the resources straight from the build via getResource is the way to go. Thank you Ross! - Rishi Verma On July 23rd, 2014, 11:33 p.m. UTC, Ross Laidlaw wrote: Review request for oodt, Lewis McGibbney, Chris Mattmann, and Paul Ramirez. By Ross Laidlaw. *Updated July 23, 2014, 11:33 p.m.* *Bugs: * OODT-685 https://issues.apache.org/jira/browse/OODT-685 *Repository: * oodt Description After changes in the file manager directory structure for OODT-685, some of the file manager unit tests required updating to point to the new location for test resources (src/testdata moved to src/test/resources). This patch updates several of the file manager unit tests that referred to src/testdata to point to src/test/resources instead. It also removes duplicate XML from the following test resources in src/test/resources: cmd-line-actions.xml, cmd-line-options.xml and product-types.xml. After applying the patch, there are still some issues when running file manager tests (one test error and one test failure), see below for details. Testing Before applying the patch, the majority of file manager tests fail, as shown by the results below: Tests run: 189, Failures: 74, Errors: 30, Skipped: 0 After applying the patch, the majority of file manager tests pass, except for the following (1 failure, 1 error): Results : Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 Tests in error: testAddProductType(org.apache.oodt.cas.filemgr.cli.TestFileManagerCli): org.apache.oodt.cas.cli.exception.CmdLineConstructionException: Invalid option: 'typeName' Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestMappedDataSourceCatalog.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestOrderedDataSourceCatalog.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/datatransfer/TestLocalDataTransferer.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/repository/TestXMLRepositoryManager.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java (1612886) - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManager.java (1612886) - /trunk
Re: Review Request 23870: OODT-685: update the file manager unit tests to reflect changes in the project directory structure
On July 25, 2014, 3:16 p.m., Rishi Verma wrote: I call this a success: BEFORE PATCH: Tests run: 190, Failures: 54, Errors: 13, Skipped: 0 AFTER PATCH: Tests run: 190, Failures: 1, Errors: 1, Skipped: 0 The single error and failure is listed below: === Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 Tests in error: testAddProductType(org.apache.oodt.cas.filemgr.cli.TestFileManagerCli): org.apache.oodt.cas.cli.exception.CmdLineConstructionException: Invalid option: 'typeName' === Will need to look more, but Ross, this is a huge step, so great job! Hi Rishi, thanks for the review! Lewis and I tried applying this patch (and variations of it) yesterday. It looked promising at first, and it fixes the majority of tests when run from the filemgr module. But we found that when running mvn test (or mvn install, etc) from the top level of the project, this patch didn't fix the problem as there were still issues with relative paths. Lewis was looking into using something like the following, which I think is the recommended way of accessing resources from unit tests: URL url = this.getClass().getResource(path to file within src/test/resources); String filePath = new File(url.getFile()).getAbsolutePath(); So for example, for src/test/resources/filemgr.properties, the first line above would be: URL url = this.getClass().getResource(/filemgr.properties); I've tried the above in a few unit tests and it seems to work so far. I'll see if I can get a patch together. - Ross --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/#review48735 --- On July 23, 2014, 11:33 p.m., Ross Laidlaw wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/ --- (Updated July 23, 2014, 11:33 p.m.) Review request for oodt, Lewis McGibbney, Chris Mattmann, and Paul Ramirez. Bugs: OODT-685 https://issues.apache.org/jira/browse/OODT-685 Repository: oodt Description --- After changes in the file manager directory structure for OODT-685, some of the file manager unit tests required updating to point to the new location for test resources (src/testdata moved to src/test/resources). This patch updates several of the file manager unit tests that referred to src/testdata to point to src/test/resources instead. It also removes duplicate XML from the following test resources in src/test/resources: cmd-line-actions.xml, cmd-line-options.xml and product-types.xml. After applying the patch, there are still some issues when running file manager tests (one test error and one test failure), see below for details. Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestMappedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestOrderedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/datatransfer/TestLocalDataTransferer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/repository/TestXMLRepositoryManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManagerClient.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestExpImpCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestMetadataBasedProductMover.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestXmlRpcStructFactory.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/validation/TestXMLValidationLayer.java 1612886 /trunk/filemgr/src/test/java/org
Re: Review Request 23870: OODT-685: update the file manager unit tests to reflect changes in the project directory structure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/ --- (Updated July 23, 2014, 11:33 p.m.) Review request for oodt, Lewis McGibbney, Chris Mattmann, and Paul Ramirez. Bugs: OODT-685 https://issues.apache.org/jira/browse/OODT-685 Repository: oodt Description (updated) --- After changes in the file manager directory structure for OODT-685, some of the file manager unit tests required updating to point to the new location for test resources (src/testdata moved to src/test/resources). This patch updates several of the file manager unit tests that referred to src/testdata to point to src/test/resources instead. It also removes duplicate XML from the following test resources in src/test/resources: cmd-line-actions.xml, cmd-line-options.xml and product-types.xml. After applying the patch, there are still some issues when running file manager tests (one test error and one test failure), see below for details. Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestMappedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestOrderedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/datatransfer/TestLocalDataTransferer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/repository/TestXMLRepositoryManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManagerClient.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestExpImpCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestMetadataBasedProductMover.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestXmlRpcStructFactory.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/validation/TestXMLValidationLayer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestBasicVersioner.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestDateTimeVersioner.java 1612886 /trunk/filemgr/src/test/resources/cmd-line-actions.xml 1612886 /trunk/filemgr/src/test/resources/cmd-line-options.xml 1612886 /trunk/filemgr/src/test/resources/xmlrpc-struct-factory/product-types.xml 1612886 Diff: https://reviews.apache.org/r/23870/diff/ Testing --- Before applying the patch, the majority of file manager tests fail, as shown by the results below: Tests run: 189, Failures: 74, Errors: 30, Skipped: 0 After applying the patch, the majority of file manager tests pass, except for the following (1 failure, 1 error): Results : Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 Tests in error: testAddProductType(org.apache.oodt.cas.filemgr.cli.TestFileManagerCli): org.apache.oodt.cas.cli.exception.CmdLineConstructionException: Invalid option: 'typeName' Thanks, Ross Laidlaw
Review Request 23870: OODT-685: update the file manager unit tests to reflect changes in the project directory structure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/ --- Review request for oodt, Lewis McGibbney, Chris Mattmann, and Paul Ramirez. Bugs: OODT-685 https://issues.apache.org/jira/browse/OODT-685 Repository: oodt Description --- After some changes in the file manager directory structure for OODT-685, some of the file manager unit tests required updating to point to the new location for some test resources (src/testdata moved to src/test/resources). This patch updates some of the file manager unit tests that referred to src/testdata to point to src/test/resources instead. It also removes some duplicate XML from the following test resources in src/test/resources: cmd-line-actions.xml, cmd-line-options.xml and product-types.xml. After applying the patch, there are still some issues when running file manager tests (one test error and one test failure), see below for details. Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestMappedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestOrderedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/datatransfer/TestLocalDataTransferer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/repository/TestXMLRepositoryManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManagerClient.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestExpImpCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestMetadataBasedProductMover.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestXmlRpcStructFactory.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/validation/TestXMLValidationLayer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestBasicVersioner.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestDateTimeVersioner.java 1612886 /trunk/filemgr/src/test/resources/cmd-line-actions.xml 1612886 /trunk/filemgr/src/test/resources/cmd-line-options.xml 1612886 /trunk/filemgr/src/test/resources/xmlrpc-struct-factory/product-types.xml 1612886 Diff: https://reviews.apache.org/r/23870/diff/ Testing --- Before applying the patch, the majority of file manager tests fail, as shown by the results below: Tests run: 189, Failures: 74, Errors: 30, Skipped: 0 After applying the patch, the majority of file manager tests pass, except for the following (1 failure, 1 error): Results : Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 Tests in error: testAddProductType(org.apache.oodt.cas.filemgr.cli.TestFileManagerCli): org.apache.oodt.cas.cli.exception.CmdLineConstructionException: Invalid option: 'typeName' Thanks, Ross Laidlaw
Re: Review Request 23870: OODT-685: update the file manager unit tests to reflect changes in the project directory structure
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23870/ --- (Updated July 23, 2014, 11:31 p.m.) Review request for oodt, Lewis McGibbney, Chris Mattmann, and Paul Ramirez. Bugs: OODT-685 https://issues.apache.org/jira/browse/OODT-685 Repository: oodt Description (updated) --- After changes in the file manager directory structure for OODT-685, some of the file manager unit tests required updating to point to the new location for some test resources (src/testdata moved to src/test/resources). This patch updates several of the file manager unit tests that referred to src/testdata to point to src/test/resources instead. It also removes duplicate XML from the following test resources in src/test/resources: cmd-line-actions.xml, cmd-line-options.xml and product-types.xml. After applying the patch, there are still some issues when running file manager tests (one test error and one test failure), see below for details. Diffs - /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestMappedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/catalog/TestOrderedDataSourceCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/cli/TestFileManagerCli.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/datatransfer/TestLocalDataTransferer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestCachedIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestLocalCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestRmiCache.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/ingest/TestStdIngester.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/repository/TestXMLRepositoryManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/structs/type/TestTypeHandler.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManager.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/system/TestXmlRpcFileManagerClient.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestExpImpCatalog.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/tools/TestMetadataBasedProductMover.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/util/TestXmlRpcStructFactory.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/validation/TestXMLValidationLayer.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestBasicVersioner.java 1612886 /trunk/filemgr/src/test/java/org/apache/oodt/cas/filemgr/versioning/TestDateTimeVersioner.java 1612886 /trunk/filemgr/src/test/resources/cmd-line-actions.xml 1612886 /trunk/filemgr/src/test/resources/cmd-line-options.xml 1612886 /trunk/filemgr/src/test/resources/xmlrpc-struct-factory/product-types.xml 1612886 Diff: https://reviews.apache.org/r/23870/diff/ Testing --- Before applying the patch, the majority of file manager tests fail, as shown by the results below: Tests run: 189, Failures: 74, Errors: 30, Skipped: 0 After applying the patch, the majority of file manager tests pass, except for the following (1 failure, 1 error): Results : Failed tests: testGetElements(org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer): There aren't exactly 4 elements in the test samples! expected:4 but was:11 Tests in error: testAddProductType(org.apache.oodt.cas.filemgr.cli.TestFileManagerCli): org.apache.oodt.cas.cli.exception.CmdLineConstructionException: Invalid option: 'typeName' Thanks, Ross Laidlaw
Re: cas-product .war artifact huge
Hi Chris Lewis, I think Chris is right, probably it's the Apache CXF dependencies that have increased the WAR size. As Chris suggested, I'll try to learn about Sergey's solution for Tika to see if we can do something similar for the CAS Product webapp. Ross On 1 Jul 2014, at 04:54, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: I think we should consider that. That is a bit of a beast and is a waste of resources, storage and bandwidth if it is not required. If we can avoid this then I think we should. Thanks for response. On Mon, Jun 30, 2014 at 11:44 PM, Chris Mattmann chris.mattm...@gmail.com wrote: @Ross Laidlaw can answer this - I'm thinking the CXF integration? Note -- this can be adapted as Sergey B in Tika has shown in some thread that we can just dep on the minimal jar for CXF Chris Mattmann chris.mattm...@gmail.com -Original Message- From: Lewis John Mcgibbney lewis.mcgibb...@gmail.com Reply-To: dev@oodt.apache.org Date: Monday, June 30, 2014 8:10 PM To: dev@oodt.apache.org dev@oodt.apache.org Subject: cas-product .war artifact huge Hi Folks, Anyone have a clue why the cas-product war artifact is 30,500KB in size? Lewis -- *Lewis* -- *Lewis*
Re: What happened to this wiki page:
Hi Chris, A while back when I was doing my GSoC project last summer, I created a set of wiki pages for the product server REST API. These were an installation guide, a developer's guide and a user guide. For the user guide, I found the existing File Manager REST API page and added some new details to it. I then renamed it to the following: https://cwiki.apache.org/confluence/display/OODT/CAS+Product+Server+Web+Application+-+User+Guide The installation guide and developers guide are as follows: https://cwiki.apache.org/confluence/display/OODT/CAS+Product+Server+Web+Application+-+Installation+Guide https://cwiki.apache.org/confluence/display/OODT/CAS+Product+Server+Web+Application+-+Developer+Guide In the new layout of wiki pages, here are the locations of the above guides: Navigation Catalog and Archive CAS Developer Guides CAS Product Server Web Application - Developer Guide Navigation Catalog and Archive CAS Installation Guides CAS Product Server Web Application - Installation Guide Navigation Catalog and Archive CAS User Guides CAS Product Server Web Application - User Guide Sorry for causing any confusion and for breaking any links you may have had to the previous versions of the File Manager REST API guide. Hopefully the document history is still available even after the renaming. Ross On 18 June 2014 17:27, ke...@apache.org wrote: HAX. --k On 2014-06-18, at 11:26 AM, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: We can view the pages on the wiki here https://cwiki.apache.org/confluence/collector/pages.action?key=OODT This enables us to track changes right back until - Home https://cwiki.apache.org/confluence/display/OODT/Home Feb 06, 2010 • created by Gavin https://cwiki.apache.org/confluence/display/%7Eipv6guru So basically, there is no history of the page being edited for the entire duration of the wiki existence!!! I am stumped on this one folks. Lewis On Wed, Jun 18, 2014 at 12:18 PM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: no worries Tom, Lewis is looking into it - I just recreated it. Just thought you may know since you reorganized the wiki ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Tom Barber tom.bar...@meteorite.bi Date: Wednesday, June 18, 2014 8:46 AM To: dev@oodt.apache.org Subject: Re: What happened to this wiki page: Well i never deleted anything (to my knowledge)! On 18/06/14 17:43, Mattmann, Chris A (3980) wrote: Hey Guys, (Tom Barber, specifically), Does anyone know what happened to this page? https://cwiki.apache.org/confluence/display/OODT/File+Manager+REST+API I created it a while back and I can find it in an email reference on the mail archives, but it's no longer on the wiki. Any ideas? Cheers, Chris ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -- Tom Barber | Technical Director meteorite bi T: +44 20 8133 3730 W: www.meteorite.bi http://www.meteorite.bi | Skype: meteorite.consulting A: Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK -- *Lewis*
Review Request 14513: OODT-612 OODT-613 - Content writers and configurations for the CAS Product Server Web Application (JAX-RS part)
/RdfWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/ReferenceFileWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/ReferenceRssWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/ReferenceZipWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/RssWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/TransfersRssWriter.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/jaxrs/writers/package-info.java PRE-CREATION trunk/webapp/fmprod/src/main/resources/georss-config.xml 1517156 trunk/webapp/fmprod/src/main/resources/logging.properties 1517156 trunk/webapp/fmprod/src/main/resources/rdfconf.xml 1517156 trunk/webapp/fmprod/src/main/resources/rss-transfer-conf.xml 1517156 trunk/webapp/fmprod/src/main/resources/rssconf.xml 1517156 trunk/webapp/fmprod/src/main/webapp/META-INF/context.xml 1517156 trunk/webapp/fmprod/src/main/webapp/WEB-INF/web.xml 1517156 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/DatasetResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/MetadataResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/ProductResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/ReferenceResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/TransferResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/jaxrs/resources/TransfersResourceTest.java PRE-CREATION trunk/webapp/fmprod/src/test/resources/test.logging.properties 1517156 Diff: https://reviews.apache.org/r/14513/diff/ Testing --- I implemented several JUnit tests for the new package. Some of these unit tests are included in this patch, while others have yet to be formalised but will be presented in future patches. I also performed a lot of manual testing by deploying the web application and testing sets of URLs via web browsers and command line tools (e.g. curl). I checked all of the outputs for validity (e.g. as valid XML, RSS, RDF, etc). Thanks, Ross Laidlaw
Review Request 13000: OODT-651: Improve parameter initialization, validation and logging for the CAS-Product JAXRS service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13000/ --- Review request for oodt, Chris Mattmann and Rishi Verma. Bugs: OODT-651 https://issues.apache.org/jira/browse/OODT-651 Repository: oodt Description --- Summary --- The aim of this patch is to make some improvements to the JAX-RS service in the CAS-Product web application. I've moved some initialization of various parameters from the resource classes into a servlet class, so that it doesn't happen every time a call is made to the RESTful service. I've also added some extra logging, along with an example (Tomcat) logging.properties file. I've tidied up some of the code, e.g. reducing some duplication. I've updated the comments and improved the Javadocs. Details --- - I moved all of the context parameter definitions from web.xml to context.xml so that they're all in one place and possibly easier to keep track of. - I created a new class CasProductJaxrsServlet that extends org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet. This new class is now our entry point to the JAX-RS service (configured in the web.xml file). - The new servlet class is used to initialize and validate certain parameters, such as the file manager's working directory and file manager client, based on context parameters (now all defined in context.xml). Previously these were initialized in the resource classes each time an HTTP request was made. After they have been initialized and validated, the working directory and file manager client instance are stored as servlet context attributes, using context.setAttribute(String name, Object value). - I abstracted some of the duplicated code in each resource class into a new abstract class called 'Resource'. This class is used to inject the servlet context and retrieve some of the initialized fields (i.e. the file manager working directory and client) using context.getAttribute(String name). - I added some extra logging in the resource and responder classes. I also added an example logging.properties file for a basic default setup when the web application is deployed. - I corrected and updated some of the comments and Javadocs. Diffs - trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/DatasetResource.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ProductResource.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ReferenceResource.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/Resource.java PRE-CREATION trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/TransferResource.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ZipResponder.java 1513410 trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/servlets/CasProductJaxrsServlet.java PRE-CREATION trunk/webapp/fmprod/src/main/resources/logging.properties PRE-CREATION trunk/webapp/fmprod/src/main/webapp/META-INF/context.xml 1513410 trunk/webapp/fmprod/src/main/webapp/WEB-INF/web.xml 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/DatasetResourceTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ProductResourceTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ReferenceResourceTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ResourceTestBase.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/FileResponderTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/NullResponderTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ResponderFactoryTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/UnrecognizedFormatResponderTest.java 1513410 trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ZipResponderTest.java 1513410 trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/product-types.xml 1513410 trunk/webapp/fmprod/src/test/resources/test.logging.properties 1513410 Diff: https://reviews.apache.org/r/13000/diff/ Testing --- I updated several of the JUnit tests for the resources and responders to take account of the code changes. All unit tests pass on my local machine (Mac OSX 10.8). Thanks, Ross Laidlaw
Re: Jenkins build is still unstable: oodt-trunk #1030
Hi All, One of my unit tests that has been passing on my local machine (Mac OSX, plus also tested on Ubuntu VM) seems to be failing during the Jenkins build. I'm looking into why this is happening and will commit an update/fix ASAP. Regards, Ross On 25 July 2013 18:27, Apache Jenkins Server jenk...@builds.apache.orgwrote: See https://builds.apache.org/job/oodt-trunk/changes
Re: Jenkins build is back to stable : oodt-trunk #1031
Hi All,
I've removed some asserts from the failing test class
(ZipResponderTest) that use instances of zip4j's FileHeader class. This
seems to have fixed the build for #1031.
I was attempting to use FileHeader instances to examine the contents of zip
files returned in HTTP responses. I'll try to learn more about this class
with a view to adding these tests back in. The failure was happening for
zips of hierarchical products, so there could be an issue with my solution
for that - I'll have a look at a more sophisticated solution to zipping
these up.
I've also made a note of the ubuntu machine used by Jenkins for the failed
builds ('ubuntu3' for builds #1029 and #1030) - I'll try to set up a VM
that resembles this to see if I can reproduce the Jenkins test failure on
my local machine.
Apologies for any inconvenience this may have caused,
Regards,
Ross
On 25 July 2013 20:02, Apache Jenkins Server jenk...@builds.apache.orgwrote:
See https://builds.apache.org/job/oodt-trunk/1031/changes
Re: Review Request 10096: OODT-611: Implement a JAX-RS service for delivery of file manager products in raw data formats (file, zip)
in the ZipResponder class and some of the test classes. I'd like to refactor these after I've implemented the other main tasks for the project: OODT-612 and OODT-613. The attached patch also contains a few minor updates to the CAS-Product POM and original servlets that I spotted while carrying out this work. Diffs (updated) - /trunk/webapp/fmprod/pom.xml 1502765 /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DataUtils.java 1502765 /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/BadRequestException.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/InternalServerErrorException.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/exceptions/NotFoundException.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/DatasetResource.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ProductResource.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/ReferenceResource.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/resources/TransferResource.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/FileResponder.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/NullResponder.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/Responder.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ResponderFactory.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/UnrecognizedFormatResponder.java PRE-CREATION /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/service/responders/ZipResponder.java PRE-CREATION /trunk/webapp/fmprod/src/main/webapp/WEB-INF/web.xml 1502765 /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/DatasetResourceTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ProductResourceTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ReferenceResourceTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/resources/ResourceTestBase.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/FileResponderTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/NullResponderTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ResponderFactoryTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/UnrecognizedFormatResponderTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/java/org/apache/oodt/cas/product/service/responders/ZipResponderTest.java PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/etc/filemgr.properties PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/etc/logging.properties PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/etc/mime-types.xml PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/location.txt PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/location.txt.met PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test.txt PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/flat/test.txt.met PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test.met PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test/file.txt PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/ingest/hierarchical/test/subdirectory/sub-file.txt PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/elements.xml PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/product-type-element-map.xml PRE-CREATION /trunk/webapp/fmprod/src/test/resources/filemgr/policy/core/product-types.xml PRE-CREATION Diff: https://reviews.apache.org/r/10096/diff/ Testing --- I've implemented several unit tests to cover various scenarios. While not comprehensive, these unit tests give some confidence that the basic functionality is working. I've also built the web application and deployed it to a Tomcat 7 web server to test it via the command line (using curl -X GET ...) and with a web browser. Thanks, Ross Laidlaw
Re: intermittent tests
Hi Lewis, I made a commit this morning to the trunk (r1495698) which triggered Jenkins build #984. When I saw the build failure I thought uh oh... have I just broken something? but the changes from the previous build were tiny (CHANGES.txt and a couple of Javadoc edits in the DataDeliveryServlet class, no code changes). I had a quick look through the console output for the build and saw the following (you've probably already seen this before posting your email): Java HotSpot(TM) Server VM warning: Insufficient space for shared memory file: /tmp/hsperfdata_jenkins/29391 Try using the -Djava.io.tmpdir= option to select an alternate temp location. I'm not sure if this was a one-off glitch or whether it also caused some of the other build failures. Ross On 22 Jun 2013, at 17:11, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Good Morning, The intermittent test can be attributed to memory allocation on whichever slave we build oodt trunk on. I am not really comfortable with the fact that the buulds fails like this... I will try my best to look in to this... regards less if it means assigning more mem to the builfd? Any thoughts? All works good for me every day and I do svn up every day Have a great weekend troops Lewis -- *Lewis*
Re: Interested in participating GSoC 2013 with OODT
Congratulations Rajith! Nice work on your project so far and well done for gaining a place on GSoC. I've also been accepted this year and I'd like to echo your comments to say a massive thank you to Chris and Rishi for giving me this opportunity. Looking forward to a great summer! Ross On 28 May 2013 01:55, Rajith Siriwardana rajithsiriward...@ieee.org wrote: Hi all, I have been notified that my proposal was accepted for GSOC 2013. Thank you so much to everybody in the OODT community for your encouragements, kind words and welcoming me to your community. Most importantly thank you very much Chris for deciding my proposal is worth sharing your valuable time during the summer. I'm very happy to have this opportunity to work with you. I'm hoping to have a very productive summer and make this project a success. Thanks Rajith T.W.R.O. Siriwardana StMIEEE Undergraduate Department of Computer Science Engineering Faculty of Engineering University of Moratuwa Sri Lanka. On Tue, Apr 23, 2013 at 9:23 PM, Mattmann, Chris A (398J) chris.a.mattm...@jpl.nasa.gov wrote: Awesome work! ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Rajith Siriwardana rajithsiriward...@ieee.org Date: Tuesday, April 23, 2013 8:38 AM To: dev dev@oodt.apache.org, jpluser chris.a.mattm...@jpl.nasa.gov Subject: Re: Interested in participating GSoC 2013 with OODT Hi Chris, I've submitted a first draft of my GSoC 2013 proposal. Here's a link to the proposal:[1] If you'd like me to make any amendments or additions, please let me know and I'll update it. Many thanks for your valuable time spent on reviewing my proposal in the first place.[1] https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/ra jithsiriwardana/1Cheers, Rajith --- T.W.R.O. Siriwardana StMIEEE Undergraduate Department of Computer Science Engineering Faculty of Engineering University of Moratuwa Sri Lanka. On Sat, Apr 20, 2013 at 11:59 AM, Mattmann, Chris A (398J) chris.a.mattm...@jpl.nasa.gov wrote: Thanks Rajith! ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Rajith Siriwardana rajithsiriward...@gmail.com Reply-To: dev@oodt.apache.org dev@oodt.apache.org Date: Friday, April 12, 2013 10:38 AM To: dev dev@oodt.apache.org Subject: Re: Interested in participating GSoC 2013 with OODT On Fri, Apr 12, 2013 at 10:54 PM, Mattmann, Chris A (398J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Rajith, Thanks, replies below! -Original Message- From: Rajith Siriwardana rajithsiriward...@ieee.org Reply-To: dev@oodt.apache.org dev@oodt.apache.org Date: Friday, April 12, 2013 10:17 AM To: dev dev@oodt.apache.org Subject: Re: Interested in participating GSoC 2013 with OODT Hi Chris, please find the comments inline. [..snip..] Do you know of one? I looked way back in 2005 but there wasn't one back then. Still I haven't found a parser compatible with the Apache license v2.0, but I found some parsers to get the idea of how it should be. I guess it won't be hard. +1, great. - In job status monitoring it will involve jvm monitoring (that is not related to the plugin as it just polling gmetad) correct me if i¹m misunderstood the job concept. Job status is resource manager job status polling. Have you seen the Job class in resource manager? yep I did that, gmetad has nothing to do with polling job status information. Should GangliaAssignmentMonitor worry about obtaining job status information then? am I missing something here? I'm not sure what you mean about it not having anything to do with job polling? gmetad collects metrics published by gmond. Some of those
Re: Review Request: Modified RSSProductTransferServlet to allow setup via XML configuration file (similar to RDF and RSS product servlets)
successfully read from the configuration file and inserted into the output RSS XML. Thanks, Ross Laidlaw
Review Request: Modified RSSProductTransferServlet to allow setup via XML configuration file (similar to RDF and RSS product servlets)
and inserted into the output RSS XML. Thanks, Ross Laidlaw
Google Summer of Code 2013 Proposal - CAS-Product JAX-RS using Apache CXF
Hi All, I'd like to propose a project for GSoC 2013 and request to be considered as a candidate to work on the project. The goal would be to implement a JAX-RS option for the CAS-Product (webapp/fmprod) module of OODT. This would initially sit alongside the existing data, RDF and RSS servlets and would offer a more powerful means for the CAS-Product module to interact with the File Manager. The implementation would use Apache CXF, to complement existing JAX-RS implementations in OODT such as those in PCS and CAS-Curator. I worked on improvements to the CAS-Product RSS servlets last year for GSoC 2012 (e.g. see refs [1] to [8] below). As a PMC member and committer for OODT, I also have experience of submitting patches via JIRA, using Review Board and committing updates to the repository. I therefore feel that I would be a good candidate to work on this project for GSoC 2013. Ross [1] https://issues.apache.org/jira/browse/OODT-402 [2] https://issues.apache.org/jira/browse/OODT-449 [3] https://issues.apache.org/jira/browse/OODT-452 [4] https://issues.apache.org/jira/browse/OODT-453 [5] https://issues.apache.org/jira/browse/OODT-458 [6] https://issues.apache.org/jira/browse/OODT-469 [7] https://issues.apache.org/jira/browse/OODT-471 [8] https://issues.apache.org/jira/browse/OODT-483
Review Request: Fix to prevent readTags method from creating empty source attributes for all tags defined in the RSS configuration file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10095/ --- Review request for oodt and Chris Mattmann. Description --- The readTags method in org.apache.oodt.cas.product.rss.RSSConfigReader assumes that org.w3c.dom.Element.getAttribute(java.lang.String) will return null if a tag does not have a 'source' attribute definition in the RSS configuration file. Instead the getAttribute method returns an empty String. This means that by default readTags will give every tag a blank source attribute. We could use the org.w3c.dom.Element.getAttributeNode(java.lang.String) instead. This method takes the attribute name as a String argument and returns null if the attribute does not exist for the element. With this change, if any tags in the RSS configuration file have an explicitly defined empty source attribute (e.g. tag name=example source=/), readTags will add the empty source attribute to the attribute list for the tag. But if a tag does not have an explicit source attribute definition, readTags will no longer create a blank source attribute for the tag. This addresses bug OODT-582. https://issues.apache.org/jira/browse/OODT-582 Diffs - /trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSConfigReader.java 1460085 Diff: https://reviews.apache.org/r/10095/diff/ Testing --- I implemented a new JUnit test class RSSConfigReaderTest to test for this bug. Please see OODT-583 for details of the test class. Thanks, Ross Laidlaw
Re: Review Request: BufferedReader needs to be closed after use in loadScript method in SqlScript class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9965/ --- (Updated March 23, 2013, 3:03 p.m.) Review request for oodt and Chris Mattmann. Description (updated) --- The loadScript method uses a BufferedReader resource but doesn't close it after use. Since the method declares 'throws IOException', we can use try...finally and close the reader in the finally block. Additionally, it looks like the method deals with files and strings only so we can remove the 'throws SQLException' from the method statement. This addresses bug OODT-576. https://issues.apache.org/jira/browse/OODT-576 Diffs - /trunk/commons/src/main/java/org/apache/oodt/commons/database/SqlScript.java 1457043 Diff: https://reviews.apache.org/r/9965/diff/ Testing --- I ran the unit tests for the commons package. I also ran the TestWorkflowDataSourceRepository tests from the workflow package, as these tests use a setUp method that makes a call to the loadScript method in SqlScript. Thanks, Ross Laidlaw
Review Request: BufferedReader needs to be closed after use in loadScript method in SqlScript class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9965/ --- Review request for oodt and Chris Mattmann. Description --- The loadScript method uses a BufferedReader resource but doesn't close it after use. Since the method declares 'throws IOException', we can use try...finally and close the reader in the finally block. Additionally, it looks like the method deals with files and strings only (no SQL stuff), so we can remove the 'throws SQLException' from the method statement. This addresses bug OODT-576. https://issues.apache.org/jira/browse/OODT-576 Diffs - /trunk/commons/src/main/java/org/apache/oodt/commons/database/SqlScript.java 1457043 Diff: https://reviews.apache.org/r/9965/diff/ Testing --- I ran the unit tests for the commons package. I also ran the TestWorkflowDataSourceRepository tests from the workflow package, as these tests use a setUp method that makes a call to the loadScript method in SqlScript. Thanks, Ross Laidlaw
Review Request: Assignments in the setWorkflowInst and setWaitforConditionSatisfy methods need 'this' keyword
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9820/ --- Review request for oodt and Chris Mattmann. Description --- The assignments in setWorkflowInst and setWaitforConditionSatisfy in the IterativeWorkflowProcessorThread class need a 'this' keyword at the beginning so that values from the parameters are assigned to member variables. This addresses bug OODT-571. https://issues.apache.org/jira/browse/OODT-571 Diffs - /trunk/workflow/src/main/java/org/apache/oodt/cas/workflow/engine/IterativeWorkflowProcessorThread.java 1454129 Diff: https://reviews.apache.org/r/9820/diff/ Testing --- Thanks, Ross Laidlaw
Review Request: Suggested refactoring for return statement in getTopNProducts method in LuceneCatalog class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9844/ --- Review request for oodt and Chris Mattmann. Description --- The getTopNProducts method has a conditional statement with two possible outcomes for return statements. It should be possible to replace this with a single return statement that will still cover all of the same options. This addresses bug OODT-573. https://issues.apache.org/jira/browse/OODT-573 Diffs - /trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/LuceneCatalog.java 1454129 Diff: https://reviews.apache.org/r/9844/diff/ Testing --- After making the changes, I ran the unit tests for the filemgr package. All tests passed. These include tests in the TestLuceneCatalog class, which has two calls to the getTopNProducts method. But the tests don't check for a null value returned from getTopNProducts, so it might be desirable to add another test for this scenario. Thanks, Ross Laidlaw
Re: Review Request: Suggested refactoring for return statement in getTopNProducts method in LuceneCatalog class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9844/ --- (Updated March 11, 2013, 2:45 a.m.) Review request for oodt and Chris Mattmann. Description --- The getTopNProducts method has a conditional statement with two possible outcomes for return statements. It should be possible to replace this with a single return statement that will still cover all of the same options. This addresses bug OODT-573. https://issues.apache.org/jira/browse/OODT-573 Diffs - /trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/LuceneCatalog.java 1454129 Diff: https://reviews.apache.org/r/9844/diff/ Testing --- After making the changes, I ran the unit tests for the filemgr package. All tests passed. These include tests in the TestLuceneCatalog class, which has two calls to the getTopNProducts method. But the tests don't check for a null value returned from getTopNProducts, so it might be desirable to add another test for this scenario. Thanks, Ross Laidlaw
Re: Review Request: Suggested refactoring for return statement in getTopNProducts method in LuceneCatalog class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9844/ --- (Updated March 11, 2013, 2:47 a.m.) Review request for oodt and Chris Mattmann. Description (updated) --- The getTopNProducts method has a conditional statement with two possible outcomes for return statements. It should be possible to replace this with a single return statement that will still cover all of the same outcomes. This addresses bug OODT-573. https://issues.apache.org/jira/browse/OODT-573 Diffs - /trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/catalog/LuceneCatalog.java 1454129 Diff: https://reviews.apache.org/r/9844/diff/ Testing --- After making the changes, I ran the unit tests for the filemgr package. All tests passed. These include tests in the TestLuceneCatalog class, which has two calls to the getTopNProducts method. But the tests don't check for a null value returned from getTopNProducts, so it might be desirable to add another test for this scenario. Thanks, Ross Laidlaw
Re: FW: [GSoC Mentors] Google Summer of Code 2013
Hi Cameron, I forgot to mention (apologies if you already know this) that GSoC is open to many types of student including undergraduates, postgraduates, mature students, full-time and part-time students, etc. Consequently, it might be difficult to devise a project that presents the same level of challenge to every GSoC student. The approach Chris took last year as my mentor was to have many interesting extensions lined up for me if I finished the original project. I really liked this approach and I believe it's a good model for other GSoC projects to follow. Hope this is useful feedback, Ross On 14 February 2013 20:51, Cameron Goodale good...@apache.org wrote: Ross, Thanks for the feedback and I am glad to hear that your GSoC experience was good enough that you are considering doing it again. Is there anything we can do differently to improve this summer? (don't worry you won't hurt my feelings) Cheers, Cameron On Wed, Feb 13, 2013 at 11:23 PM, Ross Laidlaw rlaidlaw.o...@gmail.comwrote: Hi Chris Cam, I'm listening in on the OODT and SIS lists and I've been following along with great interest. My GSoC experience last summer was very positive. I found OODT to be highly accessible because of its component-based structure. This allowed me to focus on one area (File Manager) and start working on my project without having to learn everything upfront about each part of OODT. The approach for my project was to break it down into smaller deliverable steps. I liked this idea as it helped me to gain momentum. I found the OODT and SIS communities to be very knowledgable, welcoming and encouraging. All of these aspects gave me a lot of enthusiasm to contribute and to complete GSoC. I've tried to spread the word about GSoC to my classmates and colleagues. Every year there are tons of applicants and there should be lots of healthy competition again this year for places. I'm still a student and I'm very keen to apply this year if the OODT/SIS communities would be happy for me to contribute again via GSoC. Ross On 13 February 2013 22:09, Cameron Goodale good...@apache.org wrote: Chris, I think it would be a good idea to check in with Ross who was our GSoC student last summer, and ask him for feedback on his experience with OODT. I hope he is listening to the dev list still, but if not we should check-in and see what he is doing now. My hope is that if Ross gives us feedback (good or bad or both) we can work to improve this summer for our next GSoC student. My apologies in advance if you already did this when the GSoC last year was closed out. Just a few questions to get the feedback pump primed would be: Was the mini-project challenging enough? Did the OODT Devs provide enough information and support? Did you encounter anything that was frustrating or unnecessarily difficult? Any thoughts? -Cam chee On Mon, Feb 11, 2013 at 12:24 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: [Sorry for cross posting] Guys, FYI please note that you can participate as a mentor from a PMC via Apache as they are a GSoC org. ComDev will coordinate our participation but start thinking about what projects we may want to do. Cheers, Chris From: Carol Smith car...@google.commailto:car...@google.com Date: Monday, February 11, 2013 11:02 AM To: Google Summer of Code Mentors List google-summer-of-code-mentors-l...@googlegroups.commailto: google-summer-of-code-mentors-l...@googlegroups.com Subject: [GSoC Mentors] Google Summer of Code 2013 Hi GSoC mentors and org admins, We've announced that we're doing Google Summer of Code 2013 [1]. Yay! If you would like to help spread the word about GSoC, we have presentations [2], logos [3], and flyers [4] for you to use. Please host meetups, tell your friends and colleagues about the program, go to conferences, talk to people about the program, and just generally do all the awesome word-of-mouth stuff you do every year to promote the program. The GSoC calendar, FAQ, and events timeline have all been updated with this year's important dates, so please refer to those for the milestones for this year's program. NB: the normal timeline for the program has been modified for this year. You'll probably want to examine the dates closely to make sure you know when important things are happening. Please consider translating the presentations and/or flyers into your native language and submitting them directly to me to post on the wiki. Localization for our material is integral to reaching the widest possible audience around the world. If you decide to translate a flyer, please fill out our form to request a thank you gift for your effort. [5] If you decide to host a meetup, please email me to let me know the date, time, and location so I can put it on the GSoC calendar. Also, remember
Re: [GSoC Mentors] Google Summer of Code 2013
Hi Chris, Many thanks for your kind words of encouragement, much appreciated! I'm looking forward to making more contributions in the near future. For example, I've been working on a solution for OODT-470 and hope to have a proposal ready soon for discussion/review. I'll try to use the usual channels and post to JIRA and the review board when it's ready :) Ross On 15 February 2013 02:32, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Heya Ross, You were an amazing GSoC student last year, and a great member of the OODT PMC and SIS PMC. Please continue to contribute and we would be happy to have you as a GSoC again this summer! Cheers, Chris On 2/13/13 11:23 PM, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Hi Chris Cam, I'm listening in on the OODT and SIS lists and I've been following along with great interest. My GSoC experience last summer was very positive. I found OODT to be highly accessible because of its component-based structure. This allowed me to focus on one area (File Manager) and start working on my project without having to learn everything upfront about each part of OODT. The approach for my project was to break it down into smaller deliverable steps. I liked this idea as it helped me to gain momentum. I found the OODT and SIS communities to be very knowledgable, welcoming and encouraging. All of these aspects gave me a lot of enthusiasm to contribute and to complete GSoC. I've tried to spread the word about GSoC to my classmates and colleagues. Every year there are tons of applicants and there should be lots of healthy competition again this year for places. I'm still a student and I'm very keen to apply this year if the OODT/SIS communities would be happy for me to contribute again via GSoC. Ross On 13 February 2013 22:09, Cameron Goodale good...@apache.org wrote: Chris, I think it would be a good idea to check in with Ross who was our GSoC student last summer, and ask him for feedback on his experience with OODT. I hope he is listening to the dev list still, but if not we should check-in and see what he is doing now. My hope is that if Ross gives us feedback (good or bad or both) we can work to improve this summer for our next GSoC student. My apologies in advance if you already did this when the GSoC last year was closed out. Just a few questions to get the feedback pump primed would be: Was the mini-project challenging enough? Did the OODT Devs provide enough information and support? Did you encounter anything that was frustrating or unnecessarily difficult? Any thoughts? -Cam chee On Mon, Feb 11, 2013 at 12:24 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: [Sorry for cross posting] Guys, FYI please note that you can participate as a mentor from a PMC via Apache as they are a GSoC org. ComDev will coordinate our participation but start thinking about what projects we may want to do. Cheers, Chris From: Carol Smith car...@google.commailto:car...@google.com Date: Monday, February 11, 2013 11:02 AM To: Google Summer of Code Mentors List google-summer-of-code-mentors-l...@googlegroups.commailto: google-summer-of-code-mentors-l...@googlegroups.com Subject: [GSoC Mentors] Google Summer of Code 2013 Hi GSoC mentors and org admins, We've announced that we're doing Google Summer of Code 2013 [1]. Yay! If you would like to help spread the word about GSoC, we have presentations [2], logos [3], and flyers [4] for you to use. Please host meetups, tell your friends and colleagues about the program, go to conferences, talk to people about the program, and just generally do all the awesome word-of-mouth stuff you do every year to promote the program. The GSoC calendar, FAQ, and events timeline have all been updated with this year's important dates, so please refer to those for the milestones for this year's program. NB: the normal timeline for the program has been modified for this year. You'll probably want to examine the dates closely to make sure you know when important things are happening. Please consider translating the presentations and/or flyers into your native language and submitting them directly to me to post on the wiki. Localization for our material is integral to reaching the widest possible audience around the world. If you decide to translate a flyer, please fill out our form to request a thank you gift for your effort. [5] If you decide to host a meetup, please email me to let me know the date, time, and location so I can put it on the GSoC calendar. Also, remember to take pictures at your meetup and write up a blog post for our blog using our provided template for formatting [6]. If you need promotional items for your attendees, please fill out our form [7] to request some; we're happy to send some along. We can provide up to about 25 pens, notebooks, or stickers and/or a few t-shirts. Please keep in mind, though, that shipping
Re: CSS issue with JIRA?
Hi Cam Sheryl, I think JIRA might be down at the moment. I can't access it here in the UK - I get a page saying 'Service Temporarily Unavailable'. Perhaps some maintenance/upgrades going on I guess... Ross On 10 Aug 2012, at 07:11, Sheryl John shery...@gmail.com wrote: Yep it does look like the css files are missing. Didn't get the screenshot. On Thu, Aug 9, 2012 at 11:06 PM, Cameron Goodale good...@apache.org wrote: Hey OODTers, I have been trying to access JIRA and I can, but the css files seem to be missing. Does anyone else see this same problem with: https://issues.apache.org/jira/browse/OODT-328 ? Screenshot of what I see is attached. Thanks. -Cameron -- -Sheryl
Re: Review Request: NumberFormatException when using RSS service to view transfers for large files
=124699632/link pubDateMon, 06 Aug 2012 13:07:32 BST/pubDate sourcefile manager transfers/source cas:sourcefile manager transfers/cas:source cas:bytesTransferred101998592/cas:bytesTransferred cas:totalBytes124699632/cas:totalBytes cas:percentComplete0.8179542342193921/cas:percentComplete /item /channel /rss Thanks, Ross Laidlaw
Review Request: NumberFormatException when using RSS service to view transfers for large files
BST/pubDate sourcefile manager transfers/source cas:sourcefile manager transfers/cas:source cas:bytesTransferred101998592/cas:bytesTransferred cas:totalBytes124699632/cas:totalBytes cas:percentComplete0.8179542342193921/cas:percentComplete /item /channel /rss Thanks, Ross Laidlaw
Re: OODT-470: RSSProductTransferServlet to use XML configuration file
Thanks Chris!
I'll put a patch together and post it to OODT-470 on JIRA for review.
Ross
On 28 July 2012 06:07, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
Hey Ross,
On Jul 26, 2012, at 6:08 AM, Ross Laidlaw wrote:
Dear OODT-ers,
I'm trying to put together a patch for the RSSProductTransferServlet
class [1] for File Manager. At the moment the class has some
hard-coded information and the idea is to make this configurable via
one of the XML configuration files (an example config file is: [2]).
In RSSProductTransferServlet, the 'cas' namespace is hard-coded:
private static final Map NS_MAP = new HashMap();
static { NS_MAP.put(cas, http://oodt.jpl.nasa.gov/1.0/cas;); }
Yep we should definitely make this configurable.
And the following fields within the 'cas' namespace are hard-coded:
XMLUtils.addNode(doc, item, cas:source, file manager transfers);
XMLUtils.addNode(doc, item, cas:bytesTransferred, +
status.getBytesTransferred());
XMLUtils.addNode(doc, item, cas:totalBytes, +
status.getFileRef().getFileSize());
XMLUtils.addNode(doc, item, cas:percentComplete, +
status.computePctTransferred());
If we put the 'cas' namespace definition in the config file, would it
also make sense to move the above field definitions to the config file
as well? If so, should we have a separate config file just for
transfers?
Yep it would be nice to have them generated by the existing available
Product met if they are there and then defined in the conf file.
I was thinking that some of the above fields won't really
apply to the other servlet (RSSProductServlet) as they're related
specifically to transfers (bytes transferred so far, percent complete,
etc), so RSSProductServlet and RSSProductTransferServlet could have
separate configs.
Yep we should probably make the config for RSSProductTransferServlet
specific to it. +1 to that!
Many thanks in advance for your advice,
Thanks Ross!
Cheers,
Chris
Ross
[1]
http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
[2]
http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/rssconf.xml
++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++
OODT-470: RSSProductTransferServlet to use XML configuration file
Dear OODT-ers,
I'm trying to put together a patch for the RSSProductTransferServlet
class [1] for File Manager. At the moment the class has some
hard-coded information and the idea is to make this configurable via
one of the XML configuration files (an example config file is: [2]).
In RSSProductTransferServlet, the 'cas' namespace is hard-coded:
private static final Map NS_MAP = new HashMap();
static { NS_MAP.put(cas, http://oodt.jpl.nasa.gov/1.0/cas;); }
And the following fields within the 'cas' namespace are hard-coded:
XMLUtils.addNode(doc, item, cas:source, file manager transfers);
XMLUtils.addNode(doc, item, cas:bytesTransferred, +
status.getBytesTransferred());
XMLUtils.addNode(doc, item, cas:totalBytes, +
status.getFileRef().getFileSize());
XMLUtils.addNode(doc, item, cas:percentComplete, +
status.computePctTransferred());
If we put the 'cas' namespace definition in the config file, would it
also make sense to move the above field definitions to the config file
as well? If so, should we have a separate config file just for
transfers? I was thinking that some of the above fields won't really
apply to the other servlet (RSSProductServlet) as they're related
specifically to transfers (bytes transferred so far, percent complete,
etc), so RSSProductServlet and RSSProductTransferServlet could have
separate configs.
Many thanks in advance for your advice,
Ross
[1]
http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rss/RSSProductTransferServlet.java
[2]
http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/rssconf.xml
Re: OODT-402 progress update
Hi Chris, I've committed the patches for OODT-449, 452 and 453 and set the JIRA issues to 'resolved' (hope this is ok!). Would you like me to do anything more for the main issue 402? Or if everything is ok, I could close it if all of the basic steps are complete. Regards, Ross On 24 May 2012 12:12, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Thanks Chris! :) I'll go ahead with the commits and will let you know if I have any questions or info to report. On 24 May 2012, at 04:54, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, Patches look good to me, no worries about committing directly. You are now a full PMC member and OODT committer, so use the force :) Review Board is great because me, Ricky, Thomas B., Paul R., Sheryl, BFost, and others are actively leveraging it and love it. If you put your patches there you are bound to get tons of feedback (more so than JIRA atm). But I think you are fine with OODT-449, 452 and 453, we've discussed them plenty and there are no objections so +1 to move forward go for it! Cheers, Chris On May 23, 2012, at 7:11 PM, Ross Laidlaw wrote: Hi Chris, I've uploaded patches for each sub-task for OODT-402 (these are OODT-449, 452 and 453). I noted your comments and I wanted to ask about the next steps. Would you like me to post the patches to review board for a formal review? Or are you satisfied that everything's looking good and covered in JIRA? If everything's fine and you're ok for me to commit the changes I'd be happy to go ahead. Ross On 15 May 2012 19:46, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: +1 for RB! #win! Cheers, Chris On May 15, 2012, at 3:34 AM, Ramirez, Paul M (388J) wrote: Hey Ricky, Sounds good. You can always just implement it as a patch and then when its done commit it to the repo. That would make it easy to push through review board too. --Paul On 5/14/12 9:16 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Ricky, That's awesome, glad that you started those and hope you can dig them up! Cheers, Chris On May 14, 2012, at 5:03 PM, Ross Laidlaw wrote: Hi Ricky, On 14 May 2012 21:26, Nguyen, Ricky rngu...@chla.usc.edu wrote: FYI, I've got some code lying around that ports the DataDeliverServlet and DatasetDeliveryServlet to JAX-RS using CXF. The reason I started it was because I thought we might be able to make use of CXF's oauth providers to secure this RESTful web service. But I hadn't figured out the oauth part (nor how to port the RDF/RSS servlets). If I find time, maybe I'll start a branch... -Ricky Sounds good. I'll try to close out OODT-402 and OODT-458 first, but after that I'd be happy to contribute to this work if there's anything I can do to help out. By the way, I found your emails with Chris and Andrew about the various ways of serving files from File Manager very informative! Really helped me to learn about all of the different services available. Ross ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: OODT SoCal Meetup/Hackathon
I'd like to join in. The Google hangout idea sounds good! I'm free on both the 9th and the 16th. - Ross On 25 May 2012 18:40, Sheryl John shery...@gmail.com wrote: Out of town on the 9th weekend. On Fri, May 25, 2012 at 8:28 AM, Paul Vee paul@gmail.com wrote: Can't do 9th Sent from my iPhone On May 25, 2012, at 8:21 AM, Ramirez, Paul M (388J) paul.m.rami...@jpl.nasa.gov wrote: Hey Guys, So thoughts on June 16th or June 9th? Chris the prospect is your house so either of those days work for you? --Paul On 5/21/12 10:03 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Tom, Awesome, well we will plan on having an OODT meetup in Novemeber that's for sure and I again volunteer my house :) Cheers, Chris On May 21, 2012, at 6:27 AM, Thomas Bennett wrote: Hey, Totally awesome idea. Wish I could be there to join in ;). If this becomes a more regular event - keep me posted. I'll be in the US in November - it would be great way to meet the team. Cheers, Tom On 18 May 2012 07:16, Cameron Goodale good...@apache.org wrote: I think the pool/house idea sounds great. If we start early enough, then we can hack for like 4 - 6 hours, then use the pool as a reward for a job well done. My hope is that we can get some long stretches of time to work on OODT, without email, twitter, IM, or facebook distractions...well the exception being the Google Hangout (of course). -Cam On Thu, May 17, 2012 at 7:06 PM, Ramirez, Paul M (388J) paul.m.rami...@jpl.nasa.gov wrote: Your place sounds good to me. Now we just need to lock down a date. 1st or 2nd Saturday of June work? +1 to Google hangout. Probably best if people do a SVN checkout before we meetup. --Paul On May 17, 2012, at 10:45 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Now *that* is a great idea bfost. We can set up a Google Hangout for this. OK, then I am going to throw out my pool/house for the meetup location, depending on what people think! Cheers, Chris On May 16, 2012, at 11:32 PM, Brian Foster wrote: google+ hangout 4 those out of state! On May 16, 2012, at 11:54 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Paul, This sounds like a great idea and I'm in (of course!) One thing we'll need to do is make sure that we do a really good job of trying to include everyone, even folks that are not in the SoCal area. That means potentially using the Wiki, and other forms of communication (JIRA, ReviewBoard, etc.) to make sure everyone (including those not able to attend) feels included. Great idea! Cheers, Chris On May 15, 2012, at 2:21 PM, Ramirez, Paul M (388J) wrote: Hi All, Anyone interested in doing a meetup/hackathon at the beginning of June. The goal would be to get some updates to our code base but really focused on getting 0.4 out the door. If there is enough interest we would need a place to host it. Anyone and everyone would be welcome as it would be a good way to understand the ways to contribute which go well beyond just code. If enough people seem interested I will work out the details. My initial thought is this would be on a weekend. --Paul ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA
Re: OODT-402 progress update
Thanks Chris! :) I'll go ahead with the commits and will let you know if I have any questions or info to report. On 24 May 2012, at 04:54, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, Patches look good to me, no worries about committing directly. You are now a full PMC member and OODT committer, so use the force :) Review Board is great because me, Ricky, Thomas B., Paul R., Sheryl, BFost, and others are actively leveraging it and love it. If you put your patches there you are bound to get tons of feedback (more so than JIRA atm). But I think you are fine with OODT-449, 452 and 453, we've discussed them plenty and there are no objections so +1 to move forward go for it! Cheers, Chris On May 23, 2012, at 7:11 PM, Ross Laidlaw wrote: Hi Chris, I've uploaded patches for each sub-task for OODT-402 (these are OODT-449, 452 and 453). I noted your comments and I wanted to ask about the next steps. Would you like me to post the patches to review board for a formal review? Or are you satisfied that everything's looking good and covered in JIRA? If everything's fine and you're ok for me to commit the changes I'd be happy to go ahead. Ross On 15 May 2012 19:46, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: +1 for RB! #win! Cheers, Chris On May 15, 2012, at 3:34 AM, Ramirez, Paul M (388J) wrote: Hey Ricky, Sounds good. You can always just implement it as a patch and then when its done commit it to the repo. That would make it easy to push through review board too. --Paul On 5/14/12 9:16 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Ricky, That's awesome, glad that you started those and hope you can dig them up! Cheers, Chris On May 14, 2012, at 5:03 PM, Ross Laidlaw wrote: Hi Ricky, On 14 May 2012 21:26, Nguyen, Ricky rngu...@chla.usc.edu wrote: FYI, I've got some code lying around that ports the DataDeliverServlet and DatasetDeliveryServlet to JAX-RS using CXF. The reason I started it was because I thought we might be able to make use of CXF's oauth providers to secure this RESTful web service. But I hadn't figured out the oauth part (nor how to port the RDF/RSS servlets). If I find time, maybe I'll start a branch... -Ricky Sounds good. I'll try to close out OODT-402 and OODT-458 first, but after that I'd be happy to contribute to this work if there's anything I can do to help out. By the way, I found your emails with Chris and Andrew about the various ways of serving files from File Manager very informative! Really helped me to learn about all of the different services available. Ross ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: OODT-402 progress update
Hi Chris, I've uploaded patches for each sub-task for OODT-402 (these are OODT-449, 452 and 453). I noted your comments and I wanted to ask about the next steps. Would you like me to post the patches to review board for a formal review? Or are you satisfied that everything's looking good and covered in JIRA? If everything's fine and you're ok for me to commit the changes I'd be happy to go ahead. Ross On 15 May 2012 19:46, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: +1 for RB! #win! Cheers, Chris On May 15, 2012, at 3:34 AM, Ramirez, Paul M (388J) wrote: Hey Ricky, Sounds good. You can always just implement it as a patch and then when its done commit it to the repo. That would make it easy to push through review board too. --Paul On 5/14/12 9:16 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Ricky, That's awesome, glad that you started those and hope you can dig them up! Cheers, Chris On May 14, 2012, at 5:03 PM, Ross Laidlaw wrote: Hi Ricky, On 14 May 2012 21:26, Nguyen, Ricky rngu...@chla.usc.edu wrote: FYI, I've got some code lying around that ports the DataDeliverServlet and DatasetDeliveryServlet to JAX-RS using CXF. The reason I started it was because I thought we might be able to make use of CXF's oauth providers to secure this RESTful web service. But I hadn't figured out the oauth part (nor how to port the RDF/RSS servlets). If I find time, maybe I'll start a branch... -Ricky Sounds good. I'll try to close out OODT-402 and OODT-458 first, but after that I'd be happy to contribute to this work if there's anything I can do to help out. By the way, I found your emails with Chris and Andrew about the various ways of serving files from File Manager very informative! Really helped me to learn about all of the different services available. Ross ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
request for new subdirectories in trunk branch
Please could an OODT committer create two subdirectories for me in the trunk branch? The required subdirectories are as follows: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/core http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/geo I need these for work on OODT-452 and OODT-453. As a contributor, I can create these directories locally using 'svn mkdir' and add files to them with 'svn add' or 'svn move', but when I run 'svn diff' to create a patch, the new directories aren't included in the patch. I gather this is a limitation of svn diff (but please let me know if I'm wrong about this!). Many thanks in advance, Ross
Re: request for new subdirectories in trunk branch
Awesome! Thanks Paul :) On 16 May 2012, at 15:24, Ramirez, Paul M (388J) paul.m.rami...@jpl.nasa.gov wrote: Hey Ross, Done. Committed revision 1339183. --Paul On 5/16/12 4:05 AM, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Please could an OODT committer create two subdirectories for me in the trunk branch? The required subdirectories are as follows: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/exam ples/core http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/exam ples/geo I need these for work on OODT-452 and OODT-453. As a contributor, I can create these directories locally using 'svn mkdir' and add files to them with 'svn add' or 'svn move', but when I run 'svn diff' to create a patch, the new directories aren't included in the patch. I gather this is a limitation of svn diff (but please let me know if I'm wrong about this!). Many thanks in advance, Ross
Re: OODT-402 progress update
Hi Chris! On 12 May 2012 18:13, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Yep, +1 to creating a separate task and this sounds like great progress! I've recently created OODT-458 and transferred the namespace info over from OODT-449. I would like to keep the cas namespace since it's really cas config and to me, declaring the namespace there solidified that for me. Is there any specific reason to remove it? I see what you mean and completely agree - no reason to remove it and makes sense to have it there in the config. +1 to make it use RSSConfig in both the ProductServlet and the TransferServlet. I'll try to include this in OODT-458. Also if you're feeling ambitious, a separate issue is to turn those 2 services into JAX-RS services using CXF :) You can find examples of how to do that in CAS curator (see its services) but it's really a separate issue. I just wanted to whet your whistle b/c you are doing so great! :) Definitely! Sounds very interesting! I'd like to contribute to this work. Ricky Nguyen has replied with some info on this, so I'll follow up on his comments as well. Ross
Re: OODT-402 progress update
Hi Ricky, On 14 May 2012 21:26, Nguyen, Ricky rngu...@chla.usc.edu wrote: FYI, I've got some code lying around that ports the DataDeliverServlet and DatasetDeliveryServlet to JAX-RS using CXF. The reason I started it was because I thought we might be able to make use of CXF's oauth providers to secure this RESTful web service. But I hadn't figured out the oauth part (nor how to port the RDF/RSS servlets). If I find time, maybe I'll start a branch... -Ricky Sounds good. I'll try to close out OODT-402 and OODT-458 first, but after that I'd be happy to contribute to this work if there's anything I can do to help out. By the way, I found your emails with Chris and Andrew about the various ways of serving files from File Manager very informative! Really helped me to learn about all of the different services available. Ross
Re: OODT-402 progress update
Hi Chris,
On 11 May 2012 06:01, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
Hey Ross,
Either one would be +1 for me -- if you make it easily configurable, and
changeable,
that would be even better!
Cheers,
Chris
I had a go at implementing the configurable namespaces idea and I have
something that seems to be working! I added a comment to OODT-449
with a draft implementation that processes namespace tags in the RSS
config. If you like, I could move the comment over to a separate task
and upload a patch to it.
With the implementation, it would be possible to add the cas namespace
as well to the RSS config file and this will be picked up in the rss
tag of the output. For example, as follows:
namespace prefix=cas uri=http://oodt.apache.org/components/cas; /
In theory, this means that the cas namespace wouldn't have to be
hard-coded in the RSS java classes. But there might be a couple of
related things to tidy up:
1) Do we still need the cas namespace definition at the top of the RSS
config file? This one is an inline definition for the cas:rssconf tag
itself and not for the output tags:
cas:rssconf xmlns:cas=http://oodt.apache.org/components/cas;
2) Two RSS java classes have hard-coded 'cas' namespace info:
RSSProductServlet.java and RSSProductTransferServlet.java. These
classes both use a class variable to store namespaces and write them
to the output:
private static final Map NS_MAP = new HashMap();
static { NS_MAP.put(cas, http://oodt.jpl.nasa.gov/1.0/cas;); }
...
XMLUtils.addAttribute(doc, rss, xmlns:cas, (String) NS_MAP.get(cas));
I think in RSSProductServlet.java, we could remove this and read it
from the RSS config instead, but how about
RSSProductTransferServlet.java? I'm not sure about that one because
it doesn't appear to access the RSS config file.
Many thanks again for your time and guidance!
Ross
Re: OODT-402 progress update
(Correction to my previous message: I should have written that NS_MAP
is a constant, not a class variable!)
On 11 May 2012, at 11:29, Ross Laidlaw rlaidlaw.o...@gmail.com wrote:
These classes both use a class variable to store namespaces and write them
to the output:
private static final Map NS_MAP = new HashMap();
static { NS_MAP.put(cas, http://oodt.jpl.nasa.gov/1.0/cas;); }
Re: OODT-402 progress update
... or maybe it could be: cas:rssconf xmlns:cas=http://oodt.apache.org/components/cas; On 11 May 2012 04:38, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Hi Chris, I noticed the following commented code in the rssconf.xml file for the fmprod webapp: !-- FIXME: adjust namespace URI? -- cas:rssconf xmlns:cas=http://oodt.jpl.nasa.gov/1.0/cas; Since the work on OODT-402 is based on that file, would it be ok if I include an update for the URI in the patch? Would the following be correct? cas:rssconf xmlns:cas=http://oodt.apache.org/1.0/cas; I'd also like to use the updated URI for the georss-config.xml as well. Ross On 28 April 2012 17:24, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Anytime, Ross! Cheers, Chris On Apr 27, 2012, at 9:51 AM, Ross Laidlaw wrote: Hi Chris! Thanks for responding so quickly and for the guidance. I'll set up the JIRA issues and add some specific info and questions to each item. Ross On 26 April 2012 16:25, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, No problem at all! Read on below and keep the questions coming! On Apr 26, 2012, at 5:27 AM, Ross Laidlaw wrote: Hi Chris, I'm planning my next steps for working on OODT-402. The first task is to create a default GeoRSS config in the File Manager RSS service. Do you have a preference for where this file should be stored and accessed? Yep. I think we should ship it with the File Manager inside of: http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/georss-config.xml As a first step, in the demo that I set up recently I used the default 'rssconfig.xml' file for the fmprod webapp and followed your comments to add latitude and longitude tags to this. Yep in the georss-config.xml I would copy through the default RSS and config and then just expose lat/lon with the comments/documentation. This could be a JIRA issue in itself. For the 'LocationAwareProduct' definition, I was wondering if we could make this a virtual type that could then be inherited by concrete vendor types. +1, this sounds like a good strategy, and I would do it as a full specced product-types.xml, elements.xml and product-type-element-map.xml combination. A home for it could be: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/geo As part of this, we should move the current contents of: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/ To: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/core This would be 2 separate JIRA issues (for default geo policy, and then a separate to move core, and make sure assembly.xml still works, and the unit tests, etc.) Just a few questions and thoughts to get going! Apologies if I'm off track... HTH and rock on! Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: OODT-402 progress update
Cool, thanks guys! I'll include it in the patch for 449 (sub-task of 402) On 11 May 2012 05:16, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: +1, sounds great Ross. Cheers, Chris On 11 May 2012 05:11, Ramirez, Paul M (388J) paul.m.rami...@jpl.nasa.gov wrote: Hey Ross, Yep that should be changed. --Paul Sent from my iPhone On May 10, 2012, at 8:38 PM, Ross Laidlaw wrote: Hi Chris, I noticed the following commented code in the rssconf.xml file for the fmprod webapp: !-- FIXME: adjust namespace URI? -- cas:rssconf xmlns:cas=http://oodt.jpl.nasa.gov/1.0/cas; Since the work on OODT-402 is based on that file, would it be ok if I include an update for the URI in the patch? Would the following be correct? cas:rssconf xmlns:cas=http://oodt.apache.org/1.0/cas; I'd also like to use the updated URI for the georss-config.xml as well. Ross On 28 April 2012 17:24, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Anytime, Ross! Cheers, Chris On Apr 27, 2012, at 9:51 AM, Ross Laidlaw wrote: Hi Chris! Thanks for responding so quickly and for the guidance. I'll set up the JIRA issues and add some specific info and questions to each item. Ross On 26 April 2012 16:25, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, No problem at all! Read on below and keep the questions coming! On Apr 26, 2012, at 5:27 AM, Ross Laidlaw wrote: Hi Chris, I'm planning my next steps for working on OODT-402. The first task is to create a default GeoRSS config in the File Manager RSS service. Do you have a preference for where this file should be stored and accessed? Yep. I think we should ship it with the File Manager inside of: http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/georss-config.xml As a first step, in the demo that I set up recently I used the default 'rssconfig.xml' file for the fmprod webapp and followed your comments to add latitude and longitude tags to this. Yep in the georss-config.xml I would copy through the default RSS and config and then just expose lat/lon with the comments/documentation. This could be a JIRA issue in itself. For the 'LocationAwareProduct' definition, I was wondering if we could make this a virtual type that could then be inherited by concrete vendor types. +1, this sounds like a good strategy, and I would do it as a full specced product-types.xml, elements.xml and product-type-element-map.xml combination. A home for it could be: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/geo As part of this, we should move the current contents of: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/ To: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/core This would be 2 separate JIRA issues (for default geo policy, and then a separate to move core, and make sure assembly.xml still works, and the unit tests, etc.) Just a few questions and thoughts to get going! Apologies if I'm off track... HTH and rock on! Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: OODT-402 progress update
Hi Chris! Thanks for responding so quickly and for the guidance. I'll set up the JIRA issues and add some specific info and questions to each item. Ross On 26 April 2012 16:25, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, No problem at all! Read on below and keep the questions coming! On Apr 26, 2012, at 5:27 AM, Ross Laidlaw wrote: Hi Chris, I'm planning my next steps for working on OODT-402. The first task is to create a default GeoRSS config in the File Manager RSS service. Do you have a preference for where this file should be stored and accessed? Yep. I think we should ship it with the File Manager inside of: http://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/georss-config.xml As a first step, in the demo that I set up recently I used the default 'rssconfig.xml' file for the fmprod webapp and followed your comments to add latitude and longitude tags to this. Yep in the georss-config.xml I would copy through the default RSS and config and then just expose lat/lon with the comments/documentation. This could be a JIRA issue in itself. For the 'LocationAwareProduct' definition, I was wondering if we could make this a virtual type that could then be inherited by concrete vendor types. +1, this sounds like a good strategy, and I would do it as a full specced product-types.xml, elements.xml and product-type-element-map.xml combination. A home for it could be: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/geo As part of this, we should move the current contents of: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/ To: http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examples/core This would be 2 separate JIRA issues (for default geo policy, and then a separate to move core, and make sure assembly.xml still works, and the unit tests, etc.) Just a few questions and thoughts to get going! Apologies if I'm off track... HTH and rock on! Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: [ANNOUNCE] GSoC: Ross Laidlaw, way to be man!
Superb! Thanks Paul! I really appreciate your advice and look forward to working with you :) On 24 April 2012 16:21, Ramirez, Paul M (388J) paul.m.rami...@jpl.nasa.gov wrote: Hey Ross, Unfortunately, I missed the opportunity to click yes on officially participating to be a mentor. That said, I'm more than happy to mentor unofficially =). Thanks, Paul Ramirez On 4/23/12 1:21 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Folks, Ross Laidlaw's project was accepted for Google Summer of Code 2012: http://www.google-melange.com/gsoc/project/google/gsoc2012/rlaidlaw/11001 The Apache OODT and SIS communities sincerely look forward to your continued contributions and excellent project proposals, Ross! Cheers, Chris (Ross's GSoC 2012 Mentor) ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: Google Summer of Code 2012
Hi Chris, I've submitted a first draft of my GSoC 2012 proposal to the GSoC website. Here's a link to the proposal: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/rlaidlaw/1 If you have time, please could you review it and let me know if it's up to scratch? If you'd like me to make any amendments or additions, please let me know and I'll update it. Many thanks, Ross On 17 March 2012 13:10, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Hi Chris, Many thanks for your reply. That's great news! I'm really keen to work on this project and very happy that you will be mentoring. I'll try to update you regularly on my progress and I'll definitely post any questions to the mailing lists, as requested. Ross On 16 March 2012 14:46, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, Awesome, that sounds great! I saw your message on OODT-402 and responded there as well. I will be happy to mentor this issue and am looking forward to you contributing to GSoC and more broadly, to Apache OODT as a whole! Cheers, Chris On Mar 16, 2012, at 5:23 AM, Ross Laidlaw wrote: Hi Chris, I'm a student at Oxford University in England, studying for a masters degree in Software Engineering. I'm very interested in the OODT project and I'd like to get involved. I'm hoping to participate in Google Summer of Code 2012. This would be an ideal opportunity for me to deliver a package of work within a set period of time and hopefully demonstrate my abilities to the OODT team. After that, if the OODT team feel that my work is satisfactory and would like me to contribute more, I would very much like to continue contributing to the project beyond the end of GSoC. Therefore, please could I put myself forward as a candidate to deliver OODT-402 for GSoC 2012? I've started reading through the documentation for OODT, in particular the CAS FileManager user guide and developer guide, with the aim of producing a draft proposal for GSoC. Apologies if you received this message twice - I posted a similar message to the OODT-402 page in JIRA (https://issues.apache.org/jira/browse/OODT-402). Many thanks for your time, Ross On Sun, 04 Mar 2012 17:23:01 GMT, Chris Mattmann wrote: Hey Guys, FYI here: https://issues.apache.org/jira/browse/OODT-402 I just wrote up what I think would be a cool Google Summer of Code project. I'll forward along the GSoC email information. If you have great OODT ideas that you'd like to see turned into a GSoC project, and you have the time to mentor a student, we have a shot at it being selected. So, bring em' on! Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: OODT-402 progress update
Hi Chris! On 3 April 2012 04:21, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, I checked it out. Did I mention you are amazing??! :) Keep up the awesome work! Thanks Chris! I'm glad things are going well. This is a really great community and I'm very grateful to have the opportunity to contribute. I will see if I can modify the confluence wiki to let you upload images. It looks like it is set to allow logged in users to add attachments -- is it a different permission than that? If so let me know and I or someone else can update it. These sound like the correct permissions. I'm new to confluence so it's probably something on my side that I haven't figured out yet! When I select 'Tools - Attachments' I can view attached files but I couldn't see any options to browse for files to add new attachments. Apologies if it's something obvious that I missed. Ross
Re: OODT-402 progress update
Hi Tom!
Thanks for your encouraging words. It means a lot to me that the OODT
team are happy with my work so far. I hope I can deliver something
useful that will be of benefit to you guys.
Regards,
Ross
On 2 April 2012 07:24, Thomas Bennett lmzxq@gmail.com wrote:
Hey Ross,
I also wanted to say great work so far, looks very promising!
Regards,
Tom
On 1 April 2012 18:59, Cameron Goodale good...@apache.org wrote:
Ross,
I agree with Chris that you are doing a great job on this. Thank you for
all the hard work in setting this up. Having a standard geospatial
interface from the FileManager to SIS will prove extremely helpful to
several JPL/NASA Earth Science Missions in the future.
Just wanted to say thanks!
Cameron
On Sat, Mar 31, 2012 at 2:47 PM, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
(sorry for the cross post, but of interest to both communities)
Hey Ross,
That is awesome to hear! Comments below:
On Mar 31, 2012, at 4:04 AM, Ross Laidlaw wrote:
Hi Chris,
Many thanks for your comments and advice. I have now installed SIS on
my system and I have the demo up and running. I've also been
experimenting with the connection between File Manager and SIS and I
have a basic connection working. Here are the steps I have carried so
far:
Firstly, I added elements for 'Latitude' and 'Longitude' to the
GenericFile policy. Then I created a set of test files (simple text
files) with corresponding '.met' metadata files. I added 'Latitude'
and 'Longitude' metadata to the '.met' files. I then ingested these
into my File Manager repository.
That's perfect.
After that, I enabled the 'georss' tags in the 'rssconf.xml'
configuration file (I added tags for geo:lat and geo:long to start
with) and viewed the RSS output from the CAS REST API in my browser.
I then added the URL
('http://localhost:8080/fmprod/viewRecent?channel=ALL') as an RSS feed
to the 'sis-location-config.xml' config in my SIS webapp.
Next, I ran a bounding box query from the SIS demo.jsp and my file
manager files appeared on the map! Awesome!
That is SWEET!
I have uploaded a set of screenshots showing my progress, hopefully
they're accessible and make sense (I'm new to flickr, so apologies if
things look a bit disorganised!):
http://www.flickr.com/photos/rlaidlaw/sets/72157629342438940/with/7031581313/
They look great, I checked them out. Would you be willing to do the
following?
1. Log onto the SIS wiki and add a little tutorial (small) based on the
above email
and screenshots?
wiki here: https://cwiki.apache.org/confluence/display/SIS/Home
2. Link the SIS tutorial via the OODT wiki, here:
https://cwiki.apache.org/confluence/display/OODT/Home
I tried to order the screenshots to follow the steps I outlined above,
with the final screenshot being the output from the SIS demo showing
the File Manager files on the map.
That's awesome.
For next steps, I could start to formalize some of my experimental
code and begin to address the key points you have outlined on the
OODT-402 jira page (i.e. writing a default GeoRSS config and a default
file manager policy for a 'LocationAwareProduct' type).
That would be perfect!
I will follow
up with specific questions about both of these, but for now I just
wanted to give you an update on my progress and to check that I'm
heading in the right direction.
You are definitely heading in the right direction that's precisely what I
was
hoping you would figure out how to do. Thank you Ross for the update
and your results look VERY promising, great job so far!
Cheers,
Chris
++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++
Re: OODT-402 progress update
Hi Chris, definitely, I'd be happy to do that. I've started work on it - hoping to have a draft up by end of today (April 2nd). Will send you a message as soon as I've uploaded it :) On 31 March 2012 22:47, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: They look great, I checked them out. Would you be willing to do the following? 1. Log onto the SIS wiki and add a little tutorial (small) based on the above email and screenshots? wiki here: https://cwiki.apache.org/confluence/display/SIS/Home 2. Link the SIS tutorial via the OODT wiki, here: https://cwiki.apache.org/confluence/display/OODT/Home
Re: OODT-402 progress update
Hi Chris, I've uploaded a first draft for the tutorial on the SIS wiki and linked to it from the OODT and SIS home pages as requested. I didn't have permissions to attach images directly to the wiki page, so I added the screenshot to OODT-402 first and then transferred it over from there. Apologies if this caused a lot of notifications. As I make progress on OODT-402 I'll keep updating the tutorial to reflect the latest work. Ross On 2 April 2012 13:11, Ross Laidlaw rlaidlaw.o...@gmail.com wrote: Hi Chris, definitely, I'd be happy to do that. I've started work on it - hoping to have a draft up by end of today (April 2nd). Will send you a message as soon as I've uploaded it :) On 31 March 2012 22:47, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: They look great, I checked them out. Would you be willing to do the following? 1. Log onto the SIS wiki and add a little tutorial (small) based on the above email and screenshots? wiki here: https://cwiki.apache.org/confluence/display/SIS/Home 2. Link the SIS tutorial via the OODT wiki, here: https://cwiki.apache.org/confluence/display/OODT/Home
Re: OODT-402 progress update
Hi Chris Cameron,
Thank you for your kind words. I'm honoured to have the opportunity
to contribute to these projects and to work with and learn from superb
mentors. I very much like the fact that both OODT and SIS are
developed and used by NASA. It will be a great feeling and sense of
achievement for me if I can make a positive contribution and therefore
benefit these projects.
Regards,
Ross
On 1 April 2012 17:59, Cameron Goodale good...@apache.org wrote:
Ross,
I agree with Chris that you are doing a great job on this. Thank you for
all the hard work in setting this up. Having a standard geospatial
interface from the FileManager to SIS will prove extremely helpful to
several JPL/NASA Earth Science Missions in the future.
Just wanted to say thanks!
Cameron
On Sat, Mar 31, 2012 at 2:47 PM, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
(sorry for the cross post, but of interest to both communities)
Hey Ross,
That is awesome to hear! Comments below:
On Mar 31, 2012, at 4:04 AM, Ross Laidlaw wrote:
Hi Chris,
Many thanks for your comments and advice. I have now installed SIS on
my system and I have the demo up and running. I've also been
experimenting with the connection between File Manager and SIS and I
have a basic connection working. Here are the steps I have carried so
far:
Firstly, I added elements for 'Latitude' and 'Longitude' to the
GenericFile policy. Then I created a set of test files (simple text
files) with corresponding '.met' metadata files. I added 'Latitude'
and 'Longitude' metadata to the '.met' files. I then ingested these
into my File Manager repository.
That's perfect.
After that, I enabled the 'georss' tags in the 'rssconf.xml'
configuration file (I added tags for geo:lat and geo:long to start
with) and viewed the RSS output from the CAS REST API in my browser.
I then added the URL
('http://localhost:8080/fmprod/viewRecent?channel=ALL') as an RSS feed
to the 'sis-location-config.xml' config in my SIS webapp.
Next, I ran a bounding box query from the SIS demo.jsp and my file
manager files appeared on the map! Awesome!
That is SWEET!
I have uploaded a set of screenshots showing my progress, hopefully
they're accessible and make sense (I'm new to flickr, so apologies if
things look a bit disorganised!):
http://www.flickr.com/photos/rlaidlaw/sets/72157629342438940/with/7031581313/
They look great, I checked them out. Would you be willing to do the
following?
1. Log onto the SIS wiki and add a little tutorial (small) based on the
above email
and screenshots?
wiki here: https://cwiki.apache.org/confluence/display/SIS/Home
2. Link the SIS tutorial via the OODT wiki, here:
https://cwiki.apache.org/confluence/display/OODT/Home
I tried to order the screenshots to follow the steps I outlined above,
with the final screenshot being the output from the SIS demo showing
the File Manager files on the map.
That's awesome.
For next steps, I could start to formalize some of my experimental
code and begin to address the key points you have outlined on the
OODT-402 jira page (i.e. writing a default GeoRSS config and a default
file manager policy for a 'LocationAwareProduct' type).
That would be perfect!
I will follow
up with specific questions about both of these, but for now I just
wanted to give you an update on my progress and to check that I'm
heading in the right direction.
You are definitely heading in the right direction that's precisely what I
was
hoping you would figure out how to do. Thank you Ross for the update
and your results look VERY promising, great job so far!
Cheers,
Chris
++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++
Re: OODT-402 progress update
Hi Chris,
Many thanks for your comments and advice. I have now installed SIS on
my system and I have the demo up and running. I've also been
experimenting with the connection between File Manager and SIS and I
have a basic connection working. Here are the steps I have carried so
far:
Firstly, I added elements for 'Latitude' and 'Longitude' to the
GenericFile policy. Then I created a set of test files (simple text
files) with corresponding '.met' metadata files. I added 'Latitude'
and 'Longitude' metadata to the '.met' files. I then ingested these
into my File Manager repository.
After that, I enabled the 'georss' tags in the 'rssconf.xml'
configuration file (I added tags for geo:lat and geo:long to start
with) and viewed the RSS output from the CAS REST API in my browser.
I then added the URL
('http://localhost:8080/fmprod/viewRecent?channel=ALL') as an RSS feed
to the 'sis-location-config.xml' config in my SIS webapp.
Next, I ran a bounding box query from the SIS demo.jsp and my file
manager files appeared on the map! Awesome!
I have uploaded a set of screenshots showing my progress, hopefully
they're accessible and make sense (I'm new to flickr, so apologies if
things look a bit disorganised!):
http://www.flickr.com/photos/rlaidlaw/sets/72157629342438940/with/7031581313/
I tried to order the screenshots to follow the steps I outlined above,
with the final screenshot being the output from the SIS demo showing
the File Manager files on the map.
For next steps, I could start to formalize some of my experimental
code and begin to address the key points you have outlined on the
OODT-402 jira page (i.e. writing a default GeoRSS config and a default
file manager policy for a 'LocationAwareProduct' type). I will follow
up with specific questions about both of these, but for now I just
wanted to give you an update on my progress and to check that I'm
heading in the right direction.
Speak to you soon,
Ross
On 22 March 2012 05:48, Mattmann, Chris A (388J)
chris.a.mattm...@jpl.nasa.gov wrote:
Hi Ross,
On Mar 22, 2012, at 5:34 AM, Ross Laidlaw wrote:
Hi Chris,
I'm writing to give an update on my progress on OODT-402 and getting
up to speed with the OODT project.
I've been reading through the documentation from
http://oodt.apache.org/components/maven/filemgr and the wiki pages
http://cwiki.apache.org/oodt. I've also read up on the GeoRSS format
at http://georss.org and http://en.wikipedia.org/wiki/GeoRSS.
Awesome, that's the right place to start!
By following the user guides (which are awesome by the way!)
* thanks * Many folks have been really updating those lately (Tom Bennett,
Cameron
Goodale, Sheryl John, and others) so thanks to them for that!
I have
built OODT with maven v2.2.1 on my laptop (macbook running OSX Lion)
and set up the File Manager program (I've ingested my first 'blah.txt'
file with meta data! And also tested out some queries).
You ROCK, nice!
As a next step, I'd like to install the File Manager webapp. Are
there any recent guides for installing? I watched both of your Apache
OODT Curation tutorials on YouTube (very cool) and also found some
info about webapps from one of your replies on the
u...@oodt.apache.org mailing list (with link to slides
http://oodt.apache.org/components/maven/grid/slides.pdf). I have my
webapp target folders and war files from the build and I have Tomcat
set up on my machine, so hopefully I'm not far off being up and
running! Apologies if this is something obvious that I should know
already (e.g. by learning more about Tomcat).
Great, that's the right starting point.
I built the 0.3 release of OODT on my machine. But I also had a look
through the latest webapp source code and noted your comments in the
following file:
https://svn.apache.org/repos/asf/oodt/trunk/webapp/fmprod/src/main/resources/rssconf.xml
!--
If you want to turn your CAS File Manager into a GeoRSS
simple service (see http://georss.org/simple), add the
following block, assuming that you have indexed met fields
named Latitude and Longitude for your product (update as
appropriate otherwise).
tag name=georss:point source=[Latitude] [Longitude]/
--
Yep, the above is assuming that you index 2 met fields along with your
products, the Latitude met field, and the Longitude. If you indexed them
differently (e.g., a field named geolat or geolng, then you would update
the above accordingly).
Am I right in thinking that this is the starting point for OODT-402?
Absolutely that's the right starting point for getting GeoRSS out of the
File Manager, then the next step is to move onto SIS and read about its
GeoRSS data loader and Quad Tree. Admittedly over in SIS ville there's
a lot less documentation, but it would be great because there is also
as an Incubator project a lot more ability to easily contribute and get
started
being part of the community.
I saw from georss.org
Re: Google Summer of Code 2012
Hi Chris, Many thanks for your reply. That's great news! I'm really keen to work on this project and very happy that you will be mentoring. I'll try to update you regularly on my progress and I'll definitely post any questions to the mailing lists, as requested. Ross On 16 March 2012 14:46, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Ross, Awesome, that sounds great! I saw your message on OODT-402 and responded there as well. I will be happy to mentor this issue and am looking forward to you contributing to GSoC and more broadly, to Apache OODT as a whole! Cheers, Chris On Mar 16, 2012, at 5:23 AM, Ross Laidlaw wrote: Hi Chris, I'm a student at Oxford University in England, studying for a masters degree in Software Engineering. I'm very interested in the OODT project and I'd like to get involved. I'm hoping to participate in Google Summer of Code 2012. This would be an ideal opportunity for me to deliver a package of work within a set period of time and hopefully demonstrate my abilities to the OODT team. After that, if the OODT team feel that my work is satisfactory and would like me to contribute more, I would very much like to continue contributing to the project beyond the end of GSoC. Therefore, please could I put myself forward as a candidate to deliver OODT-402 for GSoC 2012? I've started reading through the documentation for OODT, in particular the CAS FileManager user guide and developer guide, with the aim of producing a draft proposal for GSoC. Apologies if you received this message twice - I posted a similar message to the OODT-402 page in JIRA (https://issues.apache.org/jira/browse/OODT-402). Many thanks for your time, Ross On Sun, 04 Mar 2012 17:23:01 GMT, Chris Mattmann wrote: Hey Guys, FYI here: https://issues.apache.org/jira/browse/OODT-402 I just wrote up what I think would be a cool Google Summer of Code project. I'll forward along the GSoC email information. If you have great OODT ideas that you'd like to see turned into a GSoC project, and you have the time to mentor a student, we have a shot at it being selected. So, bring em' on! Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
