Re: Apache Airavata-OODT Integration
Hi Sanjaya, -Original Message- From: Sanjaya Medonsa Reply-To: "d...@airavata.apache.org" Date: Monday, June 10, 2013 5:20 PM To: "d...@airavata.apache.org" Cc: "dev@oodt.apache.org" Subject: Re: Apache Airavata-OODT Integration >Hi Chris, > On configuration, I have get rid of all the configuration files, >including pge-config.xml. All the required configurations are >programmatically set. Configurations such FileManagerServer URL are >configured in the airavata-server.properties file. I'll update the review >request with modified details. Great work! > Still I am not quite clear on how to retrieve staged file path >properly. Currently I am using getStagedFilePath method >in ApacheAiravataWorkFlowInstanceImpl to regenerate the staged file path. >While I am going through the OODT code that I have seen method in >DataTransferer to notify FileManagerServer once transfer is completed. But >I couldn't see the same for product retrieval. Example: http://svn.apache.org/repos/asf/oodt/trunk/pge/src/test/resources/pge-confi g.xml Review Board tickets: https://reviews.apache.org/r/4746/ https://reviews.apache.org/r/5382/ JIRA issue source (in OODT since 0.4): https://issues.apache.org/jira/browse/OODT-443 > As you suggested I'll improve my workflow using Apache Tika. I'd >like to continue this as an Parallal task. While modifying staging >implementation based on community feedback, currently I am looking at >ingesting output back to OODT. See above for info on file staging. I would strongly encourage you not to reimplement CAS-PGE in Airavata -- it's pretty functional and expressive anyways and I would work to figure out how to make Airavata leverage CAS-PGE. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ > > > >On Wed, Jun 5, 2013 at 12:11 AM, Mattmann, Chris A (398J) < >chris.a.mattm...@jpl.nasa.gov> wrote: > >> Hi Sanjaya, >> >> I think starting out with /bin/ls would be good, maybe like a /bin/ls >> workflow, and then for each file returned, maybe run Apache Tika and >> extract its metadata and then pipe that to a file? >> >> How about that? >> >> Cheers, >> Chris >> >> ++ >> Chris Mattmann, Ph.D. >> Senior Computer Scientist >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 171-266B, Mailstop: 171-246 >> Email: chris.a.mattm...@nasa.gov >> WWW: http://sunset.usc.edu/~mattmann/ >> ++ >> Adjunct Assistant Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++ >> >> >> >> >> >> >> -Original Message- >> From: Sanjaya Medonsa >> Reply-To: "d...@airavata.apache.org" >> Date: Tuesday, June 4, 2013 5:31 AM >> To: "d...@airavata.apache.org" >> Cc: "dev@oodt.apache.org" >> Subject: Re: Apache Airavata-OODT Integration >> >> >Hi Chris, >> > Please see my comments below on the two items. >> > >> >Configuration : It should be possible to set them programmatically. >> >Actually I have implemented partly it for file staging information. >>I'll >> >work to get rid of the other configuration files. >> > >> >Staged File Path : I'll work on the suggested approach, though I am not >> >fully understand it at the moment. I guess I need to go through bit >>more >> >on >> >CAS-PGE and come back to you on the proposed approach. >> > >> >Currently I am testing this by wrapping /bin/ls command as GFac >>service. I >> >may need to test this with real workflow. Could you please provide me >>know >> >some guidance on better scenario to test this. >> > >> >Cheers, >> >Sanjaya >> > >> > >> > >> > >> >On Mon, Jun 3, 2013 at 8:17 PM, Mattmann, Chris A (398J) < >> >chris.a.mattm...@jpl.nasa.gov> wrote: >> > >> >> Hi Sanjaya, >> >> >> >> -Original Message- >> >> >> >> From: Sanjaya Medonsa >> >> Reply-To: "d...@airavata.apache.org" >> >> Date: Thursday, May 30, 2013 5:12 AM >> >> To: "dev@oodt.apache.org" , >> >>"d...@airavata.apache.org" >> >> >> >> Subject: Apache Airavata-OODT Integration >> >> >> >> >Hi, >> >> > I have worked on the Apache Airavata integration with Apache >> >>OODT. As >> >> >a first step, I have implemented integration with Apache OODT file >> >> >manager component. >> >> >> >> Great work!! >> >> >> >> Comments below: >> >> >> >> > 1. Introduce a new GFac Schema type called OODTProduct which
Re: Apache Airavata-OODT Integration
+5000 great idea, as usual my friend. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Suresh Marru Reply-To: "d...@airavata.apache.org" Date: Wednesday, June 12, 2013 9:51 AM To: "d...@airavata.apache.org" Cc: "dev@oodt.apache.org" Subject: Re: Apache Airavata-OODT Integration >On Jun 12, 2013, at 12:34 PM, "Mattmann, Chris A (398J)" > wrote: > >> See above for info on file staging. I would strongly encourage you not >> to reimplement CAS-PGE in Airavata -- it's pretty functional and >>expressive >> anyways and I would work to figure out how to make Airavata leverage >> CAS-PGE. > >+ 1. > >Sanjaya, Airavata and OODT communities, > >Any volunteers to write a paper on "A tale of two apache workflow >systems: Airavata and OODT"? > >Given the page limit and to keep in scope, I suggest to leave out the use >cases of the systems and focus on software architectures. A detailed >technical paper comparing and contrasting the features and identifying >potential collaborative components. > >If you want a deadline, how about August 15th to WORKS workshop - >http://works.cs.cardiff.ac.uk/ > >Suresh
Re: Apache Airavata-OODT Integration
On Jun 12, 2013, at 12:34 PM, "Mattmann, Chris A (398J)" wrote: > See above for info on file staging. I would strongly encourage you not > to reimplement CAS-PGE in Airavata -- it's pretty functional and expressive > anyways and I would work to figure out how to make Airavata leverage > CAS-PGE. + 1. Sanjaya, Airavata and OODT communities, Any volunteers to write a paper on "A tale of two apache workflow systems: Airavata and OODT"? Given the page limit and to keep in scope, I suggest to leave out the use cases of the systems and focus on software architectures. A detailed technical paper comparing and contrasting the features and identifying potential collaborative components. If you want a deadline, how about August 15th to WORKS workshop - http://works.cs.cardiff.ac.uk/ Suresh
Re: Getting started with issues
Hi Guys, Having deliberated where best my skills lie to help OODT, I suspect I'll be best served in the webapp/RESTful side of stuff(excluding PHP), so I've been poking around Curator related issues. My knowledge of the stack is limited so if I should be looking at other components let me know, if not, when I get back from Austria this week, I'll start picking open issues from the Curator list and get started. If there are any specific pressing issues let me know and I'll take a look first, otherwise, its pot luck ;) Cheers Tom On 07/06/13 02:17, Mattmann, Chris A (398J) wrote: Hey Tom, Great to have you working issues on the list! We talked about classifying them along a scale (will dig up thread from Mike Joyce and Gav from earlier) but I don't think we've done too much yet in regards to curating the data for them. However if you find some you are interested in, let me or anyone on the list know and we can provide more information about them. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ -Original Message- From: Tom Barber Reply-To: "dev@oodt.apache.org" Date: Wednesday, June 5, 2013 1:03 PM To: "dev@oodt.apache.org" Subject: Getting started with issues Hi guys, I had a chat with Chris the other week and said I'd be interested in helping out fixing bugs and stuff as a part time Java & Scala dev. Obviously I don't want to work on stuff that other people are doing, so what's the best way to get started picking (easy) tasks. Thanks Tom
Re: Review Request: Apache Airvata extension to integrate Apache OODT File manager using CAS-PGE
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11565/ --- (Updated June 12, 2013, 1:12 a.m.) Review request for Airavata, oodt, Chris Mattmann, Suresh Srinivas, and Shahani Weerawarana. Changes --- Added Airavata into review group Description --- I have worked on the Apache Airavata integration with Apache OODT. As a first step, I have implemented integration with Apache OODT file manager component. 1. Introduce a new GFac Schema type called OODTProduct which takes APache OODT product IDs as input. 2. Implemented new pre GFac Handler by extending Apache OODT PgeTaskInstance to stage the corresponding file into the working directory. 3. Once file is staged, input parameter with OODT product id is replaced with path of the staged file for downstream processing I have tested the implementation with Gfac application which wraps /bin/ls command. Application takes product id as input and stage corresponding file into the working directory and /bin/ls is executed against the staged file. Hope this is a valid testing scenario. Concerns - Configurations : I have added new configuration file named and oodt-integration.properties in addition to dynamic_metadata.met and pge-config.xml files used by OODT. But at the moment there is no item configured with the oodt-integration.properties. - Staged File Name - With the current implementation of PgeTaskInstance it is not possible to retrieve path of the staged file. Due to this limitation, I have query the FileManagerServer with product id and retrieve the file name and computed the file path using information of working directory. - Currently it is not possible to execute the workflow using Xbaya due to validation failure due to new schema type. I have commented out the relevant validation code for testing purpose. Diffs - https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/java/org/apache/airavata/commons/gfac/type/MappingFactory.java 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/GFacParameterTypes.xsd 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/pom.xml 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/assembly/bin-assembly.xml 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/dynamic_metadata.met PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/gfac-config.xml 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/oodt_integration.properties PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/distribution/airavata-server/src/main/resources/conf/pge-config.xml PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/gfac-core/src/main/java/org/apache/airavata/gfac/utils/GFacUtils.java 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/pom.xml PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataToApacheOODTConfigurationMapper.java PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstance.java PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/ApacheAiravataWorkFlowInstanceImpl.java PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProduct.java PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/oodt-integration/src/main/java/org/apache/airavata/oodt/integration/file/staging/OODTProductStager.java PRE-CREATION https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/component/ws/WSComponent.java 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/gpel/script/WorkflowWSDL.java 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/workflow-model/workflow-model-core/src/main/java/org/apache/airavata/workflow/model/xsd/GFacSimpleTypesXSD.java 1487802 https://svn.apache.org/repos/asf/airavata/trunk/modules/xbaya-gui/src/main/java/org/apache/airavata/xbaya/lead/LEADTypes.java 1487802
Re: Review Request: OODT-618 [sub task of OODT-219] : Ganglia XML parser for Ganglia plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11819/ --- (Updated June 12, 2013, 5:25 a.m.) Review request for oodt and Chris Mattmann. Description --- This patch is a subtask of Ganglia plugin [OODT-219] and I have added a SAX parser to parse the XML dump which will be downloaded to Ganglia plugin and ganglia metric builder. other than that this patch contains test scenarios to test the functionality and to assure the XML tag parsing, sample ganglia xml dump also with this patch. This addresses bug OODT-618. https://issues.apache.org/jira/browse/OODT-618 Diffs - trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/exceptions/MonitoringException.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaMetKeys.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaXMLParser.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Cluster.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Host.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Metric.java PRE-CREATION trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestGangliaXMLParser.java PRE-CREATION trunk/resource/src/testdata/gangliaXMLdump.xml PRE-CREATION Diff: https://reviews.apache.org/r/11819/diff/ Testing --- Test scenarios added to test the parsing and to assure XML tags. Thanks, Rajith Siriwardana
Review Request: OODT-618 [sub task of OODT-219] : Ganglia XML parser for Ganglia plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11819/ --- Review request for oodt and ChrisRK ChrisRK. Description --- This patch is a subtask of Ganglia plugin [OODT-219] and I have added a SAX parser to parse the XML dump which will be downloaded to Ganglia plugin and ganglia metric builder. other than that this patch contains test scenarios to test the functionality and to assure the XML tag parsing, sample ganglia xml dump also with this patch. This addresses bug OODT-618. https://issues.apache.org/jira/browse/OODT-618 Diffs - trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/exceptions/MonitoringException.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaMetKeys.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaXMLParser.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Cluster.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Host.java PRE-CREATION trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Metric.java PRE-CREATION trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestGangliaXMLParser.java PRE-CREATION trunk/resource/src/testdata/gangliaXMLdump.xml PRE-CREATION Diff: https://reviews.apache.org/r/11819/diff/ Testing --- Test scenarios added to test the parsing and to assure XML tags. Thanks, Rajith Siriwardana
Re: Apache Airavata-OODT Integration
Hi Chris, On configuration, I have get rid of all the configuration files, including pge-config.xml. All the required configurations are programmatically set. Configurations such FileManagerServer URL are configured in the airavata-server.properties file. I'll update the review request with modified details. Still I am not quite clear on how to retrieve staged file path properly. Currently I am using getStagedFilePath method in ApacheAiravataWorkFlowInstanceImpl to regenerate the staged file path. While I am going through the OODT code that I have seen method in DataTransferer to notify FileManagerServer once transfer is completed. But I couldn't see the same for product retrieval. As you suggested I'll improve my workflow using Apache Tika. I'd like to continue this as an Parallal task. While modifying staging implementation based on community feedback, currently I am looking at ingesting output back to OODT. Best Regards, Sanjaya On Wed, Jun 5, 2013 at 12:11 AM, Mattmann, Chris A (398J) < chris.a.mattm...@jpl.nasa.gov> wrote: > Hi Sanjaya, > > I think starting out with /bin/ls would be good, maybe like a /bin/ls > workflow, and then for each file returned, maybe run Apache Tika and > extract its metadata and then pipe that to a file? > > How about that? > > Cheers, > Chris > > ++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++ > > > > > > > -Original Message- > From: Sanjaya Medonsa > Reply-To: "d...@airavata.apache.org" > Date: Tuesday, June 4, 2013 5:31 AM > To: "d...@airavata.apache.org" > Cc: "dev@oodt.apache.org" > Subject: Re: Apache Airavata-OODT Integration > > >Hi Chris, > > Please see my comments below on the two items. > > > >Configuration : It should be possible to set them programmatically. > >Actually I have implemented partly it for file staging information. I'll > >work to get rid of the other configuration files. > > > >Staged File Path : I'll work on the suggested approach, though I am not > >fully understand it at the moment. I guess I need to go through bit more > >on > >CAS-PGE and come back to you on the proposed approach. > > > >Currently I am testing this by wrapping /bin/ls command as GFac service. I > >may need to test this with real workflow. Could you please provide me know > >some guidance on better scenario to test this. > > > >Cheers, > >Sanjaya > > > > > > > > > >On Mon, Jun 3, 2013 at 8:17 PM, Mattmann, Chris A (398J) < > >chris.a.mattm...@jpl.nasa.gov> wrote: > > > >> Hi Sanjaya, > >> > >> -Original Message- > >> > >> From: Sanjaya Medonsa > >> Reply-To: "d...@airavata.apache.org" > >> Date: Thursday, May 30, 2013 5:12 AM > >> To: "dev@oodt.apache.org" , > >>"d...@airavata.apache.org" > >> > >> Subject: Apache Airavata-OODT Integration > >> > >> >Hi, > >> > I have worked on the Apache Airavata integration with Apache > >>OODT. As > >> >a first step, I have implemented integration with Apache OODT file > >> >manager component. > >> > >> Great work!! > >> > >> Comments below: > >> > >> > 1. Introduce a new GFac Schema type called OODTProduct which > >>takes > >> >APache OODT product IDs as input. > >> > 2. Implemented new pre GFac Handler by extending Apache OODT > >> >PgeTaskInstance to stage the corresponding file into the working > >> >directory. > >> > 3. Once file is staged, input parameter with OODT product id is > >> >replaced with path of the staged file for downstream processing > >> > > >> >I have tested the implementation with Gfac application which wraps > >>/bin/ls > >> >command. Application takes product id as input and stage corresponding > >> >file > >> >into the working directory and /bin/ls is executed against the staged > >> >file. > >> >Hope this is a valid testing scenario. > >> > > >> >Concerns > >> >- Configurations : I have added new configuration file named and > >> >oodt-integration.properties in addition to dynamic_metadata.met and > >> >pge-config.xml files used by OODT. But at the moment there is no item > >> >configured with the oodt-integration.properties. > >> > >> You probably only need the pge-config.xml file. Dynamic metadata, and > >>the > >> task configuration properties can be specified programmatically, right? > >> > >> >- Staged File Name - With the current implementation of > >>PgeTaskInstance it > >> >is not possible to retrieve path of the staged file. Due to this > >> >limitation, I have query the FileManagerServer with product id and > >> >retrieve > >> >the file name
Re: Apache Airavata-OODT Integration
Hi Chris, I have just realized that Airavata GFac Handler has been updated to include Gfac Handler specific configuration recently. I think I should move the configurations in airavata-server.properties into gfac-config.xml as properties of the GFac handler which performs the OODT File Staging Best Regards, Sanjaya On Tue, Jun 11, 2013 at 5:50 AM, Sanjaya Medonsa wrote: > Hi Chris, >On configuration, I have get rid of all the configuration files, > including pge-config.xml. All the required configurations are > programmatically set. Configurations such FileManagerServer URL are > configured in the airavata-server.properties file. I'll update the review > request with modified details. >Still I am not quite clear on how to retrieve staged file path > properly. Currently I am using getStagedFilePath method > in ApacheAiravataWorkFlowInstanceImpl to regenerate the staged file path. > While I am going through the OODT code that I have seen method in > DataTransferer to notify FileManagerServer once transfer is completed. But > I couldn't see the same for product retrieval. >As you suggested I'll improve my workflow using Apache Tika. I'd > like to continue this as an Parallal task. While modifying staging > implementation based on community feedback, currently I am looking at > ingesting output back to OODT. > > Best Regards, > Sanjaya > > > > On Wed, Jun 5, 2013 at 12:11 AM, Mattmann, Chris A (398J) < > chris.a.mattm...@jpl.nasa.gov> wrote: > >> Hi Sanjaya, >> >> I think starting out with /bin/ls would be good, maybe like a /bin/ls >> workflow, and then for each file returned, maybe run Apache Tika and >> extract its metadata and then pipe that to a file? >> >> How about that? >> >> Cheers, >> Chris >> >> ++ >> Chris Mattmann, Ph.D. >> Senior Computer Scientist >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 171-266B, Mailstop: 171-246 >> Email: chris.a.mattm...@nasa.gov >> WWW: http://sunset.usc.edu/~mattmann/ >> ++ >> Adjunct Assistant Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++ >> >> >> >> >> >> >> -Original Message- >> From: Sanjaya Medonsa >> Reply-To: "d...@airavata.apache.org" >> Date: Tuesday, June 4, 2013 5:31 AM >> To: "d...@airavata.apache.org" >> Cc: "dev@oodt.apache.org" >> Subject: Re: Apache Airavata-OODT Integration >> >> >Hi Chris, >> > Please see my comments below on the two items. >> > >> >Configuration : It should be possible to set them programmatically. >> >Actually I have implemented partly it for file staging information. I'll >> >work to get rid of the other configuration files. >> > >> >Staged File Path : I'll work on the suggested approach, though I am not >> >fully understand it at the moment. I guess I need to go through bit more >> >on >> >CAS-PGE and come back to you on the proposed approach. >> > >> >Currently I am testing this by wrapping /bin/ls command as GFac service. >> I >> >may need to test this with real workflow. Could you please provide me >> know >> >some guidance on better scenario to test this. >> > >> >Cheers, >> >Sanjaya >> > >> > >> > >> > >> >On Mon, Jun 3, 2013 at 8:17 PM, Mattmann, Chris A (398J) < >> >chris.a.mattm...@jpl.nasa.gov> wrote: >> > >> >> Hi Sanjaya, >> >> >> >> -Original Message- >> >> >> >> From: Sanjaya Medonsa >> >> Reply-To: "d...@airavata.apache.org" >> >> Date: Thursday, May 30, 2013 5:12 AM >> >> To: "dev@oodt.apache.org" , >> >>"d...@airavata.apache.org" >> >> >> >> Subject: Apache Airavata-OODT Integration >> >> >> >> >Hi, >> >> > I have worked on the Apache Airavata integration with Apache >> >>OODT. As >> >> >a first step, I have implemented integration with Apache OODT file >> >> >manager component. >> >> >> >> Great work!! >> >> >> >> Comments below: >> >> >> >> > 1. Introduce a new GFac Schema type called OODTProduct which >> >>takes >> >> >APache OODT product IDs as input. >> >> > 2. Implemented new pre GFac Handler by extending Apache OODT >> >> >PgeTaskInstance to stage the corresponding file into the working >> >> >directory. >> >> > 3. Once file is staged, input parameter with OODT product id is >> >> >replaced with path of the staged file for downstream processing >> >> > >> >> >I have tested the implementation with Gfac application which wraps >> >>/bin/ls >> >> >command. Application takes product id as input and stage corresponding >> >> >file >> >> >into the working directory and /bin/ls is executed against the staged >> >> >file. >> >> >Hope this is a valid testing scenario. >> >> > >> >> >Concerns >> >> >- Configurations : I have added new configuration file named and >> >> >oodt-integration.properties in addition to dynamic_metadata.met and >> >> >pge-config.xml files
Review Request: OODT-627 Review
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11810/ --- Review request for oodt, Chris Mattmann and Cameron Goodale. Description --- Please see [1] for a more thorough description. I decided to change the MappingFunc calls instead of the actual implementation of ReplaceFunc since I felt that this made the most sense. The ReplaceFunc implementation seems to imply the "replace 'orig' with 'with'" approach so I left that alone and changed the various calls. [1] https://issues.apache.org/jira/browse/OODT-627 Diffs - /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1490420 /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/profile/DBMSExecutor.java 1490420 /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1490420 Diff: https://reviews.apache.org/r/11810/diff/ Testing --- Tested locally with the setup from [1] and changes to the example-ps.xml file from [2]. All tests pass and functionality of product handler is as expected. [1] https://cwiki.apache.org/confluence/display/OODT/XMLPS+Quick+Start+Guide [2] https://issues.apache.org/jira/browse/OODT-627 Thanks, Michael Joyce
Re: Review Request: OODT-627 Review
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11810/ --- (Updated June 13, 2013, 5:45 a.m.) Review request for oodt, Chris Mattmann and Cameron Goodale. Changes --- Mike LGTM. This was a bug -- your expectation was my intended functionality. Thanks! Description --- Please see [1] for a more thorough description. I decided to change the MappingFunc calls instead of the actual implementation of ReplaceFunc since I felt that this made the most sense. The ReplaceFunc implementation seems to imply the "replace 'orig' with 'with'" approach so I left that alone and changed the various calls. [1] https://issues.apache.org/jira/browse/OODT-627 This addresses bug OODT-627. https://issues.apache.org/jira/browse/OODT-627 Diffs - /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java 1490420 /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/profile/DBMSExecutor.java 1490420 /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java 1490420 Diff: https://reviews.apache.org/r/11810/diff/ Testing --- Tested locally with the setup from [1] and changes to the example-ps.xml file from [2]. All tests pass and functionality of product handler is as expected. [1] https://cwiki.apache.org/confluence/display/OODT/XMLPS+Quick+Start+Guide [2] https://issues.apache.org/jira/browse/OODT-627 Thanks, Michael Joyce
Re: Review Request: OODT-627 Review
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11810/#review21841 --- Ship it! Ship It! - Chris Mattmann On June 11, 2013, 8:38 p.m., Michael Joyce wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11810/ > --- > > (Updated June 11, 2013, 8:38 p.m.) > > > Review request for oodt, Chris Mattmann and Cameron Goodale. > > > Description > --- > > Please see [1] for a more thorough description. I decided to change the > MappingFunc calls instead of the actual implementation of ReplaceFunc since I > felt that this made the most sense. The ReplaceFunc implementation seems to > imply the "replace 'orig' with 'with'" approach so I left that alone and > changed the various calls. > > [1] https://issues.apache.org/jira/browse/OODT-627 > > > Diffs > - > > > /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/product/XMLPSProductHandler.java > 1490420 > /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/profile/DBMSExecutor.java > 1490420 > /trunk/xmlps/src/main/java/org/apache/oodt/xmlps/structs/CDEResult.java > 1490420 > > Diff: https://reviews.apache.org/r/11810/diff/ > > > Testing > --- > > Tested locally with the setup from [1] and changes to the example-ps.xml file > from [2]. All tests pass and functionality of product handler is as expected. > > [1] https://cwiki.apache.org/confluence/display/OODT/XMLPS+Quick+Start+Guide > [2] https://issues.apache.org/jira/browse/OODT-627 > > > Thanks, > > Michael Joyce > >
Re: Review Request: OODT-618 [sub task of OODT-219] : Ganglia XML parser for Ganglia plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11819/#review21839 --- Ship it! Ship It! - Chris Mattmann On June 12, 2013, 5:25 a.m., Rajith Siriwardana wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11819/ > --- > > (Updated June 12, 2013, 5:25 a.m.) > > > Review request for oodt and Chris Mattmann. > > > Description > --- > > This patch is a subtask of Ganglia plugin [OODT-219] and I have added a SAX > parser to parse the XML dump which will be downloaded to Ganglia plugin and > ganglia metric builder. other than that this patch contains test scenarios to > test the functionality and to assure the XML tag parsing, sample ganglia xml > dump also with this patch. > > > This addresses bug OODT-618. > https://issues.apache.org/jira/browse/OODT-618 > > > Diffs > - > > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/exceptions/MonitoringException.java > PRE-CREATION > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaMetKeys.java > PRE-CREATION > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/GangliaXMLParser.java > PRE-CREATION > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Cluster.java > PRE-CREATION > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Host.java > PRE-CREATION > > trunk/resource/src/main/java/org/apache/oodt/cas/resource/monitor/ganglia/configuration/Metric.java > PRE-CREATION > > trunk/resource/src/test/org/apache/oodt/cas/resource/monitor/TestGangliaXMLParser.java > PRE-CREATION > trunk/resource/src/testdata/gangliaXMLdump.xml PRE-CREATION > > Diff: https://reviews.apache.org/r/11819/diff/ > > > Testing > --- > > Test scenarios added to test the parsing and to assure XML tags. > > > Thanks, > > Rajith Siriwardana > >
