Hi Ross, +1 for this proposal and it looks very interesting to me.. I would like to contribute to this project.
Regards Lahiru On Fri, Apr 29, 2011 at 10:09 AM, Ross Gardler <rgard...@apache.org> wrote: > On 29 Apr 2011, at 16:56, Waruna Ranasinghe <waruna...@gmail.com> wrote: > > > Hi all, > > > > +1 for the proposal Airavata. I would like to contribute to this project. > > Yay! Another person who sees personal value. Look forward to working with > you too :-) > > Ross > > > > > > > > Thanks, > > Waruna > > > >>>>> On 4/23/11 5:53 PM, Ross Gardler wrote: > >>>>>> > >>>>>> I would like to propose Airavata for entry into the Apache > Incubator. > >>>>>> > >>>>>> The full proposal can be found at [1] and is copied at the end of > this > >>>>>> mail. For those in a hurry here's a quick summary: > >>>>>> > >>>>>> Airavata is a software toolkit currently used to build science > >> gateways > >>>>>> but that has a much wider potential use. It provides features to > >>>>>> compose, manage, execute, and monitor large scale applications and > >>>>>> workflows on computational resources ranging from local clusters to > >>>>>> national grids and computing clouds. Users can use Airavata back end > >>>>>> services and build gadgets to deploy in open social containers such > as > >>>>>> Apache Rave and modify them to suite their needs. Airavata builds on > >>>>>> general concepts of service oriented computing, distributed > messaging, > >>>>>> and workflow composition and orchestration. > >>>>>> > >>>>>> Airavata will provide web interfaces and scalable Service Oriented > >>>>>> Architecture based backend services to build or enhance Science > >> Gateway > >>>>>> (see https://www.teragrid.org/web/science-gateways/) and similar > >>>>>> environments. Airavata will specifically focus on: > >>>>>> > >>>>>> 1. sophisticated server-side tools for registering and managing > large > >>>>>> scale applications on computational resources. > >>>>>> > >>>>>> 2. graphical user interfaces to construct, execute, control, manage > >> and > >>>>>> reuse of scientific workflows. > >>>>>> > >>>>>> 3. interfacing and interoperability with with various external > (third > >>>>>> party) data and provenance management tools > >>>>>> > >>>>>> The project team consists of a number of existing Apache Committers > >> and > >>>>>> the code comes from the same stable as some of the code donated to > >>>>>> Apache Rave (Incubating). > >>>>>> > >>>>>> We welcome your questions, suggestions, observations and support. > >>>>>> > >>>>>> Ross > >>>>>> > >>>>>> [1] http://wiki.apache.org/incubator/AiravataProposal > >>>>>> > >>>>>> FULL PROPOSAL TEXT > >>>>>> ================== > >>>>>> > >>>>>> = Airavata Proposal for Apache Incubator = > >>>>>> > >>>>>> == Abstract == > >>>>>> Airavata is a software toolkit currently used to build science > >>>>>> gateways but that has a much wider potential use. It provides > features > >>>>>> to compose, manage, execute, and monitor large scale applications > and > >>>>>> workflows on computational resources ranging from local clusters to > >>>>>> national grids and computing clouds. Users can use Airavata back end > >>>>>> services and build gadgets to deploy in open social containers such > as > >>>>>> Apache Rave and modify them to suite their needs. Airavata builds on > >>>>>> general concepts of service oriented computing, distributed > messaging, > >>>>>> and workflow composition and orchestration. > >>>>>> > >>>>>> > >>>>>> == Proposal == > >>>>>> > >>>>>> Airavata will provide web interfaces and scalable Service Oriented > >>>>>> Architecture based backend services to build or enhance Science > >>>>>> Gateway (see https://www.teragrid.org/web/science-gateways/) > >>>>>> and similar environments. Airavata will specifically focus on: > >>>>>> > >>>>>> 1. sophisticated server-side tools for registering and managing > large > >>>>>> scale applications on computational resources. > >>>>>> 2. graphical user interfaces to construct, execute, control, manage > >> and > >>>>>> reuse of scientific workflows. > >>>>>> 3. interfacing and interoperability with with various external > (third > >>>>>> party) data and provenance management tools. > >>>>>> > >>>>>> == Background == > >>>>>> Working in close quarters with Apache Axis2 committers and inspired > by > >>>>>> the true open source community driven software development of ASF, > >>>>>> Suresh Marru and Marlon Pierce have been pioneering the idea of a > >>>>>> Science Gateways software-based Apache project since late 2008. Many > >>>>>> Apache members have fostered these ideas and guided them to arrive > at > >>>>>> this proposal. > >>>>>> > >>>>>> Currently the software is a actively used in various science > >>>>>> gateways. But the tools are general purpose and build upon widely > used > >>>>>> Apache tools like Axis2, ODE engine. The core team is motivated to > >>>>>> expand the community and build a community welcoming both > synergistic > >>>>>> software components and also new usage scenarios. > >>>>>> > >>>>>> It is perhaps worth noting that one of the three seed projects that > >>>>>> make up the Apache Rave (Incubating) project is also the product of > >>>>>> this same team and is derived from the same Science Gateways > >>>>>> community. > >>>>>> > >>>>>> == Rationale == > >>>>>> > >>>>>> The nature of computational problems has evolved from simple desktop > >>>>>> calculations to complex, multidisciplinary activities that require > the > >>>>>> monitoring and analysis of remote data streams, database and web > >>>>>> search and large ensembles of simulations. In the academic domain > >>>>>> Science Gateways have emerged to address these needs and have built > >>>>>> software platforms that provide a community of users with the > ability > >>>>>> to easily solve computational problems within a specific domain. The > >>>>>> tools developed to support these gateways are potentially of value > to > >>>>>> any organisation needing to perform complex computations. Gateways > >>>>>> provide a convenient interface to the underlying infrastrucure > without > >>>>>> the need for a deep understanding of the intricacies that > >>>>>> infrastructure. > >>>>>> > >>>>>> We summarize the rationale for choosing The Apache Software > Foundation > >>>>>> (ASF) below. This is what we hope to gain from participating in the > >>>>>> ASF. > >>>>>> > >>>>>> 1. '''Broader impact''': our science gateway tool set is based on > >>>>>> Service Oriented Architecture principles, and it has always been our > >>>>>> goal to align our software with broader trends in the development of > >>>>>> software for distributed systems. Participating in the ASF provides > a > >>>>>> concrete way to implement this idea. In particular, we have done > >>>>>> extensive work on the workflow systems, messaging, and application > >>>>>> management as Web services from the perspective of computational > >> science > >>>>>> use cases (i.e., high failure rates, very long running jobs, dynamic > >>>>>> service creation, workflows not expressible as directed acyclic > >> graphs, > >>>>>> etc). These requirements and our work to implement them have already > >> had > >>>>>> direct impact on the Apache Axis 2 and Apache ODE projects. As an > >> Apache > >>>>>> project, it is hoped that our community will have an enhanced > >>>>>> opportunity for collaboration and complementary development with > >> Apache > >>>>>> Hadoop (for scientific application management), Apache QPID (for > >>>>>> messaging), Apache Rave (incubator - Open Social Container) and > >> others. > >>>>>> It is our goal to expand our software’s usage beyond just science > >>>>>> gateways to the broader enterprise community. > >>>>>> 2. '''Sustainability''': Science gateway software development (and > >>>>>> cyberinfrastructure software generally) is primarily funded in the > US > >> by > >>>>>> the National Science Foundation (NSF), so the long term > sustainability > >>>>>> of software across funding cycles is a longstanding problem. The NSF > >> is > >>>>>> attempting to solve this problem, and its vision for sustainable > >>>>>> software is described here: > >>>>>> http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating > in > >> the > >>>>>> ASF is our project’s vision for reaching software sustainability > >> that > >>>>>> underpins the NSF CF21 vision. As a successful ASF project (after > >>>>>> incubation), we will have created a community led, rather than > funding > >>>>>> led, environment for the development of our sotware. This community, > >>>>>> through our community engagement work and adoption of meritocratic > >>>>>> principles, will expand beyond our current core team and existing > >>>>>> project collaborations. This will greatly increase the chances that > >> our > >>>>>> software will continue to grow and improve beyond the participation > of > >>>>>> any individuals. > >>>>>> 3. '''Maturity''': much of the software included in this proposal > was > >>>>>> developed initially by graduate students as part of their Ph. D. > work. > >>>>>> The Open Grid Computing Environment has devoted significant effort > >>>>>> (through salaried staff and volunteers from collaborating > >> institutions) > >>>>>> to convert these research projects into mature, reliable, > >> well-written, > >>>>>> packaged components. The code is currently hosted at SourceForge, > but > >> we > >>>>>> recognize the need to go beyond just the SourceForge support tools > to > >>>>>> participate in a real community of software engineering experts. It > is > >>>>>> our desire, through the Apache Incubator, to take our software > >>>>>> engineering efforts to a higher level by learning from the > substantial > >>>>>> experience of appropraite Apache Committers. Apache mentors will > >> provide > >>>>>> initial guidance, as will the attraction of additional committers > from > >>>>>> the relevant Apache projects. > >>>>>> > >>>>>> == Initial Goals == > >>>>>> > >>>>>> * Implement a standalone version of the code base with a simple > hello > >>>>>> world service, workflow and gadget(s) to access the examples. > >>>>>> * Migration of documentation and design knowledge from existing SF > >> project > >>>>>> * Re-architect Grid based security (GSI) dependencies and adopt more > >>>>>> general purpose security implementations. > >>>>>> * Make sure Cloud (including hadoop) support is more first class. > >>>>>> * Aim to have the first Apache release within the first 6 months > >>>>>> * Verify with Apache Legal that some of the more esoteric licences > in > >>>>>> our dependencies are acceptable, or replace them as appropriate > >>>>>> > >>>>>> == Current Status == > >>>>>> > >>>>>> The proposed tools are currently hosted on SourceForge at > >>>>>> http://sourceforge.net/projects/ogce/ (source at > >>>>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and > >> are > >>>>>> described at http://www.collab-ogce.org. > >>>>>> > >>>>>> == Meritocracy == > >>>>>> > >>>>>> A significant portion of initial committers are already ASF > >>>>>> Committers/Members, > >>>>>> and the entire team is well experienced with open source software > >>>>>> development. The existing code base has resulted from > >>>>>> multi-institutional collaborative projects. The developers are well > >>>>>> aware of the Apache way and will honor the meritocracy policy of ASF > >>>>>> foundation. > >>>>>> > >>>>>> == Community == > >>>>>> > >>>>>> To date our focus has been serving our immediate partners needs > rather > >>>>>> than looking outwards in order to build a broader community with > >>>>>> diverse needs. Whilst the core team area likely to remain focussed > on > >>>>>> the Science Gateways communities we are keen to welcome community > >>>>>> members from other disciplines. > >>>>>> > >>>>>> == Core Developers == > >>>>>> Our core developers consist of participants from academic, > >>>>>> not-for-profit and for-profit organisations. Many are already well > >>>>>> versed in The Apache Way. > >>>>>> > >>>>>> Amongst our initial team we have one or more committers on the > >>>>>> following Apache top level projects; axis, geronimo, synapse, ws, > >>>>>> ws-pmc, ws-woden as well as Apache Rave (Incubating). > >>>>>> > >>>>>> == Alignment == > >>>>>> Airavata software is built upon Apache Projects like Axis2, ODE, > >>>>>> Rampart, Tomcat and Maven. We will try to closely align the project > >>>>>> with ODE to ensure BPEL workflow compatibility. We will align with > >>>>>> metadata management projects like Apache OODT. Web interfaces within > >>>>>> the Airavata software will be synergistically developed with Apache > >>>>>> Rave. > >>>>>> > >>>>>> == Known Risks == > >>>>>> === Orphaned products === > >>>>>> We acknowledge the need to seek project contributions outside the > >> current > >>>>>> developers. The core team actively travels and conducts workshops > and > >>>>>> tutorials at relevant academic conferences like Supercomputing, > >> TeraGrid, > >>>>>> Collaborative Technologies Systems and SciDAC. Previous experiences > >>>>>> have showed that these tutorials and outreach efforts will bring in > >>>>>> community participation. The general strategy will be to encourage > >>>>>> users to be active in the community and develop patches and > >>>>>> contribute. Also, the core developers use the Airavata software in > >>>>>> multiple projects with a life span ranging from 2 to 10 years, so > the > >>>>>> risk of orphaned products is very minimal. > >>>>>> > >>>>>> Furthermore, by opening our doors to non-academic organisations > >>>>>> already adopting large scale computation related projects in the ASF > >>>>>> we hope to be able to build community beyond the proposing teams > >>>>>> Science Gateway interests. > >>>>>> > >>>>>> === Inexperience with Open Source === > >>>>>> The core team is very familiar with open source practices. The > >>>>>> developers include existing Apache members who have long term > >> experience > >>>>>> with > >>>>>> the Apache Way. The OGCE project has been an active > >>>>>> open source project in SourceForge since November 2006. We welcome > the > >>>>>> new directions and are well prepared to follow the Apache way. > >>>>>> > >>>>>> === Homogenous Developers === > >>>>>> We have a semi-distributed development environment distributed among > >>>>>> Indiana University and Lanka Software Foundation. We fully expect > >>>>>> contributions from the partnering science gateways adding to the > >>>>>> heterogeneous development. > >>>>>> > >>>>>> === Reliance on Salaried Developers === > >>>>>> The core developers are self motivated on the project and also are > >>>>>> funded through various federal, state and endowment research > >>>>>> grants. Participation in these research efforts based on Airavata > >>>>>> software is mostly voluntary and above and beyond the requirements > of > >>>>>> the salaried jobs. > >>>>>> > >>>>>> The Open Gateway Computing project, from which the initial code > >>>>>> donation is sourced, is funded for the next 3 years and is mandated > by > >>>>>> the funding guidelines to open source software development - > >>>>>> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We > >>>>>> believe in the Airavata software capabilities and its vital role in > >>>>>> providing sustainable middleware for Science Gateways. Nevertheless, > >>>>>> the core team will actively build upon Airavata software and foster > >>>>>> developer community outside the current core. > >>>>>> > >>>>>> === Relationships with Other Apache Products === > >>>>>> See “Alignment†above. Airavata is based on the concepts of > >> Service > >>>>>> Oriented Architecture and all services run within Tomcat > >>>>>> container. The web services are based on Axis2. The orchestration of > >>>>>> the scientific workflows uses Orchestration Director Engine. The > >>>>>> software is built using Apache Maven. > >>>>>> > >>>>>> === An Excessive Fascination with the Apache Brand === > >>>>>> The Apache brand would certainly help promote the software suite, > but > >>>>>> gaining the brand is not the motivation for this project. Airavata > is > >>>>>> being proposed to Apache because of the belief in Apache’s > >> meritocracy > >>>>>> model for mentored, community-driven, open source software is the > best > >>>>>> way to develop sustainable software. See “Rational†above. Most > >>>>>> importantly, The Apache Software Foundation will help us create an > >>>>>> institution-neutral contribution venue and will help us build a > >>>>>> long-standing community around Airavata to sustain and improve it > >>>>>> beyond the span of specific, targeted research grants. > >>>>>> > >>>>>> == Documentation == > >>>>>> Existing documentation is available from the OGCE wiki, > >>>>>> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition, > >>>>>> there is abundance of presentation and self guided video tutorial > >>>>>> material. Effort will be put in to collect all this information into > >>>>>> meaningful documentation on the Apache websites. > >>>>>> > >>>>>> == Initial Source == > >>>>>> The initial source of the project is in SourceForge. The source is > >>>>>> available for anonymous check out from svn at > >>>>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/ > >>>>>> > >>>>>> == Source and Intellectual Property Submission Plan == > >>>>>> Indiana University is the current holder of Intellectual Property > >>>>>> rights for the software. The university has approved the code > donation > >>>>>> and signed trustees approval, Corporate Contributor Licence > Agreement > >>>>>> and Software Grant Agreement have been emailed to ASF secretary and > >>>>>> received acknowledgement. > >>>>>> > >>>>>> Specifically Indiana University will donate 4 components into > Airavata > >>>>>> project. > >>>>>> > >>>>>> 1. XBaya Scientific Workflow Suite - includes a GUI for workflow > >>>>>> composition and monitoring. The composed workflow can be exported to > >>>>>> various workflow languages like BPEL, SCUFL, Condor DAG, Jython and > >>>>>> Java. The defacto workflow enacting engine used is Apache ODE. > >>>>>> 2. GFac - an application wrapper service that can be used to wrap > >>>>>> command line-driven science applications and make them into robust, > >>>>>> network- accessible services. This component is build on Axis2 web > >>>>>> service stack. > >>>>>> 3. XRegistry - a registry service for storing deployment information > >>>>>> about wrapped application services and constructed workflows. > >>>>>> 4. WS-Messenger - a “publish-subscribe†based message broker > >>>>>> implemented on top of Apache Axis2 web services stack. It implements > >> the > >>>>>> WS-Eventing and WS-Notifications specifications and incorporates a > >>>>>> message box component that facilities communications with clients > >> behind > >>>>>> firewalls and overcomes network glitches. > >>>>>> > >>>>>> == External Dependencies == > >>>>>> > >>>>>> Following the guideline -http://www.apache.org/legal/resolved.html, > >> the > >>>>>> following are the dependent software and all of them are in binary > >>>>>> format in java archive (jar files). > >>>>>> > >>>>>> * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API > >>>>>> * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, > xmpp, > >>>>>> xsul, sigiri, atomixmiser, weps-beans. > >>>>>> * BSD: puretls, > >>>>>> * MIT: bcporv, hsqldb, dom4j, slf4j > >>>>>> * PSFL: Jython > >>>>>> * GPL 2.0: mysql-connector-java > >>>>>> * Other: > >>>>>> * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT) > >>>>>> * backport (public domain) > >>>>>> * jaxen (http://jaxen.codehaus.org/license.html) > >>>>>> > >>>>>> Licence incompatibilities (GPL) will be resolved during incubation. > >>>>>> > >>>>>> == Cryptography == > >>>>>> The software does not implement any cryptographic algorithms. > However, > >>>>>> to perform secured messaging and data movement and SSL > communications, > >>>>>> the software depends upon third party security libraries. These > >>>>>> external libraries depend in turn on Java Security, Puretls, Cryptix > >>>>>> and Bounce Castle libraries. Apache Cryptographic steps will be > >>>>>> followed to register the use of these libraries. > >>>>>> > >>>>>> == Required Resources == > >>>>>> > >>>>>> === Mailing lists === > >>>>>> 1. airavata-dev > >>>>>> 2. airavata-commits > >>>>>> 3. airavata-private > >>>>>> > >>>>>> === Subversion Directory === > >>>>>> https://svn.apache.org/repos/asf/incubator/airavata > >>>>>> > >>>>>> === Issue Tracking === > >>>>>> We intend to make use of Jira for issue tracking. Proposed key: > >> AIRAVATA > >>>>>> > >>>>>> === Other Resources === > >>>>>> > >>>>>> We intend to manage our website using the Apache CMS. > >>>>>> > >>>>>> == Initial Committers == > >>>>>> Names of initial committers with affiliation and current ASF status: > >>>>>> > >>>>>> || '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' || > >>>>>> '''ASF Status''' || '''Apache Id''' || > >>>>>> || Suresh Marru || sma...@cs.indiana.edu || Indiana University || > On > >>>>>> File || Apache Commiter || smarru || > >>>>>> || Marlon Pierce || mpie...@cs.indiana.edu || Indiana University || > >> On > >>>>>> File || Apache Commiter || mpierce || > >>>>>> || Srinath Perera || hemap...@apache.org || Lanka Software > Foundation > >> || > >>>>>> On File || Apache Member || hemapani || > >>>>>> || Aleksander Slominski || aslom at us.ibm.com || IBM || On File || > >>>>>> Apache Member || aslom || > >>>>>> || Raminderjeet Singh || rami...@indiana.edu || Indiana University > || > >> On > >>>>>> File || Apache Commiter || raminder || > >>>>>> || Archit Kulshrestha || akuls...@indiana.edu || Indiana University > >> || > >>>>>> On File || N/A || N/A || > >>>>>> || Chathura Herath || chath...@apache.org || Indiana University || > On > >>>>>> File || Apache Commiter || chathura || > >>>>>> || Eran Chinthaka || chinth...@apache.org || Indiana University || > On > >>>>>> File || Apache Member || chinthaka || > >>>>>> || Thilina Gunaratne || thil...@apache.org || Indiana University || > >> On > >>>>>> File || Apache Commiter || thilina || > >>>>>> || Wathsala Vithanage || waths...@opensource.lk || Lanka Software > >>>>>> Foundation || On File || N/A || N/A || > >>>>>> > >>>>>> All the parties are affiliated with companies and organizations that > >>>>>> are familiar with the development of open source. We expect that the > >>>>>> amount of volunteer work will increase, and more developers will > come > >>>>>> on board. > >>>>>> > >>>>>> == Champion == > >>>>>> Ross Gardler, Apache Software Foundation > >>>>>> > >>>>>> == Nominated Mentors == > >>>>>> * Ross Gardler, Member, Apache Software Foundation > >>>>>> * Alek Slominski, Member, Apache Software Foundation > >>>>>> * Ate Douma, Member, Apache Software Foundation > >>>>>> * Sanjiva Weerawarna, Member, Apache Software Foundation > >>>>>> * Paul Fremantle, Member, Apache Software Foundation > >>>>>> > >>>>>> == Sponsoring Entity == > >>>>>> Apache Incubator Project. > >>>>>> > >>>>>> > >>>>>> > --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > >>>>>> For additional commands, e-mail: general-h...@incubator.apache.org > >>>>>> > >> > > > > -- > > ----------------------------------------------------- > > Regards, > > Waruna Ranasinghe > > > > blog: http://warunapw.blogspot.com > > twitter: http://twitter.com/warunapww > > http://lk.linkedin.com/in/waruna > > www.facebook.com/waruna.ranasinghe > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > >