Hi all, +1 for the proposal Airavata. I would like to contribute to this project.
Thanks, Waruna > >>> On 4/23/11 5:53 PM, Ross Gardler wrote: > >>>> > >>>> I would like to propose Airavata for entry into the Apache Incubator. > >>>> > >>>> The full proposal can be found at [1] and is copied at the end of this > >>>> mail. For those in a hurry here's a quick summary: > >>>> > >>>> Airavata is a software toolkit currently used to build science > gateways > >>>> but that has a much wider potential use. It provides features to > >>>> compose, manage, execute, and monitor large scale applications and > >>>> workflows on computational resources ranging from local clusters to > >>>> national grids and computing clouds. Users can use Airavata back end > >>>> services and build gadgets to deploy in open social containers such as > >>>> Apache Rave and modify them to suite their needs. Airavata builds on > >>>> general concepts of service oriented computing, distributed messaging, > >>>> and workflow composition and orchestration. > >>>> > >>>> Airavata will provide web interfaces and scalable Service Oriented > >>>> Architecture based backend services to build or enhance Science > Gateway > >>>> (see https://www.teragrid.org/web/science-gateways/) and similar > >>>> environments. Airavata will specifically focus on: > >>>> > >>>> 1. sophisticated server-side tools for registering and managing large > >>>> scale applications on computational resources. > >>>> > >>>> 2. graphical user interfaces to construct, execute, control, manage > and > >>>> reuse of scientific workflows. > >>>> > >>>> 3. interfacing and interoperability with with various external (third > >>>> party) data and provenance management tools > >>>> > >>>> The project team consists of a number of existing Apache Committers > and > >>>> the code comes from the same stable as some of the code donated to > >>>> Apache Rave (Incubating). > >>>> > >>>> We welcome your questions, suggestions, observations and support. > >>>> > >>>> Ross > >>>> > >>>> [1] http://wiki.apache.org/incubator/AiravataProposal > >>>> > >>>> FULL PROPOSAL TEXT > >>>> ================== > >>>> > >>>> = Airavata Proposal for Apache Incubator = > >>>> > >>>> == Abstract == > >>>> Airavata is a software toolkit currently used to build science > >>>> gateways but that has a much wider potential use. It provides features > >>>> to compose, manage, execute, and monitor large scale applications and > >>>> workflows on computational resources ranging from local clusters to > >>>> national grids and computing clouds. Users can use Airavata back end > >>>> services and build gadgets to deploy in open social containers such as > >>>> Apache Rave and modify them to suite their needs. Airavata builds on > >>>> general concepts of service oriented computing, distributed messaging, > >>>> and workflow composition and orchestration. > >>>> > >>>> > >>>> == Proposal == > >>>> > >>>> Airavata will provide web interfaces and scalable Service Oriented > >>>> Architecture based backend services to build or enhance Science > >>>> Gateway (see https://www.teragrid.org/web/science-gateways/) > >>>> and similar environments. Airavata will specifically focus on: > >>>> > >>>> 1. sophisticated server-side tools for registering and managing large > >>>> scale applications on computational resources. > >>>> 2. graphical user interfaces to construct, execute, control, manage > and > >>>> reuse of scientific workflows. > >>>> 3. interfacing and interoperability with with various external (third > >>>> party) data and provenance management tools. > >>>> > >>>> == Background == > >>>> Working in close quarters with Apache Axis2 committers and inspired by > >>>> the true open source community driven software development of ASF, > >>>> Suresh Marru and Marlon Pierce have been pioneering the idea of a > >>>> Science Gateways software-based Apache project since late 2008. Many > >>>> Apache members have fostered these ideas and guided them to arrive at > >>>> this proposal. > >>>> > >>>> Currently the software is a actively used in various science > >>>> gateways. But the tools are general purpose and build upon widely used > >>>> Apache tools like Axis2, ODE engine. The core team is motivated to > >>>> expand the community and build a community welcoming both synergistic > >>>> software components and also new usage scenarios. > >>>> > >>>> It is perhaps worth noting that one of the three seed projects that > >>>> make up the Apache Rave (Incubating) project is also the product of > >>>> this same team and is derived from the same Science Gateways > >>>> community. > >>>> > >>>> == Rationale == > >>>> > >>>> The nature of computational problems has evolved from simple desktop > >>>> calculations to complex, multidisciplinary activities that require the > >>>> monitoring and analysis of remote data streams, database and web > >>>> search and large ensembles of simulations. In the academic domain > >>>> Science Gateways have emerged to address these needs and have built > >>>> software platforms that provide a community of users with the ability > >>>> to easily solve computational problems within a specific domain. The > >>>> tools developed to support these gateways are potentially of value to > >>>> any organisation needing to perform complex computations. Gateways > >>>> provide a convenient interface to the underlying infrastrucure without > >>>> the need for a deep understanding of the intricacies that > >>>> infrastructure. > >>>> > >>>> We summarize the rationale for choosing The Apache Software Foundation > >>>> (ASF) below. This is what we hope to gain from participating in the > >>>> ASF. > >>>> > >>>> 1. '''Broader impact''': our science gateway tool set is based on > >>>> Service Oriented Architecture principles, and it has always been our > >>>> goal to align our software with broader trends in the development of > >>>> software for distributed systems. Participating in the ASF provides a > >>>> concrete way to implement this idea. In particular, we have done > >>>> extensive work on the workflow systems, messaging, and application > >>>> management as Web services from the perspective of computational > science > >>>> use cases (i.e., high failure rates, very long running jobs, dynamic > >>>> service creation, workflows not expressible as directed acyclic > graphs, > >>>> etc). These requirements and our work to implement them have already > had > >>>> direct impact on the Apache Axis 2 and Apache ODE projects. As an > Apache > >>>> project, it is hoped that our community will have an enhanced > >>>> opportunity for collaboration and complementary development with > Apache > >>>> Hadoop (for scientific application management), Apache QPID (for > >>>> messaging), Apache Rave (incubator - Open Social Container) and > others. > >>>> It is our goal to expand our software’s usage beyond just science > >>>> gateways to the broader enterprise community. > >>>> 2. '''Sustainability''': Science gateway software development (and > >>>> cyberinfrastructure software generally) is primarily funded in the US > by > >>>> the National Science Foundation (NSF), so the long term sustainability > >>>> of software across funding cycles is a longstanding problem. The NSF > is > >>>> attempting to solve this problem, and its vision for sustainable > >>>> software is described here: > >>>> http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating in > the > >>>> ASF is our project’s vision for reaching software sustainability > that > >>>> underpins the NSF CF21 vision. As a successful ASF project (after > >>>> incubation), we will have created a community led, rather than funding > >>>> led, environment for the development of our sotware. This community, > >>>> through our community engagement work and adoption of meritocratic > >>>> principles, will expand beyond our current core team and existing > >>>> project collaborations. This will greatly increase the chances that > our > >>>> software will continue to grow and improve beyond the participation of > >>>> any individuals. > >>>> 3. '''Maturity''': much of the software included in this proposal was > >>>> developed initially by graduate students as part of their Ph. D. work. > >>>> The Open Grid Computing Environment has devoted significant effort > >>>> (through salaried staff and volunteers from collaborating > institutions) > >>>> to convert these research projects into mature, reliable, > well-written, > >>>> packaged components. The code is currently hosted at SourceForge, but > we > >>>> recognize the need to go beyond just the SourceForge support tools to > >>>> participate in a real community of software engineering experts. It is > >>>> our desire, through the Apache Incubator, to take our software > >>>> engineering efforts to a higher level by learning from the substantial > >>>> experience of appropraite Apache Committers. Apache mentors will > provide > >>>> initial guidance, as will the attraction of additional committers from > >>>> the relevant Apache projects. > >>>> > >>>> == Initial Goals == > >>>> > >>>> * Implement a standalone version of the code base with a simple hello > >>>> world service, workflow and gadget(s) to access the examples. > >>>> * Migration of documentation and design knowledge from existing SF > project > >>>> * Re-architect Grid based security (GSI) dependencies and adopt more > >>>> general purpose security implementations. > >>>> * Make sure Cloud (including hadoop) support is more first class. > >>>> * Aim to have the first Apache release within the first 6 months > >>>> * Verify with Apache Legal that some of the more esoteric licences in > >>>> our dependencies are acceptable, or replace them as appropriate > >>>> > >>>> == Current Status == > >>>> > >>>> The proposed tools are currently hosted on SourceForge at > >>>> http://sourceforge.net/projects/ogce/ (source at > >>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and > are > >>>> described at http://www.collab-ogce.org. > >>>> > >>>> == Meritocracy == > >>>> > >>>> A significant portion of initial committers are already ASF > >>>> Committers/Members, > >>>> and the entire team is well experienced with open source software > >>>> development. The existing code base has resulted from > >>>> multi-institutional collaborative projects. The developers are well > >>>> aware of the Apache way and will honor the meritocracy policy of ASF > >>>> foundation. > >>>> > >>>> == Community == > >>>> > >>>> To date our focus has been serving our immediate partners needs rather > >>>> than looking outwards in order to build a broader community with > >>>> diverse needs. Whilst the core team area likely to remain focussed on > >>>> the Science Gateways communities we are keen to welcome community > >>>> members from other disciplines. > >>>> > >>>> == Core Developers == > >>>> Our core developers consist of participants from academic, > >>>> not-for-profit and for-profit organisations. Many are already well > >>>> versed in The Apache Way. > >>>> > >>>> Amongst our initial team we have one or more committers on the > >>>> following Apache top level projects; axis, geronimo, synapse, ws, > >>>> ws-pmc, ws-woden as well as Apache Rave (Incubating). > >>>> > >>>> == Alignment == > >>>> Airavata software is built upon Apache Projects like Axis2, ODE, > >>>> Rampart, Tomcat and Maven. We will try to closely align the project > >>>> with ODE to ensure BPEL workflow compatibility. We will align with > >>>> metadata management projects like Apache OODT. Web interfaces within > >>>> the Airavata software will be synergistically developed with Apache > >>>> Rave. > >>>> > >>>> == Known Risks == > >>>> === Orphaned products === > >>>> We acknowledge the need to seek project contributions outside the > current > >>>> developers. The core team actively travels and conducts workshops and > >>>> tutorials at relevant academic conferences like Supercomputing, > TeraGrid, > >>>> Collaborative Technologies Systems and SciDAC. Previous experiences > >>>> have showed that these tutorials and outreach efforts will bring in > >>>> community participation. The general strategy will be to encourage > >>>> users to be active in the community and develop patches and > >>>> contribute. Also, the core developers use the Airavata software in > >>>> multiple projects with a life span ranging from 2 to 10 years, so the > >>>> risk of orphaned products is very minimal. > >>>> > >>>> Furthermore, by opening our doors to non-academic organisations > >>>> already adopting large scale computation related projects in the ASF > >>>> we hope to be able to build community beyond the proposing teams > >>>> Science Gateway interests. > >>>> > >>>> === Inexperience with Open Source === > >>>> The core team is very familiar with open source practices. The > >>>> developers include existing Apache members who have long term > experience > >>>> with > >>>> the Apache Way. The OGCE project has been an active > >>>> open source project in SourceForge since November 2006. We welcome the > >>>> new directions and are well prepared to follow the Apache way. > >>>> > >>>> === Homogenous Developers === > >>>> We have a semi-distributed development environment distributed among > >>>> Indiana University and Lanka Software Foundation. We fully expect > >>>> contributions from the partnering science gateways adding to the > >>>> heterogeneous development. > >>>> > >>>> === Reliance on Salaried Developers === > >>>> The core developers are self motivated on the project and also are > >>>> funded through various federal, state and endowment research > >>>> grants. Participation in these research efforts based on Airavata > >>>> software is mostly voluntary and above and beyond the requirements of > >>>> the salaried jobs. > >>>> > >>>> The Open Gateway Computing project, from which the initial code > >>>> donation is sourced, is funded for the next 3 years and is mandated by > >>>> the funding guidelines to open source software development - > >>>> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We > >>>> believe in the Airavata software capabilities and its vital role in > >>>> providing sustainable middleware for Science Gateways. Nevertheless, > >>>> the core team will actively build upon Airavata software and foster > >>>> developer community outside the current core. > >>>> > >>>> === Relationships with Other Apache Products === > >>>> See “Alignment†above. Airavata is based on the concepts of > Service > >>>> Oriented Architecture and all services run within Tomcat > >>>> container. The web services are based on Axis2. The orchestration of > >>>> the scientific workflows uses Orchestration Director Engine. The > >>>> software is built using Apache Maven. > >>>> > >>>> === An Excessive Fascination with the Apache Brand === > >>>> The Apache brand would certainly help promote the software suite, but > >>>> gaining the brand is not the motivation for this project. Airavata is > >>>> being proposed to Apache because of the belief in Apache’s > meritocracy > >>>> model for mentored, community-driven, open source software is the best > >>>> way to develop sustainable software. See “Rational†above. Most > >>>> importantly, The Apache Software Foundation will help us create an > >>>> institution-neutral contribution venue and will help us build a > >>>> long-standing community around Airavata to sustain and improve it > >>>> beyond the span of specific, targeted research grants. > >>>> > >>>> == Documentation == > >>>> Existing documentation is available from the OGCE wiki, > >>>> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition, > >>>> there is abundance of presentation and self guided video tutorial > >>>> material. Effort will be put in to collect all this information into > >>>> meaningful documentation on the Apache websites. > >>>> > >>>> == Initial Source == > >>>> The initial source of the project is in SourceForge. The source is > >>>> available for anonymous check out from svn at > >>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/ > >>>> > >>>> == Source and Intellectual Property Submission Plan == > >>>> Indiana University is the current holder of Intellectual Property > >>>> rights for the software. The university has approved the code donation > >>>> and signed trustees approval, Corporate Contributor Licence Agreement > >>>> and Software Grant Agreement have been emailed to ASF secretary and > >>>> received acknowledgement. > >>>> > >>>> Specifically Indiana University will donate 4 components into Airavata > >>>> project. > >>>> > >>>> 1. XBaya Scientific Workflow Suite - includes a GUI for workflow > >>>> composition and monitoring. The composed workflow can be exported to > >>>> various workflow languages like BPEL, SCUFL, Condor DAG, Jython and > >>>> Java. The defacto workflow enacting engine used is Apache ODE. > >>>> 2. GFac - an application wrapper service that can be used to wrap > >>>> command line-driven science applications and make them into robust, > >>>> network- accessible services. This component is build on Axis2 web > >>>> service stack. > >>>> 3. XRegistry - a registry service for storing deployment information > >>>> about wrapped application services and constructed workflows. > >>>> 4. WS-Messenger - a “publish-subscribe†based message broker > >>>> implemented on top of Apache Axis2 web services stack. It implements > the > >>>> WS-Eventing and WS-Notifications specifications and incorporates a > >>>> message box component that facilities communications with clients > behind > >>>> firewalls and overcomes network glitches. > >>>> > >>>> == External Dependencies == > >>>> > >>>> Following the guideline -http://www.apache.org/legal/resolved.html, > the > >>>> following are the dependent software and all of them are in binary > >>>> format in java archive (jar files). > >>>> > >>>> * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API > >>>> * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, xmpp, > >>>> xsul, sigiri, atomixmiser, weps-beans. > >>>> * BSD: puretls, > >>>> * MIT: bcporv, hsqldb, dom4j, slf4j > >>>> * PSFL: Jython > >>>> * GPL 2.0: mysql-connector-java > >>>> * Other: > >>>> * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT) > >>>> * backport (public domain) > >>>> * jaxen (http://jaxen.codehaus.org/license.html) > >>>> > >>>> Licence incompatibilities (GPL) will be resolved during incubation. > >>>> > >>>> == Cryptography == > >>>> The software does not implement any cryptographic algorithms. However, > >>>> to perform secured messaging and data movement and SSL communications, > >>>> the software depends upon third party security libraries. These > >>>> external libraries depend in turn on Java Security, Puretls, Cryptix > >>>> and Bounce Castle libraries. Apache Cryptographic steps will be > >>>> followed to register the use of these libraries. > >>>> > >>>> == Required Resources == > >>>> > >>>> === Mailing lists === > >>>> 1. airavata-dev > >>>> 2. airavata-commits > >>>> 3. airavata-private > >>>> > >>>> === Subversion Directory === > >>>> https://svn.apache.org/repos/asf/incubator/airavata > >>>> > >>>> === Issue Tracking === > >>>> We intend to make use of Jira for issue tracking. Proposed key: > AIRAVATA > >>>> > >>>> === Other Resources === > >>>> > >>>> We intend to manage our website using the Apache CMS. > >>>> > >>>> == Initial Committers == > >>>> Names of initial committers with affiliation and current ASF status: > >>>> > >>>> || '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' || > >>>> '''ASF Status''' || '''Apache Id''' || > >>>> || Suresh Marru || sma...@cs.indiana.edu || Indiana University || On > >>>> File || Apache Commiter || smarru || > >>>> || Marlon Pierce || mpie...@cs.indiana.edu || Indiana University || > On > >>>> File || Apache Commiter || mpierce || > >>>> || Srinath Perera || hemap...@apache.org || Lanka Software Foundation > || > >>>> On File || Apache Member || hemapani || > >>>> || Aleksander Slominski || aslom at us.ibm.com || IBM || On File || > >>>> Apache Member || aslom || > >>>> || Raminderjeet Singh || rami...@indiana.edu || Indiana University || > On > >>>> File || Apache Commiter || raminder || > >>>> || Archit Kulshrestha || akuls...@indiana.edu || Indiana University > || > >>>> On File || N/A || N/A || > >>>> || Chathura Herath || chath...@apache.org || Indiana University || On > >>>> File || Apache Commiter || chathura || > >>>> || Eran Chinthaka || chinth...@apache.org || Indiana University || On > >>>> File || Apache Member || chinthaka || > >>>> || Thilina Gunaratne || thil...@apache.org || Indiana University || > On > >>>> File || Apache Commiter || thilina || > >>>> || Wathsala Vithanage || waths...@opensource.lk || Lanka Software > >>>> Foundation || On File || N/A || N/A || > >>>> > >>>> All the parties are affiliated with companies and organizations that > >>>> are familiar with the development of open source. We expect that the > >>>> amount of volunteer work will increase, and more developers will come > >>>> on board. > >>>> > >>>> == Champion == > >>>> Ross Gardler, Apache Software Foundation > >>>> > >>>> == Nominated Mentors == > >>>> * Ross Gardler, Member, Apache Software Foundation > >>>> * Alek Slominski, Member, Apache Software Foundation > >>>> * Ate Douma, Member, Apache Software Foundation > >>>> * Sanjiva Weerawarna, Member, Apache Software Foundation > >>>> * Paul Fremantle, Member, Apache Software Foundation > >>>> > >>>> == Sponsoring Entity == > >>>> Apache Incubator Project. > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > >>>> For additional commands, e-mail: general-h...@incubator.apache.org > >>>> > -- ----------------------------------------------------- Regards, Waruna Ranasinghe blog: http://warunapw.blogspot.com twitter: http://twitter.com/warunapww http://lk.linkedin.com/in/waruna www.facebook.com/waruna.ranasinghe
