I would like to propose Airavata for entry into the Apache Incubator.
The full proposal can be found at [1] and is copied at the end of this
mail. For those in a hurry here's a quick summary:
Airavata is a software toolkit currently used to build science gateways
but that has a much wider potential use. It provides features to
compose, manage, execute, and monitor large scale applications and
workflows on computational resources ranging from local clusters to
national grids and computing clouds. Users can use Airavata back end
services and build gadgets to deploy in open social containers such as
Apache Rave and modify them to suite their needs. Airavata builds on
general concepts of service oriented computing, distributed messaging,
and workflow composition and orchestration.
Airavata will provide web interfaces and scalable Service Oriented
Architecture based backend services to build or enhance Science Gateway
(see https://www.teragrid.org/web/science-gateways/) and similar
environments. Airavata will specifically focus on:
1. sophisticated server-side tools for registering and managing large
scale applications on computational resources.
2. graphical user interfaces to construct, execute, control, manage and
reuse of scientific workflows.
3. interfacing and interoperability with with various external (third
party) data and provenance management tools
The project team consists of a number of existing Apache Committers and
the code comes from the same stable as some of the code donated to
Apache Rave (Incubating).
We welcome your questions, suggestions, observations and support.
Ross
[1] http://wiki.apache.org/incubator/AiravataProposal
FULL PROPOSAL TEXT
==================
= Airavata Proposal for Apache Incubator =
== Abstract ==
Airavata is a software toolkit currently used to build science
gateways but that has a much wider potential use. It provides features
to compose, manage, execute, and monitor large scale applications and
workflows on computational resources ranging from local clusters to
national grids and computing clouds. Users can use Airavata back end
services and build gadgets to deploy in open social containers such as
Apache Rave and modify them to suite their needs. Airavata builds on
general concepts of service oriented computing, distributed messaging,
and workflow composition and orchestration.
== Proposal ==
Airavata will provide web interfaces and scalable Service Oriented
Architecture based backend services to build or enhance Science
Gateway (see https://www.teragrid.org/web/science-gateways/)
and similar environments. Airavata will specifically focus on:
1. sophisticated server-side tools for registering and managing large
scale applications on computational resources.
2. graphical user interfaces to construct, execute, control, manage and
reuse of scientific workflows.
3. interfacing and interoperability with with various external (third
party) data and provenance management tools.
== Background ==
Working in close quarters with Apache Axis2 committers and inspired by
the true open source community driven software development of ASF,
Suresh Marru and Marlon Pierce have been pioneering the idea of a
Science Gateways software-based Apache project since late 2008. Many
Apache members have fostered these ideas and guided them to arrive at
this proposal.
Currently the software is a actively used in various science
gateways. But the tools are general purpose and build upon widely used
Apache tools like Axis2, ODE engine. The core team is motivated to
expand the community and build a community welcoming both synergistic
software components and also new usage scenarios.
It is perhaps worth noting that one of the three seed projects that
make up the Apache Rave (Incubating) project is also the product of
this same team and is derived from the same Science Gateways
community.
== Rationale ==
The nature of computational problems has evolved from simple desktop
calculations to complex, multidisciplinary activities that require the
monitoring and analysis of remote data streams, database and web
search and large ensembles of simulations. In the academic domain
Science Gateways have emerged to address these needs and have built
software platforms that provide a community of users with the ability
to easily solve computational problems within a specific domain. The
tools developed to support these gateways are potentially of value to
any organisation needing to perform complex computations. Gateways
provide a convenient interface to the underlying infrastrucure without
the need for a deep understanding of the intricacies that
infrastructure.
We summarize the rationale for choosing The Apache Software Foundation
(ASF) below. This is what we hope to gain from participating in the
ASF.
1. '''Broader impact''': our science gateway tool set is based on
Service Oriented Architecture principles, and it has always been our
goal to align our software with broader trends in the development of
software for distributed systems. Participating in the ASF provides a
concrete way to implement this idea. In particular, we have done
extensive work on the workflow systems, messaging, and application
management as Web services from the perspective of computational science
use cases (i.e., high failure rates, very long running jobs, dynamic
service creation, workflows not expressible as directed acyclic graphs,
etc). These requirements and our work to implement them have already had
direct impact on the Apache Axis 2 and Apache ODE projects. As an Apache
project, it is hoped that our community will have an enhanced
opportunity for collaboration and complementary development with Apache
Hadoop (for scientific application management), Apache QPID (for
messaging), Apache Rave (incubator - Open Social Container) and others.
It is our goal to expand our software’s usage beyond just science
gateways to the broader enterprise community.
2. '''Sustainability''': Science gateway software development (and
cyberinfrastructure software generally) is primarily funded in the US by
the National Science Foundation (NSF), so the long term sustainability
of software across funding cycles is a longstanding problem. The NSF is
attempting to solve this problem, and its vision for sustainable
software is described here:
http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating in the
ASF is our project’s vision for reaching software sustainability that
underpins the NSF CF21 vision. As a successful ASF project (after
incubation), we will have created a community led, rather than funding
led, environment for the development of our sotware. This community,
through our community engagement work and adoption of meritocratic
principles, will expand beyond our current core team and existing
project collaborations. This will greatly increase the chances that our
software will continue to grow and improve beyond the participation of
any individuals.
3. '''Maturity''': much of the software included in this proposal was
developed initially by graduate students as part of their Ph. D. work.
The Open Grid Computing Environment has devoted significant effort
(through salaried staff and volunteers from collaborating institutions)
to convert these research projects into mature, reliable, well-written,
packaged components. The code is currently hosted at SourceForge, but we
recognize the need to go beyond just the SourceForge support tools to
participate in a real community of software engineering experts. It is
our desire, through the Apache Incubator, to take our software
engineering efforts to a higher level by learning from the substantial
experience of appropraite Apache Committers. Apache mentors will provide
initial guidance, as will the attraction of additional committers from
the relevant Apache projects.
== Initial Goals ==
* Implement a standalone version of the code base with a simple hello
world service, workflow and gadget(s) to access the examples.
* Migration of documentation and design knowledge from existing SF project
* Re-architect Grid based security (GSI) dependencies and adopt more
general purpose security implementations.
* Make sure Cloud (including hadoop) support is more first class.
* Aim to have the first Apache release within the first 6 months
* Verify with Apache Legal that some of the more esoteric licences in
our dependencies are acceptable, or replace them as appropriate
== Current Status ==
The proposed tools are currently hosted on SourceForge at
http://sourceforge.net/projects/ogce/ (source at
https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and are
described at http://www.collab-ogce.org.
== Meritocracy ==
A significant portion of initial committers are already ASF
Committers/Members,
and the entire team is well experienced with open source software
development. The existing code base has resulted from
multi-institutional collaborative projects. The developers are well
aware of the Apache way and will honor the meritocracy policy of ASF
foundation.
== Community ==
To date our focus has been serving our immediate partners needs rather
than looking outwards in order to build a broader community with
diverse needs. Whilst the core team area likely to remain focussed on
the Science Gateways communities we are keen to welcome community
members from other disciplines.
== Core Developers ==
Our core developers consist of participants from academic,
not-for-profit and for-profit organisations. Many are already well
versed in The Apache Way.
Amongst our initial team we have one or more committers on the
following Apache top level projects; axis, geronimo, synapse, ws,
ws-pmc, ws-woden as well as Apache Rave (Incubating).
== Alignment ==
Airavata software is built upon Apache Projects like Axis2, ODE,
Rampart, Tomcat and Maven. We will try to closely align the project
with ODE to ensure BPEL workflow compatibility. We will align with
metadata management projects like Apache OODT. Web interfaces within
the Airavata software will be synergistically developed with Apache
Rave.
== Known Risks ==
=== Orphaned products ===
We acknowledge the need to seek project contributions outside the current
developers. The core team actively travels and conducts workshops and
tutorials at relevant academic conferences like Supercomputing, TeraGrid,
Collaborative Technologies Systems and SciDAC. Previous experiences
have showed that these tutorials and outreach efforts will bring in
community participation. The general strategy will be to encourage
users to be active in the community and develop patches and
contribute. Also, the core developers use the Airavata software in
multiple projects with a life span ranging from 2 to 10 years, so the
risk of orphaned products is very minimal.
Furthermore, by opening our doors to non-academic organisations
already adopting large scale computation related projects in the ASF
we hope to be able to build community beyond the proposing teams
Science Gateway interests.
=== Inexperience with Open Source ===
The core team is very familiar with open source practices. The
developers include existing Apache members who have long term experience
with
the Apache Way. The OGCE project has been an active
open source project in SourceForge since November 2006. We welcome the
new directions and are well prepared to follow the Apache way.
=== Homogenous Developers ===
We have a semi-distributed development environment distributed among
Indiana University and Lanka Software Foundation. We fully expect
contributions from the partnering science gateways adding to the
heterogeneous development.
=== Reliance on Salaried Developers ===
The core developers are self motivated on the project and also are
funded through various federal, state and endowment research
grants. Participation in these research efforts based on Airavata
software is mostly voluntary and above and beyond the requirements of
the salaried jobs.
The Open Gateway Computing project, from which the initial code
donation is sourced, is funded for the next 3 years and is mandated by
the funding guidelines to open source software development -
http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We
believe in the Airavata software capabilities and its vital role in
providing sustainable middleware for Science Gateways. Nevertheless,
the core team will actively build upon Airavata software and foster
developer community outside the current core.
=== Relationships with Other Apache Products ===
See “Alignment� above. Airavata is based on the concepts of Service
Oriented Architecture and all services run within Tomcat
container. The web services are based on Axis2. The orchestration of
the scientific workflows uses Orchestration Director Engine. The
software is built using Apache Maven.
=== An Excessive Fascination with the Apache Brand ===
The Apache brand would certainly help promote the software suite, but
gaining the brand is not the motivation for this project. Airavata is
being proposed to Apache because of the belief in Apache’s meritocracy
model for mentored, community-driven, open source software is the best
way to develop sustainable software. See “Rational� above. Most
importantly, The Apache Software Foundation will help us create an
institution-neutral contribution venue and will help us build a
long-standing community around Airavata to sustain and improve it
beyond the span of specific, targeted research grants.
== Documentation ==
Existing documentation is available from the OGCE wiki,
http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition,
there is abundance of presentation and self guided video tutorial
material. Effort will be put in to collect all this information into
meaningful documentation on the Apache websites.
== Initial Source ==
The initial source of the project is in SourceForge. The source is
available for anonymous check out from svn at
https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/
== Source and Intellectual Property Submission Plan ==
Indiana University is the current holder of Intellectual Property
rights for the software. The university has approved the code donation
and signed trustees approval, Corporate Contributor Licence Agreement
and Software Grant Agreement have been emailed to ASF secretary and
received acknowledgement.
Specifically Indiana University will donate 4 components into Airavata
project.
1. XBaya Scientific Workflow Suite - includes a GUI for workflow
composition and monitoring. The composed workflow can be exported to
various workflow languages like BPEL, SCUFL, Condor DAG, Jython and
Java. The defacto workflow enacting engine used is Apache ODE.
2. GFac - an application wrapper service that can be used to wrap
command line-driven science applications and make them into robust,
network- accessible services. This component is build on Axis2 web
service stack.
3. XRegistry - a registry service for storing deployment information
about wrapped application services and constructed workflows.
4. WS-Messenger - a “publish-subscribe� based message broker
implemented on top of Apache Axis2 web services stack. It implements the
WS-Eventing and WS-Notifications specifications and incorporates a
message box component that facilities communications with clients behind
firewalls and overcomes network glitches.
== External Dependencies ==
Following the guideline -http://www.apache.org/legal/resolved.html, the
following are the dependent software and all of them are in binary
format in java archive (jar files).
* CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API
* Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, xmpp,
xsul, sigiri, atomixmiser, weps-beans.
* BSD: puretls,
* MIT: bcporv, hsqldb, dom4j, slf4j
* PSFL: Jython
* GPL 2.0: mysql-connector-java
* Other:
* cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT)
* backport (public domain)
* jaxen (http://jaxen.codehaus.org/license.html)
Licence incompatibilities (GPL) will be resolved during incubation.
== Cryptography ==
The software does not implement any cryptographic algorithms. However,
to perform secured messaging and data movement and SSL communications,
the software depends upon third party security libraries. These
external libraries depend in turn on Java Security, Puretls, Cryptix
and Bounce Castle libraries. Apache Cryptographic steps will be
followed to register the use of these libraries.
== Required Resources ==
=== Mailing lists ===
1. airavata-dev
2. airavata-commits
3. airavata-private
=== Subversion Directory ===
https://svn.apache.org/repos/asf/incubator/airavata
=== Issue Tracking ===
We intend to make use of Jira for issue tracking. Proposed key: AIRAVATA
=== Other Resources ===
We intend to manage our website using the Apache CMS.
== Initial Committers ==
Names of initial committers with affiliation and current ASF status:
|| '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' ||
'''ASF Status''' || '''Apache Id''' ||
|| Suresh Marru || sma...@cs.indiana.edu || Indiana University || On
File || Apache Commiter || smarru ||
|| Marlon Pierce || mpie...@cs.indiana.edu || Indiana University || On
File || Apache Commiter || mpierce ||
|| Srinath Perera || hemap...@apache.org || Lanka Software Foundation ||
On File || Apache Member || hemapani ||
|| Aleksander Slominski || aslom at us.ibm.com || IBM || On File ||
Apache Member || aslom ||
|| Raminderjeet Singh || rami...@indiana.edu || Indiana University || On
File || Apache Commiter || raminder ||
|| Archit Kulshrestha || akuls...@indiana.edu || Indiana University ||
On File || N/A || N/A ||
|| Chathura Herath || chath...@apache.org || Indiana University || On
File || Apache Commiter || chathura ||
|| Eran Chinthaka || chinth...@apache.org || Indiana University || On
File || Apache Member || chinthaka ||
|| Thilina Gunaratne || thil...@apache.org || Indiana University || On
File || Apache Commiter || thilina ||
|| Wathsala Vithanage || waths...@opensource.lk || Lanka Software
Foundation || On File || N/A || N/A ||
All the parties are affiliated with companies and organizations that
are familiar with the development of open source. We expect that the
amount of volunteer work will increase, and more developers will come
on board.
== Champion ==
Ross Gardler, Apache Software Foundation
== Nominated Mentors ==
* Ross Gardler, Member, Apache Software Foundation
* Alek Slominski, Member, Apache Software Foundation
* Ate Douma, Member, Apache Software Foundation
* Sanjiva Weerawarna, Member, Apache Software Foundation
* Paul Fremantle, Member, Apache Software Foundation
== Sponsoring Entity ==
Apache Incubator Project.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
