date:20130603

Hi Henry,

Thanks for your support! I will leave it up to Matei and
the incoming Spark community to decide if they would like
to add you (or anyone else) to the wiki as a contributor
on the project.

Thanks!

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Henry Saputra henry.sapu...@gmail.com
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 12:38 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

Wow! I have been using Shark, which runs on top of Shark, with Mesos in
our
prototype for API analytics for a while and would LOVE to help as mentor
and initial contributors.


- Henry



On Fri, May 31, 2013 at 11:03 AM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Folks,

 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal

 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.

 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.

 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more. People
interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.

 Questions -- let's hear em'! :)

 Cheers,
 Chris
 (Champion, incoming Apache Spark)

 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.

 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark¹s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.

 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.

 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.

 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted multiple major patches
from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
 exists in which individuals who contribute major components act as
 maintainers for those modules. If

Re: [PROPOSAL] Apache Spark for the Incubator

Thanks for the support, Pei. I think the questions you had
about frameworks/etc., hopefully were answered.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Chen, Pei pei.c...@childrens.harvard.edu
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 11:45 AM
To: general@incubator.apache.org general@incubator.apache.org
Subject: RE: [PROPOSAL] Apache Spark for the Incubator

+1 (non-binding)
This seems like a really interesting project.
Q- Is Spark just a framework/API or does it also have some tools
implemented for data analytics?
--Pei

 -Original Message-
 From: Mattmann, Chris A (398J) [mailto:chris.a.mattm...@jpl.nasa.gov]
 Sent: Friday, May 31, 2013 2:04 PM
 To: general@incubator.apache.org
 Subject: [PROPOSAL] Apache Spark for the Incubator
 
 Hi Folks,
 
 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal
 
 The work originates from the Berkeley AMPLab and through a number of
 industry participants, and other institutions. Spark is a framework for
large-
 scale data analysis on clusters, with a particular focus on low latency
 operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings in
 various programming languages.
 
 The proposal text is copied to the bottom of this email. I'm going to
leave this
 thread open for the next week for discussion. Once it's died down, I'll
call an
 official VOTE.
 
 Suresh, Ross G. -- heads up -- this project may be of interest to you
both and
 would welcome you guys as additional mentors. We currently have 3
 mentors committed to the project, but would love to have more. People
 interested in contributing should declare their interest here on the
 general@incubator thread and those potential contributors will be
discussed
 by the incoming Spark community.
 
 Questions -- let's hear em'! :)
 
 Cheers,
 Chris
 (Champion, incoming Apache Spark)
 
 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.
 
 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
analysis.
 Spark provides a general purpose runtime that supports low-latency
 execution in several forms. These include interactive exploration of
very
 large datasets, near real-time stream processing, and ad-hoc SQL
analytics
 (through higher layer extensions). Spark interfaces with HDFS, HBase,
 Cassandra and several other storage storage layers, and exposes APIs in
 Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to
efficiently run
 machine learning algorithms on large datasets. Over time, it has
evolved into
 a general computing engine as outlined above. Spark¹s developer
community
 has also grown to include additional institutions, such as universities,
 research labs, and corporations. Funding has been provided by various
 institutions including the U.S. National Science Foundation, DARPA, and
a
 number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.
 
 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
 would be a great fit. Spark is a natural fit for the Apache foundation:
Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.
 
 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
Apache
 and integrate with the Apache development process. Furthermore, we plan
 for incremental development, and releases along with the Apache
 guidelines.
 
 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
Spark
 has several developers and has accepted multiple major patches from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
exists in
 which individuals who contribute major components act as maintainers for
 those

Re: [PROPOSAL] Apache Spark for the Incubator

Thanks for the support Roman!

I will leave it up to the incoming Spark community members to
decide if they need more mentors and we'll be in touch.

Thank you again.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Roman Shaposhnik r...@apache.org
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 3:25 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

Extremely enthusiastic +1!!!

If you ever need help with mentorship -- please let me know.

Also, looking forward to seeing this in Bigtop!

Thanks,
Roman.

On Fri, May 31, 2013 at 11:03 AM, Mattmann, Chris A (398J)
chris.a.mattm...@jpl.nasa.gov wrote:
 Hi Folks,

 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal

 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.

 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.

 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more. People
interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.

 Questions -- let's hear em'! :)

 Cheers,
 Chris
 (Champion, incoming Apache Spark)

 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.

 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark¹s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.

 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.

 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.

 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted multiple major patches
from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
 exists in which individuals who contribute major components act as
 maintainers for those modules. If accepted, the Spark project would
 include several of these participants as

Re: [VOTE] Accept Apache BeanShell in the Incubator

+1 (binding).

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Simone Tripodi simonetrip...@apache.org
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 24, 2013 12:23 AM
To: general@incubator.apache.org general@incubator.apache.org
Subject: [VOTE] Accept Apache BeanShell in the Incubator

Dear ASF members,

We would like to propose BeanShell for the incubator.

The proposal draft is available at:
https://wiki.apache.org/incubator/BeanShellProposal,
follows below the proposal

Open is open for at least 72h and closes approximately on May 27th at
8:20am GMT

[ ] +1 accept BeanShell in the Incubator
[ ] +/-0
[ ] -1 because (provide a reason)

Many thanks in advance, all the best!
-Simo

http://people.apache.org/~simonetripodi/
http://simonetripodi.livejournal.com/
http://twitter.com/simonetripodi
http://www.99soft.org/

~~
~

= BeanShell =
== Abstract ==
The following proposal is about BeanShell, see JSR-274: The BeanShell
Scripting Language implementation.

== Proposal ==
BeanShell is a small, free, embeddable Java source interpreter with
object scripting language features, written in Java. BeanShell
dynamically executes standard Java syntax and extends it with common
scripting conveniences such as loose types, commands, and method
closures like those in Perl and JavaScript.
Users can use BeanShell interactively for Java experimentation and
debugging as well as to extend your applications in new ways.
Scripting Java lends itself to a wide variety of applications
including rapid prototyping, user scripting extension, rules engines,
configuration, testing, dynamic deployment, embedded systems, and even
Java education.
BeanShell is small and embeddable, so users can call BeanShell from
Java applications to execute Java code dynamically at run-time or to
provide extensibility in applications. Alternatively, users can use
standalone BeanShell scripts to manipulate Java applications; working
with Java objects and APIs dynamically. Since BeanShell is written in
Java and runs in the same VM as application, users can freely pass
references to live objects into scripts and return them as results.

== Background ==
BeanShell is a long living project born in the 2000 thanks to Patrick
Niemeyer initial effort, who is still maintaining the project, with
the help of Daniel Leuck and contributions voluntarily sent by users.

== Rationale ==
Currently there are no projects hosted by the ASF focused on providing
JSR-274 implementation, moving the existing BeanShell project under
the Apache umbrella would mean the ASF provides the JSR-274 reference
implementation.

= Current Status =
== Meritocracy ==
The historical BeanShell team believes in meritocracy and always acted
as a community. Mailing list, open issue tracker and other
communication channels have always been adopted since its first
release. The adoption in a larger community, such as Apache, is the
natural evolution for BeanShell. Moreover, the Apache standards will
enforce the existing BeanShell community practices and will be a
foundation for future committers involvement.

== Core Developers ==
In alphabetical order:

 * Daniel Leuck dan at ikayzo dot com,
 * Patrick Niemeyer pat at ikayzo dot com
 * Pedro Giffuni pfg at apache dot org
 * Simone Tripodi simonetripodi at apache dot org

== Alignment ==
Main aim of the project is to develop and maintain a fully flavored
JSR-274 implementation that can be used by other Apache projects that
need a Java  Scripting Language.

= Known Risks =
== Orphaned Products ==
The increasing number of BeanShell adopters and the raising interest
for the JSR-274 technology let us believe that there is a minimal risk
for this work to being abandoned from the community.

Moreover, BeanShell has been already used by the following projects for
years:

 * Apache OpenOffice
 * Apache Maven
 * Apache JMeter

== Inexperience with Open Source ==
All of the committers have experience working in one or more open
source projects inside and outside ASF.

== Homogeneous Developers ==
The list of initial committers are geographically distributed across
the world with no one company being associated with a majority of the
developers.  Many of these initial developers are experienced Apache
committers already  and all are experienced with working in
distributed development communities.

== Reliance on

Re: [PROPOSAL] Apache Spark for the Incubator

Thanks Suresh, after conferring with the incoming Spark community
members, I will add you as a mentor on the wiki.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Suresh Marru sma...@apache.org
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Saturday, June 1, 2013 9:12 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

On May 31, 2013, at 2:03 PM, Mattmann, Chris A (398J)
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Folks,
 
 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal
 
 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data 
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.
 
 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.
 
 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more.

Thanks Chris for the alert. Great proposal indeed, if the podling needs
help I am in.

Suresh


 People interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.
 
 Questions -- let's hear em'! :)
 
 Cheers,
 Chris
 (Champion, incoming Apache Spark)
 
 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.
 
 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark¹s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.
 
 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.
 
 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.
 
 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted multiple major patches
from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
 exists in which individuals who contribute major components act as
 maintainers for those modules. If accepted, the Spark project would
 include several of these participants as committers from the onset. We
 will work to identify all committers and PPMC members for the project
and
 to

Re: [PROPOSAL] Apache Spark for the Incubator

Hi Konstantin,

Thanks for your kind words and expressed interest. I will leave it
to Matei and the incoming Spark community members to comment on adding
you (or anyone else) as a contributor to the wiki. If they are OK with
it, then I am very much too.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Konstantin Boudnik c...@apache.org
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 12:29 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

Great news!

Definitely +1 (non-binding, I guess) on adding Spark to the family
of ASF project!

I also express the interest to contribute to the project and move it
forward
to the graduation! Bigtop has been packaging and providing Spark as a
part of
Hadoop 1.x software stacks for some time; and hopefully would be able to
offer
it as a part of Hadoop 2.x line in the coming days.

Dr. Konstantin Boudnik
  Hadoop committer
  BigTop PMC

On Fri, May 31, 2013 at 06:03PM, Mattmann, Chris A (398J) wrote:
 Hi Folks,
 
 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal
 
 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data 
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.
 
 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.
 
 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more. People
interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.
 
 Questions -- let's hear em'! :)
 
 Cheers,
 Chris
 (Champion, incoming Apache Spark)
 
 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.
 
 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark╧s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.
 
 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.
 
 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.
 
 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted

Redirect of new podling urls

2013-06-03 Thread Christian Grobmeier

Hello

http://incubator.apache.org/guides/graduation.html#life-after-graduation
says we should use .htaccess to redirect incubator.apache.org/$foo to
$foo.apache.org after graduation. Now we have
$foo.incubator.apache.org.

Is it still the same?

Redirect Permanent http://onami.incubator.apache.org http://onami.apache.org

Not sure if this would work, I just know it with relative path. Does
anybody know?
This would be a good improvement for the automatic tld creation btw

Cheers

--
http://www.grobmeier.de
https://www.timeandbill.de

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

[jira] [Commented] (PODLINGNAMESEARCH-33) Establish whether Apache JSPWiki is a suitable name

2013-06-03 Thread JIRA


[ 
https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-33?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13673411#comment-13673411
 ] 

Juan Pablo Santos Rodríguez commented on PODLINGNAMESEARCH-33:
--

Hi Ant,

thanks for the advice, will contact the trademark team later tonight


br,
juan pablo

 Establish whether Apache JSPWiki is a suitable name 
 --

 Key: PODLINGNAMESEARCH-33
 URL: 
 https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-33
 Project: Podling Suitable Names Search
  Issue Type: Suitable Name Search
Reporter: Juan Pablo Santos Rodríguez

 We have to do some investigations to ensure that Apache JSPWiki is a 
 suitable name for a TLP. Here are some resources related to this issue:
 http://incubator.apache.org/guides/names.html
 http://www.apache.org/foundation/marks/
 Apache JSPWiki is a Java-based WikiWiki engine, feature-rich and built around 
 standard J2EE components (Java, servlets, JSP). 
 A similar search was perfomed 4 years ago, so this issue is mostly an update 
 on that JIRA. cfr with: https://issues.apache.org/jira/browse/JSPWIKI-547

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On 3 June 2013 05:10, Greg Stein gst...@gmail.com wrote:
 On Sun, Jun 2, 2013 at 11:08 PM, Alan Cabrera l...@toolazydogs.com wrote:
...

 Apache Incubator tools
 Copyright 2013 The Apache Software Foundation

 This product includes software developed at
 The Apache Software Foundation (http://www.apache.org/).

 This distribution contains code from the LinkedIn Corporation:

 Copyright (c) 2013 LinkedIn Corp. All rights reserved.
 Licensed under the Apache License, Version 2.0


 WDYT?

 Looks good. And given that the LinkedIn stuff is under ALv2, then 4(d)
 comes into play, requesting attribution. It is Right and Proper that
 they get a couple lines in our NOTICE file.

I'm not convinced of that.
If their attribution request is not in the NOTICE file, does it have
to go in ours?

Attribution could be added in a CREDITS file as per

 https://svn.apache.org/repos/asf/cocoon/branches/BRANCH_2_1_X/CREDITS.txt

 Note that we may even need to incorporate portions of *their* NOTICE

Indeed, see http://www.apache.org/dev/licensing-howto.html#mod-notice

 file. (I don't know the extent of inclusion/derivation, but ALv2 4(d)
 is pretty straight-forward)

 Cheers,
 -g

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

2013-06-03 Thread Alan Cabrera


On Jun 3, 2013, at 2:23 PM, sebb seb...@gmail.com wrote:

 On 3 June 2013 05:10, Greg Stein gst...@gmail.com wrote:
 On Sun, Jun 2, 2013 at 11:08 PM, Alan Cabrera l...@toolazydogs.com wrote:
 ...
 
 Apache Incubator tools
 Copyright 2013 The Apache Software Foundation
 
 This product includes software developed at
 The Apache Software Foundation (http://www.apache.org/).
 
 This distribution contains code from the LinkedIn Corporation:
 
Copyright (c) 2013 LinkedIn Corp. All rights reserved.
Licensed under the Apache License, Version 2.0
 
 
 WDYT?
 
 Looks good. And given that the LinkedIn stuff is under ALv2, then 4(d)
 comes into play, requesting attribution. It is Right and Proper that
 they get a couple lines in our NOTICE file.
 
 I'm not convinced of that.
 If their attribution request is not in the NOTICE file, does it have
 to go in ours?
 
 Attribution could be added in a CREDITS file as per
 
 https://svn.apache.org/repos/asf/cocoon/branches/BRANCH_2_1_X/CREDITS.txt

A credits file does not seem to be the appropriate place for a copyright 
declaration.


Regards,
Alan


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On Mon, Jun 3, 2013 at 5:23 PM, sebb seb...@gmail.com wrote:
 On 3 June 2013 05:10, Greg Stein gst...@gmail.com wrote:
 On Sun, Jun 2, 2013 at 11:08 PM, Alan Cabrera l...@toolazydogs.com wrote:
...

 Apache Incubator tools
 Copyright 2013 The Apache Software Foundation

 This product includes software developed at
 The Apache Software Foundation (http://www.apache.org/).

 This distribution contains code from the LinkedIn Corporation:

 Copyright (c) 2013 LinkedIn Corp. All rights reserved.
 Licensed under the Apache License, Version 2.0


 WDYT?

 Looks good. And given that the LinkedIn stuff is under ALv2, then 4(d)
 comes into play, requesting attribution. It is Right and Proper that
 they get a couple lines in our NOTICE file.

 I'm not convinced of that.
 If their attribution request is not in the NOTICE file, does it have
 to go in ours?

Maybe, maybe not. Why be so concerned? Let's be good citizens and
recognize their contribution in our NOTICE file. It doesn't hurt
anybody.

If they have their own NOTICE file, which gets incorporated into a
subdir of *our* distro, then we can/should reference it from our
NOTICE file.

I'd rather that we be nice by default.

Cheers,
-g

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On 3 June 2013 22:40, Greg Stein gst...@gmail.com wrote:
 On Mon, Jun 3, 2013 at 5:23 PM, sebb seb...@gmail.com wrote:
 On 3 June 2013 05:10, Greg Stein gst...@gmail.com wrote:
 On Sun, Jun 2, 2013 at 11:08 PM, Alan Cabrera l...@toolazydogs.com wrote:
...

 Apache Incubator tools
 Copyright 2013 The Apache Software Foundation

 This product includes software developed at
 The Apache Software Foundation (http://www.apache.org/).

 This distribution contains code from the LinkedIn Corporation:

 Copyright (c) 2013 LinkedIn Corp. All rights reserved.
 Licensed under the Apache License, Version 2.0


 WDYT?

 Looks good. And given that the LinkedIn stuff is under ALv2, then 4(d)
 comes into play, requesting attribution. It is Right and Proper that
 they get a couple lines in our NOTICE file.

 I'm not convinced of that.
 If their attribution request is not in the NOTICE file, does it have
 to go in ours?

 Maybe, maybe not. Why be so concerned? Let's be good citizens and
 recognize their contribution in our NOTICE file. It doesn't hurt
 anybody.

 If they have their own NOTICE file, which gets incorporated into a
 subdir of *our* distro, then we can/should reference it from our
 NOTICE file.

 I'd rather that we be nice by default.

I think that conflicts with the advice here:

http://www.apache.org/dev/licensing-howto.html#mod-notice

The last sentence says: Do not add anything to NOTICE which is not
legally required.

Being legal and being nice may not be completely disjoint but they are
certainly not identical.

 Cheers,
 -g

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1489076 - in /incubator/public/trunk/tools: src/asf/incubator/podlings.py tests/test_podlings.py

On 3 June 2013 18:43,  a...@apache.org wrote:
 Author: adc
 Date: Mon Jun  3 17:43:51 2013
 New Revision: 1489076

 URL: http://svn.apache.org/r1489076
 Log:
 Minor sketch of some podling code before I forget to check it in.

And perhaps add some comments explaining what the purpose is?

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On Mon, Jun 3, 2013 at 5:45 PM, sebb seb...@gmail.com wrote:
...
 I think that conflicts with the advice here:

 http://www.apache.org/dev/licensing-howto.html#mod-notice

 The last sentence says: Do not add anything to NOTICE which is not
 legally required.

 Being legal and being nice may not be completely disjoint but they are
 certainly not identical.

Geez. Come on. Now you're just being a dick.

Let me turn it around: why should it NOT go in there? Why is it not
legally required?

-g

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

As I said, it does look good. But I don't see the LinkedIn Copyright
in any files under tools/. The copyright/license header should remain
in the files that you copied into our repos. It is impossible to tell
what is from LinkedIn.

Cheers,
-g

On Sun, Jun 2, 2013 at 11:08 PM, Alan Cabrera l...@toolazydogs.com wrote:

On Jun 2, 2013, at 7:45 PM, Kevan Miller kevan.mil...@gmail.com wrote:

On Jun 2, 2013, at 10:09 PM, Alan Cabrera l...@toolazydogs.com wrote:

On Jun 2, 2013, at 5:29 PM, sebb seb...@gmail.com wrote:

On 3 June 2013 00:37, Alan Cabrera l...@toolazydogs.com wrote:

On Jun 2, 2013, at 4:01 PM, sebb seb...@gmail.com wrote:

Modified: incubator/public/trunk/tools/NOTICE.txt
URL:
http://svn.apache.org/viewvc/incubator/public/trunk/tools/NOTICE.txt?rev=1488735r1=1488734r2=1488735view=diff
==
--- incubator/public/trunk/tools/NOTICE.txt (original)
+++ incubator/public/trunk/tools/NOTICE.txt Sun Jun 2 16:21:24 2013
@@ -1,5 +1,21 @@
=
== NOTICE file corresponding to section 4(d) of the Apache License,
==
-== Version 1.0.0, in this case for the LiveTribe Utilities
==
+== Version 2.0.0, in this case for the ASF Incubator tools
==
== distribution.
==
=
+
+This distribution contains code from the LinkedIn Corporation:
+
+ Copyright (c) 2013 LinkedIn Corp. All rights reserved.
+
+ Licensed under the Apache License, Version 2.0 (the License);
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an AS IS BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.

That does not look like a standard NOTICE file.
The part between === should not be present, and the AL 2.0 should not
be there either.

Is the notice entry even required by the Linked In license?

Remember that NOTICE files are like poems - they are only correct when
nothing more can be removed!

Would this suffice:

begin
This distribution contains code from the LinkedIn Corporation:

IMO, no.

It's not a standard NOTICE file - the ASF header is missing.

Can you point to a canonical example for me to emulate?

How about the following:

http://www.apache.org/legal/src-headers.html#notice
and
http://www.apache.org/dev/licensing-howto.html

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

This distribution contains code from the LinkedIn Corporation:

WDYT?

Regards,
Alan

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On 4 June 2013 01:04, Greg Stein gst...@gmail.com wrote:
 On Mon, Jun 3, 2013 at 5:45 PM, sebb seb...@gmail.com wrote:
...
 I think that conflicts with the advice here:

 http://www.apache.org/dev/licensing-howto.html#mod-notice

 The last sentence says: Do not add anything to NOTICE which is not
 legally required.

 Being legal and being nice may not be completely disjoint but they are
 certainly not identical.


 Let me turn it around: why should it NOT go in there? Why is it not
 legally required?

Since the intention is to minimise the content of NOTICE, the onus
should be on showing that it must be in there rather showing why it
should not be there.

 -g

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: svn commit: r1488735 - in /incubator/public/trunk/tools: NOTICE.txt requirements.txt setup.py src/asf/utils/ src/asf/utils/init.py src/asf/utils/config.py src/asf/utils/file.py tests/test_conf

On Mon, Jun 3, 2013 at 8:15 PM, sebb seb...@gmail.com wrote:
 On 4 June 2013 01:04, Greg Stein gst...@gmail.com wrote:
 On Mon, Jun 3, 2013 at 5:45 PM, sebb seb...@gmail.com wrote:
...
 I think that conflicts with the advice here:

 http://www.apache.org/dev/licensing-howto.html#mod-notice

 The last sentence says: Do not add anything to NOTICE which is not
 legally required.

 Being legal and being nice may not be completely disjoint but they are
 certainly not identical.


 Let me turn it around: why should it NOT go in there? Why is it not
 legally required?

 Since the intention is to minimise the content of NOTICE, the onus
 should be on showing that it must be in there rather showing why it
 should not be there.

Already stated:

If their code is under ALv2, then the license says we must put their
copyright notice into our NOTICE file. Simple as that.

-g

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [PROPOSAL] Apache Spark for the Incubator

Hi Henry,

I've conferred with the incoming Spark community and we are
very happy to have you as a mentor on the project.
Please feel free to add yourself to the wiki.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Henry Saputra henry.sapu...@gmail.com
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 12:38 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

Wow! I have been using Shark, which runs on top of Shark, with Mesos in
our
prototype for API analytics for a while and would LOVE to help as mentor
and initial contributors.


- Henry



On Fri, May 31, 2013 at 11:03 AM, Mattmann, Chris A (398J) 
chris.a.mattm...@jpl.nasa.gov wrote:

 Hi Folks,

 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal

 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.

 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.

 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more. People
interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.

 Questions -- let's hear em'! :)

 Cheers,
 Chris
 (Champion, incoming Apache Spark)

 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.

 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark¹s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.

 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.

 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.

 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted multiple major patches
from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
 exists in which individuals who contribute major components act as
 maintainers for those modules. If accepted, the Spark project would
 include

Re: [PROPOSAL] Apache Spark for the Incubator

Hi Roman, I've conferred with the incoming Spark community and we are
happy to have you
as a mentor for the project.

Feel free to add yourself to the wiki proposal.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: Roman Shaposhnik r...@apache.org
Reply-To: general@incubator.apache.org general@incubator.apache.org
Date: Friday, May 31, 2013 3:25 PM
To: general@incubator.apache.org general@incubator.apache.org
Subject: Re: [PROPOSAL] Apache Spark for the Incubator

Extremely enthusiastic +1!!!

If you ever need help with mentorship -- please let me know.

Also, looking forward to seeing this in Bigtop!

Thanks,
Roman.

On Fri, May 31, 2013 at 11:03 AM, Mattmann, Chris A (398J)
chris.a.mattm...@jpl.nasa.gov wrote:
 Hi Folks,

 I'm pleased to bring you a proposal to the Apache Incubator for the
Apache
 Spark project: https://wiki.apache.org/incubator/SparkProposal

 The work originates from the Berkeley AMPLab and through a number of
 industry
 participants, and other institutions. Spark is a framework for
large-scale
 data
 analysis on clusters, with a particular focus on low latency operations.
 The
 source code is written in Scala, and provides a number of APIs and
bindings
 in various programming languages.

 The proposal text is copied to the bottom of this email. I'm going to
leave
 this thread open for the next week for discussion. Once it's died down,
 I'll
 call an official VOTE.

 Suresh, Ross G. -- heads up -- this project may be of interest to you
both
 and would welcome you guys as additional mentors. We currently have 3
 mentors
 committed to the project, but would love to have more. People
interested in
 contributing should declare their interest here on the general@incubator
 thread
 and those potential contributors will be discussed by the incoming Spark
 community.

 Questions -- let's hear em'! :)

 Cheers,
 Chris
 (Champion, incoming Apache Spark)

 === Abstract ===
 Spark is an open source system for large-scale data analysis on
clusters.

 === Proposal ===
 Spark is an open source system for fast and flexible large-scale data
 analysis. Spark provides a general purpose runtime that supports
 low-latency execution in several forms. These include interactive
 exploration of very large datasets, near real-time stream processing,
and
 ad-hoc SQL analytics (through higher layer extensions). Spark interfaces
 with HDFS, HBase, Cassandra and several other storage storage layers,
and
 exposes APIs in Scala, Java and Python.
 Background
 Spark started as U.C. Berkeley research project, designed to efficiently
 run machine learning algorithms on large datasets. Over time, it has
 evolved into a general computing engine as outlined above. Spark¹s
 developer community has also grown to include additional institutions,
 such as universities, research labs, and corporations. Funding has been
 provided by various institutions including the U.S. National Science
 Foundation, DARPA, and a number of industry sponsors. See:
 https://amplab.cs.berkeley.edu/sponsors/ for full details.

 === Rationale ===
 As the number of contributors to Spark has grown, we have sought for a
 long-term home for the project, and we believe the Apache foundation
would
 be a great fit. Spark is a natural fit for the Apache foundation: Spark
 already interoperates with several existing Apache projects (HDFS,
HBase,
 Hive, Cassandra, Avro and Flume to name a few). The Spark team is
familiar
 with the Apache process and and subscribes to the Apache mission - the
 team includes multiple Apache committers already. Finally, joining
Apache
 will help coordinate the development effort of the growing number of
 organizations which contribute to Spark.

 == Initial Goals ==
 The initial goals will most likely be to move the existing codebase to
 Apache and integrate with the Apache development process. Furthermore,
we
 plan for incremental development, and releases along with the Apache
 guidelines.

 === Current Status ===
 == Meritocracy ==
 The Spark project already operates on meritocratic principles. Today,
 Spark has several developers and has accepted multiple major patches
from
 outside of U.C. Berkeley. While this process has remained mostly
informal
 (we do not have an official committer list), an implicit organization
 exists in which individuals who contribute major components act as
 maintainers for those modules. If accepted, the Spark project would
 include several of these participants as

Re: [PROPOSAL] Apache Spark for the Incubator