Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread sebb
On 14 December 2013 03:13, Patrick Wendell pwend...@gmail.com wrote:
 Please vote on releasing the following candidate as Apache Spark
 (incubating) version 0.8.1.

 The tag to be voted on is v0.8.1-incubating (commit b87d31d):
 https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e

 The release files, including signatures, digests, etc can be found at:
 http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/

The source archive agrees with the tag, which is good.

However they both contain binaries, which is not good.
Third party jars should *not* be included in SCM nor in source releases.

 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/pwendell.asc

Where is the KEYS file?

 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-040/

 The documentation corresponding to this release can be found at:
 http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/

Not a blocker for the release artifacts themselves, but should be
fixed before any release announcement:

The download page

http://spark.incubator.apache.org/downloads.html

includes a link to the directory containing the sigs and hashes, but
does not link to the KEYS file.
Also it is normal to provide some instructions as to how to do the verification.

The website still invites users to sign up for the Spark 2013
conference which is over.

 For information about the contents of this release see:
 https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8

 A vote on this release has passed within the Spark PPMC [1].

 Please vote on releasing this package as Apache Spark 0.8.1-incubating!

 The vote is open until Tuesday, December 17th at 03:30 UTC and

s/until/until at least/

 passes if a majority of at least 3 +1 IPMC votes are cast.

 [ ] +1 Release this package as Apache Spark 0.8.1-incubating
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see
 http://spark.incubator.apache.org/

 [1]
 http://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201312.mbox/%3CCABPQxsuEYMn_JE0qEOcrt4J5-N1PJWgGcN7m0qzNefW7fsz2PA%40mail.gmail.com%3E

 (Note that at present the concluding message isn't shown due to lag in the
 mail archives.)

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Recruit a Champion and Java developers for a new proposal Busilet.

2013-12-14 Thread Neng Geng Huang
Hello community,

Please find below a proposal draft wish to submit to the ASF. The proposal
is only a draft and persons interesting to this project are welcome to
join.

I am new here and also need help for the proposal and the further
processing.

A brief introduction of IDTP, UTID, and Busilet could be found in
http://www.utid.org/

Thanks in advance

Best regards,

Huang

 Busilet Proposal ===

===Abstract
Busilet is a reference implementation of IDTP (Identifier Tracing Protocol)
and UTID (Universally Traceable Identifier), a new communication protocol
and an identification system, which were submitted to IETF as two
Internet-Drafts (http://tools.ietf.org/html/draft-huangng-idtp-01,
http://tools.ietf.org/html/draft-huangng-utid-01). The IDTP is an
application-level protocol for distributed and collaborative information
systems, potentially used in wide areas of computation, such as the
Internet of Things, cloud computing, pervasive computing, distributed
database, and remote procedure call.

===Proposal
I propose to move future development of Busilet to the Apache Software
Foundation in order to build a broader user and developer community.
Busilet had been under developing by me only and published as open software
in https://sourceforge.net/p/busilet. I hope that with the help of
developers all around the world, the system could become more powerful and
functional in wide areas of computation.

===Background
Busilet was originated from a project called Things Mail (tmail) System in
a small innovative company nearly three years ago. The tmail system require
full interoperation among operating system platforms, organizations
(express companies and delivery stations), physical things (mails and smart
mail boxes), and persons (senders, delivery men, and receivers). UTIDs were
designed to identify these various objects and IDTP is used as a
communication protocol to exchange message among them.
After the project finished, I recognized that the development prospects of
the UTID and IDTP so that continued to conduct further researches on them
during the last two years. I wrote specifications of UTID and IDTP while I
was developing Busilet to proof the feasibility and practicality of the
specifications. I had submitted the specifications to IETF as two Internet
Drafts. I realized that it is impossible for me alone to fulfill the task.
Therefore, I do hope more persons are involved in the task making
contributes to the computation world.

===Rationale
As the reference implementation of UTID and IDTP in Java language, Busilet
Project is developed for following reasons:
**To demonstrate the feasibility and practicality of the two
Internet-Drafts of UTID and IDTP.
**To find errors in the two Internet-Drafts and improve them.
**To provide a new communication protocol for public used in the Internet
of Things, cloud computing, pervasive computing, distributed database, and
remote procedure call.

===Recent Goals
The initial goals for Apache Busilet are:
**Improve the existing Busilet project by refactoring the codes.
**Implements all features defined in the two Internet-Drafts.
**Extends two features: session and encryption, which are already
implemented in the Busilet Project at present.
**Release a stable version of code.
**Rewrite development documents and user manuals.

===Current Status
Several versions of code of Busilet Project could be accessed in:
https://sourceforge.net/p/busilet
The latest version of Busilet Project is fully compatible with the two
Internet-Drafts. The early versions of Busilet Project have been
successfully applied in four real production projects.

===Meritocracy
The project is completely new and there is no person involved in it at
present.

===Community
Busilet will try to foster a diverse community that is open to everyone. It
is released under a Apache License 2.0 to encourage the maximum possible
adoption by all potential users and developers. The Busilet community
encourages suggestions and contributions from any potential user and
developers.

===Core Developers
The project is completely new and I am the only developer. I would like to
have more competent developers joining this project and have the project
leaded by an excellent leader.

===Alignment
The initial Busilet makes use of Jackson component and other components
such as org.json component. It is possible to use a better json component
to replace these components in the future.

===Known Risks
===Orphaned Products
There is a small risk of being orphaned. We will try to avoid it happens
from the beginning.

===Inexperience with Open Source
The Busilet has been treated as an open source project since its beginning.
While our experience with public open source is limited, we do not
anticipate difficulty in operating under Apache's development process.

===Homogeneous Developers

===Reliance on Salaried Developers

===Relationships with Other Apache Projects
This project has no know connections 

Re: [VOTE] Enable Release Checklist Experiment

2013-12-14 Thread Sergio Fernández

On 13/12/13 21:59, Marvin Humphrey wrote:

Please vote:

[ ] +1 Yes, apply the patch enabling the experiment.
[ ] -1 No, do not apply the patch enabling the experiment.


+1

--
Sergio Fernández
Senior Researcher
Knowledge and Media Technologies
Salzburg Research Forschungsgesellschaft mbH
Jakob-Haringer-Straße 5/3 | 5020 Salzburg, Austria
T: +43 662 2288 318 | M: +43 660 2747 925
sergio.fernan...@salzburgresearch.at
http://www.salzburgresearch.at

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Enable Release Checklist Experiment

2013-12-14 Thread Joseph Schaefer
+1

On Dec 13, 2013, at 11:31 PM, Henry Saputra henry.sapu...@gmail.com wrote:

 So it begins =)
 
 +1
 
 Thanks for leading the effort, Marvin
 
 - Henry
 
 
 On Fri, Dec 13, 2013 at 12:59 PM, Marvin Humphrey
 mar...@rectangular.com wrote:
 Greetings,
 
 As the next step in our ongoing efforts to reform the release voting process,
 I propose that we run an experiment allowing the PPMC members of select
 podlings to earn binding votes under limited circumstances by completing a
 release checklist.
 
 For participating podlings, the Incubator's release management guide...
 
http://incubator.apache.org/guides/releasemanagement.html
 
 ... would be supplanted by the following documents:
 
http://incubator.apache.org/guides/release_manifest.txt
http://incubator.apache.org/guides/release.html
 
 The scope of this VOTE is limited to approving the following patch to our
 policy page:
 
https://paste.apache.org/k4vJ
 
 Here is the patch content minus markup:
 
2013 Alternate Release Voting Process
 
Select podlings pre-cleared by a majority vote of the IPMC MAY
 participate in
an alternate release voting process:
 
Should a Podling decide it wishes to perform a release, the Podling SHALL
hold a vote on the Podling's dev list and create a permanently archived
Release Manifest as described in the Experimental Release Guide.  At least
three +1 votes from PPMC members are required (see the Apache Voting
Process page).  If the majority of PPMC votes is positive, then the 
 Podling
SHALL send a summary of that vote to the Incubator's general list and
formally request the Incubator PMC approve such a release.
 
Formal approval requires three binding +1 votes and more positive than
negative votes.  Votes cast by members of the Incubator PMC are always
binding.  For all releases after the first, votes cast by members
 of the PPMC
are binding if a Mentor approves the Release Manifest.
 
 Please note that the proposed change is both incremental and reversible:
 
 *   It is incremental because podlings must be opted in by vote of the IPMC 
 to
participate.
 *   It is reversible because once the experiment has run its course the
policy change can be reverted with zero impact through lazy consensus.
 
 Those who may have questions about the legitimacy of allowing binding votes
 from non-IPMC members should see this post from Roy Fielding:
 
http://s.apache.org/v7
 
 Please vote:
 
 [ ] +1 Yes, apply the patch enabling the experiment.
 [ ] -1 No, do not apply the patch enabling the experiment.
 
 This majority VOTE will run for 7 days and will close at 13:00 PST on Friday,
 December 20, 2013.  Votes cast by members of the Incubator PMC are binding.
 
 Here is my own +1.
 
 Marvin Humphrey
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Enable Release Checklist Experiment

2013-12-14 Thread Dave
+1


On Sat, Dec 14, 2013 at 9:11 AM, Joseph Schaefer joe_schae...@yahoo.comwrote:

 +1

 On Dec 13, 2013, at 11:31 PM, Henry Saputra henry.sapu...@gmail.com
 wrote:

  So it begins =)
 
  +1
 
  Thanks for leading the effort, Marvin
 
  - Henry
 
 
  On Fri, Dec 13, 2013 at 12:59 PM, Marvin Humphrey
  mar...@rectangular.com wrote:
  Greetings,
 
  As the next step in our ongoing efforts to reform the release voting
 process,
  I propose that we run an experiment allowing the PPMC members of select
  podlings to earn binding votes under limited circumstances by
 completing a
  release checklist.
 
  For participating podlings, the Incubator's release management guide...
 
 http://incubator.apache.org/guides/releasemanagement.html
 
  ... would be supplanted by the following documents:
 
 http://incubator.apache.org/guides/release_manifest.txt
 http://incubator.apache.org/guides/release.html
 
  The scope of this VOTE is limited to approving the following patch to
 our
  policy page:
 
 https://paste.apache.org/k4vJ
 
  Here is the patch content minus markup:
 
 2013 Alternate Release Voting Process
 
 Select podlings pre-cleared by a majority vote of the IPMC MAY
  participate in
 an alternate release voting process:
 
 Should a Podling decide it wishes to perform a release, the Podling
 SHALL
 hold a vote on the Podling's dev list and create a permanently
 archived
 Release Manifest as described in the Experimental Release Guide.  At
 least
 three +1 votes from PPMC members are required (see the Apache Voting
 Process page).  If the majority of PPMC votes is positive, then the
 Podling
 SHALL send a summary of that vote to the Incubator's general list and
 formally request the Incubator PMC approve such a release.
 
 Formal approval requires three binding +1 votes and more positive
 than
 negative votes.  Votes cast by members of the Incubator PMC are
 always
 binding.  For all releases after the first, votes cast by members
  of the PPMC
 are binding if a Mentor approves the Release Manifest.
 
  Please note that the proposed change is both incremental and reversible:
 
  *   It is incremental because podlings must be opted in by vote of the
 IPMC to
 participate.
  *   It is reversible because once the experiment has run its course the
 policy change can be reverted with zero impact through lazy
 consensus.
 
  Those who may have questions about the legitimacy of allowing binding
 votes
  from non-IPMC members should see this post from Roy Fielding:
 
 http://s.apache.org/v7
 
  Please vote:
 
  [ ] +1 Yes, apply the patch enabling the experiment.
  [ ] -1 No, do not apply the patch enabling the experiment.
 
  This majority VOTE will run for 7 days and will close at 13:00 PST on
 Friday,
  December 20, 2013.  Votes cast by members of the Incubator PMC are
 binding.
 
  Here is my own +1.
 
  Marvin Humphrey
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 
 
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
 


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Henry Saputra
Hi Sebb, thanks for the review.

When you said However they both contain binaries, which is not good.
were you talking about the spark-0.8.1-incubating-bin-* files ?

- Henry

On Fri, Dec 13, 2013 at 7:13 PM, Patrick Wendell pwend...@gmail.com wrote:
 Please vote on releasing the following candidate as Apache Spark
 (incubating) version 0.8.1.

 The tag to be voted on is v0.8.1-incubating (commit b87d31d):
 https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e

 The release files, including signatures, digests, etc can be found at:
 http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/

 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/pwendell.asc

 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-040/

 The documentation corresponding to this release can be found at:
 http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/

 For information about the contents of this release see:
 https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8

 A vote on this release has passed within the Spark PPMC [1].

 Please vote on releasing this package as Apache Spark 0.8.1-incubating!

 The vote is open until Tuesday, December 17th at 03:30 UTC and
 passes if a majority of at least 3 +1 IPMC votes are cast.

 [ ] +1 Release this package as Apache Spark 0.8.1-incubating
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see
 http://spark.incubator.apache.org/

 [1]
 http://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201312.mbox/%3CCABPQxsuEYMn_JE0qEOcrt4J5-N1PJWgGcN7m0qzNefW7fsz2PA%40mail.gmail.com%3E

 (Note that at present the concluding message isn't shown due to lag in the
 mail archives.)

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Marvin Humphrey
On Sat, Dec 14, 2013 at 10:37 AM, Henry Saputra henry.sapu...@gmail.com wrote:
 When you said However they both contain binaries, which is not good.
 were you talking about the spark-0.8.1-incubating-bin-* files ?

There seem to be compiled files in the source archive.

marvin@knut:~/spark $ tar -zxf spark-0.8.1-incubating.tgz
marvin@knut:~/spark $ cd spark-0.8.1-incubating
marvin@knut:~/spark/spark-0.8.1-incubating $ find . -print | grep .jar$
./assembly/lib/net/sf/py4j/py4j/0.7/py4j-0.7.jar
./core/src/test/resources/uncommons-maths-1.2.2.jar
./repl/lib/scala-jline.jar
./sbt/sbt-launch-0.11.3-2.jar
./streaming/lib/org/apache/kafka/kafka/0.7.2-spark/kafka-0.7.2-spark.jar
marvin@knut:~/spark/spark-0.8.1-incubating $

One option for addressing that issue would be to move all compiled
dependencies to an accompanying -deps archive.

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Enable Release Checklist Experiment

2013-12-14 Thread Andy Seaborne

+1

On 14/12/13 14:42, Dave wrote:

+1


On Sat, Dec 14, 2013 at 9:11 AM, Joseph Schaefer joe_schae...@yahoo.comwrote:


+1

On Dec 13, 2013, at 11:31 PM, Henry Saputra henry.sapu...@gmail.com
wrote:


So it begins =)

+1

Thanks for leading the effort, Marvin

- Henry


On Fri, Dec 13, 2013 at 12:59 PM, Marvin Humphrey
mar...@rectangular.com wrote:

Greetings,

As the next step in our ongoing efforts to reform the release voting

process,

I propose that we run an experiment allowing the PPMC members of select
podlings to earn binding votes under limited circumstances by

completing a

release checklist.

For participating podlings, the Incubator's release management guide...

http://incubator.apache.org/guides/releasemanagement.html

... would be supplanted by the following documents:

http://incubator.apache.org/guides/release_manifest.txt
http://incubator.apache.org/guides/release.html

The scope of this VOTE is limited to approving the following patch to

our

policy page:

https://paste.apache.org/k4vJ

Here is the patch content minus markup:

2013 Alternate Release Voting Process

Select podlings pre-cleared by a majority vote of the IPMC MAY
participate in
an alternate release voting process:

Should a Podling decide it wishes to perform a release, the Podling

SHALL

hold a vote on the Podling's dev list and create a permanently

archived

Release Manifest as described in the Experimental Release Guide.  At

least

three +1 votes from PPMC members are required (see the Apache Voting
Process page).  If the majority of PPMC votes is positive, then the

Podling

SHALL send a summary of that vote to the Incubator's general list and
formally request the Incubator PMC approve such a release.

Formal approval requires three binding +1 votes and more positive

than

negative votes.  Votes cast by members of the Incubator PMC are

always

binding.  For all releases after the first, votes cast by members
of the PPMC
are binding if a Mentor approves the Release Manifest.

Please note that the proposed change is both incremental and reversible:

*   It is incremental because podlings must be opted in by vote of the

IPMC to

participate.
*   It is reversible because once the experiment has run its course the
policy change can be reverted with zero impact through lazy

consensus.


Those who may have questions about the legitimacy of allowing binding

votes

from non-IPMC members should see this post from Roy Fielding:

http://s.apache.org/v7

Please vote:

[ ] +1 Yes, apply the patch enabling the experiment.
[ ] -1 No, do not apply the patch enabling the experiment.

This majority VOTE will run for 7 days and will close at 13:00 PST on

Friday,

December 20, 2013.  Votes cast by members of the Incubator PMC are

binding.


Here is my own +1.

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org




-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org







-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Patrick Wendell
 However they both contain binaries, which is not good.
 Third party jars should *not* be included in SCM nor in source releases.

These are not binary artifacts containing our project's code. They are
our build tool and immediate dependencies that are not published in
maven. I've looked around to find TL projects that also use sbt and
they also include the sbt jar in the source release. For instance
Apache Kafka does the same thing:

https://kafka.apache.org/downloads.html

 Where is the KEYS file?
The keys file is here:
https://dist.apache.org/repos/dist/release/incubator/spark/KEYS

This is where we put it based on the incubator release guidelines.
Were you actually asking where the file was or suggesting it is
required to include it in VOTE threads?

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Marvin Humphrey
On Sat, Dec 14, 2013 at 3:43 PM, Patrick Wendell pwend...@gmail.com wrote:
 However they both contain binaries, which is not good.
 Third party jars should *not* be included in SCM nor in source releases.

 These are not binary artifacts containing our project's code. They are
 our build tool and immediate dependencies that are not published in
 maven. I've looked around to find TL projects that also use sbt and
 they also include the sbt jar in the source release. For instance
 Apache Kafka does the same thing:

 https://kafka.apache.org/downloads.html

I appreciate your doing the research, and I understand why you might think
following Kafka's example is a reasonable approach.  However, that binary is a
problem for Kafka.  If Kafka's releases were like that when they graduated,
it's a failure of the Incubator as well.

Please read these messages from ASF Board member Roy Fielding:

http://s.apache.org/roy-binary-deps-0
http://s.apache.org/roy-binary-deps-1
http://s.apache.org/roy-binary-deps-2
http://s.apache.org/roy-binary-deps-3

This has to be fixed.  If some TLP PMCs have not been made aware that binary
dependencies may not be bundled in source releases, the Incubator must not
compound the problem by failing to educate current podlings.

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: IP Clearance before releasing

2013-12-14 Thread Marvin Humphrey
On Thu, Dec 12, 2013 at 12:20 AM, Upayavira u...@odoko.co.uk wrote:
 We should probably be clear about *who* can relax the rules, because
 again this could become a fighting ground amongst 180 of us.

Perhaps we can avert potential disputes by blocking graduation rather than
releases.

Every review item in the following checklist is either required by Incubator
policy or is documented outside the Incubator as required for all Apache
releases:

http://incubator.apache.org/guides/release_manifest.txt
http://incubator.apache.org/guides/release.html#review-items

Let's say that we start allowing more documented exceptions to the rules for
incubating releases.  Fine -- but the podling still needs to demonstrate that
they can make a release worthy of a TLP, or they shouldn't graduate.

The simple approach is to establish a policy that a podling must make a
release which passes all the checklist items.  I'm sure we'd end up making
occassional exceptions under the materiality test, but that's no big deal.

An audit prior to graduation has also been proposed, but forcing IPMC
members to go rooting around in source trees, build scripts and issue trackers
seems unattractive.

Blocking graduation instead of release candidates would reduce strain on both
podlings and reviewers.  Suitable policy bugs could be fixed in between
incubating releases during the normal course of development, rather than
between release candidates -- so fewer release candidates would be needed.

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Patrick Wendell
Hey Marvin,

Is this policy actually written up anywhere (along with best practices
on how to deal with this issue if you indeed have third party
dependencies)? I'm just asking because I don't see an obvious fix
for this based on the way Spark is built.

Second - this issue was not brought to our attention before - and in
particular was not raised during our 0.8.0 major release through the
incubator. Since this (0.8.1) release is a maintenance release, doing
a large change to the build system is not possible. It seems to me a
bit much to ask us to completely re-tool this entire project in order
for a simple maintenance release to pass. Especially since other top
level projects are clearly still employing this practice (not that
they are in-the-right, but just that this is a policy which it seems
is still being shaped).

We are planning to do our next major release (Spark 0.9.0) while still
under incubation in the next few weeks. Could I propose that we create
a parallel discussion about how we might re-tool or build process with
the aim towards satisfying whatever policy exists in that release?
We'll probably need guidance on that from people at Apache since,
again, there is no documented guidelines about what is allowed and
what isn't.

- Patrick

On Sat, Dec 14, 2013 at 8:20 PM, Marvin Humphrey mar...@rectangular.com wrote:
 On Sat, Dec 14, 2013 at 3:43 PM, Patrick Wendell pwend...@gmail.com wrote:
 However they both contain binaries, which is not good.
 Third party jars should *not* be included in SCM nor in source releases.

 These are not binary artifacts containing our project's code. They are
 our build tool and immediate dependencies that are not published in
 maven. I've looked around to find TL projects that also use sbt and
 they also include the sbt jar in the source release. For instance
 Apache Kafka does the same thing:

 https://kafka.apache.org/downloads.html

 I appreciate your doing the research, and I understand why you might think
 following Kafka's example is a reasonable approach.  However, that binary is a
 problem for Kafka.  If Kafka's releases were like that when they graduated,
 it's a failure of the Incubator as well.

 Please read these messages from ASF Board member Roy Fielding:

 http://s.apache.org/roy-binary-deps-0
 http://s.apache.org/roy-binary-deps-1
 http://s.apache.org/roy-binary-deps-2
 http://s.apache.org/roy-binary-deps-3

 This has to be fixed.  If some TLP PMCs have not been made aware that binary
 dependencies may not be bundled in source releases, the Incubator must not
 compound the problem by failing to educate current podlings.

 Marvin Humphrey

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Marvin Humphrey
On Sat, Dec 14, 2013 at 9:24 PM, Patrick Wendell pwend...@gmail.com wrote:
 We are planning to do our next major release (Spark 0.9.0) while still
 under incubation in the next few weeks. Could I propose that we create
 a parallel discussion about how we might re-tool or build process with
 the aim towards satisfying whatever policy exists in that release?

+1

That seems reasonable to me.

We're winging it a bit, since discussions about under what circumstances to
relax policy are ongoing (see the IP Clearance before releasing thread).
However, you've taken the initiative to make a constructive proposal.  We
should work with you.

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)

2013-12-14 Thread Patrick Wendell
Thanks Marvin and Joe,

Do you know which other projects/people we might reach out to for
institutional knowledge of how this is done?

Our specific issue is that while 99% of our dependencies are in maven,
there are a small # of dependencies that don't publish anywhere via
maven. One solution would be if apache let us publish these to maven
central - but I'm guessing Apache limits us to publishing o.i.a.XXX
artifacts.

- Patrick

On Sat, Dec 14, 2013 at 10:31 PM, Marvin Humphrey
mar...@rectangular.com wrote:
 On Sat, Dec 14, 2013 at 9:24 PM, Patrick Wendell pwend...@gmail.com wrote:
 We are planning to do our next major release (Spark 0.9.0) while still
 under incubation in the next few weeks. Could I propose that we create
 a parallel discussion about how we might re-tool or build process with
 the aim towards satisfying whatever policy exists in that release?

 +1

 That seems reasonable to me.

 We're winging it a bit, since discussions about under what circumstances to
 relax policy are ongoing (see the IP Clearance before releasing thread).
 However, you've taken the initiative to make a constructive proposal.  We
 should work with you.

 Marvin Humphrey

 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org