Re: [RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Patrick Wendell
Thanks everyone who contributed to this vote. I've created an IPMC vote here:

http://mail-archives.apache.org/mod_mbox/incubator-general/201309.mbox/%3CCABPQxsu6XwYMUxWKwRsXOvT%2B3-8%3DTTWwyHwJe_hWVb%3DNPxWuuw%40mail.gmail.com%3E

- Patrick

On Fri, Sep 20, 2013 at 4:39 PM, Roman Shaposhnik  wrote:
> On Fri, Sep 20, 2013 at 1:37 PM, Henry Saputra  
> wrote:
>> It is passing with Apache Spark podling/ community but we need extra
>> blessing from IPMCs via formal email to general@i.a.o list and in
>> particular we need one more IPMC vote.
>
> Correct, the release for a poddling can only happen once
> it passes on general@
>
> Like I said -- I intend to test the bits on a fully distributed cluster
> in Bigtop over the weekend and then I'll vote on general@
>
> Just to keep things clean, I personally, always try to make sure
> 3 IPMC votes are cast excplicitly on general@
>
> Thanks,
> Roman.


Re: [RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Roman Shaposhnik
On Fri, Sep 20, 2013 at 1:37 PM, Henry Saputra  wrote:
> It is passing with Apache Spark podling/ community but we need extra
> blessing from IPMCs via formal email to general@i.a.o list and in
> particular we need one more IPMC vote.

Correct, the release for a poddling can only happen once
it passes on general@

Like I said -- I intend to test the bits on a fully distributed cluster
in Bigtop over the weekend and then I'll vote on general@

Just to keep things clean, I personally, always try to make sure
3 IPMC votes are cast excplicitly on general@

Thanks,
Roman.


Re: [RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Henry Saputra
Thanks to Patrick for driving the first Apache Spark release. Great job so far.

A bit clarification, the release VOTE passes with more than 3 +1
binding votes from Apache Spark Podling Project Management (PPMC):

+1 (7 Total)
Andy Konwinski
Matei Zaharia
Patrick Wendell
Reynold Xin
Chris Mattmann*
Henry Saputra*

(* indicates IPMC)

Since Spark is under ASF incubator we need to send another VOTE to
general@i.a.o list.

>From the ASF release management page:
"
It is Apache policy that all releases be formally approved by the
responsible PMC. In the case of the incubator, the IPMC must approve
all releases. That means there is an additional bit of voting that the
release manager must now oversee on general@incubator in order to gain
that approval. The release manager must inform general@incubator that
the vote has passed on the podling's development list, and should
indicate any IPMC votes gained during that process. A new vote on the
release candidate artifacts must now be held on general@incubator to
seek majority consensus from the IPMC. Previous IPMC votes issued on
the project's development list count towards that goal. Even if there
are sufficient IPMC votes already, it is vital that the IPMC as whole
is informed via a VOTE e-mail on general@incubator.
"

We have 2 IPMCs Vote already so technically we need one more unless we
got veto votes against the release.

- Henry


On Fri, Sep 20, 2013 at 11:43 AM, Patrick Wendell  wrote:
> The vote is now closed. Below are the vote totals.
>
> +1 (7 Total)
> Andy Konwinski
> Matei Zaharia
> Patrick Wendell
> Konstantin Boudnik
> Reynold Xin
> Chris Mattmann*
> Henry Saputra*
>
> 0 (1 Total)
> Mark Hamstra
>
> -1 (0 Total)
>
> * = Binding Vote
>
> As per the incubator release guide [1] I'll be sending this to the
> general incubator list for a final vote from IPMC members.
>
> [1] 
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-vote
>
> - Patrick
>
> -- Forwarded message ------
> From: Roman Shaposhnik 
> Date: Fri, Sep 20, 2013 at 8:10 AM
> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
> To: dev@spark.incubator.apache.org
>
>
> On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell  wrote:
>> Hey Roman,
>>
>> We can do this in the future - I wasn't sure exactly what the right
>> standard approach was. Just so I understand, the change you are
>> proposing from what is there now is just to remove rcX from the
>> file-names, correct?
>
> Right. Basically your artifacts should look exactly like what
> is going to be released when the vote passes.
>
> Like I said -- it is a small nit, but it makes it easier for the
> guys like me to test the RCs in the automated manner.
>
> Thanks,
> Roman.


Re: [RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Henry Saputra
It is passing with Apache Spark podling/ community but we need extra
blessing from IPMCs via formal email to general@i.a.o list and in
particular we need one more IPMC vote.

- Henry

On Fri, Sep 20, 2013 at 1:35 PM, Patrick Wendell  wrote:
> Hey Henry,
>
> Sounds good. I'll send an email to general@ shortly. I didn't realize
> that this vote technically counts as passing according to those rules
> (since plenty of PPMC gave +1).
>
> On Fri, Sep 20, 2013 at 1:30 PM, Henry Saputra  
> wrote:
>> Thanks to Patrick for driving the first Apache Spark release. Great job so 
>> far.
>>
>> A bit clarification, the release VOTE passes with more than 3 +1
>> binding votes from Apache Spark Podling Project Management (PPMC):
>>
>> +1 (7 Total)
>> Andy Konwinski
>> Matei Zaharia
>> Patrick Wendell
>> Reynold Xin
>> Chris Mattmann*
>> Henry Saputra*
>>
>> (* indicates IPMC)
>>
>> Since Spark is under ASF incubator we need to send another VOTE to
>> general@i.a.o list.
>>
>> From the ASF release management page:
>> "
>> It is Apache policy that all releases be formally approved by the
>> responsible PMC. In the case of the incubator, the IPMC must approve
>> all releases. That means there is an additional bit of voting that the
>> release manager must now oversee on general@incubator in order to gain
>> that approval. The release manager must inform general@incubator that
>> the vote has passed on the podling's development list, and should
>> indicate any IPMC votes gained during that process. A new vote on the
>> release candidate artifacts must now be held on general@incubator to
>> seek majority consensus from the IPMC. Previous IPMC votes issued on
>> the project's development list count towards that goal. Even if there
>> are sufficient IPMC votes already, it is vital that the IPMC as whole
>> is informed via a VOTE e-mail on general@incubator.
>> "
>>
>> We have 2 IPMCs Vote already so technically we need one more unless we
>> got veto votes against the release.
>>
>> - Henry
>>
>>
>> On Fri, Sep 20, 2013 at 11:43 AM, Patrick Wendell  wrote:
>>> The vote is now closed. Below are the vote totals.
>>>
>>> +1 (7 Total)
>>> Andy Konwinski
>>> Matei Zaharia
>>> Patrick Wendell
>>> Konstantin Boudnik
>>> Reynold Xin
>>> Chris Mattmann*
>>> Henry Saputra*
>>>
>>> 0 (1 Total)
>>> Mark Hamstra
>>>
>>> -1 (0 Total)
>>>
>>> * = Binding Vote
>>>
>>> As per the incubator release guide [1] I'll be sending this to the
>>> general incubator list for a final vote from IPMC members.
>>>
>>> [1] 
>>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-vote
>>>
>>> - Patrick
>>>
>>> -- Forwarded message --
>>> From: Roman Shaposhnik 
>>> Date: Fri, Sep 20, 2013 at 8:10 AM
>>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>>> To: dev@spark.incubator.apache.org
>>>
>>>
>>> On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell  wrote:
>>>> Hey Roman,
>>>>
>>>> We can do this in the future - I wasn't sure exactly what the right
>>>> standard approach was. Just so I understand, the change you are
>>>> proposing from what is there now is just to remove rcX from the
>>>> file-names, correct?
>>>
>>> Right. Basically your artifacts should look exactly like what
>>> is going to be released when the vote passes.
>>>
>>> Like I said -- it is a small nit, but it makes it easier for the
>>> guys like me to test the RCs in the automated manner.
>>>
>>> Thanks,
>>> Roman.


Re: [RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Patrick Wendell
Hey Henry,

Sounds good. I'll send an email to general@ shortly. I didn't realize
that this vote technically counts as passing according to those rules
(since plenty of PPMC gave +1).

On Fri, Sep 20, 2013 at 1:30 PM, Henry Saputra  wrote:
> Thanks to Patrick for driving the first Apache Spark release. Great job so 
> far.
>
> A bit clarification, the release VOTE passes with more than 3 +1
> binding votes from Apache Spark Podling Project Management (PPMC):
>
> +1 (7 Total)
> Andy Konwinski
> Matei Zaharia
> Patrick Wendell
> Reynold Xin
> Chris Mattmann*
> Henry Saputra*
>
> (* indicates IPMC)
>
> Since Spark is under ASF incubator we need to send another VOTE to
> general@i.a.o list.
>
> From the ASF release management page:
> "
> It is Apache policy that all releases be formally approved by the
> responsible PMC. In the case of the incubator, the IPMC must approve
> all releases. That means there is an additional bit of voting that the
> release manager must now oversee on general@incubator in order to gain
> that approval. The release manager must inform general@incubator that
> the vote has passed on the podling's development list, and should
> indicate any IPMC votes gained during that process. A new vote on the
> release candidate artifacts must now be held on general@incubator to
> seek majority consensus from the IPMC. Previous IPMC votes issued on
> the project's development list count towards that goal. Even if there
> are sufficient IPMC votes already, it is vital that the IPMC as whole
> is informed via a VOTE e-mail on general@incubator.
> "
>
> We have 2 IPMCs Vote already so technically we need one more unless we
> got veto votes against the release.
>
> - Henry
>
>
> On Fri, Sep 20, 2013 at 11:43 AM, Patrick Wendell  wrote:
>> The vote is now closed. Below are the vote totals.
>>
>> +1 (7 Total)
>> Andy Konwinski
>> Matei Zaharia
>> Patrick Wendell
>> Konstantin Boudnik
>> Reynold Xin
>> Chris Mattmann*
>> Henry Saputra*
>>
>> 0 (1 Total)
>> Mark Hamstra
>>
>> -1 (0 Total)
>>
>> * = Binding Vote
>>
>> As per the incubator release guide [1] I'll be sending this to the
>> general incubator list for a final vote from IPMC members.
>>
>> [1] 
>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-vote
>>
>> - Patrick
>>
>> -- Forwarded message --
>> From: Roman Shaposhnik 
>> Date: Fri, Sep 20, 2013 at 8:10 AM
>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>> To: dev@spark.incubator.apache.org
>>
>>
>> On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell  wrote:
>>> Hey Roman,
>>>
>>> We can do this in the future - I wasn't sure exactly what the right
>>> standard approach was. Just so I understand, the change you are
>>> proposing from what is there now is just to remove rcX from the
>>> file-names, correct?
>>
>> Right. Basically your artifacts should look exactly like what
>> is going to be released when the vote passes.
>>
>> Like I said -- it is a small nit, but it makes it easier for the
>> guys like me to test the RCs in the automated manner.
>>
>> Thanks,
>> Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Henry Saputra
Hi Patrick,

Let's leave it for now. If someone veto against it the. We do the rename.

Thx,

Henry

On Friday, September 20, 2013, Patrick Wendell wrote:

> Henry - one thing is that, because the filenames are not included in
> the signatures, I could just alter the filenames now to not include
> -RCX... would that be preferable or would that necessitate another
> vote?
>
> - Patrick
>
> On Fri, Sep 20, 2013 at 6:39 AM, Henry Saputra 
> >
> wrote:
> > The RC should be just the directory where the artifact live but the final
> > name should omit the RCxx
> >
> > Hmm not sure if IPMCs will be picky about this but should not be blocker
> to
> > release.
> >
> > - Henry
> >
> > On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell
> > >
> > wrote:
> >> Hey Roman,
> >>
> >> We can do this in the future - I wasn't sure exactly what the right
> >> standard approach was. Just so I understand, the change you are
> >> proposing from what is there now is just to remove rcX from the
> >> file-names, correct?
> >>
> >> - Patrick
> >>
> >> On Thu, Sep 19, 2013 at 8:06 PM, Roman Shaposhnik 
> >> 
> >
> > wrote:
> >>> On Thu, Sep 19, 2013 at 5:56 PM, Patrick Wendell 
> >>> 
> >
> > wrote:
>  FYI this vote ends in 8 hours.
> >>>
> >>> I was going to test it on a fully distributed Bigtop cluster, but hit
> >>> a few snags. That now will extend into the weekend.
> >>>
> >>> Of course, that's not that big of a deal -- I can always vote on
> >>> incubator general once you guys move the vote over there.
> >>>
> >>> The only minor nit for the future I've noticed is that I would
> >>> highly encourage you to follow the usual RC practices where
> >>> you name all of your artifact as final bits and have a subdirectory
> >>> that reflects the RC name. E.g. here's how a very recent
> >>> Hadoop RC looks like:
> >>> http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0/
> >>>
> >>> Thanks,
> >>> Roman.
>


[RESULT] [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Patrick Wendell
The vote is now closed. Below are the vote totals.

+1 (7 Total)
Andy Konwinski
Matei Zaharia
Patrick Wendell
Konstantin Boudnik
Reynold Xin
Chris Mattmann*
Henry Saputra*

0 (1 Total)
Mark Hamstra

-1 (0 Total)

* = Binding Vote

As per the incubator release guide [1] I'll be sending this to the
general incubator list for a final vote from IPMC members.

[1] 
http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-vote

- Patrick

-- Forwarded message --
From: Roman Shaposhnik 
Date: Fri, Sep 20, 2013 at 8:10 AM
Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
To: dev@spark.incubator.apache.org


On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell  wrote:
> Hey Roman,
>
> We can do this in the future - I wasn't sure exactly what the right
> standard approach was. Just so I understand, the change you are
> proposing from what is there now is just to remove rcX from the
> file-names, correct?

Right. Basically your artifacts should look exactly like what
is going to be released when the vote passes.

Like I said -- it is a small nit, but it makes it easier for the
guys like me to test the RCs in the automated manner.

Thanks,
Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Roman Shaposhnik
On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell  wrote:
> Hey Roman,
>
> We can do this in the future - I wasn't sure exactly what the right
> standard approach was. Just so I understand, the change you are
> proposing from what is there now is just to remove rcX from the
> file-names, correct?

Right. Basically your artifacts should look exactly like what
is going to be released when the vote passes.

Like I said -- it is a small nit, but it makes it easier for the
guys like me to test the RCs in the automated manner.

Thanks,
Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Patrick Wendell
Henry - one thing is that, because the filenames are not included in
the signatures, I could just alter the filenames now to not include
-RCX... would that be preferable or would that necessitate another
vote?

- Patrick

On Fri, Sep 20, 2013 at 6:39 AM, Henry Saputra  wrote:
> The RC should be just the directory where the artifact live but the final
> name should omit the RCxx
>
> Hmm not sure if IPMCs will be picky about this but should not be blocker to
> release.
>
> - Henry
>
> On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell
> >
> wrote:
>> Hey Roman,
>>
>> We can do this in the future - I wasn't sure exactly what the right
>> standard approach was. Just so I understand, the change you are
>> proposing from what is there now is just to remove rcX from the
>> file-names, correct?
>>
>> - Patrick
>>
>> On Thu, Sep 19, 2013 at 8:06 PM, Roman Shaposhnik 
>> >
> wrote:
>>> On Thu, Sep 19, 2013 at 5:56 PM, Patrick Wendell 
>>> >
> wrote:
 FYI this vote ends in 8 hours.
>>>
>>> I was going to test it on a fully distributed Bigtop cluster, but hit
>>> a few snags. That now will extend into the weekend.
>>>
>>> Of course, that's not that big of a deal -- I can always vote on
>>> incubator general once you guys move the vote over there.
>>>
>>> The only minor nit for the future I've noticed is that I would
>>> highly encourage you to follow the usual RC practices where
>>> you name all of your artifact as final bits and have a subdirectory
>>> that reflects the RC name. E.g. here's how a very recent
>>> Hadoop RC looks like:
>>> http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0/
>>>
>>> Thanks,
>>> Roman.


[VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Henry Saputra
The RC should be just the directory where the artifact live but the final
name should omit the RCxx

Hmm not sure if IPMCs will be picky about this but should not be blocker to
release.

- Henry

On Thu, Sep 19, 2013 at 8:17 PM, Patrick Wendell
>
wrote:
> Hey Roman,
>
> We can do this in the future - I wasn't sure exactly what the right
> standard approach was. Just so I understand, the change you are
> proposing from what is there now is just to remove rcX from the
> file-names, correct?
>
> - Patrick
>
> On Thu, Sep 19, 2013 at 8:06 PM, Roman Shaposhnik 
> >
wrote:
>> On Thu, Sep 19, 2013 at 5:56 PM, Patrick Wendell 
>> >
wrote:
>>> FYI this vote ends in 8 hours.
>>
>> I was going to test it on a fully distributed Bigtop cluster, but hit
>> a few snags. That now will extend into the weekend.
>>
>> Of course, that's not that big of a deal -- I can always vote on
>> incubator general once you guys move the vote over there.
>>
>> The only minor nit for the future I've noticed is that I would
>> highly encourage you to follow the usual RC practices where
>> you name all of your artifact as final bits and have a subdirectory
>> that reflects the RC name. E.g. here's how a very recent
>> Hadoop RC looks like:
>> http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0/
>>
>> Thanks,
>> Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-20 Thread Henry Saputra
Signature looks good
Hashes good
License and notice exists
Build and run tests and looks good

+1 binding

On Tuesday, September 17, 2013, Patrick Wendell wrote:

> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.8.0. This will be the first incubator release for
> Spark in Apache.
>
> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>
> The release files, including signatures, digests, etc can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-059/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>
> Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>
> The vote is open until Friday, September 20th at 09:00 UTC and passes if
> a majority of at least 3 +1 IPMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.incubator.apache.org/
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Chris Mattmann
incubating-
rc] mattmann%


Checksums check out:

[guest-wireless-upc-nat-206-117-88-006:~/tmp/apache-spark-0.8.0-incubating-
rc] mattmann% $HOME/bin/verify_md5_checksums
md5sum: stat '*.tar.gz': No such file or directory
md5sum: stat '*.bz2': No such file or directory
md5sum: stat '*.zip': No such file or directory
spark-0.8.0-incubating-bin-cdh4-rc6.tgz: OK
spark-0.8.0-incubating-bin-hadoop1-rc6.tgz: OK
spark-0.8.0-incubating-rc6.tgz: OK
[guest-wireless-upc-nat-206-117-88-006:~/tmp/apache-spark-0.8.0-incubating-
rc] mattmann%


I was able to build:

[guest-wireless-upc-nat-206-117-88-006:~/tmp/apache-spark-0.8.0-incubating-
rc/spark-0.8.0-incubating] mattmann% ./sbt/sbt assembly
Getting net.java.dev.jna jna 3.2.3 ...
downloading 
http://repo1.maven.org/maven2/net/java/dev/jna/jna/3.2.3/jna-3.2.3.jar ...
[SUCCESSFUL ] net.java.dev.jna#jna;3.2.3!jna.jar (7552ms)
:: retrieving :: org.scala-sbt#boot-jna
confs: [default]
1 artifacts copied, 0 already retrieved (838kB/36ms)
Getting org.scala-sbt sbt 0.12.4 ...

..later


[warn] Strategy 'discard' was applied to 2 files
[warn] Strategy 'first' was applied to 823 files
[info] Checking every *.class/*.jar file's SHA-1.
[info] Done packaging.
[info] SHA-1: bf3ae9e2d032150075416faf132eb779b9df587e
[info] Packaging 
/Users/mattmann/tmp/apache-spark-0.8.0-incubating-rc/spark-0.8.0-incubating
/examples/target/scala-2.9.3/spark-examples-assembly-0.8.0-incubating.jar
...
[info] Done packaging.
[success] Total time: 1773 s, completed Sep 19, 2013 8:54:23 PM
[guest-wireless-upc-nat-206-117-88-006:~/tmp/apache-spark-0.8.0-incubating-
rc/spark-0.8.0-incubating] mattmann%

Cheers,

Chris

-Original Message-
From: Patrick Wendell 
Reply-To: "dev@spark.incubator.apache.org" 
Date: Tuesday, September 17, 2013 2:03 AM
To: "dev@spark.incubator.apache.org" 
Subject: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

>Please vote on releasing the following candidate as Apache Spark
>(incubating) version 0.8.0. This will be the first incubator release for
>Spark in Apache.
>
>The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>
>The release files, including signatures, digests, etc can be found at:
>http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>
>Release artifacts are signed with the following key:
>https://people.apache.org/keys/committer/pwendell.asc
>
>The staging repository for this release can be found at:
>https://repository.apache.org/content/repositories/orgapachespark-059/
>
>The documentation corresponding to this release can be found at:
>http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>
>Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>
>The vote is open until Friday, September 20th at 09:00 UTC and passes if
>a majority of at least 3 +1 IPMC votes are cast.
>
>[ ] +1 Release this package as Apache Spark 0.8.0-incubating
>[ ] -1 Do not release this package because ...
>
>To learn more about Apache Spark, please see
>http://spark.incubator.apache.org/




Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Patrick Wendell
Hey Roman,

We can do this in the future - I wasn't sure exactly what the right
standard approach was. Just so I understand, the change you are
proposing from what is there now is just to remove rcX from the
file-names, correct?

- Patrick

On Thu, Sep 19, 2013 at 8:06 PM, Roman Shaposhnik  wrote:
> On Thu, Sep 19, 2013 at 5:56 PM, Patrick Wendell  wrote:
>> FYI this vote ends in 8 hours.
>
> I was going to test it on a fully distributed Bigtop cluster, but hit
> a few snags. That now will extend into the weekend.
>
> Of course, that's not that big of a deal -- I can always vote on
> incubator general once you guys move the vote over there.
>
> The only minor nit for the future I've noticed is that I would
> highly encourage you to follow the usual RC practices where
> you name all of your artifact as final bits and have a subdirectory
> that reflects the RC name. E.g. here's how a very recent
> Hadoop RC looks like:
> http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0/
>
> Thanks,
> Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Roman Shaposhnik
On Thu, Sep 19, 2013 at 5:56 PM, Patrick Wendell  wrote:
> FYI this vote ends in 8 hours.

I was going to test it on a fully distributed Bigtop cluster, but hit
a few snags. That now will extend into the weekend.

Of course, that's not that big of a deal -- I can always vote on
incubator general once you guys move the vote over there.

The only minor nit for the future I've noticed is that I would
highly encourage you to follow the usual RC practices where
you name all of your artifact as final bits and have a subdirectory
that reflects the RC name. E.g. here's how a very recent
Hadoop RC looks like:
http://people.apache.org/~acmurthy/hadoop-2.1.1-beta-rc0/

Thanks,
Roman.


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Chris Mattmann
Thanks!



-Original Message-
From: Patrick Wendell 
Reply-To: "dev@spark.incubator.apache.org" 
Date: Thursday, September 19, 2013 7:33 PM
To: "dev@spark.incubator.apache.org" 
Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

>Ah I see - sounds good. For future releases we can use that URL to
>describe the tag.
>
>On Thu, Sep 19, 2013 at 7:29 PM, Henry Saputra 
>wrote:
>> I think Chris was saying the VOTE should be made against ASF git repo
>>directly:
>>
>> 
>>https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=ca6
>>9992d3b82124795d16e8e0d76a50fe10435f9
>>
>>
>> - Henry
>>
>> On Thu, Sep 19, 2013 at 7:25 PM, Patrick Wendell 
>>wrote:
>>> Hey Chris the tag in github is 3b85a85, which I listed in the original
>>> vote next to the git URL. Is there another type of tag I should be
>>> adding?
>>>
>>> On Thu, Sep 19, 2013 at 7:20 PM, Chris Mattmann 
>>>wrote:
>>>> I'm currently downloading the RC (all 127mb of the bin; then onto
>>>>source).
>>>> I have a generic set of Incubator scripts so should go fine after
>>>>that.
>>>>
>>>> I'm giving you a preview of my minor nit:
>>>>
>>>> We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag).
>>>>That
>>>> should be corrected in future RC emails. If all checks out, should be
>>>>+1
>>>> shortly.
>>>>
>>>>
>>>>
>>>> -Original Message-
>>>> From: Patrick Wendell 
>>>> Reply-To: "dev@spark.incubator.apache.org"
>>>>
>>>> Date: Thursday, September 19, 2013 5:56 PM
>>>> To: "dev@spark.incubator.apache.org" 
>>>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>>>>
>>>>>FYI this vote ends in 8 hours.
>>>>>
>>>>>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin 
>>>>>wrote:
>>>>>> +1
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Reynold Xin, AMPLab, UC Berkeley
>>>>>> http://rxin.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik
>>>>>>
>>>>>>wrote:
>>>>>>
>>>>>>> Maven package could be run with -DskipTests that will simply
>>>>>>>build...
>>>>>>>well,
>>>>>>> the package.
>>>>>>>
>>>>>>> +1 on the RC. The nits are indeed minor.
>>>>>>>
>>>>>>>   Cos
>>>>>>>
>>>>>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>>>>>> > In Maven, mvn package should also create the assembly, but the
>>>>>>> non-obvious
>>>>>>> > thing is that it needs to happen for all projects before mvn
>>>>>>>test for
>>>>>>> core
>>>>>>> > works. Unfortunately I don't know any easy way around that.
>>>>>>> >
>>>>>>> > Matei
>>>>>>> >
>>>>>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>>>>>wrote:
>>>>>>> >
>>>>>>> > > Hey Mark,
>>>>>>> > >
>>>>>>> > > Good catches here. Ya the driver suite thing is sorta annoying
>>>>>>>- we
>>>>>>> > > should try to fix that in master. The audit script I wrote
>>>>>>>first
>>>>>>>does
>>>>>>> > > an sbt/sbt assembly to avoid this. I agree though these
>>>>>>>shouldn't
>>>>>>> > > block the release (if a blocker does come up we can revisit
>>>>>>>these
>>>>>>> > > potentially when cutting a release).
>>>>>>> > >
>>>>>>> > > - Patrick
>>>>>>> > >
>>>>>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>>>>>
>>>>>>> wrote:
>>>>>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local'
>>>>>>>isn't
>&g

Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Chris Mattmann
Yep what Henry said below Patrick.

Not a huge deal I'm still likely going to be +1 (once my darned
downloads complete on my crappy cable modem lol). I just meant that
the URL reference should be an Apache URL.

Will keep you guys posted..


-Original Message-
From: Henry Saputra 
Reply-To: "dev@spark.incubator.apache.org" 
Date: Thursday, September 19, 2013 7:29 PM
To: "dev@spark.incubator.apache.org" 
Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

>I think Chris was saying the VOTE should be made against ASF git repo
>directly:
>
>https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=ca69
>992d3b82124795d16e8e0d76a50fe10435f9
>
>
>- Henry
>
>On Thu, Sep 19, 2013 at 7:25 PM, Patrick Wendell 
>wrote:
>> Hey Chris the tag in github is 3b85a85, which I listed in the original
>> vote next to the git URL. Is there another type of tag I should be
>> adding?
>>
>> On Thu, Sep 19, 2013 at 7:20 PM, Chris Mattmann 
>>wrote:
>>> I'm currently downloading the RC (all 127mb of the bin; then onto
>>>source).
>>> I have a generic set of Incubator scripts so should go fine after that.
>>>
>>> I'm giving you a preview of my minor nit:
>>>
>>> We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag).
>>>That
>>> should be corrected in future RC emails. If all checks out, should be
>>>+1
>>> shortly.
>>>
>>>
>>>
>>> -Original Message-
>>> From: Patrick Wendell 
>>> Reply-To: "dev@spark.incubator.apache.org"
>>>
>>> Date: Thursday, September 19, 2013 5:56 PM
>>> To: "dev@spark.incubator.apache.org" 
>>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>>>
>>>>FYI this vote ends in 8 hours.
>>>>
>>>>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin 
>>>>wrote:
>>>>> +1
>>>>>
>>>>>
>>>>> --
>>>>> Reynold Xin, AMPLab, UC Berkeley
>>>>> http://rxin.org
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik 
>>>>>wrote:
>>>>>
>>>>>> Maven package could be run with -DskipTests that will simply
>>>>>>build...
>>>>>>well,
>>>>>> the package.
>>>>>>
>>>>>> +1 on the RC. The nits are indeed minor.
>>>>>>
>>>>>>   Cos
>>>>>>
>>>>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>>>>> > In Maven, mvn package should also create the assembly, but the
>>>>>> non-obvious
>>>>>> > thing is that it needs to happen for all projects before mvn test
>>>>>>for
>>>>>> core
>>>>>> > works. Unfortunately I don't know any easy way around that.
>>>>>> >
>>>>>> > Matei
>>>>>> >
>>>>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>>>>wrote:
>>>>>> >
>>>>>> > > Hey Mark,
>>>>>> > >
>>>>>> > > Good catches here. Ya the driver suite thing is sorta annoying
>>>>>>- we
>>>>>> > > should try to fix that in master. The audit script I wrote first
>>>>>>does
>>>>>> > > an sbt/sbt assembly to avoid this. I agree though these
>>>>>>shouldn't
>>>>>> > > block the release (if a blocker does come up we can revisit
>>>>>>these
>>>>>> > > potentially when cutting a release).
>>>>>> > >
>>>>>> > > - Patrick
>>>>>> > >
>>>>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>>>>
>>>>>> wrote:
>>>>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local'
>>>>>>isn't
>>>>>> generating
>>>>>> > >> correct POM files because of the way the exclusions are
>>>>>>defined in
>>>>>> > >> SparkBuild.scala using wildcards; looks like there may be some
>>>>>>broken
>>>>>> doc
>>>>>> > >> links generated in that t

Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Patrick Wendell
Ah I see - sounds good. For future releases we can use that URL to
describe the tag.

On Thu, Sep 19, 2013 at 7:29 PM, Henry Saputra  wrote:
> I think Chris was saying the VOTE should be made against ASF git repo 
> directly:
>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=ca69992d3b82124795d16e8e0d76a50fe10435f9
>
>
> - Henry
>
> On Thu, Sep 19, 2013 at 7:25 PM, Patrick Wendell  wrote:
>> Hey Chris the tag in github is 3b85a85, which I listed in the original
>> vote next to the git URL. Is there another type of tag I should be
>> adding?
>>
>> On Thu, Sep 19, 2013 at 7:20 PM, Chris Mattmann  wrote:
>>> I'm currently downloading the RC (all 127mb of the bin; then onto source).
>>> I have a generic set of Incubator scripts so should go fine after that.
>>>
>>> I'm giving you a preview of my minor nit:
>>>
>>> We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag). That
>>> should be corrected in future RC emails. If all checks out, should be +1
>>> shortly.
>>>
>>>
>>>
>>> -Original Message-
>>> From: Patrick Wendell 
>>> Reply-To: "dev@spark.incubator.apache.org" 
>>> Date: Thursday, September 19, 2013 5:56 PM
>>> To: "dev@spark.incubator.apache.org" 
>>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>>>
>>>>FYI this vote ends in 8 hours.
>>>>
>>>>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin  wrote:
>>>>> +1
>>>>>
>>>>>
>>>>> --
>>>>> Reynold Xin, AMPLab, UC Berkeley
>>>>> http://rxin.org
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik 
>>>>>wrote:
>>>>>
>>>>>> Maven package could be run with -DskipTests that will simply build...
>>>>>>well,
>>>>>> the package.
>>>>>>
>>>>>> +1 on the RC. The nits are indeed minor.
>>>>>>
>>>>>>   Cos
>>>>>>
>>>>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>>>>> > In Maven, mvn package should also create the assembly, but the
>>>>>> non-obvious
>>>>>> > thing is that it needs to happen for all projects before mvn test for
>>>>>> core
>>>>>> > works. Unfortunately I don't know any easy way around that.
>>>>>> >
>>>>>> > Matei
>>>>>> >
>>>>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>>>>wrote:
>>>>>> >
>>>>>> > > Hey Mark,
>>>>>> > >
>>>>>> > > Good catches here. Ya the driver suite thing is sorta annoying - we
>>>>>> > > should try to fix that in master. The audit script I wrote first
>>>>>>does
>>>>>> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
>>>>>> > > block the release (if a blocker does come up we can revisit these
>>>>>> > > potentially when cutting a release).
>>>>>> > >
>>>>>> > > - Patrick
>>>>>> > >
>>>>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>>>>
>>>>>> wrote:
>>>>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>>>>>> generating
>>>>>> > >> correct POM files because of the way the exclusions are defined in
>>>>>> > >> SparkBuild.scala using wildcards; looks like there may be some
>>>>>>broken
>>>>>> doc
>>>>>> > >> links generated in that task, as well; DriverSuite doesn't like to
>>>>>> run from
>>>>>> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be
>>>>>>run
>>>>>> first.
>>>>>> > >>
>>>>>> > >> None of these is enough for me to give RC6 a -1.
>>>>>> > >>
>>>>>> > >>
>>>>>> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>>>>>> matei.zaha...@gmail.com>wrote:
>>&g

Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Henry Saputra
I think Chris was saying the VOTE should be made against ASF git repo directly:

https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=tag;h=ca69992d3b82124795d16e8e0d76a50fe10435f9


- Henry

On Thu, Sep 19, 2013 at 7:25 PM, Patrick Wendell  wrote:
> Hey Chris the tag in github is 3b85a85, which I listed in the original
> vote next to the git URL. Is there another type of tag I should be
> adding?
>
> On Thu, Sep 19, 2013 at 7:20 PM, Chris Mattmann  wrote:
>> I'm currently downloading the RC (all 127mb of the bin; then onto source).
>> I have a generic set of Incubator scripts so should go fine after that.
>>
>> I'm giving you a preview of my minor nit:
>>
>> We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag). That
>> should be corrected in future RC emails. If all checks out, should be +1
>> shortly.
>>
>>
>>
>> -Original Message-
>> From: Patrick Wendell 
>> Reply-To: "dev@spark.incubator.apache.org" 
>> Date: Thursday, September 19, 2013 5:56 PM
>> To: "dev@spark.incubator.apache.org" 
>> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>>
>>>FYI this vote ends in 8 hours.
>>>
>>>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin  wrote:
>>>> +1
>>>>
>>>>
>>>> --
>>>> Reynold Xin, AMPLab, UC Berkeley
>>>> http://rxin.org
>>>>
>>>>
>>>>
>>>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik 
>>>>wrote:
>>>>
>>>>> Maven package could be run with -DskipTests that will simply build...
>>>>>well,
>>>>> the package.
>>>>>
>>>>> +1 on the RC. The nits are indeed minor.
>>>>>
>>>>>   Cos
>>>>>
>>>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>>>> > In Maven, mvn package should also create the assembly, but the
>>>>> non-obvious
>>>>> > thing is that it needs to happen for all projects before mvn test for
>>>>> core
>>>>> > works. Unfortunately I don't know any easy way around that.
>>>>> >
>>>>> > Matei
>>>>> >
>>>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>>>wrote:
>>>>> >
>>>>> > > Hey Mark,
>>>>> > >
>>>>> > > Good catches here. Ya the driver suite thing is sorta annoying - we
>>>>> > > should try to fix that in master. The audit script I wrote first
>>>>>does
>>>>> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
>>>>> > > block the release (if a blocker does come up we can revisit these
>>>>> > > potentially when cutting a release).
>>>>> > >
>>>>> > > - Patrick
>>>>> > >
>>>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>>>
>>>>> wrote:
>>>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>>>>> generating
>>>>> > >> correct POM files because of the way the exclusions are defined in
>>>>> > >> SparkBuild.scala using wildcards; looks like there may be some
>>>>>broken
>>>>> doc
>>>>> > >> links generated in that task, as well; DriverSuite doesn't like to
>>>>> run from
>>>>> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be
>>>>>run
>>>>> first.
>>>>> > >>
>>>>> > >> None of these is enough for me to give RC6 a -1.
>>>>> > >>
>>>>> > >>
>>>>> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>>>>> matei.zaha...@gmail.com>wrote:
>>>>> > >>
>>>>> > >>> +1
>>>>> > >>>
>>>>> > >>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>>>> > >>>
>>>>> > >>> Matei
>>>>> > >>>
>>>>> > >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>>>>> wrote:
>>>>> > >>>
>>>>> > >>>> Please vote on releasing the following candidate as Apache Spark
>>>>> > >>>> (incubating) version 0.8.0. This will be the first incubator
>>>>> release for
>>>>> > >>>> Spark in Apache.
>>>>> > >>>>
>>>>> > >>>> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>>>>> > >>>>
>>>>>
>>>>>https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>>>>> > >>>>
>>>>> > >>>> The release files, including signatures, digests, etc can be
>>>>>found
>>>>> at:
>>>>> > >>>>
>>>>> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>>>>> > >>>>
>>>>> > >>>> Release artifacts are signed with the following key:
>>>>> > >>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>> > >>>>
>>>>> > >>>> The staging repository for this release can be found at:
>>>>> > >>>>
>>>>> https://repository.apache.org/content/repositories/orgapachespark-059/
>>>>> > >>>>
>>>>> > >>>> The documentation corresponding to this release can be found at:
>>>>> > >>>>
>>>>>http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>>>>> > >>>>
>>>>> > >>>> Please vote on releasing this package as Apache Spark
>>>>> 0.8.0-incubating!
>>>>> > >>>>
>>>>> > >>>> The vote is open until Friday, September 20th at 09:00 UTC and
>>>>> passes if
>>>>> > >>>> a majority of at least 3 +1 IPMC votes are cast.
>>>>> > >>>>
>>>>> > >>>> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>>>>> > >>>> [ ] -1 Do not release this package because ...
>>>>> > >>>>
>>>>> > >>>> To learn more about Apache Spark, please see
>>>>> > >>>> http://spark.incubator.apache.org/
>>>>> > >>>
>>>>> > >>>
>>>>> >
>>>>>
>>
>>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Patrick Wendell
Hey Chris the tag in github is 3b85a85, which I listed in the original
vote next to the git URL. Is there another type of tag I should be
adding?

On Thu, Sep 19, 2013 at 7:20 PM, Chris Mattmann  wrote:
> I'm currently downloading the RC (all 127mb of the bin; then onto source).
> I have a generic set of Incubator scripts so should go fine after that.
>
> I'm giving you a preview of my minor nit:
>
> We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag). That
> should be corrected in future RC emails. If all checks out, should be +1
> shortly.
>
>
>
> -Original Message-
> From: Patrick Wendell 
> Reply-To: "dev@spark.incubator.apache.org" 
> Date: Thursday, September 19, 2013 5:56 PM
> To: "dev@spark.incubator.apache.org" 
> Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)
>
>>FYI this vote ends in 8 hours.
>>
>>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin  wrote:
>>> +1
>>>
>>>
>>> --
>>> Reynold Xin, AMPLab, UC Berkeley
>>> http://rxin.org
>>>
>>>
>>>
>>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik 
>>>wrote:
>>>
>>>> Maven package could be run with -DskipTests that will simply build...
>>>>well,
>>>> the package.
>>>>
>>>> +1 on the RC. The nits are indeed minor.
>>>>
>>>>   Cos
>>>>
>>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>>> > In Maven, mvn package should also create the assembly, but the
>>>> non-obvious
>>>> > thing is that it needs to happen for all projects before mvn test for
>>>> core
>>>> > works. Unfortunately I don't know any easy way around that.
>>>> >
>>>> > Matei
>>>> >
>>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>>wrote:
>>>> >
>>>> > > Hey Mark,
>>>> > >
>>>> > > Good catches here. Ya the driver suite thing is sorta annoying - we
>>>> > > should try to fix that in master. The audit script I wrote first
>>>>does
>>>> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
>>>> > > block the release (if a blocker does come up we can revisit these
>>>> > > potentially when cutting a release).
>>>> > >
>>>> > > - Patrick
>>>> > >
>>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>>
>>>> wrote:
>>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>>>> generating
>>>> > >> correct POM files because of the way the exclusions are defined in
>>>> > >> SparkBuild.scala using wildcards; looks like there may be some
>>>>broken
>>>> doc
>>>> > >> links generated in that task, as well; DriverSuite doesn't like to
>>>> run from
>>>> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be
>>>>run
>>>> first.
>>>> > >>
>>>> > >> None of these is enough for me to give RC6 a -1.
>>>> > >>
>>>> > >>
>>>> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>>>> matei.zaha...@gmail.com>wrote:
>>>> > >>
>>>> > >>> +1
>>>> > >>>
>>>> > >>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>>> > >>>
>>>> > >>> Matei
>>>> > >>>
>>>> > >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>>>> wrote:
>>>> > >>>
>>>> > >>>> Please vote on releasing the following candidate as Apache Spark
>>>> > >>>> (incubating) version 0.8.0. This will be the first incubator
>>>> release for
>>>> > >>>> Spark in Apache.
>>>> > >>>>
>>>> > >>>> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>>>> > >>>>
>>>>
>>>>https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>>>> > >>>>
>>>> > >>>> The release files, including signatures, digests, etc can be
>>>>found
>>>> at:
>>>> > >>>>
>>>> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>>>> > >>>>
>>>> > >>>> Release artifacts are signed with the following key:
>>>> > >>>> https://people.apache.org/keys/committer/pwendell.asc
>>>> > >>>>
>>>> > >>>> The staging repository for this release can be found at:
>>>> > >>>>
>>>> https://repository.apache.org/content/repositories/orgapachespark-059/
>>>> > >>>>
>>>> > >>>> The documentation corresponding to this release can be found at:
>>>> > >>>>
>>>>http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>>>> > >>>>
>>>> > >>>> Please vote on releasing this package as Apache Spark
>>>> 0.8.0-incubating!
>>>> > >>>>
>>>> > >>>> The vote is open until Friday, September 20th at 09:00 UTC and
>>>> passes if
>>>> > >>>> a majority of at least 3 +1 IPMC votes are cast.
>>>> > >>>>
>>>> > >>>> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>>>> > >>>> [ ] -1 Do not release this package because ...
>>>> > >>>>
>>>> > >>>> To learn more about Apache Spark, please see
>>>> > >>>> http://spark.incubator.apache.org/
>>>> > >>>
>>>> > >>>
>>>> >
>>>>
>
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Chris Mattmann
I'm currently downloading the RC (all 127mb of the bin; then onto source).
I have a generic set of Incubator scripts so should go fine after that.

I'm giving you a preview of my minor nit:

We don't VOTE on github URLs -- we VOTE on ASF URLs (e.g., the tag). That
should be corrected in future RC emails. If all checks out, should be +1
shortly.



-Original Message-
From: Patrick Wendell 
Reply-To: "dev@spark.incubator.apache.org" 
Date: Thursday, September 19, 2013 5:56 PM
To: "dev@spark.incubator.apache.org" 
Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

>FYI this vote ends in 8 hours.
>
>On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin  wrote:
>> +1
>>
>>
>> --
>> Reynold Xin, AMPLab, UC Berkeley
>> http://rxin.org
>>
>>
>>
>> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik 
>>wrote:
>>
>>> Maven package could be run with -DskipTests that will simply build...
>>>well,
>>> the package.
>>>
>>> +1 on the RC. The nits are indeed minor.
>>>
>>>   Cos
>>>
>>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>>> > In Maven, mvn package should also create the assembly, but the
>>> non-obvious
>>> > thing is that it needs to happen for all projects before mvn test for
>>> core
>>> > works. Unfortunately I don't know any easy way around that.
>>> >
>>> > Matei
>>> >
>>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell 
>>>wrote:
>>> >
>>> > > Hey Mark,
>>> > >
>>> > > Good catches here. Ya the driver suite thing is sorta annoying - we
>>> > > should try to fix that in master. The audit script I wrote first
>>>does
>>> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
>>> > > block the release (if a blocker does come up we can revisit these
>>> > > potentially when cutting a release).
>>> > >
>>> > > - Patrick
>>> > >
>>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra
>>>
>>> wrote:
>>> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>>> generating
>>> > >> correct POM files because of the way the exclusions are defined in
>>> > >> SparkBuild.scala using wildcards; looks like there may be some
>>>broken
>>> doc
>>> > >> links generated in that task, as well; DriverSuite doesn't like to
>>> run from
>>> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be
>>>run
>>> first.
>>> > >>
>>> > >> None of these is enough for me to give RC6 a -1.
>>> > >>
>>> > >>
>>> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>>> matei.zaha...@gmail.com>wrote:
>>> > >>
>>> > >>> +1
>>> > >>>
>>> > >>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>> > >>>
>>> > >>> Matei
>>> > >>>
>>> > >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>>> wrote:
>>> > >>>
>>> > >>>> Please vote on releasing the following candidate as Apache Spark
>>> > >>>> (incubating) version 0.8.0. This will be the first incubator
>>> release for
>>> > >>>> Spark in Apache.
>>> > >>>>
>>> > >>>> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>>> > >>>>
>>> 
>>>https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>>> > >>>>
>>> > >>>> The release files, including signatures, digests, etc can be
>>>found
>>> at:
>>> > >>>>
>>> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>>> > >>>>
>>> > >>>> Release artifacts are signed with the following key:
>>> > >>>> https://people.apache.org/keys/committer/pwendell.asc
>>> > >>>>
>>> > >>>> The staging repository for this release can be found at:
>>> > >>>>
>>> https://repository.apache.org/content/repositories/orgapachespark-059/
>>> > >>>>
>>> > >>>> The documentation corresponding to this release can be found at:
>>> > >>>> 
>>>http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>>> > >>>>
>>> > >>>> Please vote on releasing this package as Apache Spark
>>> 0.8.0-incubating!
>>> > >>>>
>>> > >>>> The vote is open until Friday, September 20th at 09:00 UTC and
>>> passes if
>>> > >>>> a majority of at least 3 +1 IPMC votes are cast.
>>> > >>>>
>>> > >>>> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>>> > >>>> [ ] -1 Do not release this package because ...
>>> > >>>>
>>> > >>>> To learn more about Apache Spark, please see
>>> > >>>> http://spark.incubator.apache.org/
>>> > >>>
>>> > >>>
>>> >
>>>




Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-19 Thread Patrick Wendell
FYI this vote ends in 8 hours.

On Wed, Sep 18, 2013 at 8:56 PM, Reynold Xin  wrote:
> +1
>
>
> --
> Reynold Xin, AMPLab, UC Berkeley
> http://rxin.org
>
>
>
> On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik  wrote:
>
>> Maven package could be run with -DskipTests that will simply build... well,
>> the package.
>>
>> +1 on the RC. The nits are indeed minor.
>>
>>   Cos
>>
>> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
>> > In Maven, mvn package should also create the assembly, but the
>> non-obvious
>> > thing is that it needs to happen for all projects before mvn test for
>> core
>> > works. Unfortunately I don't know any easy way around that.
>> >
>> > Matei
>> >
>> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell  wrote:
>> >
>> > > Hey Mark,
>> > >
>> > > Good catches here. Ya the driver suite thing is sorta annoying - we
>> > > should try to fix that in master. The audit script I wrote first does
>> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
>> > > block the release (if a blocker does come up we can revisit these
>> > > potentially when cutting a release).
>> > >
>> > > - Patrick
>> > >
>> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
>> wrote:
>> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>> generating
>> > >> correct POM files because of the way the exclusions are defined in
>> > >> SparkBuild.scala using wildcards; looks like there may be some broken
>> doc
>> > >> links generated in that task, as well; DriverSuite doesn't like to
>> run from
>> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be run
>> first.
>> > >>
>> > >> None of these is enough for me to give RC6 a -1.
>> > >>
>> > >>
>> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>> matei.zaha...@gmail.com>wrote:
>> > >>
>> > >>> +1
>> > >>>
>> > >>> Tried new staging repo to make sure the issue with RC5 is fixed.
>> > >>>
>> > >>> Matei
>> > >>>
>> > >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>> wrote:
>> > >>>
>> >  Please vote on releasing the following candidate as Apache Spark
>> >  (incubating) version 0.8.0. This will be the first incubator
>> release for
>> >  Spark in Apache.
>> > 
>> >  The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>> > 
>> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>> > 
>> >  The release files, including signatures, digests, etc can be found
>> at:
>> > 
>> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>> > 
>> >  Release artifacts are signed with the following key:
>> >  https://people.apache.org/keys/committer/pwendell.asc
>> > 
>> >  The staging repository for this release can be found at:
>> > 
>> https://repository.apache.org/content/repositories/orgapachespark-059/
>> > 
>> >  The documentation corresponding to this release can be found at:
>> >  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>> > 
>> >  Please vote on releasing this package as Apache Spark
>> 0.8.0-incubating!
>> > 
>> >  The vote is open until Friday, September 20th at 09:00 UTC and
>> passes if
>> >  a majority of at least 3 +1 IPMC votes are cast.
>> > 
>> >  [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>> >  [ ] -1 Do not release this package because ...
>> > 
>> >  To learn more about Apache Spark, please see
>> >  http://spark.incubator.apache.org/
>> > >>>
>> > >>>
>> >
>>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Reynold Xin
+1


--
Reynold Xin, AMPLab, UC Berkeley
http://rxin.org



On Wed, Sep 18, 2013 at 11:06 AM, Konstantin Boudnik  wrote:

> Maven package could be run with -DskipTests that will simply build... well,
> the package.
>
> +1 on the RC. The nits are indeed minor.
>
>   Cos
>
> On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
> > In Maven, mvn package should also create the assembly, but the
> non-obvious
> > thing is that it needs to happen for all projects before mvn test for
> core
> > works. Unfortunately I don't know any easy way around that.
> >
> > Matei
> >
> > On Sep 17, 2013, at 1:46 PM, Patrick Wendell  wrote:
> >
> > > Hey Mark,
> > >
> > > Good catches here. Ya the driver suite thing is sorta annoying - we
> > > should try to fix that in master. The audit script I wrote first does
> > > an sbt/sbt assembly to avoid this. I agree though these shouldn't
> > > block the release (if a blocker does come up we can revisit these
> > > potentially when cutting a release).
> > >
> > > - Patrick
> > >
> > > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
> wrote:
> > >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
> generating
> > >> correct POM files because of the way the exclusions are defined in
> > >> SparkBuild.scala using wildcards; looks like there may be some broken
> doc
> > >> links generated in that task, as well; DriverSuite doesn't like to
> run from
> > >> the maven build, complaining that 'sbt/sbt assembly' needs to be run
> first.
> > >>
> > >> None of these is enough for me to give RC6 a -1.
> > >>
> > >>
> > >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
> matei.zaha...@gmail.com>wrote:
> > >>
> > >>> +1
> > >>>
> > >>> Tried new staging repo to make sure the issue with RC5 is fixed.
> > >>>
> > >>> Matei
> > >>>
> > >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
> wrote:
> > >>>
> >  Please vote on releasing the following candidate as Apache Spark
> >  (incubating) version 0.8.0. This will be the first incubator
> release for
> >  Spark in Apache.
> > 
> >  The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> > 
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
> > 
> >  The release files, including signatures, digests, etc can be found
> at:
> > 
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
> > 
> >  Release artifacts are signed with the following key:
> >  https://people.apache.org/keys/committer/pwendell.asc
> > 
> >  The staging repository for this release can be found at:
> > 
> https://repository.apache.org/content/repositories/orgapachespark-059/
> > 
> >  The documentation corresponding to this release can be found at:
> >  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
> > 
> >  Please vote on releasing this package as Apache Spark
> 0.8.0-incubating!
> > 
> >  The vote is open until Friday, September 20th at 09:00 UTC and
> passes if
> >  a majority of at least 3 +1 IPMC votes are cast.
> > 
> >  [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> >  [ ] -1 Do not release this package because ...
> > 
> >  To learn more about Apache Spark, please see
> >  http://spark.incubator.apache.org/
> > >>>
> > >>>
> >
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Konstantin Boudnik
Maven package could be run with -DskipTests that will simply build... well,
the package.

+1 on the RC. The nits are indeed minor.

  Cos

On Tue, Sep 17, 2013 at 07:20PM, Matei Zaharia wrote:
> In Maven, mvn package should also create the assembly, but the non-obvious
> thing is that it needs to happen for all projects before mvn test for core
> works. Unfortunately I don't know any easy way around that.
> 
> Matei
> 
> On Sep 17, 2013, at 1:46 PM, Patrick Wendell  wrote:
> 
> > Hey Mark,
> > 
> > Good catches here. Ya the driver suite thing is sorta annoying - we
> > should try to fix that in master. The audit script I wrote first does
> > an sbt/sbt assembly to avoid this. I agree though these shouldn't
> > block the release (if a blocker does come up we can revisit these
> > potentially when cutting a release).
> > 
> > - Patrick
> > 
> > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra  
> > wrote:
> >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't generating
> >> correct POM files because of the way the exclusions are defined in
> >> SparkBuild.scala using wildcards; looks like there may be some broken doc
> >> links generated in that task, as well; DriverSuite doesn't like to run from
> >> the maven build, complaining that 'sbt/sbt assembly' needs to be run first.
> >> 
> >> None of these is enough for me to give RC6 a -1.
> >> 
> >> 
> >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia 
> >> wrote:
> >> 
> >>> +1
> >>> 
> >>> Tried new staging repo to make sure the issue with RC5 is fixed.
> >>> 
> >>> Matei
> >>> 
> >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:
> >>> 
>  Please vote on releasing the following candidate as Apache Spark
>  (incubating) version 0.8.0. This will be the first incubator release for
>  Spark in Apache.
>  
>  The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>  https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>  
>  The release files, including signatures, digests, etc can be found at:
>  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>  
>  Release artifacts are signed with the following key:
>  https://people.apache.org/keys/committer/pwendell.asc
>  
>  The staging repository for this release can be found at:
>  https://repository.apache.org/content/repositories/orgapachespark-059/
>  
>  The documentation corresponding to this release can be found at:
>  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>  
>  Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>  
>  The vote is open until Friday, September 20th at 09:00 UTC and passes if
>  a majority of at least 3 +1 IPMC votes are cast.
>  
>  [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>  [ ] -1 Do not release this package because ...
>  
>  To learn more about Apache Spark, please see
>  http://spark.incubator.apache.org/
> >>> 
> >>> 
> 


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Patrick Wendell
I'm a +1 as well.

On Wed, Sep 18, 2013 at 10:39 AM, Mark Hamstra  wrote:
> "not enough", of course.
>
>
>
> On Wed, Sep 18, 2013 at 10:38 AM, Mark Hamstra wrote:
>
>> If that will make the difference between releasing and not releasing, then
>> sure; else I'd rather see the last niggles resolved, but that's enough for
>> me to vote -1 -- so I'm a 0 unless you really need another +1.
>>
>>
>> On Wed, Sep 18, 2013 at 10:34 AM, Henry Saputra 
>> wrote:
>>
>>> Mark, so I assume we are getting +1 from you?  ;)
>>>
>>> On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
>>> wrote:
>>> > There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>>> generating
>>> > correct POM files because of the way the exclusions are defined in
>>> > SparkBuild.scala using wildcards; looks like there may be some broken
>>> doc
>>> > links generated in that task, as well; DriverSuite doesn't like to run
>>> from
>>> > the maven build, complaining that 'sbt/sbt assembly' needs to be run
>>> first.
>>> >
>>> > None of these is enough for me to give RC6 a -1.
>>> >
>>> >
>>> > On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>>> matei.zaha...@gmail.com>wrote:
>>> >
>>> >> +1
>>> >>
>>> >> Tried new staging repo to make sure the issue with RC5 is fixed.
>>> >>
>>> >> Matei
>>> >>
>>> >> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>>> wrote:
>>> >>
>>> >> > Please vote on releasing the following candidate as Apache Spark
>>> >> > (incubating) version 0.8.0. This will be the first incubator release
>>> for
>>> >> > Spark in Apache.
>>> >> >
>>> >> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>>> >> >
>>> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>>> >> >
>>> >> > The release files, including signatures, digests, etc can be found
>>> at:
>>> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>>> >> >
>>> >> > Release artifacts are signed with the following key:
>>> >> > https://people.apache.org/keys/committer/pwendell.asc
>>> >> >
>>> >> > The staging repository for this release can be found at:
>>> >> >
>>> https://repository.apache.org/content/repositories/orgapachespark-059/
>>> >> >
>>> >> > The documentation corresponding to this release can be found at:
>>> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>>> >> >
>>> >> > Please vote on releasing this package as Apache Spark
>>> 0.8.0-incubating!
>>> >> >
>>> >> > The vote is open until Friday, September 20th at 09:00 UTC and
>>> passes if
>>> >> > a majority of at least 3 +1 IPMC votes are cast.
>>> >> >
>>> >> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>>> >> > [ ] -1 Do not release this package because ...
>>> >> >
>>> >> > To learn more about Apache Spark, please see
>>> >> > http://spark.incubator.apache.org/
>>> >>
>>> >>
>>>
>>
>>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Mark Hamstra
"not enough", of course.



On Wed, Sep 18, 2013 at 10:38 AM, Mark Hamstra wrote:

> If that will make the difference between releasing and not releasing, then
> sure; else I'd rather see the last niggles resolved, but that's enough for
> me to vote -1 -- so I'm a 0 unless you really need another +1.
>
>
> On Wed, Sep 18, 2013 at 10:34 AM, Henry Saputra 
> wrote:
>
>> Mark, so I assume we are getting +1 from you?  ;)
>>
>> On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
>> wrote:
>> > There are a few nits left to pick: 'sbt/sbt publish-local' isn't
>> generating
>> > correct POM files because of the way the exclusions are defined in
>> > SparkBuild.scala using wildcards; looks like there may be some broken
>> doc
>> > links generated in that task, as well; DriverSuite doesn't like to run
>> from
>> > the maven build, complaining that 'sbt/sbt assembly' needs to be run
>> first.
>> >
>> > None of these is enough for me to give RC6 a -1.
>> >
>> >
>> > On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
>> matei.zaha...@gmail.com>wrote:
>> >
>> >> +1
>> >>
>> >> Tried new staging repo to make sure the issue with RC5 is fixed.
>> >>
>> >> Matei
>> >>
>> >> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
>> wrote:
>> >>
>> >> > Please vote on releasing the following candidate as Apache Spark
>> >> > (incubating) version 0.8.0. This will be the first incubator release
>> for
>> >> > Spark in Apache.
>> >> >
>> >> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>> >> >
>> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>> >> >
>> >> > The release files, including signatures, digests, etc can be found
>> at:
>> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>> >> >
>> >> > Release artifacts are signed with the following key:
>> >> > https://people.apache.org/keys/committer/pwendell.asc
>> >> >
>> >> > The staging repository for this release can be found at:
>> >> >
>> https://repository.apache.org/content/repositories/orgapachespark-059/
>> >> >
>> >> > The documentation corresponding to this release can be found at:
>> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>> >> >
>> >> > Please vote on releasing this package as Apache Spark
>> 0.8.0-incubating!
>> >> >
>> >> > The vote is open until Friday, September 20th at 09:00 UTC and
>> passes if
>> >> > a majority of at least 3 +1 IPMC votes are cast.
>> >> >
>> >> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>> >> > [ ] -1 Do not release this package because ...
>> >> >
>> >> > To learn more about Apache Spark, please see
>> >> > http://spark.incubator.apache.org/
>> >>
>> >>
>>
>
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Mark Hamstra
If that will make the difference between releasing and not releasing, then
sure; else I'd rather see the last niggles resolved, but that's enough for
me to vote -1 -- so I'm a 0 unless you really need another +1.


On Wed, Sep 18, 2013 at 10:34 AM, Henry Saputra wrote:

> Mark, so I assume we are getting +1 from you?  ;)
>
> On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
> wrote:
> > There are a few nits left to pick: 'sbt/sbt publish-local' isn't
> generating
> > correct POM files because of the way the exclusions are defined in
> > SparkBuild.scala using wildcards; looks like there may be some broken doc
> > links generated in that task, as well; DriverSuite doesn't like to run
> from
> > the maven build, complaining that 'sbt/sbt assembly' needs to be run
> first.
> >
> > None of these is enough for me to give RC6 a -1.
> >
> >
> > On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia  >wrote:
> >
> >> +1
> >>
> >> Tried new staging repo to make sure the issue with RC5 is fixed.
> >>
> >> Matei
> >>
> >> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
> wrote:
> >>
> >> > Please vote on releasing the following candidate as Apache Spark
> >> > (incubating) version 0.8.0. This will be the first incubator release
> for
> >> > Spark in Apache.
> >> >
> >> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> >> >
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
> >> >
> >> > The release files, including signatures, digests, etc can be found at:
> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
> >> >
> >> > Release artifacts are signed with the following key:
> >> > https://people.apache.org/keys/committer/pwendell.asc
> >> >
> >> > The staging repository for this release can be found at:
> >> >
> https://repository.apache.org/content/repositories/orgapachespark-059/
> >> >
> >> > The documentation corresponding to this release can be found at:
> >> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
> >> >
> >> > Please vote on releasing this package as Apache Spark
> 0.8.0-incubating!
> >> >
> >> > The vote is open until Friday, September 20th at 09:00 UTC and passes
> if
> >> > a majority of at least 3 +1 IPMC votes are cast.
> >> >
> >> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> >> > [ ] -1 Do not release this package because ...
> >> >
> >> > To learn more about Apache Spark, please see
> >> > http://spark.incubator.apache.org/
> >>
> >>
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Henry Saputra
Mark, so I assume we are getting +1 from you?  ;)

On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra  wrote:
> There are a few nits left to pick: 'sbt/sbt publish-local' isn't generating
> correct POM files because of the way the exclusions are defined in
> SparkBuild.scala using wildcards; looks like there may be some broken doc
> links generated in that task, as well; DriverSuite doesn't like to run from
> the maven build, complaining that 'sbt/sbt assembly' needs to be run first.
>
> None of these is enough for me to give RC6 a -1.
>
>
> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia 
> wrote:
>
>> +1
>>
>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>
>> Matei
>>
>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:
>>
>> > Please vote on releasing the following candidate as Apache Spark
>> > (incubating) version 0.8.0. This will be the first incubator release for
>> > Spark in Apache.
>> >
>> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>> > https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>> >
>> > The release files, including signatures, digests, etc can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>> >
>> > Release artifacts are signed with the following key:
>> > https://people.apache.org/keys/committer/pwendell.asc
>> >
>> > The staging repository for this release can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-059/
>> >
>> > The documentation corresponding to this release can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>> >
>> > Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>> >
>> > The vote is open until Friday, September 20th at 09:00 UTC and passes if
>> > a majority of at least 3 +1 IPMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see
>> > http://spark.incubator.apache.org/
>>
>>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-18 Thread Henry Saputra
Patrick, dont forget to officially cast your own vote.

On Tue, Sep 17, 2013 at 2:03 AM, Patrick Wendell  wrote:
> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.8.0. This will be the first incubator release for
> Spark in Apache.
>
> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>
> The release files, including signatures, digests, etc can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-059/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>
> Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>
> The vote is open until Friday, September 20th at 09:00 UTC and passes if
> a majority of at least 3 +1 IPMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.incubator.apache.org/


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Josh Rosen
Maybe we could run the tests that depend on the assembly during Maven's
`integration-test` phase, which runs after the `package` phase.  I'm not
sure what its sbt equivalent is.


On Tue, Sep 17, 2013 at 7:20 PM, Matei Zaharia wrote:

> In Maven, mvn package should also create the assembly, but the non-obvious
> thing is that it needs to happen for all projects before mvn test for core
> works. Unfortunately I don't know any easy way around that.
>
> Matei
>
> On Sep 17, 2013, at 1:46 PM, Patrick Wendell  wrote:
>
> > Hey Mark,
> >
> > Good catches here. Ya the driver suite thing is sorta annoying - we
> > should try to fix that in master. The audit script I wrote first does
> > an sbt/sbt assembly to avoid this. I agree though these shouldn't
> > block the release (if a blocker does come up we can revisit these
> > potentially when cutting a release).
> >
> > - Patrick
> >
> > On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra 
> wrote:
> >> There are a few nits left to pick: 'sbt/sbt publish-local' isn't
> generating
> >> correct POM files because of the way the exclusions are defined in
> >> SparkBuild.scala using wildcards; looks like there may be some broken
> doc
> >> links generated in that task, as well; DriverSuite doesn't like to run
> from
> >> the maven build, complaining that 'sbt/sbt assembly' needs to be run
> first.
> >>
> >> None of these is enough for me to give RC6 a -1.
> >>
> >>
> >> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia <
> matei.zaha...@gmail.com>wrote:
> >>
> >>> +1
> >>>
> >>> Tried new staging repo to make sure the issue with RC5 is fixed.
> >>>
> >>> Matei
> >>>
> >>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell 
> wrote:
> >>>
>  Please vote on releasing the following candidate as Apache Spark
>  (incubating) version 0.8.0. This will be the first incubator release
> for
>  Spark in Apache.
> 
>  The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> 
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
> 
>  The release files, including signatures, digests, etc can be found at:
>  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
> 
>  Release artifacts are signed with the following key:
>  https://people.apache.org/keys/committer/pwendell.asc
> 
>  The staging repository for this release can be found at:
> 
> https://repository.apache.org/content/repositories/orgapachespark-059/
> 
>  The documentation corresponding to this release can be found at:
>  http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
> 
>  Please vote on releasing this package as Apache Spark
> 0.8.0-incubating!
> 
>  The vote is open until Friday, September 20th at 09:00 UTC and passes
> if
>  a majority of at least 3 +1 IPMC votes are cast.
> 
>  [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>  [ ] -1 Do not release this package because ...
> 
>  To learn more about Apache Spark, please see
>  http://spark.incubator.apache.org/
> >>>
> >>>
>
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Matei Zaharia
In Maven, mvn package should also create the assembly, but the non-obvious 
thing is that it needs to happen for all projects before mvn test for core 
works. Unfortunately I don't know any easy way around that.

Matei

On Sep 17, 2013, at 1:46 PM, Patrick Wendell  wrote:

> Hey Mark,
> 
> Good catches here. Ya the driver suite thing is sorta annoying - we
> should try to fix that in master. The audit script I wrote first does
> an sbt/sbt assembly to avoid this. I agree though these shouldn't
> block the release (if a blocker does come up we can revisit these
> potentially when cutting a release).
> 
> - Patrick
> 
> On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra  wrote:
>> There are a few nits left to pick: 'sbt/sbt publish-local' isn't generating
>> correct POM files because of the way the exclusions are defined in
>> SparkBuild.scala using wildcards; looks like there may be some broken doc
>> links generated in that task, as well; DriverSuite doesn't like to run from
>> the maven build, complaining that 'sbt/sbt assembly' needs to be run first.
>> 
>> None of these is enough for me to give RC6 a -1.
>> 
>> 
>> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia 
>> wrote:
>> 
>>> +1
>>> 
>>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>> 
>>> Matei
>>> 
>>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:
>>> 
 Please vote on releasing the following candidate as Apache Spark
 (incubating) version 0.8.0. This will be the first incubator release for
 Spark in Apache.
 
 The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
 https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
 
 The release files, including signatures, digests, etc can be found at:
 http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
 
 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/pwendell.asc
 
 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-059/
 
 The documentation corresponding to this release can be found at:
 http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
 
 Please vote on releasing this package as Apache Spark 0.8.0-incubating!
 
 The vote is open until Friday, September 20th at 09:00 UTC and passes if
 a majority of at least 3 +1 IPMC votes are cast.
 
 [ ] +1 Release this package as Apache Spark 0.8.0-incubating
 [ ] -1 Do not release this package because ...
 
 To learn more about Apache Spark, please see
 http://spark.incubator.apache.org/
>>> 
>>> 



Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Patrick Wendell
Hey Mark,

Good catches here. Ya the driver suite thing is sorta annoying - we
should try to fix that in master. The audit script I wrote first does
an sbt/sbt assembly to avoid this. I agree though these shouldn't
block the release (if a blocker does come up we can revisit these
potentially when cutting a release).

- Patrick

On Tue, Sep 17, 2013 at 1:26 PM, Mark Hamstra  wrote:
> There are a few nits left to pick: 'sbt/sbt publish-local' isn't generating
> correct POM files because of the way the exclusions are defined in
> SparkBuild.scala using wildcards; looks like there may be some broken doc
> links generated in that task, as well; DriverSuite doesn't like to run from
> the maven build, complaining that 'sbt/sbt assembly' needs to be run first.
>
> None of these is enough for me to give RC6 a -1.
>
>
> On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia 
> wrote:
>
>> +1
>>
>> Tried new staging repo to make sure the issue with RC5 is fixed.
>>
>> Matei
>>
>> On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:
>>
>> > Please vote on releasing the following candidate as Apache Spark
>> > (incubating) version 0.8.0. This will be the first incubator release for
>> > Spark in Apache.
>> >
>> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
>> > https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>> >
>> > The release files, including signatures, digests, etc can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>> >
>> > Release artifacts are signed with the following key:
>> > https://people.apache.org/keys/committer/pwendell.asc
>> >
>> > The staging repository for this release can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-059/
>> >
>> > The documentation corresponding to this release can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>> >
>> > Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>> >
>> > The vote is open until Friday, September 20th at 09:00 UTC and passes if
>> > a majority of at least 3 +1 IPMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see
>> > http://spark.incubator.apache.org/
>>
>>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Mark Hamstra
There are a few nits left to pick: 'sbt/sbt publish-local' isn't generating
correct POM files because of the way the exclusions are defined in
SparkBuild.scala using wildcards; looks like there may be some broken doc
links generated in that task, as well; DriverSuite doesn't like to run from
the maven build, complaining that 'sbt/sbt assembly' needs to be run first.

None of these is enough for me to give RC6 a -1.


On Tue, Sep 17, 2013 at 11:28 AM, Matei Zaharia wrote:

> +1
>
> Tried new staging repo to make sure the issue with RC5 is fixed.
>
> Matei
>
> On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:
>
> > Please vote on releasing the following candidate as Apache Spark
> > (incubating) version 0.8.0. This will be the first incubator release for
> > Spark in Apache.
> >
> > The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> > https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
> >
> > The release files, including signatures, digests, etc can be found at:
> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/pwendell.asc
> >
> > The staging repository for this release can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-059/
> >
> > The documentation corresponding to this release can be found at:
> > http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
> >
> > Please vote on releasing this package as Apache Spark 0.8.0-incubating!
> >
> > The vote is open until Friday, September 20th at 09:00 UTC and passes if
> > a majority of at least 3 +1 IPMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see
> > http://spark.incubator.apache.org/
>
>


Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Matei Zaharia
+1

Tried new staging repo to make sure the issue with RC5 is fixed.

Matei

On Sep 17, 2013, at 2:03 AM, Patrick Wendell  wrote:

> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.8.0. This will be the first incubator release for
> Spark in Apache.
> 
> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
> 
> The release files, including signatures, digests, etc can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
> 
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-059/
> 
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
> 
> Please vote on releasing this package as Apache Spark 0.8.0-incubating!
> 
> The vote is open until Friday, September 20th at 09:00 UTC and passes if
> a majority of at least 3 +1 IPMC votes are cast.
> 
> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> [ ] -1 Do not release this package because ...
> 
> To learn more about Apache Spark, please see
> http://spark.incubator.apache.org/



Re: [VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Andy Konwinski
+1 (see my comments on RC5)


On Tue, Sep 17, 2013 at 2:03 AM, Patrick Wendell  wrote:

> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.8.0. This will be the first incubator release for
> Spark in Apache.
>
> The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
> https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating
>
> The release files, including signatures, digests, etc can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-059/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/
>
> Please vote on releasing this package as Apache Spark 0.8.0-incubating!
>
> The vote is open until Friday, September 20th at 09:00 UTC and passes if
> a majority of at least 3 +1 IPMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 0.8.0-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.incubator.apache.org/
>


[VOTE] Release Apache Spark 0.8.0-incubating (RC6)

2013-09-17 Thread Patrick Wendell
Please vote on releasing the following candidate as Apache Spark
(incubating) version 0.8.0. This will be the first incubator release for
Spark in Apache.

The tag to be voted on is v0.8.0-incubating (commit 3b85a85):
https://github.com/apache/incubator-spark/releases/tag/v0.8.0-incubating

The release files, including signatures, digests, etc can be found at:
http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/files/

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc

The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-059/

The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-0.8.0-incubating-rc6/docs/

Please vote on releasing this package as Apache Spark 0.8.0-incubating!

The vote is open until Friday, September 20th at 09:00 UTC and passes if
a majority of at least 3 +1 IPMC votes are cast.

[ ] +1 Release this package as Apache Spark 0.8.0-incubating
[ ] -1 Do not release this package because ...

To learn more about Apache Spark, please see
http://spark.incubator.apache.org/