Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-27 Thread Reynold Xin 
Yup looks like I missed that. I will build a new one.

On Tuesday, October 27, 2015, Sean Owen  wrote:

> Ah, good point. I also see it still reads 1.5.1. I imagine we just need
> another sweep to update all the version strings.
>
> On Tue, Oct 27, 2015 at 3:08 AM, Krishna Sankar  > wrote:
>
>> Guys,
>>The sc.version returns 1.5.1 in python and scala. Is anyone getting
>> the same results ? Probably I am doing something wrong.
>> Cheers
>> 
>>
>> On Sun, Oct 25, 2015 at 12:07 AM, Reynold Xin > > wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark
>>> version 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and
>>> passes if a majority of at least 3 +1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.5.2
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> The release fixes 51 known issues in Spark 1.5.1, listed here:
>>> http://s.apache.org/spark-1.5.2
>>>
>>> The tag to be voted on is v1.5.2-rc1:
>>> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> *http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
>>> *
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release can be found at:
>>> - as version 1.5.2-rc1:
>>> https://repository.apache.org/content/repositories/orgapachespark-1151
>>> - as version 1.5.2:
>>> https://repository.apache.org/content/repositories/orgapachespark-1150
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>>>
>>>
>>> ===
>>> How can I help test this release?
>>> ===
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> 
>>> What justifies a -1 vote for this release?
>>> 
>>> -1 vote should occur for regressions from Spark 1.5.1. Bugs already
>>> present in 1.5.1 will not block this release.
>>>
>>> ===
>>> What should happen to JIRA tickets still targeting 1.5.2?
>>> ===
>>> Please target 1.5.3 or 1.6.0.
>>>
>>>
>>>
>>
>

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-27 Thread Sean Owen 
Ah, good point. I also see it still reads 1.5.1. I imagine we just need
another sweep to update all the version strings.

On Tue, Oct 27, 2015 at 3:08 AM, Krishna Sankar  wrote:

> Guys,
>The sc.version returns 1.5.1 in python and scala. Is anyone getting the
> same results ? Probably I am doing something wrong.
> Cheers
> 
>
> On Sun, Oct 25, 2015 at 12:07 AM, Reynold Xin  wrote:
>
>> Please vote on releasing the following candidate as Apache Spark
>> version 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and
>> passes if a majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.5.2
>> [ ] -1 Do not release this package because ...
>>
>>
>> The release fixes 51 known issues in Spark 1.5.1, listed here:
>> http://s.apache.org/spark-1.5.2
>>
>> The tag to be voted on is v1.5.2-rc1:
>> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> *http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
>> *
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>> - as version 1.5.2-rc1:
>> https://repository.apache.org/content/repositories/orgapachespark-1151
>> - as version 1.5.2:
>> https://repository.apache.org/content/repositories/orgapachespark-1150
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>>
>>
>> ===
>> How can I help test this release?
>> ===
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> 
>> What justifies a -1 vote for this release?
>> 
>> -1 vote should occur for regressions from Spark 1.5.1. Bugs already
>> present in 1.5.1 will not block this release.
>>
>> ===
>> What should happen to JIRA tickets still targeting 1.5.2?
>> ===
>> Please target 1.5.3 or 1.6.0.
>>
>>
>>
>

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-26 Thread Patrick Wendell 
I verified that the issue with build binaries being present in the source
release is fixed. Haven't done enough vetting for a full vote, but did
verify that.

On Sun, Oct 25, 2015 at 12:07 AM, Reynold Xin  wrote:

> Please vote on releasing the following candidate as Apache Spark
> version 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and
> passes if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.2
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 51 known issues in Spark 1.5.1, listed here:
> http://s.apache.org/spark-1.5.2
>
> The tag to be voted on is v1.5.2-rc1:
> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> *http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
> *
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> - as version 1.5.2-rc1:
> https://repository.apache.org/content/repositories/orgapachespark-1151
> - as version 1.5.2:
> https://repository.apache.org/content/repositories/orgapachespark-1150
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>
>
> ===
> How can I help test this release?
> ===
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> 
> What justifies a -1 vote for this release?
> 
> -1 vote should occur for regressions from Spark 1.5.1. Bugs already
> present in 1.5.1 will not block this release.
>
> ===
> What should happen to JIRA tickets still targeting 1.5.2?
> ===
> Please target 1.5.3 or 1.6.0.
>
>
>

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-26 Thread Krishna Sankar 
Guys,
   The sc.version returns 1.5.1 in python and scala. Is anyone getting the
same results ? Probably I am doing something wrong.
Cheers


On Sun, Oct 25, 2015 at 12:07 AM, Reynold Xin  wrote:

> Please vote on releasing the following candidate as Apache Spark
> version 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and
> passes if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.2
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 51 known issues in Spark 1.5.1, listed here:
> http://s.apache.org/spark-1.5.2
>
> The tag to be voted on is v1.5.2-rc1:
> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> *http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
> *
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> - as version 1.5.2-rc1:
> https://repository.apache.org/content/repositories/orgapachespark-1151
> - as version 1.5.2:
> https://repository.apache.org/content/repositories/orgapachespark-1150
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>
>
> ===
> How can I help test this release?
> ===
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> 
> What justifies a -1 vote for this release?
> 
> -1 vote should occur for regressions from Spark 1.5.1. Bugs already
> present in 1.5.1 will not block this release.
>
> ===
> What should happen to JIRA tickets still targeting 1.5.2?
> ===
> Please target 1.5.3 or 1.6.0.
>
>
>

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Sean Owen 
The signatures and licenses are fine. I continue to get failures in
these tests though, with "-Pyarn -Phadoop-2.6 -Phive
-Phive-thriftserver" on Ubuntu 15 / Java 7.

- Unpersisting HttpBroadcast on executors and driver in distributed
mode *** FAILED ***
  java.util.concurrent.TimeoutException: Can't find 2 executors before
1 milliseconds elapsed
  at 
org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561)
  at 
org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313)
  at 
org.apache.spark.broadcast.BroadcastSuite.org$apache$spark$broadcast$BroadcastSuite$$testUnpersistHttpBroadcast(BroadcastSuite.scala:238)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply$mcV$sp(BroadcastSuite.scala:149)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  ...
- Unpersisting TorrentBroadcast on executors only in local mode
- Unpersisting TorrentBroadcast on executors and driver in local mode
- Unpersisting TorrentBroadcast on executors only in distributed mode
- Unpersisting TorrentBroadcast on executors and driver in distributed
mode *** FAILED ***
  java.util.concurrent.TimeoutException: Can't find 2 executors before
1 milliseconds elapsed
  at 
org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561)
  at 
org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313)
  at 
org.apache.spark.broadcast.BroadcastSuite.org$apache$spark$broadcast$BroadcastSuite$$testUnpersistTorrentBroadcast(BroadcastSuite.scala:287)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply$mcV$sp(BroadcastSuite.scala:165)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165)
  at 
org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165)
  at 
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  ...

On Sun, Oct 25, 2015 at 7:07 AM, Reynold Xin  wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.2
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 51 known issues in Spark 1.5.1, listed here:
> http://s.apache.org/spark-1.5.2
>
> The tag to be voted on is v1.5.2-rc1:
> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> - as version 1.5.2-rc1:
> https://repository.apache.org/content/repositories/orgapachespark-1151
> - as version 1.5.2:
> https://repository.apache.org/content/repositories/orgapachespark-1150
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>
>
> ===
> How can I help test this release?
> ===
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> 
> What justifies a -1 vote for this release?
> 
> -1 vote should occur for regressions from Spark 1.5.1. Bugs already present
> in 1.5.1 will not block this release.
>
> ===
> What should happen to JIRA tickets still targeting 1.5.2?
> ===
> Please target 1.5.3 or 1.6.0.
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Mark Hamstra 
Should 1.5.2 wait for Josh's fix of SPARK-11293?

On Sun, Oct 25, 2015 at 2:25 PM, Sean Owen  wrote:

> The signatures and licenses are fine. I continue to get failures in
> these tests though, with "-Pyarn -Phadoop-2.6 -Phive
> -Phive-thriftserver" on Ubuntu 15 / Java 7.
>
> - Unpersisting HttpBroadcast on executors and driver in distributed
> mode *** FAILED ***
>   java.util.concurrent.TimeoutException: Can't find 2 executors before
> 1 milliseconds elapsed
>   at
> org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561)
>   at
> org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313)
>   at org.apache.spark.broadcast.BroadcastSuite.org
> $apache$spark$broadcast$BroadcastSuite$$testUnpersistHttpBroadcast(BroadcastSuite.scala:238)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply$mcV$sp(BroadcastSuite.scala:149)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$12.apply(BroadcastSuite.scala:149)
>   at
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
>   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   ...
> - Unpersisting TorrentBroadcast on executors only in local mode
> - Unpersisting TorrentBroadcast on executors and driver in local mode
> - Unpersisting TorrentBroadcast on executors only in distributed mode
> - Unpersisting TorrentBroadcast on executors and driver in distributed
> mode *** FAILED ***
>   java.util.concurrent.TimeoutException: Can't find 2 executors before
> 1 milliseconds elapsed
>   at
> org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561)
>   at
> org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313)
>   at org.apache.spark.broadcast.BroadcastSuite.org
> $apache$spark$broadcast$BroadcastSuite$$testUnpersistTorrentBroadcast(BroadcastSuite.scala:287)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply$mcV$sp(BroadcastSuite.scala:165)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165)
>   at
> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165)
>   at
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
>   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
>   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
>   at org.scalatest.Transformer.apply(Transformer.scala:22)
>   ...
>
> On Sun, Oct 25, 2015 at 7:07 AM, Reynold Xin  wrote:
> > Please vote on releasing the following candidate as Apache Spark version
> > 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and passes
> if a
> > majority of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Spark 1.5.2
> > [ ] -1 Do not release this package because ...
> >
> >
> > The release fixes 51 known issues in Spark 1.5.1, listed here:
> > http://s.apache.org/spark-1.5.2
> >
> > The tag to be voted on is v1.5.2-rc1:
> > https://github.com/apache/spark/releases/tag/v1.5.2-rc1
> >
> > The release files, including signatures, digests, etc. can be found at:
> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/pwendell.asc
> >
> > The staging repository for this release can be found at:
> > - as version 1.5.2-rc1:
> > https://repository.apache.org/content/repositories/orgapachespark-1151
> > - as version 1.5.2:
> > https://repository.apache.org/content/repositories/orgapachespark-1150
> >
> > The documentation corresponding to this release can be found at:
> > http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
> >
> >
> > ===
> > How can I help test this release?
> > ===
> > If you are a Spark user, you can help us test this release by taking an
> > existing Spark workload and running on this release candidate, then
> > reporting any regressions.
> >
> > 
> > What justifies a -1 vote for this release?
> > 
> > -1 vote should occur for regressions from Spark 1.5.1. Bugs already
> present
> > in 1.5.1 will not block this release.
> >
> > ===
> > What should happen to JIRA tickets still targeting 1.5.2?
> > ===
> > Please target 1.5.3 or 1.6.0.
> >
> >
>
> -
> To unsubscribe,

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Sean Owen 
I believe you still need to "clean package" and then "test"
separately. Or did the change to make that unnecessary go in to 1.5?

FWIW I do not see this (my results coming soon)

On Sun, Oct 25, 2015 at 4:28 PM, Ted Yu  wrote:
> When I ran the following command:
> ~/apache-maven-3.3.3/bin/mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.4
> -Dhadoop.version=2.6.0 package
>
> I got:
>
> testChildProcLauncher(org.apache.spark.launcher.SparkLauncherSuite)  Time
> elapsed: 0.031 sec  <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:472)
> at org.junit.Assert.assertEquals(Assert.java:456)
> at
> org.apache.spark.launcher.SparkLauncherSuite.testChildProcLauncher(SparkLauncherSuite.java:105)
>
> java version "1.7.0_67"
> Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>
> I checked
> ./launcher/target/surefire-reports/TEST-org.apache.spark.launcher.SparkLauncherSuite.xml
> but didn't get much clue.
>
> FYI
>
> On Sun, Oct 25, 2015 at 12:07 AM, Reynold Xin  wrote:
>>
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and passes if a
>> majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.5.2
>> [ ] -1 Do not release this package because ...
>>
>>
>> The release fixes 51 known issues in Spark 1.5.1, listed here:
>> http://s.apache.org/spark-1.5.2
>>
>> The tag to be voted on is v1.5.2-rc1:
>> https://github.com/apache/spark/releases/tag/v1.5.2-rc1
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.2-rc1-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>> - as version 1.5.2-rc1:
>> https://repository.apache.org/content/repositories/orgapachespark-1151
>> - as version 1.5.2:
>> https://repository.apache.org/content/repositories/orgapachespark-1150
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-v1.5.2-rc1-docs/
>>
>>
>> ===
>> How can I help test this release?
>> ===
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> 
>> What justifies a -1 vote for this release?
>> 
>> -1 vote should occur for regressions from Spark 1.5.1. Bugs already
>> present in 1.5.1 will not block this release.
>>
>> ===
>> What should happen to JIRA tickets still targeting 1.5.2?
>> ===
>> Please target 1.5.3 or 1.6.0.
>>
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org