Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Ah oops Andy I see - thought you were discussing something different.
On Tue, Jan 21, 2014 at 1:52 PM, Patrick Wendell wrote:
> This isn't a new RC or a result - I just tried to make an new vote
> thread and the mail clients get confused and merge it in with the old
> thread :(
>
> On Tue, Jan 21, 2014 at 1:04 PM, Andy Konwinski
> wrote:
>> I think we might need to change the subject to [RESULT][VOTE] or something
>> so that the apache scripts can tell when a vote thread ends, even if it's
>> simply being replaced by another vote thread for rc{n+1}.
>>
>>
>> On Tue, Jan 21, 2014 at 3:15 AM, Patrick Wendell wrote:
>>
>>> This vote is cancelled in favor of rc4.
>>>
>>>
>>> On Mon, Jan 20, 2014 at 4:05 PM, Ewen Cheslack-Postava
>>> wrote:
>>>
I'm using sbt and also tried doubling all the values in the sbt/sbt
script. Still fails during the tests, and leaves a sbt.ForkMain process
running (and eating up 100% CPU...).
I can break this out into a separate issue/thread, I just figured it
might be relevant since I'd imagine this is a fairly common setup (Macbook
Air).
-Ewen
Reynold Xin
January 20, 2014 10:15 AM
That's a perm gen issue - you need to adjust the perm gem size. In sbt it
should've been set automatically, but I think for Maven, you need to set
the maven opts, which is documented in the build instructions.
Ewen Cheslack-Postava
January 19, 2014 11:35 PM
I can't get the tests to run on a Mac, 10.7.5, java -version output:
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
For reference, Spark 0.8.* build and test find on the same configuration.
0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's
running at that time since all the tests in PrimitiveVectorSuite seem to
have finished:
[info] PrimitiveVectorSuite:
[info] - primitive value (4 milliseconds)
[info] - non-primitive value (5 milliseconds)
[info] - ideal growth (4 milliseconds)
[info] - ideal size (5 milliseconds)
[info] - resizing (6 milliseconds)
[ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4]
[ActorSystem(spark)] exception while executing timer task
org.apache.spark.SparkException: Error sending message to
BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost,
51634, 0))]
at
org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176)
at
org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52)
at org.apache.spark.storage.BlockManager.org
$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97)
at
org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135)
at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80)
at
akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241)
at
akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464)
at
akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281)
at
akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279)
at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15)
at
akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746)
at
akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
at
>
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
This isn't a new RC or a result - I just tried to make an new vote
thread and the mail clients get confused and merge it in with the old
thread :(
On Tue, Jan 21, 2014 at 1:04 PM, Andy Konwinski wrote:
> I think we might need to change the subject to [RESULT][VOTE] or something
> so that the apache scripts can tell when a vote thread ends, even if it's
> simply being replaced by another vote thread for rc{n+1}.
>
>
> On Tue, Jan 21, 2014 at 3:15 AM, Patrick Wendell wrote:
>
>> This vote is cancelled in favor of rc4.
>>
>>
>> On Mon, Jan 20, 2014 at 4:05 PM, Ewen Cheslack-Postava
>> wrote:
>>
>>> I'm using sbt and also tried doubling all the values in the sbt/sbt
>>> script. Still fails during the tests, and leaves a sbt.ForkMain process
>>> running (and eating up 100% CPU...).
>>>
>>> I can break this out into a separate issue/thread, I just figured it
>>> might be relevant since I'd imagine this is a fairly common setup (Macbook
>>> Air).
>>>
>>> -Ewen
>>>
>>> Reynold Xin
>>> January 20, 2014 10:15 AM
>>> That's a perm gen issue - you need to adjust the perm gem size. In sbt it
>>> should've been set automatically, but I think for Maven, you need to set
>>> the maven opts, which is documented in the build instructions.
>>>
>>>
>>>
>>> Ewen Cheslack-Postava
>>> January 19, 2014 11:35 PM
>>> I can't get the tests to run on a Mac, 10.7.5, java -version output:
>>>
>>> java version "1.6.0_65"
>>> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>>> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>>>
>>> For reference, Spark 0.8.* build and test find on the same configuration.
>>> 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's
>>> running at that time since all the tests in PrimitiveVectorSuite seem to
>>> have finished:
>>>
>>> [info] PrimitiveVectorSuite:
>>> [info] - primitive value (4 milliseconds)
>>> [info] - non-primitive value (5 milliseconds)
>>> [info] - ideal growth (4 milliseconds)
>>> [info] - ideal size (5 milliseconds)
>>> [info] - resizing (6 milliseconds)
>>> [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4]
>>> [ActorSystem(spark)] exception while executing timer task
>>> org.apache.spark.SparkException: Error sending message to
>>> BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost,
>>> 51634, 0))]
>>> at
>>> org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176)
>>> at
>>> org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52)
>>> at org.apache.spark.storage.BlockManager.org
>>> $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97)
>>> at
>>> org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135)
>>> at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80)
>>> at
>>> akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241)
>>> at
>>> akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464)
>>> at
>>> akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281)
>>> at
>>> akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280)
>>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>>> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>>> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>>> at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279)
>>> at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630)
>>> at
>>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582)
>>> at
>>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
>>> at
>>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
>>> at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596)
>>> at
>>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750)
>>> at
>>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753)
>>> at
>>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
>>> at
>>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
>>> at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15)
>>> at
>>> akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746)
>>> at
>>> akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
>>> at
>>> akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
>>> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
>>> at
>>> akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBa
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
I think we might need to change the subject to [RESULT][VOTE] or something
so that the apache scripts can tell when a vote thread ends, even if it's
simply being replaced by another vote thread for rc{n+1}.
On Tue, Jan 21, 2014 at 3:15 AM, Patrick Wendell wrote:
> This vote is cancelled in favor of rc4.
>
>
> On Mon, Jan 20, 2014 at 4:05 PM, Ewen Cheslack-Postava wrote:
>
>> I'm using sbt and also tried doubling all the values in the sbt/sbt
>> script. Still fails during the tests, and leaves a sbt.ForkMain process
>> running (and eating up 100% CPU...).
>>
>> I can break this out into a separate issue/thread, I just figured it
>> might be relevant since I'd imagine this is a fairly common setup (Macbook
>> Air).
>>
>> -Ewen
>>
>> Reynold Xin
>> January 20, 2014 10:15 AM
>> That's a perm gen issue - you need to adjust the perm gem size. In sbt it
>> should've been set automatically, but I think for Maven, you need to set
>> the maven opts, which is documented in the build instructions.
>>
>>
>>
>> Ewen Cheslack-Postava
>> January 19, 2014 11:35 PM
>> I can't get the tests to run on a Mac, 10.7.5, java -version output:
>>
>> java version "1.6.0_65"
>> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>>
>> For reference, Spark 0.8.* build and test find on the same configuration.
>> 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's
>> running at that time since all the tests in PrimitiveVectorSuite seem to
>> have finished:
>>
>> [info] PrimitiveVectorSuite:
>> [info] - primitive value (4 milliseconds)
>> [info] - non-primitive value (5 milliseconds)
>> [info] - ideal growth (4 milliseconds)
>> [info] - ideal size (5 milliseconds)
>> [info] - resizing (6 milliseconds)
>> [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4]
>> [ActorSystem(spark)] exception while executing timer task
>> org.apache.spark.SparkException: Error sending message to
>> BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost,
>> 51634, 0))]
>> at
>> org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176)
>> at
>> org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52)
>> at org.apache.spark.storage.BlockManager.org
>> $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97)
>> at
>> org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135)
>> at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80)
>> at
>> akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241)
>> at
>> akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464)
>> at
>> akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281)
>> at
>> akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280)
>> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>> at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279)
>> at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630)
>> at
>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582)
>> at
>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
>> at
>> akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
>> at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596)
>> at
>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750)
>> at
>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753)
>> at
>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
>> at
>> akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
>> at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15)
>> at
>> akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746)
>> at
>> akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
>> at
>> akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
>> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
>> at
>> akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
>> at
>> akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
>> at
>> akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
>> at
>> akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(Batchin
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
This vote is cancelled in favor of rc4. On Mon, Jan 20, 2014 at 4:05 PM, Ewen Cheslack-Postava wrote: > I'm using sbt and also tried doubling all the values in the sbt/sbt > script. Still fails during the tests, and leaves a sbt.ForkMain process > running (and eating up 100% CPU...). > > I can break this out into a separate issue/thread, I just figured it might > be relevant since I'd imagine this is a fairly common setup (Macbook Air). > > -Ewen > > Reynold Xin > January 20, 2014 10:15 AM > That's a perm gen issue - you need to adjust the perm gem size. In sbt it > should've been set automatically, but I think for Maven, you need to set > the maven opts, which is documented in the build instructions. > > > > Ewen Cheslack-Postava > January 19, 2014 11:35 PM > I can't get the tests to run on a Mac, 10.7.5, java -version output: > > java version "1.6.0_65" > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609) > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode) > > For reference, Spark 0.8.* build and test find on the same configuration. > 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's > running at that time since all the tests in PrimitiveVectorSuite seem to > have finished: > > [info] PrimitiveVectorSuite: > [info] - primitive value (4 milliseconds) > [info] - non-primitive value (5 milliseconds) > [info] - ideal growth (4 milliseconds) > [info] - ideal size (5 milliseconds) > [info] - resizing (6 milliseconds) > [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4] > [ActorSystem(spark)] exception while executing timer task > org.apache.spark.SparkException: Error sending message to > BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost, > 51634, 0))] > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > at > akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464) > at > akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281) > at > akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280) > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279) > at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) > at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) > at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15) > at > akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746) > at > akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) > at > akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) > at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) > at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) > at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) > at > akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) > at scala.c
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
I'm using sbt and also tried doubling all the values in the sbt/sbt script. Still fails during the tests, and leaves a sbt.ForkMain process running (and eating up 100% CPU...). I can break this out into a separate issue/thread, I just figured it might be relevant since I'd imagine this is a fairly common setup (Macbook Air). -Ewen Reynold Xin January 20, 2014 10:15 AM That's a perm gen issue - you need to adjust the perm gem size. In sbt it should've been set automatically, but I think for Maven, you need to set the maven opts, which is documented in the build instructions. Ewen Cheslack-Postava January 19, 2014 11:35 PM I can't get the tests to run on a Mac, 10.7.5, java -version output: java version "1.6.0_65" Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609) Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode) For reference, Spark 0.8.* build and test find on the same configuration. 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's running at that time since all the tests in PrimitiveVectorSuite seem to have finished: [info] PrimitiveVectorSuite: [info] - primitive value (4 milliseconds) [info] - non-primitive value (5 milliseconds) [info] - ideal growth (4 milliseconds) [info] - ideal size (5 milliseconds) [info] - resizing (6 milliseconds) [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4] [ActorSystem(spark)] exception while executing timer task org.apache.spark.SparkException: Error sending message to BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost, 51634, 0))] at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176) at org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) at org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) at akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) at akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464) at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281) at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279) at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15) at akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoin
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
running the test cases, always block after passing “Interact with Files” in ReplSuite.scalaI checked the code, it is test("local-cluster mode") {What happened here?
-- Nan Zhu
On Monday, January 20, 2014 at 1:15 PM, Reynold Xin wrote:
That's a perm gen issue - you need to adjust the perm gem size. In sbt it should've been set automatically, but I think for Maven, you need to set the maven opts, which is documented in the build instructions.
On Sun, Jan 19, 2014 at 11:35 PM, Ewen Cheslack-Postava wrote:
I can't get the tests to
run on a Mac, 10.7.5, java -version output:
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
For reference, Spark 0.8.* build and test find on the same
configuration. 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not
sure what it's running at that time since all the tests in
PrimitiveVectorSuite seem to have finished:
[info] PrimitiveVectorSuite:
[info] - primitive value (4 milliseconds)
[info] - non-primitive value (5 milliseconds)
[info] - ideal growth (4 milliseconds)
[info] - ideal size (5 milliseconds)
[info] - resizing (6 milliseconds)
[ERROR] [01/19/2014 23:16:27.508]
[spark-akka.actor.default-dispatcher-4] [ActorSystem(spark)] exception
while executing timer task
org.apache.spark.SparkException: Error sending message to
BlockManagerMaster [message = HeartBeat(BlockManagerId(,
localhost, 51634, 0))]
at
org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176)
at
org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52)
at
org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97)
at
org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135)
at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80)
at
akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241)
at
akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464)
at
akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281)
at
akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279)
at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
at
akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582)
at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
at
akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746)
at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15)
at
akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746)
at
akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
at
akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at
akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
at
scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at
akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
at
scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinP
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
That's a perm gen issue - you need to adjust the perm gem size. In sbt it should've been set automatically, but I think for Maven, you need to set the maven opts, which is documented in the build instructions. On Sun, Jan 19, 2014 at 11:35 PM, Ewen Cheslack-Postava wrote: > I can't get the tests to run on a Mac, 10.7.5, java -version output: > > java version "1.6.0_65" > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609) > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode) > > For reference, Spark 0.8.* build and test find on the same configuration. > 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's > running at that time since all the tests in PrimitiveVectorSuite seem to > have finished: > > [info] PrimitiveVectorSuite: > [info] - primitive value (4 milliseconds) > [info] - non-primitive value (5 milliseconds) > [info] - ideal growth (4 milliseconds) > [info] - ideal size (5 milliseconds) > [info] - resizing (6 milliseconds) > [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4] > [ActorSystem(spark)] exception while executing timer task > org.apache.spark.SparkException: Error sending message to > BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost, > 51634, 0))] > at > org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176) > at > org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) > at org.apache.spark.storage.BlockManager.org > $apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) > at > org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) > at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) > at > akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) > at > akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464) > at > akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281) > at > akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280) > at scala.collection.Iterator$class.foreach(Iterator.scala:727) > at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) > at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) > at scala.collection.AbstractIterable.foreach(Iterable.scala:54) > at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279) > at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) > at > akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) > at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) > at > akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) > at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15) > at > akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746) > at > akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) > at > akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) > at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) > at > scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) > at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) > at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) > at > akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > Caused by: akka.pattern.AskTimeoutException: > Recipient[Actor[akka://spark/user/BlockManagerMaster#927284646]] had > already been terminated. > at akka.pattern.Askable
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
I can't get the tests to run on a Mac, 10.7.5, java -version output: java version "1.6.0_65" Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609) Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode) For reference, Spark 0.8.* build and test find on the same configuration. 0.9.0-rc3 fails *after* PrimitiveVectorSuite, I'm not sure what it's running at that time since all the tests in PrimitiveVectorSuite seem to have finished: [info] PrimitiveVectorSuite: [info] - primitive value (4 milliseconds) [info] - non-primitive value (5 milliseconds) [info] - ideal growth (4 milliseconds) [info] - ideal size (5 milliseconds) [info] - resizing (6 milliseconds) [ERROR] [01/19/2014 23:16:27.508] [spark-akka.actor.default-dispatcher-4] [ActorSystem(spark)] exception while executing timer task org.apache.spark.SparkException: Error sending message to BlockManagerMaster [message = HeartBeat(BlockManagerId(, localhost, 51634, 0))] at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:176) at org.apache.spark.storage.BlockManagerMaster.sendHeartBeat(BlockManagerMaster.scala:52) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$heartBeat(BlockManager.scala:97) at org.apache.spark.storage.BlockManager$$anonfun$initialize$1.apply$mcV$sp(BlockManager.scala:135) at akka.actor.Scheduler$$anon$9.run(Scheduler.scala:80) at akka.actor.LightArrayRevolverScheduler$$anon$3$$anon$2.run(Scheduler.scala:241) at akka.actor.LightArrayRevolverScheduler$TaskHolder.run(Scheduler.scala:464) at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:281) at akka.actor.LightArrayRevolverScheduler$$anonfun$close$1.apply(Scheduler.scala:280) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at akka.actor.LightArrayRevolverScheduler.close(Scheduler.scala:279) at akka.actor.ActorSystemImpl.stopScheduler(ActorSystem.scala:630) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply$mcV$sp(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anonfun$_start$1.apply(ActorSystem.scala:582) at akka.actor.ActorSystemImpl$$anon$3.run(ActorSystem.scala:596) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.runNext$1(ActorSystem.scala:750) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply$mcV$sp(ActorSystem.scala:753) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$TerminationCallbacks$$anonfun$run$1.apply(ActorSystem.scala:746) at akka.util.ReentrantGuard.withGuard(LockUtil.scala:15) at akka.actor.ActorSystemImpl$TerminationCallbacks.run(ActorSystem.scala:746) at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) at akka.actor.ActorSystemImpl$$anonfun$terminationCallbacks$1.apply(ActorSystem.scala:593) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72) at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: akka.pattern.AskTimeoutException: Recipient[Actor[akka://spark/user/BlockManagerMaster#927284646]] had already been terminated. at akka.pattern.AskableActorRef$.ask$extension(AskSupport.scala:134) at org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:161) ... 39 more [ERROR] [01/19/2014 23:20:03.131] [sparkWorker2-akka.actor.default-dispatcher-19] [ActorSystem(sparkWorker2)] Uncaught fatal error from thread [sparkWorker2-akka.actor.default-dispatcher-19] shutting down ActorSystem [sparkWorker2] java.lang.OutOfMemor
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Ah yes, makes sense, thanks! - Henry On Sun, Jan 19, 2014 at 10:01 PM, Patrick Wendell wrote: > Eventually the notes get posted on the apache website. I attached them > to this e-mail so that people can get a sense of what is in the > release before they vote on it. > > On Sun, Jan 19, 2014 at 9:57 PM, Henry Saputra > wrote: >> Hi Patrick, quick question, where are you planning to add the release notes? >> I dont think it is part of the source, is it? >> >> - Henry >> >> On Sun, Jan 19, 2014 at 8:41 PM, Patrick Wendell wrote: >>> Attempting to attach the release notes again (I think it may have been >>> blocked previously due to not having an extension). >>> >>> On Sun, Jan 19, 2014 at 8:05 PM, Patrick Wendell wrote: I'll add my +1 as well On Sun, Jan 19, 2014 at 7:33 PM, Matei Zaharia wrote: > +1 > > Re-tested on Mac. > > Matei > > On Jan 19, 2014, at 7:09 PM, Tathagata Das > wrote: > >> Starting off. >> +1 >> >> >> On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell >> wrote: >> >>> Please vote on releasing the following candidate as Apache Spark >>> (incubating) version 0.9.0. >>> >>> A draft of the release notes along with the changes file is attached >>> to this e-mail. >>> >>> The tag to be voted on is v0.9.0-incubating (commit a7760eff): >>> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d >>> >>> The release files, including signatures, digests, etc can be found at: >>> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ >>> >>> Release artifacts are signed with the following key: >>> https://people.apache.org/keys/committer/pwendell.asc >>> >>> The staging repository for this release can be found at: >>> https://repository.apache.org/content/repositories/orgapachespark-1004/ >>> >>> The documentation corresponding to this release can be found at: >>> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ >>> >>> Please vote on releasing this package as Apache Spark 0.9.0-incubating! >>> >>> The vote is open until Wednesday, January 22, at 22:15 UTC and passes >>> if a majority of at least 3 +1 PPMC votes are cast. >>> >>> [ ] +1 Release this package as Apache Spark 0.9.0-incubating >>> [ ] -1 Do not release this package because ... >>> >>> To learn more about Apache Spark, please see >>> http://spark.incubator.apache.org/ >>> >
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Eventually the notes get posted on the apache website. I attached them to this e-mail so that people can get a sense of what is in the release before they vote on it. On Sun, Jan 19, 2014 at 9:57 PM, Henry Saputra wrote: > Hi Patrick, quick question, where are you planning to add the release notes? > I dont think it is part of the source, is it? > > - Henry > > On Sun, Jan 19, 2014 at 8:41 PM, Patrick Wendell wrote: >> Attempting to attach the release notes again (I think it may have been >> blocked previously due to not having an extension). >> >> On Sun, Jan 19, 2014 at 8:05 PM, Patrick Wendell wrote: >>> I'll add my +1 as well >>> >>> On Sun, Jan 19, 2014 at 7:33 PM, Matei Zaharia >>> wrote: +1 Re-tested on Mac. Matei On Jan 19, 2014, at 7:09 PM, Tathagata Das wrote: > Starting off. > +1 > > > On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell > wrote: > >> Please vote on releasing the following candidate as Apache Spark >> (incubating) version 0.9.0. >> >> A draft of the release notes along with the changes file is attached >> to this e-mail. >> >> The tag to be voted on is v0.9.0-incubating (commit a7760eff): >> >> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d >> >> The release files, including signatures, digests, etc can be found at: >> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ >> >> Release artifacts are signed with the following key: >> https://people.apache.org/keys/committer/pwendell.asc >> >> The staging repository for this release can be found at: >> https://repository.apache.org/content/repositories/orgapachespark-1004/ >> >> The documentation corresponding to this release can be found at: >> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ >> >> Please vote on releasing this package as Apache Spark 0.9.0-incubating! >> >> The vote is open until Wednesday, January 22, at 22:15 UTC and passes >> if a majority of at least 3 +1 PPMC votes are cast. >> >> [ ] +1 Release this package as Apache Spark 0.9.0-incubating >> [ ] -1 Do not release this package because ... >> >> To learn more about Apache Spark, please see >> http://spark.incubator.apache.org/ >>
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Hi Patrick, quick question, where are you planning to add the release notes? I dont think it is part of the source, is it? - Henry On Sun, Jan 19, 2014 at 8:41 PM, Patrick Wendell wrote: > Attempting to attach the release notes again (I think it may have been > blocked previously due to not having an extension). > > On Sun, Jan 19, 2014 at 8:05 PM, Patrick Wendell wrote: >> I'll add my +1 as well >> >> On Sun, Jan 19, 2014 at 7:33 PM, Matei Zaharia >> wrote: >>> +1 >>> >>> Re-tested on Mac. >>> >>> Matei >>> >>> On Jan 19, 2014, at 7:09 PM, Tathagata Das >>> wrote: >>> Starting off. +1 On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell wrote: > Please vote on releasing the following candidate as Apache Spark > (incubating) version 0.9.0. > > A draft of the release notes along with the changes file is attached > to this e-mail. > > The tag to be voted on is v0.9.0-incubating (commit a7760eff): > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d > > The release files, including signatures, digests, etc can be found at: > http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/pwendell.asc > > The staging repository for this release can be found at: > https://repository.apache.org/content/repositories/orgapachespark-1004/ > > The documentation corresponding to this release can be found at: > http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ > > Please vote on releasing this package as Apache Spark 0.9.0-incubating! > > The vote is open until Wednesday, January 22, at 22:15 UTC and passes > if a majority of at least 3 +1 PPMC votes are cast. > > [ ] +1 Release this package as Apache Spark 0.9.0-incubating > [ ] -1 Do not release this package because ... > > To learn more about Apache Spark, please see > http://spark.incubator.apache.org/ > >>>
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Attempting to attach the release notes again (I think it may have been blocked previously due to not having an extension). On Sun, Jan 19, 2014 at 8:05 PM, Patrick Wendell wrote: > I'll add my +1 as well > > On Sun, Jan 19, 2014 at 7:33 PM, Matei Zaharia > wrote: >> +1 >> >> Re-tested on Mac. >> >> Matei >> >> On Jan 19, 2014, at 7:09 PM, Tathagata Das >> wrote: >> >>> Starting off. >>> +1 >>> >>> >>> On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell wrote: >>> Please vote on releasing the following candidate as Apache Spark (incubating) version 0.9.0. A draft of the release notes along with the changes file is attached to this e-mail. The tag to be voted on is v0.9.0-incubating (commit a7760eff): https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d The release files, including signatures, digests, etc can be found at: http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1004/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ Please vote on releasing this package as Apache Spark 0.9.0-incubating! The vote is open until Wednesday, January 22, at 22:15 UTC and passes if a majority of at least 3 +1 PPMC votes are cast. [ ] +1 Release this package as Apache Spark 0.9.0-incubating [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.incubator.apache.org/ >> Spark 0.9.0 is a major release that adds significant new features. It updates Spark to Scala 2.10, simplifies high availability, and updates numerous components of the project. This release includes a first version of GraphX, a powerful new framework for graph processing that comes with a library of standard algorithms. In addition, Spark Streaming is now out of alpha, and includes significant optimizations and simplified high availability deployment. ### Scala 2.10 Support Spark now runs on Scala 2.10, letting users benefit from the language and library improvements in this version. ### Configuration System The new [SparkConf] class is now the preferred way to configure advanced settings on your SparkContext, though the previous Java system property still works. SparkConf is especially useful in tests to make sure properties donât stay set across tests. ### Spark Streaming Improvements Spark Streaming is no longer alpha, and comes with simplified high availability and several optimizations. * When running on a Spark standalone cluster with the [standalone cluster high availability mode], you can submit a Spark Streaming driver application to the cluster and have it automatically recovered if either the driver or the cluster master crashes. * Windowed operators have been sped up by 30-50%. * Spark Streamingâs input source plugins (e.g. for Twitter, Kafka and Flume) are now separate projects, making it easier to pull in only the dependencies you need. * A new StreamingListener interface has been added for monitoring statistics about the streaming computation. * A few aspects of the API have been improved: * `DStream` and `PairDStream` classes have been moved from `org.apache.spark.streaming` to `org.apache.spark.streaming.dstream` to keep it consistent with `org.apache.spark.rdd.RDD`. * `DStream.foreach` -> `DStream.foreachRDD` to make it explicit that it works for every RDD, not every element * `StreamingContext.awaitTermination()` allows you wait for context shutdown and catch any exception that occurs in the streaming computation. *`StreamingContext.stop()` now allows stopping of StreamingContext without stopping the underlying SparkContext. ### GraphX Alpha GraphX is a new API for graph processing that uses recent advances in graph-parallel computation. It lets you build a graph within a Spark program using the standard Spark operators, then process it with new graph operators that are optimized for distributed computation. It includes basic transformations, a Pregel API for iterative computation, and a standard library of graph loaders and analytics algorithms. By offering these features within the Spark engine, GraphX can significantly speed up processing tasks compared to workflows that use different engines. GraphX features in this release include: * Building graphs from arbitrary Spark RDDs * Basic operations to transform graphs or extract subgraphs * An optimized Pregel API that takes advantage of graph partitioning and indexing * Standard algorithms including PageRank, connected components, strongly connected
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
I'll add my +1 as well On Sun, Jan 19, 2014 at 7:33 PM, Matei Zaharia wrote: > +1 > > Re-tested on Mac. > > Matei > > On Jan 19, 2014, at 7:09 PM, Tathagata Das > wrote: > >> Starting off. >> +1 >> >> >> On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell wrote: >> >>> Please vote on releasing the following candidate as Apache Spark >>> (incubating) version 0.9.0. >>> >>> A draft of the release notes along with the changes file is attached >>> to this e-mail. >>> >>> The tag to be voted on is v0.9.0-incubating (commit a7760eff): >>> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d >>> >>> The release files, including signatures, digests, etc can be found at: >>> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ >>> >>> Release artifacts are signed with the following key: >>> https://people.apache.org/keys/committer/pwendell.asc >>> >>> The staging repository for this release can be found at: >>> https://repository.apache.org/content/repositories/orgapachespark-1004/ >>> >>> The documentation corresponding to this release can be found at: >>> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ >>> >>> Please vote on releasing this package as Apache Spark 0.9.0-incubating! >>> >>> The vote is open until Wednesday, January 22, at 22:15 UTC and passes >>> if a majority of at least 3 +1 PPMC votes are cast. >>> >>> [ ] +1 Release this package as Apache Spark 0.9.0-incubating >>> [ ] -1 Do not release this package because ... >>> >>> To learn more about Apache Spark, please see >>> http://spark.incubator.apache.org/ >>> >
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
+1 Re-tested on Mac. Matei On Jan 19, 2014, at 7:09 PM, Tathagata Das wrote: > Starting off. > +1 > > > On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell wrote: > >> Please vote on releasing the following candidate as Apache Spark >> (incubating) version 0.9.0. >> >> A draft of the release notes along with the changes file is attached >> to this e-mail. >> >> The tag to be voted on is v0.9.0-incubating (commit a7760eff): >> >> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d >> >> The release files, including signatures, digests, etc can be found at: >> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ >> >> Release artifacts are signed with the following key: >> https://people.apache.org/keys/committer/pwendell.asc >> >> The staging repository for this release can be found at: >> https://repository.apache.org/content/repositories/orgapachespark-1004/ >> >> The documentation corresponding to this release can be found at: >> http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ >> >> Please vote on releasing this package as Apache Spark 0.9.0-incubating! >> >> The vote is open until Wednesday, January 22, at 22:15 UTC and passes >> if a majority of at least 3 +1 PPMC votes are cast. >> >> [ ] +1 Release this package as Apache Spark 0.9.0-incubating >> [ ] -1 Do not release this package because ... >> >> To learn more about Apache Spark, please see >> http://spark.incubator.apache.org/ >>
Re: [VOTE] Release Apache Spark 0.9.0-incubating (rc3)
Starting off. +1 On Sun, Jan 19, 2014 at 2:15 PM, Patrick Wendell wrote: > Please vote on releasing the following candidate as Apache Spark > (incubating) version 0.9.0. > > A draft of the release notes along with the changes file is attached > to this e-mail. > > The tag to be voted on is v0.9.0-incubating (commit a7760eff): > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=commit;h=a7760eff4ea6a474cab68896a88550f63bae8b0d > > The release files, including signatures, digests, etc can be found at: > http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/pwendell.asc > > The staging repository for this release can be found at: > https://repository.apache.org/content/repositories/orgapachespark-1004/ > > The documentation corresponding to this release can be found at: > http://people.apache.org/~pwendell/spark-0.9.0-incubating-rc3-docs/ > > Please vote on releasing this package as Apache Spark 0.9.0-incubating! > > The vote is open until Wednesday, January 22, at 22:15 UTC and passes > if a majority of at least 3 +1 PPMC votes are cast. > > [ ] +1 Release this package as Apache Spark 0.9.0-incubating > [ ] -1 Do not release this package because ... > > To learn more about Apache Spark, please see > http://spark.incubator.apache.org/ >
