Spark Summit 2013 videos available now

2013-12-13 Thread Andy Konwinski
Hi Spark user@ and dev@ list members,

We are happy to announce that videos and slides of all talks from the first
Spark Summit last week, Dec 2-3 in Downtown SF, are now available on the
Spark Summit 2013 webpage at http://spark-summit.org/summit-2013. There is
a link for each talk's slides and video next to the talk's title in the
summit agenda. There are also links to the YouTube playlists for each
Summit session 
(Keynoteshttp://www.youtube.com/playlist?list=PL-x35fyliRwjXj33QvAXN0Vlx0gc6u0je,
Track 
Ahttp://www.youtube.com/playlist?list=PL-x35fyliRwiNcKwIkDEQZBejiqxEJ79U,
Track 
Bhttp://www.youtube.com/playlist?list=PL-x35fyliRwh6YpI80pu2t0JDVHXOtU7y,
and Training 
Dayhttp://www.youtube.com/playlist?list=PL-x35fyliRwjR1Umntxz52zv3EcKpbzCp
).

You can also now pre-register for Summit
2014http://spark-summit.org/2014/pre-registerto get notified when
tickets go on sale. Finally please let us know if you
have feedback from the Summit.

Enjoy the videos!

Andy


Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-13 Thread Patrick Wendell
The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
votes.

+1 (5 Total)
Matei Zaharia*
Nick Pentreath*
Patrick Wendell*
Prashant Sharma*
Tom Graves*

0 (0 Total)

-1 (0 Total)

* = Binding Vote

As per the incubator release guide [1] I'll be sending this to the
general incubator list for a final vote from IPMC members.

[1]
http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
vote


On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan e...@ooyala.com wrote:

 I'd be personally fine with a standard workflow of assemble-deps +
 packaging just the Spark files as separate packages, if it speeds up
 everyone's development time.


 On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra m...@clearstorydata.com
 wrote:

  I don't know how to make sense of the numbers, but here's what I've got
  from a very small sample size.
 
  For both v0.8.0-incubating and v0.8.1-incubating, building separate
  assemblies is faster than `./sbt/sbt assembly` and the times for building
  separate assemblies for 0.8.0 and 0.8.1 are about the same.
 
  For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
 the
  sum of the separate assemblies.
  For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
 the
  sum of the separate assemblies.
 
  Weird.
 
 
  On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell pwend...@gmail.com
  wrote:
 
   I'll +1 myself also.
  
   For anyone who has the slow build problem: does this issue happen when
   building v0.8.0-incubating also? Trying to figure out whether it's
   related to something we added in 0.8.1 or if it's a long standing
   issue.
  
   - Patrick
  
   On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia 
 matei.zaha...@gmail.com
  
   wrote:
Woah, weird, but definitely good to know.
   
If you’re doing Spark development, there’s also a more convenient
  option
   added by Shivaram in the master branch. You can do sbt assemble-deps to
   package *just* the dependencies of each project in a special assembly
  JAR,
   and then use sbt compile to update the code. This will use the classes
   directly out of the target/scala-2.9.3/classes directories. You have to
   redo assemble-deps only if your external dependencies change.
   
Matei
   
On Dec 11, 2013, at 1:04 AM, Prashant Sharma scrapco...@gmail.com
   wrote:
   
I hope this PR https://github.com/apache/incubator-spark/pull/252can
   help.
Again this is not a blocker for the release from my side either.
   
   
On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra 
  m...@clearstorydata.com
   wrote:
   
Interesting, and confirmed: On my machine where `./sbt/sbt
 assembly`
   takes
a long, long, long time to complete (a MBP, in my case),
 building
   three
separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
 less
   time.
   
   
   
On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma 
   scrapco...@gmail.com
wrote:
   
forgot to mention, after running sbt/sbt assembly/assembly running
sbt/sbt
examples/assembly takes just 37s. Not to mention my hardware is
 not
really
great.
   
   
On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma 
   scrapco...@gmail.com
wrote:
   
Hi Patrick and Matei,
   
Was trying out this and followed the quick start guide which says
  do
sbt/sbt assembly, like few others I was also stuck for few
 minutes
  on
linux. On the other hand if I use sbt/sbt assembly/assembly it is
   much
faster.
   
Should we change the documentation to reflect this. It will not
 be
great
for first time users to get stuck there.
   
   
On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia 
matei.zaha...@gmail.com
wrote:
   
+1
   
Built and tested it on Mac OS X.
   
Matei
   
   
On Dec 10, 2013, at 4:49 PM, Patrick Wendell 
 pwend...@gmail.com
wrote:
   
Please vote on releasing the following candidate as Apache
 Spark
(incubating) version 0.8.1.
   
The tag to be voted on is v0.8.1-incubating (commit b87d31d):
   
   
   
   
  
 
 https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
   
The release files, including signatures, digests, etc can be
  found
at:
http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
   
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
   
The staging repository for this release can be found at:
   
   
   https://repository.apache.org/content/repositories/orgapachespark-040/
   
The documentation corresponding to this release can be found
 at:
   
   http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
   
For information about the contents of this release see: