A very nice addition for us PySpark users in 0.9.1 is the addition of RDD.repartition(), which is not mentioned in the release notes<http://spark.apache.org/releases/spark-release-0-9-1.html> !
This is super helpful for when you create an RDD from a gzipped file and then need to explicitly shuffle the data around to parallelize operations on it appropriately. Thanks people! FYI, docs/latest<http://spark.apache.org/docs/latest/api/pyspark/index.html>hasn't been updated yet to reflect the new additions to PySpark. Nick On Wed, Apr 9, 2014 at 6:07 PM, Matei Zaharia <matei.zaha...@gmail.com>wrote: > Thanks TD for managing this release, and thanks to everyone who > contributed! > > Matei > > On Apr 9, 2014, at 2:59 PM, Tathagata Das <tathagata.das1...@gmail.com> > wrote: > > A small additional note: Please use the direct download links in the Spark > Downloads <http://spark.apache.org/downloads.html> page. The Apache > mirrors take a day or so to sync from the main repo, so may not work > immediately. > > TD > > > On Wed, Apr 9, 2014 at 2:54 PM, Tathagata Das <tathagata.das1...@gmail.com > > wrote: > >> Hi everyone, >> >> We have just posted Spark 0.9.1, which is a maintenance release with >> bug fixes, performance improvements, better stability with YARN and >> improved parity of the Scala and Python API. We recommend all 0.9.0 >> users to upgrade to this stable release. >> >> This is the first release since Spark graduated as a top level Apache >> project. Contributions to this release came from 37 developers. >> >> The full release notes are at: >> http://spark.apache.org/releases/spark-release-0-9-1.html >> >> You can download the release at: >> http://spark.apache.org/downloads.html >> >> Thanks all the developers who contributed to this release: >> Aaron Davidson, Aaron Kimball, Andrew Ash, Andrew Or, Andrew Tulloch, >> Bijay Bisht, Bouke van der Bijl, Bryn Keller, Chen Chao, >> Christian Lundgren, Diana Carroll, Emtiaz Ahmed, Frank Dai, >> Henry Saputra, jianghan, Josh Rosen, Jyotiska NK, Kay Ousterhout, >> Kousuke Saruta, Mark Grover, Matei Zaharia, Nan Zhu, Nick Lanham, >> Patrick Wendell, Prabin Banka, Prashant Sharma, Qiuzhuang, >> Raymond Liu, Reynold Xin, Sandy Ryza, Sean Owen, Shixiong Zhu, >> shiyun.wxm, Stevo Slavić, Tathagata Das, Tom Graves, Xiangrui Meng >> >> TD >> > > >
