If 1.0 is just round the corner, then it is fair enough to push to that, thanks for clarifying !
Regards, Mridul On Wed, Mar 19, 2014 at 6:12 PM, Tathagata Das <tathagata.das1...@gmail.com> wrote: > I agree that the garbage collection > PR<https://github.com/apache/spark/pull/126>would make things very > convenient in a lot of usecases. However, there are > two broads reasons why it is hard for that PR to get into 0.9.1. > 1. The PR still needs some amount of work and quite a lot of testing. While > we enable RDD and shuffle cleanup based on Java GC, its behavior in a real > workloads still needs to be understood (especially since it is tied to > Spark driver's garbage collection behavior). > 2. This actually changes some of the semantic behavior of Spark and should > not be included in a bug-fix release. The PR will definitely be present for > Spark 1.0, which is expected to be release around end of April (not too far > ;) ). > > TD > > > On Wed, Mar 19, 2014 at 5:57 PM, Mridul Muralidharan <mri...@gmail.com>wrote: > >> Would be great if the garbage collection PR is also committed - if not >> the whole thing, atleast the part to unpersist broadcast variables >> explicitly would be great. >> Currently we are running with a custom impl which does something >> similar, and I would like to move to standard distribution for that. >> >> >> Thanks, >> Mridul >> >> >> On Wed, Mar 19, 2014 at 5:07 PM, Tathagata Das >> <tathagata.das1...@gmail.com> wrote: >> > Hello everyone, >> > >> > Since the release of Spark 0.9, we have received a number of important >> bug >> > fixes and we would like to make a bug-fix release of Spark 0.9.1. We are >> > going to cut a release candidate soon and we would love it if people test >> > it out. We have backported several bug fixes into the 0.9 and updated >> JIRA >> > accordingly< >> https://spark-project.atlassian.net/browse/SPARK-1275?jql=project%20in%20(SPARK%2C%20BLINKDB%2C%20MLI%2C%20MLLIB%2C%20SHARK%2C%20STREAMING%2C%20GRAPH%2C%20TACHYON)%20AND%20fixVersion%20%3D%200.9.1%20AND%20status%20in%20(Resolved%2C%20Closed) >> >. >> > Please let me know if there are fixes that were not backported but you >> > would like to see them in 0.9.1. >> > >> > Thanks! >> > >> > TD >>
