Hey Mark, One constructive action you and other people can take to help us assess the quality and completeness of this release is to download the release, run the tests, run the release in your dev environment, read through the documentation, etc. This is one of the main points of releasing an RC to the community... even if you disagree with some patches that were merged in, this is still a way you can help validate the release.
- Patrick On Sun, Dec 8, 2013 at 1:30 PM, Mark Hamstra <m...@clearstorydata.com> wrote: > I'm aware of the changes file, but it really doesn't address the issue that > I am raising. The changes file just tells me what has gone into the > release candidate. In general, it doesn't tell me why those changes went > in or provide any rationale by which to judge whether that is the complete > set of changes that should go in. > > I talked some with Matei about related versioning and release issues last > week, and I've raised them in other contexts previously, but I'm taking the > liberty to annoy people again because I really am not happy with our > current versioning and release process, and I really am of the opinion that > we've got to start doing much better before I can vote in favor of a 1.0 > release. I fully realize that this is not a 1.0 release, and that because > we are pre-1.0 we still have a lot of flexibility with releases that break > backward or forward compatibility and with version numbers that have > nothing like the semantic meaning that they will eventually need to have; > but it is not going to be easy to change our process and culture so that we > produce the kind of stability and reliability that Spark users need to be > able to depend upon and version numbers that clearly communicate what those > users expect them to mean. I think that we should start making those > changes now. Just because we have flexibility pre-1.0, that doesn't mean > that we shouldn't start training ourselves now to work within the > constraints of post-1.0 Spark. If I'm to be happy voting for an eventual > 1.0 release candidate, I'll need to have seen at least one full development > cycle that already adheres to the post-1.0 constraints, demonstrating the > maturity of our development process. > > That demonstration cycle is clearly not this one -- and I understand that > there were some compelling reasons (particularly with regard too getting a > "full" release of Spark based on Scala 2.9.3 before we make the jump to > 2.10. This "patch-level" release breaks binary compatibility and contains > a lot of code that isn't anywhere close to meeting the criterion for > inclusion in a real, post-1.0 patch-level release: essentially "changes > that every, or nearly every, existing Spark user needs (not just wants), > and that work with all existing and future binaries built with the prior > patch-level version of Spark as a dependency." Like I said, we are clearly > nowhere close to that with the move from 0.8.0 to 0.8.1; but I also haven't > been able to recognize any alternative criterion by which to judge the > quality and completeness of this release candidate. > > Maybe there just isn't one, and I'm just going to have to swallow my > concerns while watching 0.8.1 go out the door; but if we don't start doing > better on this kind of thing in the future, you are going to start hearing > more complaining from me. I just hope that it doesn't get to the point > where I feel compelled to actively oppose an eventual 1.0 release > candidate. > > > On Sun, Dec 8, 2013 at 12:37 PM, Henry Saputra <henry.sapu...@gmail.com>wrote: > >> Ah, sorry for the confusion Patrick, like you said I was just trying to let >> people aware about this file and the purpose of it. >> >> On Sunday, December 8, 2013, Patrick Wendell wrote: >> >> > Hey Henry, >> > >> > Are you suggesting we need to change something about or changes file? >> > Or are you just pointing people to the file? >> > >> > - Patrick >> > >> > On Sun, Dec 8, 2013 at 11:37 AM, Henry Saputra <henry.sapu...@gmail.com> >> > wrote: >> > > HI Spark devs, >> > > >> > > I have modified the Subject to avoid polluting the VOTE thread since >> > > it related to more info how and which commits merge back to 0.8.* >> > > branch. >> > > Please respond to the previous question to this thread. >> > > >> > > Technically the CHANGES.txt [1] file should describe the changes in a >> > > particular release and it is the main requirement needed to cut an ASF >> > > release. >> > > >> > > >> > > - Henry >> > > >> > > [1] >> > https://github.com/apache/incubator-spark/blob/branch-0.8/CHANGES.txt >> > > >> > > On Sun, Dec 8, 2013 at 12:03 AM, Josh Rosen <rosenvi...@gmail.com> >> > wrote: >> > >> We can use git log to figure out which changes haven't made it into >> > >> branch-0.8. Here's a quick attempt, which only lists pull requests >> that >> > >> were only merged into one of the branches. For completeness, this >> > could be >> > >> extended to find commits that weren't part of a merge and are only >> > present >> > >> in one branch. >> > >> >> > >> *Script:* >> > >> >> > >> MASTER_BRANCH=origin/master >> > >> RELEASE_BRANCH=origin/branch-0.8 >> > >> >> > >> git log --oneline --grep "Merge pull request" $MASTER_BRANCH | cut -f >> > 2- >> > >> -d ' ' | sort > master-prs >> > >> git log --oneline --grep "Merge pull request" $RELEASE_BRANCH | cut -f >> > 2- >> > >> -d ' ' | sort > release-prs >> > >> >> > >> comm -23 master-prs release-prs > master-only >> > >> comm -23 release-prs master-prs > release-only >> > >> >> > >> >> > >> *Master Branch Only:* >> > >> Merge pull request #1 from colorant/yarn-client-2.2 >> > >> Merge pull request #105 from pwendell/doc-fix >> > >> Merge pull request #110 from pwendell/master >> > >> Merge pull request #146 from JoshRosen/pyspark-custom-serializers >> > >> Merge pull request #151 from russellcardullo/add-graphite-sink >> > >> Merge pull request #154 from soulmachine/ClusterScheduler >> > >> Merge pull request #156 from haoyuan/master >> > >> Merge pull request #159 from liancheng/dagscheduler-actor-refine >> > >> Merge pull request #16 from pwendell/master >> > >> Merge pull request #185 from mkolod/random-number-generator >> > >> Merge pull request #187 from aarondav/example-bcast-test >> > >> Merge pull request #190 from markhamstra/Stages4Jobs >> > >> Merge pull request #198 from >> > ankurdave/zipPartitions-preservesPartitioning >> > >> Merge pull request #2 from colorant/yarn-client-2.2 >> > >> Merge pull request #203 from witgo/master >> > >> Merge pull request #204 from rxin/hash >> > >> Merge pull request #205 from kayousterhout/logging >> > >> Merge pull request #206 from ash211/patch-2 >> > >> Merge pull request #207 from henrydavidge/master >> > >> Merge pull request #209 from pwendell/better-docs >> > >> Merge pull request #210 from haitaoyao/http-timeout >> > >> Merge pull request #212 from markhamstra/SPARK-963 >> > >> Merge pull request #216 from liancheng/fix-spark-966 >> > >> Merge pull request #217 from aarondav/mesos-urls >> > >> Merge pull request #22 from GraceH/metrics-naming >> > >> Merge pull request #220 from rxin/zippart >> > >> Merge pull request #225 from ash211/patch-3 >> > >> Merge pull request #226 from ash211/patch-4 >> > >> Merge pull request #233 from hsaputra/changecontexttobackend >> > >> Merge pull request #239 from aarondav/nit >> > >> Merge pull request #242 from pwendell/master >> > >> Merge pull request #3 from aarondav/pv-test >> > >> Merge pull request #36 from pwendell/versions >> > >> Merge pull request #37 from pwendell/merge-0.8 >> > >> Merge pull request #39 from pwendell/master >> > >> Merge pull request #45 from pwendell/metrics_units >> > >> Merge pull request #56 from jerryshao/kafka-0.8-dev >> > >> Merge pull request #64 from prabeesh/master >> > >> Merge pull request #66 from shivaram/sbt-assembly-deps >> > >> Merge pull request #670 from jey/ec2-ssh-improvements >> > >> Merge pull request #71 from aarondav/scdefaults >> > >> Merge pull request #78 from mosharaf/master >> > >> Merge pull request #8 from vchekan/checkpoint-ttl-restore >> > >> Merge pull request #80 from rxin/build >> > >> Merge pull request #82 from JoshRosen/map-output-t >>
