Thanks for bringing this up Sean. I think we are all happy to adopt concrete suggestions to make the release process more transparent, including pinging the list before kicking off the release build.
Technically there's still a Blocker bug: > https://issues.apache.org/jira/browse/SPARK-12000 Sorry, I misprioritized this particular issue when I thought that it was going to block the release by causing the doc build to fail. When I realized the failure was non-deterministic and isolated to OSX (i.e. the official release build on jenkins is not affected) I failed to update the issue. It doesn't show up on the dashboard that I've been using to track the release <https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20AND%20component%20not%20in%20(Tests%2C%20Documentation)%20AND%20%22Target%20Version%2Fs%22%20%3D%201.6.0%20AND%20(fixVersion%20is%20EMPTY%20OR%20fixVersion%20!%3D%201.6.0)%20AND%20(status%20%3D%20Open%20OR%20status%20%3D%20%22In%20Progress%22%20OR%20status%20%3D%20Reopened)%20ORDER%20BY%20priority%20DESC> since its labeled documentation. > The 'race' doesn't matter that much, but release planning remains the > real bug-bear here. There are still, for instance, 52 issues targeted > at 1.6.0, 42 of which were raised and targeted by committers. For > example, I count 6 'umbrella' JIRAs in ML alone that are still open. > This can be debated, but I explicitly ignored test and documentation issues. Since the docs are published separately and easy to update, I don't think its worth further disturbing the release cadence for these JIRAs. > The release is theoretically several weeks behind plan on what's > intended to be a fixed release cycle too. This is why I'm not sure why > today it's suddenly potentially ready for release. > Up until today various committers have told me that there were known issues with branch-1.6 that would cause them to -1 the release. Whenever this happened, I asked them to ensure there was a properly targeted blocker JIRA open so people could publicly track the status of the release. As long as such issues were open, I only published a preview since making an RC is pretty high cost. I'm sorry that it felt sudden to you, but as of last night all such known issues were resolved and thus I cut a release as soon as this was the case. I'm just curious, am I the only one that thinks this isn't roughly > normal, or do other people manage releases this way? I know the real > world is messy and this is better than in the past, but I still get > surprised by how each 1.x release actually comes about. > I actually did spent quite a bit of time asking people to close various umbrella issues, and I was pretty strict about watching JIRA throughout the process. Perhaps as an additional step, future preview releases or branch cuts can include a link to an authoritative dashboard that we will use to decide when we are ready to make an RC. I'm also open to other suggestions. Michael
