Hi Sean, (writing this email with my Apache hat on only and not Databricks hat)
The preview release is available here: http://spark.apache.org/downloads.html (there is an entire section dedicated to it and also there is a news link to it on the right). Again, I think this is a good opportunity to define what a release should contain. Based on http://www.apache.org/dev/release.html#where-do-releases-go "In addition to the distribution directory, project that use Maven or a related build tool sometimes place their releases on repository.apache.org beside some convenience binaries. The distribution directory is required, while the repository system is an optional convenience." So I'm reading it as that maven publication is not necessary. My understanding is that the general community (beyond who follows the dev list) should understand that preview is not a stable release, and we as the PMC should set expectations accordingly. Developers that can test the preview releases tend to be more savvy and are comfortable on the bleeding edge. It is actually fairly easy for them to add a maven repo. Now reading the page I realized no where on the page did we mention the temporary maven repo. I will fix that. I think it'd be pretty bad if preview releases in anyway become "default version", because they are unstable and contain a lot of blocker bugs. So my concrete proposal is: 1. Separate (officially voted) releases into normal and preview. 2. On the download page, have two sections. One listing the normal releases, and the other listing preview releases. 3. Everywhere we mention preview releases, include the proper disclaimer e.g. "This preview is not a stable release in terms of either API or functionality, but it is meant to give the community early access to try the code that will become Spark 2.0." 4. Publish normal releases to maven central, and preview releases only to the staging maven repo. But of course we should include the temporary maven repo for preview releases on the download page. On Wed, Jun 1, 2016 at 3:10 PM, Sean Owen <so...@cloudera.com> wrote: > I'll be more specific about the issue that I think trumps all this, > which I realize maybe not everyone was aware of. > > There was a long and contentious discussion on the PMC about, among > other things, advertising a "Spark 2.0 preview" from Databricks, such > as at > https://databricks.com/blog/2016/05/11/apache-spark-2-0-technical-preview-easier-faster-and-smarter.html > > That post has already been updated/fixed from an earlier version, but > part of the resolution was to make a full "2.0.0 preview" release in > order to continue to be able to advertise it as such. Without it, I > believe the PMC's conclusion remains that this blog post / product > announcement is not allowed by ASF policy. Hence, either the product > announcements need to be taken down and a bunch of wording changed in > the Databricks product, or, this needs to be a normal release. > > Obviously, it seems far easier to just finish the release per usual. I > actually didn't realize this had not been offered for download at > http://spark.apache.org/downloads.html either. It needs to be > accessible there too. > > > We can get back in the weeds about what a "preview" release means, > but, normal voted releases can and even should be alpha/beta > (http://www.apache.org/dev/release.html) The culture is, in theory, to > release early and often. I don't buy an argument that it's too old, at > 2 weeks, when the alternative is having nothing at all to test > against. > > On Wed, Jun 1, 2016 at 5:02 PM, Michael Armbrust <mich...@databricks.com> > wrote: > >> I'd think we want less effort, not more, to let people test it? for > >> example, right now I can't easily try my product build against > >> 2.0.0-preview. > > > > > > I don't feel super strongly one way or the other, so if we need to > publish > > it permanently we can. > > > > However, either way you can still test against this release. You just > need > > to add a resolver as well (which is how I have always tested packages > > against RCs). One concern with making it permeant is this preview > release > > is already fairly far behind branch-2.0, so many of the issues that > people > > might report have already been fixed and that might continue even after > the > > release is made. I'd rather be able to force upgrades eventually when we > > vote on the final 2.0 release. > > >
