Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-26 Thread Sean Owen
I still don't see any artifacts in maven -- did it publish? http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.spark%22 On Wed, May 25, 2016 at 10:30 AM, Reynold Xin wrote: > Yup I have published it to maven. Will post the link in a bit. > > One thing is that for

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-26 Thread Gurvinder Singh
On 05/26/2016 02:38 AM, Matei Zaharia wrote: > Just wondering, what is the main use case for the Docker images -- to > develop apps locally or to deploy a cluster? I use docker images for both development and deploying on production cluster. As it makes sure I have the correct version of Java and

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Marcin Tustin
The use case of docker images in general is that you can deploy and develop with exactly the same binary environment - same java 8, same scala, same spark. This makes things repeatable. On Wed, May 25, 2016 at 8:38 PM, Matei Zaharia wrote: > Just wondering, what is the

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Matei Zaharia
Just wondering, what is the main use case for the Docker images -- to develop apps locally or to deploy a cluster? If the image is really just a script to download a certain package name from a mirror, it may be okay to create an official one, though it does seem tricky to make it properly use

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Luciano Resende
On Wed, May 25, 2016 at 2:34 PM, Sean Owen wrote: > I don't think the project would bless anything but the standard > release artifacts since only those are voted on. People are free to > maintain whatever they like and even share it, as long as it's clear > it's not from the

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Sean Owen
I don't think the project would bless anything but the standard release artifacts since only those are voted on. People are free to maintain whatever they like and even share it, as long as it's clear it's not from the Apache project. On Wed, May 25, 2016 at 3:41 PM, Marcin Tustin

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Marcin Tustin
Ah very nice. Would it be possible to have this blessed into an official image? On Wed, May 25, 2016 at 4:12 PM, Luciano Resende wrote: > > > On Wed, May 25, 2016 at 6:53 AM, Marcin Tustin > wrote: > >> Would it be useful to start baking docker

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Luciano Resende
On Wed, May 25, 2016 at 6:53 AM, Marcin Tustin wrote: > Would it be useful to start baking docker images? Would anyone find that a > boon to their testing? > > +1, I had done one (still based on 1.6) for some SystemML experiments, I could easily get it based on a nightly

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Daniel Darabos
Awesome, thanks! It's very helpful for preparing for the migration. Do you plan to push 2.0.0-preview to Maven too? (I for one would appreciate the convenience.) On Wed, May 25, 2016 at 8:44 AM, Reynold Xin wrote: > In the past the Spark community have created preview

[ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Reynold Xin
In the past the Spark community have created preview packages (not official releases) and used those as opportunities to ask community members to test the upcoming versions of Apache Spark. Several people in the Apache community have suggested we conduct votes for these preview packages and turn