Hey Reynold, Thanks. Getting to the heart of this, I think that this project would be successful if the Apache Spark PMC decided to participate and there was some overlap. As much as I think it would be great to stand up another project, the goal here from Luciano and crew (myself included) would be to suggest it’s just as easy to start an Apache Incubator project to manage “extra” pieces of Apache Spark code outside of the release cycle and the other reasons stated that it made sense to move this code out of the code base. This isn’t a competing effort to some code on GitHub that was moved out of Apache source control from Apache Spark - it’s meant to be an enabler to suggest that code could be managed here just as easily (see the difference?)
Let me know what you think thanks Reynold. Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Director, Information Retrieval and Data Science Group (IRDS) Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA WWW: http://irds.usc.edu/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ On 4/15/16, 9:47 AM, "Reynold Xin" <r...@databricks.com> wrote: > > > >Anybody is free and welcomed to create another ASF project, but I don't think >"Spark extras" is a good name. It unnecessarily creates another tier of code >that ASF is "endorsing". >On Friday, April 15, 2016, Mattmann, Chris A (3980) ><chris.a.mattm...@jpl.nasa.gov> wrote: > >Yeah in support of this statement I think that my primary interest in >this Spark Extras and the good work by Luciano here is that anytime we >take bits out of a code base and “move it to GitHub” I see a bad precedent >being set. > >Creating this project at the ASF creates a synergy between *Apache Spark* >which is *at the ASF*. > >We welcome comments and as Luciano said, this is meant to invite and be >open to those in the Apache Spark PMC to join and help. > >Cheers, >Chris > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Chief Architect >Instrument Software and Science Data Systems Section (398) >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 168-519, Mailstop: 168-527 >Email: >chris.a.mattm...@nasa.gov <javascript:;> >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Director, Information Retrieval and Data Science Group (IRDS) >Adjunct Associate Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >WWW: http://irds.usc.edu/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > > > > >On 4/15/16, 9:39 AM, "Luciano Resende" <luckbr1...@gmail.com <javascript:;>> >wrote: > >> >> >>On Fri, Apr 15, 2016 at 9:34 AM, Cody Koeninger >><c...@koeninger.org <javascript:;>> wrote: >> >>Given that not all of the connectors were removed, I think this >>creates a weird / confusing three tier system >> >>1. connectors in the official project's spark/extras or spark/external >>2. connectors in "Spark Extras" >>3. connectors in some random organization's github >> >> >> >> >> >> >> >>Agree Cody, and I think this is one of the goals of "Spark Extras", >>centralize the development of these connectors under one central place at >>Apache, and that's why one of our asks is to invite the Spark PMC to continue >>developing the remaining connectors >> that stayed in Spark proper, in "Spark Extras". We will also discuss some >> process policies on enabling lowering the bar to allow proposal of these >> other github extensions to be part of "Spark Extras" while also considering >> a way to move code to a maintenance >> mode location. >> >> >> >> >>-- >>Luciano Resende >>http://twitter.com/lresende1975 >>http://lresende.blogspot.com/ >> >> >> >> > > >
