Re: [DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-27 Thread Jim Kleckner
Sure, it seems like an optional thing to me. Spark has a Jenkins setup for building and testing. This would only affect someone that pushes the code to gitlab. I'm happy to keep the commit in a small private branch of my own that I apply when I need to build an out of cycle build. I just

Re: [DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-26 Thread Erik Erlandson
Can a '.gitlab-ci.yml' be considered code, in the same way that the k8s related dockerfiles are code? In other words, something like: "here is a piece of code you might choose to use for building your own binaries, that is not specifically endorsed by Apache Spark"? So it would not be involved in

Re: [DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-23 Thread Jim Kleckner
I understand that "non-dev" persons could become confused and that some sort of signposting/warning makes sense. Certainly I consider my personal registry on gitlab.com as ephemeral and not intended to publish. We have our own private instance of gitlab where I put artifacts that are derived and

Re: [DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-23 Thread Sean Owen
Yeah the color on this is that 'snapshot' or 'nightly' builds are not quite _discouraged_ by the ASF, but need to be something only devs are likely to find and clearly signposted, because they aren't official blessed releases. It gets into a gray area if the project is 'officially' hosting a way

Re: [DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-23 Thread Dongjoon Hyun
Hi, Jim. Thank you for the proposal. I understand the request. However, the following key benefit sounds like unofficial snapshot binary releases. > For example, this was used to build a version of spark that included SPARK-28938 which has yet to be released and was necessary for spark-operator

[DISCUSS][SPARK-30275] Discussion about whether to add a gitlab-ci.yml file

2020-01-23 Thread Jim Kleckner
This story [1] proposes adding a .gitlab-ci.yml file to make it easy to create artifacts and images for spark. Using this mechanism, people can submit any subsequent version of spark for building and image hosting with gitlab.com. There is a companion WIP branch [2] with a candidate and example