Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-12-16 Thread Jarek Potiuk
Hello, I finally had some time after Airlfow 2.0 release and I opened discussion about the policy in legal-disc...@apache.org: https://lists.apache.org/thread.html/r7c9ceb3d6c764119b14dfedb0e22957993d93cf529792c402aaa05fc%40%3Clegal-discuss.apache.org%3E I propose we continue the discussion

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-18 Thread Jarek Potiuk
Thanks Kaxil! I agree adding scripts should not be compulsory - I already changed it to "best practice" wherever I mentioned it. If you think even this is too much - I am happy to discuss :). I think that really the definition of the packaging that would make it possible to legally comply with

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-18 Thread Kaxil Naik
I have added comments on the proposed document. I don't think we should make it compulsory to include scripts to rebuild binaries used in the projects. It is a maintenance nightmare. I agree though that we need a proper definition for the "convenience packages" and some guidelines around Docker

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-18 Thread Jarek Potiuk
@Matt Sicker - absolutely, thanks for mentioning that. My intention is to hash-out as many details as possible and get as much input as possible from relevant people to make a strong case to the board. While I am not an ASF member myself, just a PMC, I watched some of the Apache Con talks and I

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-17 Thread Justin Mclean
Hi, > I am not even sure myself if I am the only one who feels the disconnection > between reality and the current policies, that's why I started the thread > here. You would not be the only person who set this. People and project are making small steps to correct this. Recently Infra’s

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-17 Thread Matt Sicker
Keep in mind that legal doesn’t set policy. That’s the board. Legal helps inform the board (and whoever else) about the risks of different legal choices. Small nitpick, but keep in mind that legal isn’t here to block us. They’re here to help us understand the risks of which policy we choose to

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-17 Thread Jarek Potiuk
Good. So let me try to contact the legal first and try to see what they say. I indeed had comments that it will be difficult to get through the approval process, but I am eager to try this and happy to lead the effort. I think if I get at least a supportive voice from several projects who also

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Matthias Seidel
Really, I don't even know what they are talking about... Has anyone been informed about what they are trying to achieve? What "significant push back from Legal and the Board" should we expect? Regards,    Matthias Am 16.10.20 um 20:59 schrieb Joan Touzet: > Hi Jarek, > > On 16/10/2020 05:45,

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Joan Touzet
Hi Jarek, On 16/10/2020 05:45, Jarek Potiuk wrote: > Joan - I hope you are back and we can continue the discussion. Sorry, I just don't have the time or the energy to carry this further. I only brought up OO because it was one of the outliers, and the first "user application" that came to

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Dave Fisher
Let’s keep this simple. Focus on your needs for clarification for Helm Charts. I really don’t have the time or inclination to parse out what troubles there are for OpenOffice in your plan. I already have a long list with unexpected issues and mandates. The Foundation exists for its Project

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Jarek Potiuk
I do not want to make any assumptions on Windows/MacOS etc - for me this is really just proposal to clarify the ASF policies which (as far as at least I understand) are not up-to-date with some distribution mechanisms for convenience packages that have become popular way that our users can

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Dave Fisher
Questions related to OpenOffice must be discussed with the OpenOffice PMC. Joan is an observer and I don’t recall her asking the PMC. Please don’t make any assumptions about builds for Windows, macOS (we support 10.7 and newer) and Linux. Community members on their own build OS/2 and FreeBSD

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-10-16 Thread Jarek Potiuk
Hello everyone, As Apache Airflow 2.0 alpha is out, I have some time to come back to the discussion. I also add Joan who was discussing the policy in a separate thread in the builds@ devlist:

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-15 Thread Jarek Potiuk
Cool. Thanks Bertrand - I am aware of some of those, but this list will be helpful so I can make some of the references to those in my proposal (and I encourage everyone else to do so). I am super-happy to merge yours with mine. I believe the "spirit" of those is rather similar. I am fully aware

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-15 Thread Bertrand Delacretaz
Hi, On Mon, Sep 7, 2020 at 10:24 AM Bertrand Delacretaz wrote: ... > ...* https://issues.apache.org/jira/browse/ASFP-23 (accessible to ASF > Members) has links to related discussions... For the benefit of people who don't have access to that ticket, here's a list of links that are public and

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-13 Thread Jarek Potiuk
Hello Everyone. As promised, I prepared a draft of the proposal of changes that I would love if a number of interested people discuss it, comment, criticise, agree on eventually, and submit to the ASF Board for Approval. I tried to capture all the context, but also I marked clearly all the

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-09 Thread Jarek Potiuk
Issue in COMDEV created: https://issues.apache.org/jira/browse/COMDEV-382 I will prepare an early draft of the proposal shortly and ping here to spark a discussion J. On Tue, Sep 8, 2020 at 5:59 PM Matt Sicker wrote: > I've spent an inordinate amount of time at $dayjob triaging security >

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-08 Thread Matt Sicker
I've spent an inordinate amount of time at $dayjob triaging security vulnerabilities from Docker scans, so I can definitely attest to Mark's experience there. In fact, one of the biggest offenders was the official Docker Hub image for openjdk! Then there were a few years where people pushed Alpine

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-08 Thread Jarek Potiuk
Will definitely include that in my proposal Mark! BTW. Speaking of the report you got, we got the user talking to us on slack, and got the user to retest it on the refreshed image. It all boiled down to 4 "undefined" risk issues reported by the tool (seems that their - reasonable - approach is

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-08 Thread Mark J Cox
On Mon, Sep 7, 2020 at 2:21 PM Jarek Potiuk wrote: > I also talked to the Apache Security team today (there was an issue raised > about the security of the images which I think should be part of the policy > as well. > Thanks Jarek. What happened is that we got a report to secur...@apache.org

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-07 Thread Jarek Potiuk
Fantastic, and very actionable! Thanks Bertrand! I am happy to make a draft of a proposal (I will make sure to mark the parts that I know are controversial as such) and start the discussion around this. I also talked to the Apache Security team today (there was an issue raised about the security

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-07 Thread Bertrand Delacretaz
On Sun, Sep 6, 2020 at 10:37 AM Jarek Potiuk wrote: ... > How can I help to make it happen?.. It looks like what you need might require some clarifications of ASF policies and probably infrastructure support as well. IMO, the best way to make these things happen is to create concrete proposals

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-06 Thread Kaxil Naik
"The bits we build on top of external components becomes the Apache project that is ALv2 and is distributed through source archives with convenience binaries (in this case, the image layers and YAML files)." I like this statement from Matt and completely agree with it. On Sun, Sep 6, 2020, 09:40

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-06 Thread Kaxil Naik
I will be happy to help too, definitely and important topic where we need a clear guidance on it (what sources and binaries are allowed etc) Regards, Kaxil Airflow PMC On Sun, Sep 6, 2020, 09:37 Jarek Potiuk wrote: > Hey Dave, > > Thanks for this. Having the ComDev/Infra repository where PMC

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-06 Thread Jarek Potiuk
Hey Dave, Thanks for this. Having the ComDev/Infra repository where PMC can publish approved "Chart" and clear docs on what it means to follow the Release Policy is exactly what we need I think. We have a heated debate within Airflow PMC and despite exchanging many messages and (sometimes

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Kaxil Naik
Great, thanks Dave On Sun, Sep 6, 2020, 00:17 Dave Fisher wrote: > Every release must be made in SVN on dist.apache.org. All other channels > are optional and may or may not have Infra support. > > I view this as a request to have ComDev or Infra manage a Gitbox/Github > repository where any

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Dave Fisher
Every release must be made in SVN on dist.apache.org. All other channels are optional and may or may not have Infra support. I view this as a request to have ComDev or Infra manage a Gitbox/Github repository where any PMC can publish a PMC approved Helm Chart release from their project. How

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Kaxil Naik
>The point is that we MUST give the users possibility to (re)build those images on their own if they want. >From http://www.apache.org/legal/release-policy.html#source-packages, it does not say it has to be on PyPI. I will reiterate what I said in the Github issue, having a binary on PyPI is no

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Jarek Potiuk
@Matt Sicker -> agree on both, central chart repo, and per-project sources for charts. Though packaging is a bit different than Docker though. The packages contain tar.gz sources of the chart (which usually includes quite complex go templating logic, not just simple yaml files) and those

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Kaxil Naik
Does voting needs to happen for "releasing" the Helm chart ? Do we any guidelines from the ASF around releasing Helm Chart & Dockerfile? On Sat, Sep 5, 2020, 17:36 Matt Sicker wrote: > If packaging is similar to Docker, then our Helm charts will still > need some third party base images and

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Matt Sicker
If packaging is similar to Docker, then our Helm charts will still need some third party base images and such for GPL licensed system dependencies (besides the kernel itself, there's OpenJDK which is definitely used in plenty of projects here). The bits we build on top of external components

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-05 Thread Daniel Imberman
While I understand the necessity for users to have access to all source code, I'm a bit concerned about the complexity this places on the open-source project. Let's take Airflow as an example. If we want to use pgbouncer in our home chart are we expected to be able to build this external

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-04 Thread Scott Rigby
Jarek, this is fantastic! Please let me know if I/we can help with this. The Helm team has written tools to make some of these processes easier, including steps to preserve former git history, and GitHub Actions for CI (chart testing) and CD (releasing chart version packages via GitHub release

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-04 Thread Gaetan Semet
Hi, I am one of the maintainer of the Airflow Chart (stable/airflow at https://github.com/helm/charts/tree/master/stable/airflow). Our idea (described in this discussion https://github.com/apache/airflow/issues/10523) is to copy or host the current Airflow chart as it is, like a "community

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-04 Thread Jarek Potiuk
Big +1 for doing this within existing ASF infrastructure. Maybe indeed current SVN publishing a release snapshot + mirroring is the best way. IMHO such 'released' charts do not have to have commit history at all (except 'release' commits) - just release 'snapshots' is enough. For all the other

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-04 Thread Bertrand Delacretaz
Hi, On Fri, Sep 4, 2020 at 9:22 AM Jarek Potiuk wrote: > ...I think having ASF "apache/chart" for all "officially released" helm > charts is the best solution Sounds like a good idea but please use a more specific name, "charts" is very generic. If distributing those Helm charts does not

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-04 Thread Jarek Potiuk
I've been thinking a lot and discussing the issues mentioned here (as in Apache Airflow we had this very case to solve. I would like to extend what Kamil wrote - and comment on what Apache Airflow uses and some of the recent discussions we had. I think we had very recently a lot of discussions

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Matt Sicker
If Helm charts have similar licensing complexities as Docker images, then it probably makes sense to migrate charts to individual projects rather than making a larger chart repository here at Apache. If they don’t have similar licensing problems, then it could be aggregated like the Docker org,

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Dave Fisher
The ASF and the Apache Way is all about the community around an open source software project. If there is no community around this deprecated package then you will need to contact each Apache community that might be interested and ask them directly. If you have a group of developers who want

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Ween Jiann
Hi Kamil, What's your take of having the helm chart in a single repo? It will lessen the load to maintain tests and distribution. On 2020/09/03 19:54:17, Kamil Breguła wrote: > Hello, > > In the case of Airflow, we have developed our own chart and want to develop > and release it soon. For

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Ween Jiann
Hi Dave, Just to clarify, my suggestion is to move only charts related to ASF projects to the apache git repo. These charts have been deprecated as part of the repo deprecation, they have no choice but to move somewhere or be archived. If this gains some traction, I'll create an issue and

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Dave Fisher
Hi - Does the Helm Chart community wish to explore becoming an Apache Project Community? Regards, Dave Sent from my iPhone > On Sep 3, 2020, at 12:33 PM, Ween Jiann wrote: > > Hi all, > > The helm team has deprecated the charts repository and will be archiving it > in Nov. Here’s the

Re: Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Kamil Breguła
Hello, In the case of Airflow, we have developed our own chart and want to develop and release it soon. For now, there is little chance that our community will be interested in developing two independent charts. https://github.com/apache/airflow/issues/10523 That would be the 3rd method of

Migration and consolidation helm charts for ASF projects from helm/charts to apache/charts git

2020-09-03 Thread Ween Jiann
Hi all, The helm team has deprecated the charts repository and will be archiving it in Nov. Here’s the link to the notice https://github.com/helm/charts/blob/master/README.md#deprecation-timeline It looks like that are quite a few charts for ASF projects such as Airflow, Hadoop, Spark, Solr