>The point is that we MUST give the users possibility to (re)build those images on their own if they want.
>From http://www.apache.org/legal/release-policy.html#source-packages, it does not say it has to be on PyPI. I will reiterate what I said in the Github issue, having a binary on PyPI is no guarantee that it will stay there. Having a binary on Github releases https://github.com/jbub/pgbouncer_exporter/tags also does not guarantee it would not change. If the license are correct and the source is available I definitely do not see this as a problem: Every ASF release MUST contain one or more source packages, which MUST be > sufficient for a user to build and test the release provided they have > access to the appropriate platform and tools. On Sat, Sep 5, 2020 at 10:33 PM Jarek Potiuk <ja...@potiuk.com> wrote: > @Matt Sicker <boa...@gmail.com> -> agree on both, central chart repo, and > per-project sources for charts. Though packaging is a bit different than > Docker though. The packages contain tar.gz sources of the chart (which > usually includes quite complex go templating logic, not just simple yaml > files) and those templates include names of the images used. This makes > the Helm chart much more close to the -src.tar.gz files we release now than > to Docker Images we publish. So IMHO the .tgz part should follow the normal > release process with voting @Kaxilnaik <kaxiln...@apache.org>. > Unless those are the very same sources of product that are already formally > released. But then @Gaetan Semet previously pointed out as bad practice > to bind the two together. > IMHO that means that we have to vote on each Chart release. But I'd love > what others think - knowing this packaging details. > > @ermanndimb- my point for images pointed to by the helm chart is not about > copying full sources (I think from this discussion I realised where the > main confusion was) . This is not needed. I just made a PR to show how this > can be done with the pgbouncer-exporter. This is quite a good example to > base our discussion on: https://github.com/apache/airflow/pull/10759 > > The point is that we MUST give the users possibility to (re)build those > images on their own if they want. They should be able to do that using: > > a) official base images (DockerHub has the "official image program" for > platforms. openjdk, python,debian, alpine - those are all official images > in DockerHub terms > b) having access to properly licensed source code with some guarantees > that it will not disappear ("official" repo, multiple forks in Github, > heavily used by others, fork under "apache" organisation) . > c) instructions where to find the sources and build the images > (Dockerfile + necessary scripts to build the image if in case it is not > straightforward). Similarly as we provide instructions on building the > "product". > > If the first two are available but it's not obvious how the image was built > - IMHO we need to provide the user instructions on how to build those > images (ideally a script that does it automatically). > > This is all fulfilled by the PR above as an example: > a) based on official, alpine image > b) the pgbouncer code is properly licenced and we forked the sources > several times to be sure it won't disappear > c) we have a script that builds and publishes the image (image is published > under apache/airflow DockerHub) > > IMHO - that's a very good example to show the kind of approach we can take, > and shows that it isn't difficult to handle it well at all. > > The alternative that I am afraid of is - that we require our users to use > binaries which are not "official" but maintained and controlled by > 3rd-party (without a straightforward way of building the binary from > official images + properly licensed sources + instructions). If we do not > give those to the user but we simply point to a 3rd-party binary bimage - > we both endorse the 3rd party, and introduce dependency on the 3rd-party. > The user then has to rely on the 3rd-party to use the Airflow Helm Chart - > which I think is a very bad idea. > > J. > > > On Sat, Sep 5, 2020 at 7:42 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > > > Does voting needs to happen for "releasing" the Helm chart ? Do we any > > guidelines from the ASF around releasing Helm Chart & Dockerfile? > > > > On Sat, Sep 5, 2020, 17:36 Matt Sicker <boa...@gmail.com> wrote: > > > > > If packaging is similar to Docker, then our Helm charts will still > > > need some third party base images and such for GPL licensed system > > > dependencies (besides the kernel itself, there's OpenJDK which is > > > definitely used in plenty of projects here). The bits we build on top > > > of external components becomes the Apache project that is ALv2 and is > > > distributed through source archives with convenience binaries (in this > > > case, the image layers and YAML files). > > > > > > I do like the idea of having a central Apache Helm chart repo where > > > releases can be published. That makes it simpler for users to combine > > > multiple Apache charts together, too, based on my past experience with > > > using Helm (especially with the new decentralized charts distribution > > > model they've switched to). I do think it makes sense to try and > > > maintain the sources and such for those charts in the individual > > > projects since each PMC may have different workflows on how that's > > > created, and it makes it easier to release alongside the normal > > > project's releases. > > > > > > On Sat, 5 Sep 2020 at 11:13, Daniel Imberman <dimber...@apache.org> > > wrote: > > > > > > > > While I understand the necessity for users to have access to all > source > > > code, I'm a bit concerned about the complexity this places on the > > > open-source project. Let's take Airflow as an example. If we want to > use > > > pgbouncer in our home chart are we expected to be able to build this > > > external project? > > > > > > > > I think that if this is ASF policy, then maybe we need to modify the > > > policy to meet the current ecosystem. Helm charts are made to handle > > > complex deployments, and expecting a project to own every other piece > of > > > their ecosystem is a pretty heavy burden. Is there some compromise we > can > > > come to? Can we store certain pieces such as source code, Dockerfile, > and > > > docker binary in some Apache cold storage to ensure they are > > reproduceable? > > > > > > > > Ideally I'd just like to keep the burden on project maintainers to a > > > minimum. This could heavily discourage projects from creating official > > helm > > > charts. > > > > > > > > > > > > On 2020/09/04 15:05:56, Scott Rigby <sc...@r6by.com> wrote: > > > > > Jarek, this is fantastic! Please let me know if I/we can help with > > > this. The Helm team has written tools to make some of these processes > > > easier, including steps to preserve former git history, and GitHub > > Actions > > > for CI (chart testing) and CD (releasing chart version packages via > > GitHub > > > release artifacts). > > > > > > > > > > Bertrand, many orgs are adopting a git repo naming convention > > > "helm-charts" to make this explicit. > > > > > > > > > > Fun follow-up question: a helm repo is really just an index (YAML) > > > file of metadata in a format the helm client expects, including links > to > > > packaged tarballs for each immutable version of a chart. The idea of > > Apache > > > mirror hosting is on the table, one issue we have is that the GCP > object > > > storage buckets that currently host all previous versions of stable and > > > incubator repo charts will be deleted after support for these repos > ends > > on > > > Nov 13th ( > > > > > > https://github.com/helm/charts/blob/master/README.md#deprecation-timeline > > ). > > > Would it be possible for Apache to host a mirror of all these packages > > > (note, they are all Apache licenced), or only chart packages related to > > > Apache software (airflow, solr, spark, etc)? > > > > > > > > > > Scott > > > > > > > > > > On 2020/09/04 08:47:30, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > Big +1 for doing this within existing ASF infrastructure. Maybe > > > indeed > > > > > > current SVN publishing a release snapshot + mirroring is the > best > > > way. > > > > > > > > > > > > IMHO such 'released' charts do not have to have commit history at > > all > > > > > > (except 'release' commits) - just release 'snapshots' is enough. > > For > > > all > > > > > > the other development uses, whatever the source repository is, > you > > > will be > > > > > > able too install 'development' chart from the sources. > > > > > > > > > > > > Still having clear guidelines on how to approach Docker images > and > > > > > > 'rebuildability' by the users from sources is an important aspect > > > IMHO. > > > > > > J. > > > > > > > > > > > > pt., 4 wrz 2020, 10:15 użytkownik Bertrand Delacretaz < > > > > > > bdelacre...@apache.org> napisał: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > On Fri, Sep 4, 2020 at 9:22 AM Jarek Potiuk <ja...@potiuk.com> > > > wrote: > > > > > > > > ...I think having ASF "apache/chart" for all "officially > > > released" helm > > > > > > > > charts is the best solution.... > > > > > > > > > > > > > > Sounds like a good idea but please use a more specific name, > > > "charts" > > > > > > > is very generic. > > > > > > > > > > > > > > If distributing those Helm charts does not requires more than a > > web > > > > > > > server (with some metadata files I suppose) that can probably > go > > > in a > > > > > > > specific folder under > > https://dist.apache.org/repos/dist/release/ > > > > > > > > > > > > > > The next question is what kind of traffic is expected there, > and > > > can > > > > > > > the clients which will download those Helm charts handle > > redirects > > > to > > > > > > > distribution mirrors? If yes the mechanism of > > > > > > > http://www.apache.org/dev/release-download-pages.html comes to > > > mind, > > > > > > > if you can piggyback on that mirroring infrastructure that's a > > big > > > > > > > plus IMO. > > > > > > > > > > > > > > I guess the way to move forward is for the people interested in > > > this > > > > > > > to list the technical and licensing/legal requirements on a > wiki > > > page > > > > > > > or equivalent, to discuss with the ASF infrastructure team how > to > > > best > > > > > > > support them. > > > > > > > > > > > > > > -Bertrand > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > > > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > > > -- > > > Matt Sicker <boa...@gmail.com> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > -- > +48 660 796 129 >
