Thanks for starting this discussion. In general, i am also in favor of making docker image release could be partly decoupled from Flink release. However, i think it should only happen when we want to make some changes that are independent from Flink majar release(e.g. JDK11, change base image, install more tools, upgrade the docker entrypoint, etc.). If the Flink release branch codes have changed, we should not build them in a new docker image. Instead, if we could provide the daily updating snapshot for docker image, it will be very helpful for the users who want to have a taste of new features or benefit from urgent hotfix.
Best, Yang Ismaël Mejía <ieme...@gmail.com> 于2020年4月28日周二 下午11:07写道： > > To me this means that the docker images should be released at the same > time the other artifacts are released. This also includes shapshot > releases. So the build of the docker images should be an integral part of > the build. > > This is already the case since the last release, what this thread is > about are the intermediary releases that correspond to an upstream > release. Let’s call those the 1.10.0-dockerN. Let’s not forget that > our released docker images are not fully immutable artifacts because > upstream docker official images maintainers may (and regularly do) > update them for example to fix security issues on the parent images or > to catch new versions of Java or the base OS in our case openjdk and > debian. > > These releases may be required to fix issues or include new features > that are independent of Flink releases (we can also ignore and be > tight to the Flink release cycle to simplify the process), but > remember that the upstream docker images maintainers may ‘force us’ to > do updates (this rarely happens but when it happens is something we > cannot ignore). > > > I do wonder though whether the Dockerfiles really are binary releases. > > The Dockerfiles are not binary releases but are the base of the docker > image releases let’s not forget that since we want the images to be > official docker-images release > https://github.com/docker-library/official-images we have to align > with their processes too. > > I suppose the wisest is to align with the ASF vote/release process and > that this produces a new PR of and image in the official-images repo > (which would be the dockee equivalent of publishing to central). There > are probably some minor issues that we might find in the way, so far > the only one I can think of is how would we identify intermediary > docker releases but apart of this I am +1 on ‘formal’ release process > even if I found it too heavy we are an ASF project so we should align > with its rules. > > Also this goes inline with the existing proposal by Chesnay on “Moving > docker development into versioned branches” > > https://lists.apache.org/thread.html/rbb2569d24b35a40adb01b7abfe62fc1ada0408539403ef826491d0ee%40%3Cdev.flink.apache.org%3E > > Any other opinions? > > If we reach consensus maybe we should do an initial extra release and > VOTE to test the process. I would like to volunteer to do this and > document the process (we could for example do this to include the Java > 11 version), but well let’s wait for other opinions first. > > Ismaël > > ps. I should probably be accompanied by a committer at some steps > because I could lack some permissions to do the release. > > On Tue, Apr 28, 2020 at 1:17 PM Chesnay Schepler <ches...@apache.org> > wrote: > > > > I agree with Niels that if a Flink release is made it should be > > accompanied by the Dockerfiles for that version, and release them at > once. > > This makes sense for testing purposes alone, and I view Docker as just > > another distribution channel like Maven central. > > The reverse isn't necessarily true though, as in we can release an > > updated version of the Dockerfiles without an accompanying Flink release. > > This is just for convenience so we can fix issues on the Docker side > faster. > > This is a fair compromise imo. > > > > I do wonder though whether the Dockerfiles really are binary releases. > > Aren't they more of an instruction set to assemble one? > > Given that we actively point other parties to files&commits in our repo > > (which I suppose by definition is the source), then it seems > > questionable to categorize this as a binary release. > > > > On 28/04/2020 09:30, Aljoscha Krettek wrote: > > > Hi Niels, > > > > > > I think Robert was referring to the fact that Apache considers only > > > the source release to be "the release", everything else is called > > > convenience release. > > > > > > Best, > > > Aljoscha > > > > > > On 27.04.20 19:43, Niels Basjes wrote: > > >> Hi, > > >> > > >> In my opinion the docker images are essentially simply differently > > >> packed > > >> binary releases. > > >> > > >> This becomes more true when in the future deploying a Flink > > >> application to > > >> kubernetes simply pulls the correct binary from a docker hub. Because > of > > >> these kinds of use cases I disagree with Robert that these are just > > >> convenience releases. > > >> > > >> To me this means that the docker images should be released at the > > >> same time > > >> the other artifacts are released. This also includes shapshot > > >> releases. So > > >> the build of the docker images should be an integral part of the > build. > > >> > > >> Do note that there will be multiple sub-versions for each release with > > >> variations in for example the used Scala version and/or JDK. All > > >> published > > >> at the same moment. > > >> > > >> Niels Basjes > > >> > > >> On Mon, Apr 27, 2020, 10:26 Robert Metzger <rmetz...@apache.org> > wrote: > > >> > > >>> Thanks for starting the thread! > > >>> > > >>> I would consider the docker images of Flink convenience binary > releases > > >>> that can happen any time. I believe a simplified, but formal release > > >>> process would be appropriate (preview / staging images for the > > >>> community to > > >>> validate & vote, then release to docker hub). > > >>> > > >>> On Wed, Apr 22, 2020 at 3:29 PM Chesnay Schepler <ches...@apache.org > > > > >>> wrote: > > >>> > > >>>> We can create additional releases independent of Flink, but they > will > > >>>> have to go through a formal release process in any case. > > >>>> > > >>>> On 22/04/2020 14:55, Ismaël Mejía wrote: > > >>>>> Hello, > > >>>>> > > >>>>> I wanted to discuss about a subject that was shortly mentioned > during > > >>>> the docker > > >>>>> unification thread (and in some other PR) that is the release of > > >>>>> docker > > >>>> images > > >>>>> independent of major Flink releases. > > >>>>> > > >>>>> In the past when the docker images were maintained outside of the > > >>> Apache > > >>>>> repository we usually did intermediary releases to fix issues or > > >>>>> to add > > >>>> new > > >>>>> functionalities that were independent of the Flink releases and > > >>> specific > > >>>> of the > > >>>>> docker images. > > >>>>> > > >>>>> There are two major cases when this happened: > > >>>>> > > >>>>> 1. If the upstream official docker images maintainers requested us > to > > >>> do > > >>>> the > > >>>>> changes or there was some breakage because some upstream > change > > >>>> (this is rare > > >>>>> but has happened in the past). > > >>>>> > > >>>>> 2. If we wanted to fix issues or add new functionality to the > images > > >>>> that was > > >>>>> independent of the full Flink release. > > >>>>> > > >>>>> We have been working on having Java 11 based images available and > > >>>>> this > > >>>> is an > > >>>>> example of (2), where we want to publish these images based on the > > >>>> already > > >>>>> released 1.10.0 version. > > >>>>> > > >>>>> So I would like to know your opinion on how should we proceed in > the > > >>>> future. > > >>>>> Ideally we should wait for the major release, but the reality is > that > > >>>> (1) can > > >>>>> happen and (2) can benefit end users. > > >>>>> > > >>>>> So what should we do? Can we do these updates without a formal > > >>>>> release > > >>>> as we did > > >>>>> before, or does it make sense to follow a release process with the > > >>>> corresponding > > >>>>> vote for the docker images? or are there other alternatives? > > >>>>> > > >>>>> Regards, > > >>>>> Ismaël > > >>>>> > > >>>> > > >>>> > > >>> > > >> > > > > > > > > >
