This also sounds good to me. The master branch will continue to be the publishing area for the release Dockerfiles and we can support changes between Flink versions without additional complexity.
My only comment would be that I hope we can still keep eventual convergence in mind—it should be possible, with some changes in Flink, to make the differences in the Docker packaging between versions negligible, to the point that this split might stop being necessary. On Mon, Apr 27, 2020 at 3:07 PM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for starting this discussion Chesnay. Your proposal sounds good to > me. I can see how the current setup makes the development of version > specific features impractical. Hence, +1 for the proposed changes. > > Cheers, > Till > > On Mon, Apr 27, 2020 at 12:19 PM David Anderson <da...@alpinegizmo.com> > wrote: > > > Makes sense to me. I think this would align well enough with user > > expectations, and be more straightforward. > > > > David > > > > On Thu, Apr 23, 2020 at 1:23 PM Chesnay Schepler <ches...@apache.org> > > wrote: > > > > > Hello everyone, > > > > > > Currently, all development in docker-flink occurs on the master branch, > > > for all releases at once. The master branch also serves as a publishing > > > area for our docker files. > > > > > > This means that all versions share the same generators, templates > (i.e., > > > production code that we plugin in some variables like the Flink > version) > > > and test setup. > > > > > > So far this worked fine because all versions worked the same, but we > are > > > now coming to a point where we are adding version-specific features, > > > like support for Java 11 (FLINK-16260) and per-job-clusters > > (FLINK-17164). > > > > > > By virtue of all files being shared this means that we would have to > > > introduce version checks in both the generators, templates and test > > > code, so things continue to work for all versions. This is rather > > > painful to do, and presents maintenance problems as these conditions > > > must of course be updated at some point, and diverging behaviors > require > > > all versions to be tested on each PR (in contrast to the current state > > > where we only test 1 version). > > > > > > It is also simply different to the process we're using for > > > flink/flink-shaded, creating an additional entry barrier. > > > > > > I propose moving the development of the docker files into dedicated > > > branches (dev-master, dev-1.10, dev-1.9). PullRequests will usually be > > > opened against dev-master, possibly ported to other versions, and when > > > it is time for a release we will update the master branch with the > > > latest dockerfiles. Quite similar to how things work in the Flink repo. > > > > > > The master branch would continue to be our release publishing area to > > > stick to docker conventions. > > > > > > Regards, > > > > > > Chesnay > > > > > > > > >
