Sounds good. +1 On Mon, Apr 12, 2021 at 1:23 PM Chesnay Schepler <ches...@apache.org> wrote:
> Hello all, > > I would like to propose a change in how the Flink master interacts with > Flink-shaded. > > TL;DR: Release snapshot artifacts for flink-shaded, and have the Flink > master rely on specific snapshot versions for earlier dependency bumps. > > > Aa a project we have come to the general conclusion that dependencies > should be bumped as early in the release cycle as possible. This both > prevents cases where some undefined amount of work is still waiting for > as when we want to release the next version (working against the goal of > always being in a releasable state), and it gives us more time to > evaluate the stability and performance of system. Finally it gives us > ample time to look for alternatives if an issue is found. > > Currently, this conclusion is at odds with how we handle flink-shaded. > Flink has always relied on flink-shaded artifacts that went through a > proper release cycle. However, since we want to create as few releases > as possible due to the overhead/noise/etc., flink-shaded releases are > typically relegated to the end of the release cycle. > This is particularly troublesome since flink-shaded dependencies are > used in the core of Flink, and hence usage of them cannot be avoided. > > As a compromise between these 2 goals I propose the following: > - we deploy SNAPSHOT artifacts for flink-shaded for every change made > - every deployed artifact has a unique version, that is automatically > set via maven (=> no overhead on our side) > - once such an artifact is released we update the Flink dependency to > point to this _specific_ flink-shaded snapshot artifact > - to be clear, this is a manual step, which implies that things > cannot break all of a sudden because something was pushed to flink-shaded > - once the Flink release cycle ends, we publish a proper flink-shaded > release, and change the Flink dependency in the release branch accordingly > > This should give us the best of both worlds: We have as few releases as > necessary (at most 1 per Flink release cycle), but can update the > dependencies in Flink as soon as possible. > Furthermore, this can also be considered a test run for how multiple > repos with the same release cycle could be developed in sync with each > other. > > Let me know what you think. > > Regards, > > Chesnay > > -- Konstantin Knauf https://twitter.com/snntrable https://github.com/knaufk
