I don't think there is a delay per se, because there is no hard release date to begin with, to delay with respect to. It's been driven by, "feels like enough stuff has gone in" and "someone is willing to roll a release", and that happens more like every 8-9 months. This would be a shift not only in expectation - lower the threshold for 'enough stuff has gone in' to probably match a 6 month cadence - but also a shift in policy to a release train-like process. If something isn't ready then it just waits another 6 months.
You're right, the problem is kind of - what is something is in process in a half-baked state? you don't really want to release half a thing, nor do you want to develop it quite separately from the master branch. It is worth asking what prompts this, too. Just, we want to release earlier and more often? On Wed, Feb 15, 2023 at 1:19 PM Maciej <mszymkiew...@gmail.com> wrote: > Hi, > > Sorry for a silly question, but do we know what exactly caused these > delays? Are these avoidable? > > It is not a systematic observation, but my general impression is that we > rarely delay for sake of individual features, unless there is some soft > consensus about their importance. Arguably, these could be postponed, > assuming we can adhere to the schedule. > > And then, we're left with large, multi-task features. A lot can be done > with proper timing and design, but in our current process there is no way > to guarantee that each of these can be delivered within given time window. > How are we going to handle these? Delivering half-baked things is hardly > satisfying solution and more rigid schedule can only increase pressure on > maintainers. Do we plan to introduce something like feature branches for > these, to isolate upcoming release in case of delay? > > On 2/14/23 19:53, Dongjoon Hyun wrote: > > +1 for Hyukjin and Sean's opinion. > > Thank you for initiating this discussion. > > If we have a fixed-predefined regular 6-month, I believe we can persuade > the incomplete features to wait for next releases more easily. > > In addition, I want to add the first RC1 date requirement because RC1 > always did a great job for us. > > I guess `branch-cut + 1M (no later than 1month)` could be the reasonable > deadline. > > Thanks, > Dongjoon. > > > On Tue, Feb 14, 2023 at 6:33 AM Sean Owen <sro...@gmail.com> wrote: > >> I'm fine with shifting to a stricter cadence-based schedule. Sometimes, >> it'll mean some significant change misses a release rather than delays it. >> If people are OK with that discipline, sure. >> A hard 6-month cycle would mean the minor releases are more frequent and >> have less change in them. That's probably OK. We could also decide to >> choose a longer cadence like 9 months, but I don't know if that's better. >> I assume maintenance releases would still be as-needed, and major >> releases would also work differently - probably no 4.0 until next year at >> the earliest. >> >> On Tue, Feb 14, 2023 at 3:01 AM Hyukjin Kwon <gurwls...@gmail.com> wrote: >> >>> Hi all, >>> >>> *TL;DR*: Branch cut for every 6 months (January and July). >>> >>> I would like to discuss/propose to make our release cadence predictable. >>> In our documentation, we mention as follows: >>> >>> In general, feature (“minor”) releases occur about every 6 months. Hence, >>> Spark 2.3.0 would generally be released about 6 months after 2.2.0. >>> >>> However, the reality is slightly different. Here is the time it took for >>> the recent releases: >>> >>> - Spark 3.3.0 took 8 months >>> - Spark 3.2.0 took 7 months >>> - Spark 3.1 took 9 months >>> >>> Here are problems caused by such delay: >>> >>> - The whole related schedules are affected in all downstream >>> projects, vendors, etc. >>> - It makes the release date unpredictable to the end users. >>> - Developers as well as the release managers have to rush because of >>> the delay, which prevents us from focusing on having a proper >>> regression-free release. >>> >>> My proposal is to branch cut every 6 months (January and July that >>> avoids the public holidays / vacation period in general) so the release can >>> happen twice >>> every year regardless of the actual release date. >>> I believe it both makes the release cadence predictable, and relaxes the >>> burden about making releases. >>> >>> WDYT? >>> >> > -- > Best regards, > Maciej Szymkiewicz > > Web: https://zero323.net > PGP: A30CEF0C31A501EC > >
