While the point about being less time is probably correct, yeah if something is half-done, we should keep them in the master branch, and/or don't expose it to the end users (which I believe we usually do). Good thing is that we can make the schedule predictable. Suppose that the branchcut date is pinned, then people can schedule/estimate their work based on the pinned schedule :-).
On Thu, 16 Feb 2023 at 04:32, Sean Owen <sro...@gmail.com> wrote: > I don't think there is a delay per se, because there is no hard release > date to begin with, to delay with respect to. It's been driven by, "feels > like enough stuff has gone in" and "someone is willing to roll a release", > and that happens more like every 8-9 months. This would be a shift not only > in expectation - lower the threshold for 'enough stuff has gone in' to > probably match a 6 month cadence - but also a shift in policy to a release > train-like process. If something isn't ready then it just waits another 6 > months. > > You're right, the problem is kind of - what is something is in process in > a half-baked state? you don't really want to release half a thing, nor do > you want to develop it quite separately from the master branch. > It is worth asking what prompts this, too. Just, we want to release > earlier and more often? > > On Wed, Feb 15, 2023 at 1:19 PM Maciej <mszymkiew...@gmail.com> wrote: > >> Hi, >> >> Sorry for a silly question, but do we know what exactly caused these >> delays? Are these avoidable? >> >> It is not a systematic observation, but my general impression is that we >> rarely delay for sake of individual features, unless there is some soft >> consensus about their importance. Arguably, these could be postponed, >> assuming we can adhere to the schedule. >> >> And then, we're left with large, multi-task features. A lot can be done >> with proper timing and design, but in our current process there is no way >> to guarantee that each of these can be delivered within given time window. >> How are we going to handle these? Delivering half-baked things is hardly >> satisfying solution and more rigid schedule can only increase pressure on >> maintainers. Do we plan to introduce something like feature branches for >> these, to isolate upcoming release in case of delay? >> >> On 2/14/23 19:53, Dongjoon Hyun wrote: >> >> +1 for Hyukjin and Sean's opinion. >> >> Thank you for initiating this discussion. >> >> If we have a fixed-predefined regular 6-month, I believe we can persuade >> the incomplete features to wait for next releases more easily. >> >> In addition, I want to add the first RC1 date requirement because RC1 >> always did a great job for us. >> >> I guess `branch-cut + 1M (no later than 1month)` could be the reasonable >> deadline. >> >> Thanks, >> Dongjoon. >> >> >> On Tue, Feb 14, 2023 at 6:33 AM Sean Owen <sro...@gmail.com> wrote: >> >>> I'm fine with shifting to a stricter cadence-based schedule. Sometimes, >>> it'll mean some significant change misses a release rather than delays it. >>> If people are OK with that discipline, sure. >>> A hard 6-month cycle would mean the minor releases are more frequent and >>> have less change in them. That's probably OK. We could also decide to >>> choose a longer cadence like 9 months, but I don't know if that's better. >>> I assume maintenance releases would still be as-needed, and major >>> releases would also work differently - probably no 4.0 until next year at >>> the earliest. >>> >>> On Tue, Feb 14, 2023 at 3:01 AM Hyukjin Kwon <gurwls...@gmail.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> *TL;DR*: Branch cut for every 6 months (January and July). >>>> >>>> I would like to discuss/propose to make our release cadence >>>> predictable. In our documentation, we mention as follows: >>>> >>>> In general, feature (“minor”) releases occur about every 6 months. >>>> Hence, >>>> Spark 2.3.0 would generally be released about 6 months after 2.2.0. >>>> >>>> However, the reality is slightly different. Here is the time it took >>>> for the recent releases: >>>> >>>> - Spark 3.3.0 took 8 months >>>> - Spark 3.2.0 took 7 months >>>> - Spark 3.1 took 9 months >>>> >>>> Here are problems caused by such delay: >>>> >>>> - The whole related schedules are affected in all downstream >>>> projects, vendors, etc. >>>> - It makes the release date unpredictable to the end users. >>>> - Developers as well as the release managers have to rush because >>>> of the delay, which prevents us from focusing on having a proper >>>> regression-free release. >>>> >>>> My proposal is to branch cut every 6 months (January and July that >>>> avoids the public holidays / vacation period in general) so the release can >>>> happen twice >>>> every year regardless of the actual release date. >>>> I believe it both makes the release cadence predictable, and relaxes >>>> the burden about making releases. >>>> >>>> WDYT? >>>> >>> >> -- >> Best regards, >> Maciej Szymkiewicz >> >> Web: https://zero323.net >> PGP: A30CEF0C31A501EC >> >>
