> > would definitely be in favour of that approach and using it more > liberally. I think SemVer does not say anything about this case - "the > software still supports it but you need to flip a flg" sounds like a nice > way of introducing behavioural changes without breaking changes.
This doesn't sound right to me. According to what you're proposing, we could change the default to `catchup=False` and add an option to flip it back. But then when a user upgrades, many dags in the wild would stop "catching up", since it's not a required parameter. This feels like it is totally forbidden by semver. I think it's different from e.g., moving the executors out and forcing user to install them, because that's effectively an "installation instructions" change whereas this is a behavior change. On Sat, Sep 2, 2023 at 1:21 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > What about more frequently using news fragments + using configuration > settings as a way to introduce breaking changes that users can revert? > > I would definitely be in favour of that approach and using it more > liberally. I think SemVer does not say anything about this case - "the > software still supports it but you need to flip a flg" sounds like a nice > way of introducing behavioural changes without breaking changes. > > We could even introduce some way (adding interactive `airflow db migrate`) > - to ask the user during migration what to do. That would make the > automated and generally unattended Helm Chart migrations, but I am sure we > can figure out something there but I easily imagine that when we detect > that user migrates from 2.7.0 to 2.8.0 and we have some feature disabled > there by default we could ask the question "This feature in 2.8.0 is now > disabled by default, do you want to enable it ? + link to documentation > explaining what it is and why we disabled it". > > J. > > > On Fri, Sep 1, 2023 at 11:30 PM Ryan Hatter > <ryan.hat...@astronomer.io.invalid> wrote: > > > What about more frequently using news fragments + using configuration > > settings as a way to introduce breaking changes that users can revert? > > Disabling > > "trigger dag with config" without Params by default > > < > > > https://airflow.apache.org/docs/apache-airflow/2.7.0/release_notes.html#the-trigger-ui-form-is-skipped-in-web-ui-if-no-parameters-are-defined-in-a-dag-33351 > > > > > is more or less what I'm referring to. While this was a breaking change, > > users were given the option to "roll back". > > > > On Thu, Aug 31, 2023 at 12:14 AM Amogh Desai <amoghdesai....@gmail.com> > > wrote: > > > > > Very good discussions going on here. > > > > > > Semver has been a point of concern for us too in our internal product. > > > Some ideas emerging out of this could help me there. Thanks, Jarek and > > > Niko. > > > > > > There are two points I'd like to stress on to say why semver is that > > > important: > > > > > > - Compatibility: Without versioning that indicates compatibility, > > > users might hesitate to upgrade due to concerns about potential > > > breaking changes. This could lead to fragmentation, where different > > > users are using different versions of the software, making support and > > > maintenance more challenging. > > > > > > - Release Communication: Semantic versioning helps communicate the > > > impact of a release to users, maintainers, and contributors. Without > > > clear versioning, understanding the significance of a release becomes > > > more difficult. > > > > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Thu, Aug 31, 2023 at 3:56 AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > > > > > Now, one elephant in the room - the 5 year security patches thing > > Jarek > > > > brought up. I freely admit I haven't tracked this at all, so please > > > correct > > > > me if I'm wrong. If that ends up panning out though, I think we will > > have > > > > to reassess our strategy with providers too. > > > > > > > > Just to answer the last point. Yes. I believe so. I was never a big > fan > > > of > > > > introducing breaking changes in providers to make maintainers' lives > a > > > bit > > > > easier. While I was a big fan of doing it when we had a very good > > reason. > > > > > > > > I think we have too many breaking changes in providers now. And we do > > it > > > > because we "can" - we mostly only consider maintainer's lives easier > > > > currently. But I think when those regulations are in place we will > have > > > to > > > > make a much more deliberate judgment on when we make a major release > > > > in providers and take on a deliberate decision "is it worth doing it, > > > > knowing that we will have to deliver patches for previous major > > > versions". > > > > This will be something that the regulations might make us think about > > > when > > > > making a decision "should we break?". And when we do - we should be > > > > prepared to cherry-pick security fixes to those old versions. We > > > currently > > > > can have a process for it - and it is off-loaded mainly to > > stakeholders. > > > > > > > > > > https://github.com/apache/airflow/blob/main/PROVIDERS.rst#mixed-governance-model > > > > - where we mainly take care about releasing cherry-picked code. > > > > > > > > I believe the overwhelming majority of those "breaking" releases that > > we > > > > really needed, were in providers where there is an active stakeholder > > > > already cooperating with us. I have - honestly quite an expectation > > that > > > > this will stay like that. In the proposed regulations, the > stakeholders > > > are > > > > (much more than the OSS foundations) responsible for security of the > > > > software they provide to their users and they will have incentive to > > fix > > > > and provide fixes also for past releases of those integration. And I > > > think > > > > we can work out a collaborative model on that - very close to the > > "mixed > > > > governance" we already have. And in other cases where we have no > active > > > > stakeholders, we might simply refrain from making breaking changes if > > not > > > > absolutely needed. > > > > > > > > That would be my approach to the elephant. > > > > > > > > J, > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
