Re: [Development] Changing Qt's Binary Compatibility policy
On Tuesday 18 June 2024 07:53:32 GMT-7 Giuseppe D'Angelo via Development wrote: > I think this is now effectively a separate thread of discussion, since > these deprecations create source incompatibilities, not binary > incompatibilities. > > Should we discuss it at the QtCS in a couple of slots? Please note we *can* have a binary-incompatible, source-compatible version, technically speaking. We do have two version numbers in our binaries: libQt6Core.so.6 The first 6 is the source version; the second one is the binary version. How to communicate the difference to users, how to market the versions, and whether it is worth the hassle are completely separate topics. -- Thiago Macieira - thiago.macieira (AT) intel.com Principal Engineer - Intel DCAI Fleet Systems Engineering smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
Hi, On 18/06/2024 10:15, Alex Blasche via Development wrote: Our biggest issue is the adoption of Qt by users moving from one major release to the next. The deprecations start to become a liability and while they keep SC compatibility in check for Qt 6 they become a serious concern for any adoption of Qt 7. While doing changes when it is necessary is OK as part of a major patch release, I totally agree with Andre that a lot of deprecations are nice to have and not mandatory. I would like to propose a far more restrictive process on when a deprecation is accepted (unless somebody has a better idea on how to curb this flood gate any other way). If you need any further prove of the seriousness of this problem, have a look at how long it took for KDE to adopt Qt 6 or look at the still incoming issues for Qt 5.15. Based on what I see, performance differences are the most common reason why Qt 6 is not ported to. The second one is the effort to port to Qt 6 due to mechanical porting from one API to the next. Qt 7 deprecations show an alarming trend and those mostly don't even involve truly needed architecture changes like potential QIODevice or QFileSystemEngine changes yet. (I only use these two classes examples to highlight the point and not that there is actual work being decided or ongoing) I think this is now effectively a separate thread of discussion, since these deprecations create source incompatibilities, not binary incompatibilities. Should we discuss it at the QtCS in a couple of slots? Thanks, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - Trusted Software Excellence smime.p7s Description: S/MIME Cryptographic Signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
>From: Development on behalf of apoenitz > >Sent: Friday, 14 June 2024 19:32 >There are by now ~240 QT_DEPRECATED_VERSION_X_6_* in qtbase alone. Fixing them >today involves touching thousands line of code in my code base(s). Very few of >them are truly, clearly necessary. The bulk is "nice to have" or "should have >been like that from the beginning (but the benefit is not sufficient to >the drop the original)", and a few are just pain without (significant) gain. Our biggest issue is the adoption of Qt by users moving from one major release to the next. The deprecations start to become a liability and while they keep SC compatibility in check for Qt 6 they become a serious concern for any adoption of Qt 7. While doing changes when it is necessary is OK as part of a major patch release, I totally agree with Andre that a lot of deprecations are nice to have and not mandatory. I would like to propose a far more restrictive process on when a deprecation is accepted (unless somebody has a better idea on how to curb this flood gate any other way). If you need any further prove of the seriousness of this problem, have a look at how long it took for KDE to adopt Qt 6 or look at the still incoming issues for Qt 5.15. Based on what I see, performance differences are the most common reason why Qt 6 is not ported to. The second one is the effort to port to Qt 6 due to mechanical porting from one API to the next. Qt 7 deprecations show an alarming trend and those mostly don't even involve truly needed architecture changes like potential QIODevice or QFileSystemEngine changes yet. (I only use these two classes examples to highlight the point and not that there is actual work being decided or ongoing) >As an example for the latter: QCryptographicHash::addData() went from >QByteArray plus ptr/len overload to a single addData(QByteArrayView bytes). >Most of the existing uses were of the addData(reinterpret_castY*(), sizeof(something)) kind - ugly enough already, but now it needs >an additional QByteArrayView() wrapped around. Why? Why can't the ptr/len >overload just stay? One line, inline, on the library side? Because someone >doesn't like raw pointers but a struct with a raw pointer and an int will >save the world? Based on current adoption issues with Qt 5.15, these changes constitute an alarming adoption problem for Qt 7. I cannot even argue to a customer what good this change would do to his code or even Qt. The right way should have been to add the new function, leave the old one in and remove the old one from the docs/mark it as \internal. On Friday 24 May 2024 13:29:41 GMT-3 Giuseppe D'Angelo via Development wrote: >2) We stop guaranteeing forward binary compatibility within the same >minor version. For what it is worth, I agree too. -- Alex -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
On Mon, May 27, 2024 at 09:55:37AM +0200, Giuseppe D'Angelo wrote: > Hi, > > On 25/05/2024 13:03, apoenitz wrote: > > As an open source application developer I mostly care about SC, as SC > > breakages > > means actual work for/me/. > > To better understand: when you talk about SC, are you referring to literally > source compatibility (application can be recompiled with newer version of > Qt, without any source changes) or more in general to behavioural > compatibility? In the text of my response I distinguished between source compatibility and behavioural compatibility, "SC" was refering to literal source compatibility. In any case, I see already this part of the discussion as indication that the focus is not where I as a user would have hoped to be: As a user I would like a library to be beneficial in some way, typically by saving me from doing similar work myself or finding/buying/.. elsewhere, minus, of course, the effort/time/cost/.. needed to use this library, including adapting my own code to other, usually newer, versions of the library. Ideally, the various compatibility guarantees would help, or even ensure, that such upgrades are cheap. Full source compatibility would e.g. save me from having to change my code at all, strong behavioural compatibility guarantee would save me from having to worry about my working applications to suddenly break. But, as mentioned, binary compatibility does not really help me much. This all is not white and black, the effort needed for the required adaptation scales with the severeness and the number of the compatibilty breakages. Often, /some/ effort to overcome those breakages is tolerable, for instance when other people/projects benefit in total much more from the new behaviour then I suffer. My problem currently is that I do not see that balance since ~5.12 anymore. Instead we have now an industrial scale process to deprecate stuff, giving me the choice of taking the pain of a thousand little cuts spread out over a few years be heeding all the warnings or to ignore them and take the heavy blow come 7.0. There are by now ~240 QT_DEPRECATED_VERSION_X_6_* in qtbase alone. Fixing them today involves touching thousands line of code in my code base(s). Very few of them are truly, clearly necessary. The bulk is "nice to have" or "should have been like that from the beginning (but the benefit is not sufficient to the drop the original)", and a few are just pain without (significant) gain. As an example for the latter: QCryptographicHash::addData() went from QByteArray plus ptr/len overload to a single addData(QByteArrayView bytes). Most of the existing uses were of the addData(reinterpret_cast(), sizeof(something)) kind - ugly enough already, but now it needs an additional QByteArrayView() wrapped around. Why? Why can't the ptr/len overload just stay? One line, inline, on the library side? Because someone doesn't like raw pointers but a struct with a raw pointer and an int will save the world? Similarly, QString::fromUcs4() went from uint to char32_t. The first works well to interface with C89 code (and seriously, where else do you get UCS4 data from?). char32_t doesn't even exist in that world. So it's ugly casts to "fix" again, and again: what's the point of forcing thousands of people and projects to adapt to that instead of having /one line/, inline, on the library side? Now, these two cases require only mechanical changes, conceptually easy to do, but annoying and eating time that cannot be spend for actual improvements of the user's application. Others are worse, and I don't want to get started with the behaviour changes here. To come back to why I think the focus in the compatibility discussion is wrong: The topics there are simply irrelevant from the perspective of someone /only/ /using/ Qt: Whether something is foo-bar incompatible of type A or zig-zag incompatible of type B does not matter. For the user it matters how much effort it takes to get back to a working state after an update of Qt. And for us the discussion should not be foo-bar vs zig-zag, but /actually/ /helping/ users vs having fun with ivory tower discussions and processes. Andre' -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
On Thursday 30 May 2024 15:56:26 CEST Volker Hilsheimer via Development wrote: > My larger concern is that for patch releases, we have no processes to avoid > that we end up adding poor APIs. We don’t do a header review, and we don’t > have a whatsnew documentation file. The changelog should be enough for the > kinds of things we are presumably talking about here. But agreeing to this > will give ourselves the freedom to add new symbols within a patch cycle, so > we should perhaps narrow this down a bit. > - must fix a P1 bug > - add symbols required to support new platforms (e.g. new > QOperatingSystemVersion constants) or otherwise realities out of our > control (e.g. new locale enum values - not a symbol anyway) > - doesn’t > touch any template code, because otherwise we *must* have a header review > ;) > We could carve out an exception for new symbols already approved (and released) for a newer version of Qt in case it is needed in a patch release. One tricky point which is already causing issues sometimes on minor version updates is adding new enum values. Code that assumes it knows all possible values given to it by Qt, can break if a new value is issued from Qt code. But since such code could already break with a minor update, we could take the stance that it is also allowed to break with a patch update. Best regards Allan -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
> On 29 May 2024, at 05:30, Kevin Kofler via Development > wrote: > > Giuseppe D'Angelo via Development wrote: >> 2) We stop guaranteeing forward binary compatibility within the same >> minor version. >> >> In other words, code compiled against Qt X.Y.Z may or may not work if at >> runtime Qt X.Y.W is used, with W> >> Details: no user downgrades Qt and therefore has ever needed this. This >> is something that only Qt developers themselves have possibly needed -- >> seems to be a historical remnant at this point. >> >> Concretely, this means that we can make our lives easier; for instance, >> when backporting fixes, we'll allow new symbols to appear in patch >> releases. > > Hmmm… I do indeed think that this can help getting some bug fixes backported > more easily, and indeed, within one (GNU/Linux) distribution, we normally > care only about backwards compatibility, not forwards compatibility. Guilty as charged for not following up on last year's Qt CS discussion by sending the email to the mailing list to put forth a specific proposal - also because we very last minute (and evidently after the note taker had closed his laptop) had some new concerns raised. - a 6.5.5 release might introduce symbols not present in 6.6.1 (but presumably present in 6.6.2, released chronologically after 6.5.5) We can explicitly exclude support for this upgrade path. - 3rd party, binary-only libraries A user project might use several binary-only libraries that have been built against different patch levels of the same minor Qt version. This is where things might get hairy. I don’t know how. One of those binary-only libraries might suffer from all sorts of issues when it runs against version of Qt with fewer bug fixes, but at least the symbols should be satisfied. My larger concern is that for patch releases, we have no processes to avoid that we end up adding poor APIs. We don’t do a header review, and we don’t have a whatsnew documentation file. The changelog should be enough for the kinds of things we are presumably talking about here. But agreeing to this will give ourselves the freedom to add new symbols within a patch cycle, so we should perhaps narrow this down a bit. - must fix a P1 bug - add symbols required to support new platforms (e.g. new QOperatingSystemVersion constants) or otherwise realities out of our control (e.g. new locale enum values - not a symbol anyway) - doesn’t touch any template code, because otherwise we *must* have a header review ;) Volker -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
On Wednesday 29 May 2024 16:54:33 GMT-3 Narolewski Jakub wrote: > Sorry for chiming in uninvited but isn't this how the Qt Online Installer > thingy works? Very carefully. It's built on a reasonably old Linux distribution, the oldest of all of them that are still supported. It also builds with all or almost all third-party content bundled, instead of depending on system libraries, and some features disabled where system libraries would be required (for example, journald integration, zstd compression support, etc.). Your Linux distribution's packages are superior builds in almost every aspect, except that of being run in other distributions. Because the pre-built binaries from download.qt.io contain bundled third- parties that do get a couple of security advisories per year, you must be able to rebuild the binaries on a moment's notice. So don't trust the binaries you download from download.qt.io except for development: for your releases, you should build Qt from source. -- Thiago Macieira - thiago.macieira (AT) intel.com Principal Engineer - Intel DCAI Fleet Engineering and Quality smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
Sorry for chiming in uninvited but isn't this how the Qt Online Installer thingy works? I know that with it I can install some prebuilt Qt distribution on my Linux - openSUSE Tumbleweed - and I kinda always assumed that the actual build host could be a different distro. I never really gave it much thought to be honest. Maybe there is some kind of matching mechanism somewhere in there during the download process. -- Best regards / Pozdrawiam serdecznie Narolewski Jakub Software Developer śr., 29 maj 2024 o 21:29 Thiago Macieira napisał(a): > On Wednesday 29 May 2024 00:30:12 GMT-3 Kevin Kofler via Development wrote: > > There is, however, one use case you are overlooking, and that is binaries > > compiled on one distribution and run on another. > > That's not supported at all and that has nothing to do with Qt. Unless the > two > distributions are coordinating and testing each other's binaries, it's not > guaranteed to work and is in fact a recipe for disaster. But if they are > coordinating, then the problem you're talking about doesn't exist. > > Some distributions have adopted the no-direct-external-access support and > others have not. In order to run against a Qt that was compiled with it, > you > must compile your software with it too, otherwise the loader will refuse > to > start your application. > > I don't understand why some distros explicitly disable that support. It > might > be because they wanted to retain compatibility with the negligible amount > of > Qt 6 software that existed before the option was introduced. But now > they're > locked into it for the duration of Qt 6.x. > > -- > Thiago Macieira - thiago.macieira (AT) intel.com > Principal Engineer - Intel DCAI Fleet Engineering and Quality > -- > Development mailing list > Development@qt-project.org > https://lists.qt-project.org/listinfo/development > -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
On Wednesday 29 May 2024 00:30:12 GMT-3 Kevin Kofler via Development wrote: > There is, however, one use case you are overlooking, and that is binaries > compiled on one distribution and run on another. That's not supported at all and that has nothing to do with Qt. Unless the two distributions are coordinating and testing each other's binaries, it's not guaranteed to work and is in fact a recipe for disaster. But if they are coordinating, then the problem you're talking about doesn't exist. Some distributions have adopted the no-direct-external-access support and others have not. In order to run against a Qt that was compiled with it, you must compile your software with it too, otherwise the loader will refuse to start your application. I don't understand why some distros explicitly disable that support. It might be because they wanted to retain compatibility with the negligible amount of Qt 6 software that existed before the option was introduced. But now they're locked into it for the duration of Qt 6.x. -- Thiago Macieira - thiago.macieira (AT) intel.com Principal Engineer - Intel DCAI Fleet Engineering and Quality smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
Giuseppe D'Angelo via Development wrote: > 2) We stop guaranteeing forward binary compatibility within the same > minor version. > > In other words, code compiled against Qt X.Y.Z may or may not work if at > runtime Qt X.Y.W is used, with W > Details: no user downgrades Qt and therefore has ever needed this. This > is something that only Qt developers themselves have possibly needed -- > seems to be a historical remnant at this point. > > Concretely, this means that we can make our lives easier; for instance, > when backporting fixes, we'll allow new symbols to appear in patch > releases. Hmmm… I do indeed think that this can help getting some bug fixes backported more easily, and indeed, within one (GNU/Linux) distribution, we normally care only about backwards compatibility, not forwards compatibility. There is, however, one use case you are overlooking, and that is binaries compiled on one distribution and run on another. This is how a user can end up with an older Qt than what the application was built against. That said, most other libraries do not promise any kind of forward compatibility either, starting from glibc, so it might not be that big an issue in practice. (If one wants to build a cross-distro binary, one is best off building on an enterprise/LTS distribution with old libraries.) But still, I wanted to bring up this use case. Kevin Kofler -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
Hi, On 25/05/2024 13:03, apoenitz wrote: As an open source application developer I mostly care about SC, as SC breakages means actual work for/me/. To better understand: when you talk about SC, are you referring to literally source compatibility (application can be recompiled with newer version of Qt, without any source changes) or more in general to behavioural compatibility? Thank you, -- Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com KDAB - Trusted Software Excellence smime.p7s Description: S/MIME Cryptographic Signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Changing Qt's Binary Compatibility policy
On Fri, May 24, 2024 at 06:29:41PM +0200, Giuseppe D'Angelo via Development wrote: > Hi, > > One of the discussions at the past Qt Contributors' Summit was about our BC > policy: > > https://wiki.qt.io/Two-way_BC_in_Patch_Releases > > We had a consensus for implementing some changes, but I don't remember the > action points have been taken, so here we are. > > > We propose to change Qt's BC policy as follows: > > > 1) Qt continues to promise backwards binary compatibility for end-users > within the same major version. Specifically, user code that is linked > against a given Qt version will continue to work when linked against a > version of Qt which > a) is higher, and > b) has been released later than the one used to originally link the code. > > Details: > > * "Qt" means the essential modules, except for QtTest. Addon modules may or > may not promise BC (or even SC). > > * "continue to work" just means that user code will be able to load and run > with the newer version of Qt. We're not making any behavioural guarantees in > this policy. > > * "a higher version of Qt" means to follow semantic versioning > (lexicographic / component-wise) ordering. For instance, 6.6.1 is higher > than 6.5.5. > > * "that has been released later": due to the branching policy / LTS / etc., > it is perfectly possible that a higher version of Qt has actually been > released *before* one with a lower version number. > > In the example above, 6.6.1 has been released on 27/Nov/2023, and 6.5.5 on > 4/Mar/2024. Therefore, we _do not support_ upgrading from 6.5.5 to 6.6.1: > 6.5.5 is allowed to have new symbols that are not present in 6.6.1, and > would cause user code to fail to dynamically link/load. > > (6.5.5 is also allowed to have some minor SICs, like the list of > countries/languages in QLocale. 6.6.1 may have bugs that have been fixed in > 6.5.5, thus causing regressions user code.) > > > > 2) We stop guaranteeing forward binary compatibility within the same minor > version. > > In other words, code compiled against Qt X.Y.Z may or may not work if at > runtime Qt X.Y.W is used, with W > Details: no user downgrades Qt and therefore has ever needed this. This is > something that only Qt developers themselves have possibly needed -- seems > to be a historical remnant at this point. > > Concretely, this means that we can make our lives easier; for instance, when > backporting fixes, we'll allow new symbols to appear in patch releases. > > Does this sound good? TL;DR: "Mostly Harmless", i.e. +/- 0 from my side. -- The longer version: I believe the suggested change is harmless, as the current BC policy (guaranteed BC in some cases, but neither source nor behavior compatibility) already today does not actually help in a lot of cases anyway, so changes there cannot possibly do much harm: As an open source application developer I mostly care about SC, as SC breakages means actual work for /me/. My applications target a wide variety of systems, with varying Qt installations. Porting to specific system should be easy, preferably not require patches to the sources, and I don't want to clutter my code with Qt version #ifdefs, not just for readability reasons, but also because realistically I won't test some code paths that way. For distributions BC _might_ be interesting as it theoretically allows to update the libraries without having to recompile the applications. Given the lack of behavioral compatibility one would, however, assume proper testing of the result, and the effort for that easily dwarfs the effort for recompiling the whole thing. As a closed source application developer I am often (more) free to target only one specific Qt version, but there source incompatibily also means that I have to spend extra time when switching versions, and I "of course" also have to test for behavioral changes when doing so, again dwarfing any compile-time savings. So no real gain either. So there's not much difference here, and I am not aware of other typical use cases, i.e. effectively BC as such does not look like huge benefit in general, and for me personally an as-large-as-possible degree of source compatibility and strict behavioral compatibility (except for true bug fixes) would be a /much/ better working environment. Note that I am not saying that BC is useless. In the past it helped a lot, even if not by itself, but by restricting people in making certain incompatible changes. Together with a somewhat conservative mindset on what kind of change is good in general this effectively meant for a long time that "it just worked". This has changed in mid-Qt-5 times. We are now more willing to break SC even for little, sometimes absurdly low, not measurable benefit, we are more diligent to break SC while formally keeping BC, and the (IMHO far too) easy to use "Pick-to:" mechanism means that not only desperately needed fixes are backported but also a higher chance that fumbling affects "stable"
Re: [Development] Changing Qt's Binary Compatibility policy
On Friday 24 May 2024 13:29:41 GMT-3 Giuseppe D'Angelo via Development wrote: > Details: no user downgrades Qt and therefore has ever needed this. This > is something that only Qt developers themselves have possibly needed -- > seems to be a historical remnant at this point. > > Concretely, this means that we can make our lives easier; for instance, > when backporting fixes, we'll allow new symbols to appear in patch > releases. This is what prevents the 6.5.5-to-6.6.1 upgrade. So it makes some people's lives easier in one way by those people's and our users' more difficult in another. > Does this sound good? Yes. For me, there's no such thing as LTS anyway, so I don't care about anything 6.5 beyond 6.5.3. This policy doesn't affect *me*. -- Thiago Macieira - thiago.macieira (AT) intel.com Principal Engineer - Intel DCAI Fleet Engineering and Quality smime.p7s Description: S/MIME cryptographic signature -- Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
