Re: [libvirt] Limiting old version back compat for language bindings ?
On Mon, 2019-10-07 at 16:29 +0100, Daniel P. Berrangé wrote: > On Mon, Oct 07, 2019 at 04:58:34PM +0200, Andrea Bolognani wrote: > > It seems to me that people who want to run the latest version of > > whatever application will also use a non-obsolete operating system, > > and conversely people stuck with an old OS will rather also stick to > > the vendor-provided (and -supported) versions of the various > > components rather than installing newer ones from source. > > > > It's basically the same argument we used to justify libvirt dropping > > support for old operating systems and old QEMU versions, and I think > > it still applies when you take it one layer up the stack. > > It is the difference between the OS infrastructure layer and > the application layer. Essentially what I'm saying is that > libvirt is part of the OS infrastructure, and the libvirt > language bindings are part of the application infrastructure. > > It is valid to deploy on an old OS with vendor supplied libvirt, > while still using brand new libvirt python/Go bundled with the > application, not using the OS vendor provided version. > > The latter is in fact the recommended approach for application > developers in RHEL these days. We ship libvirt-python in RHEL-8 > built against the system python for use by virt tools we ship > like virt-install, virt-manager. From a support POV 3rd party > application developers are not supposed to use system python, > instead they must pick a python module stream. If they're lucky > their python module is the same version as system python and > they can use the system libvirt-python, but in general they > are expected to ship libvirt-python themselves as part of their > application install. I expected that to happen for languages like Rust and Go, but I thought for Python the common behavior would be to either used the packaged version (in the RHEL 8 case perhaps from a stream rather than the system one, doesn't change much) or perhaps bring in the necessary dependency with pip, not bundle it. > > If anything, the higher you go up the stack and the more developers > > are okay with having tighter coupling between their application and > > libvirt/QEMU versions, so I'd say it actually applies even more to > > them. > > I don't believe you can make such a blanket assertion about tight > coupling. I've seen both from apps developing against a very > specific min libvirt version only, vs apps developing against a > range of libvirt versions. I don't know. The latest version of oVirt requires RHEL 7.7, virt-manager is Python 3 only these days so it would be quite a pain to get it running on RHEL 6, if at all possible... The libvirt support matrix is fairly reasonable IMHO, and if you're stuck with a base OS which is outside of it I think your best bet is to stick with older versions of everything else as well. At some point, even applications that try to support arbitrarily old operating systems will end up in a situation where they're not actually testing on them (we have no CI test for libvirt-python or libvirt-go on CentOS 6, for example) and issues will either be ignored or a massive pain for developers to address. With all that said, I don't actually do any work on bindings so my own take only matters so much :) -- Andrea Bolognani / Red Hat / Virtualization -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Limiting old version back compat for language bindings ?
On Mon, Oct 07, 2019 at 04:58:34PM +0200, Andrea Bolognani wrote: > On Mon, 2019-10-07 at 13:35 +0100, Daniel P. Berrangé wrote: > > On Mon, Oct 07, 2019 at 01:58:10PM +0200, Andrea Bolognani wrote: > > > Can't we follow the same policy as the main library? That would make > > > it more straightforward to reason about. Also note that our CI only > > > runs jobs on the platforms targeted by the main library, which means > > > RHEL 6 and Ubuntu 14.04 are out already... > > > > I don't think this is the same kind of situation at play, because of how > > it interacts with application developers expressing their dependancies > > for the language bindings. If an app expresses a dep on the oldest > > version of libvirt-python they support they can't use APIs newer than > > that. If an app expresses a dep on the newest version of libvirt-python > > they can use, then they can conditionally use the new APIs while still > > being compatible with the older ihnstalls. This works regardless of > > our support policy wrt the main libvirt EOL. > > > > If we put the same EOL policy on the language bindings, we're either > > forcing the application onto the same support policy as libvirt, or > > making their build / deployment process more complicated, neither of > > which I think are reasonable. > > > > With main libvirt library our EOL policy is a great benefit so us as > > it dramatically lowers our maint burden. This makes it worth the cost > > for people who might wish to deploy libvirt on older systems. > > > > The language bindings do not have a high maint cost from supporting > > old versions, so it doesn't justify creating pain for application > > developers by dropping support so aggressively as for main libvirt. > > > > A time based scheme for dropping old versions in language bindings > > is very easy to describe to people & apply ourselves, more so than > > our main policy which needs us to research versions across distros > > every time we change something. > > It seems to me that people who want to run the latest version of > whatever application will also use a non-obsolete operating system, > and conversely people stuck with an old OS will rather also stick to > the vendor-provided (and -supported) versions of the various > components rather than installing newer ones from source. > > It's basically the same argument we used to justify libvirt dropping > support for old operating systems and old QEMU versions, and I think > it still applies when you take it one layer up the stack. It is the difference between the OS infrastructure layer and the application layer. Essentially what I'm saying is that libvirt is part of the OS infrastructure, and the libvirt language bindings are part of the application infrastructure. It is valid to deploy on an old OS with vendor supplied libvirt, while still using brand new libvirt python/Go bundled with the application, not using the OS vendor provided version. The latter is in fact the recommended approach for application developers in RHEL these days. We ship libvirt-python in RHEL-8 built against the system python for use by virt tools we ship like virt-install, virt-manager. From a support POV 3rd party application developers are not supposed to use system python, instead they must pick a python module stream. If they're lucky their python module is the same version as system python and they can use the system libvirt-python, but in general they are expected to ship libvirt-python themselves as part of their application install. > If anything, the higher you go up the stack and the more developers > are okay with having tighter coupling between their application and > libvirt/QEMU versions, so I'd say it actually applies even more to > them. I don't believe you can make such a blanket assertion about tight coupling. I've seen both from apps developing against a very specific min libvirt version only, vs apps developing against a range of libvirt versions. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Limiting old version back compat for language bindings ?
On Mon, 2019-10-07 at 13:35 +0100, Daniel P. Berrangé wrote: > On Mon, Oct 07, 2019 at 01:58:10PM +0200, Andrea Bolognani wrote: > > Can't we follow the same policy as the main library? That would make > > it more straightforward to reason about. Also note that our CI only > > runs jobs on the platforms targeted by the main library, which means > > RHEL 6 and Ubuntu 14.04 are out already... > > I don't think this is the same kind of situation at play, because of how > it interacts with application developers expressing their dependancies > for the language bindings. If an app expresses a dep on the oldest > version of libvirt-python they support they can't use APIs newer than > that. If an app expresses a dep on the newest version of libvirt-python > they can use, then they can conditionally use the new APIs while still > being compatible with the older ihnstalls. This works regardless of > our support policy wrt the main libvirt EOL. > > If we put the same EOL policy on the language bindings, we're either > forcing the application onto the same support policy as libvirt, or > making their build / deployment process more complicated, neither of > which I think are reasonable. > > With main libvirt library our EOL policy is a great benefit so us as > it dramatically lowers our maint burden. This makes it worth the cost > for people who might wish to deploy libvirt on older systems. > > The language bindings do not have a high maint cost from supporting > old versions, so it doesn't justify creating pain for application > developers by dropping support so aggressively as for main libvirt. > > A time based scheme for dropping old versions in language bindings > is very easy to describe to people & apply ourselves, more so than > our main policy which needs us to research versions across distros > every time we change something. It seems to me that people who want to run the latest version of whatever application will also use a non-obsolete operating system, and conversely people stuck with an old OS will rather also stick to the vendor-provided (and -supported) versions of the various components rather than installing newer ones from source. It's basically the same argument we used to justify libvirt dropping support for old operating systems and old QEMU versions, and I think it still applies when you take it one layer up the stack. If anything, the higher you go up the stack and the more developers are okay with having tighter coupling between their application and libvirt/QEMU versions, so I'd say it actually applies even more to them. -- Andrea Bolognani / Red Hat / Virtualization -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Limiting old version back compat for language bindings ?
On Mon, Oct 07, 2019 at 01:58:10PM +0200, Andrea Bolognani wrote: > On Mon, 2019-10-07 at 12:13 +0100, Daniel P. Berrangé wrote: > > Given this is only low/moderate maint cost, I'm tempted to be quite > > generous to applications and say that in January each year, we purge > > support for versions older than 5 years. > > > > This would imply... > > > > - Jan 2020 - purge older than 1.2.12 (Jan 2015) (Drops Trusty) > > - Jan 2021 - purge older than 1.3.1 (Jan 2016) > > - Jan 2022 - purge older than 3.0.0 (Jan 2017) (Drops Xenial) > > - Jan 2023 - purge older than 4.0.0 (Jan 2018) > > - Jan 2024 - purge older than 5.0.0 (Jan 2019) (drops RHEL-7, Bionic) > > - Jan 2025 - purge older than 6.0.0 (Jan 2020) > > Can't we follow the same policy as the main library? That would make > it more straightforward to reason about. Also note that our CI only > runs jobs on the platforms targeted by the main library, which means > RHEL 6 and Ubuntu 14.04 are out already... I don't think this is the same kind of situation at play, because of how it interacts with application developers expressing their dependancies for the language bindings. If an app expresses a dep on the oldest version of libvirt-python they support they can't use APIs newer than that. If an app expresses a dep on the newest version of libvirt-python they can use, then they can conditionally use the new APIs while still being compatible with the older ihnstalls. This works regardless of our support policy wrt the main libvirt EOL. If we put the same EOL policy on the language bindings, we're either forcing the application onto the same support policy as libvirt, or making their build / deployment process more complicated, neither of which I think are reasonable. With main libvirt library our EOL policy is a great benefit so us as it dramatically lowers our maint burden. This makes it worth the cost for people who might wish to deploy libvirt on older systems. The language bindings do not have a high maint cost from supporting old versions, so it doesn't justify creating pain for application developers by dropping support so aggressively as for main libvirt. A time based scheme for dropping old versions in language bindings is very easy to describe to people & apply ourselves, more so than our main policy which needs us to research versions across distros every time we change something. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Limiting old version back compat for language bindings ?
On Mon, 2019-10-07 at 12:13 +0100, Daniel P. Berrangé wrote: > Given this is only low/moderate maint cost, I'm tempted to be quite > generous to applications and say that in January each year, we purge > support for versions older than 5 years. > > This would imply... > > - Jan 2020 - purge older than 1.2.12 (Jan 2015) (Drops Trusty) > - Jan 2021 - purge older than 1.3.1 (Jan 2016) > - Jan 2022 - purge older than 3.0.0 (Jan 2017) (Drops Xenial) > - Jan 2023 - purge older than 4.0.0 (Jan 2018) > - Jan 2024 - purge older than 5.0.0 (Jan 2019) (drops RHEL-7, Bionic) > - Jan 2025 - purge older than 6.0.0 (Jan 2020) Can't we follow the same policy as the main library? That would make it more straightforward to reason about. Also note that our CI only runs jobs on the platforms targeted by the main library, which means RHEL 6 and Ubuntu 14.04 are out already... -- Andrea Bolognani / Red Hat / Virtualization -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] Limiting old version back compat for language bindings ?
In at least the Python and Go bindings for libvirt we use conditional compilation to allow the bindings to be build against old versions of libvirt. For Python this goes back to 0.9.11, from Apr 2012 For Go this goes back to 1.2.0, from Dec 2013 I'm wondering whether it would be worthwhile to define some rule to set a historical maximum, beyond which we will drop conditional compilation, or whether we're ok letting it grow without bound. The conditional compilation of code has some maint cost, but the cost is not huge, so I don't think this is something we need to be too aggressive on. At the same time I'm sceptical anyone is using latest Python bindings with libvirt 0.9.11, or latest Go bindings with libvirt 1.2.0 The challenge with language bindings is that users will often not use the language binding provided by the host OS vendor, instead preferring to download & build themselves. IOW, the host OS can have libvirt 3.0.0, but the app will be blindly pulling latest python binding (4.8.0) from PyPI and expect it to work. Of course if they're on an old distro, they could just pull an older version of the binding. Having to write code to download different version of the binding code on each OS is costly though, and indeed not even supported by common build tools. eg in python requirements.txt you can allow it to pick the latest version, or you can set an explicit version. The problem comes if you want to build on an OS with version 3.0.0, but also want to be able to use APIs from 4.0.0 if its available. AFAIK, you can't express this with distutils/setuptools. The same issue arises with the way you express deps in Go modules. You can ask for a specific version, but can't say to use a different version on certain OS. The key question is thus how far back applications should reasonably expect us to support language bindings. LTS distros live for quite a long time & its not unreasonable for apps to target them. In RHEL we've tended to rebase libvirt frequently while that RHEL version was the latest. That means we have RHEL-6 - 0.10.2 RHEL-7 - 4.5.0 RHEL-8 - 4.5.0 Considering Ubuntu LTS which doesn't rebase we have Trusty 14.04 - 1.2.2 Xenial 16.04 - 1.3.1 Bionic 18.04 - 4.0.0 Given this is only low/moderate maint cost, I'm tempted to be quite generous to applications and say that in January each year, we purge support for versions older than 5 years. This would imply... - Jan 2020 - purge older than 1.2.12 (Jan 2015) (Drops Trusty) - Jan 2021 - purge older than 1.3.1 (Jan 2016) - Jan 2022 - purge older than 3.0.0 (Jan 2017) (Drops Xenial) - Jan 2023 - purge older than 4.0.0 (Jan 2018) - Jan 2024 - purge older than 5.0.0 (Jan 2019) (drops RHEL-7, Bionic) - Jan 2025 - purge older than 6.0.0 (Jan 2020) Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
