Daniel P. Berrangé <berra...@redhat.com> writes:
> On Fri, Feb 17, 2023 at 01:41:50PM +0100, Paolo Bonzini wrote:
[...]
>> This proposed update to the support policy chooses the last of these
>> possibilities. It does by modifying two aspects of the support policy:
>>
>> * it introduces different support periods for *native* vs. *non-native*
>> dependencies. Non-native dependencies are currently Python ones only,
>> and for simplicity the policy only mentions Python; however, the concept
>> generalizes to other languages with a well-known upstream package
>> manager, that users of older distributions can fetch dependencies from;
>
> I guess this would mean its applicable to perl/ruby/etc, but really
> we should be exclusively using Python for our scripting needs, except
> for legacy stuff we already have. So in practice non-native == python.
>
> Potentially if we want to use Rust, we'd want vendoring of deps needed
> by Rust code, and that's native deps arguably, rather than non-native.
>
> Still I think it is unlikely we would replace existing working code in
> qemu.git with a rust impl, while discarding the original impl. More
> likely we'd have a Rust impl in parallel, or use Rust only for a brand
> new feature, where we don't need to support all possible old platforms.
I'm not sure. However, there's no real need to anticipate right now how
we will use Rust; we can update our support policy when we know.
>> * it limits the support period for non-native dependencies to a fixed
>> amount of 4 years. This is intended to be close to the Python 5-year
>> lifecycle while accounting for the time between a distro's feature freeze
>> and the day it's released. This limit applies to all distro versions,
>> not just the previous one, in order to cater for the delay of SLE 16.
>>
>> The 4 year cutoff in practice means that QEMU will be able to drop Python
>> 3.6 support for QEMU 7.1 (RHEL8 becomes 4 year old next May, while SLE
>> is already over the threshold).
>
> So the 4 year timeframe enables us to move off 3.6, but it is still
> potentially limiting.
>
> That doesn't explicitly allow for the case where we decide we want
> to increae min version of flake8/pylint/mypy to one that is merly
> 1 year old. As noted, covering multiple versions of these linting
> tools is a burden, as the issues reported from new versions are
> not always a superset of the old version. So it is somewhat
> desirable to fixate on a specific release, or a narrow range of
> releases.
Concur.
> I guess we can argue that any optional build time components that
> only affect testing don't need to be constrained by our platform
> policy in the first, since they are optional and don't affect the
> ability to build QEMU. Still if I'm a distro maintainer, and
> backporting patches I like to run all static analysis tools to
> catch silly python mistake that happen when cherry-picking.
Yes, but you want to run the static analysis tools required by the QEMU
you're backporting to, not the one you're backporting from (unless
you're backporting the patches needed to make the newer linters happy,
and then bump the dependencies).
> Equally if I'm a contributor working on QEMU I would like to
> run all the static linting tests available.
This one is valid. Still, it's a "want", not a "must have". If it was
a hard "must have", then CentOS 8 support would be in trouble: no mypy
out of the box, as far as I can tell.
We can rely on CI to find the lint. Delays the feedback, which not
ideal, but also not unlike how we use CI to keep the build working on
all hosts (because not every developer has access to all hosts) and in a
multitude of configurations (requiring all developers to test all
configurations at all times would be impractical, so we don't).
Let's consider the special case mypy. Right now, you have to run it
manually, but John has been working on integrating it into the build
process. Now compare two extremes:
(A) Commit to keeping mypy happy for all versions of mypy provided by
any supported build host
The build runs mypy if the build host provides it. Immediate
feedback then, but only from a single mypy version.
Developers still have to run all the mypy versions somehow, or else
punt to maintainers, who may in turn punt to CI. Delayed feedback
from all mypy versions but one.
We operate with the intersection of mypy features and the union of
mypy bugs, which is bound to complicate the typing job, and may well
limit typing power.
(B) Pick a single mypy version
Developers have to run this mypy version, or else punt to
maintainers. We'll naturally pick a mypy version the maintainers
are happy to run, so no need for them to punt to CI. We can exploit
the full typing power of that mypy version. We don't have to work
around issues older versions may have.
Developers unable to run this mypy version locally get delayed
feedback. Affects the intersection of "people working on Python
code we check with mypy" and "people developing exclusively on older
hosts". If we further intersect with "people doing so on a somewhat
regular basis", I posit we get the empty set.
I think (A) is pretty much all downsides. I like (B), but could be too
rigid in practice. We could also
(C) Pick a narrow range range of mypy versions
> Implicit in the 4 year timeframe is that it is actually easy to
> cope with the fallout, because the long life distros support many
> newer python runtime versions. The only impact is on the need to
> use pip to grab extra pieces.
>
> So a different way of expressing this is to not limit ourselves
> by time. Instead declare that we reserve the right to bump the
> python runtime version, provided the new version is available
> from the disto as an optional alternative to the default python
> version.
>
> This would mean we would have had the option to adopt python
> 3.8 a year ago (or whenever it first became an option in RHEL
> /SLES). Today 4 years would suit our immediate needs, but John
> probably would have liked to have bumped the version last year
> already if it weren't for our support policy, and 4 year cut
> off wouldn't have helped last year.
Good points.
> Basically if it is acceptable to require a non-default python
> version, it should be ok to require that are more or less
> any time, as long as the version is available from the OS
> vendor officially.
>
>> Note that all "non-native" packages are currently build dependencies.
>> If in the future some non-native packages became runtime dependencies for
>> parts of QEMU, it would still be possible to choose any of the first
>> three possibilities for them.
>>
>> Another possible future change is to the way that these dependencies
>> have to be obtained by the person building QEMU. Right now they have to
>> run pip before the build; it may be desirable for configure to set up a
>> virtual environment and download them in the same way that it populates
>> git submodules. Just like with submodules, this would make things
>> easier for people that can afford accessing the network in their build
>> environment; the option to populate the build environment manually with
>> pip would remain for people whose build machines lack network access.
>> The change to the support policy neither requires nor forbids this change.
>>
>> [Thanks to Daniel P. Berrangé, Peter Maydell and others for discussions
>> that were copied or summarized in the above commit message]
>>
>> Cc: Markus Armbruster <arm...@redhat.com>
>> Cc: Daniel P. Berrangé <berra...@redhat.com>
>> Cc: Peter Maydell <peter.mayd...@linaro.org>
>> Cc: John Snow <js...@redhat.com>
>> Cc: Kevin Wolf <kw...@redhat.com>
>> Signed-off-by: Paolo Bonzini <pbonz...@redhat.com>
>> ---
>> docs/about/build-platforms.rst | 19 +++++++++++++++++++
>> 1 file changed, 19 insertions(+)
>>
>> diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
>> index 1c1e7b9e11c3..e1ea09789107 100644
>> --- a/docs/about/build-platforms.rst
>> +++ b/docs/about/build-platforms.rst
>> @@ -86,6 +86,25 @@ respective ports repository, while NetBSD will use the
>> pkgsrc repository.
>> For macOS, `Homebrew`_ will be used, although `MacPorts`_ is expected to
>> carry
>> similar versions.
>>
>> +Python build dependencies
>> +~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +The minimum supported version of Python is currently 3.6.
>> +
>> +Distributions with long-term support often provide multiple
>> +versions of the Python runtime. QEMU aims to support the default
>> +Python runtime for 4 years after the initial release of a new version.
>> +Afterwards, you may have to point QEMU to a newer version of the Python
>> +runtime using the ``--python`` command line option of the ``configure``
>> +script.
>
> My counter proposal is to remove the 4 year marker, and give us
> more flexibility:
>
> Distributions with long-term support often provide multiple
> versions of the Python runtime. QEMU will initially aim to
> support the default python runtime. QEMU reserves the right
> to increase its minimum version to any newer python that is
> available as an option from the vendor.
Sounds good to me.
>> +
>> +Some of QEMU's build dependencies are written in Python and available
>> +through the Python Package Index (PyPI). QEMU aims to be compatible
>> +with the versions packaged by common Linux distributions for the first
>> +4 years after the major release of the distribution. After 4 years,
>> +you may have to use ``pip`` to install some of these build dependencies.
>
> And for this I'd say
>
> If QEMU bumps its minimum python version to a non-default version,
> then it may be neccessary to fetch python modules from 'pip' to
> build or test QEMU.
Likewise.
