Re: Proposal: Let's update virtualenv in Rawhide (drops Python 3.6 support)
On Mon, Aug 26, 2024 at 01:08:35PM +0200, Miro Hrončok wrote: > On 26. 08. 24 12:58, Neal Gompa wrote: > > On Mon, Aug 26, 2024 at 6:11 AM Miro Hrončok wrote: > > > > > > Hello Pythonistas. > > > > > > The virtualenv version in Fedora is currently stuck at 20.21.1 (April > > > 2023). > > > > > > This is because version 20.22+ dropped support for Python 2.7 and 3.6 > > > virtual > > > environments. > > > > > > Now when Python 2.7 was retired via > > > https://fedoraproject.org/wiki/Changes/RetirePython2.7 the only > > > difference is > > > Python 3.6. > > > > > > If we upgrade virtualenv, developers targeting Python 3.6 (e.g. RHEL 8 > > > platform > > > Python) can use `python3.6 -m venv` to create Python 3.6 virtual > > > environments. > > > Developers using tox can follow > > > https://tox.wiki/en/4.18.0/faq.html#testing-end-of-life-python-versions > > > > > > I propose we bite the bullet and update virtualenv to the latest version > > > in > > > Fedora 42+. > > > > Why not do it for 41? The reason we didn't do it before was because of > > Python 2.7, which we dropped now. > > Because I think kit's too late in the F41 dev cycle to introduce such change > and I am willing to keep backporting fixes to old virtualenv in Fedora 41. > > The reason was both 2.7 and 3.6. And as of now, 2.7 is still in Fedora 41. +1 to doing this just in rawhide now. Zbyszek -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: The -O3 flag leaked into sysconfig CFLAGS, should we keep it?
On Wed, Aug 14, 2024 at 11:30:16AM +0200, Miro Hrončok wrote:
> On 14. 08. 24 11:04, Zbigniew Jędrzejewski-Szmek wrote:
> > On Wed, Aug 14, 2024 at 01:22:19AM +0200, Miro Hrončok wrote:
> > > On 13. 08. 24 21:45, Zbigniew Jędrzejewski-Szmek wrote:
> > > > On Tue, Aug 06, 2024 at 11:29:01AM +0200, Miro Hrončok wrote:
> > > > > On 06. 08. 24 1:20, Miro Hrončok wrote:
> > > > > > Hello Pythonistas.
> > > > > >
> > > > > > For years, the CFLAGS embedded in Python sysconfig contained -O2 in
> > > > > > Fedora. This was how Python was built and by default, all flags
> > > > > > used to
> > > > > > build Python were embedded.
> > > > > >
> > > > > > Later, the flag was removed in Fedora 39 via this change:
> > > > > > https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction
> > > > > >
> > > > > > We wanted to remove as many flags as possible, with this motivation:
> > > > > >
> > > > > > """
> > > > > > Python developers will get more upstream-like experience when
> > > > > > building
> > > > > > Python extension modules and also closer to building vanilla C
> > > > > > programs.
> > > > > > """
> > > > > >
> > > > > > Note that removing -O2 specifically was not the primary motivation,
> > > > > > but
> > > > > > the removal was intentional at that time.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Now we build Python 3.13 with -O3 via this change:
> > > > > > https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3
> > > > > >
> > > > > > The change proposal said:
> > > > > >
> > > > > > """
> > > > > > Other Python extension modules will remain bulidng as before, e.g.
> > > > > > in
> > > > > > RPM packages, they will still be built with -O2...
> > > > > > """
> > > > > >
> > > > > > However, I made a mistake and the -O3 flag leaked into sysconfig
> > > > > > CFLAGS.
> > > > > >
> > > > > > The good news is this does not seem to affect RPM packages, they are
> > > > > > still being built with -O2, like this:
> > > > > >
> > > > > > <$CFLAGS from automatic
> > > > > > %set_build_flags>
> > > > > >
> > > > > > E.g. ... -fcf-protection -fexceptions -O3 -O2 -flto=auto ...
> > > > > >
> > > > > > The latter flag wins.
> > > > > >
> > > > > >
> > > > > >
> > > > > > OTOH users building their own extension modules will get -O3.
> > > > > >
> > > > > > This is not what was intended. However, "Python developers will get
> > > > > > more
> > > > > > upstream-like experience" is more true now, because upstream Python
> > > > > > builds use -O3:
> > > > > >
> > > > > > $ podman run --rm -ti python:3.12 /usr/bin/bash
> > > > > > # python
> > > > > >>>> import sysconfig
> > > > > >>>> sysconfig.get_config_var('CFLAGS')
> > > > > > '-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall'
> > > > > >
> > > > > >
> > > > > > So the question is:
> > > > > >
> > > > > > Do we keep -O3 for user-built extension modules for speed and
> > > > > > upstream-like experience? (I would update the -O3 change proposal.)
> > > > > >
> > > > > > Or do we loose the flag, as currently documented?
> > > > > >
> > > > > > Alternatively, do something else entirely (e.g. embed -O2, or other
> > > > > > flag...)?
> > > > >
> > > > > I slept on it and I support keeping the -O3 flag in sysconfig CFLAGS.
> > > > >
> > > > > The original motivation for Python_Extension_Flags_Reduction was:
> > > > >
> > > > > &
Re: The -O3 flag leaked into sysconfig CFLAGS, should we keep it?
On Wed, Aug 14, 2024 at 01:22:19AM +0200, Miro Hrončok wrote:
> On 13. 08. 24 21:45, Zbigniew Jędrzejewski-Szmek wrote:
> > On Tue, Aug 06, 2024 at 11:29:01AM +0200, Miro Hrončok wrote:
> > > On 06. 08. 24 1:20, Miro Hrončok wrote:
> > > > Hello Pythonistas.
> > > >
> > > > For years, the CFLAGS embedded in Python sysconfig contained -O2 in
> > > > Fedora. This was how Python was built and by default, all flags used to
> > > > build Python were embedded.
> > > >
> > > > Later, the flag was removed in Fedora 39 via this change:
> > > > https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction
> > > >
> > > > We wanted to remove as many flags as possible, with this motivation:
> > > >
> > > > """
> > > > Python developers will get more upstream-like experience when building
> > > > Python extension modules and also closer to building vanilla C programs.
> > > > """
> > > >
> > > > Note that removing -O2 specifically was not the primary motivation, but
> > > > the removal was intentional at that time.
> > > >
> > > >
> > > >
> > > > Now we build Python 3.13 with -O3 via this change:
> > > > https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3
> > > >
> > > > The change proposal said:
> > > >
> > > > """
> > > > Other Python extension modules will remain bulidng as before, e.g. in
> > > > RPM packages, they will still be built with -O2...
> > > > """
> > > >
> > > > However, I made a mistake and the -O3 flag leaked into sysconfig CFLAGS.
> > > >
> > > > The good news is this does not seem to affect RPM packages, they are
> > > > still being built with -O2, like this:
> > > >
> > > > <$CFLAGS from automatic
> > > > %set_build_flags>
> > > >
> > > > E.g. ... -fcf-protection -fexceptions -O3 -O2 -flto=auto ...
> > > >
> > > > The latter flag wins.
> > > >
> > > >
> > > >
> > > > OTOH users building their own extension modules will get -O3.
> > > >
> > > > This is not what was intended. However, "Python developers will get more
> > > > upstream-like experience" is more true now, because upstream Python
> > > > builds use -O3:
> > > >
> > > > $ podman run --rm -ti python:3.12 /usr/bin/bash
> > > > # python
> > > > >>> import sysconfig
> > > > >>> sysconfig.get_config_var('CFLAGS')
> > > > '-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall'
> > > >
> > > >
> > > > So the question is:
> > > >
> > > > Do we keep -O3 for user-built extension modules for speed and
> > > > upstream-like experience? (I would update the -O3 change proposal.)
> > > >
> > > > Or do we loose the flag, as currently documented?
> > > >
> > > > Alternatively, do something else entirely (e.g. embed -O2, or other
> > > > flag...)?
> > >
> > > I slept on it and I support keeping the -O3 flag in sysconfig CFLAGS.
> > >
> > > The original motivation for Python_Extension_Flags_Reduction was:
> > >
> > > """
> > > Python developers will get more upstream-like experience when building
> > > Python extension modules and also closer to building vanilla C programs.
> > > """
> > >
> > > Keeping -O3 supports "more upstream-like experience".
> > > Getting rid of it supports "closer to building vanilla C programs".
> > >
> > > When choosing from the two, I prefer the first one.
> > >
> > > Every time somebody pip installs something without a wheel, or even builds
> > > their own extension module code, on CI etc. they would benefit from the
> > > added speed. If we want Fedora to succeed on the CI field, we should be
> > > competitive.
> >
> > I think the argument about pip is important: with -O3, the users will
> > get a "reasonable default". Local pip installs intended for computations
> > are certainly better with '-O2' or '-O3' rather than the compiler default
> > of -O0
Re: The -O3 flag leaked into sysconfig CFLAGS, should we keep it?
On Tue, Aug 06, 2024 at 11:29:01AM +0200, Miro Hrončok wrote:
> On 06. 08. 24 1:20, Miro Hrončok wrote:
> > Hello Pythonistas.
> >
> > For years, the CFLAGS embedded in Python sysconfig contained -O2 in
> > Fedora. This was how Python was built and by default, all flags used to
> > build Python were embedded.
> >
> > Later, the flag was removed in Fedora 39 via this change:
> > https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction
> >
> > We wanted to remove as many flags as possible, with this motivation:
> >
> > """
> > Python developers will get more upstream-like experience when building
> > Python extension modules and also closer to building vanilla C programs.
> > """
> >
> > Note that removing -O2 specifically was not the primary motivation, but
> > the removal was intentional at that time.
> >
> >
> >
> > Now we build Python 3.13 with -O3 via this change:
> > https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3
> >
> > The change proposal said:
> >
> > """
> > Other Python extension modules will remain bulidng as before, e.g. in
> > RPM packages, they will still be built with -O2...
> > """
> >
> > However, I made a mistake and the -O3 flag leaked into sysconfig CFLAGS.
> >
> > The good news is this does not seem to affect RPM packages, they are
> > still being built with -O2, like this:
> >
> > <$CFLAGS from automatic %set_build_flags>
> >
> > E.g. ... -fcf-protection -fexceptions -O3 -O2 -flto=auto ...
> >
> > The latter flag wins.
> >
> >
> >
> > OTOH users building their own extension modules will get -O3.
> >
> > This is not what was intended. However, "Python developers will get more
> > upstream-like experience" is more true now, because upstream Python
> > builds use -O3:
> >
> > $ podman run --rm -ti python:3.12 /usr/bin/bash
> > # python
> > >>> import sysconfig
> > >>> sysconfig.get_config_var('CFLAGS')
> > '-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall'
> >
> >
> > So the question is:
> >
> > Do we keep -O3 for user-built extension modules for speed and
> > upstream-like experience? (I would update the -O3 change proposal.)
> >
> > Or do we loose the flag, as currently documented?
> >
> > Alternatively, do something else entirely (e.g. embed -O2, or other
> > flag...)?
>
> I slept on it and I support keeping the -O3 flag in sysconfig CFLAGS.
>
> The original motivation for Python_Extension_Flags_Reduction was:
>
> """
> Python developers will get more upstream-like experience when building
> Python extension modules and also closer to building vanilla C programs.
> """
>
> Keeping -O3 supports "more upstream-like experience".
> Getting rid of it supports "closer to building vanilla C programs".
>
> When choosing from the two, I prefer the first one.
>
> Every time somebody pip installs something without a wheel, or even builds
> their own extension module code, on CI etc. they would benefit from the
> added speed. If we want Fedora to succeed on the CI field, we should be
> competitive.
I think the argument about pip is important: with -O3, the users will
get a "reasonable default". Local pip installs intended for computations
are certainly better with '-O2' or '-O3' rather than the compiler default
of -O0.
But then the setup becomes inconsistent. The flags are:
$ python -c "import sysconfig; print(sysconfig.get_config_var('CFLAGS'))"
-fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG
-fcf-protection -fexceptions -fcf-protection -fexceptions -fcf-protection
-fexceptions
-O3
I'd argue that the goal is to have flags that provide a reasonable
efficient default that integrates well with the cpython code provided
by the distribution,
-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer
should be included too. (Those flags allow whole-system profiling of
the system, and this works best if all of the system has frame pointers,
so the extensions should have them too.)
Also, can we get of rid of the repeats?
Zbyszek
--
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
Re: New to the list, keen to contribute!
On Thu, Jul 11, 2024 at 11:03:44AM -, Igor Jagec wrote: > Hey all, > > I'm Igor, a long-time Linux and middleware admin (Red Hat/Fedora user since > 2001!). I'm currently diving into Python and figuring it all out – always > been a big fan of learning by doing, so I thought this would be a great place > to get stuck in. > > Hoping to find my place within the Fedora community and contribute properly. > Any pointers on where I could start would be much appreciated. Welcome Igor! Most of our infrastructure and distro tooling is in Python, and there's always plenty to do. E.g. https://fedoraproject.org/wiki/Infrastructure/GettingStarted. But also fedpkg/rpkg/rust2rpm/koji/mock are all Python. Zbyszek -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Fedora 41 Python 3.13 rebuild in a side tag has now started
On Fri, Jun 07, 2024 at 08:48:03AM +0200, Karolina Surma wrote: > Hello, > > We have just started the Python 3.13 mass rebuild in the side tag for Fedora > 41 with Python 3.13.0b2. > > Please, follow the original instructions when planning to build your Python > packages. > We'll let you know when we'll plan to merge the side tag. > > If you'd like to build a package after we already rebuilt it, you should > > be able to build it in the side tag via: > > > > on branch rawhide: > > $ fedpkg build --target=f41-python I built rust-add-determinism-0.2.0-11.fc41 in the side tag. It drops the dependency on python3-libs and marshalparser and provides an internal reimplementation of marshalparser. Subsequent builds of python packages should have the pyc handler active. I sincerely hope this doesn't interrupt the rebuild in any way! Zbyszek -- ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: 16 packages still need a Python 3.12 rebuild, final freeze in 6 days
On Wed, Sep 27, 2023 at 11:56:01AM +0200, Miro Hrončok wrote: > Hello packagers, > here is the list of packages that still need a Python 3.12 rebuild for Fedora > 39+. > zbyszekpython-igor I retired it now in f39 and rawhide. The first attempt failed halfway because I didn't have a valid token. I repeated the 'fedpkg retire …' command, but I'm not sure if the state got updated correctly. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: JupyterLab finally in Fedora and the future of Jupyter Notebook
On Sun, Mar 05, 2023 at 04:19:41PM +0100, Miro Hrončok wrote: > On 05. 03. 23 12:32, Lumír Balhar wrote: > > Fedora 39 now contains JupyterLab 4.0.0 alpha 34 which allows me to > > update Jupyter Notebook to 7.0.0 alpha 14. ... > > Any help testing the prereleases is appreciated. > > Thank you, Lumír, for doing this! +1 Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Making PyPy 3.9 the "main" pypy3 for Fedora 36?
On Tue, Mar 01, 2022 at 05:44:56PM +0100, Miro Hrončok wrote: > Hello Pythonistas. > > Do you think we should make /usr/bin/pypy3 PyPy 3.9 in Fedora 36? It is > rather late in the release cycle, but considering this is essentially a leaf > package I think this might be a good idea. +1 Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: package request: PyWaffle
In general, I agree that the docs could use a short "tl;dr — the common case"
section at the top. This also applies to the "simplified" versioning guidelines
with full version info the the Version field. I wrote a chunk of the latest
text and I'm having trouble finding the right paragraph in all that prose.
The descriptions that we have are correct, but just not very easy to
consume casually. This is part of a bigger problem: docs have regularly
been coming up in various surveys as the number one problem in Fedora.
IMO, one of the causes is that we've been keeping the descriptions of
older workflows. In case of Versioning it's because some ancient RHEL baggage
doesn't support the new stuff, and also because some people just *like*
the old approach. In case of Python macros, it's because not all packages
can use the new build systems, etc. But I think this isn't working, and we
should
just settle on the best version and prune the rest or move to appendices.
People can look at archived versions of the docs if they want to do
technoarcheology.
Right now, rpmdevtools recommends 'rm -rf $RPM_BUILD_ROOT' and outdated python
macros. Almost no guides have been updated for rpmautospec. (Though thankfully
at least fedora-review has been fixed not to warn about %autorelease.) Etc, etc.
Experienced packagers know what to ignore, but new packagers are very likely
to fall into the numerous traps we set out for them.
On Sun, Dec 19, 2021 at 02:26:15PM -0500, Matthew Miller wrote:
> 2. You mentioned in the review request — and Zbyszek said something similar
>— that it'd be better to repeat the package name throughout the spec
> file in the various place it's needed. That doesn't feel right to me.
>
> I know the macros replace a lot of boilerplate, which is a good direction,
> but having the name be different in every file _throughout_ the file makes
> it harder to pick up when something is different from standard. It seems
> that a lot of python packages could be identical _except_ for the header.
> (And maybe then all of that eventually hidden completely with just some
> thing near the top that indicates "standard python package".)
I find 'pywaffle' easier to read that '%{pypi_name}'. When I see e.g.
'%{_bindir}/%{pypi_name}',
I *can* do the substitutions in my head and arrive at '/usr/bin/pywaffle', but
it's certainly a bit of an overhead.
Using macros for things that *change* is reasonable: we certainly wouldn't want
to search&replace the version string when it changes during version upgrades,
and we can't hardcode the library path, because it changes between
architectures.
But for things that *never change* during the lifetime of the package,
what is the point of using macros? When you create a package for a different
python module, you're going to start afresh. The spec file should be as readable
as possible for _that particular package_. This also means not using unnecessary
macros and making the spec file as short as is reasonable.
> It seems that a lot of python packages could be identical _except_ for the
> header.
Not really. There's still a lot of variation between packages: even Python
packages have different build systems, various workarounds and required, etc.
The spec files have been becoming shorter, but right now it isn't practical
to completely hide all the details. If the technology improves, _and_ upstreams
clean up their metadata, maybe we'll be able to handle common Python packaging
scenarios with a one-line boilerplate.
But right now we're not there yet. In your spec file, we have 'pywaffle',
'%{pypi_name}', and '%{githubname}' used in various places. When reading the
spec file, I need to jump up to the header to check the definition of the macros
to remember which one is which.
(There's also the tiny annoyance that I can't ctrl-click on
https://github.com/gyli/%{githubname} to open the repo, I need to do
the substitution… Firefox doesn't understand the macro either.)
> 3. The machinery to make every package be python-... while generating an
>actual python3-... subpackage is awkward. Are we stuck with that forever?
Yes, I think we'll have this for the foreseeable future. On EPEL, you can
have multiple python versions, so we *need* the subpackages there.
Maybe we'll be able to hide the binary package name once we have package
generators (https://github.com/rpm-software-management/rpm/issues/329).
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: Introduction
On Sat, Sep 04, 2021 at 11:41:01PM -0700, Gordon Messmer wrote: > Hello, Python SIG. I've been meaning to join, but I've been busy > for a while in pursuit of a new job (which I will start in about a > week). > > I'm a maintainer of the pyp2rpm application, and have done some > other work to improve translation of Python dependencies to rpm. Hi Gordon, welcome to the fold! (and congrats on the job ;)) Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On Fri, Jun 11, 2021 at 03:24:28PM +0200, Petr Viktorin wrote: > I've proposed the new guidelines as a Fedora change: > https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x > The Package-x-generic-16.pngpython3-devel package, a mandatory > build-time requirement for all packages that use Python, will > require Package-x-generic-16.pngpyproject-rpm-macros, helpers > designed for the new guidelines. What about making this conditional: Requires: (pyproject-rpm-macros if rpm-build) You can't really *use* the macros without rpmbuild, and one less hard dependency is always good. With the conditional dep, pretty much any real system will get the dep, but it's easier to build a leaner custom containers and such. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: Draft of New Python Packaging Guidelines
On Fri, Jun 11, 2021 at 03:24:28PM +0200, Petr Viktorin wrote:
> I've proposed the new guidelines as a Fedora change:
> https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x
Some comments:
> For example, if a user runs pip install requests[security]
Please quote 'requests[security]' in the command. [] are special to the shell.
> You MUST NOT disable the entire testsuite or ignore the result to solve a
> build failure.
I understand the sentiment, but I think this should be SHOULD NOT.
There are some projects where the testsuite it an unmanagable mess
of failures that change in each release. Saying MUST NOT will just mean
that more packages will be violating the guidelines.
> BuildRequires: %{py3_dist pytest}
Does it make sense to recommend py3_dist here? python3dist(pytest) is
not more complex but can be fed to 'dnf install' directly, so in the
end it's more flexible. I always find the macro more obfuscation than
anything for package names which don't require normalization.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: Plan to deprecate pytz in Fedora 35+
On Thu, Feb 18, 2021 at 05:39:30PM +0100, Miro Hrončok wrote: > On 18. 02. 21 17:16, Felix Schwarz wrote: > >Hi Miro, > > > >I can understand the desire to remove pytz and I have to mention > >that I don't have any specific upstream feedback. > > > >However certbot/babel do support pretty old versions of Python. > >Adding a separate code path for Python 3.x+ might be a pretty > >tough sell and might contribute to the "let's advertise snaps only > >for distribution" mindset. I think there shouldn't be resistance from certbot to drop the dependency. certbot should be deployable in really minimal environments, so I expect people would be happy to drop the dependency. Zbyszek > Hi Felix. > > Disclaimer: We are not removing pytz as long as there are packages > that depend on it. > > For babel, I follow this: https://github.com/python-babel/babel/issues/716 > > Since we have marked the pytz package unwanted for ELN and we have > babel in ELN (trough sphinx), we'll most likely try to help there > (or get rid of babel from ELN's sphinx). > > BTW There's backports.zoneinfo for Python 3.6+, so we'll see how > long babel needs to support Python 2. ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: My Introduction
On Sun, Sep 27, 2020 at 09:13:54AM +, Antariksha Verma (Student Gurgaon) wrote: > Hey, I am a new member of the Fedora family and have just joined the Python > SIG mailing list. My name is Antariksh Verma and I am 14 years old. I am > excited to work with and contribute to the Fedora family. Hi and welcome to Fedora! Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Draft of New Python Packaging Guidelines
On Tue, May 05, 2020 at 08:04:35PM +0200, Petr Viktorin wrote: > Zbyszek said: > >Please don't advertise macros like %distname, %srcname, %origname, and > >the countless other variants in use. They made sense when people used > > >editors which didn't support search&replace easily. Having the name > >verbatim in there makes the whole thing easier to read and copy&paste. > > The idea was to standardize these, so we'd get well known names > instead of each packager defining their own. Then, macros like > %pypi_url could use the well-known name by default. This would also > take care of the subtle differences between names: in Django's case, > you need to use capital D in the URL, but automatic provides use the > canonicalized lowercase. Even search/replace is likely to miss these > (but, inevitably packagers themselves will also be confused :/) > > It's definitely possible to get rid of %py_set_name, and deprecate > %pypi_url's weird default -- you'd be expected to write out > `%pypi_url Django` every time. > I'm not sure if that would prevent people from using > %distname/%srcname/%origname/%pypiname, but it looks like the right > thing to do. %pypi_url Django sounds reasonable. This is nice an explicit, and if pypi returns 404, it is obvious what needs to change. Better then figuring out some cryptic macro to change. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Current plan: Build python3, python3-libs etc. from python39 SRPM on F33+
On Tue, Mar 24, 2020 at 04:12:29PM +, José Abílio Matos wrote: > On Tuesday, 24 March 2020 12.40.24 WET Miro Hrončok wrote: > > We would like ro rename the "python3" component (SRPM) to "python39" to > > make maintaining various Python versions in various Fedora versions > > easier. The names of binary RPMs would be unchanged; you still do `dnf > > install python3`. > > > > The change should only affect packagers of Python itself; not users or > > packagers of software written in Python. > > The only user-visible or general-packager-visible change is a different > > component (SRPM) name. > > That seems like a good idea. What about bugzilla? Will it follow the renaming? > That means that we need to know the python version (not difficult I admit) to > know the component where to submit the bug report. That's a good point. Maybe it'd be useful to set up a periodic job to move any new bugs from python to pythonNN as appropriate for any given release. > What about the upgrade path? Will it be preserved. > Case: assume that I install python39 for F33. Then in F35 will it upgrade all > the python39* to python310*? Or will the python39 remain installed? The binary path only cares about binary package names, so it is unaffected by this change. > There is some work to be done here although IMHO I agree that it is worth. :-) Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Current plan: Build python3, python3-libs etc. from python39 SRPM on F33+
On Tue, Mar 24, 2020 at 01:40:24PM +0100, Miro Hrončok wrote: > Hello Pythonistas. > > (I've CC'ed the devel list for further exposure. But let's discuss > this on python-devel list please to avoid noise.) > > > We would like ro rename the "python3" component (SRPM) to > "python39" to make maintaining various Python versions in various > Fedora versions easier. > The names of binary RPMs would be unchanged; you still do `dnf install > python3`. +1, sounds very reasonable. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: RFC: Python minimization in Fedora
On Thu, Jan 16, 2020 at 03:36:11PM +0100, Nicolas Mailhot via devel wrote: > Le 2020-01-16 15:10, Felix Schwarz a écrit : > >Am 16.01.20 um 13:37 schrieb Nicolas Mailhot via devel: > >>If we start messing with the Python tree it would be nice to put > >>each shared > >>python component in a separate zip/xz/whatever, and allow > >>versioning those > >>archives > >> > >>(ie use the highest semver zip present unless the code > >>explicitely requests > >>another version, and this version is available on the filesystem) > >> > >>That would heal the breach between venv users and Fedora/rpm. > >>We’re alienating > >>a lot of users, because un-versioned python components, do not > >>permit the > >>version divergence, some third party software requires > > > >Could you give a specific example? Even though my $DAYJOB is > >mostly about > >working with Python I don't have a clue which "un-versioned python > >components" > >you are referring to. > > Right now we (in Fedora) deploy things like > > /usr/lib/pythonxx/site-packages/something > > That means only one something may exist on-disk at a given time. > Python users workaround this with venvs and blame rpm and Fedora for > making a single something possible. > > Accommodating component versioning would mean deploying > > /usr/lib/pythonxx/site-packages/something-semver.zip This path includes xx, which contains the major and minor numbers. So adding "semver" would only allow accommodating different patch levels. Would that be useful? Different patch levels are supposed to be about bug fix only changes, so there's usually very little reason to carry anything except the latest one for any specific major.minor combination. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: RFC: Python minimization in Fedora
On Wed, Jan 15, 2020 at 06:05:42PM +0100, Miro Hrončok wrote: > ### File types (and bytecode caches) > > The orthogonal dimension is the file type. Python standard library > contains directories with both "extension modules" (written in C > (usually) and compiled to `*.cpython-38-x86_64-linux-gnu.so` shared > object file) and "pure Python" modules (written in Python and saved > as `*.py` source file). > > Each pure Python module comes in 4 files: > > - `module.py` -- the source > - `__pycache__/module.cpython-38.pyc` -- regular (not optimized) bytecode > cache > - `__pycache__/module.cpython-38.opt-1.pyc` -- optimized bytecode cache > (level 1) > - `__pycache__/module.cpython-38.opt-2.pyc` -- optimized bytecode cache > (level 2) I suspect that the difference in speed between loading various .pyc files is negligible. Do you have actual benchmarks for this? > ### Solution 5: Stop shipping mandatory bytecode cache > > This solution sounds simple: We do no longer ship the bytecode cache > mandatorily. Technically, we move the `.pyc` files to a subpackage > of `python3-libs` (or three different subpackages, that is not > important here). And we only *Recommend* them from `python3-libs` -- > by default, the users get them, but for space critical Fedora > flavors (such as container images) the maintainers can opt-out and > so can the powerusers. > > This would **save 18.6 MiB / 50%** -- quite a lot. > > However, as said earlier, if the bytecode cache files are not there, > Python attempts to create them upon first import. That can result in > several problems, here we will try to propose how to workaround > them. Below using a flag file in each __pycache__ directory is suggested. What about a different route: having a flag file for all descendants of a directory? For example, /usr/lib/python3.8/.dont_write_bytecode would cover all modules under /usr/lib/python3.8/. If a .pyc file is present, python could still make use of it. This would be a nicer solution because it wouldn't require modifying individual packages, but would still avoid the selinux issues and slowdowns from failed attempts to write the optimized files. The __pycache__ files wouldn't need to exist at all. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: update to spyder4
On Sun, Dec 22, 2019 at 08:33:00AM -0500, Mukundan Ragavan wrote: > I am looking to update Spyder-IDE to version 4.x [0] on rawhide and ran > into some issues. Spyder 4 introduces a few new dependencies are not yet > available in Fedora or have been retired. These dependencies are > qdarksuite, diff-match-patch and python-language-server. > > qdarksuite is not necessarily a problem. It depends on qtsass and > helpdev. These three are fairly easily packaged [1]. > > diff-patch-match is already present in Fedora but has not been updated > for sometime by the looks of it. Although I have not tested if the > latest upstream version builds, I suspect this should not be a problem. > > python-language-server (pyls), on the other hand, may be a problem. pyls > depends on several packages including jsonrpc-server, versioneer and > backports.functools_lru_cache in the simple configuration. The first two > packages are straightforward to build [1]. However, lru_cache seems to > python2 only as far as I can tell. Furthermore, it was recently retired > in Fedora for the exact same reason [2]. functools.lru_cache() is provided by the standard library in python3. We have diff-match-patch from 2012, latest version is 2018. I'd say it's ripe for an update. > How do I proceed? Can the SIG give me some pointers on how I can move > forward? Anyone has experience with packaging Pyls? I'd just go ahead with updating everything to the latest versions. Feel free to put me in CC on any package reviews or PRs, I'll be happy to review them. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Let's update pytest to 4.x
On Wed, Apr 03, 2019 at 09:28:47PM +0200, Miro Hrončok wrote: > On 03. 04. 19 20:47, Robert-André Mauchin wrote: > >On Wednesday, 3 April 2019 20:20:58 CEST Miro Hrončok wrote: > >>Hi. > >> > >>I'd like to update pytest as soon as possible to 4.3 (or 4.4 if agreed > >>upon). > >>This update will bring a lot of small breakage (long deprecated stuff got > >>removed), but it is needed for Python 3.8 support. And I'd rather deal with > >>this > > breakage now than firefighting it in the 3.8 side tag in May/June. > >>Thoughts? I think it's better to update. Stuff that may fail to build will need to be fixed, but delaying this is just going to make the pain more acute. We're still early in the F31 cycle, and such breakage is more appropriate now than later. Zbyszek > >> > >>Relevant bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1671167 > >>Relevant PR: https://src.fedoraproject.org/rpms/pytest/pull-request/8 > >> > >>-- > >>Miro Hrončok > >>-- > >>Phone: +420777974800 > >>IRC: mhroncok > > > >How many of these small breakages do you expect? > > The amount of bugs blocking this at this point (incl. CLOSED, > excluding pacage reviews is 7: > > https://bugzilla.redhat.com/showdependencytree.cgi?id=1671167&hide_resolved=0 > > I've built cca 250 packages in my Python 3.8 + pytest 4 Copr. > Cca 90 from them use pytest (based on a quick dirty grep). > > This is not a very good statistics but let's say that if we predict > this to 3662 Python packages tracked on portingdb and/or 440 > packages buildrequiring pytest, we can except 35-100 failed builds. > > >Will they mainly affect old unmaintained packages? > > No, this already affects e.g. pip. > Problem is that a lot of upstreams pin pytest to a specific version. > > >Will these breakages be easy to solve in your opinion? > > Half of the problems so far was trivial (e.g. renaming a setup.cfg section). > Others might be harder. I can offer help. Worst case, tests can be > temporarily skipped. > > > -- > Miro Hrončok > -- > Phone: +420777974800 > IRC: mhroncok > ___ > python-devel mailing list -- python-devel@lists.fedoraproject.org > To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: sphinx-build conf.py circular dependency
On Fri, Feb 15, 2019 at 06:19:10PM +0100, Miro Hrončok wrote:
> On 15. 02. 19 17:57, Gabriel L. Somlo wrote:
> >I'm trying to package 'migen':
> >
> >http://www.contrib.andrew.cmu.edu/~somlo/python-migen-rpm/python-migen.spec
> >
> >The sources come with a ./doc directory that uses sphinx-build to
> >generate documentation, and a ./doc/conf.py file which imports
> >the package being built, in order to automatically paste various
> >code and doc bits from the module into the resulting documentation
> >...
> >make: Entering directory
> >'/builddir/build/BUILD/migen-afe4405becdbc76539f0195c319367187012b05e/doc'
> >sphinx-build -b man -d _build/doctrees . _build/man
> >Running Sphinx v1.7.5
> >make: Leaving directory
> >'/builddir/build/BUILD/migen-afe4405becdbc76539f0195c319367187012b05e/doc'
> >
> >Configuration error:
> >There is a programable error in your configuration file:
> >
> >Traceback (most recent call last):
> > ...
> >pkg_resources.DistributionNotFound: The 'migen' distribution was not found
> >and is required by the application
>
>
> This is tricky. It doesn't import it (that is easily tricked with
> PYTHONPATH), but uses pkg_resources to fetch data about installed
> packages.
>
> I believe the easiest way is to sed conf.py to replace
>
> pkg_resources.require("migen")[0].version
Argh, upstream is doing the typical overengineered crap. Do they realize
that they are embedding the identifier of the _previously_ installed
version in the docs?
Also, using make to call sphinx-build is so python-2.
The following diff works for me:
diff --git python-migen.spec python-migen.spec
index 9ad29027c8..8dc929a214 100644
--- python-migen.spec
+++ python-migen.spec
@@ -35,9 +35,12 @@ A Python toolbox for building complex digital hardware.
%prep
%autosetup -n %{srcname}-%{commit0}
+sed -r -i 's/(migen_version = ).*/\1"%{version}-%{release}"/' doc/conf.py
+
%build
%py3_build
-%make_build -C doc latexpdf man
+PYTHONPATH=. sphinx-build-3 -b latex doc/ _build/doc/
+PYTHONPATH=. sphinx-build-3 -b man doc/ _build/doc/
%install
%py3_install
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Introduction
On Sun, Nov 04, 2018 at 04:10:49AM -, Daniel Newkirk wrote: > Hi! > My name is Daniel, and I'd love to help out the Fedora project in some > manner. Been a Fedora user on and off since Fedora 1 (used RH 9 back in the > day), and I've been coding with Python for about a decade now--things like > bioinformatics software, analysis pipelines, and automation--and may be of > some use. There appear to be some python2 packages that still need some > packaging/porting to python3, but that may not be of use much longer? If > there is an area of particular need, and someone has a few spare moments to > point me in the right direction, that would be wonderful. I'm not very > experienced with packaging in Fedora yet, but I'm a quick learner and > self-motivated, so it should be relatively painless :). Hi, welcome to Fedora! The list of python2 packages to port over is long enough that this will still be a subject for the next few years [0]. Currently Fedora is shedding some leaf python2 packages [1], but there is still plenty of stuff to update. And apart from the stuff that is already in Fedora, there is Python software which could be added to Fedora in the future if it is updated for Python 3. There is a new initiative to add more scientific software to Fedora [2], something that might be interesting for you if you care about bioinformatics. [0] http://fedora.portingdb.xyz/ [1] https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal [2] https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/VXN4UBCNIL6BUCDRN55IGVK3IMO3HF6C/#3WFYAQ35Q7HAIESVNQYY3UE6JAOY7KON Nowadays it is possible/exptected/welcome to open pull requests under https://src.fedoraproject.org/ for the spec files of various packages. If you want to become a packager (either for some new packages or just to work on improving the existing ones) and are looking for sponsorship into the packagers group, let me know off-list. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Mass Python 2 packages Removal
> >>On Tue, Sep 18, 2018 at 03:14:41PM +0200, Miro Hrončok wrote: > >>>Is anyone willing to look into automating this? Write a spec > >>>depython2izer? Hi, I wrote something that mostly seems to work: https://pagure.io/pyrenamer/blob/master/f/depython2ize.py ../pyrenamer/depython2ize.py -b -d python-alchimia python-gzipstream \ python-ivi python-pdfkit python-simpleparse python-walkdir \ python-XStatic-Jasmine python-yourls ... If does: > >>> - removing python2- and python- BRs yes > >>> - removing %package -n python2 and it's metadata, description & %files yes > >>> - removing all %py2_build, %py2_install or their older manual variants yes > >>> - removing python2 tests from %check yes > >>> - (sometimes) removing %py3dir or python3 dir pushd/popd dance yes (by default, can be disabled with a switch) > >>> - (sometimes) removing with_python3 conditionals as they make no sense no The workflow is to first calls it with just -bd for review, and then with -bwn to actually commit. I tested it for packages from $(bugzilla query --blocked=1625773 -s NEW). For the ones that are called python-* it seems to work, and quite a lot of them even build fine. Sometimes manual fixups will be necessary (e.g. when the py2 directories or files are referenced from %build in non-trivial ways). For packages with generic names it doesn't work because python2 subpackage detection is broken. Should be relatively easy to fix. I'll work on that tomorrow, so please just ignore them for now. PTAL. Let me know what bits are missing or buggy. I'd also be happy to help with pushing some of those patches to dist-git and doing the builds, but for this we need to coordinate. Example patches: http://in.waw.pl/~zbyszek/fedora/depy2/. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Mass Python 2 packages Removal
On Tue, Sep 18, 2018 at 03:14:41PM +0200, Miro Hrončok wrote: > Is anyone willing to look into automating this? Write a spec depython2izer? > > This usually means: > > - removing python2- and python- BRs > - removing %package -n python2 and it's metadata, description & %files > - removing all %py2_build, %py2_install or their older manual variants > - removing python2 tests from %check > - (sometimes) removing %py3dir or python3 dir pushd/popd dance > - (sometimes) removing with_python3 conditionals as they make no sense > > IIRC Zbyszek said at Flock that he might work on that. Is that still > true? If not, shall I draft something? Yeah, I wanted to work on this. When do you need it? Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Python 2 Package Removal and when to use fedora-obsolete-packages
Hi, there's a misunderstanding, so I'll reply to the part of your email where I think we're in mostly in agreement first: On Thu, Sep 13, 2018 at 01:51:10PM -0500, Jason L Tibbitts III wrote: > >>>>> "ZJ" == Zbigniew Jędrzejewski-Szmek writes: > ZJ> With the policy of "obsolete from fedora-obsolete-packages > ZJ> sometimes" we'll always be playing whack-a-mole because with > ZJ> approx. 2800 subpackages becoming obsolete, it is absolutely > ZJ> guaranteed that some maintainers get it wrong. > > I would think that centralizing the obsoletes would make things better, > not worse. It's certainly better than "obsolete from some other random > package". And if we _know_ that there will be dependency problems (such > as the old python2 package itself having to be obsoleted) then there isn't > much of a question here, is there? The obsoletes would need to be > added. Yes, I think obsoleting from fedora-obsolete-packages is perfectly OK. My beef was with obsoleting being optional. I'd like the policy to be "python2- subpackage SHOULD be obsoleted from fedora-obsolete-packages.". > ZJ> IMHO a simple policy of "always obsolete" is the only thing that can > ZJ> work at this scale. > > What scale are you talking about? It's not clear to me if you're > disagreeing with the entire concept of the distribution being > conservative about removing packages from end-user systems, or if you > have an argument that all python2 packages will need to be obsoleted > regardless. If that's the case, then talking about the robustness of > the policy seems odd because you're not actually disagreeing with it. > You should instead simply make your argument so we can get on with > business. I'm disagreeing with the part of the policy that makes obsoleting conditional on some unspecified future problems. I'm convinced that this is not robust in the sense that not-obsoleted package will to many small problems for users during upgrades. > ZJ> Heh, I don't think the FPC policy is very robust. > > It's as robust as is reasonable to implement. > > When fedora-obsolete-packages was introduced, there was considerable > controversy over whether it is remotely acceptable to remove installed > packages from end user systems when those packages aren't causing actual > problems for anyone. The decision was made that they would be removed > only when they cause dependency issues, and that this would be limited > as much as possible to updates between Fedora releases. Ack. "between releases" is important. > So, that's fedora-obsolete-packages. If you think it should be changed, > feel free to bring it before FESCo and go through the discussion again. > Personally I agree with the original decision: We should not simply be > yanking software off of someone's system unless we simply have to do so > because the system cannot be updated otherwise. I understand the motivations for the policy, but to me the assumptions about the effect of various choices that guided the creation of that policy are not entirely valid anymore. The primary now-invalid assumption is that users can a) tell which packages have been abandoned by the distribution, b) make a choice whether to keep them or not. I'm pretty sure that with gnome-software (or any other graphical updater) all the user can say is "some packages are not being updated". With dnf, it's a bit better, but still pretty hard to figure out. > ZJ> We know from experience that when users see "80 packages cannot be > ZJ> upgraded and were skipped", they don't like it. > > Unless they are relying on those packages for something, of course. If > you've figured out how to tell that's the case, feel free to give > details. I would rather have an occasional message whenever possible > rather than breaking someone's setup, but that's just me. As I wrote before, and I don't think anybody disagrees, users learn about abandoned packages only when the package managers tells them that some *other* packages cannot be upgraded. It is not obvious which of the huge set of packages that dnf lists is the "real offender". Instead, when we obsolete packages from fedora-obsolete-packages, the message is clearer: "Removing python-foo, obsoleted by fedora-obsolete-packages". A power user can do 'dnf --exclude python-foo' and keep the old package if they wish. Normal users just get the package removed. This would only happen during Fedora version upgrades, where users are faces with changes anyway and learn to adapt to them. To summarize, I'd rather have users see "Removing python2-fo
Re: Python 2 Package Removal and when to use fedora-obsolete-packages
On Sun, Sep 09, 2018 at 11:14:57PM -0400, Scott Talbert wrote: > On Sun, 9 Sep 2018, Miro Hrončok wrote: > > >>>I'm thinking about removing some of my Python 2 subpackages (that > >>>are no longer used). However, I'm wondering about the case where > >>>the removed packages should be added to fedora-obsolete-packages. I > >>>don't _think_ the removal of my packages would cause any problems > >>>upon upgrade. > >>> > >>>Can someone provide an example of a situation where a removed > >>>package could cause a problem on upgrade (and should thus be added > >>>to fedora-obsolete-packages). > >> > >>Hi, > >> > >>you should always add the old subpackage to fedora-obsolete-packages. > > > >This is not the FPC's conclusion. You should only do it if it is > >problematic (as in your example 1 bellow). Heh, I don't think the FPC policy is very robust. We know from experience that when users see "80 packages cannot be upgraded and were skipped", they don't like it. I'm myself an experienced packager and I still hate it, because it takes quite a bit of time to figure out what exactly is the issue. Sometimes it's indeed an obsolete package, but sometimes maintainers screw up and there's a conflict between packages, sometimes it's a mirror issue [1]. With the policy of "obsolete from fedora-obsolete-packages sometimes" we'll always be playing whack-a-mole because with approx. 2800 subpackages becoming obsolete, it is absolutely guaranteed that some maintainers get it wrong. There's even 700+ non-noarch packages which are much more likely to cause problems. IMHO a simple policy of "always obsolete" is the only thing that can work at this scale. [1] https://pagure.io/releng/issue/7792 > >https://pagure.io/packaging-committee/issue/754#comment-512716 > > > >>There are two possibilities for how the subpackage affects upgrade: > >>1. if your subpackage depends on other packages, then the presence > >>of your subpackage can prevent the upgrade of those other packages. > >>Users might need to use --allowerasing to work around that, and we > >>don't want that, upgrades must be possible without --allowerasing. > > Okay, after thinking about this example, my package _would_ actually > have a problem on upgrade, so thanks for the example. > > >>2. if your subpackage does not depend on anything except python2, then > >>there is no immediate problem with upgrade (since python2 is staying > >>around for now), but the users of your package have an obsolete package > >>installed that is not being upgraded or supported. > > Actually, I think Zbyszek may be right that we need to worry about > this case. Say there is a python2 package that only depends on > python(abi) = 2.7 and this package is stopped from building. This > package will remain on user's systems. Then in Fedora 58, Python 2 > is finally removed. If the Python 2 packages are obsoleted, won't > this cause a problem with the original Python 2 subpackage? Yes, it will. Users will get a message from 'dnf upgrade' like "450 packages were not upgraded" (with a long list including python2 and the offending python2 subpackages). If users switch to "--best --allowerasing", the upgrade will go through, but at this scale users will not be able to tell if they are removing just obsolete packages or if in this list there are interspersed packages that they care about. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Python 2 Package Removal and when to use fedora-obsolete-packages
On Fri, Sep 07, 2018 at 09:49:40PM -0400, Scott Talbert wrote: > I'm thinking about removing some of my Python 2 subpackages (that > are no longer used). However, I'm wondering about the case where > the removed packages should be added to fedora-obsolete-packages. I > don't _think_ the removal of my packages would cause any problems > upon upgrade. > > Can someone provide an example of a situation where a removed > package could cause a problem on upgrade (and should thus be added > to fedora-obsolete-packages). Hi, you should always add the old subpackage to fedora-obsolete-packages. There are two possibilities for how the subpackage affects upgrade: 1. if your subpackage depends on other packages, then the presence of your subpackage can prevent the upgrade of those other packages. Users might need to use --allowerasing to work around that, and we don't want that, upgrades must be possible without --allowerasing. 2. if your subpackage does not depend on anything except python2, then there is no immediate problem with upgrade (since python2 is staying around for now), but the users of your package have an obsolete package installed that is not being upgraded or supported. Note that policy is that python2- subpackages should be removed before the release of any given Fedora version, the same as for normal packages. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Re: Getting rid of python3- backports packages
On Sat, Jun 30, 2018 at 06:33:22AM -0700, Toshio Kuratomi wrote: > On Sat, Jun 30, 2018, 5:59 AM Miro Hrončok wrote: > > > During the 3.7 boostrapping of the interstitial sequence of Python 3 > > packages, I have noticed it includes a lot of packages that are only > > intended as Python 3 stdlib backports for older Pythons. > > > > In general, I think this is a good effort however, it's not always > straightforward and not always about python3. There's also at least one > item on your list (simplejson) which should remain packaged in Fedora > because it is useful as more than forwards compatibility. > > Here's an example of how you might need to replace "import contextlib2": > > try: > import contextlib > except ImportError: > contextlib = False > else: > if not hasattr(contextlib, 'nullcontext'): > contextlib = False > if not contextlib: > import contextlib2 as contextlib That's a generally useful formula if support for diverse python versions is needed. But in Fedora we know that we have python3.7 (assuming that this effort would be directed at F29+), so just the above collapses to just a single import contextlib Not appropriate as an upstream patch of course, but good enough for Fedora. > > https://pypi.org/project/contextlib2/ is a backport of the standard > > library’s contextlib module to earlier Python versions > > > > https://pypi.org/project/linecache2/ is a backport of linecache to older > > supported Pythons. > > > > https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a > > backport of standard pathlib module which tracks the standard library > > module, so all the newest features of the standard pathlib can be used > > also on older Python versions. > > > > https://pypi.org/project/traceback2/ is a backport of traceback to older > > supported Pythons. > > > > https://pypi.org/project/unittest2/ is a backport of the new features > > added to the unittest testing framework in Python 2.7 and onwards. > > > > https://pypi.org/project/simplejson/ is the externally maintained > > development version of the json library included with Python 2.6 and > > Python 3.0, but maintains backwards compatibility with Python 2.5. > > > > https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function > > signature features from Python 3.3 > > > > https://pypi.org/project/mock/ is now part of the Python standard > > library, available as unittest.mock in Python 3.3 onwards. > > > > > > Now I see fairly no real reason to have python3-contextlib2, > > python3-linecache2 etc. packaged and maintained in Fedora. The only > > reason there is that other packages actually depend on them. > > > > I think we should actively change the packages not to depend on those > > and eventually get rid of them. This involves changing how upstreams are > > using those. Thoughts? I think it'd be good to start with the most egregious ones (e.g. the ones that "backport" features from python 3.3), and try to kill all dependencies and then the package itself. The nice thing about this problem it can be done piecemeal. > > $ whatrequires python3-mock > > copr-keygen > > module-build-service > > python3-ara-tests > > python3-certbot > > python3-certbot-dns-* > > python3-conda > > python3-congressclient-tests > > python3-keystoneclient-tests > > python3-kubernetes-tests > > python3-magnumclient-tests > > python3-music21 > > python3-netdiff > > python3-octaviaclient-tests > > python3-osc-lib-tests > > python3-oslo-cache-tests > > python3-oslo-log-tests > > python3-oslo-middleware-tests > > python3-oslo-policy-tests > > python3-oslo-rootwrap-tests > > python3-oslo-serialization-tests > > python3-oslo-service-tests > > python3-oslo-versionedobjects > > python3-oslo-vmware-tests > > python3-oslotest > > python3-pytest-spec > > python3-pywbem > > python3-sphinx > > python3-testfixtures-tests > > python3-testify > > python3-testinfra > > python3-tinyrpc-tests > > python3-vcrpy > > waiverdb > > > > ...320 source packages buildrequire this one! For the ones that have the most deps, maybe it would be worth to add all the Provides to python3-libs package? Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/Q7LXDJPQSER74GUIVXCM2OXUKI3R7EXV/
Re: [DONE] Mass package change (python2- binary package renaming)
On Thu, Apr 05, 2018 at 12:21:45AM +0100, Sérgio Basto wrote:
> On Sun, 2017-08-20 at 20:49 +0100, Sérgio Basto wrote:
> > On Sun, 2017-08-20 at 18:49 +, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Sun, Aug 20, 2017 at 05:09:04PM +0100, Sérgio Basto wrote:
> > > > On Sun, 2017-08-20 at 14:46 +0000, Zbigniew Jędrzejewski-Szmek
> > > > wrote:
> > > > > On Sun, Aug 20, 2017 at 05:16:54AM +0100, Sérgio Basto wrote:
> > > > > > On Sun, 2017-08-20 at 03:26 +, Zbigniew Jędrzejewski-
> > > > > > Szmek
> > > > > > wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I pushed the changes and rebuilt the packages today.
> > > > > >
> > > > > > Hi,
> > > > > > I got this [1] in log of wammu-0.43-1.fc28, so in [2] line 50
> > > > > > you
> > > > > > need
> > > > > > replace
> > > > > > Requires: %{name} = %{version}-%{release}
> > > > > > by
> > > > > > Requires: python2-%{name} = %{version}-%{release}
> > > > > >
> > > > > > isn't it ?
> > > > >
> > > > > Thanks. That's two bugs:
> > > > > - the Provides for the old name without %{_isa} is missing
> > > > >(Requires: %{name}%{_isa} is resolvable, but not Requires:
> > > > > %{name})
> > > > > - internal Requires should be updated to use the new name
> > > > >(using the old name does not break anything in general, but
> > > > > is
> > > > > inelegant).
>
> I found a 3rd bug , provides / obsoletes misses epoch
>
> %{version}-%{release} needs be replaced by :
> %{?epoch:%{epoch}:}%{version}-%{release}
>
> where packages have epoch , IIRC this happens with python-blivet
I looked at python-blivet, and it wasn't modified as part of my mass update,
_and_ it looks all fine to me wrt. to epoch handling.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 04:11:46PM +, Zbigniew Jędrzejewski-Szmek wrote:
> On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
> > On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
> > >Indeed, I'm using those python packages like a dinosaur ;)
> >
> > :D
> >
> > >What about adding conditionals like
> > >
> > >%if 0%{?rhel} > 7 || 0%{?fedora} > 31
> > ># Disable python2 build by default
> > >%bcond_with python2
> > >%else
> > >%bcond_without python2
> > >%endif
> > >
> > >starting now?
> >
> > I am not against that. However currently that number (31) is
> > somewhat artificial. it's up to the maintainer to choose if it's 28
> > or 32 (or anything in between). Should we somehow recommend a
> > specific Fedora release here? Why 31?
> Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
> up current biannual release schedule, we'll have 28 and 29 this year,
> 30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
> because we need python2 support for the lifetime of the release, not
> just at the start. So actually 29 which will be supported until
> 2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
> by python2 support. This right conditional would be:
>
> %if 0%{?rhel} > 7 || 0%{?fedora} > 28
>
> but that round around the corner. So it's not even necessary to add
> the conditional, as long as the patch to drop support is only added
> in rawhide, not F28.
>
> OK, so I withdraw my objections. Removing python2 support in rawhide
> is fine.
Pff, sorry I can't count. The right conditional would be
%if 0%{?rhel} > 7 || 0%{?fedora} > 29
so dropping support in rawhide right now would be premature. It seems
that adding a conditional like that is the way to go for now.
> > Should a Fedora Change for that release be crafted that says "Mass
> > python2 packages removal"?
>
> Yeah, I think it should be filed as a F29 change.
F30 might be more appropriate.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
> On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
> >Indeed, I'm using those python packages like a dinosaur ;)
>
> :D
>
> >What about adding conditionals like
> >
> >%if 0%{?rhel} > 7 || 0%{?fedora} > 31
> ># Disable python2 build by default
> >%bcond_with python2
> >%else
> >%bcond_without python2
> >%endif
> >
> >starting now?
>
> I am not against that. However currently that number (31) is
> somewhat artificial. it's up to the maintainer to choose if it's 28
> or 32 (or anything in between). Should we somehow recommend a
> specific Fedora release here? Why 31?
Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
up current biannual release schedule, we'll have 28 and 29 this year,
30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
because we need python2 support for the lifetime of the release, not
just at the start. So actually 29 which will be supported until
2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
by python2 support. This right conditional would be:
%if 0%{?rhel} > 7 || 0%{?fedora} > 28
but that round around the corner. So it's not even necessary to add
the conditional, as long as the patch to drop support is only added
in rawhide, not F28.
OK, so I withdraw my objections. Removing python2 support in rawhide
is fine.
> Should a Fedora Change for that release be crafted that says "Mass
> python2 packages removal"?
Yeah, I think it should be filed as a F29 change.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 12:23:06PM +0100, Miro Hrončok wrote:
> On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote:
> >to push the whole ecosystem. The proposed model of nipping python2 support
> >at the edges is the same thing, in reverse. First some leaf packages
> >are dropped, and then some somewhat more important packages, and then
> >suddenly it becomes hard to use python2 for anything "serious", even
> >though it's still "available". I think that's a bad way to proceed for
> >our users. Let them have a single moment where python2 support goes away.
> >Announce this moment at least a year in advance. Let them keep running
> >the last release before that for as long as they need.
>
> I think there is a fundamental difference of how I see python RPMs
> in Fedora and how you do. In my POV, purpose of pythonX-foo packages
> that only bring libraries is to provide dependency for an
> application that happens to be written in python, runs on pythonX
> and need the foo library. If there is no such application packaged
> in Fedora, I see no purpose of such package. (Except maybe if such
> app is packaged in another repository used by our users. That's a
> corner case and if the maintainer of pythonX-foo is aware of that,
> it can always be used as argument to keep pythonX-foo.)
Indeed, I'm using those python packages like a dinosaur ;)
But more seriously, I think the distinction between library and app in
the python world is neither clear nor particularly significant. Distro
packaging provides clear benefits in both cases: easy and quick
install, easy and quick uninstall, testing, provenance history,
license checks, general reliability.
So instead of losing packages, I'd like to see more automatic
packaging and updates of packages from pypi and other "forges".
We are making steps in that direction, but not quickly enough.
> We live in an era of pip (and rubygems, and cpan, and npm etc.).
> Trying to mirror those systems with RPM packages just because it's
> possible is tedious, incomplete and mostly pointless IMHO. An era
> when upstream projects were eager to get their tool into the distros
> has passed. Having pythonX-foo packaged where no other
> tool/app/service needs it is not beneficial (if foo is a library
> only thing, not a tool itself). I have this opinion even with
> python3 packages.
>
> Developers who wants to use python2 for whatever reasons (rational
> or emotional ones) can do that on Fedora. We encourage developers to
> use virtual environments and pip [1]. Removing leaf python2 packages
> does not block them here at all.
>
> > Let them have a single moment where python2 support goes away.
>
> Removing all python2 things at one point is impossible. We've tried
> to test this with Django [2] on a much smaller scale. (Tens instead
> of thousands packages.) It took us too much time and too much
> energy. At the end, too much packages were just retired because
> their maintainers are "dead". I cannot imagine doing this on current
> scale in 2020.
> We can just retire python2 then and see what breaks. Honestly,
> everything would break.
I don't think this is entirely comparable. Django20 was about updating
packages and switching support from python2 to python3. Just dropping
existing support for python2 is something much easier.
Nevertheless, I agree that this still is a very significant chunk of
work at this scale. So spreading this out makes a lot of sense, but
imho python2 subpackages don't need to go away, they can just be
conditionalized.
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Intent to orphan Python 2
On Tue, Mar 20, 2018 at 10:51:30AM +0100, Petr Viktorin wrote: > Hello, > Here is a message I want to post to devel-announce later. Let me > know if you see anything wrong (or would like to take over python2 > maintainership). > > --- > Intent to orphan Python 2 First a disclaimer: I'm all for moving to python3, so I'm not disagreeing with the general idea, but with the transition plan. > tl;dr: Unless someone steps up to Python 2 after 2020, we need to > start dropping python2 packages now. I think this is a contentious statement. "need" is the wrong word. "One of the ways to prepare for the declared EOS from upsteam is to" would be more accurate. And it's just one of the possible ways. Another would be prepare a conditional in all spec files and simply flip it over before a mass rebuild for F32 (or whatever). > Python 2.7 will reach end of upstream support on 1st of January, > 2020, after almost 10 years (!) of volunteer maintenance. > > Fedora still has more than 3000 packages depending on python2 – many > more than we can support without upstream help. This is also misleading. Those 3000 packages have different upstreams, and many of those upstreams will support python2 even after python-eol. > We (rightly) don't have the authority to say "please drop your > unneeded python2 subpackages, or let us drop them for you" [0]. > The next best thing we *can* say is: "if Fedora is to keep python2 > alive, we won't be the ones doing it – at least not at the current > magnitude". The way you propose is essentially a mirror of how python3 support was introduced. For a long time it was hard to use python3 for anything serious because there was _always_ one or two more unported dependencies. It took many years to reach this critical mass where everything "important" was available. This period was harsh on users, because python3 was nice and shiny but people were unable to switch and unable to _influence_ this state in any significant way, the way that it is hard to push the whole ecosystem. The proposed model of nipping python2 support at the edges is the same thing, in reverse. First some leaf packages are dropped, and then some somewhat more important packages, and then suddenly it becomes hard to use python2 for anything "serious", even though it's still "available". I think that's a bad way to proceed for our users. Let them have a single moment where python2 support goes away. Announce this moment at least a year in advance. Let them keep running the last release before that for as long as they need. If we want to prepare for the python2-eol, IMHO we should first finish all the steps that don't impact users: convert our tooling, make sure python2 is not used in build for anything where python3 can be used, rename packages, fix provides and requires, update she-bangs, add conditionals to make dropping python2 support easy. In particular, let's first finish Changes/Avoid_usr_bin_python_in_RPM_Build [1]. [1] https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Mass Pull Request filing for ambiguous Python 2 requirements
On Wed, Dec 06, 2017 at 12:19:24PM +0100, Iryna Shcherbina wrote: > Hi all, Hi, > It has been a while since the last discussion on Python packages > naming policy, i.e. using "python2" instead of just "python", in > preparation for making "python" mean Python 3. Thanks to zbyszek's > effort, most of the ambiguously named Python packages were renamed > in Rawhide. That has unblocked even more packages which use > "python-" prefixed names in requirements (which is banned by the > policy [0]). oh, can you wait a bit: I want to do a another batch of renaming. I started working on the renaming script to handle some more complicated cases, but I put it off for F27 release and then never got back to it. I'd try to wrap that up. This would mean that there'd a few more names to updated, and I think it'd be better to handle everything at the same time. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: [DONE] Mass package change (python2- binary package renaming)
On Sun, Aug 20, 2017 at 05:09:04PM +0100, Sérgio Basto wrote:
> On Sun, 2017-08-20 at 14:46 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > On Sun, Aug 20, 2017 at 05:16:54AM +0100, Sérgio Basto wrote:
> > > On Sun, 2017-08-20 at 03:26 +, Zbigniew Jędrzejewski-Szmek
> > > wrote:
> > > > Hi,
> > > >
> > > > I pushed the changes and rebuilt the packages today.
> > >
> > > Hi,
> > > I got this [1] in log of wammu-0.43-1.fc28, so in [2] line 50 you
> > > need
> > > replace
> > > Requires: %{name} = %{version}-%{release}
> > > by
> > > Requires: python2-%{name} = %{version}-%{release}
> > >
> > > isn't it ?
> >
> > Thanks. That's two bugs:
> > - the Provides for the old name without %{_isa} is missing
> > (Requires: %{name}%{_isa} is resolvable, but not Requires:
> > %{name})
> > - internal Requires should be updated to use the new name
> > (using the old name does not break anything in general, but is
> > inelegant).
>
> I still got Error: nothing provides python2-wxPython = 3.0.2.0-17.fc28
> needed by wxPython-devel-3.0.2.0-17.fc28.armv7hl
>
> Now it is because python2-wxpython != python2-wxPython , P is need to
> be a capital letter
Yikes. Building again.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: [DONE] Mass package change (python2- binary package renaming)
On Sun, Aug 20, 2017 at 05:16:54AM +0100, Sérgio Basto wrote:
> On Sun, 2017-08-20 at 03:26 +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > Hi,
> >
> > I pushed the changes and rebuilt the packages today.
>
> Hi,
> I got this [1] in log of wammu-0.43-1.fc28, so in [2] line 50 you need
> replace
> Requires: %{name} = %{version}-%{release}
> by
> Requires: python2-%{name} = %{version}-%{release}
>
> isn't it ?
Thanks. That's two bugs:
- the Provides for the old name without %{_isa} is missing
(Requires: %{name}%{_isa} is resolvable, but not Requires: %{name})
- internal Requires should be updated to use the new name
(using the old name does not break anything in general, but is inelegant).
I now applied a fix and started building about a hundred affected packages.
Things should be fixed in ~2h.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: [DONE] Mass package change (python2- binary package renaming)
Hi,
I pushed the changes and rebuilt the packages today.
In the process I found a significant bug:
for arch-full packages, Provides: %{oldname}%{_isa} = %{version}-%{release}
was generated, but Provides: %{oldname} = %{version}-%{release} is also
necessary. I added that in various packages and rebuilt them, but it's likely
that I missed some. I'll be looking into this tomorrow too, but if you
encounter an issue with some package, feel free to ping me so that I don't
miss it (or rebuild it yourself: the fix is trivial, just copy the Provides
line and remove %{?_isa}).
Some packages failed to build:
xmms2-0.8-46.fc28 No matching package to install:
'ecore-devel'
v8-6.2.91-2.fc28 s390x trouble
wiredtiger-2.9.0-5.fc28 core dump on arm64
python-urwid-1.3.0-11.fc28tests
python-soaplib-0.8.1-16.fc28 tests
python-pylons-1.0.1-9.fc28AttributeError:
'HTTPRequestEntityTooLarge' object has no attribute 'exception'
python-pyblock-0.53-15.fc28 error: comparison of constant '0'
with boolean expression is always false [-Werror=bool-compare]
python-openstackclient-3.2.0-4.fc28 tests
python-mwlib-0.15.14-10.fc28 tests
python-libasyncns-0.7.1-18.fc28 libasyncns.c:137:13: error:
'ns_t_zxfr' undeclared (first use in this function); did you mean 'ns_t_axfr'?
python-lettuce-0.2.20-6.fc28 tests
python-keyczar-0.71c-10.fc28 tests
python-httpretty-0.8.14-5.20161011git70af1f8.fc28 tests
python-gensim-0.10.0-12.fc28 IndexError: only integers, slices
(`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays
are valid indices
python-flask-silk-0.1.2-10.fc28 ImportError: cannot import name Module
python-gear-0.5.9-7.fc28
pkg_resources.ContextualVersionConflict: (extras 0.0.3
(/usr/lib/python2.7/site-packages), Requirement.parse('extras>=1.0.0'),
set(['testtools']))
python-djvulibre-0.3.9-11.fc28error: invalid application of
'sizeof' to incomplete type 'enum ddjvu_status_e'
python-django-mptt-0.8.6-5.fc28 tests
python-django-countries-4.0-4.fc28AttributeError: 'module' object has
no attribute 'BaseSettings'
python-blist-1.3.6-13.fc28tests on s390x
python-amqpclt-0.6-4.fc28 KeyError: u'No such transport:
amqplib. Did you mean amqp?'
mesos-0.23.0-0.4ce5475.fc28.9 configure: error: can not find mvn on
your path
libsvm-3.21-6.fc28%add_maven_depmap
JPP.tw.edu.ntu.csie-libsvm.pom tw.edu.ntu.csie/libsvm.jar
libkdtree++-0.7.0-17.fc28 ./kdtree++/kdtree.hpp:1200:28: error:
no match for 'operator<<'
gdal-2.1.4-6.fc28 configure: error: C compiler cannot
create executables
fts-rest-3.6.3-3.fc28 AttributeError: 'HTTPForbidden'
object has no attribute 'exception'
csound-6.03.2-16.fc28 cc1: error: -Wformat-security ignored
without -Wformat [-Werror=format-security]
bro-2.4.1-7.fc28 error: ambiguous overload for
'operator=='
cjdns-19.1-7.fc28 tests
atomic-reactor-1.6.23.2-3.fc28AttributeError: module 'docker' has
no attribute 'Client
gis-2.18.11-4.fc28error: conflicting declaration
'PyObject* sipExportedExceptions__core [3]'
'
I also added missing BR:/usr/bin/2to3 to a bunch of packages which
other failed to build. Those are OK now:
958814 python-tablib-0.11.5-3.fc28 2to3
958774 python-setproctitle-1.1.10-4.fc28 2to3
958707 python-pypng-0.0.18-8.fc282to3
958648 python-openopt-0.5625-7.fc28 2to3
958645 python-openoffice-0.1-0.23.20110209.fc28 2to3
958512 python-google-apputils-0.4.2-10.fc28 2to3
958362 python-beautifulsoup4-4.6.0-3.fc282to3
958209 brltty-5.5-9.fc28 2to3
vtk and vdsm are still building as I'm writing this.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Self-introduction
On Fri, Aug 11, 2017 at 02:13:16PM +0200, Miro Hrončok wrote: > On 11.8.2017 08:25, Elliott Sales de Andrade wrote: > >Hello, > > > >My name is Elliott, but you may know me as QuLogic on > >IRC/FAS/GitHub/etc. I am interested in Python 3 porting, > >scientific libraries, Jupyter, etc., some of which you can find in > >my coprs [1]. I am also an upstream developer on libraries such as > >Matplotlib and Cartopy. > > > >I have opened a few review requests for such things: > >* python-click-plugins: https://bugzilla.redhat.com/show_bug.cgi?id=1468873 > >* python-descartes: https://bugzilla.redhat.com/show_bug.cgi?id=1468872 > >* python-fiona: https://bugzilla.redhat.com/show_bug.cgi?id=1468877 > >* python-geopandas: https://bugzilla.redhat.com/show_bug.cgi?id=1468995 > > > >Any reviews on these packages would be greatly appreciated. I am > >also looking for a sponsor. I have left a comment referencing some > >informal review work I've done on all the above bugs. > > Hi Elliott, > I may be able to sponsor you, but I'm busy with Fedora 27 change > deadlines for now. Ping me in a week if you cannot find a sponsor by > then. Hi Elliott and Miro, I looked over QuLogic's review requests and reviews, and sponsored him. Elliott, welcome to Fedora. Have fun and use your powers for good ;) If you have any questions or issues feel free to write to bug me, I'll do my best to help. I approved three of your packages. I don't know if you have been following #fedora-devel and the mailing list closely, but we are in the process of switching to pagure as the frontend for spec files, and various issues with have been reported with the process of creating new packages. You probably should install the latest built fedrepo-req and ask on #fedora-devel if you encounter any issues. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
[UPDATE] Re: Mass package change (python2- binary package renaming)
I've done mock rebuilds for all the packages I plan to change (results
below), and I'm prepared to go through with the changes.
dreampie and tiled have been dropped.
(Those two packages are "applications". dreampie installs a python3
module, but no python3 binaries. For tiled the python module seems to
be a minor addition.)
Packages part of https://fedoraproject.org/wiki/Changes/Platform_Python_Stack
are also excluded (to avoid conflicts):
dbus-python, gpgme, libcomps, libdnf, librepo, pytest,
python-argcomplete, python-cffi, python-contextlib2, python-coverage,
python-cryptography, python-decorator, python-extras,
python-fixtures, python-funcsigs, python-hypothesis, python-idna,
python-iniparse, python-jinja2, python-jsonschema, python-keyring,
python-linecache2, python-markupsafe, python-mimeparse, python-mock,
python-nose, python-pbr, python-pexpect, python-pip, python-ply,
python-ptyprocess, python-py, python-pyasn1, python-pycparser,
python-pygpgme, python-pytest-timeout, python-rpm-macros,
python-SecretStorage, python-setuptools, python-setuptools_scm,
python-six, python-testtools, python-traceback2, python-unittest2,
python-wheel, pyxattr, pyxdg, rpm,
Packages that had build issues unrelated to the python2- renaming:
abiword/ ftbfs (empty debugsources list for -docs, looks like
generator bug.
I hope to get this fixed before pushing.)
atomic-reactor/ ftbfs (AttributeError: module 'docker' has no
attribute 'Client')
bro/ ftbfs (error: invalid use of incomplete type
'EVP_PKEY {aka struct evp_pkey_st}')
cjdns/ftbfs
csound/ ftbfs (also needs update to 6.03 → 6.09)
fts-rest/ ftbfs (AttributeError: 'HTTPForbidden' object has no
attribute 'exception')
libkdtree++/ ftbfs (C++11 stuff?)
libsvm/ ftbfs (%mvn_add_dep)
libvoikko/fixed and built
mesos/ftbfs
mlt/ fixed and built
python-amqpclt/ ftbfs
python-django-countries/ ftbfs
python-django-mptt/ ftbfs
python-djvulibre/ ftbfs
python-flask-silk/ftbfs
python-gear/ ftbfs
python-gensim/ftbfs
python-hwdata/fixed upstream
python-larch/ fixed and built
python-lettuce/ ftbfs
python-libasyncns/ftbfs
python-mwlib/ ftbfs
python-openstackclient/ ftbfs
python-pyblock/ ftbfs
python-pygal/ ftbfs
python-pylons/ftbfs
python-rhev/ fixed and built
python-urwid/ ftbfs
qgis/ ftbfs (error: conflicting declaration 'PyObject*
sipExportedExceptions__core [3]')
qpid-proton/ ftbfs (Make Error at
proton-c/bindings/perl/cmake_install.cmake:44 (file):
file INSTALL cannot find
"/builddir/build/BUILD/qpid-proton-0.17.0/proton-c/bindings/perl/libcproton_perl.so".)
vtk/ ftbfs (looks like a local issue with lack of memory,
might build in koji.
I hope to get this fixed before pushing.)
(ftbfs: doesn't build for unrelated reasons, I'll push the python2- changes
because they don't hurt,
fixed upstream: an independent fix in the meantime,
fixed and built: fixed and built in koji w/o the python2- renaming, I'll push
the python2- changes
at the same time as other packages.)
For some packages I also renamed the python3 subpackage manually,
and/or added %python_provides for it.
Updated patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
Zbyszek
On Tue, Aug 08, 2017 at 10:14:26PM +, Zbigniew Jędrzejewski-Szmek wrote:
> Hello Fedora Python package maintainers!
>
> This is an announcement of a mass package renaming:
> Python 2 binary packages will be renamed to python2-*.
>
> This will happen soon after the F27 branching on August 15th.
>
>
> Currently ~1330 source packages already generate a binary package with
> the python2- prefix, and 835 remain to be updated. The spec files for
> approximately 740 packages will be renamed, and 95 will be left for
> fixing by maintainers or proven packagers.
>
>
> At the end of this e-mail are two lists of maintainers and packages:
>
> List 1. for those packages which will be taken care of by the mass remaining
>Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
>
>Maintainers don't have to do anything.
>
> List 2. for the remaining packages
>
>Maintainers are encouraged to update packages manually.
>
>
> ### Details of the change ###
>
> The change is to ensure that as many as possible python packages which
> provide a version for python2 hav
Re: Naming policy for application vs library packages in Python
On Thu, Aug 10, 2017 at 11:46:42AM +0200, Petr Viktorin wrote:
> On 08/10/2017 02:49 AM, Ben Rosser wrote:
> >Hi,
> >
> >This is following up on a brief conversation I was part of in IRC
> >(#fedora-devel) earlier:
> >
> >I've never been quite sure how to name packages written in Python when
> >they are just applications. Many "applications" written in Python
> >still install themselves using distutils or setuptools, and so install
> >a "module" into the site_packages directory. By a strict
> >interpretation of our Python packaging guidelines [1] [2], python 3
> >"modules" MUST be packaged as "python3-name".
> >
> >Since technically any nontrivial Python application that can be
> >installed via distutils or setuptools likely installs a module, this
> >strict interpretation of the guidelines would mean that software that
> >happens to be written in Python but does not intend to provide a
> >library must be named with the python3-* prefix.
> >
> >I am not sure that this is intended (since there's definitely software
> >in the distribution written in Python not named with a python prefix).
> >If this is not intended, then it would be nice if we could reword the
> >guideline to be more clear.
> >
> >If it is intended, I'm prepared to argue that the current guideline is
> >too strict and should be revisited, both from a practical position
> >(users looking for software called "foobar" who many not care whether
> >or not it's written in Python will be confused if it is packaged under
> >the name "python-foobar" in Fedora), and from a philosophical one (we
> >don't put the "lib" prefix on every library package if upstream
> >doesn't use that as their name because we generally try to respect the
> >way upstream names their software).
>
>
> Hello,
>
> You're right that this is currently missing from the guidelines.
> Would you be interested in writing up a draft change?
>
>
> Python modules are a special case of "Addon Packages" [0]:
>
> > If a new package is considered an "addon" package that
> > enhances or adds a new functionality to an existing Fedora
> > package without being useful on its own, its name should >
> > reflect this fact.
>
> So, if you're not extending Python, but providing a standalone app,
> you don't need to use the python3- prefix.
>
>
> Specifically, the de-facto policy (which we should write down) is
> that if nothing else is expected to import the module, then don't
> use the python3- prefix. So, by not using the prefix, you're
> signaling that the Python module is an implementation detail and can
> be removed (or moved, for example to a different Python
> implementation) at any time.
This guidelines are just too vague, and I think this is the source of
many disagreements over naming. I think we should stop guessing, and
assume that if a package installs any importable module, something
might start using that module and behave accordingly.
I'd use the following rules:
1. If "primarily an application", use upstream name for both the srpm and main
binary rpm
2. Except for packages which are strongly python-version dependent, and
install executables for both python versions (sympy, sphinx, ipython, ...).
In this case use python2- and python3- prefixes for the binary
packages, and have one of those packages Provide the original name.
3. If "primarily a library", use python- prefix for srpm, and
python2-, python3- prefixes for the binary packages
4. For all 1., 2., and 3., always add '%python_provide python2-foobar'
or '%python_provide python3-foobar' or
'%python_provide python%{py3_pkgversion}-foobar'.
This way we provide consistent naming for installation (dnf install
python2-foobar and dnf install python3-foobar DTRT), and "applications"
still follow upstream naming.
> If others *are* expected to import the module, some people make a
> "python3-foobar" sub-package, and make the main "foobar" package
> require it.
>
>
> [0]
> https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines#Addon_Packages
>
>
> >As a counter-proposal I would suggest that, as we currently do, we
> >require the python3-prefix to be provided by the package, but
> >explicitly leave it to the packager+reviewer's discretion whether or
> >not the prefix must be part of the real name, too.
Yes! I think what I wrote above matches this.
> >Some other
> >languages do already do this: nodejs [3] and ocaml [4] both explicitly
> >have "if this primarily provides a tool or application" clauses in
> >their naming guidelines. I think it makes sense to have something
> >similar for Python, to help avoid confusion.
>
> That's a nice idea.
>
> I'll note that modules installed using Python's mechanisms
> (setuptools, flit, pip, etc.) automatically provide
> "python3dist(name)". If we write new guidelines, I think we should
> promote that rather than the "python3-" prefix.
python3dist(name) provides have no %_isa suffix.
Right now packages which are multilib and have such p
Re: Mass package change (python2- binary package renaming)
On Wed, Aug 09, 2017 at 07:13:21PM -0600, Orion Poplawski wrote:
> >>>The Provides satisfies any dependancies on the old name, but you're
> >>>missing an Obsoletes to tell RPM the upgrade path. Trying to installing
> >>>the new python2-libvirt RPM on an existing system fails because it
> >>>clashes with libvirt-python.
> >>Good catch. Obsoletes: python-libvirt is generated by %python_provide,
> >>but I forgot to add Obsoletes: libvirt-python.
> >>Thanks, I'll fix this and other packages in the same situation.
> >I added that. New patches are in https://in.waw.pl/~zbyszek/fedora/pyrename/.
> >
> >This new Obsoletes is generated with %{_isa}, and I also added
> >%{_isa} to the matching Provides.
> >
> >Zbyszek
>
> IIRC - Obsoletes are done only against package names, not
> "provides", so the Obsoletes with %{_isa} is not needed. This is
> why %python_provide doesn't generate it with %{_isa} anymore.
Indeed. I removed %{_isa} from my Obsoletes.
That brings an interesting question:
automatically generated python2.7dist() and python2dist tags don't
have %_isa either. Is this a bug?
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Mass package change (python2- binary package renaming)
On Wed, Aug 09, 2017 at 04:48:42PM +0100, Daniel P. Berrange wrote:
> On Tue, Aug 08, 2017 at 10:14:26PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
> > Hello Fedora Python package maintainers!
> >
> > This is an announcement of a mass package renaming:
> > Python 2 binary packages will be renamed to python2-*.
> >
> > This will happen soon after the F27 branching on August 15th.
> >
> >
> > Currently ~1330 source packages already generate a binary package with
> > the python2- prefix, and 835 remain to be updated. The spec files for
> > approximately 740 packages will be renamed, and 95 will be left for
> > fixing by maintainers or proven packagers.
> >
> >
> > At the end of this e-mail are two lists of maintainers and packages:
> >
> > List 1. for those packages which will be taken care of by the mass remaining
> >Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
> >
> >Maintainers don't have to do anything.
>
> > Example:
> > +%package -n python2-atpy
> > +Summary: %summary
> > +Requires: numpy python-astropy
> > +%{?python_provide:%python_provide python2-atpy}
> > +# Remove before F30
> > +Provides: ATpy = %{version}-%{release}
>
> This looks incomplete & broken to me.
>
> The Provides satisfies any dependancies on the old name, but you're
> missing an Obsoletes to tell RPM the upgrade path. Trying to installing
> the new python2-libvirt RPM on an existing system fails because it
> clashes with libvirt-python.
Good catch. Obsoletes: python-libvirt is generated by %python_provide,
but I forgot to add Obsoletes: libvirt-python.
Thanks, I'll fix this and other packages in the same situation.
> A further flaw in your script is that its changed libvirt-python to
> python2-libvirt, but not changed libvirt-python3 to python3-libvirt,
> so the naming inconsistency is worse than before your proposed change.
Yeah, that was a conscious decision. In the draft I sent to fedora-devel
last week I asked if python3 subpackages should be renamed, but nobody
answered, so I assumed that people don't care that much either way.
My motivation for not touching this right now was to limit the number
of changes and potential for screwups.
> Also the %_description stuff is just ugly
Not the prettiest, but rpm just doesn't give us nice tools. The
alternative is to repeat the description text three times, which is
not only at least as ugly, but also harder to maintain.
Zbyszek
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Mass package change (python2- binary package renaming)
On Wed, Aug 09, 2017 at 11:04:06AM -0400, R P Herrold wrote: > On Tue, 8 Aug 2017, Zbigniew Jędrzejewski-Szmek wrote: > > > Hello Fedora Python package maintainers! > > > > This is an announcement of a mass package renaming: > > Python 2 binary packages will be renamed to python2-*. > > > List 1. for those packages which will be taken care of by the mass remaining > >Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/ > > Looking at a sample patch file, it apperas that more than a > simple > s/python-/python2-/g > replacement is being done [1] Right. A simple replacement like that wouldn't be backwards compatible: at the very least it would break dependent packages and violate packaging guidelines for Python. > The patch in question also contains 'tampering' with the > %description stanza: > > -%description > -ATpy is a high-level Python package providing a way to manipulate tables of > -astronomical data in a uniform way. It provides built-in support for NumPy > -recarrays and common astronomical file/database formats (FITS, VO, HDF5, > +%global _description\ > +ATpy is a high-level Python package providing a way to manipulate tables of\ > +astronomical data in a uniform way. It provides built-in support for NumPy\ > +recarrays and common astronomical file/database formats (FITS, VO, HDF5,\ > and ASCII tables) with a very simple API. > > +%description %_description > + > > ... this does not seem 'low risk', as substitutions happen > form many fields. It is also not mentioned in your post When a subpackage is added, a new %description is always required. So this is implied by "a new subpackage is added". A lot of effort was put into making the patches minimalistic, so they should resemble what a human would do when updating the spec file. > Who shall be expected to repair FTB's after the patch? No FTBs are expected. I'll rebuild all packages locally before pushing and I'll fix any breakage caused by my changes. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Mass package change (python2- binary package renaming)
Hello Fedora Python package maintainers!
This is an announcement of a mass package renaming:
Python 2 binary packages will be renamed to python2-*.
This will happen soon after the F27 branching on August 15th.
Currently ~1330 source packages already generate a binary package with
the python2- prefix, and 835 remain to be updated. The spec files for
approximately 740 packages will be renamed, and 95 will be left for
fixing by maintainers or proven packagers.
At the end of this e-mail are two lists of maintainers and packages:
List 1. for those packages which will be taken care of by the mass remaining
Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
Maintainers don't have to do anything.
List 2. for the remaining packages
Maintainers are encouraged to update packages manually.
### Details of the change ###
The change is to ensure that as many as possible python packages which
provide a version for python2 have a python2- subpackage as required by
the guidelines
[https://fedoraproject.org/wiki/Packaging:Naming#Python2_binary_package_naming].
For source packages which do *not* have a python- prefix, but already
have a python-foo subpackage, that subpackage will be renamed to python2-foo.
For packages which are called python-foo and just build a main binary package
python-foo, a new python2-foo subpackage will be created.
Provides/Requires/Conflicts/Obsoletes/Recommends/Suggests are moved from the
main package to the new subpackage.
In all cases, the new python2- subpackage will use lower-case naming.
This may lead to a situation where an existing python3- subpackage has
a differently-cased name than the python2- subpackage.
An effort is made (thanks Miro!) to preserve the style which is used
in the spec file, so that if e.g. python-%{real_name} was used originally,
this is changed to python2-%{real_name}. If this macro cannot be guessed or
if it resolves to a non-lowercase name, a non-macro lower-case string will
be used.
%{python_provide python2-foobar} are added as required by the guidelines
[https://fedoraproject.org/wiki/Packaging:Python#The_.25python_provide_macro].
One or two: %{python_provide python2-foobar} is always added, and if the package
has a non-lower-case name, %{python_provide python2-FooBar} is also added.
This provides backward compatible names to the package (python-foobar,
python-FooBar),
as long as %{python_provide} is defined as it is now. Once %python_provide macro
is switched to prefer python3 subpackages, those compat names will be gone.
In case the source package name does not have a python- prefix, a Provides
for the old name is also added, with a comment that it should be removed later.
Example:
+%package -n python2-atpy
+Summary: %summary
+Requires: numpy python-astropy
+%{?python_provide:%python_provide python2-atpy}
+# Remove before F30
+Provides: ATpy = %{version}-%{release}
Changes are implemented in an automated way using
https://pagure.io/pyrenamer.
rpmdev-bumpspec is used to add changelog entries will be added that look like:
* Sat Aug 05 2017 Zbigniew Jędrzejewski-Szmek - 3.3-4
- Python 2 binary package renamed to python2-foobar
See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
### Which packages need to change ###
http://fedora.portingdb.xyz/namingpolicy/ "Misnamed subpackage" lists 877
subpackages.
The real number is a bit lower, because there are some dead packages
and some that have been fixed in the meanwhile on that list.
The automated rename currently works for 737 packages, 7 have manual patches,
47 don't need work.
The remaining 95 packages will require manual fixes.
I do *not* want maintainers to manually fix packages which are covered
by the automated changes (list 1 below), since that provides no
benefit. Maintainers are of course encouraged to fix packages which
are not covered (list 2 below), or to take the automated patch and use
it as a basis of their own fix if the automated change is deficient
for some reason.
Help with the remaining ~95 packages which cannot be done automatically
is very much welcome.
### Opting out and independent fixes ###
The script skips all packages which already have a python2- subpackage
and %python_provide, so all packages which are fixed in the meantime will be
skipped. If packages need to be skipped even without that, let me know and
I'll add the package to the blacklist.
### Timeline ###
2017-08-08: this e-mail is sent to fedora-announce
soon after 2017-08-15 (F27 branching): automatically and manually
generated patches are applied, and packages are rebuilt.
Depending on the results, I'll fix some and follow up with bugzilla
tickets for the remaining packages.
### List 1. Packages handled by the automated rename ###
[https://in.waw.pl/~zbyszek/fedora/pyrename/]
Maintainers by package:
ATpy sergiopr
CheMPS2 talcite
NLoptbesser82
OpenIPMI branto jridky pknirsch
OpenIma
[DRAFT] Mass package change proposal
Hi!
This is a continuation of the "Finalizing Fedora's Switch to Python 3"
thread on fedora-devel, using the procedure from
https://fedoraproject.org/wiki/Mass_package_changes.
I'm proposing an automated change to ~723 packages, and manual fixes
or follow-up bug filing for the remaining ~114 packages.
### Proposed changes ###
The change is to ensure that as many as possible python packages which
provide a version for python2 have a python2- subpackage as required by
the guidelines
[https://fedoraproject.org/wiki/Packaging:Naming#Python2_binary_package_naming].
For source packages which do *not* have a python- prefix, but already
have a python-foo subpackage, that subpackage will be renamed to python2-foo.
For packages which are called python-foo and just build a main binary package
python-foo, a new python2-foo subpackage will be created.
Provides/Requires/Conflicts/Obsoletes are moved from the main package to
the new subpackage.
In all cases, the new python2- subpackage will use lower-case naming.
This may lead to a situation where an existing python3- subpackage has
a differently-cased name than the python2- subpackage.
[Q: maybe the python3- subpackage should be renamed automatically in this case?]
An effort is made (thanks Miro!) to preserve the style which is used
in the spec file, so that if e.g. python-%{real_name} was used originally,
this is changed to python2-%{real_name}. If this macro cannot be guessed or
if it resolves to a non-lowercase name, a non-macro lower-case string will
be used.
%{python_provide python2-foobar} are added as required by the guidelines
[https://fedoraproject.org/wiki/Packaging:Python#The_.25python_provide_macro].
One or two: %{python_provide python2-foobar} is always added, and if the package
has a non-lower-case name, %{python_provide python2-FooBar} is also added.
This provides backward compatible names to the package (python-foobar,
python-FooBar),
as long as %{python_provide} is defined as it is now. Once %python_provide macro
is switched to prefer python3 subpackages, those compat names will be gone.
In case the source package name does not have a python- prefix, a Provides
for the old name is also added, with a comment that it should be removed later.
Example:
+%package -n python2-atpy
+Summary: %summary
+Requires: numpy python-astropy
+%{?python_provide:%python_provide python2-atpy}
+# Remove before F30
+Provides: ATpy = %{version}-%{release}
Changes are implemented in an automated way using
https://pagure.io/pyrenamer.
rpmdev-bumpspec is used to add changelog entries will be added that look like:
* Sat Aug 05 2017 Zbigniew Jędrzejewski-Szmek - 3.3-4
- Python 2 subpackage renamed to python2-foobar
See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
Current diff: https://in.waw.pl/~zbyszek/fedora/python2-renaming.diff
[https://in.waw.pl/~zbyszek/fedora/python2-renaming.color.diff is a version
will color escapes, to be downloaded and viewed with less:
curl https://in.waw.pl/~zbyszek/fedora/python2-renaming.color.diff | less -RF]
### Which packages need to change ###
http://fedora.portingdb.xyz/namingpolicy/ "Misnamed subpackage" lists 877
subpackages.
It seems that the real number is a bit lower, because there are some
dead packages and some that have been fixed in the meanwhile on that
list. I'll generate a canonical maintainer: package, package: maintainer list
for the final version.
The automated rename currently works for 723 packages, 39 don't need work.
The remaining 141 packages will require manual fixes.
Some patches will be generated manually. They will be applied at the same
time.
I do *not* want maintainers to fix packages which are covered by the
automated changes by hand, since that provides no benefit. Maintainers are
of course encouraged to fix packages which are not covered, or to take the
automated patch and use it as a basis of their own fix if the automated change
is deficient for some reason.
I'm looking for volunteers to help with the remaining ~141 packages which
cannot be done automatically. If you can participate (either as a proven
packager or not), let me know. It is possible that automated cleanup can
be applied to some more packages (e.g. I just noticed that there's a
bunch of packages which have python%{python3_pkgversion}- subpackage,
which could done automatically...), so please coordinate with me to avoid
duplicated work.
[I'll be generating all the diffs as separate files and uploading them
somewhere later this weekend.]
Currently unhandled packages:
CONDITIONALS: ahkab mod_wsgi pystache python-alembic python-ansi2html
python-backports-ssl_match_hostname python-beaker python-bugzilla
python-cairocffi python-chameleon python-cliff python-cssmin
python-datanommer-models python-deltasigma python-django-dynamite
python-django-filter python-django-post_office
python-django-socialregistration python-docker-py python-flask-images
python-f
Re: Finalizing Fedora's Switch to Python 3
On Tue, Aug 01, 2017 at 11:29:30AM -0500, Jason L Tibbitts III wrote: > > "MH" == Miro Hrončok writes: > > MH> I just had a discussion with Tomáš Orsava and Petr Viktorin on > MH> #fedora-python. Rather than asking FESCo now to allow mass > MH> fully-automated spec changing, we'll open bugs as planned, but we'll > MH> attach patches generated by your script to them. > > It's not really required that you get an ACK from FESCo for automated > spec changes as long as you have a clear plan, people on hand to fix any > breakage, and let the devel list know about the changes you're making. > The packaging guidelines are pretty clear about maintainers needing to > be prepared for automated changes to their specs. But of course you can > run it by either FESCo or the packaging committee if you really want to > cover your bases. > > The relevant guideline section is > https://fedoraproject.org/wiki/Packaging:Guidelines#Spec_Maintenance_and_Canonicity > > Of course, if you wait just a bit you'll be able to submit pull > requests to pagure, but that could get kind of messy. > > To be honest, though, just fixing things for people where you can tell > those fixes are clean and easy is far preferable to opening bugs. In > fact, it's specifically encouraged. In any case, before you get to that > please do follow the policy for pass package changes: > https://fedoraproject.org/wiki/Mass_package_changes Thank you! That was my feeling too, but I was missing the formal text to back it up. I think the best way would indeed be to follow the "Mass package changes" procedure, in the "automated cleanup" flavour, but also file a FESCo ticket to "cover all bases". I don't think it's necessary to file an FPC ticket because in a sense FPC has already spoken: proposed change is to make the packages follow current guidelines, so I don't think FPC could be reasonably opposed. ~580 can be done automatically with the current version of the script. ~60 packages already have a subpackage with a bad name. It is possible that those could be done automatically too. That leaves maybe 150 packages that would require further manual changes, and for those it might make sense to file bugs. Further research would be needed which packages are left to convert. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Finalizing Fedora's Switch to Python 3
On Fri, Jul 28, 2017 at 08:44:37PM +, Zbigniew Jędrzejewski-Szmek wrote: > On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote: > > On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek > > wrote: > > > Do you think it'd be possible to script the python-foo to python2-foo > > > renaming? If yes, then maybe it'd make sense to just get some pps to > > > do it in rawhide now and get the "easy" part done with? That should > > > significantly cut down on the number of misnamed packages and let > > > packagers spend their times on the ones where the automatic way is not > > > obvious. > > > > I was going to ask whether it might be possible to tweak > > http://fedora.portingdb.xyz/ to also report on compliance with the > > naming policy, but then I went and saw that it *does* already report > > on that: http://fedora.portingdb.xyz/namingpolicy/ > > > > While it also turns out the wiki page already links to that page, it > > may be good to call it out a second time in a "How can I help?" > > section. > > > > Checking an initial sampling of spec files (python-d2to1, > > python-BeautfulSoup, python-amqplib) suggests to me that a script > > implementing the following rules might offer a reasonably start point, > > at least for Python-2-only modules that are remaining Python-2-only: > > > > - immediately before the first BuildRequires or Requires entry, add a > > %package section header for "-n python2-" (where "" is the > > lowercased package source name with any "python-" prefix stripped) > > - add a %python_provides entry after the new package header in > > accordance with the current guidelines > > - if the original package provided a non-lowercase "python-*" > > provides, remote it and add a second %python_provides with the > > original capitalisation > > - if the source package lacks the "python-" prefix, add a virtual > > provides for the unqualified package name > > - add a "-n python2-" qualifier to any currently unqualified > > description and files sections > > > > A script like that may even do a tolerable job for packages that *do* > > offer Python 3 subpackages (since those will already have qualifiers, > > and will necessarily appear after any unqualified runtime and build > > requirements for the default subpackage). > > I hacked up a script, it's on pagure now [1]. > See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.). > This does 561 python-* packages, out of 645. > I took the list from portingdb, but it seems partially outdated, 32 > packages already have %python_provide python2-*. > One more by hand (patch in repo). > That leaves 51 packages to convert by hand (mostly because of conditionals > which are too complicated to do automatically). > > So, please take a look at the diff [3]. If you want to adapt the > script, ping me, and I'll add you to the repo. > > After I started working on this, I limited it to python-* packages, > because those are easiest. There's 237 other packages on the portingdb > list. I *would* be possible to make the script work for those packages > which are pure python and just have an old name. I don't think it makes > sense to do other packages which just have a python subpackage this way. > > This would be a good start, I think. Before being pushed anywhere, > it'd need to be checked carefully of course and extended to some of > the remaining packages. I'd apply this as one step, and rebuild everything. > After that, it'd be easier to convert *dependencies*, because all or most > python2- names should be available. Some time after current mass rebuild > is done? > > Zbyszek > > [1] https://pagure.io/pyrenamer/tree/master > [2] https://pagure.io/pyrenamer/blob/master/f/results.txt > [3] https://paste.fedoraproject.org/paste/57oqggmOwvZFQjDbndRfcg I made some fixes and updates. Some packages were converted incorrectly, and are now skipped, and I updated the script to work for source packages without the python- prefix. Current stats: 'DONE': 584 (OK) 'MANY-SUBPACKAGES': 148 (not a python package) 'NEEDS-RENAME': 52, (already has a python- package, needs manual handling) 'CONDITIONALS': 47, (conditional provides/requires, needs manual handling) 'LOOKS-GOOD': 38, (already has %python_provide, probably got converted) 'DEAD': 10, 'SKIP-FILE': 4(to complicated, like vimsyntastic) 'EXECUTABLES': 88 — just a warning that the package has stuff in /usr/bin, and users might be confused that the package is suddenly python2-foobar. The real solution would be to convert the executable to python3. log: https://paste.fedoraproject.org/paste/OmhyHFGRGAOXRZf7ZVLH3g/ diff: https://paste.fedoraproject.org/paste/EaBgJRpNokY1BJc~4PtGmg/ Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Finalizing Fedora's Switch to Python 3
On Fri, Jul 28, 2017 at 11:11:05PM +0200, Miro Hrončok wrote: > On 28.7.2017 22:44, Zbigniew Jędrzejewski-Szmek wrote: > >On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote: > >>On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek > >>wrote: > >>>Do you think it'd be possible to script the python-foo to python2-foo > >>>renaming? If yes, then maybe it'd make sense to just get some pps to > >>>do it in rawhide now and get the "easy" part done with? That should > >>>significantly cut down on the number of misnamed packages and let > >>>packagers spend their times on the ones where the automatic way is not > >>>obvious. > >> > >>I was going to ask whether it might be possible to tweak > >>http://fedora.portingdb.xyz/ to also report on compliance with the > >>naming policy, but then I went and saw that it *does* already report > >>on that: http://fedora.portingdb.xyz/namingpolicy/ > >> > >>While it also turns out the wiki page already links to that page, it > >>may be good to call it out a second time in a "How can I help?" > >>section. > >> > >>Checking an initial sampling of spec files (python-d2to1, > >>python-BeautfulSoup, python-amqplib) suggests to me that a script > >>implementing the following rules might offer a reasonably start point, > >>at least for Python-2-only modules that are remaining Python-2-only: > >> > >>- immediately before the first BuildRequires or Requires entry, add a > >>%package section header for "-n python2-" (where "" is the > >>lowercased package source name with any "python-" prefix stripped) > >>- add a %python_provides entry after the new package header in > >>accordance with the current guidelines > >>- if the original package provided a non-lowercase "python-*" > >>provides, remote it and add a second %python_provides with the > >>original capitalisation > >>- if the source package lacks the "python-" prefix, add a virtual > >>provides for the unqualified package name > >>- add a "-n python2-" qualifier to any currently unqualified > >>description and files sections > >> > >>A script like that may even do a tolerable job for packages that *do* > >>offer Python 3 subpackages (since those will already have qualifiers, > >>and will necessarily appear after any unqualified runtime and build > >>requirements for the default subpackage). > > > >I hacked up a script, it's on pagure now [1]. > >See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.). > >This does 561 python-* packages, out of 645. > > Wow, nice job! > > >I took the list from portingdb, but it seems partially outdated, 32 > >packages already have %python_provide python2-*. > >One more by hand (patch in repo). > >That leaves 51 packages to convert by hand (mostly because of conditionals > >which are too complicated to do automatically). > > > >So, please take a look at the diff [3]. If you want to adapt the > >script, ping me, and I'll add you to the repo. > > > >After I started working on this, I limited it to python-* packages, > >because those are easiest. There's 237 other packages on the portingdb > >list. I *would* be possible to make the script work for those packages > >which are pure python and just have an old name. I don't think it makes > >sense to do other packages which just have a python subpackage this way. > > > >This would be a good start, I think. Before being pushed anywhere, > >it'd need to be checked carefully of course and extended to some of > >the remaining packages. I'd apply this as one step, and rebuild everything. > >After that, it'd be easier to convert *dependencies*, because all or most > >python2- names should be available. Some time after current mass rebuild > >is done? > > That sounds like something that would need to be approved by FESCo, > isn't it? Yes. That's what I said before too. I would like to see buy-in from you and Nick and other python gurus before submitting it anywhere else though. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Finalizing Fedora's Switch to Python 3
On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote: > On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek > wrote: > > Do you think it'd be possible to script the python-foo to python2-foo > > renaming? If yes, then maybe it'd make sense to just get some pps to > > do it in rawhide now and get the "easy" part done with? That should > > significantly cut down on the number of misnamed packages and let > > packagers spend their times on the ones where the automatic way is not > > obvious. > > I was going to ask whether it might be possible to tweak > http://fedora.portingdb.xyz/ to also report on compliance with the > naming policy, but then I went and saw that it *does* already report > on that: http://fedora.portingdb.xyz/namingpolicy/ > > While it also turns out the wiki page already links to that page, it > may be good to call it out a second time in a "How can I help?" > section. > > Checking an initial sampling of spec files (python-d2to1, > python-BeautfulSoup, python-amqplib) suggests to me that a script > implementing the following rules might offer a reasonably start point, > at least for Python-2-only modules that are remaining Python-2-only: > > - immediately before the first BuildRequires or Requires entry, add a > %package section header for "-n python2-" (where "" is the > lowercased package source name with any "python-" prefix stripped) > - add a %python_provides entry after the new package header in > accordance with the current guidelines > - if the original package provided a non-lowercase "python-*" > provides, remote it and add a second %python_provides with the > original capitalisation > - if the source package lacks the "python-" prefix, add a virtual > provides for the unqualified package name > - add a "-n python2-" qualifier to any currently unqualified > description and files sections > > A script like that may even do a tolerable job for packages that *do* > offer Python 3 subpackages (since those will already have qualifiers, > and will necessarily appear after any unqualified runtime and build > requirements for the default subpackage). I hacked up a script, it's on pagure now [1]. See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.). This does 561 python-* packages, out of 645. I took the list from portingdb, but it seems partially outdated, 32 packages already have %python_provide python2-*. One more by hand (patch in repo). That leaves 51 packages to convert by hand (mostly because of conditionals which are too complicated to do automatically). So, please take a look at the diff [3]. If you want to adapt the script, ping me, and I'll add you to the repo. After I started working on this, I limited it to python-* packages, because those are easiest. There's 237 other packages on the portingdb list. I *would* be possible to make the script work for those packages which are pure python and just have an old name. I don't think it makes sense to do other packages which just have a python subpackage this way. This would be a good start, I think. Before being pushed anywhere, it'd need to be checked carefully of course and extended to some of the remaining packages. I'd apply this as one step, and rebuild everything. After that, it'd be easier to convert *dependencies*, because all or most python2- names should be available. Some time after current mass rebuild is done? Zbyszek [1] https://pagure.io/pyrenamer/tree/master [2] https://pagure.io/pyrenamer/blob/master/f/results.txt [3] https://paste.fedoraproject.org/paste/57oqggmOwvZFQjDbndRfcg ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Finalizing Fedora's Switch to Python 3
On Fri, Jul 28, 2017 at 09:49:42AM +0200, Miro Hrončok wrote: > On 27.7.2017 19:15, Zbigniew Jędrzejewski-Szmek wrote: > >Do you think it'd be possible to script the python-foo to python2-foo > >renaming? > > May be possible but given the variety in the spec files, we didn't > think it would be easy. IMHO taking a bunch of provenpackagers and > updating the packages manually without asking the owners will be > less time consuming than creating an automated tool. > > However, in both scenarios (people or tools), I consider that > approach aggressive and would rather only do it when the deadlines > are close. > We'd rather not change other people packages and make them mad about > what have we done. The transition to Python 3 is unfortunately > considered a failure in eyes of some Fedora contributors. [My original comment here has been deleted to protect the innocent ;) ] > I consider communicating our efforts with the package owners and > explaining them what and how to do as more friendly approach. > However, definitively more time expensive. > > > On the other hand, maybe we can start updating the packages co-owned > by python-sig, as I would say those are considered to be maintained > by the group rather than a person. In case you'd like to try to > write an automated tool for that, please go ahead (but please let us > know). I don't know if delaying this will help. Actually stretching out the transition might make things more painful, because it'll increase the window where you don't know if any given python package has python2- or python- prefix, so you need to check all deps. I'm sure getting rid of this ambiguity would be welcome. So instead of taking this slowly, I'd rather go through FESCo and public announcement on fedora-devel, and then just apply the change to as many packages as possible at once. Technically, I think it would be reasonable to try to script this, w/o pushing to dist git, then put up the diffs for review somewhere, and then finally push at once, at get a few pps to do the rebuilds (some stuff is bound to fail for related or unrelated reasons, so some debugging is expected). In the other mail Nick Coghlan listed various steps: it's far from trivial, but OTOH doing 800+ packages by hand ain't gonna be fun either. I'll try to look into this over the weekend, it sounds like a fun challenge. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Finalizing Fedora's Switch to Python 3
Do you think it'd be possible to script the python-foo to python2-foo renaming? If yes, then maybe it'd make sense to just get some pps to do it in rawhide now and get the "easy" part done with? That should significantly cut down on the number of misnamed packages and let packagers spend their times on the ones where the automatic way is not obvious. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Should Python 3 macros us UTF-8 locale?
On Thu, Jun 01, 2017 at 11:02:25PM +0200, Miro Hrončok wrote: > > > On 1.6.2017 20:49, Zbigniew Jędrzejewski-Szmek wrote: > >On Thu, Jun 01, 2017 at 11:26:55AM -0700, Toshio Kuratomi wrote: > >>I think it is better if the build system sets LANG=C.UTF-8, not the > >>python specific macros. > > > >Yeah, setting it through python macros would mean that e.g. any > >python program invoked during the build would see the obsolete non-utf8 > >encoding. Seems better to just set LANG=C.UTF-8 globally. > > Who could make that happen? Should I propose this at devel@? It seems that mock uses $LANG to configure $LANG and all $LC_* variables in the build environment. But when mock is invoked with LANG=C, it ignores that and uses C.UTF-8 instead, because of the "Python 3 C.UTF-8 locale" change in python. So with new-enough python, we'd get C.UTF-8 anyway on the builders, once the builders are upgraded to F26. But then you'd still get a warning on every mock invocation. It seems that right solution, that would work while the builders are still not F26, would be to configure their locale to C.UTF-8. It'd just be a matter of dropping LANG=C.UTF-8 into /etc/locale.conf. I think a ticket on https://pagure.io/fedora-infrastructure would be the way to request the change. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Should Python 3 macros us UTF-8 locale?
On Thu, Jun 01, 2017 at 11:26:55AM -0700, Toshio Kuratomi wrote:
> I think it is better if the build system sets LANG=C.UTF-8, not the
> python specific macros.
Yeah, setting it through python macros would mean that e.g. any
python program invoked during the build would see the obsolete non-utf8
encoding. Seems better to just set LANG=C.UTF-8 globally.
Zbyszek
>
> On Thu, Jun 1, 2017 at 8:56 AM, Miro Hrončok wrote:
> > Hi Pythonistas.
> >
> > Regarding our Python 3 C.UTF-8 locale coercing [1], aka PEP 538 [2].
> >
> > As you probably know, we build RPM packages with the C locale. So everytime
> > we use python3 in the spec file, the coercing message is shown. This can be
> > more problematic than just spamming the build logs, see for example the
> > related rpmlint bug [3][4].
> >
> > Our macros, such as %{python3_sitelib}, %{python3_version} etc. all call
> > python3 and generate the warning. Should we "fix" our macros to set the LANG
> > to C.UTF-8?
> >
> > If we change the %{__python3} macro entirely, we might get rid of all of
> > those warnings and we will workaround the fact that we build RPM packages
> > with the C locale. On the other hand the packager would not be able to set a
> > desired locale because it will always be overwritten:
> >
> > # The crazy test suite needs Czech locale
> > LANG=cs_CZ.utf8 %{__python3} -m pytest
> >
> > Will become:
> >
> > LANG=cs_CZ.utf8 LANG=C.utf8 /usr/bin/python3 -m pytest
> >
> > So I would not do that.
> >
> > But we can change all other macros in /usr/lib/rpm/macros.d/macros.python3
> > to set the UTF-8 locale. Would that be wise? Desired?
> >
> > Any thoughts?
> >
> >
> > [1] https://fedoraproject.org/wiki/Changes/python3_c.utf-8_locale
> > [2] https://www.python.org/dev/peps/pep-0538/
> > [3] https://bugzilla.redhat.com/show_bug.cgi?id=1457786
> > [4] https://bugzilla.redhat.com/show_bug.cgi?id=1436345
> >
> > --
> > Miro Hrončok
> > --
> > Phone: +420777974800
> > IRC: mhroncok
> > ___
> > python-devel mailing list -- python-devel@lists.fedoraproject.org
> > To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
> ___
> python-devel mailing list -- python-devel@lists.fedoraproject.org
> To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Missing executables in Python 3 subpackages
On Thu, Mar 02, 2017 at 06:06:49PM +0100, Iryna Shcherbina wrote: > python-igor This one is functional under python3, but the output is slightly nicer under python2 (python3 shows some spurious b'' prefixes). So it's on py2 on purpose. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Tue, Jan 17, 2017 at 08:14:36PM +, Zbigniew Jędrzejewski-Szmek wrote: > On Tue, Jan 17, 2017 at 08:29:16AM -0700, Jerry James wrote: > > On Tue, Jan 17, 2017 at 7:08 AM, Zbigniew Jędrzejewski-Szmek > > wrote: > > > On Mon, Jan 16, 2017 at 10:27:19PM -0800, Adam Williamson wrote: > > >> On Tue, 2017-01-17 at 05:33 +, Zbigniew Jędrzejewski-Szmek wrote: > > >> > Oh, Jerry beat me to it by taking a different route and disabling most > > >> > test. That should work too, but I think it hides real problems with > > >> > the python3 code... > > >> > > >> Yeah. Please don't just disable tests. Tests exist for a reason. > > > > > > We need to disable *some* tests, at least if we hope to build this > > > quickly. It's just a question whether to disable more or less of > > > them. > > > > > > Most tests fail because of sympy/pyglet/xvfb/opengl interaction > > > (#904851), not from sympy error per se — the same tests run fine on a > > > "real" display. > > > > Oh, sorry if I stepped on somebody's toes. i didn't realize others > > were trying to fix sympy, too. I guess I should join the python SIG > > mailing list, but I'm already on far too many mailing lists... > No, no, there were no toes to step on. I didn't announce anywhere > that I'm working on this. And anyway, I think our work is complementary: > I didn't know about the patch you pulled in, > but otoh, I managed to fix some of the tests by passing options to xvfb. Sympy is now built. I had to disable a few tests... Those seems to be actual (and known) bugs in sympy, I added some links to upstream bug reports. I tried building nipy [1], but it fails in tests. No too badly though, apparently just three tests, out of 2599. Zbyszek [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=17326509 ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Tue, Jan 17, 2017 at 08:29:16AM -0700, Jerry James wrote: > On Tue, Jan 17, 2017 at 7:08 AM, Zbigniew Jędrzejewski-Szmek > wrote: > > On Mon, Jan 16, 2017 at 10:27:19PM -0800, Adam Williamson wrote: > >> On Tue, 2017-01-17 at 05:33 +, Zbigniew Jędrzejewski-Szmek wrote: > >> > Oh, Jerry beat me to it by taking a different route and disabling most > >> > test. That should work too, but I think it hides real problems with > >> > the python3 code... > >> > >> Yeah. Please don't just disable tests. Tests exist for a reason. > > > > We need to disable *some* tests, at least if we hope to build this > > quickly. It's just a question whether to disable more or less of > > them. > > > > Most tests fail because of sympy/pyglet/xvfb/opengl interaction > > (#904851), not from sympy error per se — the same tests run fine on a > > "real" display. > > Oh, sorry if I stepped on somebody's toes. i didn't realize others > were trying to fix sympy, too. I guess I should join the python SIG > mailing list, but I'm already on far too many mailing lists... No, no, there were no toes to step on. I didn't announce anywhere that I'm working on this. And anyway, I think our work is complementary: I didn't know about the patch you pulled in, but otoh, I managed to fix some of the tests by passing options to xvfb. I'll try to push some of my changes too, let's see if that builds. > But I don't understand what you mean about me disabling tests. I > haven't disabled any tests, much less "most tests". i did add a BR on > pyglets, then discovered that that CAUSES test failures due to missing > GLX support, and so I removed the pyglets BR again. Maybe a BR on > mesa is needed as well as pyglets? I don't know. > > Anyway, I checked in an experimental patch in an attempt at fixing the > test failures, and now there is a different test failure. :-( So > either my experimental patch isn't good enough, or it works perfectly > and has unmasked another problem. I probably won't have time to > figure that out for a couple of days, so if somebody else is > interested in looking at it, here is the latest build failure: > > https://koji.fedoraproject.org/koji/buildinfo?buildID=828867 Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Mon, Jan 16, 2017 at 10:27:19PM -0800, Adam Williamson wrote: > On Tue, 2017-01-17 at 05:33 +0000, Zbigniew Jędrzejewski-Szmek wrote: > > On Tue, Jan 17, 2017 at 05:29:35AM +, Zbigniew Jędrzejewski-Szmek wrote: > > > On Wed, Jan 11, 2017 at 04:28:18AM +, Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > > python3-sympy-0:1.0-3.fc25.noarch > > > > > > > > Pfff, it still fails in the test suite :( > > > > > > I'm chipping away at this one. There are some failing python2 tests > > > which I had to disable, but it seems that things are mostly with python2. > > > But it seems that the python3 code is plain broken (bad non-absolute > > > imports). > > > > Oh, Jerry beat me to it by taking a different route and disabling most > > test. That should work too, but I think it hides real problems with > > the python3 code... > > Yeah. Please don't just disable tests. Tests exist for a reason. We need to disable *some* tests, at least if we hope to build this quickly. It's just a question whether to disable more or less of them. Most tests fail because of sympy/pyglet/xvfb/opengl interaction (#904851), not from sympy error per se — the same tests run fine on a "real" display. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Tue, Jan 17, 2017 at 05:29:35AM +, Zbigniew Jędrzejewski-Szmek wrote: > On Wed, Jan 11, 2017 at 04:28:18AM +0000, Zbigniew Jędrzejewski-Szmek wrote: > > > python3-sympy-0:1.0-3.fc25.noarch > > Pfff, it still fails in the test suite :( > I'm chipping away at this one. There are some failing python2 tests > which I had to disable, but it seems that things are mostly with python2. > But it seems that the python3 code is plain broken (bad non-absolute > imports). Oh, Jerry beat me to it by taking a different route and disabling most test. That should work too, but I think it hides real problems with the python3 code... > > > This one depends on sympy, so might be ready to build when sympy is done: > > > python3-nipy-0:0.4.0-5.fc25.x86_64 Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Wed, Jan 11, 2017 at 04:28:18AM +, Zbigniew Jędrzejewski-Szmek wrote: > > python3-sympy-0:1.0-3.fc25.noarch > Pfff, it still fails in the test suite :( I'm chipping away at this one. There are some failing python2 tests which I had to disable, but it seems that things are mostly with python2. But it seems that the python3 code is plain broken (bad non-absolute imports). > > This one depends on sympy, so might be ready to build when sympy is done: > > python3-nipy-0:0.4.0-5.fc25.x86_64 Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Wed, Jan 11, 2017 at 03:58:18AM +, Zbigniew Jędrzejewski-Szmek wrote: > On Wed, Jan 11, 2017 at 03:38:59AM +0000, Zbigniew Jędrzejewski-Szmek wrote: > > PS. What's the status of the python3.6 rebuild? Can somebody post > > a list of work-needing packages? > > I guess that the right invocation: > > $ dnf repoquery --whatrequires 'python(abi) = 3.5' > dpm-python3-0:1.9.0-2.fc26.x86_64 > lfc-python3-0:1.9.0-2.fc26.x86_64 > python3-bsddb3-0:6.2.0-3.fc26.x86_64 > python3-django-admin-honeypot-0:1.0.0-1.fc26.noarch > python3-django-avatar-0:3.0.0-1.fc26.noarch > python3-django-countries-0:3.4.1-2.fc24.noarch > python3-django-filter-0:0.10.0-3.fc24.noarch > python3-flower-0:0.9.1-2.fc25.noarch Blocked by #1410864. > python3-gensim-addons-0:0.10.0-5.fc24.x86_64 > python3-gensim-core-0:0.10.0-5.fc24.noarch > python3-gensim-test-0:0.10.0-5.fc24.noarch > python3-kdcproxy-0:0.3.2-3.fc24.noarch > python3-moss-0:0.3.4-7.fc26.noarch > python3-os-brick-0:1.3.0-2.fc25.noarch > python3-oslo-messaging-0:5.10.0-1.fc26.noarch > python3-oslo-messaging-tests-0:5.10.0-1.fc26.noarch > python3-oslo-middleware-0:3.19.0-1.fc26.noarch > python3-oslo-middleware-tests-0:3.19.0-1.fc26.noarch > python3-oslo-versionedobjects-0:1.8.0-1.fc26.noarch > python3-oslo-versionedobjects-tests-0:1.8.0-1.fc26.noarch > python3-profilehooks-0:1.8.0-3.fc25.noarch > python3-pyopencl-0:2016.1-4.fc26.x86_64 > python3-recommonmark-0:0.4.0-7.git7ca5247.fc25.noarch > python3-repoze-who-plugins-sa-0:1.0.1-11.20160106gite1a36c5.fc25.noarch > python3-shogun-0:4.1.0-7.fc26.x86_64 > rb_libtorrent-python3-0:1.1.1-1.fc26.x86_64 > root-python3-0:6.06.08-2.fc26.i686 > root-python3-0:6.06.08-2.fc26.x86_64 > > I fixed those two: > python3-sphinxcontrib-programoutput-0:0.8-6.fc24.noarch > python3-mdp-0:3.5-4.fc25.noarch > > This might be fixed, I fired off a build in koji now: > python3-sympy-0:1.0-3.fc25.noarch Pfff, it still fails in the test suite :( > This one depends on sympy, so might be ready to build when sympy is done: > python3-nipy-0:0.4.0-5.fc25.x86_64 > > All in all, this doesn't look to bad. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Wed, Jan 11, 2017 at 03:38:59AM +, Zbigniew Jędrzejewski-Szmek wrote: > PS. What's the status of the python3.6 rebuild? Can somebody post > a list of work-needing packages? I guess that the right invocation: $ dnf repoquery --whatrequires 'python(abi) = 3.5' dpm-python3-0:1.9.0-2.fc26.x86_64 lfc-python3-0:1.9.0-2.fc26.x86_64 python3-bsddb3-0:6.2.0-3.fc26.x86_64 python3-django-admin-honeypot-0:1.0.0-1.fc26.noarch python3-django-avatar-0:3.0.0-1.fc26.noarch python3-django-countries-0:3.4.1-2.fc24.noarch python3-django-filter-0:0.10.0-3.fc24.noarch python3-flower-0:0.9.1-2.fc25.noarch python3-gensim-addons-0:0.10.0-5.fc24.x86_64 python3-gensim-core-0:0.10.0-5.fc24.noarch python3-gensim-test-0:0.10.0-5.fc24.noarch python3-kdcproxy-0:0.3.2-3.fc24.noarch python3-moss-0:0.3.4-7.fc26.noarch python3-os-brick-0:1.3.0-2.fc25.noarch python3-oslo-messaging-0:5.10.0-1.fc26.noarch python3-oslo-messaging-tests-0:5.10.0-1.fc26.noarch python3-oslo-middleware-0:3.19.0-1.fc26.noarch python3-oslo-middleware-tests-0:3.19.0-1.fc26.noarch python3-oslo-versionedobjects-0:1.8.0-1.fc26.noarch python3-oslo-versionedobjects-tests-0:1.8.0-1.fc26.noarch python3-profilehooks-0:1.8.0-3.fc25.noarch python3-pyopencl-0:2016.1-4.fc26.x86_64 python3-recommonmark-0:0.4.0-7.git7ca5247.fc25.noarch python3-repoze-who-plugins-sa-0:1.0.1-11.20160106gite1a36c5.fc25.noarch python3-shogun-0:4.1.0-7.fc26.x86_64 rb_libtorrent-python3-0:1.1.1-1.fc26.x86_64 root-python3-0:6.06.08-2.fc26.i686 root-python3-0:6.06.08-2.fc26.x86_64 I fixed those two: python3-sphinxcontrib-programoutput-0:0.8-6.fc24.noarch python3-mdp-0:3.5-4.fc25.noarch This might be fixed, I fired off a build in koji now: python3-sympy-0:1.0-3.fc25.noarch This one depends on sympy, so might be ready to build when sympy is done: python3-nipy-0:0.4.0-5.fc25.x86_64 All in all, this doesn't look to bad. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Packages FTBFS with Python 3.6
On Thu, Jan 05, 2017 at 02:45:37PM -0800, Adam Williamson wrote: > On Wed, 2016-12-21 at 00:18 +0100, Miro Hrončok wrote: > > python-sphinxcontrib-programoutput > > This one was abandoned by its upstream years ago. He actually entirely > deleted the repository from github, though someone else put up a copy > of it recently (they didn't pick up maintenance, just put up a > 'historical copy'). > > It's only used by certbot and python-acme (which is actually a part of > certbot that gets its own releases on pypi and which we treat as a > separate source package). > > Both have stopped using it upstream now, so we can quite trivially > backport those fixes to our certbot and python-acme packages, and > retire sphinxcontrib-programoutput. > > Zbyszek is the owner of the package...do you agree with this plan, > Zbyszek? I'm fixing up certbot and python-acme, but you'd have to > retire programoutput. Hi Adam, I'm just back from the winter break. It's always great to see other people take care of my packages, thanks! You're right that the original repo is gone, but I think that this extension remains to be useful. There's a bunch of forks on github, so hopefully somebody will pick up maintenance. If that doesn't happen, or if a good replacement appears, I'll drop the package. But for now I prefer to let it live. I fixed the test suite, it's building in rawhide. Zbyszek PS. What's the status of the python3.6 rebuild? Can somebody post a list of work-needing packages? ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Python 3.6, Fedora, and the "C" locale
On Sun, Dec 11, 2016 at 11:24:54AM +, Thomas Spura wrote: > Zbigniew Jędrzejewski-Szmek schrieb am Sa., 10. Dez. > 2016 um 23:50 Uhr: > > > On Sat, Dec 10, 2016 at 05:41:45PM -0500, Neal Gompa wrote: > > > On Sat, Dec 10, 2016 at 3:40 PM, Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > On Sat, Dec 10, 2016 at 11:56:44PM +1000, Nick Coghlan wrote: > > > >> Along similar lines, what do folks think of the idea of patching > > > >> Python 3.6 in Fedora to assume UTF-8 if it's told that it should use > > > >> ASCII to communicate with the OS? > > > > +1 > > > > > > > > Non-utf8 environments are nowadays a rarity, OTOH misconfigured > > > > installations which do support utf8 but are just missing an env var > > > > are rather common (e.g. mock). > > > > > > Why aren't we fixing Fedora Cloud/Atomic and the container images to > > > be C.UTF-8 instead of just plain C, then? > > > > It's a game of whack-a-mole. You can always fix that place you just > > noticed where it's missing, but then a few days later it's in another > > place. Not saying that we should be initializing the locale properly, > > but rather than we can do both independently. > > > > > To change the default encoding for python was proposed a while ago [1], but > was finally dropped again, as upstream didn't agree to this change. Did > anything changed here from upstream python? > > Best, >Thomas > > [1] https://fedoraproject.org/wiki/Features/PythonEncodingUsesSystemLocale That was 2010, now it's 2016, going on 2017. That was Python 2, now it's Python 3.6 ;) But seriously, with Python 2 such changes were much more risky: Unicode was bolted on, and it was (is) very easy to cause things to fail unpredictably when Unicode was added to the mix. But with Python 3, Unicode is the basis upon which everything is built, and things more often start to fail when (unexpectedly) input is suddenly not Unicode. I think the considerations are a bit different for Python upstream, which has a lot of Windows users and utf-16, and for Linux, where utf-8 is pervasive. To make things less hand-wavy: there are concrete examples where this would be known to help: containers as mentioned in the original proposal, and mock (in my experience tests most often require setting an utf-8 environment to work). Do we know of examples where the proposed change would make things worse? Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Python 3.6, Fedora, and the "C" locale
On Sat, Dec 10, 2016 at 05:41:45PM -0500, Neal Gompa wrote: > On Sat, Dec 10, 2016 at 3:40 PM, Zbigniew Jędrzejewski-Szmek > wrote: > > On Sat, Dec 10, 2016 at 11:56:44PM +1000, Nick Coghlan wrote: > >> Along similar lines, what do folks think of the idea of patching > >> Python 3.6 in Fedora to assume UTF-8 if it's told that it should use > >> ASCII to communicate with the OS? > > +1 > > > > Non-utf8 environments are nowadays a rarity, OTOH misconfigured > > installations which do support utf8 but are just missing an env var > > are rather common (e.g. mock). > > Why aren't we fixing Fedora Cloud/Atomic and the container images to > be C.UTF-8 instead of just plain C, then? It's a game of whack-a-mole. You can always fix that place you just noticed where it's missing, but then a few days later it's in another place. Not saying that we should be initializing the locale properly, but rather than we can do both independently. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Python 3.6, Fedora, and the "C" locale
On Sat, Dec 10, 2016 at 11:56:44PM +1000, Nick Coghlan wrote: > Along similar lines, what do folks think of the idea of patching > Python 3.6 in Fedora to assume UTF-8 if it's told that it should use > ASCII to communicate with the OS? +1 Non-utf8 environments are nowadays a rarity, OTOH misconfigured installations which do support utf8 but are just missing an env var are rather common (e.g. mock). Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: upstream dev. asks suggestions about howto make packagers work easier (bundled libraries, etc.)
On Sun, Nov 20, 2016 at 08:48:27AM +0100, Germano Massullo wrote: > We often deal with upstream developers that bundle libraries in their > code, so to make a package we have to debundle them, etc. > This time, an upstream dev. asked me what he could do to make easier > the work of packagers. > In this case the software is python-netjsongraph [1] that bundles > javascript-d3 library and that is being reviewd at [2] > > I think it would be nice to make a discussion even for non Python > packages, so we can elaborate a sort of vademecum that a packager > could show to upstreams when there is a collaboration between them. I think it comes down to: - don't bundle, - if you have to bundle, provide an easy and unambiguous configure switch to use the system version of the dependency, - never, never, patch stuff in-tree. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: List of packages owning %{python3_sitelib}/__pycache__
On Mon, Nov 14, 2016 at 11:49:32PM -0200, Athos Ribeiro wrote: > python-cma > python-pandocfilters Fixed those two. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Python packages tests directory co-ownership
On Mon, Oct 31, 2016 at 12:00:58PM -0200, Athos Ribeiro wrote: > tl;dr: different packages own /usr/lib/pythonX/site-packages/tests and > different files with the same name inside that directory. > > Hi all, > > I was going through a package review and realized the package under > review owns > > /usr/lib/pythonX/site-packages/tests > > and some files inside the directory. > > since there were no subdirectories and there were no namespaces for the > file names inside the directory, I went further and realized that the > following packages are doing the same: > > python3-custodia-0:0.1.0-3.fc24.noarch > python3-django-federated-login-0:1.0.0-9.fc24.noarch > python3-journal-brief-0:1.1.3-3.fc24.noarch > python3-oauth2-0:1.9.0-2.post1.fc24.noarch > python3-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch > python-custodia-0:0.1.0-3.fc24.noarch > python-django-federated-login-0:1.0.0-9.fc24.noarch > python-libturpial-0:1.7.0-4.fc24.noarch > python-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch > > what's the problem with that? Well what if we would have a package named > python-tests? shouldn't it own that directory? No, it's a bug in each and every one of those packages. Usually this is an upstream issue, and the fix is to make tests/ a submodule of the main module. For example of doing this properly, let's take numpy: it has numpy and numpy.testing. After installation, you can test your *installed* numpy by running numpy.testing.test(). Countless other packages get this right too. Squatting on a common name like "test", "tests", "testing" in the global namespace should be caught in review. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: including EOL and vulnerable software in Fedora
On Fri, Oct 07, 2016 at 06:43:10PM +0200, Dominik 'Rathann' Mierzejewski wrote: > Dear All, > I was made aware that EOL software with known security bugs that will > not be fixed upstream (due to EOL status) was reviewed and accepted into > Fedora recently. This came on the back of the FPC ticket [1] asking to > make some changes in the Python Packaging Guidelines. I did go back and > re-read our current guidelines and found that we don't have any policy > on that. As a result, I opened a FESCo ticket [2] with the aim of > establishing a clear policy on how to treat EOL software with known > security vulnerabilities. A parallel could be drawn between previous python versions and previous C standards, like c89, c90, c99, etc. One could say that they are obsolete, but it is still very convenient to be able to add CFLAGS=-ansi. The difference is that gcc has this built in, while python does not have compatibility with previous "standards", so the only way to test with previous versions is to run those previous versions. It's damn useful for testing, and it's much more convenient to do it through dnf install than through virtualization/containers/cloud/hand-compilation/copr/other-nonstandard-things. So from my side, a vote for 1. labelling old pythons very clearly as such, 2. allowing people to install them using dnf. Zbyszek ___ python-devel mailing list -- python-devel@lists.fedoraproject.org To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Re: Automatic Provides: Discussion summary and plan
On Tue, Aug 23, 2016 at 12:48:40PM -0500, Jason L Tibbitts III wrote: > >>>>> "ZJ" == Zbigniew Jędrzejewski-Szmek writes: > > ZJ> %changelog would have to be stored separately. This would apply even > ZJ> for qthe case when no "external" changes were made — just normal updates > ZJ> by the package managers — since there's no way for pyp2rpm to know > ZJ> what happened in the past. > > It certainly could; it has the existing spec. Yeah, that'd be even better to pull the changelog from existing spec. Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org
Re: Automatic Provides: Discussion summary and plan
On Tue, Aug 23, 2016 at 11:10:08AM -0500, Jason L Tibbitts III wrote: > > "PV" == Petr Viktorin writes: > > PV> - Make it standard practice in Fedora to use this data and treat the > PV> spec file as an immutable generated artifact. > > If you're saying that any changes which are made to the spec file (say, > by release engineering doing a rebuild or by someone tweaking the > package for pretty much any reason) will be overwritten, this is not a > particularly good idea. %changelog would have to be stored separately. This would apply even for the case when no "external" changes were made — just normal updates by the package managers — since there's no way for pyp2rpm to know what happened in the past. Anyway, I think it's a good idea to try to make automatically generated spec files work, but making this mandatory sound premature. Even in the future, there's always be packages which need some manual fixup to whatever pyp2rpm produces. Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org
Re: __pycache__ in python2 directory? O_o
On Sat, Aug 20, 2016 at 05:17:59PM +0200, Igor Gnatenko wrote: > RPM build errors: > Installed (but unpackaged) file(s) found: >/usr/lib/python2.7/site-packages/__pycache__/site.cpython-35.pyc > > > This is from build of Flask. Looks really weird. > > Also it has some weird file inside: > /usr/lib/python3.5/site-packages/flask/testsuite/test_apps/lib/python2.5/site-packages/SiteEgg.egg > > Should we skip such things from RPM generator or we should remove that > or what? Thoughts? Looks like a in the build system or packaging script. Any automatic tools (like the RPM generator) should probably just ignore those, just like Python itself does. Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org
Re: Inconsistencies in the Fedora Packaging Guidelines for Python
On Mon, Mar 21, 2016 at 05:35:03PM -0400, Neal Gompa wrote:
> On Mon, Mar 21, 2016 at 3:55 PM, Miro Hrončok wrote:
> > On 21.3.2016 20:13, Zbigniew Jędrzejewski-Szmek wrote:
> >>
> >> On Mon, Mar 21, 2016 at 06:46:00PM -, Tomas Orsava wrote:
> >>>
> >>> Since the spec file does package both p2 and p3 versions of the
> >>> executable
> >>
> >>
> >> There's a difference between *modules* (in the Python sense,
> >> i.e. Python libraries) and *executables*. We almost always want
> >> to provide modules for both Python versions, but executables
> >> only rarely so.
> >>
> >> The example spec file does *not* package both versions of the
> >> executable.
> >
> >
> > Yes, it does.
Ah, OK. It didn't at some point and I didn't check.
It seems that the Guidelines:Python page could still use some
editing. I think most of the info is there, but it's not very clear.
In particular, the multiple-executables case is again very prominent
(as this thread shows), and it's really applicable to a miniscule
percentage of packages (literally: sphinx, pytest, nosetest, a bunch
of python development and installation tools. There's a spattering
of other random packages, which might be mistakes. E.g. python-nibabel
also provides versioned executables, but that I don't think there's a
good reason for that). The way that Guidelines are written only serves
to confuse packagers.
> > %files -n python2-%{srcname}
> > %license COPYING
> > %doc README.rst
> > %{python2_sitelib}/*
> > %{_bindir}/sample-exec-2.7< HERE
> >
> > %files -n python3-%{srcname}
> > %license COPYING
> > %doc README.rst
> > %{python3_sitelib}/*
> > %{_bindir}/sample-exec< HERE
> > %{_bindir}/sample-exec-3.4< HERE
> >
> > https://fedoraproject.org/wiki/Packaging:Python#Example_common_spec_file
>
> I would suggest that the unversioned binary shouldn't necessarily be
> part of a versioned python package (like pythonX-), but I
> guess this is something that people expect these days anyway...
The alternative would to add yet another subpackage. Most of the
time that would be overkill. In the common case it works just fine
to put the binaries in py2 or py3 subpackage.
Zbyszek
___
python-devel mailing list
python-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org
Re: Inconsistencies in the Fedora Packaging Guidelines for Python
On Mon, Mar 21, 2016 at 06:46:00PM -, Tomas Orsava wrote: > Since the spec file does package both p2 and p3 versions of the executable There's a difference between *modules* (in the Python sense, i.e. Python libraries) and *executables*. We almost always want to provide modules for both Python versions, but executables only rarely so. The example spec file does *not* package both versions of the executable. Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org
Re: matplotlib 1.5.0 tagged
On Fri, Oct 30, 2015 at 01:33:49PM -0400, Neal Becker wrote:
> An initial srpm is here:
>
> https://nbecker.fedorapeople.org/python-cycler-0.9.0-1.fc22.src.rpm
This is so simple, let's get this done quickly.
1. Please change Source to
https://github.com/matplotlib/%{srcname}/archive/v%{version}.tar.gz#/python-%{srcname}.tar.gz
The archive on pypi is missing license. It seems more reasonable to use
github directly than to fight around that.
2. I think the summary should be something less cute. "Composable style
cycles" only works if you already know what the package does.
(I think that this package shows the power of the new Python macros from
bkabrda and others... The spec file is very readable and maintainable.)
If you submit a Review Request, I promise to hit the fedora-review+ button
promptly.
Zbyszek
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Stray .pyc files from tests
On Tue, Oct 20, 2015 at 05:53:45PM -0500, Jason L Tibbitts III wrote: > In the new python macros that some folks have been writing, we could > easily pass -B or export PYTHONDONTWRITEBYTECODE in the %check section. > Would this be a reasonable thing to do? I think it would, considering that we run bytecode compilation later on anyway. Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Stray .pyc files from tests
On Mon, Oct 19, 2015 at 10:11:12PM -0600, Orion Poplawski wrote:
> I just ran into an issue with a package under review where stray
> python2 .pyc files were getting put into the python3_sitelib
> directory. The tests were run with:
>
> PYTHONPATH=%{buildroot}%{python3_sitelib} py.test-%{python3_version} -vv
>
> and one of the tests ended up calling /usr/bin/python.setup.py
> --version triggering the loading and byte-compiling of the installed
> module. I don't know if this is common issue or not (I suspect
> not).
I think it happens regularly:
$ dnf whatprovides '/usr/lib64/python3.4/site-packages/*'|grep fc|cut -d' ' -f1
> /tmp/list
$ dnf repoquery -l $(cat /tmp/list)|grep -v __pycache__|grep
'python3.4.*py[co]$'|grep -v doc
/usr/lib64/python3.4/site-packages/arc/__init__.pyc
/usr/lib64/python3.4/site-packages/arc/__init__.pyo
/usr/lib64/python3.4/site-packages/arc/common.pyc
/usr/lib64/python3.4/site-packages/arc/common.pyo
/usr/lib64/python3.4/site-packages/arc/communication.pyc
/usr/lib64/python3.4/site-packages/arc/communication.pyo
/usr/lib64/python3.4/site-packages/arc/compute.pyc
/usr/lib64/python3.4/site-packages/arc/compute.pyo
/usr/lib64/python3.4/site-packages/arc/credential.pyc
/usr/lib64/python3.4/site-packages/arc/credential.pyo
/usr/lib64/python3.4/site-packages/arc/data.pyc
/usr/lib64/python3.4/site-packages/arc/data.pyo
/usr/lib64/python3.4/site-packages/arc/delegation.pyc
/usr/lib64/python3.4/site-packages/arc/delegation.pyo
/usr/lib64/python3.4/site-packages/arc/loader.pyc
/usr/lib64/python3.4/site-packages/arc/loader.pyo
/usr/lib64/python3.4/site-packages/arc/message.pyc
/usr/lib64/python3.4/site-packages/arc/message.pyo
/usr/lib64/python3.4/site-packages/arc/security.pyc
/usr/lib64/python3.4/site-packages/arc/security.pyo
/usr/lib64/python3.4/site-packages/cairo/__init__.pyc
/usr/lib64/python3.4/site-packages/cairo/__init__.pyo
I filed
https://bugzilla.redhat.com/show_bug.cgi?id=1273661
https://bugzilla.redhat.com/show_bug.cgi?id=1273662
but since this shouldn't cause any problems, it's a minor
issue. Probably not worth an automated check. There's some more
.py[co] files in documentation directories, which are probably not
useful either.
Zbyszek
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: Python guidelines cleanup
On Thu, Jul 30, 2015 at 07:34:48PM -0500, Jason L Tibbitts III wrote:
> After some additional discussion and cleanup, I've gone ahead and moved
> my drafts into place in the main guidelines. Please let me know if I've
> made any mistakes or typos or left out anything important.
There's a dead link in "Example common spec file".
> However, there is one thing I'm trying to understand about the new
> guidelines (which came from one of the submitted drafts, not something
> that I wrote). They say:
>
> "
> The following is a very simple spec file for a module building for both
> python2 and python3. It builds both versions in the same directory; this
> is possible because setuptools uses different build directories for
> different python versions and architectures. In addition, python3 will
> include the version of the interpreter in the names of generated files,
> so the build products don't conflict. (Of course this only works if a
> package builds for a single python2 version, which should always be the
> case in Fedora.)
> "
>
> Which is fine, except that I must be missing something about the second
> sentence. Setuptools in f22 and rawhide (which I know are different
> versions) seems to use "build" regardless of which python version is
> used to execute setup.py. Am I misunderstanding what that sentence is
> trying to tell me?
I wrote that. I turns out, which I didn't know at the time, that this is
only true for binary modules
(for example python-systemd has build/{lib,temp}.linux-x86_64-{2.7,3.4}).
For pure-python modules it seems to use a single build directory,
but separate build/scripts directories.
[ It seems that there's no nice way to tell setuptools/distutils/distribute
to always use separate build directories. It still uses the same build
directory even if 2to3 translation is specified in the setup.py file, which is
probably a bug. In a pinch, it is possible to override the build dir with
--build-lib. So something like this works, even if 2to3 is used:
%{__python2} setup.py build --build-lib build/2
%{__python3} setup.py build --build-lib build/3
Fortunately this is rarely necessary.]
So I think the detailed justification should be removed, and the first
paragraph of "Example common spec file" replaced with something like this:
"The following is a very simple spec file for a module building for
both python2 and python3. It builds both versions in the same
directory; this is possible because build products for different
versions of Python usually do not conflict."
> For the package I'm using for testing (python-requests) it turns out
> that the results of %py2_build and %py3_build are completely identical,
> but this might just be coincidence.
Zbyszek
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: python-sig and retiring python3-dateutil
On Tue, Jan 27, 2015 at 08:59:13AM -0500, Bohuslav Kabrda wrote:
> Packaging Python extension modules should keep working the way it
did up until F21. python-dateutil will still be the Python2 build,
python3-datetuil will still be the Python3 build. There are some
guidelines changes that I proposed for F22 [1], but I think they
don't influence dateutil.
python{,3}-dateutil have no executables, so this does not apply to
them at all.
Zbyszek
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel
Re: python-sig and retiring python3-dateutil
On Mon, Jan 26, 2015 at 10:57:25AM -0700, Pete Travis wrote: > On 01/26/2015 10:45 AM, Thomas Spura wrote: > > Pete Travis mailto:m...@petetravis.com>> schrieb am > > Mon Jan 26 2015 at 5:24:32 PM: > > > > On 01/26/2015 08:53 AM, Zbigniew Jędrzejewski-Szmek wrote: > > > Hi, > > > > > > since bug #1126521 seems to be progressing nicely, I think it > > would be > > > nice to get in touch with python-sig, the maintainers of > > > python3-dateutil, about retiring python3-dateutil and adding a > > python3 > > > subpackage to python-dateutil. They might want to do it the > > other way > > > around, which would be fine too, but either way, something should be > > > arranged. > > > > > > I tried to sign up for the python-sig mailing list, but it is > > > "private" and I haven't received any welcome letter, so I think I'm > > > stuck in some moderation queue. > > > > > > Zbyszek > > > > Yes, I think it's a good time to bring up retiring python3-dateutil. > > I've also requested membership for that list, and included the list > > owners here, maybe they can expedite our requests. > > > > > > As python-dateutil is not (yet) under the python-sig group > > maintainership [1], you could reach the maintainers of python-dateutil > > under ther -owner mail address (CC'ed). > > python-sig@fp.o is for all group maintained package maintainers and > > general python related questions are handled on python-devel@fp.o > > (also CC'ed). (Sorry for the confusion, the python-sig > > groupmaintainership started very recently and is still a work in process.) I applied for the group membership just in case. > > I'd say it depends, how python3 will be introduced in F22, which > > package should provide the subpackage and the other one should be retired. > > Maybe someone from [2] could comment on what they like to see in the > > future. I don't mind having two separate packages until we approach > > F22 and the future process is here. Building python3-du from python-du seems more standard, i.e. less confusing, but of course there's no difference from the technical side. No matter which name is chosen, it'd be nice to kill two birds with one stone, and do the merge together with an update of python3-dateutil to version 2.4. Scratch build is here: http://koji.fedoraproject.org/koji/taskinfo?taskID=8689481 > > [1] https://admin.fedoraproject.org/pkgdb/package/python-dateutil/ > > [2] http://fedoraproject.org/wiki/Changes/Python_3_as_Default > > We are proceeding with a planned Change[0] with Zbyszek acting on behalf > of the python-dateutil owner. He has already adapted > python-dateutil.spec to provide a python3-dateutil and, with the > relevant maintainers, we have been testing[1] dependent packages with a > scratch build of that package. > > The python3-dateutil package is around because upstream did not have > sources that would work with both anymore. That is no longer the case, > so the logical route to us is to retire python3-dateutil as a unique > srpm. The participation of they python-sig as package owners, and > insight on how to best transition, would be very helpful. > > My apologies if this is encroaching on python-sig territory; we saw a > need for change and took initiative. > > [0] https://fedoraproject.org/wiki/Changes/python-dateutil_2.x > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1126521 Zbyszek ___ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
