Re: Handling --enable-experimental-jit in Python 3.13

[Neal Gompa]

On Wed, Apr 10, 2024 at 2:23 PM Miro Hrončok  wrote:
>
> Hello Pythonistas,
>
> Python 3.13 has an experimental JIT compiler:
>
> https://docs.python.org/dev/whatsnew/3.13.html#experimental-jit-compiler
>
> Enabling it is a configure (hence build-time) option.
>
> How do we handle this in Fedora?
>
> - We can keep it disabled, as it is experimental.
> - We can enable it, but be ready to revert if it causes problems.
> - We can add yet another build variant, but we already have 4 of those
> (regular, debug, freethreading, freethreading-debug), so I'd rather not make 
> it
> 6 (or 8, if we include freethreading+jit combinations). I don't know yet if it
> would be co-installable.
>
> Opinions?
>

I lean toward just going ahead and shipping it and reverting if it
causes enough problems that cannot be fixed.



-- 
真実はいつも一つ！/ Always, there's only one truth!
--
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Rust bindings for Python (pyo3 versions <0.19, cpython) broken with Python 3.12

[Neal Gompa]

On Sun, Nov 12, 2023 at 12:35 PM Fabio Valentini  wrote:
>
> On Tue, Aug 1, 2023 at 5:43 PM Fabio Valentini  wrote:
> >
> > There are applications in Fedora that still rely on *ancient* versions
> > of PyO3, potentially affected by this:
> >
> > - cpython: mercurial
> > - pyo3 v0.15: fapolicy-analyzer, python-bcrypt, python-cryptography
> > - pyo3 v0.16: python-y-py
> > - pyo3 v0.17: unused compat packages, will be retired
> > - pyo3 v0.18: matrix-synapse
>
> Hello again.
>
> It is now three months later, and three packages have moved to the
> latest available version of pyo3:
>
> - python-cryptography
> - python-y-py
> - matrix-synapse
>
> That leaves two packages that are stuck on pyo3 v0.15:
>
> - fapolicy-analyzer
> - python-bcrypt
>
> I have now filed bugs against both packages that they need to move to
> pyo3 v0.19.2+ ASAP on both Fedora 39 and Rawhide.
> Due to ABI changes in Python 3.12, they are not guaranteed to even
> work correctly on Python 3.12.
>
> - https://bugzilla.redhat.com/show_bug.cgi?id=2249378
> - https://bugzilla.redhat.com/show_bug.cgi?id=2249381
>

The mainline git version of python-bcrypt has already been adapted for
python 3.12, they just haven't made a release yet, it seems?

https://github.com/pyca/bcrypt

That said, it doesn't look like it needed code changes to make that
work, so it should be possible to just bump things?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: I propose sphinx no longer Requires sphinxcontrib-{applehelp,devhelp,jsmath,htmlhelp,serializinghtml,qthelp}

[Neal Gompa]

On Wed, Nov 8, 2023 at 7:02 AM Miro Hrončok  wrote:
>
> On 08. 11. 23 12:57, Neal Gompa wrote:
> >> Packages that need those would need to BuildRequire them specifically, as
> >> proposed in:
> >>
> >>-https://src.fedoraproject.org/rpms/xca/pull-request/1
> >>
> >> -https://src.fedoraproject.org/rpms/python-django-haystack/pull-request/1
> >>-https://src.fedoraproject.org/rpms/pykka/pull-request/3
> >>-https://src.fedoraproject.org/rpms/extra-cmake-modules/pull-request/2
> >>
> >> The change has been offered upstream, but I suppose it will take a while 
> >> before
> >> it is actually landed there.
> >>
> >> Let me know if you think this needs more discussion before shipping it.
> >>
> >> (The change is only intended for Fedora 40+ and ELN.)
> >>
> > Do we know what the impact of this is going to be beyond those four?
>
> All Fedora packages that BuildRequire python3-sphinx (even transitively) were
> tested. Those that successfully built before this change are known not to be
> impacted, except for the listed 4 and
> https://src.fedoraproject.org/rpms/python-fastavro/pull-request/12 (which is
> special, because the removal only uncovered a bug in tests).
>
> This was tested in
> https://copr.fedorainfracloud.org/coprs/churchyard/sphinxcontrib-optional/builds/
> and
> https://copr.fedorainfracloud.org/coprs/churchyard/sphinxcontrib-mandaotry/builds/
>
> A handful of packages timed out after 5 hours in Copr, happy to fix them later
> if they they are impacted. Same for the packages that FTBFS now for unrelated
> reasons.
>

Then I'm fine with this, let's do it. :)


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: I propose sphinx no longer Requires sphinxcontrib-{applehelp,devhelp,jsmath,htmlhelp,serializinghtml,qthelp}

[Neal Gompa]

On Wed, Nov 8, 2023 at 6:56 AM Miro Hrončok  wrote:
>
> Hello Pythonistas.
>
> In https://src.fedoraproject.org/rpms/python-sphinx/pull-request/59 I propose 
> a
> change that would make the runtime requirements of:
>
>   - python3-sphinxcontrib-applehelp
>   - python3-sphinxcontrib-devhelp
>   - python3-sphinxcontrib-jsmath
>   - python3-sphinxcontrib-htmlhelp
>   - python3-sphinxcontrib-serializinghtml
>   - python3-sphinxcontrib-qthelp
>
> ...optional (Recommends) in python3-sphinx.
>
> Packages that need those would need to BuildRequire them specifically, as
> proposed in:
>
>   - https://src.fedoraproject.org/rpms/xca/pull-request/1
>   - https://src.fedoraproject.org/rpms/python-django-haystack/pull-request/1
>   - https://src.fedoraproject.org/rpms/pykka/pull-request/3
>   - https://src.fedoraproject.org/rpms/extra-cmake-modules/pull-request/2
>
> The change has been offered upstream, but I suppose it will take a while 
> before
> it is actually landed there.
>
> Let me know if you think this needs more discussion before shipping it.
>
> (The change is only intended for Fedora 40+ and ELN.)
>

Do we know what the impact of this is going to be beyond those four?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Need eyes on Django 4.2.3 PR

[Neal Gompa]

On Fri, Jul 21, 2023 at 4:31 PM Michel Alexandre Salim
 wrote:
>
> Dear all,
>
> I just put up a PR to update Django in Rawhide to 4.2.3:
>
> https://src.fedoraproject.org/rpms/python-django/pull-request/33
>
> Also - Fedora 37 and 38 are on Django 4.0.x, which is no longer supported -
> should we just update them to 4.2.x as well?
>
> Any version before 4.1.10 and 4.2.3 are affected by this CVE:
> https://bugzilla.redhat.com/show_bug.cgi?id=2219383
> https://nvd.nist.gov/vuln/detail/CVE-2023-36053
>
> NIST NVD gave it a base score of 7.5; and once we switch series anyway,
> maybe we might as well jump to 4.2 which is an LTS, while 4.1 reaches
> end of extended support in Dec 2023 (when Fedora 38 will still be
> supported)
>
> https://www.djangoproject.com/download/
>
> To update to 4.2, asgiref needs to be updated as well, but that seems to
> be the only dependency that is too old.
>
> If we decide against bumping Django on stable releases, we can see if
> the CVE fix can be easily backported to 4.0 or not.
>

Is there any reason why 4.2 would be incompatible with anything using
4.0? If not, then I'd lean toward upgrading things unless upgrading
asgiref would be too painful.

A quick query shows the following packages require asgiref:

ngompa@fedora ~> dnf -q repoquery --whatrequires "python3.11dist(asgiref)"
python3-daphne-0:3.0.2-4.fc38.noarch
python3-django-0:4.0.10-1.fc38.noarch
python3-django-0:4.0.2-6.fc37.noarch
python3-django3-0:3.2.18-1.fc38.noarch
python3-django3-0:3.2.19-1.fc38.noarch
python3-opentelemetry-instrumentation-asgi+instruments-1:0.38~b0-10.fc38.noarch
python3-opentelemetry-instrumentation-asgi+instruments-1:0.39~b0-12.fc38.noarch
python3-opentelemetry-instrumentation-asgi-1:0.38~b0-10.fc38.noarch
python3-opentelemetry-instrumentation-asgi-1:0.39~b0-12.fc38.noarch
python3-opentelemetry-test-utils-0:0.38~b0-1.fc38.noarch
python3-opentelemetry-test-utils-0:0.39~b0-1.fc38.noarch
python3-uvicorn-0:0.15.0-5.fc38.noarch

This might be fine or a bit much depending on how strict the dependencies are.




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Pre-review of a new python package: uswid

[Neal Gompa]

On Sat, Feb 4, 2023 at 10:32 AM Richard Hughes
 wrote:
>
> Many thanks all; I've fixed up all the issues I think and submitted an actual 
> review here: https://bugzilla.redhat.com/show_bug.cgi?id=2167067
>

Reviewed. :)



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Pre-review of a new python package: uswid

[Neal Gompa]

On Tue, Jan 31, 2023 at 1:12 PM Richard Hughes
 wrote:
>
> Hey all,
>
> I'm building python-uswid as a rpm as it's going to be needed by the 
> fwupd-efi package at build time in the near future. I'm also the upstream 
> maintainer, so I'm not against changing upstream and then tagging a new 
> release if there's something that needs to be fixed to build a Fedora 
> package. I'm no python expert, so advice very welcome.
>
> I've uploaded a srpm here and I'd appreciate some early 40,000ft checks 
> before I submit a Fedora package review: 
> https://people.freedesktop.org/~hughsient/temp/python-uswid-0.4.1-1.fc37.src.rpm
>  -- the upstream is https://github.com/hughsie/python-uswid/ for the curious.
>
> Thanks!
>

It mostly looks okay, but there's some spec details that need tuning
up, as I've written below:

> License:LGPLv2+

This should be "LGPL-2.1-or-later"

Cf. 
https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuidelines/#_valid_license_short_names

> Source0:
> https://files.pythonhosted.org/packages/af/b5/f935629332e1e3bf5bdc0d108c42d487df7ab6dfeafc7dfb35fd83fff0f1/%{srcname}-%{version}.tar.gz

You can simply this by using "%{pypi_source %{srcname}}"

Cf. 
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_201x/#_source_files_from_pypi

> %files -n %{name}

This needs to be python3-%{srcname}, and you need to define "%package
-n python3-%{srcname}"

Cf. 
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_201x/#_naming

You may optionally decide to split out the main binary into another
subpackage named "%{srcname}" if you choose. If you do, it needs a
hard exact dependency on "python3-%{srcname}".




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: What if we excluded 32bit ARM from Python 3.12

[Neal Gompa]

On Thu, Oct 13, 2022 at 5:08 AM Miro Hrončok  wrote:
>
> Hello Pythonistas,
>
> we are probably going to package python3.12 soon for all Fedora releases.
>
> Unfortunately, building Python for 32bit ARM has been very tedious lately, as
> the Koji build keeps restarting and the build takes 24+ hours to finish.
>
> Considering 32bit ARM is gone from Fedora 37+, I was considering the
> ExcludeArch it from the python3.12 package even on older Fedoras, to make the
> builds easier.
>
> Would anybody be sad about that?
>

I'm fine with it. The less ARM in my life, the better.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Retiring Python 3.7 before Python 3.6?

[Neal Gompa]

On Fri, May 27, 2022 at 11:35 AM Miro Hrončok  wrote:
>
> On 27. 05. 22 14:34, Neal Gompa wrote:
> > While unfortunate, I think it makes sense to retire Python 3.7 when
> > Debian 10 goes EOL.
>
> I don't understand why do you consider this unfortunate.
>

The situation is unfortunate, not the result.

> > I think the larger question is how do we write
> > down a multi-Python maintenance policy to codify this?
>
> It has been so far implicitly (but indeed not written) somehow like this:
>
> "We keep a Python version in Fedora as long as:
>
> 1) A "significant enough" platform that our users could be deploying into runs
> that Python version. That has always been at least RHEL, Debian and Ubuntu 
> LTS.
>
> 2) It is bearable for the maintainers."
>
> 1 means we drop it when it's no longer useful.
> 2 means we drop when we cannot longer make it work with the current resources.
>
> > The implicit
> > assumption here is that we're going to be able to leverage security
> > support work in other distributions to keep other Python runtimes
> > alive, but I don't think we've got that written down anywhere.
>
> At least for RHEL, we still use Fedora as upstream for our patches. So as long
> as a Python is supported in RHEL, we would (in most cases) try to fix Fedora
> together with RHEL.
>
> For upstream EOLed Pythons that are not in RHEL but are supported in other
> distros (e.g. Python 3.7 will  be supported in Debian for ~1 additional year
> after upstream EOL, but is not included in RHEL), we don't know yet (it never
> happened until now).
>
> > Moreover, we don't have anything written down for where to reference
> > and source security fixes across distributions for multi-Python
> > (assuming that's a goal here).
>
> Not necessarily my goal, but a nice goal nevertheless.
>
> > If we're not doing any of that, then I'm not sure why other
> > distributions matter for our multi-Python support beyond just making
> > it easier to target those distributions. And if that's true, then we
> > should have that codified too.
>
> Making it easier to use Fedora when you target those distributions was my
> primary goal here.
>

Makes sense to me.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Retiring Python 3.7 before Python 3.6?

[Neal Gompa]

On Fri, May 27, 2022 at 6:07 AM Miro Hrončok  wrote:
>
> Hey Pythonistas,
>
> let me include you in my dilemma I have wrt different Python versions we
> support in Fedora for testing.
>
> tl;dr Should we retire Python 3.7 before Python 3.6? 3.6 will stick around for
> RHEL 8, but 3.7 will no longer be "needed". However, retiring mid-order might
> be confusing.
>
> Note that this topic is not urgent at all, I am just curious about what is the
> best plan for the future.
>
>
> Python 3.7 retirement date
> ==
>
> Python 3.7 has security support upstream until June, 2023.
>
> It is also used in Debian 10 “Buster” -- LST ends June, 2024.
>
> Ubuntu seem to have skipped this Python version, so no LTS to consider there.
> Same with the SUSE Linux Enterprise Server.
>
> If nothing changes significantly, Fedora 40 will be released in April 2024,
> hence the natural thing to do would be to retire Python 3.7 in Fedora 40.
>
> (I might have made an error, evaluating the dates. Please, do correct me if
> there is a significant platform that supports Python 3.7 beyond Debian 10 LTS
> EOL or if some dates are wrong.)
>
>
> Python 3.6 retirement date
> ==
>
> Python 3.6 will be supported until RHEL 8 goes EOL. Which will likely not
> happen until 2029 [1]. That means, we might consider retiring it in Fedora 50
> (wow).
>
>
> My dilemma
> ==
>
> If we retire 3.7 before 3.6, the situation will be harder to explain to our
> users. Why is there a gap, they might ask. In the future, the list of 
> supported
> Python versions might look weird and contain multiple gaps as we are 
> eventually
> going to face this situation again with 3.8, 3.10...
>
> If we keep 3.7 around until 3.6 goes down, it will be harder to maintain. If 
> it
> was just 3.7, that would probably be acceptable, but assuming no release
> schedules change, we will likely have Python 3.18 in Fedora 50. That means we
> would need to support 12 different Python 3.X versions by then if we haven't
> started introducing those gaps. Now we support 6.
>
> As I written this down, I think the better thing to do is to accept the gaps
> and retire Python 3.7 before Python 3.6 right away and not decide to "keep it
> around, as we still support both 3.6 and 3.8". WDYT?
>
>
> [1] https://access.redhat.com/support/policy/updates/errata
>

While unfortunate, I think it makes sense to retire Python 3.7 when
Debian 10 goes EOL. I think the larger question is how do we write
down a multi-Python maintenance policy to codify this? The implicit
assumption here is that we're going to be able to leverage security
support work in other distributions to keep other Python runtimes
alive, but I don't think we've got that written down anywhere.
Moreover, we don't have anything written down for where to reference
and source security fixes across distributions for multi-Python
(assuming that's a goal here).

If we're not doing any of that, then I'm not sure why other
distributions matter for our multi-Python support beyond just making
it easier to target those distributions. And if that's true, then we
should have that codified too.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Python installation paths, pip, F36

[Neal Gompa]

On Thu, May 26, 2022 at 12:10 AM Owen Taylor  wrote:
>
> [ At Miro's request, resending this to python-devel so the discussion can be 
> public ]
>
> Hi Miro -
>
> When rebuilding a package to include in a Flatpak, we want to install 
> *everything* under prefix=/app - that includes Python modules. (Paths will be 
> adjusted properly when running the Flatpak.)
>
> For years, the way we did that is by installing a 
> /usr/lib64/python3.10/distutils/distutils.cfg:
> ===
> [install]
> prefix=/app
> ===
>
> This is part of the flatpak-rpm-macros package that gets installed in the 
> buildroot for Flatpak rebuilds
>
> In Fedora 36, this no longer works for 'pip install' - it seems that pip 
> unconditionally uses 'sysconfig' rather than 'distutils' to determine install 
> paths for Python >= 3.10.
>
> I see that you filed https://github.com/pypa/pip/issues/10647 which landed in 
> pip for pip-22, but Fedora 36 / pip-21 this isn't present? And in any case, 
> that seems like it's something for 'sudo pip install' rather than package 
> installation. I actually don't really follow how the Python macros are 
> getting things installed into /usr/lib rather than /usr/local/lib.
>
> Any advice about how we can make the installation into /app work? Hopefully 
> in a future-proof way...
>

What's stopping us from upgrading pip in F36? It's packaged separately
from Python itself already...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Splitting alternative Python packages into subpackages, e.g. python3.11{,libs,devel,...}

[Neal Gompa]

On Fri, Mar 18, 2022 at 4:28 PM Miro Hrončok  wrote:
>
> On 16. 03. 22 17:12, Tomáš Orsava wrote:
> > Hi Python-devel,
> > we are considering splitting the alternative Python versions from a
> > single-package format (e.g. python3.11) to multiple subpackages (e.g.
> > python3.11{,-libs,-devel,-tkinter,-test,-idle}). We do this already with the
> > main `python3` package: it requires less disk space to install and speeds up
> > download times, because you can chose which bits are important to you. For
> > example, if you decide you don't need python3-tkinter, you save yourself ~18
> > dependent packages leading to a total savings of ~20MBs, while skipping
> > python3-test saves you further ~10MBs.
> >
> > What do you think?
> >
> > The push came from [BZ#2063227] where the reporters would welcome to have a
> > smaller python3.11 package for containers and VMs for local testing, CI
> > purposes and more.
> > This would be a larger amount of work, so our initial reaction was hesitant.
> > We'll have to change the already complicated spec file %bcond logic, and 
> > adjust
> > the ecosystem to work with the new subpackages. For example tox would need 
> > to
> > Recommend `python3.11-devel`, as `python3.11` would bring in only the bare
> > interpreter. And of course a thorough integration testing would be in order.
> >
> > However, we already do separate subpackages for alternative stacks in
> > Enterprise Linux (CentOS /Stream, RHEL, EPEL) and as a general rule we 
> > consider
> > it good to have fewer differences between Fedora and EL. This helps to test
> > things earlier, and there are fewer surprises in user experience. So perhaps
> > the effort in doing this would be well spent.
> >
> > To cut down on the amount of work, we're considering changing only the
> > `python3.11` package (and any future newer versions) right now. If later we
> > consider it worth it, we could switch the older alternative interpreters as
> > well, or we might let them die out as they are.
> >
> > We're currently in the brainstorming stage, so you're feedback is welcome.
> >
> > [BZ#2063227] https://bugzilla.redhat.com/show_bug.cgi?id=2063227
>
> Note that I was originally against this because I though it would make things
> more complex/complicated and the "flat" Python packages are simple.
>
> However, we build the "flat" package vs. the "split" package based on a %bcond
> -- the spec is needlessly complex/complicated already and this could make it
> much simpler (so it would no longer resemble a nuclear power plant).
>
> My only remaining fear was that it will be a huge amount of work across all 
> our
> interpreters, but the idea to only do it in 3.11+ for now is intriguing. (In
> fact, I'd do it in 3.10+ so when we update main python to 3.10 in Fedora 37,
> Python 3.10 remains split.)
>
> Hence, I am in favor of this idea. Let's see how much cleanup this can bring 
> to
> the spec file and how much integration work would be needed (we use reverse
> weak deps in tox, so it should be painless).
>
> Thanks, Tomáš, for writing it down.
>

I'm in favor of it myself as well. I brought it up a while back on
this list last year even:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/WNNAZWWHDU7LE4EJBDKREJO5FJQ6SXRX/

So I'd love to see this happen.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Is missing python3-docs in EL9 a problem?

[Neal Gompa]

On Sat, Mar 5, 2022 at 1:35 PM Orion Poplawski  wrote:
>
> I've been poking at some packages that BR python3-docs, like
> python-zope-event.  This apparently comes from a sphinx inventory:
>
> # Use local objects.inv for intersphinx
> sed -i "s|\('https://docs\.python\.org/':
> \)None|\1'%{_docdir}/python3-docs/html/objects.inv'|" docs/conf.py
>
> If I build without python3-docs present I get the following message but
> the build succeeds:
>
> Running Sphinx v3.4.3
> making output directory... done
> loading intersphinx inventory from
> /usr/share/doc/python3-docs/html/objects.inv...
> WARNING: failed to reach any of the inventories with the following issues:
> intersphinx inventory '/usr/share/doc/python3-docs/html/objects.inv' not
> fetchable due to : [Errno 2] No such file or
> directory: '/usr/share/doc/python3-docs/html/objects.inv'
>
>
> Is this a problem or can we safely ignore it?  Is there a better way to
> remove the intersphinx dep?
>
> If not, how best to get python3-docs into EL9?
>

File a BZ against python3 in RHEL 9 to ask for it to be shipped.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: python-flit for EPEL9?

[Neal Gompa]

Top posting because replying from my phone, but I think it'd be great to
have in EPEL 9 because it's essential for some packages to build. The same
goes for Poetry, which is now used for software made by Fedora
Infrastructure these days.

On Thu, Dec 23, 2021, 3:10 PM Michel Alexandre Salim <
sali...@fedoraproject.org> wrote:

> I'm starting to try and grok pyproject.toml and its assorted PEPs in
> more details, and it looks like flit is one of the more straightforward
> way to build compliant wheels out there.
>
> Should we get it into EPEL9? I filed a tracking bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=2035376
>
> looks like it's retired in CS9 so it should be fine for us to have.
>
> https://gitlab.com/redhat/centos-stream/rpms/python-flit/-/blob/main/dead.package
>
> It currently does not build because of some missing dependencies - I
> listed them in the bug above - if we do want this in EPEL9 I'll file
> those bug requests and make them block this.
>
> Also, to note, the EPEL maintainer is currently set to orphan. While I'm
> in python-sig so I can branch and build just fine as is, Miro, as the
> owner could you add me to the ACL and/or set me as the EPEL assignee?
>
> We could probably add epel-packagers-sig as collaborator on epel*
> branches too.
>
> Best regards,
>
> --
> Michel Alexandre Salim
> profile: https://keyoxide.org/mic...@michel-slm.name
>
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Python RPM dependency generators and LegacyVersion, time to error?

[Neal Gompa]

On Thu, Nov 4, 2021 at 7:50 AM Miro Hrončok  wrote:
>
> Hello Pythonistas.
>
> After some recent improvements in the Python RPM dependency generators, a
> regression was discovered [0].
>
> Turns out the error happened when the upstream metadata contained a 
> requirement
> with a PEP 440 [1] incompatible version. A fix is pending and only one package
> in Fedora was affected.
>
> However, this brings me to a question: What shall we do with such incompatible
> versions in the future? Implementation-wise, the generators use the
> pypa/packaging library to parse the requirements. When a PEP 440 incompatible
> version is found, it's parsed as a LegacyVersion [3]. A LegacyVersion has been
> deprecated for 1 year now and when found it might indicate a packaging 
> problem.
>
> I've experimented with a patched version of the generators that would error 
> out
> right away when a LegacyVersion is parsed. I found the following categories of
> problems:
>
> 1) A requirement uses invalid/legacy version (2 packages):
>
> python-celery: pytz>dev
>
> This is a pip-compatible shortcut for 0.dev.0, which is only required as a 
> hack
> for PEP 440 incompatible versions of pytz from before 2013 (2012a, 2012b,
> 2012c...).
>
> python-pvc: pyvmomi>=5.5.0-2014.1.1
>
> This is an existing PEP 440 incompatible version of pyvmomi that has been 
> since
> replaced with PEP 440 compatible version on PyPI.
>
> 2) The package itself has an invalid/legacy version (3 packages):
>
> python-haversion: main
> python-ipmi: unknown
> python-lacrosse: unknown
>
> This is most likely a packaging problem introduced by the Fedora packager by
> using a GitHub tarball whereas the version information is only stored in 
> sdist.
> I believe the generators should error out immediately when this happens to
> prevent packaging errors, but there might be a need to allow it to proceed if
> we want to package e.g. pyvmomi 5.5.0-2014.1.1 or pytz 2012d.
>
>
> tl;dr:
>
>   - there might be cases where legacy versions are needed
>   - but in most cases, they should be avoided
>
> Hence, I propose we do the following in Rawhide:
>
> We turn LegacyVersions to failures, but we provide a stop-gap measure, such as
> (%global python_dependency_allow_legacy_version_provides 1 /  %global
> python_dependency_allow_legacy_version_requires 1 ) for packages that need to
> override this. When pypa/packaging actually drops LegacyVersion, this stop-gap
> measure will no longer work.
>
>
> [0] https://bugzilla.redhat.com/show_bug.cgi?id=2019954
> [1] https://www.python.org/dev/peps/pep-0440/
> [2] https://packaging.pypa.io/
> [3]
> https://packaging.pypa.io/en/latest/version.html#packaging.version.LegacyVersion
>

This makes sense to me, but would it also make sense to have some
documentation about this case and how to resolve it to PEP 440
compatible versions, too?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Draft of New Python Packaging Guidelines

[Neal Gompa]

On Tue, Jun 15, 2021 at 5:35 AM Miro Hrončok  wrote:
>
> On 14. 06. 21 7:31, Zbigniew Jędrzejewski-Szmek wrote:
> >> BuildRequires: %{py3_dist pytest}
> > Does it make sense to recommend py3_dist here? python3dist(pytest) is
> > not more complex but can be fed to 'dnf install' directly, so in the
> > end it's more flexible. I always find the macro more obfuscation than
> > anything for package names which don't require normalization.
>
> Replaced with python3dist(pytest). The %py3_dist remains documented in the
> guidelines but is not longer used in examples like this (this was the only
> instance).
>

The case where we'd use macros would be if we want to extend for
building for multiple Pythons from one spec file again (which is
something that'd be nice to do in the future)...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Is there a reason to do Python runtime flatpackages anymore?

[Neal Gompa]

Hey all,

Things have changed in Python runtime packaging since we started
introducing alternative Python versions years ago. For one, we now
always have fully versioned source packages, and now we have a flag
for whether the packages are "main runtime" vs "alternate runtime".
Another is that RHEL now offers multiple Python runtimes that you can
build packages from.

I'm wondering if it makes sense to continue having the logic in the
Python runtime packaging for "flatpackage" when we can now just have
them build as alternative runtimes. This doesn't get rid of the
concept of a "main runtime" that is generally supported by the macros,
but it brings us closer in line with what our downstreams are doing.
This could also ease Python transitions in the future, since we
wouldn't have the Python runtime ripped out from under us for DNF as
we rebootstrap the whole environment to a new Python version default.

At least for me, it would also make it easier for me to trivially
rebuild packages in COPR against an alternate Python version for
specific purposes, too, since the only required change to switch to an
alternate runtime would be setting %__python/%__python3 and making
sure the subpackage has the fully qualified Python version name in it.

What do you all think? Is this crazy talk or something we might want
to think about?

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Macronize %package -n python3-foo?

[Neal Gompa]

On Tue, Mar 16, 2021 at 5:32 AM Miro Hrončok  wrote:
>
> Hello Pythonistas.
>
>
> I find myself cop-pasting this boring snippet each time I create a Python
> package (using the old macros or the new):
>
>
>%package -n python3-foo
>Summary:%{summary}
>
>%description -n python3-foo %_description
>
> And using one of those in %files:
>
>%files -n python3-foo
>%files -n python3-foo -f %{pyproject_files}
>
> I wonder whether it makes sense to macronize this.
>
> For example:
>
>Name: python-foo
>...
>
>%global _description %{expand:
>This is the description for both SRPM and the python3-foo package.}
>
>%description %_description
>
>%py3_package %_description
>
>...
>
>%py3_files
>...
>
>
> Or maybe even (if possible):
>
>Name: python-foo
>...
>
>%py3_package_with_description
>This is the description for both SRPM and the python3-foo package.
>
>...
>
>%py3_files
>...
>
>
> Both macros would figure the package name by replacing the python- prefix from
> %{name} with python3-.
>
>
> Pros: No more copy-paste-edit \o/
>
> Cons: The more is hidden from the reader behind automagic macros, the less
> obvious is the spec file to somebody who tries to read or modify it :(
>
>
> What is your opinion?

I would actually like this, and make it easy for us to build for
multiple Pythons (even if we don't do it by default in Fedora). This
is how openSUSE works with their "singlespec" model[1], and I have
come to appreciate it, even though I don't like some of the details of
the openSUSE implementation (such as weird parsing and rewriting of
stanzas for dependencies instead of the use of a generator and the
usage of alternatives, as shown in this example[2]).

[1]: https://en.opensuse.org/openSUSE:Packaging_Python#What_is_single-spec.3F
[2]: 
https://build.opensuse.org/package/view_file/devel:languages:python:flask/python-flask-oidc/python-flask-oidc.spec?expand=1



--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: package request: PyWaffle

[Neal Gompa]

On Fri, Feb 19, 2021 at 4:57 PM Matthew Miller  wrote:
>
> Hi! I'm using this https://github.com/gyli/PyWaffle for some visualizations 
> for Fedora Project stats.
>
> I'm kind of out of the loop on the state of the art of python packaging, and 
> wondered if some kind Python SIG person would like to take it on for me.

It looks like it's just a classic setuptools build, so the same
packaging style that's been in use for ~5+ years should work fine
here.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Let's retire Python 3.5 from Fedora 35+ ?

[Neal Gompa]

On Tue, Dec 29, 2020 at 8:47 AM Miro Hrončok  wrote:
>
> On 29. 12. 20 14:27, Neal Gompa wrote:
> > On Tue, Dec 29, 2020 at 8:18 AM Miro Hrončok  wrote:
> >>
> >> Hello,
> >> I plan to retire Python 3.5 from Fedora 35+:
> >>
> >> https://fedoraproject.org/wiki/Changes/RetirePython3.5
> >>
> >> Let me know if there is some reason to extend the deadline to a later 
> >> release.
> >>
> >
> > No reason to extend it, but I would also note that Ubuntu 16.04 LTS
> > EOL is in April, and that's the primary reason that people used Python
> > 3.5 at all. It's basically the reason I deal with it at all for work
> > (since work used Ubuntu 16.04 everywhere and we're moving to 20.04
> > now...).
>
> Done. Ubuntu 16.04 has some extended support trough 2024, but I'm not sure 
> this
> counts. Do you know any details regarding that?
>

That's minimal security updates for a subset of Ubuntu 16.04 LTS if
you pay for it. I'm not sure if Python is covered in that subset. It's
functionally equivalent to the RHEL EUS thing if you give Red Hat a
bag of money for post-EOL support.

For all practical purposes, Ubuntu 16.04 is EOL. Ubuntu 20.04 LTS is
EOL in 2025 unless you pay, too.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Let's retire Python 3.5 from Fedora 35+ ?

[Neal Gompa]

On Tue, Dec 29, 2020 at 8:18 AM Miro Hrončok  wrote:
>
> Hello,
> I plan to retire Python 3.5 from Fedora 35+:
>
> https://fedoraproject.org/wiki/Changes/RetirePython3.5
>
> Let me know if there is some reason to extend the deadline to a later release.
>

No reason to extend it, but I would also note that Ubuntu 16.04 LTS
EOL is in April, and that's the primary reason that people used Python
3.5 at all. It's basically the reason I deal with it at all for work
(since work used Ubuntu 16.04 everywhere and we're moving to 20.04
now...).




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

RFC: Mangle Python shebangs to fully qualified interpreter (/usr/bin/pythonX.Y)

[Neal Gompa]

Hey all,

An "interesting" Linux support issue at work with another Linux
distribution opened my eyes to the possibility of how badly things
would break if /usr/bin/python3 was swapped or otherwise altered by an
external force. In such a scenario where it was overridden to point to
another Python 3 interpreter that wasn't the system one, things would
break pretty badly.

Now that we've been moving toward a model where it becomes possible
for first-party or third-party multi-Python RPMs, I'm wondering if it
would make sense to evolve the shebang mangling for Python to resolve
to the fully qualified Python interpreter path (that is,
/usr/bin/pythonX.Y instead of /usr/bin/pythonX) for packaged Python
code. Presumably this may make things easier for switching Python
versions without everything breaking, too.

In practice, I don't think this is a *huge* issue Fedora-side, since
we don't manage /usr/bin/python3 via alternatives. But I can easily
see people wanting /usr/bin/python3 to be swappable in Red Hat
Enterprise Linux, while everything is still parallel installable and
fully functional. The Python interpreter packaging even mostly
supports this now.

What do y'all think? Is this doable? Is this desirable?


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Python spec template violates guidelines?

[Neal Gompa]

On Wed, Aug 26, 2020 at 6:25 PM Michel Alexandre Salim
 wrote:
>
> On Wed, 2020-08-26 at 15:16 -0700, Michel Alexandre Salim wrote:
> > On Wed, 2020-08-26 at 20:23 +0200, Miro Hrončok wrote:
> > >
> > > I don't really know who maintains `rpmdev-newspec python-foo` but
> > > the
> > > output
> > > (when I run this on Fedora 32) is really severely outdated beyond
> > > being any
> > > useful. I remember trying to update the template many years ago but
> > > got stuck at
> > > EPEL-compatibility issues.
> > >
> > Neal maintains rpmdevtools, both on src.fp.o and on Pagure
> >
> > I'll get a PR in to make the Python spec a bit more up to date. This
> > brings to mind that I've been intendeing to have a spec template and
> > a
> > packaging guideline for Lua for a while; that can go next.
> >
> So:
> F32 has rpmdevtools 8.10, which was first released... over three years
> ago
>
> * Sat Jan 14 2017 Ville Skyttä  - 8.10-1
>
> The spec in pagure is updated (including a patch Miro committed last
> year), but looks like it's only built for F33 and F34 right now.
>
> Neal, we should be able to release this for F32, right? Most packagers
> are likely to still be on it rather than on the recently branched F33
> so it probably would save some review time.
>

I am not releasing rpmdevtools 9.x to F32 or older because I've
changed default behaviors for 9.0 and replaced the spectool
implementation.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Draft of New Python Packaging Guidelines - 0.2

[Neal Gompa]

On Wed, Aug 12, 2020 at 12:02 PM Petr Viktorin  wrote:
>
> On 2020-08-12 17:22, Neal Gompa wrote:
> > On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin  wrote:
> >>
> >> On 2020-08-12 16:53, Neal Gompa wrote:
> >>> On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin  
> >>> wrote:
> >>>>
> >>>> I'll move some discussion here, where it doesn't become part of the
> >>>> document:
> >>>>
> >>>>>
> >>>>> On 2020-08-11 14:19, Petr Viktorin wrote:
> >>>>>> These Guidelines represent a major rewrite and paradigm shift, and not
> >>>>>> all packages are updated to reflect this.
> >>>>>> Older guidelines are still being kept up to date, and existing packages
> >>>>>> **MAY** use them instead of this document:
> >>>>>> * 201x-era [Python packaging
> >>>>>> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
> >>>>>> (for packages that use e.g. `%py3_install` or `setup.py install`)
> >>>>>> * [Python 2
> >>>>>> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
> >>>>>> (for e.g. `%py2_install`) (Python 2 packages require a FESCo 
> >>>>>> exception.)
> >>>>
> >>>> @Conan-Kudo:
> >>>>> Is there something you've done here to make the new macros 
> >>>>> unequivocally better? Do you have automated flavor builds, for example? 
> >>>>> Right now there is no effective difference.
> >>>>
> >>>>
> >>>> The new macros allow other build tools than just setuptools.
> >>>> They also use upstream metadata for BuildRequires & file lists.
> >>>>
> >>>> Of course, where possible the changes were backported to the existing
> >>>> macros. I don't necessarily see the macros as the main thing that
> >>>> changes. But when you look at a specfile, you can usually tell what
> >>>> conventions it uses by what macros you see.
> >>>>
> >>>
> >>> Those are all nice things, for sure. I think it'd be important to
> >>> spell that out somewhere.
> >>
> >> Do you mean providing a summary of the changes between the existing
> >> guidelines and these?
> >>
> >
> > Yes. Also indicating the *why* for pyproject stuff is useful within
> > the context of why the old macros are deprecated.
>
> Is that necessary for the beta release of the packaging guidelines?
> (i.e. when they would be in effect, but optional?)
>
> You're asking for quite a lot of work around describing a document that
> might still change.
>
>

I am only asking for what makes sense if the document as it stands
were to be finalized.

> >>>>>> ## PyPI parity
> >>>>>>
> >>>>>> Every Python package in Fedora **SHOULD** also be available
> >>>>>> on [the Python Package Index](https://pypi.org) (PyPI).
> >>>>>>
> >>>>>> The command `pip install PROJECTNAME` **MUST** install the same package
> >>>>>> (possibly in a different version),
> >>>>>> install nothing,
> >>>>>> or fail with a reasonable error message.
> >>>>>>
> >>>>>> If this is not the case, the packager **MUST** contact upstream about 
> >>>>>> this.
> >>>>>> The goal is to get the project name registered or blocked on PyPI,
> >>>>>> or to otherwise ensure the rule is followed.
> >>>>>>
> >>>>>> To block a project name on PyPI, email
> >>>>>> [ad...@pypi.org](mailto:ad...@pypi.org),
> >>>>>> giving the project name and explaining the situation
> >>>>>> (for example: the package cannot currently be installed via `pip`).
> >>>>>> You can ask questions and discuss the process at the [Python
> >>>>>> Discourse](https://discuss.python.org/t/block-names/4045).
> >>>>>>
> >>>>>>> NOTE: Project names that were in Fedora but not on PyPI
> >>>>>>> when these guidelines were proposed
> >>>>>>> are *blocked* from being uploaded to PyPI.
> >>>>>>> This prevents potential trolls from ta

Re: Draft of New Python Packaging Guidelines - 0.2

[Neal Gompa]

On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin  wrote:
>
> On 2020-08-12 16:53, Neal Gompa wrote:
> > On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin  wrote:
> >>
> >> I'll move some discussion here, where it doesn't become part of the
> >> document:
> >>
> >>>
> >>> On 2020-08-11 14:19, Petr Viktorin wrote:
> >>>> These Guidelines represent a major rewrite and paradigm shift, and not
> >>>> all packages are updated to reflect this.
> >>>> Older guidelines are still being kept up to date, and existing packages
> >>>> **MAY** use them instead of this document:
> >>>> * 201x-era [Python packaging
> >>>> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
> >>>> (for packages that use e.g. `%py3_install` or `setup.py install`)
> >>>> * [Python 2
> >>>> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
> >>>> (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
> >>
> >> @Conan-Kudo:
> >>> Is there something you've done here to make the new macros unequivocally 
> >>> better? Do you have automated flavor builds, for example? Right now there 
> >>> is no effective difference.
> >>
> >>
> >> The new macros allow other build tools than just setuptools.
> >> They also use upstream metadata for BuildRequires & file lists.
> >>
> >> Of course, where possible the changes were backported to the existing
> >> macros. I don't necessarily see the macros as the main thing that
> >> changes. But when you look at a specfile, you can usually tell what
> >> conventions it uses by what macros you see.
> >>
> >
> > Those are all nice things, for sure. I think it'd be important to
> > spell that out somewhere.
>
> Do you mean providing a summary of the changes between the existing
> guidelines and these?
>

Yes. Also indicating the *why* for pyproject stuff is useful within
the context of why the old macros are deprecated.

> >>>> ## PyPI parity
> >>>>
> >>>> Every Python package in Fedora **SHOULD** also be available
> >>>> on [the Python Package Index](https://pypi.org) (PyPI).
> >>>>
> >>>> The command `pip install PROJECTNAME` **MUST** install the same package
> >>>> (possibly in a different version),
> >>>> install nothing,
> >>>> or fail with a reasonable error message.
> >>>>
> >>>> If this is not the case, the packager **MUST** contact upstream about 
> >>>> this.
> >>>> The goal is to get the project name registered or blocked on PyPI,
> >>>> or to otherwise ensure the rule is followed.
> >>>>
> >>>> To block a project name on PyPI, email
> >>>> [ad...@pypi.org](mailto:ad...@pypi.org),
> >>>> giving the project name and explaining the situation
> >>>> (for example: the package cannot currently be installed via `pip`).
> >>>> You can ask questions and discuss the process at the [Python
> >>>> Discourse](https://discuss.python.org/t/block-names/4045).
> >>>>
> >>>>   > NOTE: Project names that were in Fedora but not on PyPI
> >>>>   > when these guidelines were proposed
> >>>>   > are *blocked* from being uploaded to PyPI.
> >>>>   > This prevents potential trolls from taking them,
> >>>>   > but it also blocks legitimate owners.
> >>>>   > If your package is affected, contact the Python SIG
> >>>>   > or [file a PyPA
> >>>> issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> >>>>
> >>>>   > and mention `@encukou`.
> >>>>
> >>>> This is necessary to protect users,
> >>>> avoid confusion,
> >>>> and enable automation as Fedora and upstream ecosystems grow more
> >>>> integrated.
> >>>>
> >>>> As always, [specific exceptions can be granted by the Packaging
> >>>> Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
> >>
> >>
> >> @Conan-Kudo:
> >>> This is not reasonable to ask of packagers to do. Such a check is 
> >>> difficult to do, and the

Re: Draft of New Python Packaging Guidelines - 0.2

[Neal Gompa]

On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin  wrote:
>
> I'll move some discussion here, where it doesn't become part of the
> document:
>
> >
> > On 2020-08-11 14:19, Petr Viktorin wrote:
> >> These Guidelines represent a major rewrite and paradigm shift, and not
> >> all packages are updated to reflect this.
> >> Older guidelines are still being kept up to date, and existing packages
> >> **MAY** use them instead of this document:
> >> * 201x-era [Python packaging
> >> guidelines](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/)
> >> (for packages that use e.g. `%py3_install` or `setup.py install`)
> >> * [Python 2
> >> appendix](https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/#_python_2_packages)
> >> (for e.g. `%py2_install`) (Python 2 packages require a FESCo exception.)
>
>
> @Conan-Kudo:
> > Is there something you've done here to make the new macros unequivocally 
> > better? Do you have automated flavor builds, for example? Right now there 
> > is no effective difference.
>
>
> The new macros allow other build tools than just setuptools.
> They also use upstream metadata for BuildRequires & file lists.
>
> Of course, where possible the changes were backported to the existing
> macros. I don't necessarily see the macros as the main thing that
> changes. But when you look at a specfile, you can usually tell what
> conventions it uses by what macros you see.
>

Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.

> >
> >> ## PyPI parity
> >>
> >> Every Python package in Fedora **SHOULD** also be available
> >> on [the Python Package Index](https://pypi.org) (PyPI).
> >>
> >> The command `pip install PROJECTNAME` **MUST** install the same package
> >> (possibly in a different version),
> >> install nothing,
> >> or fail with a reasonable error message.
> >>
> >> If this is not the case, the packager **MUST** contact upstream about this.
> >> The goal is to get the project name registered or blocked on PyPI,
> >> or to otherwise ensure the rule is followed.
> >>
> >> To block a project name on PyPI, email
> >> [ad...@pypi.org](mailto:ad...@pypi.org),
> >> giving the project name and explaining the situation
> >> (for example: the package cannot currently be installed via `pip`).
> >> You can ask questions and discuss the process at the [Python
> >> Discourse](https://discuss.python.org/t/block-names/4045).
> >>
> >>  > NOTE: Project names that were in Fedora but not on PyPI
> >>  > when these guidelines were proposed
> >>  > are *blocked* from being uploaded to PyPI.
> >>  > This prevents potential trolls from taking them,
> >>  > but it also blocks legitimate owners.
> >>  > If your package is affected, contact the Python SIG
> >>  > or [file a PyPA
> >> issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541=pep541-request.md=PEP+541+Request%3A+PROJECT_NAME)
> >>
> >>  > and mention `@encukou`.
> >>
> >> This is necessary to protect users,
> >> avoid confusion,
> >> and enable automation as Fedora and upstream ecosystems grow more
> >> integrated.
> >>
> >> As always, [specific exceptions can be granted by the Packaging
> >> Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
>
>
> @Conan-Kudo:
> > This is not reasonable to ask of packagers to do. Such a check is difficult 
> > to do, and there is no particularly compelling reason for making everything 
> > written in Python using Python build system available in PyPI. Unless you 
> > plan to provide an automated system to inform PyPI of these things, this is 
> > not only unreasonable, it is unenforceable.
>
> A lot of stuff in the guidelines is unenforceable, so let's focus on the
> "unreasonable" part.
>
> There is no reason to have everything available on PyPI, but I do
> believe it's reasonable to *reserve the name* in such cases.
>
> Here's a reason why I want this:
>
> The issue here is that Python tools have access to project names. For
> example, I can get the version of Requests using:
>
>  >>> from importlib.metadata import version
>  >>> version('requests')
> '2.22.0'
>
> Things like this are only useful if we use a common namespace. Upstream,
> that namespace *is* PyPI; there's little we can do about that.
> If project names mean something else in Fedora than in the wider
> ecosystem, we'll get user confusion at best.
>
> (If you use a private package index, like they do at CERN or some
> closed-source shops, there can be some collisions -- but in that case
> there's a limited number of software authors and users, and a lot more
> control over the package set. Conflicts in global repositories of
> free/open-source software are much harder to manage.)
>
>
>
> Lately, I think about another way to handle this: packages that aren't
> on PyPI could not ship the .dist-info at all, and so, they would not
> have things like `python3dist(...)` provides. They'd only be usable with
> Fedora tooling, not 

Fwd: [pypa/pip] New Resolver: Rollout, Feedback Loops and Development Flow (#6536)

[Neal Gompa]

-- Forwarded message -
From: Sumana Harihareswara 
Date: Thu, Jul 30, 2020 at 11:36 AM
Subject: Re: [pypa/pip] New Resolver: Rollout, Feedback Loops and
Development Flow (#6536)
To: pypa/pip 
Cc: Neal Gompa (ニール・ゴンパ) , Comment <
comm...@noreply.github.com>


Per #8511 <https://github.com/pypa/pip/issues/8511> we have now released *pip
20.2*. This release includes the *beta of the next-generation dependency
resolver*. It is significantly stricter and more consistent when it
receives incompatible instructions, and reduces support for certain kinds
of constraints files, so some workarounds and workflows may break. *Please
test it* with the --use-feature=2020-resolver flag. Please see our guide on
how to test and migrate, and how to report issues
<https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020>.
The new dependency resolver is *off by default* because it is *not yet
ready for everyday use*.

We plan to make pip's next quarterly release, 20.3, in October 2020. *We
are preparing to change the default dependency resolution behavior and make
the new resolver the default in pip 20.3.*

Please *spread the word* by pointing to this blog post
<https://blog.python.org/2020/07/upgrade-pip-20-2-changes-20-3.html> --
spread the word on Hacker News, Reddit, Twitter, Facebook, Dev.to,
Telegram, relevant Stack Overflow answers, and your favorite Slacks and
Discords. Most of the people this will affect do not keep up with
Python-specific developer news. Help them get the heads-up before October,
and help us get their bug reports.

(Copying my note from #988 <https://github.com/pypa/pip/issues/988>.)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://github.com/pypa/pip/issues/6536#issuecomment-666464812>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACDSPEX6J7RKU4J4V4X2UDR6GHOJANCNFSM4HPTXQ3A>
.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Redesigning the %python_provide macro from scratch

[Neal Gompa]

On Tue, Apr 28, 2020 at 5:00 AM Petr Viktorin  wrote:
>
> I finally got around to this mail...
>
> On 2020-04-19 16:55, Miro Hrončok wrote:
> >
> > As always, this leaves us with the name problem, but I'd very much like
> > to use %python_provides (note the s). The only problem I see is that it
> > is likely to be mistaken for the old one, but IMHO it shouldn't really
> > hurt that much.
>
> I guess something like %py_provides would also work, and maybe fit more
> nicely with the new generation of macros. Do you like full "python" prefix?
> But that's bikeshedding.
>

I know it's a bit of bikeshedding, but I think it's weird how
inconsistent we are with using py_* vs python_*. I'd rather us just
fully switch over to using python_* prefix everywhere.

It's annoying and I'd rather be verbose and clear rather than terse
and confusing.




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Renaming pythonXY packages to pythonX.Y (e.g. python39 to python3.9)

[Neal Gompa]

On Wed, Apr 29, 2020 at 10:28 AM Tomas Orsava  wrote:
>
> Hello everyone.
> I’m working on a change to rename pythonXY packages to pythonX.Y, e.g. 
> python39 to python3.9.
>
> Motivation:
> When you install an additional Python interpreter, the command that runs it 
> contains a dot (e.g. /usr/bin/python3.9) but the package name does not (e.g. 
> dnf install python39).
> The name without a dot is a technical debt that we carry since (at least) the 
> python26 package in RHEL 5 for consistency. The name with the dot makes much 
> more sense to Red Hat Python maintainers.
> It’s a minor inconsistency, but we'd like to get it right in RHEL, and since 
> RHEL splits off from Fedora, it will be bugging us still in 2030+ unless we 
> fix it now. Especially with the likely coming version 3.10 the package name 
> python310 is confusing. This will also get us in sync with e.g. Debian 
> package names.
>
> Backwards compatibility:
> We plan to create new components in rawhide containing the dot (e.g. 
> python3.9) for all Python interpreters (except soon-to-be-retired python34 
> and python26) and push the git commits from pythnoXY to them to preserve the 
> history.
> The packages will Provide and Obsolete the old name without a dot (e.g. 
> python39). The current packages already provide the name with the dot, so 
> this will be just a switch between real name and virtual provide.
> Therefore any package that depends on these components will continue to work 
> just fine. And in time we’d like to fix all of those to use the new name, 
> which is backwards compatible, because it is already provided now (as a side 
> note, the packages are generally just recommended, not required).
>
> Technical details:
> There has been a recent rawhide-only change to the %python_provide macro that 
> acts on packages named `python3-foo` and adds for them a new Provide 
> `python38-foo`.
> We’d like to change this to Provide `python3.8-foo` instead.
> Since this has been rawhide-only so far, and because there’s no real reason 
> to depend on these provides yet, we don’t expect any breakage.
>
> What do you think? Do you foresee any problems?

I'm good with this plan, except for one thing I thought of we need to
address: How do we do comparisons for python versions? Will we still
have a %python_version_nodots macro so that we can do integer
comparisons? I know that people have been abusing the
%python3_pkgversion macro for doing this (which you shouldn't be
doing!), so we need an official guideline for how to do those
comparisons.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: [Fedora-packaging] Re: Macronize %pytest

[Neal Gompa]

On Mon, Apr 27, 2020 at 11:58 AM Nicolas Mailhot
 wrote:
>
> Le lundi 27 avril 2020 à 11:34 +0200, Miro Hrončok a écrit :
> > Hello Python packagers,
> >
> > Usage:
> >
> >%check
> >%pytest
> >
> > Or with options passed to pyets:
> >
> >%check
> >%pytest -v -m "not network" tests/
>
> I ended up renaming similar macros as %foocheck (after several false
> starts).
>
> Having a domain-specific macro named slightly differently than the rpm
> section it is supposed to occur in was tripping packagers all the time.
>

The problem in Python land is that there are multiple different
options for tests, even with setuptools or whatnot. So some knowledge
of the tool is required, since there is no standardized interface
anymore (setup.py test is going away).


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Co-Maintainers wanted for python-lockfile EPEL branches

[Neal Gompa]

On Mon, Apr 20, 2020 at 9:51 AM Miro Hrončok  wrote:
>
> On 20. 04. 20 13:45, Fabio Valentini wrote:
> > and it seems I
> > can't even figure out how to determine which EPEL packages require
> > python*-lockfile.
>
> Take the attached repo files.
>
> They are good for repoquery, adapted from epel-release.
>
> They don't have -testing repos, but -testingx, so you don't accidentally 
> enable
> them  with dnf --enablerepo=\*-testing.
>
> Usage:
>
> $ repoquery --repo=epel8{,-source} --whatrequires python2-lockfile
> pungi-legacy-0:4.1.38-1.el8.2.noarch
> python2-pungi-0:4.1.38-1.el8.2.noarch
>

I also have a simple little tool I use for querying distributions and
repos with DNF: https://pagure.io/rpmdistro-repoquery

It comes with collections of repo definitions for Fedora, CentOS +
EPEL, Mageia, and openSUSE, and might be useful for things like
this...



--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Why is python(abi) generated? (Re: Rawhide python-rpm-generators news)

[Neal Gompa]

On Tue, Apr 7, 2020 at 8:26 AM Petr Viktorin  wrote:
>
> On 2020-04-07 12:40, Miro Hrončok wrote:
> > On 07. 04. 20 11:06, Petr Viktorin wrote:
> >> On 2020-04-03 20:44, Miro Hrončok wrote:
> >>> Hello Python packagers.
> >>>
> >>> I have just updated python-rpm-generators to
> >>> python-rpm-generators-11-1.fc33 in Rawhide. It uses some fresh stuff
> >>> from RPM 4.16 and will not be backported to older releases.
> >>>
> >>>
> >>> The python(abi) requirement is now added from a RPM Lua macro,
> >>> instead of by executing a Shell script. This is a bit faster
> >>> especially for packages with a lot of files. For 10 000 files in one
> >>> package, the speedup is roughly from ~80 to ~2 seconds (time of the
> >>> entire package build incl. creating the files from a Python script).
> >>> That is a lot, but most of the real packages have much less files, so
> >>> I am afraid you won't feel it.
> >>
> >> Hi,
> >> One thing I'm wondering about is: why is python(abi) provided by a
> >> generator, anyway?
> >> It's provided by a very small set of packages (python2 and python3 in
> >> Fedora). Couldn't it just be listed in those specfiles?
> >
> > Just to put this into more context: The time overhead is for the
> > requires. The provides are filtered on a certain path and even if we are
> > automating stuff for 2 packages which indeed might not be needed, it
> > doesn't really hurt anything.
> >
> > See also
> > https://lists.fedoraproject.org/archives/list/packag...@lists.fedoraproject.org/thread/UFKUM5UKCTNGIT3KJVYEI5VXPI23QMBN/
> > for me trying to figure out when this provides are useful -- depending
> > on the answer there, we might want to keep the automated provides of
> > python(abi) only and remove the manual ones.
>
> The part I don't get is: why keep the automated provides over manual
> ones. Compared to a line in the relevant spec files, the macro looks
> like it adds complexity for no benefit.
>
> I get the automatic *requires* on python(abi) -- that's used by
> thousands of packages and easy to forget. But the provides will only
> ever be in CPython.

I'd use the same argument to get rid of the manual Provides, but the
reason for why they exist in both manual and generated form is simple:
to prevent human error.

There have been cases in the past where the python RPM was being built
in places where generators are disabled or not going to work properly
(e.g. in this multi-python case), then the manual Provides would
ensure things came out as intended. The automatic Provides generation
is used to ensure that if things are using the generator, that
everything is _ensured_ to be consistent. It's a very bad idea to have
generated requires without generated provides, because you cannot
guarantee the names will match up.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Current plan: Build python3, python3-libs etc. from python39 SRPM on F33+

[Neal Gompa]

On Tue, Mar 24, 2020 at 8:40 AM Miro Hrončok  wrote:
>
> Hello Pythonistas.
>
> (I've CC'ed the devel list for further exposure. But let's discuss this on
> python-devel list please to avoid noise.)
>
>
> We  would like ro rename the "python3" component (SRPM) to "python39" to  make
> maintaining various Python versions in various Fedora versions  easier.
> The names of binary RPMs would be unchanged; you still do `dnf install 
> python3`.
>
> The change should only affect packagers of Python itself; not users or 
> packagers
> of software written in Python.
> The only user-visible or general-packager-visible change is a different
> component (SRPM) name.
>
> Details follow.
>
>
>
> Currently, we build the Python interpreters this way:
>
>   - the "main" Python 3 interpreter component is called python3
> - it builds a couple of subpackages, like python3-libs or python3-devel
>   - the "alternate" Python 3 versions components are called python3X 
> (python37,
> python39...)
>
> For example, Fedora 31 has:
>
> - python36
> - python3  (the "main" Python, which is 3.7.x)
> - python38
> - python39
>
> (python37 is retired in f31, but present in Fedora versions where python3 != 
> 3.7)
>
> This makes updating the "main" Python fairly complicated. To upgrade from 3.7 
> to
> 3.8 in f32, we had to:
>
>   1) retire python38
>   2) upgrade python3 from 3.7 to 3.8 (this makes python3 obsolete python38)
>   3) re-introduce (unretire) python37
>
> Such an update comes every few years:
>   - https://fedoraproject.org/wiki/Changes/Python3.7 (f29)
>   - https://fedoraproject.org/wiki/Changes/Python3.8 (f32)
>   - https://fedoraproject.org/wiki/Changes/Python3.9 (f33)
>
> What's even more complicated is backporting patches across Fedora releases.
> Assume there is an important upgrade/fix for Python 3.7 we wish to apply for 
> all
> Fedoras including stable. That currently means we need to apply the fix in:
>
>   - master branch of python37
>   - f32 branch of python37
>   - f31 branch of python3 (!)
>   - f30 branch of python3
>
> While cherry-picking or merging patches between branches of one git repo is
> quite straight-forward (the only major obstacle is the release and 
> %chaneglog),
> doing this across multiple git repos (and spec filenames) is very tedious and
> error-prone (albeit possible).
>
> With the 3.9 update approaching in Fedora 33, we would end up with the 
> following
> scheme when updating Python 3.8:
>
>   - master branch of python38
>   - f32 branch of python3 (!)
>   - f31 branch of python38 (!)
>   - f30 branch of python38
>
> Apart from technological obstacles, this causes nontrivial cognitive overhead.
> So far, I've been carrying that overhead myself, but others (who don't work on
> the Fedora Python packages daily) always need my assistance with this, because
> it's overly complicated and hard to understand without drawing pictures.
>
>
>
> For this to be easier for us, while keeping the change minimal for others, we
> have decided to retire the python3 component and only continue maintaining the
> interpreters in "python3X" components. The names of "binary" packages remain
> unchanged.
>
> Before:
>
> SRPM: python3
> builds:
>   - python3
>   - python-unversioned-command
>   - python3-{libs,devel,tkinter,idle,test,debug}
>   - python3-debug{info,source}
>
> After:
>
> SRPM: python39
> builds:
>   - python3 (unchanged)
>   - python-unversioned-command (unchanged)
>   - python3-{libs,devel,tkinter,idle,test,debug} (unchanged)
>   - python39-debug{info,source} (usually not installed by name)
>
> To avoid further confusion, the "python3-libs" etc. packages will provide
> "python39-libs" as well.
>
> We plan to do this together with the Python 3.9 upgrade. I'll update the 
> change
> page after some feedback happens here.
>
> The initial implementation is in:
>https://src.fedoraproject.org/rpms/python39/pull-request/32
>
>
> Here are other approaches we have considered:
>
>
> Why not rename the packages to python39-libs etc. (and only provide 
> python3-libs)?
>
>This would be inconsistent with the way other Python 3 packages are called
> (e.g. python39-setuptools). We consider that confusing.
>
>The goal here is to make our work easier while introducing minimal changes 
> to
> our users and contributors.
>
>
> Why not rename all the Python packages to python39-setuptools etc.?
>
>This would create a mess upon distro boundary Python upgrades.
>All python39-foo packages would need to obsolete all python38-foo packages.
>Even if we add the obsoletes metadata to all the packages, it would prevent
> users from having the two parallel stacks of Python installed on the same OS.
> (We don't have such stacks in Fedora, but we intend to continue to make this
> possible in third party repos, downstreams of Fedora or other systems built on
> top or Fedora).
>
>See also the "minimal changes" point above.
>
>
> Why not package the alternate stacks as modular streams and keep 

Re: Help needed to get dependencies in EPEL 8 for pagure

[Neal Gompa]

On Sat, Mar 21, 2020 at 3:41 AM Neal Gompa  wrote:
>
> On Sat, Feb 22, 2020 at 11:57 PM Neal Gompa  wrote:
> >
> > On Sat, Feb 8, 2020 at 9:59 PM Neal Gompa  wrote:
> > >
> > > Hey all,
> > >
> > > I've been trying to get Pagure into EPEL 8 for a couple of months now
> > > so that we can upgrade our Pagure instances to RHEL 8[1].
> > >
> > > Thankfully, most of Pagure's dependencies *are* now present in EPEL 8,
> > > so there's only a few that need to be added.
> > >
> >
> > So, an update: after this most recent push, most of Pagure's
> > dependencies are now in EPEL 8.
> >
> > We're still missing celery...
> >
>
> Celery is now in EPEL 8, so we're good here now!
>
> >
> > And we're still missing pygit2...
> >
> > > * python-pygit2: pwalter
> > ...
> > > A note here: RHEL 8 ships libgit2 0.26.8, so we need pygit2 0.26.x.
>
> And now pygit2 is the only remaining missing dependency! We're nearly
> to the finish line!
>

And now it's all done, with Pagure 5.9.0 landing in EPEL 8:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-a504acc25e

Thanks to everyone who helped make this happen!



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Pagure for EL8 (EPEL8)

[Neal Gompa]

On Sat, Mar 21, 2020 at 3:42 AM Neal Gompa  wrote:
>
> On Sat, Feb 22, 2020 at 11:50 PM Neal Gompa  wrote:
> >
> > On Mon, Nov 18, 2019 at 1:16 PM Neal Gompa  wrote:
> > >
> > > On Mon, Nov 18, 2019 at 11:18 AM Kevin Fenzi  wrote:
> > > >
> > > > On Sat, Nov 16, 2019 at 05:37:16PM -0500, Neal Gompa wrote:
> > > > > I've done an early build locally to determine what's needed to make
> > > > > this possible. The following report from DNF indicates the missing
> > > > > packages that need to be added to EPEL 8 before I can introduce Pagure
> > > > > into EPEL8:
> > > > >
> > > > >  Problem 1: conflicting requests
> > > > >   - nothing provides python3-jenkins needed by 
> > > > > pagure-ci-5.8-1.el8.noarch
> > > > >  Problem 2: conflicting requests
> > > > >   - nothing provides gitolite3 needed by pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(binaryornot) needed by
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(celery) needed by 
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(flask-wtf) needed by 
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(redis) needed by 
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(straight.plugin) needed by
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(wtforms) needed by 
> > > > > pagure-5.8-1.el8.noarch
> > > > >   - nothing provides python3.6dist(pygit2) >= 0.26.0 needed by
> > > > > pagure-5.8-1.el8.noarch
> > > > >  Problem 3: conflicting requests
> > > > >   - nothing provides python3-trololio needed by 
> > > > > pagure-ev-5.8-1.el8.noarch
> > > > >
> > > > > One of the reasons I'd like to have this done sooner rather than later
> > > > > is so that we can drop Python 2 support from Pagure with version 6.0.
> > > > > I think it's quite reasonable to say that version 6.0 isn't going to
> > > > > happen until we can get our Pagure servers running on EL8 using Python
> > > > > 3.
> > > > >
> > > > > So now, I need some help making this happen. I already own trololio,
> > > > > and I'm going to make that available in EPEL 8 ASAP. Can anyone help
> > > > > with some of the other dependencies here?
> > > >
> > > > Can you give a list with maintainers? I'm not sure off hand how many of
> > > > those are maintained by me/infra-sig, but any I can I would be happy to
> > > > add in. There's a few that are in testing I think already...
> > > >
> > >
> > > Sure, here's a list so far (package: maintainer):
> > >
> > > * gitolite3: limb
> > > * python-jenkins: cottsay
> > > * python-binaryornot: pingou
> > > * python-celery: bowlofeggs
> > > * python-flask-wtf: pingou
> > > * python-redis: kevin
> > > * python-straight-plugin: pingou
> > > * python-wtforms: kumarpraveen
> > > * python-pygit2: pwalter
> > > * python-trololio: ngompa
> > >
> > > I've already got trololio going:
> > > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-6c443851dd
> > >
> >
> > So an update on this...
> >
> > We're now only missing two dependencies:
> >
> > * python-celery: abompard
> > * python-pygit2: pwalter
> >
> > The rest of them are now in EPEL 8.
> >
>
> We're now only missing python-pygit2, as celery just landed in EPEL 8.
>

And now it's all done, with Pagure 5.9.0 landing in EPEL 8:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-a504acc25e

Thanks to everyone who helped make this happen!


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Pagure for EL8 (EPEL8)

[Neal Gompa]

On Sat, Feb 22, 2020 at 11:50 PM Neal Gompa  wrote:
>
> On Mon, Nov 18, 2019 at 1:16 PM Neal Gompa  wrote:
> >
> > On Mon, Nov 18, 2019 at 11:18 AM Kevin Fenzi  wrote:
> > >
> > > On Sat, Nov 16, 2019 at 05:37:16PM -0500, Neal Gompa wrote:
> > > > I've done an early build locally to determine what's needed to make
> > > > this possible. The following report from DNF indicates the missing
> > > > packages that need to be added to EPEL 8 before I can introduce Pagure
> > > > into EPEL8:
> > > >
> > > >  Problem 1: conflicting requests
> > > >   - nothing provides python3-jenkins needed by 
> > > > pagure-ci-5.8-1.el8.noarch
> > > >  Problem 2: conflicting requests
> > > >   - nothing provides gitolite3 needed by pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(binaryornot) needed by
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(celery) needed by 
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(flask-wtf) needed by 
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(redis) needed by 
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(straight.plugin) needed by
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(wtforms) needed by 
> > > > pagure-5.8-1.el8.noarch
> > > >   - nothing provides python3.6dist(pygit2) >= 0.26.0 needed by
> > > > pagure-5.8-1.el8.noarch
> > > >  Problem 3: conflicting requests
> > > >   - nothing provides python3-trololio needed by 
> > > > pagure-ev-5.8-1.el8.noarch
> > > >
> > > > One of the reasons I'd like to have this done sooner rather than later
> > > > is so that we can drop Python 2 support from Pagure with version 6.0.
> > > > I think it's quite reasonable to say that version 6.0 isn't going to
> > > > happen until we can get our Pagure servers running on EL8 using Python
> > > > 3.
> > > >
> > > > So now, I need some help making this happen. I already own trololio,
> > > > and I'm going to make that available in EPEL 8 ASAP. Can anyone help
> > > > with some of the other dependencies here?
> > >
> > > Can you give a list with maintainers? I'm not sure off hand how many of
> > > those are maintained by me/infra-sig, but any I can I would be happy to
> > > add in. There's a few that are in testing I think already...
> > >
> >
> > Sure, here's a list so far (package: maintainer):
> >
> > * gitolite3: limb
> > * python-jenkins: cottsay
> > * python-binaryornot: pingou
> > * python-celery: bowlofeggs
> > * python-flask-wtf: pingou
> > * python-redis: kevin
> > * python-straight-plugin: pingou
> > * python-wtforms: kumarpraveen
> > * python-pygit2: pwalter
> > * python-trololio: ngompa
> >
> > I've already got trololio going:
> > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-6c443851dd
> >
>
> So an update on this...
>
> We're now only missing two dependencies:
>
> * python-celery: abompard
> * python-pygit2: pwalter
>
> The rest of them are now in EPEL 8.
>

We're now only missing python-pygit2, as celery just landed in EPEL 8.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Help needed to get dependencies in EPEL 8 for pagure

[Neal Gompa]

On Sat, Feb 22, 2020 at 11:57 PM Neal Gompa  wrote:
>
> On Sat, Feb 8, 2020 at 9:59 PM Neal Gompa  wrote:
> >
> > Hey all,
> >
> > I've been trying to get Pagure into EPEL 8 for a couple of months now
> > so that we can upgrade our Pagure instances to RHEL 8[1].
> >
> > Thankfully, most of Pagure's dependencies *are* now present in EPEL 8,
> > so there's only a few that need to be added.
> >
>
> So, an update: after this most recent push, most of Pagure's
> dependencies are now in EPEL 8.
>
> We're still missing celery...
>

Celery is now in EPEL 8, so we're good here now!

>
> And we're still missing pygit2...
>
> > * python-pygit2: pwalter
> ...
> > A note here: RHEL 8 ships libgit2 0.26.8, so we need pygit2 0.26.x.

And now pygit2 is the only remaining missing dependency! We're nearly
to the finish line!



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Help needed to get dependencies in EPEL 8 for pagure

[Neal Gompa]

On Sat, Feb 8, 2020 at 9:59 PM Neal Gompa  wrote:
>
> Hey all,
>
> I've been trying to get Pagure into EPEL 8 for a couple of months now
> so that we can upgrade our Pagure instances to RHEL 8[1].
>
> Thankfully, most of Pagure's dependencies *are* now present in EPEL 8,
> so there's only a few that need to be added.
>

So, an update: after this most recent push, most of Pagure's
dependencies are now in EPEL 8.

We're still missing celery...

> * python-celery: abompard, mrunge, pingou, ngompa
...
> Celery is in a special position here, because it has a number of
> missing dependencies, too (which is why I haven't branched and built
> it yet):
>
> * python-kombu: mrunge, pingou, fab, pjp, sundaram
> * python-billiard: mrunge, pingou, fab, pjp, sundaram
> * python-amqp: eharney
> * python-vine: mrunge
> * python-case: mrunge

And we're still missing pygit2...

> * python-pygit2: pwalter
...
> A note here: RHEL 8 ships libgit2 0.26.8, so we need pygit2 0.26.x.
> This is already what we ship in the EPEL 7 branch, so that can just be
> branched into a new epel8 branch and built.

We're nearly there, though. Thanks for everyone who has helped so far!



--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Pagure for EL8 (EPEL8)

[Neal Gompa]

On Mon, Nov 18, 2019 at 1:16 PM Neal Gompa  wrote:
>
> On Mon, Nov 18, 2019 at 11:18 AM Kevin Fenzi  wrote:
> >
> > On Sat, Nov 16, 2019 at 05:37:16PM -0500, Neal Gompa wrote:
> > > I've done an early build locally to determine what's needed to make
> > > this possible. The following report from DNF indicates the missing
> > > packages that need to be added to EPEL 8 before I can introduce Pagure
> > > into EPEL8:
> > >
> > >  Problem 1: conflicting requests
> > >   - nothing provides python3-jenkins needed by pagure-ci-5.8-1.el8.noarch
> > >  Problem 2: conflicting requests
> > >   - nothing provides gitolite3 needed by pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(binaryornot) needed by
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(celery) needed by 
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(flask-wtf) needed by 
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(redis) needed by 
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(straight.plugin) needed by
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(wtforms) needed by 
> > > pagure-5.8-1.el8.noarch
> > >   - nothing provides python3.6dist(pygit2) >= 0.26.0 needed by
> > > pagure-5.8-1.el8.noarch
> > >  Problem 3: conflicting requests
> > >   - nothing provides python3-trololio needed by pagure-ev-5.8-1.el8.noarch
> > >
> > > One of the reasons I'd like to have this done sooner rather than later
> > > is so that we can drop Python 2 support from Pagure with version 6.0.
> > > I think it's quite reasonable to say that version 6.0 isn't going to
> > > happen until we can get our Pagure servers running on EL8 using Python
> > > 3.
> > >
> > > So now, I need some help making this happen. I already own trololio,
> > > and I'm going to make that available in EPEL 8 ASAP. Can anyone help
> > > with some of the other dependencies here?
> >
> > Can you give a list with maintainers? I'm not sure off hand how many of
> > those are maintained by me/infra-sig, but any I can I would be happy to
> > add in. There's a few that are in testing I think already...
> >
>
> Sure, here's a list so far (package: maintainer):
>
> * gitolite3: limb
> * python-jenkins: cottsay
> * python-binaryornot: pingou
> * python-celery: bowlofeggs
> * python-flask-wtf: pingou
> * python-redis: kevin
> * python-straight-plugin: pingou
> * python-wtforms: kumarpraveen
> * python-pygit2: pwalter
> * python-trololio: ngompa
>
> I've already got trololio going:
> https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-6c443851dd
>

So an update on this...

We're now only missing two dependencies:

* python-celery: abompard
* python-pygit2: pwalter

The rest of them are now in EPEL 8.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Help needed to get dependencies in EPEL 8 for pagure

[Neal Gompa]

Hey all,

I've been trying to get Pagure into EPEL 8 for a couple of months now
so that we can upgrade our Pagure instances to RHEL 8[1].

Thankfully, most of Pagure's dependencies *are* now present in EPEL 8,
so there's only a few that need to be added.

The list of Pagure dependencies missing are below:

* gitolite3: limb
* python-jenkins: cottsay
* python-binaryornot: pingou
* python-celery: bowlofeggs, mrunge, pingou, ngompa
* python-flask-wtf: pingou
* python-wtforms: kumarpraveen, pjp, sundaram
* python-pygit2: pwalter

Celery is in a special position here, because it has a number of
missing dependencies, too (which is why I haven't branched and built
it yet):

* python-kombu: mrunge, pingou, fab, pjp, sundaram
* python-billiard: mrunge, pingou, fab, pjp, sundaram
* python-amqp: eharney
* python-vine: mrunge
* python-case: mrunge

A note here: RHEL 8 ships libgit2 0.26.8, so we need pygit2 0.26.x.
This is already what we ship in the EPEL 7 branch, so that can just be
branched into a new epel8 branch and built.

Fortunately, I only need Python 3 variants of all these things, as
Pagure is Python 3 in Fedora and I intend to keep it that way for EPEL
8. This is also a prerequisite for dropping Python 2 support in Pagure
for 6.0, as we need all Pagure production servers to be able to move
to Python 3 first (and our servers run RHEL instead of Fedora).

Thank you all in advance!

[1]: 
https://lists.fedoraproject.org/archives/list/infrastruct...@lists.fedoraproject.org/thread/RI5RQOPLWHV2V3XPVD3YDYNZEOIO4SYL/

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Python 3.9 package review request

[Neal Gompa]

On Wed, Nov 20, 2019 at 5:57 AM Miro Hrončok  wrote:
>
> Python 3.9.0a1 is out: https://www.python.org/downloads/release/python-390a1/
>
> Here is the package review request for python39:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1774417

I've grabbed the review. Let's see how this goes...


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Building the python executable statically with libpython*.a for better performance

[Neal Gompa]

On Tue, Oct 29, 2019 at 6:58 PM Charalampos Stratakis
 wrote:
>
> Through our latest benchmarks the speedup could be up to 27% so I would say 
> it's definitely worth it, and we plan to work with the affected packages to 
> see how to best resolve the issues, if they arise, on a case by case basis. 
> Currently on rawhide there are 113 packages requiring libpython.
>
> Ubuntu and Debian are doing this for years so I would expect that at least 
> the packages that are also included there, would either be working or require 
> slight modifications.
>
> The change will be posted soon for F32 on the devel list, so you can check 
> all the details and we can continue this conversation by then.
>

Debian/Ubuntu packaging of Python works quite a bit differently. They
do install-time byte-compilation, and they made modifications
throughout the stack to make it as hard as possible to create a
dependency on a specific minor version of Python. We do exactly the
opposite (for VERY good reasons), and it's important to keep that in
mind.

Python performance improvement by nearly 30% is no joke, but we need
to figure out how to account for non-standard builds before
implementing it.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Building the python executable statically with libpython*.a for better performance

[Neal Gompa]

On Sat, Sep 21, 2019 at 5:16 PM Miro Hrončok  wrote:
>
> Hello,
>
> we've been recently approached by a colleague from Red Hat working on
> performance (CCed).
>
> According to their testing, Fedora Python performance could be improved by 
> ~15%
> by building /usr/bin/python* statically with libpython*.a. That sounds like a
> worthy thing to do.
>
> https://bugzilla.redhat.com/show_bug.cgi?id=1749479
>
> Since Python 3.8 Python extension modules are no longer linked to libpython.so
> and we can do the following:
>
>   * build /usr/bin/python3(.8) statically with libpython*.a
>   * build and ship libpython3.8.so.1.0 for packages that "embed" Python
>
> The change in the python3 package is trivial:
>
> https://src.fedoraproject.org/rpms/python3/pull-request/133
>
> However it can have serious impact on Python extension modules that are linked
> to libpython3.8.so.1.0 by various "nonstandard" build mechanisms or by 
> compiling
> code for Python extension module and code that embeds Python into one file.
>
> We will likely propose a Fedora 32 Change for this, however I'm opening this
> topic for discussion before we do so.
>
> Testing the proposed Pull Request with your code is also helpful. Let me know
> how can we make that easier (e.g. if you want a Copr or a Fedora 30/31 
> python38
> package with this change).
>
> WDYT?

One major concern: We currently rely on the libpython shared object
linkage for non-standard builds to ensure we get everything for moving
to new Python versions. I've personally experienced quirks with trying
to force linking via the shared object when the Python interpreter
itself is not. For example, perl-Inline-Python determines how to build
itself by asking the interpreter. In Fedora, this works correctly
because the interpreter returns the shared object. In other
distributions where I've built the module, it doesn't and uses the
static link object, which makes it difficult to automatically generate
the link dependency to the system Python we are using.

In Mageia, we actually reverted Python's change to not link libpython
with extension modules so that we still have the dependency link for
binary objects:
https://svnweb.mageia.org/packages/cauldron/python3/current/SOURCES/link-C-modules-with-libpython.patch?view=markup

Is there no other way to get better performance? Do we want to
potentially give up the easily trackable dependency web for that? Is
it worth breaking non-standard build mechanisms that interrogate the
interpreter for how to link to it?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Should python3-rpm-macros depend on python3?

[Neal Gompa]

On Wed, Jun 19, 2019 at 6:05 AM Miro Hrončok  wrote:
>
> We have an interesting request for python3-rpm-macros to depend on python3.
>
> See https://bugzilla.redhat.com/show_bug.cgi?id=1563789
>
> Highlights:
>
>   - users who build for Python 3 are told (in the guidelines) to BR 
> python3-devel
> (that brings in both python3 and python3-rpm-macros)
>   - the existence of python3-rpm-macros is left as an implementation detail 
> for ^
>   - in theory, those macros can be used with other Pythons
> (such as pypy3 or python36, that is most likely not done in practice)
>
> Arguments:
>
>   - to require: the macros are broken without a python3 interpreter
>   - to not: the macros should work with any python3 interpreter
>
> Solutions?
>
>   - declare direct BR of macros without a python3 interpreter unsupported
>   - add dependency on python3. unused if used with another interpreter
>   - add a common virtual provide for all python3 interpreters
> or require (python3 or pypy3 or python36...) - very tedious
>

Have we ever tried using it with pypy3? Does it even work with that
(that is, pypy3-foo packages can be made)? If it does, we should just
add a common virtual provide to supported interpreters and require
that. Else, just require python3.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Heads-up: rpm 4.15 alpha coming to rawhide

[Neal Gompa]

On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen  wrote:
>
>
>
> A pile of language-specific macros and scripts have been eliminated from
> rpm upstream, notably %__python and %__perl and everything built around
> them, such as %python_sitelib and %perl_sitelib and their relatives.
> Python packages should be using the version specific macros already,
> provided by respective pythonN-devel packages so this shouldn't be an
> issue. On perl-side, those macros have been temporarily added to
> redhat-rpm-config instead to avoid breaking things right now, later to
> be moved to final destination in perl-macros or such.
>

Unversioned Python macros need to get added to python-rpm-macros, as
there are a number of packages that do rely on their existence (as a
means of supporting only a single version of Python depending on
distro releases or legacy reasons).

CC'd python-devel, Miro, and Igor about this.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: I found 2 problems on remove python2-foo packages

[Neal Gompa]

On Tue, Jun 4, 2019 at 3:26 AM Nico Kadel-Garcia  wrote:
>
> On Mon, Jun 3, 2019 at 10:51 AM Sérgio Basto  wrote:
> >
> > Hi,
> > These removed python2 packages are breaking upgrade path
> >
> > 1- when you remove python2-foo , you should add to main package
> > Obsoletes: python2-foo
>
> If I may suggest, *absolutely not*. "Obsoletes: python2-foo"
> specifically means that it includes and provides "python2-foo"
> functionality, and "python3-foo" packages *do not* provide this. There
> are a very few tools that put things in "/usr/bin" for typical use,
> such as the "rpm" package itself which might benefit from such
> options. But I think it's quite dangerous to replace them willy-nilly
> due to compatibility for old packages which would have their
> requirements replaced with something quite incompatible without
> notification.
>

I think you misunderstand here. Obsoletes does not specifically
indicate it is equivalent. It is saying that you're replacing it. You
will still get a broken dep chain if something Requires something that
was obsoleted.

It is absolutely the correct strategy to do Obsoletes (without Provides) here.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Let's update tornado to 6 and drop python2-torando

[Neal Gompa]

On Wed, May 15, 2019 at 10:59 AM Miro Hrončok  wrote:
>
> Hi.
>
> Tornado 6 doesn't support Python 2. Let's update the python-torando package to
> Python 3 only. There are several consumers of python2-torando and if their
> maintainers are interested, they can package it separately.
>
> $ dnf repoquery --repo=compose{,-source} --whatrequires python2-tornado
> bup-0:0.29.2-3.fc30.src
> bup-web-0:0.29.2-3.fc30.x86_64
> python-httpretty-0:0.9.5-5.fc30.src
> python-pika-0:1.0.1-1.fc31.src
> python-urllib3-0:1.24.2-1.fc31.src
> salt-0:2019.2.0-1.fc31.noarch
> uwsgi-plugin-python2-tornado-0:2.0.17.1-10.fc31.x86_64
>
> Note that tornado is often used to test things. We can (and should) just skip
> such tests from Python 2 httpretty, pika and urllib3.
>
> Is the plan OK? I'll talk to the dependent packages maintainers, but wanted to
> check with torando co-maintainers first.

I'm okay with this for pika. :)


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: [EPEL-devel] EPEL7: Adapting %python_provide to provide python3- for python36-

[Neal Gompa]

On Wed, Apr 24, 2019 at 6:23 AM Miro Hrončok  wrote:
>
> Hey,
>
> since the plan is to have some python3-... packages in RHEL proper, should we
> adapt the %python_provide macro to provide python3-... when it gets 
> python36-...?
>
> %{python_provide python36-foo} currently does nothing.
> I propose to change it to do: Provides: python3-foo = %{version}-%{release}.
>
> Note: %{python_provide python2-foo} currently adds obsoletes, let's not add 
> them
> for the python36 case (there is nothing to obsolete here, quite the opossite -
> python3-foo from RHEL would in theory obsolete python36-foo from EPEL).
>
> Arguably, this discussion should have happened before we did the mass rebuild
> for 3.4 -> 3.6 transition, however I don't consider such provides essential. 
> The
> idea is to change the macro, but don't mass rebuild - instead start providing
> the provides gradually.
>
> Note that not all EPEL7 Python 3 packages use this macro, but many do.
>

That would actually be great, since it simplifies how people can use it.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Koji 1.17.0 and Python 3

[Neal Gompa]

On Mon, Mar 11, 2019 at 9:44 AM Peter Robinson  wrote:
>
> > I disabled autopushing precisely so that there's no accidental pushing
> > to stable updates. The worst thing that can happen is that we have to
> > unpush the update from testing. Seriously, that's really not the end
> > of the world.
> >
> > Nowhere did I say that releng needs to deploy this RIGHT NOW. Having
> > this in rawhide and F30 updates-testing makes it possible for the
> > dependency map for Python 2 to cleanup even more, allowing the Python
> > SIG to continue its work to clean out Python 2 subpackages in Fedora
> > 30/31.
>
> So if you've disabled/removed python2 support in Fedora 30 you've
> broken image building at least.
>

I double-checked this and you're sadly right. I've fixed this so
koji-builder is now using Python 2 again.

I thought I had validated all the dependencies for Koji we use in
infra to have been ported, but alas, imgfac hasn't been done yet.

Sorry all. I've learned my lesson. I'm not going to touch
infrastructure in Fedora directly again.



--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Koji 1.17.0 and Python 3

[Neal Gompa]

On Sun, Mar 10, 2019 at 7:12 PM Peter Robinson  wrote:
>
> > > > > Hey all,
> > > > >
> > > > > I've proposed a pull request to switch our Koji package to use Python
> > > > > 3 wherever possible:
> > > > > https://src.fedoraproject.org/rpms/koji/pull-request/4
> > > > >
> > > > > The PR is a bit complex, but it's based on the upstream spec for Koji,
> > > > > which accounts for all the variations (Py2 Koji + Py3 client for
> > > > > Fedora < 30, Py3 Koji + client + Py2 API for Fedora 30+, Py2 Koji for
> > > > > EPEL).
> > > > >
> > > > > I'd like to merge this in and submit updates for all currently
> > > > > supported releases we ship the Koji package to (Fedora and EPEL).
> > > > >
> > > > > Note that this is independent of testing and upgrading the
> > > > > infrastructure. But I'd like to merge this in now so that we could
> > > > > look at having staging Koji switch over now.
> > > >
> > > > Could you wait til Tuesday/Wednesday for EPEL, we are in the middle of
> > > > putting the python36 there and various packages build funny until we
> > > > are done.
> > >
> > > Please wait for Tues/Wed for everything to allow rel-eng and infra to
> > > replay. It should also go to rawhide first.
> > >
> > > Not everyone works on the weekend, and Monday is often catching up
> > > from issue from the weekend for people.
> > >
> >
> > I'm not pushing to stable automatically, but they are proposed for
> > syncing out to updates-testing:
>
> I think pushing it at all with such little notice and on a weekend is
> disingenuous, it wasn't even 24 hrs between when you sent the above
> email and merging your own PR with out a single comment on the PR from
> either the koji maintainers (of which you're not one, and I believe
> one of them requested on IRC that you don't) or the rel-eng team.

I specifically waited for a response from someone in the releng/admin
team. The packaging is literally copied from upstream, which has
already been beaten to death under multiple pull requests. In
addition, the PR had some review from someone before I asked about
merging it.

When Stephen gave me the okay, I went with it. I specifically asked
because I wanted permission before pushing it to testing.

And for what it's worth, the original 1.17.0 update to Koji in Rawhide
was broken. It did a directory -> file replacement improperly which
causes rpm to choke. Moreover, the transition wasn't even necessary
because the directory was completely unused and had been for a while.

All I did was synchronize what upstream offered, and made a small
tweak based on the feedback in the PR.

> Even on a week day, due to timezones and people having things like getting
> the Beta done, I would expect you to wait a good 48 hours after
> posting to the list, and if done on a weekend 48 hours of working time
> to ensure review and response before just pushing it anyway.
>
> What's more you should at LEAST just push it to rawhide first and let
> it bake there for some time before then shoving it out to every
> release!
>
> > * Fedora 30: https://bodhi.fedoraproject.org/updates/FEDORA-2019-6b2e124a4a
> > * Fedora 29: https://bodhi.fedoraproject.org/updates/FEDORA-2019-940d3922ce
> > * Fedora 28: https://bodhi.fedoraproject.org/updates/FEDORA-2019-c4107ac9d3
> > * Fedora EPEL 7:
> > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-798356a949
> > * Fedora EPEL 6:
> > https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-210b4d1b32
> >
> > This should give people the opportunity to test the new version and
> > get it pushed once it's validated.
>
> But it doesn't give rel-eng and the infrastructure teams to review
> your changes, and we're also in freeze which means if there's
> something in existing stable builds that needs to be fixed for that
> it's quite likely that they may have to revert it all, bump an epoch
> to get things working again rather than having the re-validate and
> entire stack.
>

First of all, you're blowing it way out of proportion. It's only in
testing, and that's pretty much the only way anyone can really see
whether it works. And like it or not, it has to make its way into
stable releases at some point, so that all the stuff can use it.

I disabled autopushing precisely so that there's no accidental pushing
to stable updates. The worst thing that can happen is that we have to
unpush the update from testing. Seriously, that's really not the end
of the world.

Nowhere did I say that releng needs to deploy this RIGHT NOW. Having
this in rawhide and F30 updates-testing makes it possible for the
dependency map for Python 2 to cleanup even more, allowing the Python
SIG to continue its work to clean out Python 2 subpackages in Fedora
30/31.

> PLEASE do not do this sort of stuff, it's not helpful or kind to the
> teams that have to deal with the infra and releases. There's no rush
> to get it out, it should go through the process of the maintainers
> reviewing your changes, it being tested in rawhide and then eventually
> making 

Re: Koji 1.17.0 and Python 3

[Neal Gompa]

On Sun, Mar 10, 2019 at 11:42 AM Peter Robinson  wrote:
>
> > > Hey all,
> > >
> > > I've proposed a pull request to switch our Koji package to use Python
> > > 3 wherever possible:
> > > https://src.fedoraproject.org/rpms/koji/pull-request/4
> > >
> > > The PR is a bit complex, but it's based on the upstream spec for Koji,
> > > which accounts for all the variations (Py2 Koji + Py3 client for
> > > Fedora < 30, Py3 Koji + client + Py2 API for Fedora 30+, Py2 Koji for
> > > EPEL).
> > >
> > > I'd like to merge this in and submit updates for all currently
> > > supported releases we ship the Koji package to (Fedora and EPEL).
> > >
> > > Note that this is independent of testing and upgrading the
> > > infrastructure. But I'd like to merge this in now so that we could
> > > look at having staging Koji switch over now.
> >
> > Could you wait til Tuesday/Wednesday for EPEL, we are in the middle of
> > putting the python36 there and various packages build funny until we
> > are done.
>
> Please wait for Tues/Wed for everything to allow rel-eng and infra to
> replay. It should also go to rawhide first.
>
> Not everyone works on the weekend, and Monday is often catching up
> from issue from the weekend for people.
>

I'm not pushing to stable automatically, but they are proposed for
syncing out to updates-testing:

* Fedora 30: https://bodhi.fedoraproject.org/updates/FEDORA-2019-6b2e124a4a
* Fedora 29: https://bodhi.fedoraproject.org/updates/FEDORA-2019-940d3922ce
* Fedora 28: https://bodhi.fedoraproject.org/updates/FEDORA-2019-c4107ac9d3
* Fedora EPEL 7:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-798356a949
* Fedora EPEL 6:
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-210b4d1b32

This should give people the opportunity to test the new version and
get it pushed once it's validated.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Koji 1.17.0 and Python 3

[Neal Gompa]

On Sun, Mar 10, 2019 at 10:57 AM Stephen John Smoogen  wrote:
>
> On Sun, 10 Mar 2019 at 10:55, Neal Gompa  wrote:
> >
> > On Sun, Mar 10, 2019 at 10:53 AM Stephen John Smoogen  
> > wrote:
> > >
> > > On Sat, 9 Mar 2019 at 22:19, Neal Gompa  wrote:
> > > >
> > > > Hey all,
> > > >
> > > > I've proposed a pull request to switch our Koji package to use Python
> > > > 3 wherever possible:
> > > > https://src.fedoraproject.org/rpms/koji/pull-request/4
> > > >
> > > > The PR is a bit complex, but it's based on the upstream spec for Koji,
> > > > which accounts for all the variations (Py2 Koji + Py3 client for
> > > > Fedora < 30, Py3 Koji + client + Py2 API for Fedora 30+, Py2 Koji for
> > > > EPEL).
> > > >
> > > > I'd like to merge this in and submit updates for all currently
> > > > supported releases we ship the Koji package to (Fedora and EPEL).
> > > >
> > > > Note that this is independent of testing and upgrading the
> > > > infrastructure. But I'd like to merge this in now so that we could
> > > > look at having staging Koji switch over now.
> > >
> > > Could you wait til Tuesday/Wednesday for EPEL, we are in the middle of
> > > putting the python36 there and various packages build funny until we
> > > are done.
> > >
> >
> > Even for exclusively Python 2? Koji isn't building Python 3 components
> > in EPEL at all right now, as there's too much stuff missing for that
> > to work anyway.
>
> OK in that case, we can look at adding the missing stuff afterwords.
>

So then, no objection to me merging and submitting updates to Bodhi
for this now then?




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Koji 1.17.0 and Python 3

[Neal Gompa]

On Sun, Mar 10, 2019 at 10:53 AM Stephen John Smoogen  wrote:
>
> On Sat, 9 Mar 2019 at 22:19, Neal Gompa  wrote:
> >
> > Hey all,
> >
> > I've proposed a pull request to switch our Koji package to use Python
> > 3 wherever possible:
> > https://src.fedoraproject.org/rpms/koji/pull-request/4
> >
> > The PR is a bit complex, but it's based on the upstream spec for Koji,
> > which accounts for all the variations (Py2 Koji + Py3 client for
> > Fedora < 30, Py3 Koji + client + Py2 API for Fedora 30+, Py2 Koji for
> > EPEL).
> >
> > I'd like to merge this in and submit updates for all currently
> > supported releases we ship the Koji package to (Fedora and EPEL).
> >
> > Note that this is independent of testing and upgrading the
> > infrastructure. But I'd like to merge this in now so that we could
> > look at having staging Koji switch over now.
>
> Could you wait til Tuesday/Wednesday for EPEL, we are in the middle of
> putting the python36 there and various packages build funny until we
> are done.
>

Even for exclusively Python 2? Koji isn't building Python 3 components
in EPEL at all right now, as there's too much stuff missing for that
to work anyway.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Koji 1.17.0 and Python 3

[Neal Gompa]

Hey all,

I've proposed a pull request to switch our Koji package to use Python
3 wherever possible:
https://src.fedoraproject.org/rpms/koji/pull-request/4

The PR is a bit complex, but it's based on the upstream spec for Koji,
which accounts for all the variations (Py2 Koji + Py3 client for
Fedora < 30, Py3 Koji + client + Py2 API for Fedora 30+, Py2 Koji for
EPEL).

I'd like to merge this in and submit updates for all currently
supported releases we ship the Koji package to (Fedora and EPEL).

Note that this is independent of testing and upgrading the
infrastructure. But I'd like to merge this in now so that we could
look at having staging Koji switch over now.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: F30 Self-Contained Change proposal: Retire YUM 3

[Neal Gompa]

On Thu, Feb 28, 2019 at 8:23 PM Dennis Gregorovic  wrote:
>
> I have an update on the koji end.  The 1.17 release will not only drop the 
> yum dependency, it will also have full python 3 support (except for image 
> building that uses oz / imagefactory).  Unfortunately, there is only medium 
> confidence that the 1.17 release will be ready by the F30 devel freeze on 
> Tuesday.  It depends on whether QE uncovers any issues in its final testing.  
> If we're not able to land the release on Tuesday, what is the backup plan?
>

I'm not sure. Honestly, I'd rather take a snapshot of git master
that's going to be Koji 1.17 in Python 3 form for F30+ so that we can
iterate and get to the final release.

None of the Koji components are shipped on any of the media, it's only
accessed through the repositories, so there's a very low risk there.

Moreover, releng redeploys post-GA for prod, so that gives us a long
window to suss out issues. We could even have staging upgraded early
to "kick the tires" if need be.

The worst thing that could happen if 1.17.0 goes out and there's a
problem is that 1.17.1 has to be issued. In the grand scheme of
things, that's really not that bad.






--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: We need to get rid of python2-flake8 on F30+

[Neal Gompa]

On Tue, Feb 26, 2019 at 9:12 AM Miro Hrončok  wrote:
>
> Long story short:
>
> - we've updated pycodestyle and broke flake8
> - we need to update flake8
> - we cannot update on python2
>
> Hence, I'd like to get rid of python2-flake8.
>
> https://src.fedoraproject.org/rpms/python-flake8/pull-request/4
>
> We will probably just do it and deal with the breakage later. Let me know if
> this is somehow problematic.
>

I'm generally okay with this. Most usage of flake8 is through the
binary, and so the python-flake8 package and all the stuff depending
on it should be safe to switch to Python 3 only...




--
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

urlgrabber 4.0.0 released!

[Neal Gompa]

Hey all,

I've released urlgrabber 4.0.0, which is notable for being Python 3 compatible!

We're still working out the kinks for getting it updated in all the
right places (urlgrabber.baseurl.org, PyPI, etc.), but for now, the
release source tarball can be downloaded from GitHub:
https://github.com/rpm-software-management/urlgrabber/releases/tag/urlgrabber-4-0-0

It is also making its way into Fedora 30 and Rawhide now, so this
should unblock efforts to port packages dependent on urlgrabber to
Python 3.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: [EPEL-devel] Moving EPEL7 to python3.6

[Neal Gompa]

On Tue, Oct 23, 2018 at 1:28 PM Jason L Tibbitts III  wrote:
>
> > "OP" == Orion Poplawski  writes:
>
> OP> - Can we make epel7-py36 branches, and somehow have
> OP> %python3_version, et. al. be 3.6 for those builds?
>
> I can't think of any way to do that without extra magic.  And if you
> require something in the spec, you might as well just hardcode it.
>
> OP> - Can we just make it easier for people to create python3X- packages
> OP> from existing python3- or python3(X-n)- packages?  And just drop the
> OP> whole macro thing altogether, since sed -i -e s/pyton34-/python36-/
> OP> *.spec does 90% of the work?
>
> Right now the process is:
>
> fedpkg request-repo --summary "Summary of foo" --exception python36-foo
> fedpkg request-branch --repo python36-foo epel7
> (wait)
> fedpkg clone python36-foo
> cd python36-foo
> fedpkg retire "This is an EPEL-only package"
> fedpkg switch-branch epel7
>
> And then copy in your spec and sources, edit and go.  There is no need
> for a package review.
>
> That's certainly a few steps but far from the worst process in the
> distro.  What could we do to make it easier?
>

Wait, we can do that? I thought we couldn't use the exception process for this?



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Your package will fail to build due to ambiguous Python shebang

[Neal Gompa]

On Mon, Sep 10, 2018 at 7:23 AM Miro Hrončok  wrote:
>
> Hi, according to my query, latest build of your package failed the
> Taskotron check that checks if the brp-magle-sehbang script mangled the
> shebangs from python to python2.
>
> See something like this in the build.log:
>
> *** WARNING: mangling shebang in ... from #!/usr/bin/env python to
> #!/usr/bin/python2. This will become an ERROR, fix it manually!
>
> This will soon become an error in F30 as promised 2 releases ago.
>
> https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error
>
> The motivation is to not drag dependency on python2 when it is not
> needed. If you are hit by this error, you should check whether the
> executable Python file with the Python shebang is actually needed in
> your package. If it is needed, fix the shebang manually to python3 or
> python2 explicitly.
>
> Here's how:
>
> https://fedoraproject.org/wiki/Changes/Make_ambiguous_python_shebangs_error#Using_pathfix.py_to_fix_shebangs
>
> And here is the list:
>
> Packages by maintainer:
> ngompa appliance-tools pagure zypper

I'm working on fixing these. "pagure" will change to Python 3 in F29+
soon, and I'm working on porting appliance-tools and zypper to Python
3.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org

Re: Python 3.7 status in Fedora

[Neal Gompa]

On Tue, May 29, 2018 at 11:14 AM Miro Hrončok  wrote:

> Hi, this is just a summary about what is currently happening with Python
> 3.7 in Fedora [0].

> Upstream has delayed the first RC a bit [1]. So we are not pushing
> anything to rawhide just yet. However I'm working on a "topsort rebuild"
> in Copr [2] on 3.7.0b3 to see what order [3] we'll need once this goes
> to rawhide.

> I'm currently trying to approach a state where the following packages
> are successfully built:

>* dnf
>* cloud-init
>* anaconda
>* abrt
>* python-flask
>* python-django
>* koji
>* bodhi
>* fedpkg
>* dnf-plugins-core

> (Anything else that should be on the list? Note that sphinx, pytest etc,
> is included as a dependency, I'm mostly trying to build apps.)

> Several packages are blocking this effort, all have open bugs at [4], if
> you'd like some upstream involvement, go ahead and look at some, I won't
> be able to fix them all alone :)

Mock and its dependencies should be included in there, too. I recall that
at least there was one issue encountered by OpenMandriva during their
Python 3.7 transition in which one of Mock's dependencies broke due to a
newly reserved keyword. I don't remember which one it was, though...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/O6ROVCAVEOYZWDPJ2M44OTTIY2GVFJ7O/

Re: RFC: Drop python-rpm-generators for the subpackage to be built as part of the rpm package itself

[Neal Gompa]

On Mon, Jan 15, 2018 at 3:06 AM, Panu Matilainen <pmati...@redhat.com> wrote:
> On 01/14/2018 04:46 PM, Neal Gompa wrote:
>>
>> On Sun, Jan 14, 2018 at 7:22 AM, Igor Gnatenko
>> <ignatenkobr...@fedoraproject.org> wrote:
>>>
>>> On Thu, 2018-01-11 at 10:37 -0500, Neal Gompa wrote:
>>>>
>>>> Hey,
>>>>
>>>> As I was looking at the rpm.spec and fixing up a small bit of the
>>>> build for the Python bindings, it occurred to me that we have left
>>>> python-rpm-generators out of sync with rpm.
>>>>
>>>> I'm wondering if it even makes sense to keep that as a separate source
>>>> package, and instead build the python3-rpm-generators subpackage from
>>>> the main rpm package. I'd much rather do that as it makes it much
>>>> easier to keep everything in sync.
>>>>
>>>> We inherited the split from Fedora, who did it for their failed
>>>> modularity effort to try to make a base system environment that was
>>>> only C and shell but also contain the rpm-build package (I didn't say
>>>> it made sense...). From the point of view of Mageia, who is unlikely
>>>> to attempt what Fedora is doing, I'm not sure there's a particular
>>>> value in doing so. I was not even particularly enthused about this
>>>> when Fedora did it...
>>>>
>>>> Insofar as bytecompilation issues are concerned, we should probably
>>>> rename the file on install to not have the .py suffix, so that the
>>>> Python interpreter doesn't consider it as something to bytecompile.
>>>> That will allow us to avoid the issues that necessitate python -> rpm
>>>> -> python rebuilds.
>>>>
>>>> What do you all think?
>>>
>>>
>>> To be honest, I would like to separate even upstream to its own
>>> repository. I
>>> don't think it makes sense to keep it part of RPM itself.
>>>
>>> CCing python SIG and RPM maintainers for their feedback.
>>> - --
>>> - -Igor Gnatenko
>>
>>
>> If we were going to do something like that, I'd actually like to have
>> a repository in rpm-software-management where we can contribute
>> dependency generators of all kinds beyond the basic ones included in
>> RPM for everyone to use. It makes no sense for people to reinvent the
>> same things over and over simply because they didn't know it was
>> already written somewhere.
>
>
> Yes, that's kinda been the idea ever since making it possible in rpm 4.9.x,
> and which is why I've encouraged people splitting their generators out of
> main rpm package. It's just the common upstream part that is lacking...
>

Now that we have the GitHub organization, I don't see why we couldn't
have a repository that could serve this purpose.

Don't get me wrong, I think there needs to be a basic set of
dependency generators in the main RPM repo. The current ones are fine
in the main repository, but there's *a lot* of generators out there,
written by Fedora, SUSE, Mageia, and others. Putting them somewhere in
one place can help us reduce the duplication and ensure things are
consistent, as well as helping to improve the quality of those
generators anyway.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: RFC: Drop python-rpm-generators for the subpackage to be built as part of the rpm package itself

[Neal Gompa]

On Sun, Jan 14, 2018 at 7:22 AM, Igor Gnatenko
<ignatenkobr...@fedoraproject.org> wrote:
> On Thu, 2018-01-11 at 10:37 -0500, Neal Gompa wrote:
>> Hey,
>>
>> As I was looking at the rpm.spec and fixing up a small bit of the
>> build for the Python bindings, it occurred to me that we have left
>> python-rpm-generators out of sync with rpm.
>>
>> I'm wondering if it even makes sense to keep that as a separate source
>> package, and instead build the python3-rpm-generators subpackage from
>> the main rpm package. I'd much rather do that as it makes it much
>> easier to keep everything in sync.
>>
>> We inherited the split from Fedora, who did it for their failed
>> modularity effort to try to make a base system environment that was
>> only C and shell but also contain the rpm-build package (I didn't say
>> it made sense...). From the point of view of Mageia, who is unlikely
>> to attempt what Fedora is doing, I'm not sure there's a particular
>> value in doing so. I was not even particularly enthused about this
>> when Fedora did it...
>>
>> Insofar as bytecompilation issues are concerned, we should probably
>> rename the file on install to not have the .py suffix, so that the
>> Python interpreter doesn't consider it as something to bytecompile.
>> That will allow us to avoid the issues that necessitate python -> rpm
>> -> python rebuilds.
>>
>> What do you all think?
>
> To be honest, I would like to separate even upstream to its own repository. I
> don't think it makes sense to keep it part of RPM itself.
>
> CCing python SIG and RPM maintainers for their feedback.
> - --
> - -Igor Gnatenko

If we were going to do something like that, I'd actually like to have
a repository in rpm-software-management where we can contribute
dependency generators of all kinds beyond the basic ones included in
RPM for everyone to use. It makes no sense for people to reinvent the
same things over and over simply because they didn't know it was
already written somewhere.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

openSUSE Python packaging and Fedora

[Neal Gompa]

Hello all,

I've been recently somewhat involved in openSUSE stuff, and I caught
wind of a recent initiative to update their Python packaging.

Some background here: As some of you may know, for whatever unknown
reason, openSUSE has historically had separate source packages and
projects for Python 2 and Python 3 packages[1][2]. While this approach
does make packaging more trivial in some regards, it has had the
unfortunate consequence of making it so that the Python 2 and Python 3
packages of the same module often were wildly out of sync.

Last month, Jan Matejek (SUSE's Python stack maintainer, who is CC'd
to this email) unveiled his "singlespec" project[3][4][5] and began
transitioning Factory (their equivalent of Rawhide) to it. Factory has
been undergoing the transition for the last month rather successfully.

Now, in Fedora (and Mageia), "single spec" is not new to us. We've
been doing this for years. However, what is new is that openSUSE's
packaging structure allows for dynamically generating "flavors" as
subpackages to target Python 2, Python 3, PyPy, or whatever other
implementation of Python you wish to support. For people familiar with
openSUSE's Ruby packaging, this is not new, as it works the same way
there.

I've taken a look at how the new openSUSE Python packaging works, and
it appears to have shades of how Jason Tibbs' prototype Python
packaging works[6]. I've not dug too deeply into Jason's work, but
they do appear conceptually similar.

A deeper examination of the openSUSE packaging macros[7] reveals that
it's designed to be mostly a superset of our macros in Fedora and
Mageia. All the macros we currently use are defined in their macros,
except for one: %python_provide. For the openSUSE macros, the
"flavors" capability supersede the usage of %python_provide.

My main critique of the openSUSE packaging is that it enforces the
usage of alternatives to select the "unversioned" binary. In my view,
openSUSE uses alternatives too much across Python, Ruby, and other
languages where they have multiple stacks supported. I'd much rather
enforce a particular version of the binaries being unversioned
(preferably the Python 3 version, unless there's some compelling
reason otherwise) and have that always be used.

Another issue is that it depends on Python packages being reformatted
to match the new convention. Unlike openSUSE, we have not been very
good about pushing to restructure all of our Python packages to match
our new preferred convention. That being said, we also have the
advantage of having a Python dependency generator (though we are only
using it for generating Provides at the moment, Mageia uses an earlier
version of it to generate Provides and Requires, and openSUSE doesn't
have this at all). If we were to incorporate their work, it could be
modified to leverage our dependency generator so that this is less of
a problem. We could fall back to the structured Python module package
names when we're attempting to use modules that don't have the
Provides (which are basically modules not built using setuptools or
distutils).

However, I think that their work is complementary, and at least we
could incorporate their macros into our standard set of Python macros.
I would like to see us work with the openSUSE guys to see if we can
come together on a unified standard of macros. I believe there's
certainly a desire on both sides (especially with the OpenStack
packaging teams, who'd like to not have to duplicate each other's work
for their respective distributions).

What do you guys think?

Best regards,
Neal


[1]: https://build.opensuse.org/project/show/devel:languages:python
[2]: https://build.opensuse.org/project/show/devel:languages:python3
[3]: https://lists.opensuse.org/opensuse-packaging/2017-02/msg00105.html
[4]: https://lists.opensuse.org/opensuse-packaging/2017-02/msg00107.html
[5]: https://build.opensuse.org/project/show/devel:languages:python:singlespec
[6]: https://pagure.io/python-macros
[7]: https://github.com/openSUSE/python-rpm-macros

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: Making sudo pip Safe

[Neal Gompa]

On Wed, Dec 7, 2016 at 7:53 AM, Michal Cyprian  wrote:
> - system-python (i.e. what all programs installed via DNF will use) is 
> limited to site-packages under /usr, so DNF-installed software is unaffected 
> by anything installed with pip

system-python is not intended for this use-case. It was designed to
support DNF itself and other related system software. It's also
intentionally set up to not have everything that regular python3 has.
I would really hesitate to allow this to be used for more than that,
because then we're back to square one, again...


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: [RFC] RPM's Python dependency generator

[Neal Gompa]

On Thu, Dec 1, 2016 at 8:36 AM, Igor Gnatenko <ignate...@redhat.com> wrote:
> On Wed, Nov 30, 2016 at 2:53 PM, Tomas Orsava <tors...@redhat.com> wrote:
>> On 11/30/2016 02:44 PM, Neal Gompa wrote:
>>>
>>> On Wed, Nov 30, 2016 at 8:41 AM, Tomas Orsava <tors...@redhat.com> wrote:
>>>>
>>>> I don't think the depgen should be enabled by default, at least not in
>>>> the
>>>> foreseeable future. IIRC it's not that well implemented—e.g. I believe it
>>>> doesn't read requirements.txt for example (but I might be wrong).
>>>> There will be a lot of cases where the generated requirements are
>>>> incomplete, or contain unnecessary entries, etc. I think it should remain
>>>> an
>>>> opt-in.
>>>>
>>> According to various Python people, we're not actually supposed to
>>> read requirements.txt. That file is explicitly designed for people to
>>> individualized deployments. The proper place to get this information
>>> is from the egg-info/dist-info data, which is what we read. The fact
>>> that some people abuse requirements.txt and have it read in by their
>>> setup.py is beside the point. Whatever the setup.py (thus
>>> pip/easy_install/etc.) says it needs, the generator will dutifully
>>> report.
>>
>>
>> The fact remains in too many cases it will need to be manually adjusted, it
>> won't be foolproof.
>> Therefore I argue it would be better for it to be an opt-in so that new
>> packagers don't immediately have to jump in into debugging a depgen they
>> have no clue how really works.
> We'll see how it will go. we have depgen for pkgconfig, libraries,
> etc. for many years and people don't go and debug it immediately, but
> for many of packages it will help a lot. Anyhow, we'll see after
> couple of releases.
>
> Neal suggested to have:
> %__python_requires
> %{_rpmconfigdir}/%{?pythondistdeps_enable:pythondistdeps.py}%{!?pythondistdeps_enable:pythondeps.sh}
> --requires
> in python.attr inside RPM.
>
> I tested it and it just works once I specify `%global
> pythondistdeps_enable 1` in spec. Can you help me to get this
> included? With RPM part it's clear how to get this, but updating
> guidelines and other stuff...

This will also drastically simplify the work of tools like pyp2rpm,
since instead of having to do crazy processing of module names, it can
just use the appropriate pypi provides/requires. If the macro template
enables the requires generator, it won't even need to specify requires
at all, as they'll be generated from the wheel data anyway.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: [RFC] RPM's Python dependency generator

[Neal Gompa]

On Wed, Nov 30, 2016 at 8:41 AM, Tomas Orsava  wrote:
>
> I don't think the depgen should be enabled by default, at least not in the
> foreseeable future. IIRC it's not that well implemented—e.g. I believe it
> doesn't read requirements.txt for example (but I might be wrong).
> There will be a lot of cases where the generated requirements are
> incomplete, or contain unnecessary entries, etc. I think it should remain an
> opt-in.
>

According to various Python people, we're not actually supposed to
read requirements.txt. That file is explicitly designed for people to
individualized deployments. The proper place to get this information
is from the egg-info/dist-info data, which is what we read. The fact
that some people abuse requirements.txt and have it read in by their
setup.py is beside the point. Whatever the setup.py (thus
pip/easy_install/etc.) says it needs, the generator will dutifully
report.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: upstream dev. asks suggestions about howto make packagers work easier (bundled libraries, etc.)

[Neal Gompa]

On Mon, Nov 21, 2016 at 4:42 AM, Piotr Ozarowski  wrote:
> Hi,
>
> [Germano Massullo, 2016-11-20]
>> We often deal with upstream developers that bundle libraries in their
>> code, so to make a package we have to debundle them, etc.
>> This time, an upstream dev. asked me what he could do to make easier
>> the work of packagers.
>> In this case the software is python-netjsongraph [1] that bundles
>> javascript-d3 library and that is being reviewd at [2]
>>
>> I think it would be nice to make a discussion even for non Python
>> packages, so we can elaborate a sort of vademecum that a packager
>> could show to upstreams when there is a collaboration between them.
>
> In Debian we have https://wiki.debian.org/UpstreamGuide
> I guess more official, cross-distro document that we all point upstream
> authors to would be a nice idea as well.

It would be a great idea if there were principles that all
distributions could agree on and have a document codify that so that
all distributions can reference it to upstream.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: [opensuse-buildservice] introducing new macros across the whole OBS?

[Neal Gompa]

On Thu, Nov 10, 2016 at 7:49 AM, jan matejek  wrote:
> Hello,
> in relation to the python single-spec initiative, I have designed a set
> of new macros that allow significant automation in building Python packages.
>
> However, these are constrained by the %python_module macro used in
> BuildRequires.
> The purpose of this macro is to expand %{python_module foo} to
> python2-foo, python3-foo, etc., based on which python flavors are build
> targets for the package. Obviously, the buildservice itself must be able
> to do this expansion, otherwise it can't resolve the build requirements.
>
> So far I have solved this by placing all the macros in prjconf. But this
> is not viable if I want my macros to be available everywhere. I can
> create a "python-macros" package for all the other macros, and
> buildrequire it, in order to make the packages build in any distro that
> has this package, but I can't do the same for %python_module.
>
> Is it possible to make the buildservice itself recognize this macro, or
> place it in some sort of global prjconf? Ideally without triggering a
> full rebuild of everything?
> Or do you have any tips for alternate solutions?
>
> One thing that occured to me, instead of relying on macros, turn
> "python-foo" packages into metapackages requiring the respective
> "python2-foo", "python3-foo" etc., and then BuildRequire the
> metapackage. I don't have a clear idea about how well this would work,
> it seems rather problematic.
>
> I don't want to make packagers list the buildrequires by hand, because
> the requirements can be numerous and we also want to be able to add more
> python flavors and expand the build requirements accordingly.
>
> thanks for any tips
> m.
>

Why not pull in the dependency generator I upstreamed into RPM[1] into
openSUSE? Fedora is using it now in Fedora 25 and Rawhide[2][3], and
Mageia is using an earlier variant of it now. It establishes common
names for Python packages that provide egg-info or dist-info data
(egg/wheel metadata installed on the system) that can be used for
standardizing how Python dependencies are referenced.

Fedora is currently only using it for Provides generation at the
moment, but Mageia uses it for Provides and Requires generation. That
would make it less onerous in general to package Python stuff, as it
would gain useful dependency generation like Perl and Ruby have. Tools
like pyp2rpm[4] can use these names for generating good spec files
that can build packages properly. Petr Viktorin gave a presentation at
Flock 2016[5][6] about the future of Python packaging in Fedora, with
this as a key staple.

[1]: 
https://github.com/rpm-software-management/rpm/blob/master/scripts/pythondistdeps.py
[2]: http://pkgs.fedoraproject.org/cgit/rpms/rpm.git/tree/
[3]: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/UZ5IZFW7AZVIU3XH6EX4262NVVJRJZUZ/
[4]: https://github.com/fedora-python/pyp2rpm
[5]: https://speakerdeck.com/encukou/python-packaging-in-fedora
[6]: https://www.youtube.com/watch?v=rp5jq-25nZg

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: PEP: Distributing a Subset of the Standard Library

[Neal Gompa]

On Tue, Sep 6, 2016 at 12:41 PM, Tomas Orsava  wrote:
> Hi!
>
> I'm currently writing a PEP titled "Distributing a Subset of the Standard
> Library" to standardize and hopefully improve the behavior of Python without
> the its full standard library. This is relevant to Fedora, as we exclude
> several standard library modules into separate optional packages (e.g.
> python3-tkinter).
>
> I have a draft of the first two sections: Motivation and Specification.
> https://fedora-python.github.io/pep-drafts/pep-A.html
>
> The source can be found here: https://github.com/fedora-python/pep-drafts

Why doesn't Python offer dist data indicating the modules that are
included in the standard library? That way there doesn't need to be
much special handling to support whether it's an external package or a
standard library module. Things like dependency generators can pick
this up properly.

I think Perl does it that way...



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: Renaming python to python2

[Neal Gompa]

On Thu, Sep 1, 2016 at 8:33 AM, Avram Lubkin <av...@rockhopper.net> wrote:
> On Thu, Sep 1, 2016 at 8:14 AM, Neal Gompa <ngomp...@gmail.com> wrote:
>>
>> Alternatives doesn't work in this case because Python 2.x and Python
>> 3.x versions don't share resources, even with minor versions of the
>> same series. Enabling alternatives for it is a recipe for disaster.
>
>
> Could you elaborate? The use case we are talking about is what gets executed
> when you run /usr/bin/python. i.e. symlink management, which is what
> alternatives is intended for.
>

Sure, but those scripts may not actually work because modules that are
supposed to be there, aren't. For example, if you depend on a
non-standard lib module, then that means it needs to be installed for
each python version supported. How do you expect to guarantee that?

Again, alternatives is only for things that operate functionally
identically. That's not even true between Python 2.6 and Python 2.7 if
they are installed in parallel because they can't use the same module
path. Likewise for Python 3.5 and 3.6.

You're essentially asking for unpredictable breakage.

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: RFC: Using the new optional python module dependency generator in Fedora

[Neal Gompa]

On Wed, Apr 13, 2016 at 8:43 AM, Miro Hrončok  wrote:
>
> Hi, this looks very good.
>
> I tried to build some arched packages and was wondering, if there shouldn't
> be %{?_isa} included at the end of the provide name?
>
> Something like:
>
> Provided form the 64bit package:
>
> python3.5dist(foo)(x86-64)
> python3.5dist(foo)
>
>
> Provided form the 32bit package:
>
> python3.5dist(foo)
>

I'm not sure. Historically, dependency generators of this kind don't
include the architecture information because of the issues related to
determining whether a package is a noarch package or an archful
package (or if it needs to transition from one to the other). It's
somewhat easier to tell on 64-bit (due to path differences), but
harder on 32-bit, because the paths are the same.

I'm also not sure what happens when you just naively add %{?_isa} to a
Provides/Requires set in a noarch package.



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

RFC: Using the new optional python module dependency generator in Fedora

[Neal Gompa]

Hello all,

It's been a while since I messaged this list about the new dependency
generator being upstreamed into RPM[0]. Since then, I've taken your
valuable feedback and incorporated it into the version that now sits
in RPM git master[1].

A little bit ago, I pushed a package to Copr[2] that includes this
updated dependency generator, and I would love for people to take a
look at it, try it out, and see how well it works. Instructions on
setting up and configuring are on the Copr page.

I'd also love this become enabled in Fedora's Python stack for Fedora
25, giving us nice Provides/Requires chains that are independent of
the package name, but map more closely to PyPi names that people would
recognize from using pip or other tools.

We already have similar things in place for Ruby, Mono, Perl, and even
MinGW in Fedora, so it'd be nice to have it on the Python side, too.

Best regards,
Neal

[0]: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/SQBSAS4T25HK5YJBNBSFDD7KDQWDELL6/

[1]: 
https://github.com/rpm-software-management/rpm/blob/master/scripts/pythondistdeps.py

[2]: https://copr.fedorainfracloud.org/coprs/ngompa/rpm-depgen-pythondistdeps/

-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: New (optional) python egg dependency generator for RPM

[Neal Gompa]

On Sun, Nov 22, 2015 at 10:11 PM, Nick Coghlan <ncogh...@gmail.com> wrote:

> On 22 November 2015 at 04:18, Neal Gompa <ngomp...@gmail.com> wrote:
> > Based on the feedback from you guys, I've made the changes to move to
> > pythonX.Ydist() in the dependency generator. That code has been
> > submitted as a pull request to the RPM GitHub repository[0]. I also
> > added a switch for those who want pythonXdist() Provides, but it is
> > opt-in rather than opt-out. The option is only for distributions that
> > intend to carry only one Python runtime per major version.
>
> Very cool, thank you!
>
> Cheers,
> Nick.
>
>
​I do still have one quirk that I can't quite figure out what is causing
it. The quirk I mentioned at the beginning of this thread, where it adds an
unversioned python(abi) Requires when being executed under Python 3.x. This
doesn't happen on Python 2.7, and I don't know why.

Anyone have any thoughts on why that's happening? I'm totally bamboozled on
it, so I'd love some help to fix it. :)



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org

Re: New (optional) python egg dependency generator for RPM

[Neal Gompa]

On Wed, Nov 18, 2015 at 2:48 AM, Nick Coghlan  wrote:
>
> On 18 November 2015 at 02:29, Jason L Tibbitts III  wrote:
> >> "NC" == Nick Coghlan  writes:
> >
> > NC> If so, then there's some relevant work currently under way upstream
> > NC> to improve the interaction between Python installation tools and
> > NC> build systems to improve the metadata extraction process, rather
> > NC> than relying on implementation details of setuptools.
> >
> > If that's the case, could someone bang out a few paragraphs that we
> > could use as a blueprint for some packaging guidelines?  An example
> > spec, or even just the file layout and some idea of how autogenerated
> > dependencies would work would be enough.  I know this stuff is a bit
> > new, but we've been doing a really big overhaul of the python stuff and
> > we'd like to at least design to accommodate this rather than having to
> > do it all over again once this new format comes out.
>
> The main policy changes would be to update these two section to
> mention keeping dist-info directories:
>
> * https://fedoraproject.org/wiki/Packaging:Python#Files_to_include
> * https://fedoraproject.org/wiki/Packaging:Python#Reviewer_checklist
>
> It would also be desirable to state a preference for dist-info over egg-info.
>
> Thinking about it a bit further, I don't think the latest round of
> upstream changes should impact the metadata analysis step, as the
> metadata querying changes are designed to support getting at the
> metadata without building and installing the package first, and that's
> not a consideration for the RPM use case.
>
> However, they could potentially affect the py2/3_build macros, as
> we're looking to finally migrate away from *requiring* the presence of
> a setup.py file in every source tree, and instead allow out-of-tree
> build tools, with machine readable instructions for bootstrapping them
> into the build environment.
>
> The draft PEP for that is at
> https://github.com/pypa/interoperability-peps/pull/54/files and
> upstream discussions are on distutils-sig
>
> I'd been thinking using "pip install" instead of "setup.py install" in
> the build macros would be sufficient, but I now realise that isn't the
> case - if a project uses flit (for example) as its build utility, then
> we're going to need to generate a suitable BuildRequires in pyp2rpm
> and similar tools (perhaps using the "BuidlRequires:
> pythonX.Ydist(flit)" format). The build macros themselves could still
> delegate the task of working out the right build command to invoke to
> pip, though.
>

The main issue I see with that is how to make it so that python upgrades
aren't obnoxiously painful. If BuildRequires use pythonXdist(module) format,
but all *generated* runtime requirements use pythonX.Ydist(module) format,
this problem goes away. But as Toshio mentioned, how do we solve that in a
multi-version environment (like Enterprise Linux, for instance)?

Using pythonX.Ydist(module) for BuildRequires effectively locks the module
to a specific Python version until each and every maintainer upgrades them.
That is an awful thing to have to do, and no other programming environment
in any RPM-based distribution requires that. Most of the time, this is an
unnecessary burden on the package maintainers.

My view on pythonXdist(module) vs pythonX.Ydist(module) for
BuildRequires is that DNF/Zypper may actually solve this issue for us.
Perhaps presenting it with pythonXdist(module) and a package that provides
the appropriate "python(ABI) = X.Y" as part of the builddep grab will actually
pick the right one (after all, each module would Require a specific
"python(ABI)" anyway). I'm not sure if Yum would do the same, though
(I hope it does!). I suppose the key is whether or not the depsolver analyzes
the whole request before creating its proposed transaction, rather than
iteratively solving and presenting the results.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
python-devel mailing list
python-devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/python-devel@lists.fedoraproject.org