On 2020-08-12 18:19, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 12:02 PM Petr Viktorin <pvikt...@redhat.com> wrote:
On 2020-08-12 17:22, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 11:19 AM Petr Viktorin <pvikt...@redhat.com> wrote:
On 2020-08-12 16:53, Neal Gompa wrote:
On Wed, Aug 12, 2020 at 10:23 AM Petr Viktorin <pvikt...@redhat.com> wrote:
@Conan-Kudo:
Is there something you've done here to make the new macros unequivocally
better? Do you have automated flavor builds, for example? Right now there is no
effective difference.
The new macros allow other build tools than just setuptools.
They also use upstream metadata for BuildRequires & file lists.
Of course, where possible the changes were backported to the existing
macros. I don't necessarily see the macros as the main thing that
changes. But when you look at a specfile, you can usually tell what
conventions it uses by what macros you see.
Those are all nice things, for sure. I think it'd be important to
spell that out somewhere.
Do you mean providing a summary of the changes between the existing
guidelines and these?
Yes. Also indicating the *why* for pyproject stuff is useful within
the context of why the old macros are deprecated.
Is that necessary for the beta release of the packaging guidelines?
(i.e. when they would be in effect, but optional?)
You're asking for quite a lot of work around describing a document that
might still change.
I am only asking for what makes sense if the document as it stands
were to be finalized.
OK, you're right. I added a note to the checklist.
## PyPI parity
Every Python package in Fedora **SHOULD** also be available
on [the Python Package Index](https://pypi.org) (PyPI).
The command `pip install PROJECTNAME` **MUST** install the same package
(possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager **MUST** contact upstream about this.
The goal is to get the project name registered or blocked on PyPI,
or to otherwise ensure the rule is followed.
To block a project name on PyPI, email
[ad...@pypi.org](mailto:ad...@pypi.org),
giving the project name and explaining the situation
(for example: the package cannot currently be installed via `pip`).
You can ask questions and discuss the process at the [Python
Discourse](https://discuss.python.org/t/block-names/4045).
> NOTE: Project names that were in Fedora but not on PyPI
> when these guidelines were proposed
> are *blocked* from being uploaded to PyPI.
> This prevents potential trolls from taking them,
> but it also blocks legitimate owners.
> If your package is affected, contact the Python SIG
> or [file a PyPA
issue](https://github.com/pypa/pypi-support/issues/new?labels=PEP+541&template=pep541-request.md&title=PEP+541+Request%3A+PROJECT_NAME)
> and mention `@encukou`.
This is necessary to protect users,
avoid confusion,
and enable automation as Fedora and upstream ecosystems grow more
integrated.
As always, [specific exceptions can be granted by the Packaging
Committee](https://docs.fedoraproject.org/en-US/packaging-guidelines/#_general_exception_policy).
@Conan-Kudo:
This is not reasonable to ask of packagers to do. Such a check is difficult to
do, and there is no particularly compelling reason for making everything
written in Python using Python build system available in PyPI. Unless you plan
to provide an automated system to inform PyPI of these things, this is not only
unreasonable, it is unenforceable.
A lot of stuff in the guidelines is unenforceable, so let's focus on the
"unreasonable" part.
There is no reason to have everything available on PyPI, but I do
believe it's reasonable to *reserve the name* in such cases.
Here's a reason why I want this:
The issue here is that Python tools have access to project names. For
example, I can get the version of Requests using:
>>> from importlib.metadata import version
>>> version('requests')
'2.22.0'
Things like this are only useful if we use a common namespace. Upstream,
that namespace *is* PyPI; there's little we can do about that.
If project names mean something else in Fedora than in the wider
ecosystem, we'll get user confusion at best.
(If you use a private package index, like they do at CERN or some
closed-source shops, there can be some collisions -- but in that case
there's a limited number of software authors and users, and a lot more
control over the package set. Conflicts in global repositories of
free/open-source software are much harder to manage.)
Lately, I think about another way to handle this: packages that aren't
on PyPI could not ship the .dist-info at all, and so, they would not
have things like `python3dist(...)` provides. They'd only be usable with
Fedora tooling, not in the wider Python ecosystem.
It's a major case to think out and test, but maybe it would be worth it?
I think omitting the Provides is unfair.
Why?
In my view, `python3dist(...)` is using the namespace of the wider
Python ecosystem.
In Fedora, you can always use Fedora package names. But for Python
tools, we need to use names from PyPI.
I disagree here. There is nothing inherent about pythonXdist()
metadata that indicates it is associated with PyPI. It's *nice* if
it's in PyPI, but not even Fedora projects themselves do it, so...
Well, this *is* the main change this document brings: it says that
`pythonXdist(...)` now *will* inherently use the PyPI namespace, the
default repository of open-source packages. I'm not documenting the
status quo. I am saying the Fedora projects that don't currently do this
are should be considered wrong and should be fixed.
I `pip install foo` (in a venv with system site packages), and the PyPI
package `foo` depends on `bar`, I should get the `bar` from PyPI even if
I have some unrelated, Fedora-only `python3-bar` package.
This is what people expect, because the Fedora and PyPI namespaces
already *almost* always overlap. When they don't, users are confused.
And as we automate stuff to use upstream metadata more, the tools will
"get confused" as well.
If you do need a Fedora-only namespace, regular package names are
completely adequate. But we need a *Python-ecosystem-wide* namespace.
And that is PyPI.
I'm saying that we do not necessarily need it. The *reason* I pushed
for the pythonXdist() names to be supported in Fedora and did the work
to make the generator acceptable for Fedora was because I explicitly
wanted to stop caring about how people named the packages. They
differed across distributions, and that made packaging software that
depended on them quite difficult.
Well, for syncing names across distros, wouldn't a cross-distro, or
distro-agnostic, namespace be useful?
I *never* intended for it to be used
as a hammer to force everyone to go through the drudgery of pushing
stuff into PyPI.
Is that drudgery? I never thought of it that way.
Is there something to improve so that it's not that much of a drudgery?
I do not necessarily disagree that PyPI and Fedora pythonXdist() names
should match, but I disagree on burdening packagers with this. I also
fundamentally disagree on conditioning those names on PyPI, because
frankly no other ecosystem in Fedora works that way. Not Rust, not Go,
not Perl, not OCaml, not anything.
Who *should* be burdened with this?
I'm aware that no other ecosystem in Fedora does this, but I believe
that they should. (though I'm not an expert on those ecosystems)
RPM predates global per-ecosystem directories of free/open software, but
they are here now, and in my opinion, they offer a very good way to
standardize the names.
Acatually, it's already a de-facto standard: if you see a name in
install_requires in setup.py of some random upstream project you want to
package, you can assume that it's the PyPI name.
From my perspective, you have not demonstrated enough proof that it's
worth making people manually go through that effort. And you're making
an *automatic feature* a reason to compel people to do extra work
without any potential concept of how to simplify this.
Could you elaborate a bit on "that effort"? I guess that from my
position, it's hard to see what the issues are, and what could be
simplified.
If you want PyPI and Fedora Provides to match and upstream doesn't
release to PyPI, you need a process to auto-register them so that
upstream can be contacted to upload to PyPI.
OK.
Currently the process is to e-mail Python SIG. I can't speak for Python
SIG, but I could make it a duty of Red Hat's python-maint team to handle
such requests promptly. If it gets out of hand, we can add automation
(or amend the guidelines), but I doubt that will happen.
Actually, at this point I'd actually like to have all such requests go
through me. I'm quite interested in reasons why people don't release to
PyPI.
(Could you share yours?)
_______________________________________________
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org
