Re: RFC: Mangle Python shebangs to fully qualified interpreter (/usr/bin/pythonX.Y)

2020-11-03 Thread Miro Hrončok

On 11/3/20 6:32 PM, Neal Gompa wrote:

Hey all,

An "interesting" Linux support issue at work with another Linux
distribution opened my eyes to the possibility of how badly things
would break if /usr/bin/python3 was swapped or otherwise altered by an
external force. In such a scenario where it was overridden to point to
another Python 3 interpreter that wasn't the system one, things would
break pretty badly.

Now that we've been moving toward a model where it becomes possible
for first-party or third-party multi-Python RPMs, I'm wondering if it
would make sense to evolve the shebang mangling for Python to resolve
to the fully qualified Python interpreter path (that is,
/usr/bin/pythonX.Y instead of /usr/bin/pythonX) for packaged Python
code. Presumably this may make things easier for switching Python
versions without everything breaking, too.

In practice, I don't think this is a *huge* issue Fedora-side, since
we don't manage /usr/bin/python3 via alternatives. But I can easily
see people wanting /usr/bin/python3 to be swappable in Red Hat
Enterprise Linux, while everything is still parallel installable and
fully functional. The Python interpreter packaging even mostly
supports this now.

What do y'all think? Is this doable? Is this desirable?


Hey Neal. I am glad you brought this up. In fact, we've been discussing this 
internally in the Python Maint team on several occasions.


Some of the conclusions:

We don't want to support /usr/bin/python3 to be swappable in RHEL. The 
alternatives mechanism in RHEL 8 proved to be painful to get right both from the 
maintenance perspective and users/sysadmin perspective. We want people to be 
able to to use a well-known entry point (/usr/bin/python3) as "the Python that's 
running the system". /usr/libexec/platform-python is a downstream-only hack (we 
plan to keep it for backwards compatibility, but that's it).


IMHO people who want swappable python3 commond don't *actually* need 
/usr/bin/python3, they can have /usr/local/bin/python3 or ~/.local/bin/python3.


Changing shebangs to fully qualified (X.Y) Python version brings additional 
bootstrapping issues when upgrading Python. For example, in Fedora, we currently 
have ~3500 source packages that build at least one package requiring python(abi) 
and/or libpython X.Y specifically. Such packages need to be bootstrapped when we 
upgrade Python and it is not trivial. On top of that, we have ~400 more 
requiring /usr/bin/python3 "only". Adding them to the problem is not 
significant, but it is more work nevertheless.


Additionally, if we support swappable /usr/bin/python3, shebangs are the easy 
part. We have learned the hard way (with /usr/bin/python) that shebnags are 
easily checked/asserted/mangled/counted, but it's the non-shebang invocations 
that are lurking everywhere. The only way to actually discover the usage in 
Fedora packages was to remove the executable from the Python package and it took 
many releases to get rid of all of them (but honestly, we cannot be sure if we did).


OTOH I've been thinking we should set the shebang of dnf to a fully qualified 
Python version as an exception: In case people will get ideas about changing the 
/usr/bin/python3 symbolic link target (as I am sure they will), at least they 
would be able to run `sudo dnf reinstall python3` to recover.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org


RFC: Mangle Python shebangs to fully qualified interpreter (/usr/bin/pythonX.Y)

2020-11-03 Thread Neal Gompa
Hey all,

An "interesting" Linux support issue at work with another Linux
distribution opened my eyes to the possibility of how badly things
would break if /usr/bin/python3 was swapped or otherwise altered by an
external force. In such a scenario where it was overridden to point to
another Python 3 interpreter that wasn't the system one, things would
break pretty badly.

Now that we've been moving toward a model where it becomes possible
for first-party or third-party multi-Python RPMs, I'm wondering if it
would make sense to evolve the shebang mangling for Python to resolve
to the fully qualified Python interpreter path (that is,
/usr/bin/pythonX.Y instead of /usr/bin/pythonX) for packaged Python
code. Presumably this may make things easier for switching Python
versions without everything breaking, too.

In practice, I don't think this is a *huge* issue Fedora-side, since
we don't manage /usr/bin/python3 via alternatives. But I can easily
see people wanting /usr/bin/python3 to be swappable in Red Hat
Enterprise Linux, while everything is still parallel installable and
fully functional. The Python interpreter packaging even mostly
supports this now.

What do y'all think? Is this doable? Is this desirable?


-- 
真実はいつも一つ!/ Always, there's only one truth!
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org