Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-05-01 Thread Nick Coghlan 
On 26 April 2017 at 19:19, Michal Cyprian  wrote:
> The other possibility is to limit the pip install location change
> to distutils and pip [2]. This is the "safer" option, but does
> not cover all corner cases. For example, Python software built
> locally using cmake or similar tools will be installed into
> /usr/lib and can conflict with system tools. Debian chose to
> implement similar solution.

Re-reading https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
after the discussion on the main devel list, I have a few further
comments on this point:

1. I think the tone of the change proposal should be softened to say
it's about make "sudo pip install" *safer*, rather than about making
it absolutely safe - you can still break your system if you're not
careful, the layout change just makes it easier to recover if you do
(at least in most cases)
2. The question of installation directories other than site-packages
(mostly notably /usr/bin) should be explicitly covered, and either
also moved under /usr/local, or else a rationale given for not moving
them yet
3. The proposal should be more explicitly scoped by specifying some
sequences of operations that you want to make safe, such as:

$ sudo pip3 install contextlib2 && sudo pip3 uninstall contextlib2
$ sudo pipsi install ansible && sudo pipsi uninstall ansible

(Getting the latter example to be safe would require changing the
install directory to /usr/local in a way that "pipsi" detects)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-05-01 Thread Nick Coghlan 
On 1 May 2017 at 22:47, Nico Kadel-Garcia  wrote:
> On Sun, Apr 30, 2017 at 10:30 PM, Nick Coghlan  wrote:
>> If the intended benefit of this change remains unclear, it may help to
>> focus on a specific concrete case, which would be that the following
>> operations should be completely indistinguishable at the system level
>> from not having done anything at all (except in the sudo logs):
>>
>> $ sudo pip3 install contextlib2
>> $ sudo pip3 uninstall contextlib2
>
> And this successfully ignores *all* the dependencies which contextlib2
> may add at installation time.

There's a reason I chose contextlib2 as my example rather than
something more complex like ansible - I'm the maintainer of
contextlib2, and I know it doesn't have any runtime dependencies (and
likely never will, since it's a stdlib module backport).

Given the proposal at hand though, writing a
`remove-pip-installed-modules` cleaner utility becomes a lot simpler
that it is today, since it just needs to clear out any Python packages
it finds in /usr/local (based on the Python level installation
records), rather than needing to interact with the RPM database,
figure out which system packages may have potentially been corrupted
and reinstall them.

If the proposal is adjusted to also affect other installation
directories (like the one for scripts), it will also have the
significant benefit that any pip installed binaries will go into
`/usr/local/bin`, so they'll appear on the default path for all
regular users, but *won't* appear on the default path for "root".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-30 Thread Nick Coghlan 
On 1 May 2017 at 09:59, Nico Kadel-Garcia  wrote:
> Wouldn't it be more effective, more safe, and more efficient to
> improve the existing support for doing "the right thing", rather than
> trying to outsmart a build and deployment system that is essentially
> outside your control? To encourage the use of "pyvenv" for local such
> modules, in a local userspace, rather than as a default system
> version?

This is already happening, and is entirely independent of the
discussion at hand (e.g. see [1], a presentation I did with Graham
Dumpleton at Red Hat Summit/DevNation last year advocating for Red
Hat's customers to adopt exactly this model). On Fedora's Developer
Portal, virtual environments share the front page of the Python
section with running Python itself:
https://developer.fedoraproject.org/tech/languages/python/python-installation.html

Responding to "We want to make running 'sudo pip install' less harmful
and easier to recover from when people do it by mistake" with "We
should discourage the use of 'sudo pip install'" remains an
unproductive non sequitur - we *know* it's inherently fragile, and
requires deep knowledge of both Python packaging and RPM packaging to
do reliably. However, Red Hat can't (and won't) answer a commercial
support call from a distressed customer with "Ooh, you shouldn't have
done that - sucks to be you" (although these supportability problems
*are* one of the reasons we only ship "pip" commercially through
Software Collections, and not as part of the system Python runtime).

Fortunately, we have the ability to influence *all* of the moving
parts (upstream pip, upstream CPython, downstream distros) here, and
help to incrementally adjust their respective default behaviours so
that the key risks involved in running "sudo pip install" are
comparable to those involved in running "curl $REMOTE_SCRIPT | sudo
sh".

If the intended benefit of this change remains unclear, it may help to
focus on a specific concrete case, which would be that the following
operations should be completely indistinguishable at the system level
from not having done anything at all (except in the sudo logs):

$ sudo pip3 install contextlib2
$ sudo pip3 uninstall contextlib2

At the moment, that's *not* the case if you had previously done "sudo
dnf install python3-contextlib2", as both of the above commands will
mess with the system contextlib2 packages. With the proposed changes,
contextlib2 may behave unexpectedly while the non-system version is
installed, but removing the pip installed version will be sufficient
to restore the affected machine to its previous state. (I chose
contextlib2 as the example here, rather than something more complex
like ansible, as pip installing components with dependencies makes the
cleanup process more complicated. However, the /usr/local/ split helps
there as well, by making the installed-via-pip versions easier to
identify even without checking the PEP 376 installation metadata)

Cheers,
Nick.

[1] 
https://www.slideshare.net/ncoghlan_dev/developing-in-python-on-red-hat-platforms-devnation-2016

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-28 Thread Donald Stufft 

> On Apr 27, 2017, at 11:08 AM, Barry Warsaw  wrote:
> 
> Also note that `pip install` defaults to --user in Debian, and that alleviates
> a lot of the "well that didn't work, let me add sudo" urges.  Upstream, this
> bug tracks that, although it's long standing without much recent progress:
> 
> https://github.com/pypa/pip/issues/1668 
> 
> 
> IMHO, that's the right direction to go, and I think Donald is philosophically
> aligned (apologies if I'm misreading that, DS!).  There are of course plenty
> of knock-on issues to work out, which is probably why that issue is still
> open.


I think solving this problem (to the degree it can be solved) relies on a 
multifaceted solution.

One part is certainly defaulting to —user in upstream where it makes sense to 
do so, so that people are not incentivized by a permission error by default to 
just throw a sudo in front of their command to make it work.

I think the other part is recognizing that there are valid use cases for ``sudo 
pip install`` (just like there are for doing ``sudo make install``) and 
striving to make that function as best we can. The two things I think need done 
for that are to standardize the “vendor” vs “local” split for site-packages 
that Debian has, so that the two different sources of a package are not 
fighting over dropping files into the same directory. The other thing I think 
that needs to get done is for pip to get better at the concept of overlaying 
installs on sys.path. We shouldn’t need to uninstall something in the “Vendor 
site-packages” in order to install it into the “local site-packages” (And 
similarly for user site-packages).

—
Donald Stufft



___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Nick Coghlan 
On 28 April 2017 at 02:01, Charalampos Stratakis  wrote:
> Fedora's equivalent to dirtbike is rewheel [0][1] and it would be great if 
> these projects could be unified somehow, however that's a separate 
> conversation for another time.
>
> I'd be happy to take a closer look at dirtbike and maybe initiate some 
> discussions when I get some free time.

My recollection from the last time I looked at this was that where
rewheel assumes that the Python level PEP 376 package metadata will be
available, dirtbike instead queried the operating system level
metadata.

So while they're technically solving the same problem (injecting a
system level Python package or bindings into a virtual environment),
they took radically different paths to get there.

The biggest benefit I see to the dirtbike approach is that it has the
potential to work with Python bindings that aren't built with the
standard Python packaging tools, and hence don't have any PEP 376
metadata.

Conversely, the biggest benefit I see to the rewheel approach is that
it lets system packages more clearly indicate whether or not they want
to support rewheel by including or omitting the PEP 376 metadata.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Barry Warsaw 
On Apr 27, 2017, at 12:01 PM, Charalampos Stratakis wrote:

>Fedora's equivalent to dirtbike is rewheel [0][1] and it would be great if
>these projects could be unified somehow, however that's a separate
>conversation for another time.
>
>I'd be happy to take a closer look at dirtbike and maybe initiate some
>discussions when I get some free time.

+1

-Barry


pgph6l0nHuHNe.pgp
Description: OpenPGP digital signature
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Charalampos Stratakis 
Fedora's equivalent to dirtbike is rewheel [0][1] and it would be great if 
these projects could be unified somehow, however that's a separate conversation 
for another time.

I'd be happy to take a closer look at dirtbike and maybe initiate some 
discussions when I get some free time.

[0] 
http://pkgs.fedoraproject.org/cgit/rpms/python3.git/plain/00189-add-rewheel-module.patch
[1] https://github.com/fedora-python/rewheel (kinda outdated)

Charalampos Stratakis
Associate Software Engineer
Python Maintenance Team, Red Hat

- Original Message -
From: "Barry Warsaw" 
To: python-devel@lists.fedoraproject.org
Sent: Thursday, April 27, 2017 5:27:55 PM
Subject: Re: What is your opinion on "sudo pip" fix for Fedora 27?

On Apr 27, 2017, at 04:52 PM, Nick Coghlan wrote:

>From my perspective, the main goal of the change is to make it
>possible to fully recover from "sudo pip install " by doing
>"sudo pip uninstall ", and the approach Debian took is
>sufficient to achieve that in almost all cases. So +1 from me for
>replicating something similar to Debian's approach in Fedora rather
>than blazing new trails by fiddling with `sys.prefix` directly.

All of Debian's deltas from upstream pip are here:

https://anonscm.debian.org/cgit/python-modules/packages/python-pip.git/tree/debian/patches

We also have a somewhat more elaborate machinery to unbundle packages using a
tool called dirtbike to turn installed (at build time) packages into
wheels.  That's a separate issue I'd be happy to talk more about.

In any case, to the extend that we can upstream the more generally useful
bits, or at least share behavior so users will have a similar experience on
both ecosystems, I'd be happy to help represent Debian.  Probably not much
will change there until after the upcoming release though, since the archive
is currently frozen.

Cheers,
-Barry

___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Barry Warsaw 
On Apr 27, 2017, at 04:52 PM, Nick Coghlan wrote:

>From my perspective, the main goal of the change is to make it
>possible to fully recover from "sudo pip install " by doing
>"sudo pip uninstall ", and the approach Debian took is
>sufficient to achieve that in almost all cases. So +1 from me for
>replicating something similar to Debian's approach in Fedora rather
>than blazing new trails by fiddling with `sys.prefix` directly.

All of Debian's deltas from upstream pip are here:

https://anonscm.debian.org/cgit/python-modules/packages/python-pip.git/tree/debian/patches

We also have a somewhat more elaborate machinery to unbundle packages using a
tool called dirtbike to turn installed (at build time) packages into
wheels.  That's a separate issue I'd be happy to talk more about.

In any case, to the extend that we can upstream the more generally useful
bits, or at least share behavior so users will have a similar experience on
both ecosystems, I'd be happy to help represent Debian.  Probably not much
will change there until after the upcoming release though, since the archive
is currently frozen.

Cheers,
-Barry


pgpwOOQCUGDZq.pgp
Description: OpenPGP digital signature
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Barry Warsaw 
On Apr 27, 2017, at 06:21 AM, Donald Stufft wrote:

>For whatever it’s worth, at some point in time I plan on attempting to
>formalize the Debian solution (or something akin to it) within a PEP and try
>to get baked in support for it in Python.
>
>I also feel required to point out that ``sudo pip uninstall`` isn’t always
>enough, because while the two systems install to different locations, pip
>currently will uninstall the files in /usr when installing to
>/usr/local. They’ve patched their copy of pip to refuse to uninstall files
>from /usr, but upstream pip will still do it. Again, something I hope to get
>sorted at some point (and of course, an upgrade totally restores the files in
>/usr).

Also note that `pip install` defaults to --user in Debian, and that alleviates
a lot of the "well that didn't work, let me add sudo" urges.  Upstream, this
bug tracks that, although it's long standing without much recent progress:

https://github.com/pypa/pip/issues/1668

IMHO, that's the right direction to go, and I think Donald is philosophically
aligned (apologies if I'm misreading that, DS!).  There are of course plenty
of knock-on issues to work out, which is probably why that issue is still
open.

Defaulting to --user won't solve all the problems but I think it's very much a
step in the right direction.

Even with this, over in Debuntu-land we sometimes still see bug reports from
people who were confused by importing an unexpected version of a package, and
not remembering (or knowing) that some third party version outside of the
archive had been installed, and /usr/local/lib does come before /usr/lib in
sys.path.  This lead to https://bugs.python.org/issue29546 which will be in
3.7 and should at least help in debugging such situations.

Cheers,
-Barry


pgpRQPCy_NxVn.pgp
Description: OpenPGP digital signature
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Nick Coghlan 
On 27 April 2017 at 23:04, Daniel P. Berrange  wrote:
> On Thu, Apr 27, 2017 at 04:32:09PM +1000, Nick Coghlan wrote:
>> Their approach means that any harm caused by "sudo pip install X" can
>> subsequently be fully reversed by doing "sudo pip uninstall X".
>>
>> At this moment, this is NOT true for Fedora and derivatives. Instead,
>> the remediation step here is "sudo pip uninstall X && sudo dnf
>> reinstall " where you have to:
>>
>> 1. Figure out what "" needs to be
>> 2. Hope that whatever you broke didn't affect your ability to run
>> "sudo dnf reinstall"
>
> Yep, recovering the system python install to a pristine state after
> 'devstack' has done its stuff is very painful right now :-(
>
> devstack was originally written for Ubuntu systems, so I guess they
> don't suffer as much due to the Debian specific change you describe
> above.
>
> Which location takes priority in Debian if the same module is installed
> in both places ? The DPKG location, or the Pip location ? Presumably it
> would have to be the pip version that takes priority if you want it to
> be usable for updating the (likely older) distro provided versions.

Aye, /usr/local/lib has priority:

$ docker run --rm ncoghlan/debian-python python3 -m site
sys.path = [
'/',
'/usr/lib/python3.4',
'/usr/lib/python3.4/plat-x86_64-linux-gnu',
'/usr/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/dist-packages',
'/usr/lib/python3/dist-packages',
]
USER_BASE: '/root/.local' (doesn't exist)
USER_SITE: '/root/.local/lib/python3.4/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

User level package installs take priority over system level ones
(regardless of distro), so the current container build use case for
root level pip installations can typically be met equally well by
running as a non-root user inside the container and doing "pip install
--user ...". Unfortunately, as with the venv option, there are a *lot*
of container build scripts out there that don't do that :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Donald Stufft 

> On Apr 27, 2017, at 2:32 AM, Nick Coghlan  wrote:
> 
> Debian and derivatives already mitigate the potential harm for these
> cases by requiring the "--install-layout=deb" option to be passed to
> get distutils to install into the system directories rather than doing
> it by default: https://wiki.debian.org/Python#Deviations_from_upstream 
> 
> 
> Their approach means that any harm caused by "sudo pip install X" can
> subsequently be fully reversed by doing "sudo pip uninstall X".


For whatever it’s worth, at some point in time I plan on attempting to 
formalize the Debian solution (or something akin to it) within a PEP and try to 
get baked in support for it in Python.

I also feel required to point out that ``sudo pip uninstall`` isn’t always 
enough, because while the two systems install to different locations, pip 
currently will uninstall the files in /usr when installing to /usr/local. 
They’ve patched their copy of pip to refuse to uninstall files from /usr, but 
upstream pip will still do it. Again, something I hope to get sorted at some 
point (and of course, an upgrade totally restores the files in /usr).

—
Donald Stufft



___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Nick Coghlan 
On 26 April 2017 at 19:19, Michal Cyprian  wrote:
> The other possibility is to limit the pip install location change
> to distutils and pip [2]. This is the "safer" option, but does
> not cover all corner cases. For example, Python software built
> locally using cmake or similar tools will be installed into
> /usr/lib and can conflict with system tools. Debian chose to
> implement similar solution.
>
> I would like to ask which solution would work for your applications
> better, and what is in your opinion the right way to go.

From my perspective, the main goal of the change is to make it
possible to fully recover from "sudo pip install " by doing
"sudo pip uninstall ", and the approach Debian took is
sufficient to achieve that in almost all cases. So +1 from me for
replicating something similar to Debian's approach in Fedora rather
than blazing new trails by fiddling with `sys.prefix` directly.

For the latter concern with CMake/autotools/Scons/etc, anyone doing
local builds of C/C++ applications already needs to be careful not to
be interfere with system packages, so I'm comfortable leaving that in
the category where running C/C++ build system install commands outside
a package build process without due care and attention is a generally
bad idea (e.g. a "sudo make altinstall" of the Python 3.5 maintenance
branch will clobber the system Python binaries in F25, and a "sudo
make install" from the main Python development branch will mess with
the default symlinks if you didn't set a suitable `/opt/` prefix).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org

Re: What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-27 Thread Nick Coghlan 
On 27 April 2017 at 11:47, Nico Kadel-Garcia  wrote:
> On Wed, Apr 26, 2017 at 7:13 AM, Charalampos Stratakis
>> At the present time, running sudo pip3 in Fedora is not safe.
>> Pip shares its installation directory with dnf, can remove
>> dnf-managed files and generally break the Python 3 interpreter.
>
> This is true of every version of pip, not merely pip3. It was also
> true of CPAN, and of many gradle, maven, and ant working environments
> that semi-randomly collate the very latest versions of indeterminate
> components and spray them on top of your system workspace with their
> own quite distinct ideas about packaging and versioning.
>
> There is no solution to this problem, except to use tools like
> "pyvenv" to set aside secluded workspaces for Python modules and their
> dependencies assembly. So, for most developers, they *should not* use
> "sudo pip". They should remain in user space, or possibly in shared
> workspaces, to avoid overwriting system components.

Nothing is changing in terms of *recommended* practices. This change
proposal is entirely about harm mitigation for the cases where users
*do* run "sudo pip ...", either because that's their instinctive
reaction to a permissions error, or because some misguided
installation instructions for a 3rd party package advised them to do
it.

Debian and derivatives already mitigate the potential harm for these
cases by requiring the "--install-layout=deb" option to be passed to
get distutils to install into the system directories rather than doing
it by default: https://wiki.debian.org/Python#Deviations_from_upstream

Their approach means that any harm caused by "sudo pip install X" can
subsequently be fully reversed by doing "sudo pip uninstall X".

At this moment, this is NOT true for Fedora and derivatives. Instead,
the remediation step here is "sudo pip uninstall X && sudo dnf
reinstall " where you have to:

1. Figure out what "" needs to be
2. Hope that whatever you broke didn't affect your ability to run
"sudo dnf reinstall"

>> Our first attempt to make sudo pip safe on Fedora [0] was
>> based on using a different binary (/usr/libexec/system-python)
>> for RPM packaging purposes and changing the behavior
>> of /usr/bin/python3 (and pip that uses its settings) to install
>> under /usr/local by default. Switching all the Python 3 packages
>
> Oh, Oh my. The extent to which this unworkable is... fascinating.

Nico, the contemptuous tone you have adopted in this post is entirely
inappropriate. Please give other list participants the courtesy of
assuming expertise at least equivalent to your own, rather than
engaging in attempted displays of assumed superiority.

> You
> will inevitably install incompatible future versions of critical
> packages which are incompatible with the Fedora system packages. And
> as soon as you install one that breaks RPM, if /usr/local/ if ahead of
> /usr in your search path for modules, you are *screwed*.

This isn't true, as long as the fix is as simple as using pip to
uninstall whatever problematic package you just installed.

At the moment, that isn't the case on Fedora and derivatives, since
"sudo pip install ..." can actually *corrupt* a system package
installation, rather than merely shadowing it at import time the way
it does on Debian et al.

> And if
> /usr/local is first, well, an RPM update may introduce a component
> that is incompatible with the pip installed modules at arbitrary
> times.

The goal of this change isn't to allow co-existence of arbitrary
upstream and downstream versions as supported combinations - it's to
make sure that forced installation of an incompatible upstream version
is an easily reversible error, rather than a potentially
system-breaking mistake.

>> We decided to try a different approach. The main idea is to detect
>>  an ongoing RPM build and modify the behavior of the Python 3
>> executable only when in normal user environment so that RPM builds
>> won't be affected at all.
>
> See https://xkcd.com/1172/ . You're inventing a workflow that isn't
> supported by *anyone*.

The problem is that the default behaviour needs to change (to mitigate
the potential harm caused by a relatively common end user error), but
unlike Debian package builds (which know they need to set
"--install-layout=deb"), most RPM builds for Python packages currently
rely on that default behaviour as part of the build process. Changing
the default indiscriminately would thus be a breaking change that not
only affects Fedora RPM builds, but also COPR and other third party
RPM builds.

So we need a solution that retains the existing behaviour for RPM
builds, but changes the behaviour to be less potentially destructive
when run interactively at a shell prompt, or as part of a
configuration management script.

>> The adjustment of the behavior can be done on different levels.
>> The first option is to set the sys.prefix variable to /usr/local
>> when the interpreter is initialized. This will

What is your opinion on "sudo pip" fix for Fedora 27?

2017-04-26 Thread Michal Cyprian 
At the present time, running sudo pip3 in Fedora is not safe.
Pip shares its installation directory with dnf, can remove 
dnf-managed files and generally break the Python 3 interpreter.

Our first attempt to make sudo pip safe on Fedora [0] was 
based on using a different binary (/usr/libexec/system-python) 
for RPM packaging purposes and changing the behavior 
of /usr/bin/python3 (and pip that uses its settings) to install 
under /usr/local by default. Switching all the Python 3 packages
to system-python turned out to be much more problematic than 
expected and we had to postpone the Change for F27.

We decided to try a different approach. The main idea is to detect
 an ongoing RPM build and modify the behavior of the Python 3
executable only when in normal user environment so that RPM builds 
won't be affected at all.

The adjustment of the behavior can be done on different levels.
The first option is to set the sys.prefix variable to /usr/local 
when the interpreter is initialized. This will affect
all the install methods, but the solution can cause some
problems in applications that rely on the value of sys.prefix.
A prototype of this implementation can be seen here [1].

The other possibility is to limit the pip install location change 
to distutils and pip [2]. This is the "safer" option, but does 
not cover all corner cases. For example, Python software built 
locally using cmake or similar tools will be installed into 
/usr/lib and can conflict with system tools. Debian chose to 
implement similar solution.

I would like to ask which solution would work for your applications 
better, and what is in your opinion the right way to go. I will
appreciate any costructive comments or ideas on how to improve these 
patches. If you want to check some specific use-case you can use the images
docker.io/mcyprian/sudo_pip_sys_prefix and 
docker.io/mcyprian/sudo_pip_install_prefix
to try out the two mentioned implementations.
 
[0] 
https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe?rd=User:Torsava/Making_sudo_pip_safe
[1] 
https://github.com/mcyprian/python3/commit/6077c96460c03ddc334bae654fffcce44eb6873d
[2] 
https://github.com/mcyprian/python3/commit/ac2f266a90871d67ba8d0b70d2b0ebe47732f064

Michal Cyprian
___
python-devel mailing list -- python-devel@lists.fedoraproject.org
To unsubscribe send an email to python-devel-le...@lists.fedoraproject.org