Bug#994045: python-pip-whl: pip 18.1 raises EnvironmentError on 404 from index, which leads to PIP_FIND_LINKS being ignored

2021-10-20 Thread Stefano Rivera
Hi MrMino (2021.09.10_08:45:58_-0700)
> Since python-virtualenv will try to install pkg_resources==0.0.0 as a
> separate package (which does not exist on any Python index, given that
> it's a part of setuptools), it will instruct pip to do something akin to
> "pip install pkg_resources". Pip will ask the index about pkg_resources,
> which will return a 404. Then, pip will move to resolving the dependency
> using PIP_FIND_LINKS.

Somewhat related is #994952. The patch I'm proposing for Buster will
avoid trying to install pkg_resources==0.0.0. But setuptools and pip
still need to be installable from your index or you'll still hit this
bug.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  +1 415 683 3272



Bug#994045: python-pip-whl: pip 18.1 raises EnvironmentError on 404 from index, which leads to PIP_FIND_LINKS being ignored

2021-09-10 Thread MrMino
Package: python-pip-whl
Version: 18.1-5
Severity: important

Dear Maintainers,

TLDR: Current version of python-pip-whl breaks python-virtualenv if a custom
index-url is used inside ~/.pip/pip.conf.

Debian version of virtualenv uses PIP_FIND_LINKS environment variable to
inject files from /usr/share/python-wheels into pip's dependency resolution.

Since python-virtualenv will try to install pkg_resources==0.0.0 as a
separate package (which does not exist on any Python index, given that
it's a part of setuptools), it will instruct pip to do something akin to
"pip install pkg_resources". Pip will ask the index about pkg_resources,
which will return a 404. Then, pip will move to resolving the dependency
using PIP_FIND_LINKS.

That is, only if the original PyPI index is used. If you use a custom
index via ~/.pip/pip.conf file, Pip will fail with the following
message:

Could not install packages due to an EnvironmentError: 404
Client Error: Not Found for url: 

... and exit with RC=1 without going through PIP_FIND_LINKS directory.

This means, that if one has a custom index set in his ~/.pip/pip.conf,
virtualenv will fail with the following log:

New python executable in /env/bin/python2
Also creating executable in /env/bin/python
Installing setuptools, pkg_resources, pip, wheel...
  Complete output from command /env/bin/python2 - setuptools 
pkg_resources pip wheel:
  Looking in indexes: 
Looking in links: /usr/lib/python3/dist-packages, 
/usr/share/python-wheels/
Collecting setuptools
  Downloading 
Collecting pkg_resources
Could not install packages due to an EnvironmentError: 404 Client 
Error: Not Found for url: 


...Installing setuptools, pkg_resources, pip, wheel...done.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 2379, in 

main()
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 724, in main
symlink=options.symlink)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 996, in 
create_environment
download=download,
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 926, in 
install_wheel
call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 817, in 
call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /env/bin/python2 - setuptools pkg_resources pip wheel 
failed with error code 1
Running virtualenv with interpreter /usr/bin/python2

None of this happens when using the upstream 18.1 version of pip, which
leads me to believe this bug is specific to python-pip-whl/python-pip.

During analysis, I created a simple Dockerfile for reproducing this
issue, but it requires setting up a custom index. Currently the behavior
has been observed when using Artifactory. We have at least one index
hosted internally via HTTP that does not lead to the aforementioned
buggy behavior, so the problem seems specific to HTTPS requests.

(_Replace  with a custom index_)

FROM debian:buster

RUN apt update
RUN apt install -y python2 virtualenv
RUN mkdir ~/.pip/
RUN echo "[global]" >> ~/.pip/pip.conf
RUN echo "index-url = " >> ~/.pip/pip.conf
CMD virtualenv env


Best Regards
Blazej Michalik


-- System Information:
Debian Release: 10.9
  APT prefers oldstable-updates
  APT policy: (500, 'oldstable-updates'), (500, 'oldstable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.11.0-27-generic (SMP w/8 CPU cores)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: unable to detect

Versions of packages python-pip-whl depends on:
ii  ca-certificates  20200601~deb10u2

python-pip-whl recommends no packages.

python-pip-whl suggests no packages.

-- no debconf information