Re: [OpenStack-Infra] pip-10.0.0 will make tox library in openstack-infra/zuul-jobs won't work

2018-04-16 Thread Jeremy Stanley 
On 2018-04-16 20:52:22 +0800 (+0800), 张幸 wrote:
> I have noticed that pip release a new version days ago, and when I
> test on my own CI system trigered by job
> openstack-zuul-jobs-linters, there was an error, looks like pip
> version installed in nodepool image is higher the the code need:
> 
> File:
> https://github.com/openstack-infra/zuul-jobs/blob/master/roles/tox/library/tox_install_sibling_packages.py#L114
[...]

Yes, we missed that we were importing from pip's internals here,
even knowing that pip 10 would intentionally break anything that
did. There are a few possibilities:

1. We could temporarily get it from pip._internal.req if we want a
fast stop-gap while discussing more thorough solutions.

2. We could implement our own parse_requirements() based on a
supported API from pkg_resources (like PBR does) or the packaging
library.

3. We can revisit whether we need to perform fancy parsing the `pip
freeze` output at all.

Clark's https://review.openstack.org/561659 is attempting to do #3.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] pip-10.0.0 will make tox library in openstack-infra/zuul-jobs won't work

2018-04-16 Thread 张幸 
Hi all,

I have noticed that pip release a new version days ago, and when I test on my 
own CI system
trigered by job openstack-zuul-jobs-linters, there was an error, looks like pip 
version installed in nodepool image is higher the the code need:

File:
https://github.com/openstack-infra/zuul-jobs/blob/master/roles/tox/library/tox_install_sibling_packages.py#L114

Logs:
2018-04-16 11:23:07.578344 | TASK [tox : Run tox without tests]
2018-04-16 11:23:07.970229 | ubuntu-xenial | linters create: 
/home/zuul/src/git.xxx.com/xxx/xxx-zuul-jobs/.tox/linters
2018-04-16 11:23:17.073502 | ubuntu-xenial | linters installdeps: 
-r/home/zuul/src/git.xxx.com/xxx/xxx-zuul-jobs/test-requirements.txt
2018-04-16 11:24:57.020033 | ubuntu-xenial | linters installed: 
alabaster==0.7.10,ansible==2.3.3.0,ansible-lint==3.4.21,asn1crypto==0.24.0,Babel==2.5.3,bashate==0.5.1,bcrypt==3.1.4,certifi==2018.1.18,cffi==1.11.5,chardet==3.0.4,cryptography==2.2.2,docutils==0.14,flake8==2.5.5,hacking==0.12.0,idna==2.6,imagesize==1.0.0,Jinja2==2.10,MarkupSafe==1.0,mccabe==0.2.1,packaging==17.1,paramiko==2.4.1,pbr==4.0.2,pep8==1.5.7,pyasn1==0.4.2,pycparser==2.18,pycrypto==2.6.1,pyflakes==0.8.1,Pygments==2.2.0,PyNaCl==1.2.1,pyparsing==2.2.0,pytz==2018.4,PyYAML==3.12,requests==2.18.4,six==1.11.0,snowballstemmer==1.2.1,Sphinx==1.7.2,sphinxcontrib-websupport==1.0.1,urllib3==1.22,zuul-sphinx==0.2.2
2018-04-16 11:24:57.020616 | ubuntu-xenial | 
___ summary 
2018-04-16 11:24:57.020680 | ubuntu-xenial |   linters: skipped tests
2018-04-16 11:24:57.020716 | ubuntu-xenial |   congratulations :)
2018-04-16 11:24:57.240157 | ubuntu-xenial | ok: Runtime: 0:01:49.290302
2018-04-16 11:24:57.301782 | 
2018-04-16 11:24:57.302127 | TASK [tox : Install any sibling python packages]
2018-04-16 11:24:58.153887 | ubuntu-xenial | ERROR
2018-04-16 11:24:58.154949 | ubuntu-xenial | {
2018-04-16 11:24:58.155064 | ubuntu-xenial |   "failed": true,
2018-04-16 11:24:58.155138 | ubuntu-xenial |   "log": "Processing siblings for 
ustack-zuul-jobs from src/git.xxx.com/xxx/xxx-zuul-jobs\nSiblingxxx-zuul-jobs 
at src/git.xxx.com/xxx/xxx-zuul-jobs\nSibling zuul-jobs at 
src/github.com/openstack-infra/zuul-jobs\n'module' object has no attribute 
'req'\nTraceback (most recent call last):\n  File 
\"/tmp/ansible_o6uPfQ/ansible_module_tox_install_sibling_packages.py\", line 
185, in main\nfor package in get_installed_packages(tox_python):\n  File 
\"/tmp/ansible_o6uPfQ/ansible_module_tox_install_sibling_packages.py\", line 
114, in get_installed_packages\nreturn 
pip.req.req_file.parse_requirements(\nAttributeError: 'module' object has no 
attribute 'req'\n",
2018-04-16 11:24:58.155201 | ubuntu-xenial |   "msg": "'module' object has no 
attribute 'req'"
2018-04-16 11:24:58.155259 | ubuntu-xenial | }
2018-04-16 11:24:58.194821 | 
2018-04-16 11:24:58.195028 | PLAY RECAP
2018-04-16 11:24:58.195148 | ubuntu-xenial | ok: 4 changed: 4 unreachable: 0 
failed: 1


I have tested in pip-8.x, pip-9.x and pip-10.x simple by import the pip like 
this:

root@xxx:~# pip --version
pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)
root@xxx:~# python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> pip.req

>>>


Successfully installed pip-9.0.3
You are using pip version 9.0.3, however version 10.0.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@test:~# python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> pip.req

>>>


Successfully installed pip-10.0.0
root@test:~# python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> pip.req
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'req'
>>>


Maybe there are some code won't work on pip-10.x
Hope we can fix it asop.



Best,
Angeiv Zhang___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra