I wanted to try out local version identifiers for packages. Or
actually, I already used them, but with a wrong notation, and ran into
trouble, also after fixing the notation. So I did some testing and
made notes on what does or does not work. See conclusions at the
bottom.
Local version identifiers: general idea
---------------------------------------
PEP440 is here, specifically the local version identifiers:
http://legacy.python.org/dev/peps/pep-0440/#local-version-identifiers
Use case in short, for those who do not know what local version
identifiers are. Upstream community package largetrout has version
1.0. As integrator you notice a problem, fix it in the code, and
create a pull request. While waiting for a new official release you
already want to use the new code in production. If you release a
version 1.1 yourself, this will conflict with a later official
version. This is where local version identifiers come in. You create
largetrout version 1.0+local.1 that you put in a directory on a
webserver. You add the url of the package as find-link for
pip/zc.buildout and can now use your local version.
That is the general idea. How does it work in practice? Two things
need to work: creating a distribution, and installing the distribution
with pip or buildout.
Creating a distribution
-----------------------
I have created a very basic python project called 'myproject'. It
does nothing. I have released a few versions here:
http://pypi.zestsoftware.nl/public/packagingtest/
The various versions only differ in their version numbers and in the
way in which the distributions were created. All were created with
Python 2.7.8. on Mac OSX 10.9.5, with ``python setup.py sdist
--formats=zip``.
- setup.py version = 1.0
* Created with: setuptools 2.2
* Distribution file: myproject-1.0.zip
* PKG-INFO version: 1.0
* works everywhere
- setup.py version = 1.1+maurits.1
* Created with: setuptools 2.2
* Distribution file: myproject-1.1-maurits.1.zip
* PKG-INFO version: 1.1-maurits.1
* NOT installable with zc.buildout 2.3 and setuptools 8 (see below)
- setup.py version = 1.1+maurits.2
* Created with: setuptools 7.0
* Distribution file: myproject-1.1-maurits.2.zip
* PKG-INFO version: 1.1-maurits.2
* NOT installable with zc.buildout 2.3 and setuptools 8 (see below)
- setup.py version = 1.1+maurits.3
* Created with: setuptools 8.0.4
* Distribution file: myproject-1.1+maurits.3.zip
* PKG-INFO version: 1.1+maurits.3
* NOT installable with current pip (see below)
* hard to install with development pip (BUG, see below)
* NOT installable with zc.buildout 2.2 and setuptools<8
So: with any setuptools version you can specify a local version
identifier (``+something``) in the setup.py, but you need setuptools
8.0+ to correctly handle it. In earlier versions, the plus sign is
replaced by a dash. This may lead to problems while installing; we
will see that below.
Note: ``python2.7 setup.py --version`` always correctly returns the
version number as specified in setup.py.
Installing with pip
-------------------
Now let's see how installing goes. Again we are using Python 2.7.
Below, some warnings or errors are expected (possibly with a
workaround, marked with TIP), some may be surprising
(marked with DANGER), others may be bugs (marked with BUG).
I picked two combinations of pip and setuptools versions. Other
combinations may give other results.
current pip, setuptools 7.0
---------------------------
A basic virtualenv:
$ pip list
pip (1.5.6)
setuptools (7.0)
wsgiref (0.1.2)
$ pip install -U -f http://pypi.zestsoftware.nl/public/packagingtest/
myproject==1.0
...
$ python -c "import myproject" && echo "importing works"
importing works
We try ``pip install`` for the available versions.
- myproject==1.0
* importing works
- myproject=1.1-maurits.1
* importing works
- myproject==1.1-maurits.2
* importing works
- myproject==1.1-maurits.3
* pip says:
Could not find a version that satisfies the requirement
myproject==1.1-maurits.3 (from versions: 1.0, 1.1, 1.1-maurits.1,
1.1-maurits.2)
- myproject==1.1+maurits.3
* pip says:
Exception:
Traceback (most recent call last):
...
File ".../pip/_vendor/pkg_resources.py", line 2583, in scan_list
"Expected ',' or end-of-list in",line,"at",line[p:]
ValueError: ("Expected ',' or end-of-list in",
'myproject==1.1+maurits.3', 'at', '+maurits.3')
* DANGER: older pip versions cannot install packages with local
version identifiers.
- pip install
http://pypi.zestsoftware.nl/public/packagingtest/myproject-1.1+maurits.3.zip
* importing works
development pip, setuptools 8.0
-------------------------------
A basic virtualenv:
$ pip list
pip (6.0.dev1)
setuptools (8.0.4)
$ pip install -U --trusted-host pypi.zestsoftware.nl -f
http://pypi.zestsoftware.nl/public/packagingtest/ myproject==1.0
...
$ python -c "import myproject" && echo "importing works"
importing works
We try ``pip install`` for the available versions.
- myproject==1.0
* importing works
- myproject==1.1-maurits.1
* pip warns:
Requested myproject==1.1-maurits.1, but installing version
1.1+maurits.1
* importing works
- myproject==1.1-maurits.2
* pip warns:
Requested myproject==1.1-maurits.2, but installing version
1.1+maurits.2
* importing works
- myproject==1.1-maurits.3
* pip says:
Collecting myproject==1.1+maurits.3
Could not find a version that satisfies the requirement
myproject==1.1-maurits.3 (from versions: )
No distributions matching the version for myproject==1.1-maurits.3
- myproject==1.1+maurits.3
* pip says:
Collecting myproject==1.1+maurits.3
Could not find a version that satisfies the requirement
myproject==1.1+maurits.3 (from versions: )
No distributions matching the version for myproject==1.1+maurits.3
* BUG: this should work, right?
- pip install
http://pypi.zestsoftware.nl/public/packagingtest/myproject-1.1+maurits.3.zip
* importing works
Installing with buildout
------------------------
We try the same with buildout. I use this basic buildout config:
[buildout]
find-links = http://pypi.zestsoftware.nl/public/packagingtest/
eggs-directory = eggs
download-cache = downloads
parts = mypy test
show-picked-versions = true
[mypy]
recipe = zc.recipe.egg
eggs = myproject
interpreter = mypy
[test]
recipe = plone.recipe.command
command = grep myproject bin/mypy && bin/mypy -c "import myproject" &&
echo "importing works"
update-command = ${:command}
[versions]
myproject = 1.0
plone.recipe.command = 1.1
#setuptools = ...
#zc.buildout = ...
zc.recipe.egg = 2.0.1
This buildout creates a bin/mypy python interpreter with the myproject
package installed. And it greps bin/mypy for 'myproject' so I can
easily see whether this has the version I expect. And it checks to
see it importing the module works.
Using previous zc.buildout and setuptools
-----------------------------------------
zc.buildout = 2.2.5
setuptools = 7.0
- myproject = 1.0
* all is well
- myproject = 1.1-maurits.1
* all is well
- myproject = 1.1-maurits.2
* all is well
- myproject = 1.1-maurits.3
* all is well
- myproject = 1.1+maurits.3
* buildout fails:
Traceback (most recent call last):
...
File ".../setuptools-7.0-py2.7.egg/pkg_resources.py", line 2690,
in scan_list
raise ValueError(msg, line, "at", line[p:])
ValueError: ("Expected ',' or end-of-list in",
'myproject[]==1.1+maurits.3', 'at', '+maurits.3')
* TIP: you can use a dash instead of a plus in the version pin.
Using latest zc.buildout and setuptools
---------------------------------------
zc.buildout = 2.3.1
setuptools = 8.0.2
- myproject = 1.0
* all is well
- myproject = 1.1-maurits.1
* buildout fails:
Installing myproject 1.1-maurits.1
Caused installation of a distribution:
myproject 1.1+maurits.1
with a different version.
Got None.
While:
Updating mypy.
Error: There is a version conflict.
We already have: myproject 1.1+maurits.1
We require myproject==1.1-maurits.1
* when I do not specify a version in the buildout config, but the
latest on server is 1.1-maurits.1, buildout succeeds and importing
works. buildout reports this (note the plus-sign):
Versions had to be automatically picked.
The following part definition lists the versions picked:
[versions]
myproject = 1.1+maurits.1
* BUG? pip only warns, buildout may want to do the same. Maybe
somehow normalize versions before comparison.
- myproject = 1.1-maurits.2
* buildout fails:
Installing myproject 1.1-maurits.2
Caused installation of a distribution:
myproject 1.1+maurits.2
with a different version.
...
* So: same remarks as for 1.1-maurits.1 apply.
- myproject = 1.1-maurits.3
* buildout fails:
Error: Couldn't find a distribution for 'myproject==1.1-maurits.3'.
- myproject = 1.1+maurits.3
* importing works
Conclusions
-----------
- BUG in pip: development version of pip cannot find local version
identifiers (version numbers with a plus sign).
- TIP: do NOT use setuptools 7.0 or older to create a distribution
with a local version identifier. Installation of this may fail,
because the version according to the filename differs from the
actual version.
- BUG in buildout? Where pip only warns about non-matching versions
from the above tip, buildout actually quits with an error. Maybe
somehow normalize versions before comparison.
- TIP: only use local version identifiers when you use setuptools 8 or
higher both when creating and when using the distribution. Then
installing with zc.buildout will work. Installing with pip can be
done by pointing to the exact file url.
--
Maurits van Rees: http://maurits.vanrees.org/
Zest Software: http://zestsoftware.nl
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
