[Distutils] Metadataformat PEP 426 on PyPI?
Hello and sorry if I am on the wrong mailing list. I'm working on a dependency resolution resolver and wanted to implement support for the new Metadata format. (As of now i'm parsing the setup.py for dependency information and am dependent on a self build metadata database of all PyPI packages) Will there be build-in Metadata in the new PEP 426 format online on PyPI for all packages? If yes, are there already specifications on how they will be retreavable? Thanks, Alexander Schneider ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
On 3 July 2013 19:17, Alexander Schneider alexander.schnei...@uni-duesseldorf.de wrote: Hello and sorry if I am on the wrong mailing list. I'm working on a dependency resolution resolver and wanted to implement support for the new Metadata format. (As of now i'm parsing the setup.py for dependency information and am dependent on a self build metadata database of all PyPI packages) Will there be build-in Metadata in the new PEP 426 format online on PyPI for all packages? If yes, are there already specifications on how they will be retreavable? Yes there will, but actually figuring out the details of those APIs is some time away. Note that the version currently referenced from the PEP is a little out of date ( http://mail.python.org/pipermail/distutils-sig/2013-June/021357.html). I will hopefully get it updated at the PyCon AU sprints next week. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
Alexander Schneider Alexander.Schneider at uni-duesseldorf.de writes:
Will there be build-in Metadata in the new PEP 426 format online on PyPI
for all packages? If yes, are there already specifications on how they
will be retreavable?
I have experimental support for PEP 426 metadata available, which is
up-to-date with the spec apart from the changes Nick linked to. For any
given package, you can access some JSON at an URL based on the project name.
For example, setuptools 0.7.5 metadata is available at
http://www.red-dove.com/pypi/projects/S/setuptools/package-0.7.5.json
If you deserialize the JSON at an URL like the above into a dict, the PEP
426 metadata is available in the subdict at key index-metadata in the
top-level dict. Example from setuptools 0.7.5:
index-metadata: {
description: omitted for brevity,
license: PSF or ZPL,
contacts: [
{
role: author,
name: The fellowship of the packaging,
email: distutils-sig@python.org
}
],
summary: Easily download, build, install, upgrade, and uninstall
Python packages,
project_urls: {
Home: https://pypi.python.org/pypi/setuptools;
},
keywords: [
CPAN,
PyPI,
distutils,
eggs,
package,
management
],
metadata_version: 2.0,
extras: [
certs,
ssl
],
version: 0.7.5,
run_may_require: [
{
environment: sys_platform=='win32',
dependencies: [
wincertstore (== 0.1)
],
extra: ssl
},
{
environment: sys_platform=='win32' and python_version=='2.4',
dependencies: [
ctypes (== 1.0.2)
],
extra: ssl
},
{
dependencies: [
certifi (== 0.0.8)
],
extra: certs
},
{
environment: python_version in '2.4, 2.5',
dependencies: [
ssl (== 1.16)
],
extra: ssl
}
],
classifiers: [
Development Status :: 5 - Production/Stable,
Intended Audience :: Developers,
License :: OSI Approved :: Python Software Foundation License,
License :: OSI Approved :: Zope Public License,
Operating System :: OS Independent,
Programming Language :: Python :: 2.4,
Programming Language :: Python :: 2.5,
Programming Language :: Python :: 2.6,
Programming Language :: Python :: 2.7,
Programming Language :: Python :: 3,
Programming Language :: Python :: 3.1,
Programming Language :: Python :: 3.2,
Programming Language :: Python :: 3.3,
Topic :: Software Development :: Libraries :: Python Modules,
Topic :: System :: Archiving :: Packaging,
Topic :: System :: Systems Administration,
Topic :: Utilities
],
name: setuptools
},
I expect this metadata to track the PEP as changes to it are published.
Currently, the top-level dict contains some legacy representations of the
metadata which will be removed in due course.
I would hope that once the dust settles on the PEP, this metadata (the PEP
426 part) can be migrated to PyPI.
Regards,
Vinay Sajip
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
On Wed, Jul 3, 2013 at 10:51 AM, Vinay Sajip vinay_sa...@yahoo.co.uk wrote:
If you deserialize the JSON at an URL like the above into a dict, the PEP
426 metadata is available in the subdict at key index-metadata in the
top-level dict. Example from setuptools 0.7.5:
index-metadata: {
name: setuptools
},
I expect this metadata to track the PEP as changes to it are published.
Currently, the top-level dict contains some legacy representations of the
metadata which will be removed in due course.
Just an FYI, not sure if this is an issue with your converter or with
the new spec, but the metadata shown for setuptools is missing
something important: 0.7.x pins specific distributions of its
dependencies using dependency_links URLs with #md5 hashes, so that SSL
support can be installed in a reasonably secure manner, as long as
you're starting from a trusted copy of the distribution. The
converted metadata you show lacks this pinning.
Granted, the pinning is somewhat kludged, and the specific need is
perhaps a rare use case outside of installer tools themselves. But I
thought it worth pointing out as a limitation of either the converter
or with the spec itself in relation to version support.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
PJ Eby pje at telecommunity.com writes: Just an FYI, not sure if this is an issue with your converter or with the new spec, but the metadata shown for setuptools is missing something important: 0.7.x pins specific distributions of its dependencies using dependency_links URLs with #md5 hashes, so that SSL support can be installed in a reasonably secure manner, as long as you're starting from a trusted copy of the distribution. The converted metadata you show lacks this pinning. True, although I do capture the dependency links under the 'dependency-urls' key of the top level dict of the JSON I linked to. While dependency_links is not directly supported by PEP 426, the intent is there via direct references. When installing using distlib/distil, SSL host verification and hash verification are done, even when direct references are not specified, since the versions of dependencies are pinned. For example, if I install setuptools into a fresh venv: $ pyvenv-3.3 /tmp/venv $ distil -e /tmp/venv install setuptools [ssl,certs] Checking requirements for setuptools (0.7.7) ... done. The following new packages will be downloaded and installed: certifi (0.0.8) [for setuptools] setuptools (0.7.7) Proceed? (y/n) y Downloading certifi-0.0.8.tar.gz to /tmp/tmpccek0f [for setuptools] 115KB @ 667 KB/s 100 % Done: 00:00:00 Unpacking ... done. Downloading setuptools-0.7.7.tar.gz to /tmp/tmpchxc1x 736KB @ 393 KB/s 100 % Done: 00:00:01 Unpacking ... done. [installation feedback snipped] Below is an extract from distil.log for the above installation, showing the downloading and verification operations: Downloading certifi-0.0.8.tar.gz to /tmp/tmpccek0f [for setuptools] Digest specified: dc5f5e7f0b5fc08d27654b17daa6ecec Host verified: pypi.python.org Digest verified: dc5f5e7f0b5fc08d27654b17daa6ecec Library location: venv site-packages Downloading setuptools-0.7.7.tar.gz to /tmp/tmpchxc1x Digest specified: 0d7bc0e1a34b70a97e706ef74aa7f37f Host verified: pypi.python.org Digest verified: 0d7bc0e1a34b70a97e706ef74aa7f37f Library location: venv site-packages Distil includes the Mozilla certs and thus is able to do SSL host validation. The hash support is currently limited to MD5 because PyPI has not supported other formats, but I expect that will be rectified in due course. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
On Wed, Jul 3, 2013 at 2:34 PM, Donald Stufft don...@stufft.io wrote: PEP426 does not support dependency_links. Right - I would've expected direct references in this scenario, assuming the PEP still has them. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Metadataformat PEP 426 on PyPI?
On Jul 3, 2013, at 7:38 PM, PJ Eby p...@telecommunity.com wrote: On Wed, Jul 3, 2013 at 2:34 PM, Donald Stufft don...@stufft.io wrote: PEP426 does not support dependency_links. Right - I would've expected direct references in this scenario, assuming the PEP still has them. Yea PEP440 direct references can be used as an approximate dependency_links replacement with the caveat you can't publish them on PyPI. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
