Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
Donald, I've added this to the list at https://bitbucket.org/pypa/pypi-metadata-formats/issue/49/address-pep-440-v6-feedback (the comments from Marcus and Paul were already noted). one other thing Donald and I discussed was being clear what was compatible with pkg_resources. my understanding is that *only* local versions are incompatible, in that they don't sort correctly relative to public versions. I'd like to see mention of that somewhere. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On Aug 16, 2014, at 11:37 AM, Jason R. Coombs jar...@jaraco.com wrote: Thanks Donald for the extensive work on this. It all looks generally good. One thing that stuck out as slightly surprising – the use of ‘_’ as a separator. I imagine most people consider the underscore to be yet another alpha character, similar to [a-zA-Z]. If it is to be a separator, I suggest that the PEP give some examples. As is, I didn’t see any examples that showed ‘_’ as a separator. It’s not a preferred separator, it’s use is mostly because of various normalization schemes, particularly Wheel, which will convert a ``-`` to a `` _`` in the version. Right now pip considers them equal because it was causing some breakage with Wheels. This rationale should probably be given in the PEP though. The original bug report in pip for this was https://github.com/pypa/pip/issues/1150. Related, I would suggest a consistent scheme for local version tags. Why not have local version tags have the same syntax as primary version numbers? That is, allow the same character set, pre and post versions, etc, just separated by a +. At the very least, I would expect local versions to allow underscores. Originally local versions were just integers separated by dots. We wound up where we are so that downstream distributions can put more information inside of it. For instance Instead of Ubuntu taking a version like 1.0, and making it 1.0+1, they can do 1.0+ubuntu.1 or similar so that it’s obvious from the version number that this version is different than say Fedora, who might have also patched it and have 1.0+fedora.1. I’m not sure what the use case is for pre/post/etc versions in the local version. At the point you’re doing more than simple patching you’re probably making a full fledged fork which should have it’s own name and version numbers I think? These are all just nitpicks, though. A lot of work has gone into this spec, and it’s good to see the community coalescing on an implementable standard. It’s surely acceptable in the current form. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 17 August 2014 02:11, Donald Stufft don...@stufft.io wrote: On Aug 16, 2014, at 11:37 AM, Jason R. Coombs jar...@jaraco.com wrote: Thanks Donald for the extensive work on this. It all looks generally good. One thing that stuck out as slightly surprising – the use of ‘_’ as a separator. I imagine most people consider the underscore to be yet another alpha character, similar to [a-zA-Z]. If it is to be a separator, I suggest that the PEP give some examples. As is, I didn’t see any examples that showed ‘_’ as a separator. It’s not a preferred separator, it’s use is mostly because of various normalization schemes, particularly Wheel, which will convert a ``-`` to a `` _`` in the version. Right now pip considers them equal because it was causing some breakage with Wheels. This rationale should probably be given in the PEP though. The original bug report in pip for this was https://github.com/pypa/pip/issues/1150. Related, I would suggest a consistent scheme for local version tags. Why not have local version tags have the same syntax as primary version numbers? That is, allow the same character set, pre and post versions, etc, just separated by a +. At the very least, I would expect local versions to allow underscores. Originally local versions were just integers separated by dots. We wound up where we are so that downstream distributions can put more information inside of it. For instance Instead of Ubuntu taking a version like 1.0, and making it 1.0+1, they can do 1.0+ubuntu.1 or similar so that it’s obvious from the version number that this version is different than say Fedora, who might have also patched it and have 1.0+fedora.1. Right. In the Fedora (et al) case, the local version ordering scheme also matches the way yum orders the RPM release field, so it becomes possible to map the release field (which tracks patches and spec file changes) directly to the Python level local version. One big advantage of the more relaxed rules around the local version identifier component is that DVCS hashes are allowed, you just want to include a serial number (e.g. number of commits since the last rebase) first to ensure sensible ordering. I’m not sure what the use case is for pre/post/etc versions in the local version. At the point you’re doing more than simple patching you’re probably making a full fledged fork which should have it’s own name and version numbers I think? Agreed. For use of the local version field to be appropriate, we should be looking at full API compatibility with the public version identifier. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 17 August 2014 02:11, Donald Stufft don...@stufft.io wrote: It’s not a preferred separator, it’s use is mostly because of various normalization schemes, particularly Wheel, which will convert a ``-`` to a `` _`` in the version. Right now pip considers them equal because it was causing some breakage with Wheels. This rationale should probably be given in the PEP though. The original bug report in pip for this was https://github.com/pypa/pip/issues/1150. Donald, I've added this to the list at https://bitbucket.org/pypa/pypi-metadata-formats/issue/49/address-pep-440-v6-feedback (the comments from Marcus and Paul were already noted). If you could prepare a patch to clarify those, we can get it merged and published, and I'll accept the PEP. (Since they're just clarifications rather than real changes, I'm actually prepared to accept the PEP now, but I don't believe there's any pressing need for urgency on that front - may as well get the clarifications in first) Thanks for all your work in finally bringing this to fruition, as well as to all those who contributed to the long migration from the original CPAN style versioning to an approach more in line with the Zen of Python :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On Aug 11, 2014, at 11:11 AM, Marcus Smith qwc...@gmail.com wrote: Public index servers SHOULD NOT allow the use of local version identifiers for uploaded distributions. I'm thinking this should just say PyPI and not Public broadly. The point is for local versions not to confused with the one authoritative upstream version sequence, not that it couldn't be public. I can imagine locally versioned distributions needing to be distributed or available publicly (e.g. for a specific platform or system). Considering the recommendation to use the python.integrator extension, which is generally about downstream modification and redistribution, it seems inconsistent to say that this redistribution couldn't be public. We actually have a definition for Public Index Server, It’s Public index servers are index servers which allow distribution uploads from untrusted third parties. The Python Package Index [3] is a public index server.”. This is defined in PEP 426. But thinking about it, that doesn't particularly match it either, because something like binstar allows this, but should allow local versions. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 12 Aug 2014 01:23, Donald Stufft don...@stufft.io wrote: On Aug 11, 2014, at 11:11 AM, Marcus Smith qwc...@gmail.com wrote: Public index servers SHOULD NOT allow the use of local version identifiers for uploaded distributions. I'm thinking this should just say PyPI and not Public broadly. The point is for local versions not to confused with the one authoritative upstream version sequence, not that it couldn't be public. I can imagine locally versioned distributions needing to be distributed or available publicly (e.g. for a specific platform or system). Considering the recommendation to use the python.integrator extension, which is generally about downstream modification and redistribution, it seems inconsistent to say that this redistribution couldn't be public. We actually have a definition for Public Index Server, It’s Public index servers are index servers which allow distribution uploads from untrusted third parties. The Python Package Index [3] is a public index server.”. This is defined in PEP 426. But thinking about it, that doesn't particularly match it either, because something like binstar allows this, but should allow local versions. In this particular case, I think Marcus is right - the restriction is specific to PyPI as the keeper of the authoritative shared namespace, rather than applying to public index servers in general. Cheers, Nick. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 10 August 2014 01:03, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Paul Moore wrote: FWIW, it looks like file:myserver/share/WINDOWS/clock.avi is how you'd refer to \\myserver\share\WINDOWS\clock.avi. Where did you get that from? According to this it's wrong: http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx It should be file://myserver/share/WINDOWS/clock.avi from urllib.request import pathname2url pathname2url(r'\\myserver\share\WINDOWS\clock.avi') 'myserver/share/WINDOWS/clock.avi' Bug in urllib? Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 8 August 2014 22:53, Donald Stufft don...@stufft.io wrote: Direct references = Some automated tools may permit the use of a direct reference as an alternative to a normal version specifier. A direct reference consists of the specifier ``@`` and an explicit URL. Whether or not direct references are appropriate depends on the specific use case for the version specifier. Automated tools SHOULD at least issue warnings and MAY reject them entirely when direct references are used inappropriately. Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers. Depending on the use case, some appropriate targets for a direct URL reference may be a valid ``source_url`` entry (see PEP 426), an sdist, or a wheel binary archive. The exact URLs and targets supported will be tool dependent. For example, a local source archive may be referenced directly:: pip @ file:///localbuilds/pip-1.3.1.zip Alternatively, a prebuilt archive may also be referenced:: pip @ file:///localbuilds/pip-1.3.1-py33-none-any.whl One very minor question. Is URL format required here? I guess so. On Windows, file URLs are confusing and annoying to type - use of / rather than \ makes tab-completion useless, it's difficult to remember how many slashes go at the start and where the drive letter goes, and do UNC paths need 157 or 158 slashes at the start (:-)), and the documentation is inconsistent and hard to find. Allowing a pathname here would be convenient for users, but it's 100% not important enough to need a new version of the spec. A clarification (presumably, that a URL is required) with a pointer to a document that explains the relevant filename-URL translation algorithm, would be good, though. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On Aug 9, 2014, at 4:26 AM, Paul Moore p.f.mo...@gmail.com wrote: On 8 August 2014 22:53, Donald Stufft don...@stufft.io wrote: Direct references = Some automated tools may permit the use of a direct reference as an alternative to a normal version specifier. A direct reference consists of the specifier ``@`` and an explicit URL. Whether or not direct references are appropriate depends on the specific use case for the version specifier. Automated tools SHOULD at least issue warnings and MAY reject them entirely when direct references are used inappropriately. Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers. Depending on the use case, some appropriate targets for a direct URL reference may be a valid ``source_url`` entry (see PEP 426), an sdist, or a wheel binary archive. The exact URLs and targets supported will be tool dependent. For example, a local source archive may be referenced directly:: pip @ file:///localbuilds/pip-1.3.1.zip Alternatively, a prebuilt archive may also be referenced:: pip @ file:///localbuilds/pip-1.3.1-py33-none-any.whl One very minor question. Is URL format required here? I guess so. On Windows, file URLs are confusing and annoying to type - use of / rather than \ makes tab-completion useless, it's difficult to remember how many slashes go at the start and where the drive letter goes, and do UNC paths need 157 or 158 slashes at the start (:-)), and the documentation is inconsistent and hard to find. Allowing a pathname here would be convenient for users, but it's 100% not important enough to need a new version of the spec. A clarification (presumably, that a URL is required) with a pointer to a document that explains the relevant filename-URL translation algorithm, would be good, though. Paul To be clear, the direct reference is mostly for use in the install_requires. On the CLI pip can still just take a path to a file or whatever. This feature is intended to replace dependency_links in a way that people can use them for private packages but that they won't be allowed on PyPI or the like. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 9 August 2014 17:02, Donald Stufft don...@stufft.io wrote: To be clear, the direct reference is mostly for use in the install_requires. On the CLI pip can still just take a path to a file or whatever. This feature is intended to replace dependency_links in a way that people can use them for private packages but that they won't be allowed on PyPI or the like. Understood. I'd still like an explicit pointer to the canonical URL details tools should (or must?) use. See http://en.wikipedia.org/wiki/File_URI_scheme (specifically the Windows sections) for why I'm bothered - specifically the quote Here are *some* examples which *may* be accepted by some applications (emphasis mine). I'd be more than happy if the spec was just tools will use XXX from the Python standard library. But note (the first 4 are the supposedly acceptable URLs from that Wikipedia article): py Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)] on win32 Type help, copyright, credits or license for more information. from urllib.request import url2pathname url2pathname(file://localhost/c|/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|//localhost/c|/WINDOWS/clock.avi url2pathname(file:///c|/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|///c|/WINDOWS/clock.avi url2pathname(file://localhost/c:/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|//localhost/c|/WINDOWS/clock.avi url2pathname(file:///c:/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|///c|/WINDOWS/clock.avi url2pathname(file:WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file:///WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file://WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file:/WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' I have no idea where Python got that E drive from - my current drive is C:. At this point in time, I have no idea how to correctly write a file URL for the file C:\Windows\clock.avi on my PC. While it's very definitely a corner case in the PEP, and it's clearly not the PEP's responsibility to explain the file URL spec, I do think it's relevant to the usability of the PEP. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On Aug 9, 2014, at 1:41 PM, Paul Moore p.f.mo...@gmail.com wrote: On 9 August 2014 17:02, Donald Stufft don...@stufft.io wrote: To be clear, the direct reference is mostly for use in the install_requires. On the CLI pip can still just take a path to a file or whatever. This feature is intended to replace dependency_links in a way that people can use them for private packages but that they won't be allowed on PyPI or the like. Understood. I'd still like an explicit pointer to the canonical URL details tools should (or must?) use. See http://en.wikipedia.org/wiki/File_URI_scheme (specifically the Windows sections) for why I'm bothered - specifically the quote Here are *some* examples which *may* be accepted by some applications (emphasis mine). I'd be more than happy if the spec was just tools will use XXX from the Python standard library. But note (the first 4 are the supposedly acceptable URLs from that Wikipedia article): py Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)] on win32 Type help, copyright, credits or license for more information. from urllib.request import url2pathname url2pathname(file://localhost/c|/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|//localhost/c|/WINDOWS/clock.avi url2pathname(file:///c|/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|///c|/WINDOWS/clock.avi url2pathname(file://localhost/c:/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|//localhost/c|/WINDOWS/clock.avi url2pathname(file:///c:/WINDOWS/clock.avi) Traceback (most recent call last): File stdin, line 1, in module File C:\Apps\Python34\lib\nturl2path.py, line 26, in url2pathname raise OSError(error) OSError: Bad URL: file|///c|/WINDOWS/clock.avi url2pathname(file:WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file:///WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file://WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' url2pathname(file:/WINDOWS/clock.avi) 'E:\\WINDOWS\\clock.avi' I have no idea where Python got that E drive from - my current drive is C:. At this point in time, I have no idea how to correctly write a file URL for the file C:\Windows\clock.avi on my PC. While it's very definitely a corner case in the PEP, and it's clearly not the PEP's responsibility to explain the file URL spec, I do think it's relevant to the usability of the PEP. Paul Does: file:///c:/WINDOWS/clock.avi work? That’s pointed out in the wikipage you linked as to what IE has done since IE4 and seems to be the most reasonable candidate. It also uses file://remotehost/sharename/dir/file.txt for UNC paths apparently. I misunderstood the problem though, I thought you were worried about needing to type out the file:// and the slash order on the CLI, not referencing files at all. We can absolutely add more details to the PEP. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
On 9 August 2014 18:50, Donald Stufft don...@stufft.io wrote: Does: file:///c:/WINDOWS/clock.avi work? Never mind, it's my mistake. Reading the docs more closely, Convert the path component path from a percent-encoded URL to the local syntax for a path. This does not accept a complete URL. (Note that second sentence...) url2pathname(///c:/WINDOWS/clock.avi) 'C:\\WINDOWS\\clock.avi' Looks like the PEP would be covered by referring to pathname2url/url2pathname (with a note that you strip the file: prefix for idiots like me ;-)) and giving a couple of examples. FWIW, it looks like file:myserver/share/WINDOWS/clock.avi is how you'd refer to \\myserver\share\WINDOWS\clock.avi. See what I mean about the slashes? I can't rationalise the 4 slashes here resulting in two backslashes, when the 3 slashes above disappear entirely... Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Round 6 - PEP 440 - Version Identification and Dependency Specification Version
Paul Moore wrote: FWIW, it looks like file:myserver/share/WINDOWS/clock.avi is how you'd refer to \\myserver\share\WINDOWS\clock.avi. Where did you get that from? According to this it's wrong: http://blogs.msdn.com/b/ie/archive/2006/12/06/file-uris-in-windows.aspx It should be file://myserver/share/WINDOWS/clock.avi i.e. the hosthame part of the path goes into the hostname part of the URI, which makes sense. If the path uses a drive letter instead, there is no hostname, so that part of the URI is empty, resulting in three consecutive slashes: file:///c:/WINDOWS/clock.avi -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
