Re: [Distutils] PEP 440 and direct references
On 27 June 2013 13:12, PJ Eby wrote: > On Wed, Jun 26, 2013 at 1:40 PM, Vinay Sajip wrote: >> (a) Repeatability (not possible with a generic "latest version" URL). > > ISTM that forcing repeatability in the spec implies the inability to > do development with requirements that are also in development, unless > there is also an out-of-band channel for communicating the URL. Yeah, I think you might be right here. Specific tools that want to ensure repeatability are free to impose restrictions like that (just as they're free to rule out the use of direct references entirely), but we don't want to constraint them too much at the specification level or they lose their effectiveness as an escape clause. The description already says "The exact URLs and targets supported will be tool dependent." I think it's OK for distlib to require that by the time any associated info is handed over as a requirements specification, it has been converted to a specific version number. It isn't distlib's problem whether that was extracted from the URL directly, or by downloading it and looking at (or possibly even generating) the metadata. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
On Wed, Jun 26, 2013 at 1:40 PM, Vinay Sajip wrote: > (a) Repeatability (not possible with a generic "latest version" URL). ISTM that forcing repeatability in the spec implies the inability to do development with requirements that are also in development, unless there is also an out-of-band channel for communicating the URL. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
Nick Coghlan gmail.com> writes: > Hmm, you're right. So maybe it makes sense to declare that rather than be > a particular kind of clause, a direct reference is instead an alternative > to the entire version specifier system. That way it wouldn't allow > composition with other clauses at all. Another way to look at this is: the purpose of XXX in a requirement 'dist (XXX)' is to identify a specific version of "dist" to operate on. This can happen in two ways: * XXX is a direct reference - it identifies the version via URL, and no matching of any kind is required - that's the version required, plain and simple. If the resource is obtainable, then it's used, else it leads to an exception when an attempt to actually use it is made. In this case, the reason for constraining the form of the URL to a specific versioned source_url or wheel URL is twofold: (a) Repeatability (not possible with a generic "latest version" URL). (b) The ability to get the version represented by the URL, without doing any downloading, e.g. to see if a compatible version is already installed, or perhaps to see if it leads to any conflicts. * XXX is a set of version constraints which is used when querying indexes to find the most recent version which matches the constraints. According to this view, distlib does not need to provide specific support for direct references (beyond what is already provided for secure downloads, wheels etc.) - a direct reference is treated exactly the same as a single clause "(== )" for dependency resolution purposes. Of course, distlib should extend the existing version matching support to cover the additional cases in PEP 440 (such as explicit prefix matching and compatible version matching). Does that seem right? Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
On 26 Jun 2013 19:36, "Vinay Sajip" wrote: > > Nick Coghlan gmail.com> writes: > > > > Hmm, you're right. So maybe it makes sense to declare that rather than be > > a particular kind of clause, a direct reference is instead an alternative > > to the entire version specifier system. That way it wouldn't allow > > composition with other clauses at all. > > That seems fitting. Can I also suggest that "from" is better than "is", > though of course it's a little longer to type? I think it reads better when > it says "dist (from URL)". Oh, I like it. Nice :) Cheers, Nick. > > > I'll update the PEP accordingly (not sure exactly when, though). > > Okay, I'll proceed with distlib along the lines we've discussed here. > > Regards, > > Vinay Sajip > > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
Nick Coghlan gmail.com> writes: > Hmm, you're right. So maybe it makes sense to declare that rather than be > a particular kind of clause, a direct reference is instead an alternative > to the entire version specifier system. That way it wouldn't allow > composition with other clauses at all. That seems fitting. Can I also suggest that "from" is better than "is", though of course it's a little longer to type? I think it reads better when it says "dist (from URL)". > I'll update the PEP accordingly (not sure exactly when, though). Okay, I'll proceed with distlib along the lines we've discussed here. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
On 26 Jun 2013 08:52, "Vinay Sajip" wrote: > > Nick Coghlan gmail.com> writes: > > > While the intended use case is to pin a specific version, you could also > > use a more general "latest" link and use other clauses to trigger an > > error when the version changes beyond what you expect. > > > > It's mostly because I don't think it's worth the hassle of banning such > > combinations, though. > > My question arose from thinking about how you would implement matching in > cases where multiple such clauses arise. The only thing that seems to make > sense is that URLs in "is" clauses must conform to the source URL format > specified in PEP 426, or refer to wheels - either one allows a version > number to be unambiguously derived from the URL itself. That qualification > on allowable formats for URLs is not made in the "Direct Reference" section, > but perhaps it should be. However, your comment above about generic "latest" > links has confused me. If you can't figure out the version from the URL itself, > surely making matching decisions would require downloading the URL resource > and inspecting its contents? > > > I'm guided by the fact that pip and similar tools all allow direct references > > in requirement files. This is aimed directly at providing a standard, command > > line friendly notation for such references, rather than requiring the > > information be conveyed out of band. > > Okay, but in those cases, aren't we dealing with a single clause for a > particular distribution (i.e. a single "is clause" with no others), usually > referring to a local "editable" project or a VCS checkout? If we have a > specification "is URLA, is URLB", then unless URLA and URLB are equivalent in > terms of the version they represent, both can't match, or is there some way > that I've missed where they can? Hmm, you're right. So maybe it makes sense to declare that rather than be a particular kind of clause, a direct reference is instead an alternative to the entire version specifier system. That way it wouldn't allow composition with other clauses at all. More clearly constraining the URL in a direct reference to be a valid source URL, *or* have a valid wheel name as the final path component also makes a lot of sense. I'll update the PEP accordingly (not sure exactly when, though). Cheers, Nick. > > Most of PEP 426/440 are about the shiny, happy packaging utopia we're trying > > to create. This feature is aimed squarely at the messy world of the system > > integrator. > > I get that, but I'm assuming that there should be support in distlib for this > matching logic, and thinking about how to implement it is where I'm coming > from. > > > It is already - none of the other clauses allow the trailing wildcard as > > legal syntax. > > Okay, I wanted to make sure that was the intention. > > > Prefix matching is intended for cases where the prefix is shorter than > > constraints in other clauses. For example, "~= 1.3" means ">= 1.3, == 1.*", > > which is not the same thing as "== 1.3.*". > > OK, I see. Thanks for the answers. > > Regards, > > Vinay Sajip > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
Nick Coghlan gmail.com> writes: > While the intended use case is to pin a specific version, you could also > use a more general "latest" link and use other clauses to trigger an > error when the version changes beyond what you expect. > > It's mostly because I don't think it's worth the hassle of banning such > combinations, though. My question arose from thinking about how you would implement matching in cases where multiple such clauses arise. The only thing that seems to make sense is that URLs in "is" clauses must conform to the source URL format specified in PEP 426, or refer to wheels - either one allows a version number to be unambiguously derived from the URL itself. That qualification on allowable formats for URLs is not made in the "Direct Reference" section, but perhaps it should be. However, your comment above about generic "latest" links has confused me. If you can't figure out the version from the URL itself, surely making matching decisions would require downloading the URL resource and inspecting its contents? > I'm guided by the fact that pip and similar tools all allow direct references > in requirement files. This is aimed directly at providing a standard, command > line friendly notation for such references, rather than requiring the > information be conveyed out of band. Okay, but in those cases, aren't we dealing with a single clause for a particular distribution (i.e. a single "is clause" with no others), usually referring to a local "editable" project or a VCS checkout? If we have a specification "is URLA, is URLB", then unless URLA and URLB are equivalent in terms of the version they represent, both can't match, or is there some way that I've missed where they can? > Most of PEP 426/440 are about the shiny, happy packaging utopia we're trying > to create. This feature is aimed squarely at the messy world of the system > integrator. I get that, but I'm assuming that there should be support in distlib for this matching logic, and thinking about how to implement it is where I'm coming from. > It is already - none of the other clauses allow the trailing wildcard as > legal syntax. Okay, I wanted to make sure that was the intention. > Prefix matching is intended for cases where the prefix is shorter than > constraints in other clauses. For example, "~= 1.3" means ">= 1.3, == 1.*", > which is not the same thing as "== 1.3.*". OK, I see. Thanks for the answers. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] PEP 440 and direct references
On 26 Jun 2013 05:50, "Vinay Sajip" wrote: > > I would find it helpful to understand the motivation for direct references as > defined in PEP 440, and some clarifications of how they can be used. For > example: > > * Can they be used together with other clauses? As PEP 440 appears now, it > seems to indicate that they can, but what does it mean in practice to have a > clause which is so specific with other clauses which are less specific? While the intended use case is to pin a specific version, you could also use a more general "latest" link and use other clauses to trigger an error when the version changes beyond what you expect. It's mostly because I don't think it's worth the hassle of banning such combinations, though. > * Can we have information in the "Direct reference" section summarising where > such references can and can't appear? No, because that will be up to tool developers. > * What direct references provide which cannot be provided by the other version > clauses together with (separately) URIs or paths for where archives can be > found? The fact that the format of this clause is so different from all the > others suggests that it doesn't really belong with them. I'm guided by the fact that pip and similar tools all allow direct references in requirement files. This is aimed directly at providing a standard, command line friendly notation for such references, rather than requiring the information be conveyed out of band. Most of PEP 426/440 are about the shiny, happy packaging utopia we're trying to create. This feature is aimed squarely at the messy world of the system integrator. > > There are also questions on other areas: > > * It would appear that explicit prefix matching only makes sense for == and != > clauses (as it is implicit for e.g.<, > clauses). If so, can this be stated > explicitly? It is already - none of the other clauses allow the trailing wildcard as legal syntax. > * Since pre-/post-/dev- suffixes are ignored for the purposes of prefix > matching, and given that implicit prefix matching is used with ~= , < and >, > does this mean that explicit prefix matching is only useful for >=, <=? > (given that == with prefix matching appears equivalent to ~=) Prefix matching is intended for cases where the prefix is shorter than constraints in other clauses. For example, "~= 1.3" means ">= 1.3, == 1.*", which is not the same thing as "== 1.3.*". The only time a compatible version clause and prefix matching are equivalent is for the "~= X.0" and "== X.*" case. > > It would also be useful to document what kinds of matches would not be possible > without ~= and explicit prefix matching (trailing .*), assuming we had implicit > prefix matching for ==/!= clauses. That was the approach I took in earlier versions of the spec, but people objected loudly to "==" not giving them exactly the stated version. So I switched the prefix matching to being explicit instead. The compatible release clause notation is mostly about encouraging the use of appropriate API versioning semantics (although again, with plenty of alternatives to handle the fact the real world isn't that neat and tidy). Cheers, Nick. > > Regards, > > Vinay Sajip > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] PEP 440 and direct references
I would find it helpful to understand the motivation for direct references as defined in PEP 440, and some clarifications of how they can be used. For example: * Can they be used together with other clauses? As PEP 440 appears now, it seems to indicate that they can, but what does it mean in practice to have a clause which is so specific with other clauses which are less specific? * Can we have information in the "Direct reference" section summarising where such references can and can't appear? * What direct references provide which cannot be provided by the other version clauses together with (separately) URIs or paths for where archives can be found? The fact that the format of this clause is so different from all the others suggests that it doesn't really belong with them. There are also questions on other areas: * It would appear that explicit prefix matching only makes sense for == and != clauses (as it is implicit for e.g.<, > clauses). If so, can this be stated explicitly? * Since pre-/post-/dev- suffixes are ignored for the purposes of prefix matching, and given that implicit prefix matching is used with ~= , < and >, does this mean that explicit prefix matching is only useful for >=, <=? (given that == with prefix matching appears equivalent to ~=) It would also be useful to document what kinds of matches would not be possible without ~= and explicit prefix matching (trailing .*), assuming we had implicit prefix matching for ==/!= clauses. Regards, Vinay Sajip ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
