Re: [Distutils] Unexpected VersionConflict
That does indeed fix this problem, but requiring an egg writer to
interrogate all dependent packages (and their dependent packagesŠ) and
then hoist the dependencies up won't be robust if those dependent packages
change their requirements between the time the egg is written and the time
it's loaded.
It seems to me that if a requirement has no version specified, then it
shouldn't have a way to cause a VersionConflict. One possible way of
implementing this would be to have resolve() only check that a
distribution exists if no version is specified, do not update 'best'.
'to_activate' would need to be updated with 'generic' distributions only
if a requirement with a version specifier hadn't been seen.
On 8/8/13 7:44 PM, PJ Eby p...@telecommunity.com wrote:
On Thu, Aug 8, 2013 at 3:19 PM, Townsend, Scott E. (GRC-RTM0)[Vantage
Partners, LLC] scott.e.towns...@nasa.gov wrote:
During easy_install of an egg where two versions of pyparsing were
available
(1.5.2 and 1.5.6), a VersionConflict was raised:
pkg_resources.VersionConflict: (pyparsing 1.5.6
(/usr/lib/python2.7/dist-packages),
Requirement.parse('pyparsing==1.5.2'))
This was unexpected since sys.path (via virtualenv) has version 1.5.2
before
1.5.6. And the system gets 1.5.2 from 'import pyparsing', not 1.5.6.
Have you tried declaring the 1.5.2 dependency from your main project?
IIRC, that should make it take precedence over either of the indirect
dependencies.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Unexpected VersionConflict
On Fri, Aug 9, 2013 at 5:41 PM, Nick Coghlan ncogh...@gmail.com wrote: On 10 August 2013 04:06, PJ Eby p...@telecommunity.com wrote: Probably a better way would be to change the version resolution algorithm to be less greedy, and simply rule out unacceptable versions as the process goes along, then picking the most recent versions left when everything necessary has been eliminated. (Ideally, such an algorithm would still track which distributions had the conflicting requirements, though.) The part I find most surprising is the fact that pkg_resources ignores sys.path order entirely when choosing between multiple acceptable versions. Technically, it doesn't ignore it: if a distribution is listed in sys.path, it takes precedence over any distribution listed later, or that has to be found *in* a directory on sys.path, and will in fact cause a VersionConflict if you ask for a version spec that it doesn't match. However, where the distributions aren't listed in sys.path, but merely *found in a directory on sys.path*, then sys.path has no bearing. It would make things a lot more complicated, and not just in an implementation is hard to explain kind of way. (In principle, you could write an Environment subclass that had a different precedence, but I'm not sure what benefit you would gain from the added complexity. The core version resolution algorithm wouldn't be affected, though, since it delegates the find me something I haven't already got on sys.path operation to an Environment instance's best_match() method.) ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Unexpected VersionConflict
On Fri, Aug 9, 2013 at 9:04 AM, Townsend, Scott E. (GRC-RTM0)[Vantage Partners, LLC] scott.e.towns...@nasa.gov wrote: That does indeed fix this problem, but requiring an egg writer to interrogate all dependent packages (and their dependent packagesŠ) and then hoist the dependencies up won't be robust if those dependent packages change their requirements between the time the egg is written and the time it's loaded. That's why it's generally left up to the application installer/integrator to address these sorts of conflicts, and why it's usually a bad idea for anybody to be requiring exact versions. (I'd suggest asking your dependency to not specify exact point releases, too.) There is one other possibility, though: have you tried reversing the list of your project's dependencies so that the more-specific project's dependencies are processed first? (i.e., so that 1.5.2 will be selected as best before the non-version-specific one is used) That might fix it without requiring you to pin a version yourself. It seems to me that if a requirement has no version specified, then it shouldn't have a way to cause a VersionConflict. One possible way of implementing this would be to have resolve() only check that a distribution exists if no version is specified, do not update 'best'. 'to_activate' would need to be updated with 'generic' distributions only if a requirement with a version specifier hadn't been seen. Thing is, the complete lack of a version requirement is pretty rare, AFAIK, and so is the exact version match that's causing your problem. The combination existing on the same library is therefore that much rarer, so such a change would just be something of a complex kludge that wouldn't improve any other use cases. Probably a better way would be to change the version resolution algorithm to be less greedy, and simply rule out unacceptable versions as the process goes along, then picking the most recent versions left when everything necessary has been eliminated. (Ideally, such an algorithm would still track which distributions had the conflicting requirements, though.) That would be a pretty significant change, but potentially worth someone investigating. There are some big downsides, however: * It's not really a suitable algorithm for installation tools that don't have access to a universal dependency graph, because they can't tell what the next level of dependencies will be * Recursion causes a combinatorial explosion, because what if you select a different version and it has different dependencies (recursively)? Now you need backtracking, and there's a possibility that the algorithm will take a ridiculous amount of time to still conclude that there's nothing you can do about the conflict. These drawbacks are basically why I just wrote a simple greedy match algorithm in the first place, figuring it could always be improved later if it turned out to be needed in practice. There have been occasional comments over the last decade or so by people with ideas for better algorithms, but no actual code yet, as far as I know. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Unexpected VersionConflict
On 10 August 2013 04:06, PJ Eby p...@telecommunity.com wrote: Probably a better way would be to change the version resolution algorithm to be less greedy, and simply rule out unacceptable versions as the process goes along, then picking the most recent versions left when everything necessary has been eliminated. (Ideally, such an algorithm would still track which distributions had the conflicting requirements, though.) The part I find most surprising is the fact that pkg_resources ignores sys.path order entirely when choosing between multiple acceptable versions. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] Unexpected VersionConflict
During easy_install of an egg where two versions of pyparsing were available
(1.5.2 and 1.5.6), a VersionConflict was raised:
pkg_resources.VersionConflict: (pyparsing 1.5.6
(/usr/lib/python2.7/dist-packages), Requirement.parse('pyparsing==1.5.2'))
This was unexpected since sys.path (via virtualenv) has version 1.5.2 before
1.5.6. And the system gets 1.5.2 from 'import pyparsing', not 1.5.6.
I've traced this to the line calling _sort_dists(dists), line 801 in my copy of
pkg_resources.py:
def __getitem__(self,project_name):
Return a newest-to-oldest list of distributions for `project_name`
try:
return self._cache[project_name]
except KeyError:
project_name = project_name.lower()
if project_name not in self._distmap:
return []
if project_name not in self._cache:
dists = self._cache[project_name] = self._distmap[project_name]
_sort_dists(dists)
return self._cache[project_name]
The problem is that one dependent package of the egg has a requirement of
'pyparsing' while a subsequent dependent package has a requirement of
'pyparsing==1.5.2'. The intent was that by using virtualenv with a correct
sys.path version 1.5.2 would be used for both requirements. Unfortunately,
because of the call to _sort_dists(), the 'pyparsing' requirement is resolved
to 1.5.6 by env.best_match() in WorkingSet.resolve(). Once that resolution was
made, the more explicit requirement fails.
Note that without the _sort_dists() call the egg loads and runs correctly,
using pyparsing 1.5.2.
Its not clear to me that removing the _sort_dists() call is correct in general,
but it appears to be a bug that an egg which would load and run correctly
reports a VersionConflict.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Unexpected VersionConflict
On Thu, Aug 8, 2013 at 3:19 PM, Townsend, Scott E. (GRC-RTM0)[Vantage
Partners, LLC] scott.e.towns...@nasa.gov wrote:
During easy_install of an egg where two versions of pyparsing were available
(1.5.2 and 1.5.6), a VersionConflict was raised:
pkg_resources.VersionConflict: (pyparsing 1.5.6
(/usr/lib/python2.7/dist-packages), Requirement.parse('pyparsing==1.5.2'))
This was unexpected since sys.path (via virtualenv) has version 1.5.2 before
1.5.6. And the system gets 1.5.2 from 'import pyparsing', not 1.5.6.
Have you tried declaring the 1.5.2 dependency from your main project?
IIRC, that should make it take precedence over either of the indirect
dependencies.
___
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
