Florian Festi wrote:
Hi Tim!
Having a first look over you patches the following comes to my mind:
Tim Lauridsen wrote:
I have created at patch to add skip-broken functionality to YumBase.
I works the following way:
1. I have added some code to the depsolver to store packages with
problems in the self.po_with_problems dict. So we have the po with
problems when the depsolving is completed.
* This should be a set. These key:1 dicts are ugly IMHO.
Agree, it should be a set (fixed)
2. I have added a extra 'skip-broken' parameter to
YumBase.buildTransaction to control if we want to skip packages with
problem from the transaction. (default = no)
May be this should be done by accessing a global config option.
Yes, this is the right way to do it. (fixed)
3. if the skip-broken parameter is True, then we do
While "There is problems"
Remove the packages from the transaction
Depsolve
* From my understanding of the architecture of the depsolver the
handling of po_with_problems should be done in the process* methods. But
that might be my personal taste.
Sounds sane to me, (fixed)
* Your patch fails to make sure that the problematic packages are not
readded by dependencies. If that happens you run into an endless loop.
The right solution would be an excluded_pkgs set that is ignored for all
depsolving operations. If pkgs are removed from the transaction they get
put into that set. This way we can never get back to the same situation
and can not run into an endless loop.
Added some code to clean out the added deps to a package with problems
* While your patch tries to keep the impact on the core code minimal
(which is a good idea for a first proof of concept implementation) I
think we need a deeper integration into the depsolver in the long run.
In fact there is no need of an outer loop around the depsolver. It would
be enough if the process* methods would just remove the problematic pkg
and adds them to the excluded_pkgs set.
Yes, Agree
* As your patch works on the packages that have problems and not on the
packages that cause the problems there might be interesting side effects.
Yes, this is the problem with trying to solve depsolve problem
foo-2.0 can't update foo-1.0, because bar require foo = 1.0.
So bar is causing the issue, foo-2.0, but i would not be sane to remove
bar. So we used remove foo-2.0 from the transaction, to get on with
other stuff.
* Like readding a package (by removing the erase txmbr) that has been
updated - resulting in the old and the new packages beeing installed
(welcome to the world of multilib file conflicts!)
I am not sure i understand this, please give some example.
* Although I added some code to handle removing transaction_members a
while ago this code still requires some testing to make sure everything
works fine.
* There's also still the Depsolver.cheaterlookup. You need to make sure
that nothing goes wrong with it (your patch should be ok, but some of
the stuff I sketched above might not)
Florian
Thanks for your comments.
I have added a new patch with some of the changes you suggested.
I have added a new unittest to test it, without needing some special
crafted rpms.
The patch also fixes some issues where a package is added to transaction
by the depsolver, but the requering package is not stored in the
TransactionMember.
Tim
diff --git a/test/skipbroken-tests.py b/test/skipbroken-tests.py
new file mode 100644
index 0000000..07e9f22
--- /dev/null
+++ b/test/skipbroken-tests.py
@@ -0,0 +1,78 @@
+import unittest
+from testbase import *
+
+class SkipBrokenTests(DepsolveTests):
+
+ def testMissingReqNoSkip(self):
+ ''' bar fails because foobar is not provided '''
+ po = FakePackage('foo', '1', '0', '0', 'noarch')
+ po.addRequires('bar', None, (None,None,None))
+ self.tsInfo.addInstall(po)
+
+ xpo = FakePackage('bar', '1', '0', '0', 'noarch')
+ xpo.addRequires('foobar', None, (None,None,None))
+ self.xsack.addPackage(xpo)
+ self.assertEquals('err', *self.resolveCode(skip=False))
+ self.assertResult((po,xpo))
+
+ def testMissingReqSkip(self):
+ ''' foo + bar is skipped, because foobar is not provided '''
+ po = FakePackage('foo', '1', '0', '0', 'noarch')
+ po.addRequires('bar', None, (None,None,None))
+ self.tsInfo.addInstall(po)
+
+ xpo = FakePackage('bar', '1', '0', '0', 'noarch')
+ xpo.addRequires('foobar', None, (None,None,None))
+ self.xsack.addPackage(xpo)
+ self.assertEquals('empty', *self.resolveCode(skip=True))
+ self.assertResult([])
+
+ def testDepWithMissingReqSkip(self):
+ ''' foo + foobar is skipped because barfoo is not provided
+ bar stays in the transaction
+ '''
+ po1 = FakePackage('foo', '1', '0', '0', 'noarch')
+ po1.addRequires('foobar', None, (None,None,None))
+ self.tsInfo.addInstall(po1)
+
+ po2 = FakePackage('bar', '1', '0', '0', 'noarch')
+ self.tsInfo.addInstall(po2)
+
+ xpo1 = FakePackage('foobar', '1', '0', '0', 'noarch')
+ xpo1.addRequires('barfoo', None, (None,None,None))
+ self.xsack.addPackage(xpo1)
+ self.assertEquals('ok', *self.resolveCode(skip=True))
+ self.assertResult([po2])
+
+ def resolveCode(self,skip = False):
+ solver = YumBase()
+ solver.conf = FakeConf()
+ solver.conf.skip_broken = skip
+ solver.tsInfo = solver._tsInfo = self.tsInfo
+ solver.rpmdb = self.rpmdb
+ solver.pkgSack = self.xsack
+
+ for po in self.rpmdb:
+ po.repoid = po.repo.id = "installed"
+ for po in self.xsack:
+ po.repoid = po.repo.id = "TestRepository"
+ for txmbr in solver.tsInfo:
+ if txmbr.ts_state in ('u', 'i'):
+ txmbr.po.repoid = txmbr.po.repo.id = "TestRepository"
+ else:
+ txmbr.po.repoid = txmbr.po.repo.id = "installed"
+
+ res, msg = solver.buildTransaction()
+ #self.tsInfo = copy.copy(solver._tsInfo)
+ return self.res[res], msg
+
+
+
+
+def suite():
+ suite = unittest.TestSuite()
+ suite.addTest(unittest.makeSuite(SkipBrokenTests))
+ return suite
+
+if __name__ == "__main__":
+ unittest.main(defaultTest="suite")
diff --git a/yum/__init__.py b/yum/__init__.py
index f9618bc..44894d2 100644
--- a/yum/__init__.py
+++ b/yum/__init__.py
@@ -580,9 +580,42 @@ class YumBase(depsolve.Depsolve):
if self.tsInfo.changed:
(rescode, restring) = self.resolveDeps()
+
+ # if depsolve failed and skipbroken is enabled
+ # The remove the broken packages from the transactions and
+ # Try another depsolve
+ if self.conf.skip_broken and rescode==1:
+ rescode, restring = self._skipPackagesWithProblems()
+
return rescode, restring
+ def _skipPackagesWithProblems(self):
+ ''' Remove the packages with depsolve errors and depsolve again '''
+ rescode = 1
+ # Keep removing packages & Depsolve until all errors is gone
+ # or the transaction is empty
+ while len(self.po_with_problems) > 0 and rescode == 1:
+ for po in self.po_with_problems:
+ self.verbose_logger.debug("skipping %s because of depsolving problems" % po)
+ self._skipDeps(po)
+ self.tsInfo.remove(po.pkgtup)
+ rescode, restring = self.resolveDeps()
+ return rescode, restring
+
+ def _skipDeps(self,pkg):
+ '''
+ Remove the deps for a package with dep problem
+ from the Transaction
+ '''
+ removelist = []
+ txmbr = self.tsInfo.getMembers(pkg.pkgtup)[0]
+ for dep in txmbr.depends_on:
+ self.verbose_logger.debug("skipping dependency %s because of depsolving problems" % dep)
+ self.tsInfo.remove(dep.pkgtup)
+
+
+
def runTransaction(self, cb):
"""takes an rpm callback object, performs the transaction"""
diff --git a/yum/config.py b/yum/config.py
index 174118d..f1fee12 100644
--- a/yum/config.py
+++ b/yum/config.py
@@ -555,6 +555,7 @@ class YumConf(StartupConf):
mirrorlist_expire = IntOption(86400) # time in seconds (1 day)
rpm_check_debug = BoolOption(True)
disable_excludes = ListOption()
+ skip_broken = BoolOption(False)
_reposlist = []
diff --git a/yum/depsolve.py b/yum/depsolve.py
index 55f7130..b309826 100644
--- a/yum/depsolve.py
+++ b/yum/depsolve.py
@@ -258,6 +258,12 @@ class Depsolve(object):
CheckDeps, missingdep = self._requiringFromTransaction(po, requirement, errormsgs)
else:
CheckDeps, missingdep = self._requiringFromInstalled(po, requirement, errormsgs)
+
+ # Check packages with problems
+ if missingdep:
+ if not po in self.po_with_problems:
+ self.po_with_problems.add(po)
+
return (CheckDeps, missingdep, errormsgs)
def _requiringFromInstalled(self, requiringPo, requirement, errorlist):
@@ -559,13 +565,13 @@ class Depsolve(object):
name))
# FIXME: we should probably handle updating multiple packages...
txmbr = self.tsInfo.addUpdate(best, inst[0])
- txmbr.setAsDep()
+ txmbr.setAsDep(requiringPo)
txmbr.reason = "dep"
else:
self.verbose_logger.debug('TSINFO: Marking %s as install for %s', best,
name)
txmbr = self.tsInfo.addInstall(best)
- txmbr.setAsDep()
+ txmbr.setAsDep(requiringPo)
# if we had other packages with this name.arch that we found
# before, they're not going to be installed anymore, so we
@@ -648,6 +654,9 @@ class Depsolve(object):
CheckDeps, conflicts = self._unresolveableConflict(conf, name, errormsgs)
self.verbose_logger.log(logginglevels.DEBUG_1, '%s conflicts: %s',
name, conf)
+ if conflicts:
+ if not po in self.po_with_problems:
+ self.po_with_problems.add(po)
return (CheckDeps, conflicts, errormsgs)
@@ -701,6 +710,8 @@ class Depsolve(object):
# reset what we've seen as things may have changed between calls
# to resolveDeps (rh#242368, rh#308321)
self._dcobj.already_seen = {}
+ self.po_with_problems = set()
+
CheckDeps = True
CheckRemoves = False
@@ -768,6 +779,8 @@ class Depsolve(object):
self.tsInfo.changed = False
if len(errors) > 0:
errors = unique(errors)
+ for po in self.po_with_problems:
+ self.verbose_logger.debug("%s has depsolving problems" % po)
return (1, errors)
if len(self.tsInfo) > 0:
@@ -802,6 +815,8 @@ class Depsolve(object):
missing_in_pkg = False
for po, dep in thisneeds:
(checkdep, missing, errormsgs) = self._processReq(po, dep)
+ # If there is missing deps then add the po to
+ # problem dictionary
CheckDeps |= checkdep
errors += errormsgs
missing_in_pkg |= missing
diff --git a/yum/transactioninfo.py b/yum/transactioninfo.py
_______________________________________________
Yum-devel mailing list
[email protected]
https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
