On Jan 17, 2009, at 10:53 AM, Anders F Björklund wrote:
Jeff Johnson wrote:
The --stepped option mentioned causes Smart to do multiple commits
(after splitting the ChangeSet), with each commit() doing a ts.run
Is smart calling ts.run() once or twice per-commit?
There should be only one call per commit / ts object,
but several such transactions created/commited within
one single run (and thus one python memory heap...)
For hysterical up2date reasons, yum is likely detecting file
conflicts
with a call to ts.run(), then calling ts.run() a 2nd time to
install/erase packages. The 2nd call is triggering the
fragmentation in the bug report.
Smart kinda bombs out on file conflict, when not
specified elsewhere as an explicit file dependency.
i.e. it will try the transaction, and then fail it
(reporting the error given from rpm, i.e. conflict)
Heh, rpmtsRun() has insufficient means to return info
to calling application. In fact, rpmtsRun() still does
return -1;
which was whacked into RHL 6.0 at a dead run, isn't right,
and sadly returning -1 is seen as a necessity for rpmlib
because of the number of applications using rpmlib that just
can't be changed. But I digress ...
There's likely some other code rearrangements in python, deleting
objects that assist in removing fragmentation. But that likely
cannot be done with yum, but I'm perfctly happy to help with smart.
I believe all that is being done is deleting the object.
ts = getTS(True)
# does getTS.ts = rpm.ts("/")
...
try:
probs = ts.run(cb, None)
finally:
del getTS.ts
That may not be sufficient. A rpmts object picks up a rpmdb
object with a refcnt, and there's associated state (and possible
malloc fragmentation) tied to the rpmdb object.
There's ways to force the issue if necessary. But the best way
for an application like smart to lose the state is to keep track
of __ALL__ rpmts objects, and delete __ALL__ rpmts objects to
trigger the rpmdb close. The semantics for --stepped transactions
should decompose into a series of (essentially) rpm CLI invocations.
If smart needs persistent locking, well, then smart should add
persistent application locking.
Yes, I would expect the problem to show up with --stepped, which
has more persistence, and a better chance to cause fragmentation.
Deleting a rpmts to close/open an rpmdb per-stepped transaction
will free whatever memory happens to be in use by rpmlib, and
as long as there are <100 stepped transactions, close/open an rpmdb
won't hurt much (yum was doing open-get-close cycles on every access,
that too is/was very peculier).
Will see if I get a reproducer with some -vv spew.
(and _rpmts_debug sounds like a likely candidate)
Thank you.
73 de Jeff______________________________________________________________________
RPM Package Manager http://rpm5.org
Developer Communication List rpm-devel@rpm5.org
