Hi!
I did some test runs to find out the current state of yum and to see if the
new resolver is an improvement over the rpmlib.
The candidates:
===============
* yum-3.0.3 as used in FC6
* yum from CVS head (2007/05/07), which is very similar to the yum in F7
* pyrpmyum CVS head (2007/05/07) - a pure Python implementation developed
for testing purposed [1].
The test cases:
===============
Tests are based in FC-5 run on FC-6.x86_64. All operations are aborted by
"pressing n".
Installs:
Try to install into an empty build root. There is a big number of excluded
pkgs to make the install '*' work.
* Normal install: several groups, 856 Pkgs
* Full install: "*", 6057 Pkgs
* Install "[a-k]*", 3052 Pkgs - needs to drag in a lot of
pkgs to satisfy dependencies
Updates:
Before the Updates the "Normal Install" is run and the Pkgs are installed
into the build root.
* Empty update
* Update libgnome: 1 Pkg
* Big update: 346u+12i Pkgs
* Dist upgrade: Enable FC6 repos and do an upgrade; fails for both yum
versions,so don't take results too serious
Erase:
* glibc: 828 Pkgs
Warm ups:
Do a install '*'/upgrade to build the caches. Each program has different
issues with that. yum-3.0.3 needs to download the rpm headers (Test was run
with repos in a NFS mount). All need to build the sqlite databases. pyrpmyum
currently uses a Python only implementation that uses a lot of time. Warm up
operations fail for all programs.
This all make the results of the warm ups a bit unreliable and questionable.
They are included for completeness.
Measured Values:
================
All data is from just one test run - so don't expect too high precision. I
put 3 different test results into tables:
* Real time passed during the test
* Heap peak - maximum of Memory used
* Heap total - amount of heap memory allocated
The last one gives an idea of how much data has been moved around.
Conclusions:
============
Please read the data tables first to get your own impression.
Memory usage:
The new yum saves a lot of memory for big operations (about 50%). The
comparison to pyrpmyum shows that it is possible to save even more memory
and to drop memory usage below 100MB for all cases. In pyrpmyum this is
achieved with dynamic loading of rpm tags. Some are not even kept in the
pkgs but just cached in a least recently used list.
For really small updates the old 3.0.3 yum wins. I guess this is because it
does not do any obsolete handling.
Runtime:
Although the new yum save the very costly rpm header downloads (which take
nearly 3 minutes for the whole repository) it is still significantly slower
for big operations. Things may look a bit different in real world scenarios
where downloading the headers is even more expensive.
Nevertheless the performance of the current yum is poor. Runtime behavior
has dropped from O(#prco * #rpmlib_calls) to O(#prco^2) (prco = Provides,
Requires, Conflicts, Obsoletes). The results of pyrpmyum show that is it
possible to do the depsolving in O(#prco) (Probably just O(#prco * log
#prco) but that doesn't matter much).
The good results for small operations for the CVS yum are "achieved" by
insufficient conflict and obsolete checks[2]. So even these good results are
no reason for cheering.
To get the performance fixed all quadratic behavior has to be removed. This
includes linear searches during resolving that can be replaced by hashes or
sqlite queries. In addition each list has to be checked and replaced by
hashes or sets if necessary. After this is done we can try to get rid of all
loops over file lists. After that the performance should be OK.
Have fun
Florian Festi
[1] https://hosted.fedoraproject.org/projects/pyrpm
[2] Detailed description will follow in a separate mail
Heap peak (MB)
Operation | 3.0.3 | 05/07 | pyrpm
------------------------------------------------
Warmup: Full install | 475.3 | 226.6 | 128.6
Resolve normal install | 94.6 | 74.1 | 27.0
Resolve full install | 462.4 | 218.5 | 59.8
Resolve install [a-k]* | 278.5 | 137.8 | 52.0
Resolve empty update | 25.9 | 52.9 | 45.2
Resolve update libgnome | 30.3 | 57.0 | 48.8
Resolve big update | 112.5 | 108.2 | 60.3
Resolve erase glibc | 210.0 | 99.8 | 39.6
Warmup: Dist upgrade | 162.7 | 107.3 | 112.0
Resolve dist upgrade | 162.4 | 107.3 | 62.9
Heap total (MB)
This is the total amount of memory requested.
The amount of memory being allocated at one time
is much less!
Operation | 3.0.3 | 05/07 | pyrpm
-----------------------------------------------
Warmup: Full install | 20003 | 90859 | 10829
Resolve normal install | 2088 | 9448 | 639
Resolve full install | 15402 | 84951 | 4987
Resolve install [a-k]* | 17542 | 24805 | 4121
Resolve empty update | 167 | 80 | 747
Resolve update libgnome | 282 | 136 | 1247
Resolve big update | 3497 | 2699 | 2574
Resolve erase glibc | 1732 | 2192 | 3369
Warmup: Dist upgrade | 6203 | 2644 | 7395
Resolve dist upgrade | 5869 | 2644 | 2154
Real time (s)
Operation | 3.0.3 | 07/05/07| pyrpm
----------------------------------------------------
Warmup: Full install | 653.51 | 1064.35 | 147.23
Resolve normal install | 40.59 | 134.11 | 8.04
Resolve full install | 477.67 | 1093.57 | 51.01
Resolve install [a-k]* | 313.56 | 887.40 | 33.52
Resolve empty update | 6.91 | 6.57 | 13.36
Resolve update libgnome | 3.34 | 2.57 | 7.51
Resolve big update | 33.04 | 45.56 | 24.81
Resolve erase glibc | 29.83 | 37.72 | 30.95
Warmup: Dist upgrade | 81.42 | 40.67 | 108.50
Resolve dist upgrade | 52.07 | 40.29 | 27.20
_______________________________________________
Yum-devel mailing list
[email protected]
https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
