On Sat, 2006-12-16 at 13:48 +0200, Panu Matilainen wrote: > On Sat, 16 Dec 2006, Paul Nasrat wrote: > > > >> The only way I can think of would be a different format so we don't have > >> to parse the xml or pre-parsing the metadata into a sqlite db. This > >> would make downloads of the metadata larger but maybe it would be faster > >> for operations. > >> > >> For example - fedora extras: > >> -rw-r--r-- 1 root root 1.6M Dec 16 03:06 primary.xml.gz > >> -rw-r--r-- 1 root root 2.2M Dec 16 03:09 primary.xml.sqlite.bz2 > >> > >> bzipped the primary xml sqlite db is 2.2M vs 1.6M for the xml itself. > > > > The reason I didn't go this route for FC5 anaconda is that it's just > > the same problem as having hdlist, etc. Multiple versions of the same > > metadata, the problem we were trying to avoid by moving to repodata. > > I'd strongly argue this is the wrong approach. > > +1 > > I suggest looking closer at where the time is *really* spent. Remember the > libxml2 "slowness" which turned out to be something in the way things are > copied between C and python? Is it really the xml parsing where most of > the time is spent, or is it something else like sqlite interactions or...? > Parsing those xml files sure isn't cheap, but it's not *that* slow in > C/C++ - I'd look for other places first. > > Here's one easy target for optimization (the time difference is > consistent over successive runs): > > [EMAIL PROTECTED] yum]# yum clean dbcache > Loading "installonlyn" plugin > 3 cache files removed > [EMAIL PROTECTED] yum]# time ./yummain.py -C --disablerepo='*' > --enablerepo='core' makecache > Loading "installonlyn" plugin > Setting up repositories > ################################################## 2931/2931 > ################################################## 2931/2931 > ################################################## 2931/2931 > Metadata Cache Created >
Here's a few makecache runs of just one repository (extras) on the olpc box: -bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras' makecache >> /dev/null real 4m16.016s user 3m14.860s sys 0m14.140s -bash-3.1# yum clean dbcache Loading "installonlyn" plugin 3 cache files removed -bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras' makecache >> /dev/null real 4m3.399s user 3m12.690s sys 0m12.400s -bash-3.1# yum clean dbcache Loading "installonlyn" plugin 3 cache files removed -bash-3.1# time yum -d0 --disablerepo='*' --enablerepo='extras' makecache >> /dev/null real 4m8.325s user 3m13.450s sys 0m12.300s That's a time that cripples usability. -sv _______________________________________________ Yum-devel mailing list [email protected] https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
