On Sat, 16 Dec 2006, seth vidal wrote:
If we were going to get rid of the xml format altogether I would
recommend: 1. using the sqlitedb's as an optimization and
non-api-breaking test in 3.0.X or so
2. figure out what improvements we could make to the db format to make
searching faster or to make it smaller on disk. This would mean working
out the right indexes, etc.
3. See if we could figure out a nice or simple way of providing an
sql-diff or even a binary-diff for updates to the metadata.
The thing I like about having the xml around though is that it is human
readable in a pinch. The sqlite is not.
The xml isn't really human readable either unless it's done with
createrepo -p or converted to prettyprint manually :)
Some observations on the subject...
Sqlite dump of the database is roughly the same size as the current xml
data (at least when compressed). So transferring the repodata as sql dump
would keep bandwidth requirements to roughly the same as now. Of course
that would leave some work for the clients still but surely initiating the
database from sql statements is helluva lot faster than parsing those huge
xml files and creating the sql from that. Didn't time it yet though. Raw
sql could serve as "diffs" as well, and is "human readable". Not that I
particularly like the idea of repodata as sql statements, but ...
The overall repodata size could be cut down somewhat by at least couple of
ways:
- Drop the filenames redundancy from primary.xml. It's going to require
of course the full filelists file to be downloaded at all times (diffs
would help a lot of course), but that's what apt and smart need to do
anyway (because both calculate full dependency tree at all times). Only
yum benefits from the primary.xml stuff to some extent, and sooner or
later it needs the full filelists too.
- other.xml is not typically loaded, but it could be made quite a bit
smaller by storing the changelogs just once by source rpm. The
difference is *huge* - eg FC6 SRPMS/repodata/other.xml.gz is roughly ~2M,
but ~6M for i386 and ~8M for x86_64. With that kind of size savings
somebody might even want to use it for something :)
- Panu -
_______________________________________________
Yum-devel mailing list
[email protected]
https://lists.dulug.duke.edu/mailman/listinfo/yum-devel
