On 11/05/17 02:54, Dave Reisner wrote: > On Tue, May 09, 2017 at 10:54:44PM +1000, Allan McRae wrote: >> Hi all, >> >> Every time I attempt to work on repo-add, I find it to be a very >> difficult endeavour. Even though it is half the size of makepkg >> (without even including any of libmakepkg), it is much more convoluted >> to work on. >> >> We also have a weird repository database system. We have: >> - .db dbs with package information, signatures and delta information >> - .files dbs that are the same as .db dbs but additionally include filelists >> >> There are two reasons the .files dbs replicate all information in the >> .db dbs >> - .db and .files dbs getting out of sync could cause issues >> - a complete database is useful for things like archweb, mostly to >> avoid the above >> >> I would also like to include information on source packages to these >> databases. The files information is separate due to wanting our primary >> database to be small. Likewise, source package information needs to be >> separate (the signatures take most of the size in the .db dbs, so adding >> source package signatures effectively doubles the size). >> >> So two points up for discussion: >> >> >> 1) Sync repository layout? I don't see any point in leaving the tar >> based format, as reading of sync databases is not a bottleneck. (The >> local db format can be a bottleneck, but that is a separate discussion...) > > Isn't this a historical reversal? IIRC, the sync DBs used to be expanded > onto disk, and we decided to leave them as tarballs to address > performance/fragmentation concerns.
To be clear, I was saying to stay tar based and not to move to something else. >> Do we split the information in .db out of .files and add a .full db with >> complete information? Then any .src db could follow suit and just have >> source package information. How do we get around the out of sync issue >> (e.g., a package is removed from .db, but we have an old .files database >> with it). Do we add timestamps, and print a warning on -F operations >> when the two are out of sync? >> >> >> 2) Do we need a better (read "more easily maintainable") tool for >> handling database generation and updates? libalpm already can read in >> information package files, so we could add libalpm/db_write.c with the >> database creation functions. Should we unify our repo format with our >> local database format which we already write? >> > > I'd urge you not to make this a part of pacman. It's too far off the > beaten path for most users to make it a part of an already complicated > tool. > Definitely not part of pacman. I was suggesting another program with a libalpm backend. >> >> I am looking for ideas here. Please brainstorm to your hearts content. > > WRT replacing repo-add, I'd suggest we come up with a the use cases we > want to support, design an interface to meet them, and then come up with > the implementation. Might be nice to start with the Arch Linux > repository layout as an example that we'd want to support (pooled > packages with symlinks into repo dirs). > >> Cheers, >> Allan > . >
