Re: Learning dpkg/apt
On Sat, 19 Aug 2000, Simon Richter wrote: Actuallu the slowest thing about dpkg is the database of files. I would be cool if dpkg could use some sort of relational database for that. As long as it is still text-based, so that I can edit it by hand if neccessary. ;-) Yust to say: PLEASE do not make it binary. There may be many things to speed up, but binary is evil in this situation. (Imagine a broken database and you can do just nothing against it.) Hochachtungsvoll, Bernhard R. Link
Re: Learning dpkg/apt
On 19-Aug-00, 18:56 (CDT), Ethan Benson [EMAIL PROTECTED] wrote: for things like querying (dpkg -s and such) install dlocate it solves that problem the Right Way. (unfortunatly it got removed from potato for less then critical bugs) man apt-cache. (Assuming you're using apt-get either directly or via dselect, and there is no reason not to!) Steve
Re: Learning dpkg/apt
On 19-Aug-00, 18:56 (CDT), Ethan Benson [EMAIL PROTECTED] wrote: for things like querying (dpkg -s and such) install dlocate it solves that problem the Right Way. (unfortunatly it got removed from potato for less then critical bugs) * Steve == Steve Greenland [EMAIL PROTECTED] wrote: Steve man apt-cache. (Assuming you're using apt-get either directly Steve or via dselect, and there is no reason not to!) apt-cache and dlocate cover quite different areas. dlocate is for dpkg queries like -L and -S (and more). And it is so damn _fast_. $ time dpkg -l \*applet\* /dev/null real0m2.010s user0m1.880s sys 0m0.110s $ time dlocate -l applet /dev/null real0m0.037s user0m0.030s sys 0m0.010s $ time apt-cache --names-only search applet /dev/null real0m0.703s user0m0.350s sys 0m0.350s [These are the times of the second invocation, so memory cache/buffers are used] IMHO, it is a must have. Cudos to Craig. Ciao, Martin
Learning dpkg/apt
I want to learn the total innards of dpkg/apt. I recently filed a bug complaining about the fact that dpkg is too slow, but I want to actually _do_ something about it (other than ordering other developers around). So Can someone who knows dpkg give me a good list of the stuff I should read, or the order in which I should read the dpkg/apt source code? -- Dwayne C. Litzenberger - [EMAIL PROTECTED] - Please always Cc to me when replying to me on the lists. - See the mail headers for GPG/advertising/homepage information. pgprq6HpVYeNv.pgp Description: PGP signature
Re: Learning dpkg/apt
On Fri, Aug 18, 2000 at 09:30:18PM -0600, Dwayne C . Litzenberger wrote: I want to learn the total innards of dpkg/apt. I recently filed a bug complaining about the fact that dpkg is too slow, but I want to actually _do_ something about it (other than ordering other developers around). So Can someone who knows dpkg give me a good list of the stuff I should read, or the order in which I should read the dpkg/apt source code? Source is the best documentation for a program...all hail the invention of comments :) -- ---===-=-==-=---==-=-- / Ben Collins -- ...on that fantastic voyage... -- Debian GNU/Linux \ ` [EMAIL PROTECTED] -- [EMAIL PROTECTED] -- [EMAIL PROTECTED] ' `---=--===-=-=-=-===-==---=--=---'
Re: Learning dpkg/apt
On Fri, 18 Aug 2000, Dwayne C . Litzenberger wrote: I want to learn the total innards of dpkg/apt. I recently filed a bug complaining about the fact that dpkg is too slow, but I want to actually _do_ something about it (other than ordering other developers around). Actuallu the slowest thing about dpkg is the database of files. I would be cool if dpkg could use some sort of relational database for that. Simon -- PGP public key available from http://phobos.fs.tum.de/pgp/Simon.Richter.asc Fingerprint: 10 62 F6 F5 C0 5D 9E D8 47 05 1B 8A 22 E5 4E C1 Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!
Re: Learning dpkg/apt
Source is the best documentation for a program...all hail the invention of comments :) I know that, but dpkg/apt is a huge pile of source. I'd like to know what order I should read things in, so not to get too confused with too many forward references. -- Dwayne C. Litzenberger - [EMAIL PROTECTED] - Please always Cc to me when replying to me on the lists. - See the mail headers for GPG/advertising/homepage information. pgp4aZS1RL9FQ.pgp Description: PGP signature
Re: Learning dpkg/apt
On Sat, Aug 19, 2000 at 05:07:41PM +0200, Simon Richter wrote: On Fri, 18 Aug 2000, Dwayne C . Litzenberger wrote: I want to learn the total innards of dpkg/apt. I recently filed a bug complaining about the fact that dpkg is too slow, but I want to actually _do_ something about it (other than ordering other developers around). Actuallu the slowest thing about dpkg is the database of files. I would be cool if dpkg could use some sort of relational database for that. no it wouldn't, as soon as that database gets corrupted in whatever way your completly screwed and have to reinstall. this happened to me once and the only thing that saved me was the fact that the dpkg databases were in human readable text format. ill take slow over unrecoverable any day. for things like querying (dpkg -s and such) install dlocate it solves that problem the Right Way. (unfortunatly it got removed from potato for less then critical bugs) -- Ethan Benson http://www.alaska.net/~erbenson/ pgpiQMnRN865i.pgp Description: PGP signature
Re: Learning dpkg/apt
no it wouldn't, as soon as that database gets corrupted in whatever way your completly screwed and have to reinstall. That's why I suggested a *cacheing* system for the text database. Every write operation happens twice (once in the binDB, once in the txtDB), but every read operation can come straight from the binDB. Hashing could be used to check if the text database is still equal to the binary database. ill take slow over unrecoverable any day. Same here, but it is possible to have the best of both worlds. -- Dwayne C. Litzenberger - [EMAIL PROTECTED] - Please always Cc to me when replying to me on the lists. - See the mail headers for GPG/advertising/homepage information. pgp4RzulMDyyT.pgp Description: PGP signature
Re: Learning dpkg/apt
On Sat, Aug 19, 2000 at 06:00:54PM -0600, Dwayne C . Litzenberger wrote: no it wouldn't, as soon as that database gets corrupted in whatever way your completly screwed and have to reinstall. That's why I suggested a *cacheing* system for the text database. Every write sorry i have not followed this thread very close. operation happens twice (once in the binDB, once in the txtDB), but every read operation can come straight from the binDB. Hashing could be used to check if the text database is still equal to the binary database. it seems to me that this would make things slower since everything is written twice, the text database would still have to be processed in order to update it properly, so i don't think installing packages would be much faster, if not slower. querying would indeed be fast but as i mentioned that can be better achieved by rebuilding a binary database with a cron job as in dlocate. -- Ethan Benson http://www.alaska.net/~erbenson/ pgpMxwPV4b8ir.pgp Description: PGP signature
Re: Learning dpkg/apt
it seems to me that this would make things slower since everything is written twice, the text database would still have to be processed in order to update it properly, so i don't think installing packages would be much faster, if not slower. querying would indeed be fast but as i mentioned that can be better achieved by rebuilding a binary database with a cron job as in dlocate. You underestimate how many times the database is read from. For example, AFAIK, for each file installed, it must be checked against the diversions database to see if it has been diverted elsewhere. Anyway, the actual details have not been worked out. Another way of doing it would be as follows: 1. The checksum (md5, sha, etc) is computed for all the txtDB files. If they do not match the checksums in the binDB, the binDB files are recomputed. If they do, things are left as they are. 2. Packages are manipulated (installed, removed, configured, etc) using the binDB only. Changes are also written to a textmode journal, in case the binDB is corrupted. 3. The txtDB is then recomputed and written. Or, the changes journal is optimised, then merged with the txtDB. This occurs *once* at the end of an install cycle, rather than for each and every package manipulated. I think that would speed things up substantially. Anyway, this is getting a bit offtopic. The main question was that if someone who is familiar with dpkg could give me a few pointers as to where to start, or how I should go about learning the internals of dpkg from scratch. -- Dwayne C. Litzenberger - [EMAIL PROTECTED] - Please always Cc to me when replying to me on the lists. - See the mail headers for GPG/advertising/homepage information. pgpePiaCnzqAq.pgp Description: PGP signature
Re: Learning dpkg/apt
On Sat, Aug 19, 2000 at 02:20:26PM -0600, Dwayne C . Litzenberger wrote: I know that, but dpkg/apt is a huge pile of source. I'd like to know what order I should read things in, so not to get too confused with too many forward references. Try cflow. BTW, avoiding forward refs = bottom-up parsing. IMHO, attacking a new piece of source top-down seems to make comprehension easier. I'd start with main(). -- Steve Bowman [EMAIL PROTECTED] (preferred) Buckeye, AZ [EMAIL PROTECTED] [EMAIL PROTECTED] http://www.goodnet.com/~sbowman/ Powered by Debian GNU/Linux http://www.debian.org
