Re: gpg and trustdb very slow
on Tue, Sep 18, 2001 at 08:37:33AM -0300, Henrique de Moraes Holschuh ([EMAIL PROTECTED]) wrote: On Tue, 18 Sep 2001, Christian Kurz wrote: On 01-09-18 Joey Hess wrote: It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. I know that the Werner revamped the whole keyring code about 1 or two weeks ago. I just tried the --list-keys with my private and the debian keyring specified in the options file and I didn't took more then 5 minutes for me. (AMD K6-200). So I don't think if anyone would really want to optimize the code more, he should look at the current cvs code. As well as reading the docs, where it is said that one must reinsert all keys in the trustdb to take advantage of a new caching algorithm... Could you point us to the relevant docs and/or the command for re-reading keys? Peace. -- Karsten M. Self kmself@ix.netcom.comhttp://kmself.home.netcom.com/ What part of Gestalt don't you understand? Home of the brave http://gestalt-system.sourceforge.net/Land of the free Free Dmitry! Boycott Adobe! Repeal the DMCA! http://www.freesklyarov.org Geek for Hire http://kmself.home.netcom.com/resume.html pgpmTtP8aIl64.pgp Description: PGP signature
Re: gpg and trustdb very slow
On Sat, 22 Sep 2001, Karsten M. Self wrote: on Tue, Sep 18, 2001 at 08:37:33AM -0300, Henrique de Moraes Holschuh ([EMAIL PROTECTED]) wrote: On Tue, 18 Sep 2001, Christian Kurz wrote: On 01-09-18 Joey Hess wrote: It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. I know that the Werner revamped the whole keyring code about 1 or two weeks ago. I just tried the --list-keys with my private and the debian keyring specified in the options file and I didn't took more then 5 minutes for me. (AMD K6-200). So I don't think if anyone would really want to optimize the code more, he should look at the current cvs code. As well as reading the docs, where it is said that one must reinsert all keys in the trustdb to take advantage of a new caching algorithm... Could you point us to the relevant docs and/or the command for re-reading keys? Nope, I have no idea where the hell I read that. I'd try the changelogs of upstream and the Debian package, however: that rings a bell on my mind. As for the useful information, that I can help you with. All you have to do is to let gnupg readd all your keys to a keyring (as in 'new insertions to a keyring are made in the new cache scheme'). Simple enough to do with any RW keyrings you might have: 1. move every keyring out of the way, including the RO ones. 2. feed a keyring to gnupg. You might want to use --fast-import followed by an --update-trustdb for that. 3. replace the old keyring you fed gnupg with with the new one 4. repeat as needed. I actually find it better to have just one RW keyring, and feed any RO keyrings I might need (such as Debian's) to gnupg to get them added to that RW keyring. Since gnupg's handling of multiple keyrings is not really that hot, keeping track of only a simple RW keyring helps a lot. Peace. Hmm, sorry about that. I sounded far more annoyed/annoying ;) than I should have. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
Re: gpg and trustdb very slow
On Mon, Sep 17, 2001 at 11:54:56PM -0400, Joey Hess wrote: Has anyone else noticed that using gpg with debian-keyring 2001.09.03 results in excessively slow trustdb-related things? This is exactly what I was complaining about on IRC a few days ago. I simply am not going to use the Debian keyring if the precence of so many keys causes verifying a single mail to take barely-finite lengths of time. That's too bad, since there's a lot to be gained by actually verifying Debian signatures. Another example, I ran gpg --list-keys 6 hours ago. It's not done yet. This on a P2-300, 256mb ram. The slowdown is because it is evidently inserting each key into the trustdb in turn, and a single insert takes several minutes. Did you even TRY --check-sigs? :)
Re: gpg and trustdb very slow
Aaron Lehmann wrote: On Mon, Sep 17, 2001 at 11:54:56PM -0400, Joey Hess wrote: Has anyone else noticed that using gpg with debian-keyring 2001.09.03 results in excessively slow trustdb-related things? This is exactly what I was complaining about on IRC a few days ago. I simply am not going to use the Debian keyring if the precence of so many keys causes verifying a single mail to take barely-finite lengths of time. That's too bad, since there's a lot to be gained by actually verifying Debian signatures. I should note that one workaround is to add --always-trust to your gpg command line. At your own risk, but it does completly eliminate the long trustdb step, reducing signature verification to its former speed. I _hope_ that after --list-keys forces it to insert each and every key into the trust db and do all the long calculations, and hopefully cache them, the speed issue will be dealt with. We'll see; after 7 hours gpg is still churning away on that command. It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. [FWIW, this is a strace of a single gpg verify operation. I didn't even let it run all the way though, giving up when the strace log reached 700 MB: [EMAIL PROTECTED]:~grep llseek log |wc -l 5036905 ] -- see shy jo
Re: gpg and trustdb very slow
On Tue, Sep 18, 2001 at 01:27:20AM -0400, Joey Hess uttered: It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. I doubt very strongly that would end up being implemented, as gnupg likes to lock the memory it plays with, so it can be considered secure, I doubt upstream would consider what is described above as secure. :-/ [FWIW, this is a strace of a single gpg verify operation. I didn't even let it run all the way though, giving up when the strace log reached 700 MB: [EMAIL PROTECTED]:~grep llseek log |wc -l 5036905 ] Whoa. -- Steve StevenK Nah, I'm allergic to python. StevenK It makes me swear and break stuff. pgplhbAkz5DPj.pgp Description: PGP signature
Re: gpg and trustdb very slow
On 01-09-18 Joey Hess wrote: It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. I know that the Werner revamped the whole keyring code about 1 or two weeks ago. I just tried the --list-keys with my private and the debian keyring specified in the options file and I didn't took more then 5 minutes for me. (AMD K6-200). So I don't think if anyone would really want to optimize the code more, he should look at the current cvs code. Christian -- Debian Developer (http://www.debian.org) 1024/26CC7853 31E6 A8CA 68FC 284F 7D16 63EC A9E6 67FF 26CC 7853 pgpKkCna6h751.pgp Description: PGP signature
Re: gpg and trustdb very slow
On Tue, 18 Sep 2001, Christian Kurz wrote: On 01-09-18 Joey Hess wrote: It'd be nice if someone would look at optimizing it sometime; the behavior I see with strace is absurd, and could easily be done with no syscalls, at least, by just reading the whole trustdb into memory. I know that the Werner revamped the whole keyring code about 1 or two weeks ago. I just tried the --list-keys with my private and the debian keyring specified in the options file and I didn't took more then 5 minutes for me. (AMD K6-200). So I don't think if anyone would really want to optimize the code more, he should look at the current cvs code. As well as reading the docs, where it is said that one must reinsert all keys in the trustdb to take advantage of a new caching algorithm... I just import all the Debian keyring into a work keyring, all at once. It takes no more than 10 minutes in my k6-200. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
gpg and trustdb very slow
Has anyone else noticed that using gpg with debian-keyring 2001.09.03 results in excessively slow trustdb-related things? For example, it often takes me 10 minutes to verify a gpg signed mail now. Strace shows literally hundreds of thousands of seeks and reads in the trustdb file, with many thousands of duplicated reads of each record. Another example, I ran gpg --list-keys 6 hours ago. It's not done yet. This on a P2-300, 256mb ram. The slowdown is because it is evidently inserting each key into the trustdb in turn, and a single insert takes several minutes. It does seem that after doing this thing with the trustdb for a key, future uses of that key are as fast as I used to expect. -- see shy jo
