Re: [Sks-devel] SKS RAM usage gone haywire

2009-02-15 Thread Ari Trachtenberg
From memory, the theory predicts that recon running time and memory should grow roughly linearly with delta; only interaction should increase. In practice, everything depends on the actual implementation and how careful OCAML is in reclaiming unused memory. Maybe Yaron can chime in.

Re: [Sks-devel] SKS RAM usage gone haywire

2009-02-15 Thread Yaron Minsky
On Sat, Feb 14, 2009 at 6:14 PM, Phil Pennock sks-devel-p...@spodhuis.orgwrote: I begin to wonder if recon is sub-optimal with a large delta of keys to send and also to wonder if I should bump learn to read OCaml up my priority list -- I'm managing to navigate the sks source faster already,

Re: [Sks-devel] Has anyone looked into bdb alternatives?

2009-02-15 Thread John Marshall
On Sat, 14 Feb 2009, 14:59 -0800, Alex Roper wrote: I've been having terrible corruption issues (need to run reconstruction every day), as I usually notice with bdb-based applications. I was Perhaps the port of BDB to your platform isn't very good; or perhaps there's some underlying disc

Re: [Sks-devel] Has anyone looked into bdb alternatives?

2009-02-15 Thread Alex Roper
We're using bdb 4.6.21 on Debian Stable so I doubt it's a software issue. Hardware is possible but unlikely, as we're properly raided and lvm-ed. I'm really not sure what to do here. I suppose I could cronjob a shutdown sks, recover, start sks, but this really shouldn't be necessary. I could

Re: [Sks-devel] SKS RAM usage gone haywire

2009-02-15 Thread Phil Pennock
On 2009-02-15 at 10:05 -0500, Yaron Minsky wrote: Ari is right: there's nothing inherent about the algorithm that should require an ever-growing use of memory. OCaml itself is very careful about reclaiming unreferenced memory, but that of course does not preclude a memory leak in the code.

Re: [Sks-devel] SKS RAM usage gone haywire

2009-02-15 Thread Daniel Kahn Gillmor
On 02/15/2009 06:33 PM, Ryan wrote: Yea, I am pretty sure I am the one going haywire.. sorry guys, trying to figure out exactly what the hell is wrong... Thanks for the quick and excellent detective work, Phil and Ryan! Since i restarted sks on zimmermann.mayfirst.org, and Ryan took his system

Re: [Sks-devel] SKS RAM usage gone haywire

2009-02-15 Thread John Marshall
On Sun, 15 Feb 2009, 19:19 -0500, Daniel Kahn Gillmor wrote: I agree with Ryan here that even if his host *had* been compromised and doing what amounts to a gossip DoS attack, the other nodes in the network should not have tried to gobble up all the RAM on their respective systems. We should