On Sun, Sep 17, 2006 at 12:07:54PM +0100, Michael Rogers wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Jusa Saari wrote: > > No, this is a bug. A design bug perhaps, but still a bug. The question is, > > can it be solved at this time ? After all, getting rid of threads usually > > means asynchronous IO, and that is both difficult, labor-intensive, and > > error-prone. Still, just because fixing a bug is difficult or impossible > > doesn't mean it isn't a bug. > > If most of the threads belong to requests/inserts, we can get rid of > them by converting requests and inserts into state machines - see my > simulation code for an idea of how this might work.
Of course, but if that code is one line longer then it isn't worth it. Developer time costs several orders of magnitude more than the marginal amount of RAM and CPU used by threads. And if it really is a problem, the solution is to use continuations. I am perfectly willing to consider converting requests and inserts to state machines, however you will have to convince both me and ian that the code would be significantly simpler. > > If the threads belong to transfers (BlockTransmitter), getting rid of > them would be more difficult. Not really. > We could make PeerNode responsible for all > packets sent to the peer - transfers, searches, keepalives, etc. > However, this would introduce extra complexity as PeerNode would have to > interleave transfers and searches - a 32 kB transfer can easily take > several seconds, so we can't allow it to block other traffic. I don't see why this can't be done as a state machine too. But if we have hundreds of block transfers going on, something is wrong. > > Matthew and I discussed some ideas for interleaving transfers and > searches: one possibility is to have a "fast queue" for searches etc, > and a separate "slow queue" for each transfer. When assembling a packet, > PeerNode takes one message from the fast queue, then if there's space > left in the packet it takes one message from the current slow queue (the > slow queues are processed in round-robin order). Then it fills any > remaining space with messages from the fast queue. This is sensible, and irrelevant to the matter under discussion. > This ensures that > neither searches nor transfers starve, as long as messages in the fast > queue are generally small enough to share a packet with a 1 kB transfer > message. This is currently OK because most search messages are small, > but in the future if we want to put large messages in the fast queue (eg > for one-to-one streams), we'll have to rethink this. Those would still be slow-queue items. > > > Since no one seems to be able to figure out where the 1000 threads are > > coming from, I'd say that the current system is impossible to debug ;). > > Hardly impossible - the person who claimed 1000 threads hasn't tried > using a debugger to list the threads. Everything's impossible to debug > if you won't use a debugger. ;-) You don't even need a debugger. (The last time I tried to use eclipse's debugger on Freenet it crashed...) ./run.sh dump (on linux; on windows everything is harder) > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20060918/8537719d/attachment.pgp>
