On Sat, 09 Sep 2006 16:05:56 +0100, toad wrote: > Saving a LOT of lines of code. Threads are entirely legitimate tools. > Hundreds of threads is reasonable on modern chips, especially as they are > likely double the number of cores every 2 years for some time to come. > However, ***IF*** it proves to be a serious problem there is a lot we can > do - later on - to reduce freenet's usage of threads.
How much memory do the threads use ? AFAIK JVM defaults to operating system stack size default, and that size is 2Mb in Linux. This means that a 100 threads use 200 megabytes of memory in stack space alone. Then there's the matter of wildly varying routing time if these threads are competing from the CPU simultaneously. Finally, in Linux each thread is scheduled independently of any other thread of the same application, so even if Freenet is running at nice 19, it will end up using the vast majority of CPU time even if there's nice 0 programs runnable (a hundred threads getting 1/20th of CPU time (compared to nice 0 application) each add to 5 times as much CPU time as a nice 0 application). Besides, having lots of threads becoming runnable or blocked at an unsteady rythm will cause other programs to "hiccup" randomly. This, of course, is really a Linux scheduler problem, but it will discourage users from leaving Freenet running nonetheless. And it will happen in *any* machine, no matter how powerfull the CPU or how many cores there might be, since a hundred threads is perfectly capable of bogging them all down. No, this is a bug. A design bug perhaps, but still a bug. The question is, can it be solved at this time ? After all, getting rid of threads usually means asynchronous IO, and that is both difficult, labor-intensive, and error-prone. Still, just because fixing a bug is difficult or impossible doesn't mean it isn't a bug. > We cannot avoid having *some* threads. That means we cannot avoid the bugs > that come with threads. Which means there is nothing to be gained from a > development time perspective from using fewer threads (except in specific > cases e.g. there are many checkpointed threads that could be simplified to > run on the ticker and use one less thread at the same time; these only > make up 10 or so of the threads, so they're not a priority from a > performance perspective). And there is much to be lost. State machines are > usually far more complex, and far harder to debug. Since no one seems to be able to figure out where the 1000 threads are coming from, I'd say that the current system is impossible to debug ;). > On Thu, Sep 07, 2006 at 09:17:49PM +0000, Thomas Bruderer wrote: >> >> > > Someone who develops and tells me that 1000 threads for one single >> > > application is normal... THATS what I am flameing about. >> > >> > That always depends on the application. If I have to write an >> > application that solves a problem that warrants 1000 threads, why the >> > hell not? And Freenet has a hell of a lot stuff to do. Using threads >> > is a logical and fast way to solve a certain class of problems and >> > there's nothing wrong with doing it that way. >> >> Logical? Only for programmers not knowing the gigantic overhead of >> threads. Fast? Its probrably for most problems not even faster to >> develop, especially since multithreading always leads to the nastiest >> bug ever. And for the application its obvious that it gets slower with >> each thread. >> >> There is much wrong with using so many threads. >> >> debugging impossible, still nobody can tell me where all those threads >> come from. Thats a very good security aproach to make it undebuggable. >> >> How do you think are IRC Servers written? NO, they dont use a sheer >> amount of threads. Even though in that case the benefit would be >> obvious. You know why they dont use many threads? because they couldnt >> serve thousands or tenths of thousands users simultanously. It would >> break any reasonable machine. >> >> In particular in Freenet I dont know of any bigger Problem which needs >> using multiple threads besides the different services, (fred, fcp), and >> one thread per >> connection. >> >> Maybe its too easy to write new Thread in java, maybe posix threads had >> its value in its complexity. People using them knew how difficult they >> are. >> >> >> > as you have mentioned quite correctly - threads are indeed expensive. >> > Unfortunately in most cases it's either "use a thread" or "write >> > several hundred lines of code and not use a thread." >> >> Yes, I really got upset when I read the answer of toad. Maybe I was a >> bit harsh. But I think this careless programming leads to more problems >> than saving some hundred lines of code. >> >> >> >> >> >> _______________________________________________ Devl mailing list >> Devl@freenetproject.org >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> _______________________________________________ Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl _______________________________________________ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
