----- Original Message ----- From: "Andrew Piskorski" <[EMAIL PROTECTED]> To: <sqlite-users@sqlite.org> Sent: Friday, July 15, 2005 1:27 PM Subject: Re: [sqlite] Multi-threading.
> On Fri, Jul 15, 2005 at 01:04:50PM -0400, Paul G wrote: > > > the issue wasn't necessarily the thread implementation per se, but the fact > > that threads were treated as processes for scheduling purposes and hence > > scheduled with the regular process scheduler, which was not efficient for a > > large number of processes. these problems went away when ingo molnar's O(1) > > scheduler was adopted (not sure when it was merged into linus' 2.4, but > > Interesting. Then that may explain why I never heard any first hand > reports of notable Linux threading problems. The people I tended to > talk to were probably all running with well under 100 threads per > process, and only 2 or 3 such processes at most. Presumably even the > earlier lousy process scheduler could handle that ok. all depends on what these threads were doing and the activation pattern - a bunch of compute intensive, fixed work slice threads in a constant pool are a completely different animal to, say, threads handling network and disk i/o with variable workload in dynamically growing/shrinking thread pools. another notable issue was the so-called 'thundering herd' problem - processes (traditional or threads) waiting on an event being *all* woken up when the event comes in. this would happen on accept(), for example, and the scheduler would thrash. windows had an elegant solution for this (since at least nt5) in completion ports. linux 2.6 has epoll, which has been backported to 2.4 and available in distro kernels at least for quite some time. we're getting a bit too off-topic here. in short, multi-threading and event driven models are equivalent (this has been demonstrated formally) for ideal implementations of both. the trouble is, if w're talking c, that while there are many good event driven model implementations, threads invariably suck. they are more intuitive from a comprehension standpoint, but the devil is in the details, namely concurrency/locking issues. for bonus points, use both at the same time and watch your brain (and debugger) melt. practically speaking, threads are non-trivial to do right in non-trivial applications without higher-level primitives, cf. erlang and various actor-based languages. richard's advice is solid. use async io/event loops if possible, separate processes if possible, threads as a last resort, in that order. the grey area is the 'if possible' test, since it's a multi-way tradeoff between performance, simplicity and provable (to an extent) correctness. i fully expect that a lot of folks *do* need to use threads and the probability of that being the case on windows is much higher than on posixish platforms. -p
