Re: O_CLOEXEC (was Re: thread rant)
dean gaudet wrote: > suppose you're building a threaded server to scale to 20k connections, > and you want to support fork()/exec() of CGIs dirctly (rather than > through a separate daemon process). > > if you don't use close-on-exec, then after fork before exec you have > to iterate over the 20k descriptors to close them all -- or you have > to walk your own data structures (a list of open sockets) to find what > descriptors are open and close them. > if you use O_CLOEXEC then it's not possible for an fd to be open which > isn't marked to be closed on the exec, and so you've no extra work to > do before exec. > > but if you try to use FD_CLOEXEC there's a critical section between > the open/socket/accept/pipe/... and the fcntl(). you pretty much have > to put it under a semaphore (although you can get away with a rwlock), > or iterate over all the descriptors. Yes, this is the worst part. Having to synchronise threads. > i've a hairbrained idea for replacing fd integers with pointers which > might actually scale better if anyone wants to hear it again. Completely hairbrained, would make no difference to scalability, and make accessing your user space data structures slower. (Think X window ids and the need to use a hash table instead of a flat table). O_ANYFD would, in principle, improve scalability. It was proposed a few years ago -- it means "kernel can return any fd, not necessarily the next free one". I don't think it was considered worth the effort. -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
O_CLOEXEC (was Re: thread rant)
On Sat, 2 Sep 2000, Alexander Viro wrote: > On Sat, 2 Sep 2000, Jamie Lokier wrote: > > > dean gaudet wrote: > > > an example of brokenness in the traditional fd API is close-on-exec -- > > > there's a race between open()/socket()/pipe() and fcntl(FD_CLOEXEC) during > > > which if another thread does a fork() it's possible the child will inherit > > > an fd it shouldn't... working around it is painful. the model which > > Really? Like, say it, close() before exec()? > > > > NT/OS2 use for creating a new process scales better in the 99.99% case of > > > stdin/out/err -- you only specify those fds you want to keep in the new > > > process. > > > > An obvious solution presents itself. O_CLOEXEC. yeah i proposed this and non-blocking inheritance from accept() when i was doing the initial modelling for threaded apache-2.0 about 3 years ago. you need O_CLOEXEC to be inherited through accept() as well. oh, and you also want it on pipe(), socket(), and socketpair() to be complete. (if you search linux-kernel archives you can find that this topic has come up maybe 4 times in 3 years.) > Even more obvious solution: close what you need to close if you have > sensitive descriptors around. Close-on-exec is a kludge. If you have > sensitive pieces of descriptor table you want to do some other things too > - e.g. make sure that it gets unshared before exec(). Because new > descriptors of that kind are very likely to follow... if you'll forgive me for using webservers as an example again i can maybe give some motivation. suppose you're building a threaded server to scale to 20k connections, and you want to support fork()/exec() of CGIs dirctly (rather than through a separate daemon process). if you don't use close-on-exec, then after fork before exec you have to iterate over the 20k descriptors to close them all -- or you have to walk your own data structures (a list of open sockets) to find what descriptors are open and close them. if you want to walk your own data structures then these structures need to be coherent at the time of the fork -- which means semaphores to add/delete from the list. this would mean an extra synchronization after accept(). i suspect this would suck worse than iterating over 3..2 and calling close -- think of all the cache line activity on an SMP box. if you use O_CLOEXEC then it's not possible for an fd to be open which isn't marked to be closed on the exec, and so you've no extra work to do before exec. but if you try to use FD_CLOEXEC there's a critical section between the open/socket/accept/pipe/... and the fcntl(). you pretty much have to put it under a semaphore (although you can get away with a rwlock), or iterate over all the descriptors. opendir() has a similar race, except it's hidden inside libc. the most portable, and arguably cleaner solution is to use a separate unthreaded process for forking and fd passing... as long as this meets your needs. fork/exec and threads just don't mix so well with lots of fds. (a spawnve() API with an additional argument to indicate which fds to dup would be better... similar to NT's CreateProcess) i've a hairbrained idea for replacing fd integers with pointers which might actually scale better if anyone wants to hear it again. -dean - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: thread rant
> I've heard comments from Alan, and others in the past bashing > threads, and I can understand the "threads are for people who > can't write state machines" comments I've heard, but what other > ways are there of accomplishing the goals that threads solve in > an acceptable manner that gives good performance without coding > complexity? That's a nonsensical complaint anyway, since you need to use state machines with threads. When there's work to be done, you hand it to a thread. That thread needs to know what state the object it's working on was in. When that thread is finished working, that object is in some other state. The reason you need threads is because you may have 12,000 state machines and 3 of them may need to be worked on at the same time. You may not know ahead of time whether the needed work is CPU intensive, I/O intensive, or neither. > This is all just curiosity. I've considered trying some thread > programming, but if it is as stupid as it sounds, I'd rather > learn the "right" way of writing code that would ordinarily be > done with threads, etc.. Right now, I'm using fork() all over > the place and don't much care how much waste it is... I'd like > to though. Fork is good if you really need a whole process. It's also good if you're doing something potentially risky and can't afford to the process that is forking fail. It makes life miserable, however, if you need tight communication with the forked process. There are plenty of things I don't like about pthreads. But pthreads is not threads. A lot of the ugliness in pthreads was to allow operating systems that couldn't do threads right to claim to 'fully support' threading. One major problem with a big 'poll loop' is that you're vulnerable to ambush. A sudden page fault from a slow disk (or worse, NFS mount *shudder*) and your entire process screeches to a halt. Obviously, one process per connection/task is out of the question. Threads give you a very satisfying in-between point. DS - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
Hi, On Sat, Sep 02, 2000 at 09:41:03PM +0200, Ingo Molnar wrote: > > On Sat, 2 Sep 2000, Alexander Viro wrote: > > > unlink() and the last munmap()/exit() will get rid of it... > > yep - and this isnt possible with traditional SysV shared memory, and isnt > possible with traditional SysV semaphores. (forget my filesystem comment, > it's a thinko.) Using the SysV APIs, you can do this for shared memory: do an IPC_RMID and the segment will disappear on the last detach/exit. But yes, you can't do the same for SysV semaphores. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
Richard Gooch <[EMAIL PROTECTED]> writes: > Ingo Molnar writes: > > yep - and this isnt possible with traditional SysV shared memory, > > and isnt possible with traditional SysV semaphores. (forget my > > filesystem comment, it's a thinko.) > > Huh? After attaching, if you "delete" a SHMID, it will be removed > when all processes referring to it unattach or die. Fairly similar > semantics. Would be nice if it really would be _that_ similar. We had to reintroduce some linuxism that you can still attach to an deleted SHMID. This is quite uncommon for filesystems (and for SYSV shm on other *NIX) Christoph - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: thread rant [semi-OT]
just an aside on asynchronous i/o: concurrency by asychronous I/O actually predates concurrency via thread-like models, and goes back to the earliest OS-precursors. Early work on thread-like concurrency models were, in part, a response to the difficulties inherent in getting asynchronous I/O right, and so now the pendulum is swinging back. A pedantic nit: the basic Un*x I/O model, with syncrhronous interfaces predates Un*x networking by some time. I woudl make the case that the people who grafted Un*x networking (most notably sockets) onto Un*x didn't really understand the I/O model, and crafted cruft, that just happened to have a few usable features, one being a sort-of way of sometimes getting asynchrony. Prior to posix, by the way, there were any number of Un*x variants that asynchronous I/o models, supporting any number of i/o completion notification models, buffering schemes and (lack of) cancellation semantics. My personal favorite was the variant in the Cray-2 Unicos; my personal least favorite was Intergraph's Clix. Marty -Original Message- From: Dan Maas [mailto:[EMAIL PROTECTED]] Sent: Friday, September 01, 2000 11:50 PM To: [EMAIL PROTECTED] Subject: Re: thread rant [semi-OT] [...] Can we do better? Yes, thanks to various programming techniques that allow us to keep more of the system busy. The most important bottleneck is probably the network - it makes no sense for our server to wait while a slow client takes its time acknowledging our packets. By using standard UNIX multiplexed I/O (select()/poll()), we can send buffers of data to the kernel just when space becomes available in the outgoing queue; we can also accept client requests piecemeal, as the individual packets flow in. And while we're waiting for packets from one client, we can be processing another client's request. The improved program performs better since it keeps the CPU and network busy at the same time. However, it will be more difficult to write, since we have to maintain the connection state manually, rather than implicitly on the call stack. So now the server handles many clients at once, and it gracefully handles slow clients. Can we do even better? Yes, let's look at the next bottleneck - disk I/O. If a client asks for a file that's not in memory, the whole server will come to a halt while it read()s the data in. But the SCSI/IDE controller is smart enough to handle this alone; why not let the CPU and network take care of other clients while the disk does its work? How do we go about doing this? Well, it's UNIX, right? We talk to disk files the same way we talk to network sockets, so let's just select()/poll() on the disk files too, and everything will be dandy... (Unfortunately we can't do that - the designers of UNIX made a huge mistake and decided against implementing non-blocking disk I/O as they had with network I/O. Big booboo. For that reason, it was impossible to do concurrent disk I/O until the POSIX Asynchronous I/O standard came along. So we go learn this whole bloated API, in the process finding out that we can no longer use select()/poll(), and must switch to POSIX RT signals - sigwaitinfo() - to control our server***). After the dust has settled, we can now keep the CPU, network card, and the disk busy all the time -- so our server is even faster. [...] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
RE: thread rant
I'm confused. Threads are harder than *what* to get right? If you need concurrency, you need concurrency, and any existing model is hard. Besides, at some level, all of the concurrency models come down to a variant on threads, anyway. -Original Message- From: Alexander Viro [mailto:[EMAIL PROTECTED]] Sent: Saturday, September 02, 2000 1:07 AM To: dean gaudet Cc: Mike A. Harris; Michael Bacarella; Linux Kernel mailing list Subject: Re: thread rant On Sat, 2 Sep 2000, dean gaudet wrote: > the thread bashing is mostly bashing programs which do things such as a > thread per-connection. this is the most obvious, and easy way to use > threads, but it's not necessarily the best performance, and certainly > doesn't scale. (on the scalability side just ask yourself how much RAM is > consumed by stacks -- how many cache lines will that consume, and how many > TLB entries. it sucks pretty fast.) > state machines are hard. while people on linux-kernel may be hot shot > programmers who can do state machines in their sleep, this is definitely > not the case for the average programmer. > > fortunately fully threaded and fully state driven are two opposite ends of > a spectrum, and there are lots of useful compromises in between where > threads are used in a way that allows the average programmer to > maintain/extend a codebase; while also getting the scalability of state > machines. Lovely. You've completely sidestepped the main part: *threads* *are* *hard* *to* *write* *correctly* Average programmer will fuck up and miss tons of race conditions. Better than average programmers do. If you've got shared resource - you are in for problems. Threads are useful. But they take more efforts and are much harder to debug _and_ to prove correctness. In other words, that's one of the last resort tools, not the first one. If you can do it without shared state - don't bother with threads. Same as with rewrite in assmebler - do it with small critical parts and don't unless you absolutely have to. The first rule of optimisation: don't. Keep the critical sections small and if you can avoid them - it's worth the efforts. You'll win a lot when it will come to changing the thing. And you will have to change it, sooner or later. Same goes for debugging. Non-threaded code is easier to understand. Yes, you may be very clever. But you'll have to debug it on Friday evening after a hard week when you want only one thing - go home and sleep. Or somebody else will and he will curse you. KISS. And threads are _not_ simple. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Fri, 1 Sep 2000, Michael Bacarella wrote: > > Q: Why do we need threads? > A: Because on some operating systems, task switches are expensive. No. threads share variable and code memory, processes do not. And sometimes it can make your life a lot easier. Even if you can use things such as SHM or so. > Q: So, threads are a hack to get around operating systems that suck? > A: Basically. I don't agree. > Q: So, why must Linux support threads? > A1: : | > A2: So other programs can be easily ported to Linux! > > That can already happen. It's not the *best* implementation. It's > not as fast as it can be. But it works. And that's all it should do. If > you're not happy, cope. > > "But threads on this system are faster than on Linux!" > > The fact that the system implements threads speaks enough about > it's capabilities. ie, it's trying hard to suck less. So, from my POV, > we're looking to make Linux suck more by effectively emulating systems > that are trying to suck less. > > But, I've never done anything worthwhile for Linux, so take this for what > it's worth, from an asshole. > -MB Igmar - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
Ingo Molnar writes: > > On Sat, 2 Sep 2000, Alexander Viro wrote: > > > > what i mean is that i dont like the cleanup issues associated with SysV > > > shared memory - eg. it can hang around even if all users have exited, so > > > auto-cleanup of resources is not possible. > > > > unlink() and the last munmap()/exit() will get rid of it... > > yep - and this isnt possible with traditional SysV shared memory, > and isnt possible with traditional SysV semaphores. (forget my > filesystem comment, it's a thinko.) Huh? After attaching, if you "delete" a SHMID, it will be removed when all processes referring to it unattach or die. Fairly similar semantics. Regards, Richard Permanent: [EMAIL PROTECTED] Current: [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Alexander Viro wrote: > > what i mean is that i dont like the cleanup issues associated with SysV > > shared memory - eg. it can hang around even if all users have exited, so > > auto-cleanup of resources is not possible. > > unlink() and the last munmap()/exit() will get rid of it... yep - and this isnt possible with traditional SysV shared memory, and isnt possible with traditional SysV semaphores. (forget my filesystem comment, it's a thinko.) Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Ingo Molnar wrote: > > On Sat, 2 Sep 2000, Alexander Viro wrote: > > > Why? I would say that bad thing about SysV shared memory is that it's > > _not_ sufficiently filesystem-thing - a special API where 'create a > > file on ramfs and bloody mmap() it' would be sufficient. Why bother > > with special sets of syscalls? > > what i mean is that i dont like the cleanup issues associated with SysV > shared memory - eg. it can hang around even if all users have exited, so > auto-cleanup of resources is not possible. unlink() and the last munmap()/exit() will get rid of it... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Alexander Viro wrote: > Why? I would say that bad thing about SysV shared memory is that it's > _not_ sufficiently filesystem-thing - a special API where 'create a > file on ramfs and bloody mmap() it' would be sufficient. Why bother > with special sets of syscalls? what i mean is that i dont like the cleanup issues associated with SysV shared memory - eg. it can hang around even if all users have exited, so auto-cleanup of resources is not possible. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
[EMAIL PROTECTED] (dean gaudet) wrote on 02.09.00 in <[EMAIL PROTECTED]>: > although you know the most amusing thing to watch a thread programmer try > to do is to pass data bidirectionally between two sockets. I'm tempted to say "don't do that, then". I.e., sounds like the wrong abstraction. Of course, you may have no choice. Easy threads rule-of-thumb: if you need to juggle two concurrent jobs in one thread, you have the wrong abstraction; maybe you need to split your thread. > can we be friends and say "everything in moderation including moderation > itself"? :) That's always wise. MfG Kai - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant [semi-OT]
[EMAIL PROTECTED] (Dan Maas) wrote on 02.09.00 in <020d01c014a9$f48d3fa0$0701a8c0@morph>: > When people think of "multithreading," often they are just looking for a way > to extract more concurrency from their machine. You want all these > independent parts to be working on your task simultaneously. There are many > different mechanisms for achieveing this. Here we go... [...] > In summary, when "multithreading" floats into your mind, think > "concurrency." Think very carefully about how you might simultaneously Let me see if I get this right. Some people say "threading" when they mean "performance", so we all should think "performance" when we hear "threading"? Why am I not impressed by that logic? MfG Kai - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
[EMAIL PROTECTED] (Alexander Viro) wrote on 02.09.00 in <[EMAIL PROTECTED]>: > *threads* *are* *hard* *to* *write* *correctly* Not really. The real problem is that threads - just like pointers, incidentally - react *far* worse to getting it wrong than most abstractions do, that is, threads (and pointers) are often hard to debug. And like all abstractions, they don't do you much good unless you have a sane way of describing your problem in the terms of this abstraction. Once you *do* have that, however, threads can be a really nice fit for your problem. Just don't try to force-fit them on something different. All of this, of course, is looking from the programming-abstraction point of view. It's much harder to get right from the performance-sucks point of view. (Fortunately, getting it right the first way can lead to getting it right the second way.) Essentially, it usually comes down to the same thing: get a good high- level description of your problem. Without it, you're fucked. *That* is where the most energy should be spent. Then, code it, and debug it. > Average programmer will fuck up and miss tons of race conditions. Better s/Average/Inexperienced/ or you have a serious documentation problem. Which happens. of course. > than average programmers do. If you've got shared resource - you are in > for problems. Threads are useful. But they take more efforts and are much > harder to debug _and_ to prove correctness. In other words, that's one of Often easier than pointers, IME. Of course, in both cases using some conventions can help bigtime. > the last resort tools, not the first one. If you can do it without shared In my current job project, structure went state machines -> coroutines -> threads. Both conversions eliminated buttloads of bugs. I wish I had started out with threads in the first place (it wasn't possible in the circumstances). That would have solved *so* much headache. And there was the equivalent of race conditions even back in the state machine version. Except they were *much* harder to solve there. Oh, and the code still has state machines - *where they make sense*. Something with several tens of KB of internal state and complex logic isn't where they make sense. (Oh, and as to "one thread per connection doesn't scale" argument: thread overhead say 8KB, connection local data 50-200KB - if the *thread* overhead kills you, you were dead anyway.) > state - don't bother with threads. Same as with rewrite in assmebler - do I've never seen a problem solved with threads that *didn't* have shared state. Ok, some only had shared state in the file system, so I know what you mean :-) In any case, the canonical example of a program that absolutely *MUST* have threads is an OS kernel. You can crunch it down to a micro kernel, but that sucks in other ways, and your servers will probably need their own threading anyway. > it with small critical parts and don't unless you absolutely have to. The Right; the assembler problem is vastly different, though: here, you're *giving up* abstractions. Don't do that unless you have no choice. > first rule of optimisation: don't. Keep the critical sections small and if > you can avoid them - it's worth the efforts. You'll win a lot when it will That's why the threaded version of this project often uses producer- consumer message queues for internal communication. Much easier to get one queue right than three dozen shared variables, all subtly different. Another case of "chose the right abstraction". > come to changing the thing. And you will have to change it, sooner or > later. Same goes for debugging. Non-threaded code is easier to understand. Not for inherently parallel stuff it's not. Definitely not. Complex state machines are *hard* to understand. > Yes, you may be very clever. But you'll have to debug it on Friday evening > after a hard week when you want only one thing - go home and sleep. Or And that was often when I said "I wish I had threads instead of these bloody state machines". Now I have, and boy has it helped a lot. > somebody else will and he will curse you. KISS. And threads are _not_ > simple. In this you're wrong. MfG Kai - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
Alexander Viro <[EMAIL PROTECTED]> writes: > On Sat, 2 Sep 2000, Ingo Molnar wrote: > > > well, Linux SysV shared memory indeed has a 'software version' of > > pagetables, this way if one process faults in a new page (because > > all pages are unmapped originally), then the new physical page > > address can be discovered by all other subsequent faults in other > > process contexts. It works just fine - the thing i dislike about > > SysV shared memory is not the VM part but its 'filesystem > > characteristics' - i think anonymous shared memory is the way to > > go. > > Why? I would say that bad thing about SysV shared memory is that > it's _not_ sufficiently filesystem-thing - a special API where > 'create a file on ramfs and bloody mmap() it' would be > sufficient. Why bother with special sets of syscalls? Yes, and that's what I plan for 2.5: Redoing the shm filesystem part like (or in?) ramfs and better separation of both the VM part and the SYSV API part. I always objected against doing anonymous shared mmap over SYSV shared mem because I wanted to have this handling in the VM layer and _not_ in some horrible^wspecial API layer. So my goal would be: 1) anonymous shared mappings should be handled by the vm layer (with better integration into the pagecache) 2) the filesystem frontend should be redone to make more use of the VFS. 3) SYSV shm should only be a special API using this functionality Comments? Christoph - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
Date:Sat, 2 Sep 2000 13:08:41 +0200 (CEST) From: Ingo Molnar <[EMAIL PROTECTED]> i think anonymous shared memory is the way to go. We have it now in 2.4.x, MAP_SHARED | MAP_ANONYMOUS actually works these days. :-) Or were you talking about creating some other kind of "new semaphores"? Later, David S. Miller [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Jamie Lokier wrote: > dean gaudet wrote: > > an example of brokenness in the traditional fd API is close-on-exec -- > > there's a race between open()/socket()/pipe() and fcntl(FD_CLOEXEC) during > > which if another thread does a fork() it's possible the child will inherit > > an fd it shouldn't... working around it is painful. the model which Really? Like, say it, close() before exec()? > > NT/OS2 use for creating a new process scales better in the 99.99% case of > > stdin/out/err -- you only specify those fds you want to keep in the new > > process. > > An obvious solution presents itself. O_CLOEXEC. Even more obvious solution: close what you need to close if you have sensitive descriptors around. Close-on-exec is a kludge. If you have sensitive pieces of descriptor table you want to do some other things too - e.g. make sure that it gets unshared before exec(). Because new descriptors of that kind are very likely to follow... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Ingo Molnar wrote: > well, Linux SysV shared memory indeed has a 'software version' of > pagetables, this way if one process faults in a new page (because all > pages are unmapped originally), then the new physical page address can be > discovered by all other subsequent faults in other process contexts. It > works just fine - the thing i dislike about SysV shared memory is not the > VM part but its 'filesystem characteristics' - i think anonymous shared > memory is the way to go. Why? I would say that bad thing about SysV shared memory is that it's _not_ sufficiently filesystem-thing - a special API where 'create a file on ramfs and bloody mmap() it' would be sufficient. Why bother with special sets of syscalls? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
dean gaudet wrote: > an example of brokenness in the traditional fd API is close-on-exec -- > there's a race between open()/socket()/pipe() and fcntl(FD_CLOEXEC) during > which if another thread does a fork() it's possible the child will inherit > an fd it shouldn't... working around it is painful. the model which > NT/OS2 use for creating a new process scales better in the 99.99% case of > stdin/out/err -- you only specify those fds you want to keep in the new > process. An obvious solution presents itself. O_CLOEXEC. Glibc could easily be made to call fcntl(FD_CLOEXEC) for kernels that don't implement O_CLOEXEC, so that applications can simply use the flag and rely on it. -- Jamie - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Ingo Molnar wrote: > i dont understand why this is such an important category. If the sharing > is very high between the threads then it makes sense to use 'shared-all > threads'. But frequently the example given are webservers, which often do > not have alot of cross-request shared state. web *applications* are loaded with cross-request state. there's only so much you can stuff in a cookie, and what application servers typically do is stuff only a session id into the cookie and keep the rest of the application state in memory or on disk. a good example would be a web-based email gateway to an IMAP mail-store. (i.e. think about how to write yahoo mail or hotmail) web content is easy compared to web applications... > > file descriptors -- yeesh these are hard, you want some sharing and > > some not sharing. [...] > > well (in Linux) you can specify it on a per-filedescriptor level wether to > share or not to share, and you can pass a filedescriptor to another > process and you can establish it there. Is there any API missing in this > area? so even if CLONE_FILES is set i can specify i don't want files to be shared? how does that work? an example of brokenness in the traditional fd API is close-on-exec -- there's a race between open()/socket()/pipe() and fcntl(FD_CLOEXEC) during which if another thread does a fork() it's possible the child will inherit an fd it shouldn't... working around it is painful. the model which NT/OS2 use for creating a new process scales better in the 99.99% case of stdin/out/err -- you only specify those fds you want to keep in the new process. i know you've done a kick-ass job of making fd allocation not collide a hell of a lot, but it's another synchronization that's unnecessary really. > > other than TLB/page-table changes is there anything else i'm missing > > which makes SMP and threading "slow"? > > it's not slow, it's 'slower' in the 'common memory allocation' case. yeah malloc generally sucks because it puts synchronization in when and the avg programmer doesn't realise it. http://www.hoard.org/ is pretty interesting research in this area though. -dean - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
> This is all just curiosity. I've considered trying some thread > programming, but if it is as stupid as it sounds, I'd rather > learn the "right" way of writing code that would ordinarily be > done with threads, etc.. Right now, I'm using fork() all over > the place and don't much care how much waste it is... I'd like > to though. The single thing people forget most is that threads are expensive. Sure use one for rendering one for computation, but dont use one per socket on a web server. If you think of threading as fork() that shares more then you'll probably be making sensible use of threading. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, dean gaudet wrote:
> i don't understand why another semaphore type is needed -- about the
> only semaphore missing is the pshared portion of posix. (recently i
> ported a large commercial MTA to linux and the only thing missing was
> the pthread_mutexattr_{s,g}etpshare stuff.)
i'd like to have waitqueue based (kernel-space) semaphores such as the
SysV semaphores are, not signal or sched_yield() based ones.
> regarding using shared memory/multiprocess -- how do you handle the
> case where the shared region needs to grow?
i dont understand why this is such an important category. If the sharing
is very high between the threads then it makes sense to use 'shared-all
threads'. But frequently the example given are webservers, which often do
not have alot of cross-request shared state.
> i actually think there's far more grossness to trying to figure out
> when to grow a shared mem region -- you almost need to check every
> time you follow a pointer. and hopefully you can arrange to have the
> same mapping in all processes or else you need some sort of extra page
> table... almost forced to implement paging in software.
well, Linux SysV shared memory indeed has a 'software version' of
pagetables, this way if one process faults in a new page (because all
pages are unmapped originally), then the new physical page address can be
discovered by all other subsequent faults in other process contexts. It
works just fine - the thing i dislike about SysV shared memory is not the
VM part but its 'filesystem characteristics' - i think anonymous shared
memory is the way to go.
> file descriptors -- yeesh these are hard, you want some sharing and
> some not sharing. [...]
well (in Linux) you can specify it on a per-filedescriptor level wether to
share or not to share, and you can pass a filedescriptor to another
process and you can establish it there. Is there any API missing in this
area?
> other than TLB/page-table changes is there anything else i'm missing
> which makes SMP and threading "slow"?
it's not slow, it's 'slower' in the 'common memory allocation' case.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Alexander Viro wrote: > *threads* *are* *hard* *to* *write* *correctly* in C, yes that can be essentially true. although you can do a webserver with no mutexes pretty easily... the first threaded apache had exactly one mutex -- inside the memory allocator when it had to go to a list of free pages (which is the slow path -- apache's fast allocation path needs no mutexes). otherwise the threads successfully ran without any need to co-ordinate with each other. the kernel provided all the co-ordination necessary through accept(). i should have chosen my phrasing a bit better too -- when i was talking about the spectrum between fully threaded and fully state driven, and i said that "threads are easier to program" i was actually thinking more along the lines of "call-return stacks make programming a lot easier"... although that too is totally related to the languages we generally choose to code in these days. my biggest beef with state-machine coding is that there's essentially no call-return stack. although you know the most amusing thing to watch a thread programmer try to do is to pass data bidirectionally between two sockets. can we be friends and say "everything in moderation including moderation itself"? :) -dean p.s. you lectured me on optimising only that which needs optimising... that is exactly what i was suggesting. the only time apache, for example, needs a state machine is to handle the low bandwidth clients/long downloads. writing the whole server as a state machine is an essentially impossible task... if you don't believe me then i suggest trying it. don't forget to make your server extensible, and plan for perl, php, and java application services. that's the way people program the web, no amount of ranting on linux-kernel is going to change it. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, Ingo Molnar wrote:
>
> On Fri, 1 Sep 2000, dean gaudet wrote:
>
> > yup you can do this without threads. apache-1.3+mod_ssl for example.
> >
> > but it's not fun, and it's a lot more work on the portability side.
> > inter-process shared memory and inter-process semaphores are
> > notoriously different across platforms... [...]
>
> but it's slower (on SMP) and more error-prone to use threads. We have the
> following task: 'use multiple contexts and share some state'. The 'share
> everything' model overdoes this goal a bit. Why not share the *specific*
> state that should be shared, and nothing more? The fact that this is
> inconsistent across platforms is IMHO not a good excuse - it just shows
> the mistaken concept that threads should be used for such tasks. You are
> right about the semaphore part - i have plans to add a new type of
> semaphore to Linux, which are not as complex as SysV semaphores and are
> not persistent.
i don't understand why another semaphore type is needed -- about the only
semaphore missing is the pshared portion of posix. (recently i ported a
large commercial MTA to linux and the only thing missing was the
pthread_mutexattr_{s,g}etpshare stuff.)
regarding using shared memory/multiprocess -- how do you handle the case
where the shared region needs to grow?
you're the expert here, tell me what i'm missing:
a threaded program growing its memory requires one cross-cpu TLB update in
case other threads are currently running; this requires n-1 messages where
n is the number of CPUs.
a shared memory/multiprocess program needs some sort of note in the shared
mem to indicate another page should be mapped; and as each process
discovers this need for a new mapping it issues a syscall. this requires m
syscalls where m is the number of processes.
if i understand what you're advocating, m will be much greater than n...
does shared mem still work out faster?
i actually think there's far more grossness to trying to figure out when
to grow a shared mem region -- you almost need to check every time you
follow a pointer. and hopefully you can arrange to have the same mapping
in all processes or else you need some sort of extra page table... almost
forced to implement paging in software.
is this really what you mean?
i have no qualms about "sharing as little as possible". i totally agree
with it. i just think "sharing all of memory" is as close to as little as
you can get for some applications. signals, cwd/chroot,
uid/gid/capabilities, this is all crap i don't care about sharing. file
descriptors -- yeesh these are hard, you want some sharing and some not
sharing. (unix fds are one of the really gross places where you can see
that threading was not present in the original unix design.)
other than TLB/page-table changes is there anything else i'm missing which
makes SMP and threading "slow"?
-dean
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant [semi-OT]
On Sat, 2 Sep 2000, J. Dow wrote: > Dan, another thing to consider with multithreading is that it is a way > to avoid "convoy" effects if there is a nice priority mechanism for > first in first out messaging. [...] yep, this is a frequent problem at the level of the kernel. We fixed such a (longtime) performance bug just a couple of weeks ago in the TCP-accept logic. We are using FIFO in most places where multiple contexts are waiting. But is 'shared everything multithreading' the natural solution for this? Nope, that statement is misleading. 'Multi-contexting' is the solution. Those contexts can be 'shared-everything threads' or 'isolated processes' as well. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Fri, 1 Sep 2000, dean gaudet wrote: > yup you can do this without threads. apache-1.3+mod_ssl for example. > > but it's not fun, and it's a lot more work on the portability side. > inter-process shared memory and inter-process semaphores are > notoriously different across platforms... [...] but it's slower (on SMP) and more error-prone to use threads. We have the following task: 'use multiple contexts and share some state'. The 'share everything' model overdoes this goal a bit. Why not share the *specific* state that should be shared, and nothing more? The fact that this is inconsistent across platforms is IMHO not a good excuse - it just shows the mistaken concept that threads should be used for such tasks. You are right about the semaphore part - i have plans to add a new type of semaphore to Linux, which are not as complex as SysV semaphores and are not persistent. Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Sat, 2 Sep 2000, dean gaudet wrote: > the thread bashing is mostly bashing programs which do things such as a > thread per-connection. this is the most obvious, and easy way to use > threads, but it's not necessarily the best performance, and certainly > doesn't scale. (on the scalability side just ask yourself how much RAM is > consumed by stacks -- how many cache lines will that consume, and how many > TLB entries. it sucks pretty fast.) > state machines are hard. while people on linux-kernel may be hot shot > programmers who can do state machines in their sleep, this is definitely > not the case for the average programmer. > > fortunately fully threaded and fully state driven are two opposite ends of > a spectrum, and there are lots of useful compromises in between where > threads are used in a way that allows the average programmer to > maintain/extend a codebase; while also getting the scalability of state > machines. Lovely. You've completely sidestepped the main part: *threads* *are* *hard* *to* *write* *correctly* Average programmer will fuck up and miss tons of race conditions. Better than average programmers do. If you've got shared resource - you are in for problems. Threads are useful. But they take more efforts and are much harder to debug _and_ to prove correctness. In other words, that's one of the last resort tools, not the first one. If you can do it without shared state - don't bother with threads. Same as with rewrite in assmebler - do it with small critical parts and don't unless you absolutely have to. The first rule of optimisation: don't. Keep the critical sections small and if you can avoid them - it's worth the efforts. You'll win a lot when it will come to changing the thing. And you will have to change it, sooner or later. Same goes for debugging. Non-threaded code is easier to understand. Yes, you may be very clever. But you'll have to debug it on Friday evening after a hard week when you want only one thing - go home and sleep. Or somebody else will and he will curse you. KISS. And threads are _not_ simple. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Fri, 1 Sep 2000, Mike A. Harris wrote: > I've heard comments from Alan, and others in the past bashing > threads, and I can understand the "threads are for people who > can't write state machines" comments I've heard, but what other > ways are there of accomplishing the goals that threads solve in > an acceptable manner that gives good performance without coding > complexity? the thread bashing is mostly bashing programs which do things such as a thread per-connection. this is the most obvious, and easy way to use threads, but it's not necessarily the best performance, and certainly doesn't scale. (on the scalability side just ask yourself how much RAM is consumed by stacks -- how many cache lines will that consume, and how many TLB entries. it sucks pretty fast.) state machines are hard. while people on linux-kernel may be hot shot programmers who can do state machines in their sleep, this is definitely not the case for the average programmer. fortunately fully threaded and fully state driven are two opposite ends of a spectrum, and there are lots of useful compromises in between where threads are used in a way that allows the average programmer to maintain/extend a codebase; while also getting the scalability of state machines. this is where "worker thread pools" and message queues come in. for example, consider an IMAP server. the fully threaded implementation would have a thread per connection. but most connections sit idle most of the time. instead restructure it so that there is a thread per in-progress command; plus one thread doing a state machine for all the idle connections. this scales much better because you only have as many threads as active commands. the actual IMAP code itself can be written in the comfortable threading model rather than a state model. it's still not perfect -- large mail messages sent to slow clients chew a thread for an inordinate amount of time. an obvious next modification would be to let the state thread also handle the sending of messages. this can be done relatively cleanly. part of the apache-2.0 design was to allow for such a setup -- using a thread for the "thinking" part of each HTTP request, and once a response object is decided on, it is passed back to the state machine. (responses generally fit into the categories of disk file, pipe, or memory region.) but nobody has yet implemented beyond thread-per-connection. -dean - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant [semi-OT]
> In summary, when "multithreading" floats into your mind, think
> "concurrency." Think very carefully about how you might simultaneously
> exploit all of the independent resources in your computer. Due to the long
> and complex history of OS development, a different API is usually required
> to communicate with each device. (e.g. old-school UNIX has always handled
> non-blocking network I/O with select(), but non-blocking disk I/O is
rather
> new and must be done with AIO or threads; and don't even ask about
> asynchronous access to the graphics card =).
Dan, another thing to consider with multithreading is that it is a way to
avoid "convoy" effects if there is a nice priority mechanism for first in
first out messaging. Until NT crufted in its IO Completion model it was
highly prone to either starvation or convoy problems with certain problems.
If you fired off reads on several interfaces which all experienced about the
same problem the first on in the array searched by the multiple object
wait function was more likely to get serviced than the others. The initial
solution reordered the list each pass through the wait. IO Completion
actually added in first in first out messaging for all the messages
reaching that particular wait for io completion call. This made a VERY
measureable improvement in performance overall. You can run into
the same problems with psuedo-threading in state machines with
polling loops only it's a whole lot uglier. (Been there, done that, kicked
myself, bought the teeshirt in disgrace, and retreated to a proper
solution.)
(Heh, incidentally I was rather surprised by the time I hit NT tasking
to discover its array based message handling. I was used to the
FIFO message queueing on AmigaDOS that it enjoyed from day
zero. So I wrote as if I had that on NT I learned the hard way.)
{^_^}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant [semi-OT]
> All portability issues aside, if one is writing an application in > Linux that one would be tempted to make multithreaded for > whatever reason, what would be the better Linux way of doing > things? Let's go back to basics. Look inside your computer. See what's there: 1) one (or more) CPUs 2) some RAM 3) a PCI bus, containing: 4) -- a SCSI/IDE controller 5) -- a network card 6) -- a graphics card These are all the parts of your computer that are smart enough to accomplish some amount of work on their own. The SCSI or IDE controller can read data from disk without bothering any other components. The network card can send and receive packets fairly autonomously. Each CPU in an SMP system operates nearly independently. An ideal application could have all of these devices doing useful work at the same time. When people think of "multithreading," often they are just looking for a way to extract more concurrency from their machine. You want all these independent parts to be working on your task simultaneously. There are many different mechanisms for achieveing this. Here we go... A naively-written "server" program (eg a web server) might be coded like so: * Read configuration file - all other work stops while data is fetched from disk * Parse configuration file - all other work stops while CPU/RAM work on parsing the file * Wait for a network connection - all other work stops while waiting for incoming packets * Read request from client - all other work stops while waiting for incoming packets * Process request - all other work stops while CPU/RAM figure out what to do - all other work stops while disk fetches requested file * Write reply to client - all other work stops until final buffer transmitted I've phrased the descriptions to emphasize that only one resource is being used at once - the rest of the system sits twiddling its thumbs until the one device in question finishes its task. Can we do better? Yes, thanks to various programming techniques that allow us to keep more of the system busy. The most important bottleneck is probably the network - it makes no sense for our server to wait while a slow client takes its time acknowledging our packets. By using standard UNIX multiplexed I/O (select()/poll()), we can send buffers of data to the kernel just when space becomes available in the outgoing queue; we can also accept client requests piecemeal, as the individual packets flow in. And while we're waiting for packets from one client, we can be processing another client's request. The improved program performs better since it keeps the CPU and network busy at the same time. However, it will be more difficult to write, since we have to maintain the connection state manually, rather than implicitly on the call stack. So now the server handles many clients at once, and it gracefully handles slow clients. Can we do even better? Yes, let's look at the next bottleneck - disk I/O. If a client asks for a file that's not in memory, the whole server will come to a halt while it read()s the data in. But the SCSI/IDE controller is smart enough to handle this alone; why not let the CPU and network take care of other clients while the disk does its work? How do we go about doing this? Well, it's UNIX, right? We talk to disk files the same way we talk to network sockets, so let's just select()/poll() on the disk files too, and everything will be dandy... (Unfortunately we can't do that - the designers of UNIX made a huge mistake and decided against implementing non-blocking disk I/O as they had with network I/O. Big booboo. For that reason, it was impossible to do concurrent disk I/O until the POSIX Asynchronous I/O standard came along. So we go learn this whole bloated API, in the process finding out that we can no longer use select()/poll(), and must switch to POSIX RT signals - sigwaitinfo() - to control our server***). After the dust has settled, we can now keep the CPU, network card, and the disk busy all the time -- so our server is even faster. Notice that our program has been made heavily concurrent, and I haven't even used the word "thread" yet! Let's take it one step further. Packets and buffers are now coming in and out so quickly that the CPU is sweating just handling all the I/O. But say we have one or three more CPU's sitting there idle - how can we get them going, too? We need to run multiple request handlers at once. Conventional multithreading is *one* possible way to accomplish this; it's rather brute-force, since the threads share all their memory, sockets, etc. (and full VM sharing doesn't scale optimally, since interrupts must be sent to all the CPUs when the memory layout changes). Lots of UNIX servers run multiple *processes*- the "sub-servers" might not share anything, or they might file cache or request queue. If we were brave, we'd think carefully about what resources really should be shared between the sub-servers, and then implement it manually using
Re: thread rant
(Hm, I meant for a copy of this to go to the list, too. So here it is.)
Mike Harris comments:
> > I've heard comments from Alan, and others in the past bashing
> > threads, and I can understand the "threads are for people who
> > can't write state machines" comments I've heard, but what other
> > ways are there of accomplishing the goals that threads solve in
> > an acceptable manner that gives good performance without coding
> > complexity?
>
> Threads are a handy way to allow a prioritized state machine
> operations. State machines are nice and I use them and have
> used them to good effect. (The MX3000 SatCom modem data
> mode and fax mode are both state machines - as far as the
> Inmarsat spec vs the Facsimile spec allowed.) I also use
> multithreaded code. I use it when I want to switch from thread
> to thread based on input events and priority. I don't want to
> continue to run through a low priority "state" before I can service
> a high priority "state". Threads are the mechanism. People who
> make declarations such as you cite remind me of people who
> yank the gas pliers out of their back pockets to pound in 10
> penny nails. If you need a hammer and do not have one then
> any tool begins to look like a hammer.
>
> > This is all just curiosity. I've considered trying some thread
> > programming, but if it is as stupid as it sounds, I'd rather
> > learn the "right" way of writing code that would ordinarily be
> > done with threads, etc.. Right now, I'm using fork() all over
> > the place and don't much care how much waste it is... I'd like
> > to though.
>
> Think on what it is you want to do. State machines are REAL
> good when every state should run to completion before you
> run the next state. State machines are not good when your
> program has functions that must be run at a higher priority
> than the other's while the others must not block when the
> higher priority thread blocks. Use the two tools with some
> discrimination and you can get wonderful results.
>
> > >The fact that the system implements threads speaks enough about
> > >it's capabilities. ie, it's trying hard to suck less. So, from my POV,
> > >we're looking to make Linux suck more by effectively emulating systems
> > >that are trying to suck less.
> >
> > Makes sense... if you understand the details of why threads
> > suck. I understand there are some cache coherency issues, and
> > I've heard of other things as well, but is there an FAQ out there
> > saying "Why to not use threads?" that goes into good detail and
> > provides alternatives?
>
> Doesn't make sense to me, Mike. But then I use threads where
> threads are appropriate and state machines where state machines
> are appropriate and linear code where linear code is appropriate.
> Consider threads a tool. Learn how the tool works. Then select it
> and use it when the situation warrants it.
>
> > >But, I've never done anything worthwhile for Linux, so take this for
what
> > >it's worth, from an asshole.
> >
> > Works for me. ;o)
>
> Sometimes I remember I am a lady and that ladies don't reply to that
> level of comment. 'Sides, if I did reply it'd not be by fainting. He might
> discover a new apperature somewhere on his body that resembles
> the orifice referenced that he didn't plan on or have OEM.
>
> {^_-}
>
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
At 22:48 01/09/00, Michael Bacarella wrote: >Q: Why do we need threads? >A: Because on some operating systems, task switches are expensive. > >Q: So, threads are a hack to get around operating systems that suck? >A: Basically. urgh, i think you've missed the point. while threads /may/ be abused by many applications where fork() or decent-event-queue/state-machine would probably produce much better performance (i believe some of the java libraries are a perfect example of this), there are _MANY_ applications where threads work, work well, and are superior to the alternatives available (fork with shm segments). one such example is cpu compultationally-expensive work where some degree of working-set is required to be shared between processes. one could use fork(), run multiple instances, have them register to a shm segment and then implement some form of IPC between them. alternatively, you could create 'n' work threads where "n == NR_CPUs" with the working-set automatically available to all worker threads. for whatever synchronization is required, you don't have to write your IPC mechanism - mutexes come standard with things like pthreads. (of course, there is no excuse for bad programming or bad algorithms; mutex use should be kept to a minimum). perhaps you then need this application to then do bulk disk-i/o? one-thread-per-disk-spindle works nicely in this scenario too. threads are useful and powerful. perhaps the real problem with threads are that it is too easy to write bad code using them. caveat emptor. cheers, lincoln. -- Lincoln Dale Content Services Business Unit [EMAIL PROTECTED] cisco Systems, Inc. | | |||| +1 (408) 525-1274 bldg G, 170 West Tasman +61 (3) 9659-4294 << San Jose CA 95134..:||:..:||:.. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Fri, 1 Sep 2000, Michael Bacarella wrote: >Q: Why do we need threads? >A: Because on some operating systems, task switches are expensive. > >Q: So, threads are a hack to get around operating systems that suck? >A: Basically. Am I understanding correctly then that using fork() is faster/better? Or should one use a state machine? I'm just asking out of curiosity because thread programming is new to me. All portability issues aside, if one is writing an application in Linux that one would be tempted to make multithreaded for whatever reason, what would be the better Linux way of doing things? I've heard comments from Alan, and others in the past bashing threads, and I can understand the "threads are for people who can't write state machines" comments I've heard, but what other ways are there of accomplishing the goals that threads solve in an acceptable manner that gives good performance without coding complexity? This is all just curiosity. I've considered trying some thread programming, but if it is as stupid as it sounds, I'd rather learn the "right" way of writing code that would ordinarily be done with threads, etc.. Right now, I'm using fork() all over the place and don't much care how much waste it is... I'd like to though. >Q: So, why must Linux support threads? >A1: : | >A2: So other programs can be easily ported to Linux! > >That can already happen. It's not the *best* implementation. It's >not as fast as it can be. But it works. And that's all it should do. If >you're not happy, cope. > > "But threads on this system are faster than on Linux!" > >The fact that the system implements threads speaks enough about >it's capabilities. ie, it's trying hard to suck less. So, from my POV, >we're looking to make Linux suck more by effectively emulating systems >that are trying to suck less. Makes sense... if you understand the details of why threads suck. I understand there are some cache coherency issues, and I've heard of other things as well, but is there an FAQ out there saying "Why to not use threads?" that goes into good detail and provides alternatives? >But, I've never done anything worthwhile for Linux, so take this for what >it's worth, from an asshole. Works for me. ;o) -- Mike A. Harris Linux advocate Computer Consultant GNU advocate Capslock Consulting Open Source advocate Be up to date on nerd news and stuff that matters: http://slashdot.org - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: thread rant
On Fri, 1 Sep 2000, Michael Bacarella wrote: > Q: Why do we need threads? > A: Because on some operating systems, task switches are expensive. maybe this problem will help you understand threads better: design a webserver which supports SSL session keys. consider the performance of your webserver on single CPU and multiple CPU boxes. consider also the performance of your webserver on datasets which do not fit within RAM, hence requiring disk i/o. and finally, consider the effect of having one disk spindle versus multiple disk spindles. in this problem, the "SSL session key" requirement is there so that state information for previous connections has to be available to efficiently process future connections. (the ssl session key reduces the number of times the RSA key signature needs to be performed.) the SMP requirement is to indicate why single process select loop won't always cut it. (especially if you're doing SSL you'd like to grind crypto on all CPUs). the disk i/o, multiple spindle requirement is there so you consider what it takes to get a disk array with 50 disks in it going full-tilt. (nevermind stuffing the elevator on a single disk -- i'm just trying to make it really obvious how much parallelism may be available in a system.) yup you can do this without threads. apache-1.3+mod_ssl for example. but it's not fun, and it's a lot more work on the portability side. inter-process shared memory and inter-process semaphores are notoriously different across platforms... just look at the "mm" library from mod_ssl (which is a less puke-filled version of what's in apache-1.3/src/main/http_main.c). -dean - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
thread rant
Q: Why do we need threads? A: Because on some operating systems, task switches are expensive. Q: So, threads are a hack to get around operating systems that suck? A: Basically. Q: So, why must Linux support threads? A1: : | A2: So other programs can be easily ported to Linux! That can already happen. It's not the *best* implementation. It's not as fast as it can be. But it works. And that's all it should do. If you're not happy, cope. "But threads on this system are faster than on Linux!" The fact that the system implements threads speaks enough about it's capabilities. ie, it's trying hard to suck less. So, from my POV, we're looking to make Linux suck more by effectively emulating systems that are trying to suck less. But, I've never done anything worthwhile for Linux, so take this for what it's worth, from an asshole. -MB - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
