Re: Road to 80 cores ( Was Re: Grand Central Details)
On Wed, Jun 18, 2008 at 1:50 PM, Pierce T. Wetter III <[EMAIL PROTECTED]> wrote: > The problem is that creating a thread, destroying a thread, and switching > threads in and out have a LOT of overhead. The above thread is going to > spend most of its time waiting for I/O, but the OS has no way of knowing > that really. So the OS swaps in the thread, which looks at its mutex and > says "I have nothing to do", and then gets swapped out. I'm sorry but I have to take issue with this. Are you saying that the scheduler is not intelligent enough to keep the apps on a blocked queue until a packet comes in for that handle, at which point the process is awoken and sent a signal? > Trivia: Apache 1.x is even worse then this, it forks off a whole new > _process() per connection so you have even more overhead. That's one of the > reasons people tend to put caches in front of apache for performance > (webperfcache on OSX Server, squid on others). I'd say the fact that this model is still the default points towards its persistence as an engineering tradeoff. But overall I agree with you 100%. Threads are most useful when you have a major task (calculate pi) and a minor task (print digits as they're computed), and do not want or cannot have the minor task suffer for the major task. You still want to consume your entire timeslice with performing the single, heavy task. Web serving, on the other hand, is a bajillion tiny tasks, and better methods exist to accomplish that goal. --Kyle Sluder ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Road to 80 cores ( Was Re: Grand Central Details)
Topic/focus changed to please the moderator.
Too bad you can't avoid blocking at least occasionally with the
event-driven APIs, meaning you still have to use threads to avoid
it completely. And I fail to see what's so bad about having one
thread per socket. Is it because Threads Are Hard?
In that case, it is because threads are relatively expensive.
Every thread adds a bit of memory use -- not insignificant given
each individual thread's stack -- and scheduling overhead. In this
model, you'd expect that most threads will be blocked on I/O most of
the time, but you might also likely find that performance goes to
hell in a handbasket as soon as multiple sockets are lit up with
inbound data.
And, yes, threads are hard, though -- in this case -- that hardness
is a bit irrelevant in that the real challenge is how to get data
*out* of the thread dealing with network I/O and *into* the threads
dealing with data.
Bill and others have nailed it, but I'll expand and talk about what
I'd like to see in MacOSX to support middle-ground parallelism in Snow
Leopard ala SEDA or some other mechanism. I don't really want that
much, so who knows, maybe it will happen and Apple will provide an
NSOperationQueueGroup class. I'll use a simplified webserver as a model.
Let's start with some back of the envelope calculations.
3 GHz 64-bit processor running on a 1 GHz backplane = 64 gigabits/
second.
100 Megabit link to the internet at our colocation site, or .1
gigabits.
CPU load to keep the pipe full to the internet: .64%. Except I have
8 cores, so make that
.08%.
IntrAnet? Ok, 1 gigabit/second means .8% CPU load.
Except you never see that low of a CPU load. You never see that
because its much, much easier to write code like the following for a
webserver:
socket=openSocket(PORT_80);
listen(socket);
connection=connect(socket);
SpawnThread(socketThread(connection))
socketThread(connection)
{
while (moreData)
{
request=read(connection);
cachedFile=loadFileIntoCache(request);
send(cachedFile, connection);
}
disconnect(connection);
}
In fact, nearly every introductory programming book in the world
tells you to do it that way. Because the two sections are very linear,
its much easier to think "I need to do this, then I need to do that,
and I'll wrap it in a thread so I can be doing this and that in
parallel".
The problem is that creating a thread, destroying a thread, and
switching threads in and out have a LOT of overhead. The above thread
is going to spend most of its time waiting for I/O, but the OS has no
way of knowing that really. So the OS swaps in the thread, which looks
at its mutex and says "I have nothing to do", and then gets swapped out.
If you spawn a thread per connection, pretty soon your app grinds to
a halt and can't get anything done.
Trivia: Apache 1.x is even worse then this, it forks off a whole new
_process() per connection so you have even more overhead. That's one
of the reasons people tend to put caches in front of apache for
performance (webperfcache on OSX Server, squid on others).
Ironically, its the concept of threads themselves that created this
problem. If you look at pre-thread programming books they all told you
do write code like the following:
socketsWithData=select(opensockets)
for (socket : socketsWithData)
{
serviceSocket()
}
That is, the select call lets you pass in a whole bunch of open
sockets, and it then blocks until one of them has work to do. Remember
how we only need .08% CPU to keep up with the network pipe? Well you
can easily do that with one thread for 1000s of sockets.
The grandson of select is CFNetwork (Or the non-blocking I/O
classes in Java). What CFNetwork does is turn all of that into events,
so you can easily service all the sockets. But as Michael says, "Too
bad you can't avoid blocking at least occasionally". So what you can
then do then do to address that is split the processing of that data,
from the servicing of the sockets. If necessary, you can farm out the
processing of the data to a thread, or even better, NSOperationQueue.
So if we revist our "webserver", this means that if we factor in
CFNetwork, that turns the network code into the following:
// So first, all of the network bookkeeping gets handled by an
event model in the same
// thread.
socket=openSocket(PORT_80);
switch (event)
{
case listen:
listen(socket);
break;
case connect:
connection=connect(socket);
break;
case dataIncoming:
processData(data);
break;
case dataOutgoing:
sendMoreData(data);
break;
case disconnect:
disconnect(connection)
}
// here, we process the incoming data, a
Re: Grand Central Details
And again On Jun 17, 2008, at 8:47 PM, Pierce T. Wetter III wrote: On Jun 17, 2008, at 5:31 PM, David Wilson wrote: WWDC is still under NDA, I'm pretty sure no one's allowed to say anything here beyond whatever mentions have been made to the press. Well, Apple people are, once they obtain the appropriate permissions. Asking on the dev lists is often the best way to nudge them a bit, or sometimes someone says "Hint: check here and here on ADC". Though sometimes I wish Apple had an NDA-required list. Even if I had to send a notarized form somewhere, the whole NDA thing gets annoying sometimes. I read the press release, and it said "Grand Central is cool, it slices your bread for you". But how exactly does it do that? Discussing NDA Projects (Snow Leopard and iPhone OS) and Private API This list is not an appropriate forum for the discussion of issues that are covered by non-disclosure. This includes Snow Leopard and iPhone OS 2.0. Doing so will violate your NDA and the message will be forwarded to WWDR. The discussion of Private API is also not appropriate for this list. Using private API is strongly discouraged as it can (and often does) change in future software revisions. If you feel some private API should be made public contact WWDR directly or file a bug using bugreporter.apple.com. Please do not advocate for those changes here, it isn't effective. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
[MODERATOR] Re: Grand Central Details
On Jun 17, 2008, at 8:17 PM, Pierce T. Wetter III wrote: I had a release the week before, plus we didn't have enough tickets since WWDC was sold out, so I didn't go. Are there any details on Grand Central? Discussing NDA Projects (Snow Leopard and iPhone OS) and Private API This list is not an appropriate forum for the discussion of issues that are covered by non-disclosure. This includes Snow Leopard and iPhone OS 2.0. Doing so will violate your NDA and the message will be forwarded to WWDR. The discussion of Private API is also not appropriate for this list. Using private API is strongly discouraged as it can (and often does) change in future software revisions. If you feel some private API should be made public contact WWDR directly or file a bug using bugreporter.apple.com. Please do not advocate for those changes here, it isn't effective. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
Le 18 juin 08 à 03:40, Bill Bumgarner a écrit : On Jun 17, 2008, at 6:02 PM, Michael Gardner wrote: Too bad you can't avoid blocking at least occasionally with the event-driven APIs, meaning you still have to use threads to avoid it completely. I don't really see in what situation you have to block in the event driven model ? I did some server that are able to handle hundred of simultaneous connections, and process thousand of requests per seconds with only one I/O thread. smime.p7s Description: S/MIME cryptographic signature ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
On Jun 17, 2008, at 6:02 PM, Michael Gardner wrote: Too bad you can't avoid blocking at least occasionally with the event-driven APIs, meaning you still have to use threads to avoid it completely. And I fail to see what's so bad about having one thread per socket. Is it because Threads Are Hard? In that case, it is because threads are relatively expensive. Every thread adds a bit of memory use -- not insignificant given each individual thread's stack -- and scheduling overhead. In this model, you'd expect that most threads will be blocked on I/O most of the time, but you might also likely find that performance goes to hell in a handbasket as soon as multiple sockets are lit up with inbound data. And, yes, threads are hard, though -- in this case -- that hardness is a bit irrelevant in that the real challenge is how to get data *out* of the thread dealing with network I/O and *into* the threads dealing with data. b.bum smime.p7s Description: S/MIME cryptographic signature ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
Though sometimes I wish Apple had an NDA-required list. Even if I had to send a notarized form somewhere, the whole NDA thing gets annoying sometimes. Irrelevant. It's a legally binding agreement and this happens to be their list. Nobody likes it much (who would?) but, as has been mentioned on this list many times, trying to argue that point here is completely and utterly pointless. That's not a conceit; you're just preaching to the choir. ;-) I read the press release, and it said "Grand Central is cool, it slices your bread for you". But how exactly does it do that? If you're "in", you can ask DTS for now or wait for documentation to come out. It's presumably many months away from release, so the actual API can be considered to be "in flux", making documentation difficult (and therefore sparse) until it solidifies. In its current state, it's likely meant for you to experiment with via an example. A lack of forum is bemoaned several times a year almost as some sort of superstitious ceremony to scare away the evil spirits and end the apocalyptic drought. Dreary. If you're "out", well, I myself was not at WWDC this year and curiosity is eating me alive. I admit I read very closely to see if your message contained any juicy tidbits. ;-) -- I.S. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
On Jun 17, 2008, at 7:17 PM, Pierce T. Wetter III wrote: If you've ever used CoreNetwork you might get the idea, because CFNetwork is event-driven, rather then the shitty thread-per-socket style that CS students are taught. So you can easily handle a zillion connections with CoreNetwork. Too bad you can't avoid blocking at least occasionally with the event- driven APIs, meaning you still have to use threads to avoid it completely. And I fail to see what's so bad about having one thread per socket. Is it because Threads Are Hard? -Michael ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
On Jun 17, 2008, at 5:31 PM, David Wilson wrote: WWDC is still under NDA, I'm pretty sure no one's allowed to say anything here beyond whatever mentions have been made to the press. Well, Apple people are, once they obtain the appropriate permissions. Asking on the dev lists is often the best way to nudge them a bit, or sometimes someone says "Hint: check here and here on ADC". Though sometimes I wish Apple had an NDA-required list. Even if I had to send a notarized form somewhere, the whole NDA thing gets annoying sometimes. I read the press release, and it said "Grand Central is cool, it slices your bread for you". But how exactly does it do that? Pierce ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
On Jun 17, 2008, at 7:17 PM, Pierce T. Wetter III wrote: I had a release the week before, plus we didn't have enough tickets since WWDC was sold out, so I didn't go. Are there any details on Grand Central? First, Snow Leopard is under NDA; cannot discuss it here. If you've ever used CoreNetwork you might get the idea, because CFNetwork is event-driven, rather then the [snipped] thread-per- socket style that CS students are taught. Second, there's no need for profanity. ___ Ricky A. Sharp mailto:[EMAIL PROTECTED] Instant Interactive(tm) http://www.instantinteractive.com ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Grand Central Details
WWDC is still under NDA, I'm pretty sure no one's allowed to say anything here beyond whatever mentions have been made to the press. On Tue, Jun 17, 2008 at 8:17 PM, Pierce T. Wetter III <[EMAIL PROTECTED]> wrote: > > I had a release the week before, plus we didn't have enough tickets since > WWDC was sold out, so I didn't go. Are there any details on Grand Central? > > Things I would like it to be: > > 1. An implementation of SEDA. That stands for > staged-event-driven-architecture. The basic idea is that you break things up > into different styles of work queues. The OS then dynamically allocates > threads to the work queues and figures out the optimum number of threads for > each queue. For instance, an I/O queue might be quite happy with a single > thread servicing it, because mostly they dump their data, then wait. A queue > of tasks that did actual computation could use as many threads as there are > cores. > > If you've ever used CoreNetwork you might get the idea, because > CFNetwork is event-driven, rather then the shitty thread-per-socket style > that CS students are taught. So you can easily handle a zillion connections > with CoreNetwork. > > This could probably be built as an extension to NSOperation, but > ideally, you'd want simple wrappers for all the I/O operations in > Foundation. > > 2. A better multi-core aware scheduler, like ULE from FreeBSD. > > 3. Something unimaginably cool. > > Pierce > ___ > > Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) > > Please do not post admin requests or moderator comments to the list. > Contact the moderators at cocoa-dev-admins(at)lists.apple.com > > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/cocoa-dev/david.t.wilson%40gmail.com > > This email sent to [EMAIL PROTECTED] > -- - David T. Wilson [EMAIL PROTECTED] ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Grand Central Details
I had a release the week before, plus we didn't have enough tickets since WWDC was sold out, so I didn't go. Are there any details on Grand Central? Things I would like it to be: 1. An implementation of SEDA. That stands for staged-event- driven-architecture. The basic idea is that you break things up into different styles of work queues. The OS then dynamically allocates threads to the work queues and figures out the optimum number of threads for each queue. For instance, an I/O queue might be quite happy with a single thread servicing it, because mostly they dump their data, then wait. A queue of tasks that did actual computation could use as many threads as there are cores. If you've ever used CoreNetwork you might get the idea, because CFNetwork is event-driven, rather then the shitty thread-per-socket style that CS students are taught. So you can easily handle a zillion connections with CoreNetwork. This could probably be built as an extension to NSOperation, but ideally, you'd want simple wrappers for all the I/O operations in Foundation. 2. A better multi-core aware scheduler, like ULE from FreeBSD. 3. Something unimaginably cool. Pierce ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
