Re: R. Stevens select() Collisions Scenario
Alfred Perlstein wrote: Yes. :) When FreeBSD gets scheduler activations you'll be able to change to a single threaded process that will have excellent performance, the scheduler activations are just around the corner. Can you explain what scheduler activations are? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: R. Stevens select() Collisions Scenario
* Jonas Bulow [EMAIL PROTECTED] [000811 10:45] wrote: Alfred Perlstein wrote: Yes. :) When FreeBSD gets scheduler activations you'll be able to change to a single threaded process that will have excellent performance, the scheduler activations are just around the corner. Can you explain what scheduler activations are? Please CC. schedulder activations are like 'LWP on-demand' a really simplified explanation would be shared address space fork that only happens when you block in certain parts of the kernel, like during disk/io. the problem with normal LWP is that if you're going to do any sort of disk IO you really want an LWP _per_ disk bound thread otherwise you risk blocking on disk IO inside the kernel, scheduler activations make sure you almost never block as well as making it unnecessary to 'pre-allocate' LWP contexts to avoid such problems. -- -Alfred Perlstein - [[EMAIL PROTECTED]|[EMAIL PROTECTED]] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
R. Stevens select() Collisions Scenario
Purpose:
I am trying to write a non-blocking, preforked server, specifically to
run on FreeBSD, and have a general question as to whether or not my
strategy is sound.
Problem:
In Unix Network Programming Vol. 1 (section 27.6, p.741), Stevens
mentions a scenario under a preforked server design where multiple
children are calling select() on the same descriptor, causing collisions
for the kernel to resolve. The children would look something like...
.
.
.
for( ; ; ){
FD_SET(listfd,rset);
Select(listfd+1,rset,NULL,NULL,NULL);
if(FD_ISSET(listfd,rset)==0)
err_quit("listenfd readable");
clilen = addrlen;
connfd = Accept(listenfd, cliaddr, clilen);
process(connfd);
Close(connfd);
}
.
.
.
Hypothesis:
If you were implement the server such that the children used
non-blocking I/O, so that select() returned immediately, would this
alleviate the problem of collisions in the kernel and its associated
overhead?
If so, would you then have to worry about mutual exclusion when select
did return a ready descriptor for accept(). For example, a child gets
switched just after calling select() but just before it makes it to
accept(). Thus, the next child would also receive the same descriptor
ready for accept()? Now only one of the two children will get to
accept() first, leaving the other blocking on accept(). Stevens mentions
a similar case at the end of the Nonblocking I/O chapter (section 15.6,
p. 422) and to avoid this he recommends 1) setting the listening
descriptor to non-blocking and 2) ignoring EWOULDBLOCK, ECONNABORTED,
etc. on accept().
So, if you make the listening descriptor non-blocking, and treat
select() and accept() appropriately, you should be alright in both
avoiding select collisions and there overhead, as well as avoid children
blocking due to losing possible race conditions for the listen
descriptor in the event (albiet small) of a context switch between
select() and accept().
Apology:
The reason I bother you with all this is that while I (think I)
understand the logic, I (know I) am at the limits of my understanding,
and might be missing some important considerations as to other goings on
in the kernel, and that there might be a better way to go about all
this.
Summary:
Ultimately, all I am seeking to do is have an efficient and scalable
server design, and to my knowledge, this would consist of a number of
preforked children who are non-blocking/multiplexing themselves.
Furthermore, each child will have a pool of worker threads which will
handle jobs sent by different clients. As long as the threads' work
doesn't entail an operation that blocks for appreciable amount of time
(so that a single thread puts the whole child to sleep), it would seem
like this might be a decent proposal: multiple clients and multiple jobs
being handled simultaneously by each child, and the number of children
can be controlled by the parent according to the load and limits of the
hardware.
Does this seem like a good way to go about it?
Thanks:
Thanks.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: R. Stevens select() Collisions Scenario
* Michael Owens [EMAIL PROTECTED] [000810 17:51] wrote:
Purpose:
I am trying to write a non-blocking, preforked server, specifically to
run on FreeBSD, and have a general question as to whether or not my
strategy is sound.
Problem:
In Unix Network Programming Vol. 1 (section 27.6, p.741), Stevens
mentions a scenario under a preforked server design where multiple
children are calling select() on the same descriptor, causing collisions
for the kernel to resolve. The children would look something like...
.
.
.
for( ; ; ){
FD_SET(listfd,rset);
Select(listfd+1,rset,NULL,NULL,NULL);
if(FD_ISSET(listfd,rset)==0)
err_quit("listenfd readable");
clilen = addrlen;
connfd = Accept(listenfd, cliaddr, clilen);
process(connfd);
Close(connfd);
}
.
.
.
Hypothesis:
If you were implement the server such that the children used
non-blocking I/O, so that select() returned immediately, would this
alleviate the problem of collisions in the kernel and its associated
overhead?
Select ignores non-blocking IO, if you have a non-blocking socket and
perform a select on it, you _will_ block (unless you use a zero valued
timeout).
If so, would you then have to worry about mutual exclusion when select
did return a ready descriptor for accept(). For example, a child gets
switched just after calling select() but just before it makes it to
accept(). Thus, the next child would also receive the same descriptor
ready for accept()? Now only one of the two children will get to
accept() first, leaving the other blocking on accept(). Stevens mentions
a similar case at the end of the Nonblocking I/O chapter (section 15.6,
p. 422) and to avoid this he recommends 1) setting the listening
descriptor to non-blocking and 2) ignoring EWOULDBLOCK, ECONNABORTED,
etc. on accept().
Yes, that is correct and the correct thing to do.
So, if you make the listening descriptor non-blocking, and treat
select() and accept() appropriately, you should be alright in both
avoiding select collisions and there overhead, as well as avoid children
blocking due to losing possible race conditions for the listen
descriptor in the event (albiet small) of a context switch between
select() and accept().
Another way of doing this would be to have a 'master' accept process
that uses fd passing (it's in your book) to hand off connections to
children, this would avoid select collisions at the cost of an additional
context switch.
Apology:
The reason I bother you with all this is that while I (think I)
understand the logic, I (know I) am at the limits of my understanding,
and might be missing some important considerations as to other goings on
in the kernel, and that there might be a better way to go about all
this.
Well for one thing I wouldn't be using select, I would use FreeBSD's
kqueue mechanism, it's vastly superior to select and poll, although
given the choice of select vs poll I would use poll().
Summary:
Ultimately, all I am seeking to do is have an efficient and scalable
server design, and to my knowledge, this would consist of a number of
preforked children who are non-blocking/multiplexing themselves.
Furthermore, each child will have a pool of worker threads which will
handle jobs sent by different clients. As long as the threads' work
doesn't entail an operation that blocks for appreciable amount of time
(so that a single thread puts the whole child to sleep), it would seem
like this might be a decent proposal: multiple clients and multiple jobs
being handled simultaneously by each child, and the number of children
can be controlled by the parent according to the load and limits of the
hardware.
Does this seem like a good way to go about it?
Yes. :) When FreeBSD gets scheduler activations you'll be able to change
to a single threaded process that will have excellent performance, the
scheduler activations are just around the corner.
Thanks:
you're welcome.
Thanks.
too much coffee today? :)
-Alfred
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
