Re: poll() vs select()
In article [EMAIL PROTECTED], Archie Cobbs [EMAIL PROTECTED] wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. Yes! Yes! Yes! (I agree.) John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA "No matter how cynical I get, I just can't keep up."-- Nora Ephron To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
In article [EMAIL PROTECTED], Brian F. Feldman [EMAIL PROTECTED] wrote: On Sun, 4 Jul 1999, Archie Cobbs wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. I.e., signal delivery, child exit notification, maybe even support for an arbitrary number of (independent) timers. And make the events independent from each other -- to avoid problems like when an application completely hangs for 90 seconds when it calls gethostbyname(). An async thread to do hostname lookups would be great! Wouldn't be too hard in libc_r, would it? The application itself has to get involved if it wants to do async name lookups, or async anything else, for that matter. Suppose you do have an async thread to do hostname lookups as you propose. What is the application going to do while that thread is waiting for the lookup to complete? It depends on the application, and thus it has to be coded into the application. Maybe there's nothing useful the application could do until the lookup returns. I've been told that it works fine to use libc_r and put the name lookups into a separate thread. But to take advantage of it, the application has to have something useful it wants to do (and can do) in the meantime. John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA "No matter how cynical I get, I just can't keep up."-- Nora Ephron To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Tue, 6 Jul 1999, John Polstra wrote: In article [EMAIL PROTECTED], The application itself has to get involved if it wants to do async name lookups, or async anything else, for that matter. Suppose you do have an async thread to do hostname lookups as you propose. What is the application going to do while that thread is waiting for the lookup to complete? It depends on the application, and thus it has to be coded into the application. Maybe there's nothing useful the application could do until the lookup returns. I've been told that it works fine to use libc_r and put the name lookups into a separate thread. But to take advantage of it, the application has to have something useful it wants to do (and can do) in the meantime. It would let the other threads run more while the lookup is occurring. Wouldn't that be the most natural expectation of it? Or would this be too hard without kernel-assisted threading? John -- John Polstra [EMAIL PROTECTED] John D. Polstra Co., Inc.Seattle, Washington USA "No matter how cynical I get, I just can't keep up."-- Nora Ephron To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
In article 199907050103.saa51...@bubba.whistle.com, Archie Cobbs arc...@whistle.com wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. Yes! Yes! Yes! (I agree.) John -- John Polstra j...@polstra.com John D. Polstra Co., Inc.Seattle, Washington USA No matter how cynical I get, I just can't keep up.-- Nora Ephron To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
In article pine.bsf.4.10.9907042155090.66085-100...@janus.syracuse.net, Brian F. Feldman gr...@freebsd.org wrote: On Sun, 4 Jul 1999, Archie Cobbs wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. I.e., signal delivery, child exit notification, maybe even support for an arbitrary number of (independent) timers. And make the events independent from each other -- to avoid problems like when an application completely hangs for 90 seconds when it calls gethostbyname(). An async thread to do hostname lookups would be great! Wouldn't be too hard in libc_r, would it? The application itself has to get involved if it wants to do async name lookups, or async anything else, for that matter. Suppose you do have an async thread to do hostname lookups as you propose. What is the application going to do while that thread is waiting for the lookup to complete? It depends on the application, and thus it has to be coded into the application. Maybe there's nothing useful the application could do until the lookup returns. I've been told that it works fine to use libc_r and put the name lookups into a separate thread. But to take advantage of it, the application has to have something useful it wants to do (and can do) in the meantime. John -- John Polstra j...@polstra.com John D. Polstra Co., Inc.Seattle, Washington USA No matter how cynical I get, I just can't keep up.-- Nora Ephron To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
In article local.mail.freebsd-hackers/199907061636.jaa15...@vashon.polstra.com you write: In article 199907050103.saa51...@bubba.whistle.com, Archie Cobbs arc...@whistle.com wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. Yes! Yes! Yes! (I agree.) There was another message in this thread indicating that this has the potential to grow creeping featuritis. :-) I've been thinking this over, and unlimited timers should be easy to do. I suppose you could fold signal delivery into an event notification mechanism as well. What other events are on the various wishlists out there? -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Tue, 6 Jul 1999, John Polstra wrote: In article pine.bsf.4.10.9907042155090.66085-100...@janus.syracuse.net, The application itself has to get involved if it wants to do async name lookups, or async anything else, for that matter. Suppose you do have an async thread to do hostname lookups as you propose. What is the application going to do while that thread is waiting for the lookup to complete? It depends on the application, and thus it has to be coded into the application. Maybe there's nothing useful the application could do until the lookup returns. I've been told that it works fine to use libc_r and put the name lookups into a separate thread. But to take advantage of it, the application has to have something useful it wants to do (and can do) in the meantime. It would let the other threads run more while the lookup is occurring. Wouldn't that be the most natural expectation of it? Or would this be too hard without kernel-assisted threading? John -- John Polstra j...@polstra.com John D. Polstra Co., Inc.Seattle, Washington USA No matter how cynical I get, I just can't keep up.-- Nora Ephron To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ gr...@freebsd.org _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
From: John Polstra j...@polstra.com Date: 1999-07-06 09:36:51 -0700 To: arc...@whistle.com Subject: Re: poll() vs select() Cc: hack...@freebsd.org In-reply-to: 199907050103.saa51...@bubba.whistle.com Delivered-to: freebsd-hackers@freebsd.org X-Loop: FreeBSD.ORG Organization: Polstra Co., Seattle, WA In article 199907050103.saa51...@bubba.whistle.com, Archie Cobbs arc...@whistle.com wrote: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. Yes! Yes! Yes! (I agree.) To add to the confusion, we've implemented something very similar for Mac OS X Server, designed to be a replacement for select(), and for use in a similar way to select's use in existing code. You can see it in the Darwin code. Check out sys/ev.h for a typically terse description of the API. In the released Darwin source, the code deals only with sockets. The Darwin release that deals with Mac OS X (possibly by fall) should have a more involved design that deals with a number of different kinds of events, including Mach messages (not a big deal for this group, of course). See www.publicsource.apple.com and follow the links to Darwin OS. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Manager, CoreOS Networking| Men are from Earth. Apple Computer, Inc. | Women are from Earth. 2 Infinite Loop | Deal with it. Cupertino, CA 95014 | *-*---* To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
Brian F. Feldman wrote: On Tue, 6 Jul 1999, John Polstra wrote: In article pine.bsf.4.10.9907042155090.66085-100...@janus.syracuse.net, The application itself has to get involved if it wants to do async name lookups, or async anything else, for that matter. Suppose you do have an async thread to do hostname lookups as you propose. What is the application going to do while that thread is waiting for the lookup to complete? It depends on the application, and thus it has to be coded into the application. Maybe there's nothing useful the application could do until the lookup returns. I've been told that it works fine to use libc_r and put the name lookups into a separate thread. But to take advantage of it, the application has to have something useful it wants to do (and can do) in the meantime. It would let the other threads run more while the lookup is occurring. Wouldn't that be the most natural expectation of it? Or would this be too hard without kernel-assisted threading? What I'm saying is, we already have that in multi-threaded applications. The system can't just provide it automatically to single-threaded applications; they wouldn't know how to take advantage of it. In other words: * Multi-threaded applications already have it. * Single-threaded applications can't use it. John --- John Polstra j...@polstra.com John D. Polstra Co., Inc.Seattle, Washington USA No matter how cynical I get, I just can't keep up.-- Nora Ephron To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Peter Wemm wrote: Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? yes, its been done, and its called posix real time signal queues :) check out http://www.redhat.com/~zab/phhttpd it basically registers sigio on the fds it cares about, but the signal it registers is a real time signal ( 32) so the kernel queues them in a fifo rather than usually delivering them. for each signal in the queue there is an attached siginfo struct that has things like which fd the event is for, which POLL_ event caused the signal, etc. phhttpd is a quick silly static web server that masks the signal and has threads spinning on sigwaitinfo() popping events off the queue. phhtpd isn't going anywhere, but it's event model is planned to go into apache's mpm architecture once dean has it solid, and I'm debating writing up a super-mega network server based around it.. If anyone's near ottawa in late july, head over to ols and we'll chat about this stuff over drinks ;) -- zach - - - - - - 007 373 5963 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
But poll() copies in HUGE amounts of data compared to the few bytes for thousands of FDs that select does. but the size of the select() mask is dependant on the highest numbered fd that we care about, rather than the number of fds we actually care about. this becomes highly uncool in a mondo threaded server that shares fd space :) -- zach - - - - - - 007 373 5963 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Peter Wemm wrote: "Brian F. Feldman" wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 [EMAIL PROTECTED] you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? You can do much the same thing by using aio functions. I do this now with an added syscall aio_waitcomplete, which allows a process to sleep waiting for the next aio operation to finish. If more work was done on the aio routines to improve their performance (the existing ones are better than select() when you exceed about 40 descriptors), they would be faster than poll or select, and could function in a similar fashion to the event queue scheme above. I've only had about 140 or so connections open in my experiments, but aio has no trouble with these, to the point where my switched 100Mb line is the bottleneck. I experiment more with NNTP than HTTP, though I have toyed with the idea of hacking an existing web server to use aio. I like the event queue idea, but I'd like it for aio completions rather than an enhanced select() function. All IMHO, -Chris To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Archie Cobbs wrote: Christopher Sedore writes: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. I.e., signal delivery, child exit notification, maybe even support for an arbitrary number of (independent) timers. And make the events independent from each other -- to avoid problems like when an application completely hangs for 90 seconds when it calls gethostbyname(). An async thread to do hostname lookups would be great! Wouldn't be too hard in libc_r, would it? But in regular apps, setitimer and sigsetjmp() would be a solution. -Archie ___ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com Brian Fundakowski Feldman _ __ ___ ___ ___ ___ [EMAIL PROTECTED] _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Peter Wemm wrote: Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? yes, its been done, and its called posix real time signal queues :) check out http://www.redhat.com/~zab/phhttpd it basically registers sigio on the fds it cares about, but the signal it registers is a real time signal ( 32) so the kernel queues them in a fifo rather than usually delivering them. for each signal in the queue there is an attached siginfo struct that has things like which fd the event is for, which POLL_ event caused the signal, etc. phhttpd is a quick silly static web server that masks the signal and has threads spinning on sigwaitinfo() popping events off the queue. phhtpd isn't going anywhere, but it's event model is planned to go into apache's mpm architecture once dean has it solid, and I'm debating writing up a super-mega network server based around it.. If anyone's near ottawa in late july, head over to ols and we'll chat about this stuff over drinks ;) -- zach - - - - - - 007 373 5963 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
But poll() copies in HUGE amounts of data compared to the few bytes for thousands of FDs that select does. but the size of the select() mask is dependant on the highest numbered fd that we care about, rather than the number of fds we actually care about. this becomes highly uncool in a mondo threaded server that shares fd space :) -- zach - - - - - - 007 373 5963 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Peter Wemm wrote: Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 0...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? You can do much the same thing by using aio functions. I do this now with an added syscall aio_waitcomplete, which allows a process to sleep waiting for the next aio operation to finish. If more work was done on the aio routines to improve their performance (the existing ones are better than select() when you exceed about 40 descriptors), they would be faster than poll or select, and could function in a similar fashion to the event queue scheme above. I've only had about 140 or so connections open in my experiments, but aio has no trouble with these, to the point where my switched 100Mb line is the bottleneck. I experiment more with NNTP than HTTP, though I have toyed with the idea of hacking an existing web server to use aio. I like the event queue idea, but I'd like it for aio completions rather than an enhanced select() function. All IMHO, -Chris To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
Christopher Sedore writes: Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? You can do much the same thing by using aio functions. I do this now with an added syscall aio_waitcomplete, which allows a process to sleep waiting for the next aio operation to finish. If more work was done on the aio routines to improve their performance (the existing ones are better than select() when you exceed about 40 descriptors), they would be faster than poll or select, and could function in a similar fashion to the event queue scheme above. I've only had about 140 or so connections open in my experiments, but aio has no trouble with these, to the point where my switched 100Mb line is the bottleneck. I experiment more with NNTP than HTTP, though I have toyed with the idea of hacking an existing web server to use aio. I like the event queue idea, but I'd like it for aio completions rather than an enhanced select() function. A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. I.e., signal delivery, child exit notification, maybe even support for an arbitrary number of (independent) timers. And make the events independent from each other -- to avoid problems like when an application completely hangs for 90 seconds when it calls gethostbyname(). rant I've seen a zillion programs that have had to manually solve the problems caused by UNIX's lack of a unified event notification system. Eg., ppp(8) has its own timer library, etc. See also bind's event library. What a shameful waste/duplication of effort.. Really the underlying problem is that UNIX (originally) did not have any lightweight threads and options for IPC were (and still are) limited. This is one of the reasons I prefer programming in Java, even if it's slower. /rant -Archie ___ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Archie Cobbs wrote: Christopher Sedore writes: A new, faster event notification system would be great. But don't forget to include *all* events, not just file descriptor readability/writability. I.e., signal delivery, child exit notification, maybe even support for an arbitrary number of (independent) timers. And make the events independent from each other -- to avoid problems like when an application completely hangs for 90 seconds when it calls gethostbyname(). An async thread to do hostname lookups would be great! Wouldn't be too hard in libc_r, would it? But in regular apps, setitimer and sigsetjmp() would be a solution. -Archie ___ Archie Cobbs * Whistle Communications, Inc. * http://www.whistle.com Brian Fundakowski Feldman _ __ ___ ___ ___ ___ gr...@freebsd.org _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
"Brian F. Feldman" wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 [EMAIL PROTECTED] you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: "Brian F. Feldman" wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 [EMAIL PROTECTED] you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out "change lists"; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 01:51:28PM +0930, Greg Lehey wrote: On Saturday, 3 July 1999 at 23:10:29 -0500, Jonathan Lemon wrote: On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out "change lists"; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). Did you see the presentation "A scalable and explicit event delivery mechanism for UNIX" at USENIX? It sounded quite interesting. Page 253 of the proceedings. Is this the paper by Mogul, et al? I didn't make it to USENIX, and don't have the proceedings at hand, but my implementation is fairly similar to a series of papers that Jeff Mogul has produced regarding web scalability. -- Jonathan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 01:01:07AM -0400, Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Yup, I have. poll() is faster; copying the parameters takes less time than bit twiddling. As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). Whichever you're comfortable with. poll() isn't a pain once you know how to use it, and it does bring additional benefits. -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sat, 3 Jul 1999, Jonathan Lemon wrote: On Jul 07, 1999 at 01:01:07AM -0400, Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). Whichever you're comfortable with. poll() isn't a pain once you know how to use it, and it does bring additional benefits. I don't see how you can not find poll() a pain when compared to select(). It requires so much set-up, much like (for instance) aio_write() as opposed to write(). I suppose if you're masochistic, you won't mind doing that :) -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ gr...@freebsd.org _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 11:27:57AM -0400, Brian F. Feldman wrote: On Sat, 3 Jul 1999, Jonathan Lemon wrote: On Jul 07, 1999 at 01:01:07AM -0400, Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). Whichever you're comfortable with. poll() isn't a pain once you know how to use it, and it does bring additional benefits. I don't see how you can not find poll() a pain when compared to select(). It requires so much set-up, much like (for instance) aio_write() as opposed to write(). I suppose if you're masochistic, you won't mind doing that :) Yes, it does require more initial setup. But consider: - you don't have to re-initialize the fd sets every time around the loop, as you do with select(). This administrative overhead is moved into the initial setup, not into the main loop. - it becomes simple to ignore a slot entry; simply set the fd value to -1. - you get notification when the fd is closed. E.g.: you can poll() on a fd, ignoring both read ready and write ready state, and get POLLHUP returned when it closes. for select(), the only way to know that the fd closed is to actually do a read() on the descriptor, and have it return EOF. -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sat, 3 Jul 1999, Jonathan Lemon wrote: On Jul 07, 1999 at 11:27:57AM -0400, Brian F. Feldman wrote: On Sat, 3 Jul 1999, Jonathan Lemon wrote: On Jul 07, 1999 at 01:01:07AM -0400, Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). Whichever you're comfortable with. poll() isn't a pain once you know how to use it, and it does bring additional benefits. I don't see how you can not find poll() a pain when compared to select(). It requires so much set-up, much like (for instance) aio_write() as opposed to write(). I suppose if you're masochistic, you won't mind doing that :) Yes, it does require more initial setup. But consider: - you don't have to re-initialize the fd sets every time around the loop, as you do with select(). This administrative overhead is moved into the initial setup, not into the main loop. - it becomes simple to ignore a slot entry; simply set the fd value to -1. - you get notification when the fd is closed. E.g.: you can poll() on a fd, ignoring both read ready and write ready state, and get POLLHUP returned when it closes. for select(), the only way to know that the fd closed is to actually do a read() on the descriptor, and have it return EOF. I agree. With a proper algorithm you can also track the fds in use and migrate them to lower slots in the pollfd array as well as make an allocator that attempts to grab slots at the beginning. Just something i'm working on right now :) -Alfred Perlstein - [bri...@rush.net|bri...@wintelcom.net] systems administrator and programmer Win Telecom - http://www.wintelcom.net/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
Brian F. Feldman wrote: On Sat, 3 Jul 1999, Jonathan Lemon wrote: On Jul 07, 1999 at 01:01:07AM -0400, Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). Whichever you're comfortable with. poll() isn't a pain once you know how to use it, and it does bring additional benefits. I don't see how you can not find poll() a pain when compared to select(). It requires so much set-up, much like (for instance) aio_write() as opposed to write(). I suppose if you're masochistic, you won't mind doing that :) He wouldn't be a programmer unless masochistic. -- Where am I, and what am I doing in this handbasket? Wes Peters Softweyr LLC http://softweyr.com/ w...@softweyr.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 0...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? Cheers, -Peter -- Peter Wemm - pe...@freebsd.org; pe...@yahoo-inc.com; pe...@netplex.com.au To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 0...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out change lists; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Saturday, 3 July 1999 at 23:10:29 -0500, Jonathan Lemon wrote: On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 0...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. This sounds rather like what Tandem did in its TOS^H^H^HGuardian^H^H^H^H^H^H^H^HNonStop Kernel operating system. At the user level, you had the choice of calling awaitio (-1), which would return the next completion on any fd, or awaitio (4251), which would return information on that specific fd. And of course you have the choice of blocking or not if the conditions aren't met. Of course, the performance implications of waiting on a specific fd are another issue. Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out change lists; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). Did you see the presentation A scalable and explicit event delivery mechanism for UNIX at USENIX? It sounded quite interesting. Page 253 of the proceedings. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Jul 07, 1999 at 01:51:28PM +0930, Greg Lehey wrote: On Saturday, 3 July 1999 at 23:10:29 -0500, Jonathan Lemon wrote: On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out change lists; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). Did you see the presentation A scalable and explicit event delivery mechanism for UNIX at USENIX? It sounded quite interesting. Page 253 of the proceedings. Is this the paper by Mogul, et al? I didn't make it to USENIX, and don't have the proceedings at hand, but my implementation is fairly similar to a series of papers that Jeff Mogul has produced regarding web scalability. -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Sun, 4 Jul 1999, Peter Wemm wrote: Brian F. Feldman wrote: On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/Pine.LNX.3.95.990702160538.27513C-10 0...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually go ing to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. Actually.. select() has three copyins and three copyouts per call. poll() has one copyin and one copyout per call. But poll() copies in HUGE amounts of data compared to the few bytes for thousands of FDs that select does. Now what I particular like is the event queue system that David Filo put together for Yahoo. In a nutshell you create a queue (a fd), and then register the descriptors you want to monitor with the queue. You then run an accept()-like loop where the accept returns the fd number that has met the conditions you asked for. For example, if you wanted to know if fd number 4251 becomes readable, then the accept would return 4251. This has potential to work across multiple processes sharing a queue so that events could get round robined or whatever. The other good part is that it maintains the state and lists persistantly and doesn't have to keep copying it to/from the kernel. It handles 50,000 to 100,000 connections without too much trouble. You can still use this with select as the queue fd becomes readable when there is an event waiting for your process. Is there interest in doing something like this in general? Yes! That would not replace select() or poll(), but it would be awesome to have! Cheers, -Peter -- Peter Wemm - pe...@freebsd.org; pe...@yahoo-inc.com; pe...@netplex.com.au To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ gr...@freebsd.org _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Saturday, 3 July 1999 at 23:31:20 -0500, Jonathan Lemon wrote: On Jul 07, 1999 at 01:51:28PM +0930, Greg Lehey wrote: On Saturday, 3 July 1999 at 23:10:29 -0500, Jonathan Lemon wrote: On Jul 07, 1999 at 12:04:35PM +0800, Peter Wemm wrote: Is there interest in doing something like this in general? YES! As a matter of fact, I've done something similar to this already, but instead of a queue, it's a variant of poll which passes in and out change lists; a list of fd's which have had status changes since the last call. I've been trying to bring it up for discussion on the -arch list, but it's been dead. (I think it was just fixed recently). Did you see the presentation A scalable and explicit event delivery mechanism for UNIX at USENIX? It sounded quite interesting. Page 253 of the proceedings. Is this the paper by Mogul, et al? Yes, this is the one. I didn't make it to USENIX, and don't have the proceedings at hand, but my implementation is fairly similar to a series of papers that Jeff Mogul has produced regarding web scalability. Good. I thought the paper (presented by Gaurav Banga) was quite interesting, but I wasn't convinced it was the only way to do it. Unfortunately, I haven't found time to look at it in more detail. Greg -- See complete headers for address, home page and phone numbers finger g...@lemis.com for PGP public key To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
In article local.mail.freebsd-hackers/pine.lnx.3.95.990702160538.27513c-100...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually going to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: poll() vs select()
On Fri, 2 Jul 1999, Jonathan Lemon wrote: In article local.mail.freebsd-hackers/pine.lnx.3.95.990702160538.27513c-100...@crb.crb-web.com you write: now supports the select() and poll() system calls. My question is really one of usage. Why would one us poll() over select()? Is select eventually going to go away for some reason? select() as a user-level call will never go away; there is a large base of code that uses it. poll() is faster (it doesn't have to do bit twiddling), and it's interface is cleaner (it can report invalid fd's, something select() can't do). As its functionality is a superset of select()'s, it is used as the internal implementation for select(). Actually, select() doesn't require horrendous amounts of copyin()s, which poll() does. So have you benchmarked the two? I'd expect select to be faster. As for new code, use whichever you are comfortable with. Personally, I would recommend poll(), since it provides some added functionality over select() that makes for easier programming. poll() is a huge pain to use, which is why I recommend select(). -- Jonathan To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message Brian Fundakowski Feldman _ __ ___ ___ ___ ___ gr...@freebsd.org _ __ ___ | _ ) __| \ FreeBSD: The Power to Serve!_ __ | _ \._ \ |) | http://www.FreeBSD.org/ _ |___/___/___/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
