Re: Writing to multiple descriptors with one system call

2010-03-23 Thread David Laight
On Wed, Mar 17, 2010 at 03:40:13PM +, Sad Clouds wrote: > On Wed, 17 Mar 2010 16:07:04 +0100 > Manuel Bouyer wrote: > > > On Wed, Mar 17, 2010 at 12:50:51PM +, Sad Clouds wrote: > > > [...] > > > I profiled a simple event driven server, with 5000 simultaneous > > > connections. About 80%

Re: Writing to multiple descriptors with one system call

2010-03-18 Thread Matthew Mondor
On Thu, 18 Mar 2010 21:36:47 +0100 Jean-Yves Migeon wrote: > Pretty much all servers use the accept loop thing and fork/pthread right > after, but this was not my point. High performance non-single-threaded servers often maintain a pool of persistent processes or threads which accept(2) concurr

Re: Writing to multiple descriptors with one system call

2010-03-18 Thread Jean-Yves Migeon
On 03/18/10 01:10, Sad Clouds wrote: Instead of calling write() 50 times for all 50 new sockets, you could just call write2v() once. So basically, between the 1st accept(2) and the last, all the clients are waiting for input (which they will get sequentially, when you will perform your write2v(

Re: Writing to multiple descriptors with one system call

2010-03-18 Thread Sad Clouds
On Wed, 17 Mar 2010 21:50:45 -0400 Matthew Mondor wrote: > I also have a question: did your test really use non-blocking sockets > for writing, and an efficient polling mechanism like kqueue or > libevent used, while disabling write polling when the sendq is empty, > enabling it back when there's

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Matthew Mondor
On Wed, 17 Mar 2010 16:22:44 + Sad Clouds wrote: > On Wed, 17 Mar 2010 16:01:28 + > Quentin Garnier wrote: > > > Do you have a real world use for that? For instance, I wouldn't call > > a web server that sends the same data to all its clients *at the same > > time* realistic. > > > Why

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Vlad Galu
On Thu, Mar 18, 2010 at 1:10 AM, Sad Clouds wrote: > On Thu, 18 Mar 2010 00:53:26 +0100 > Jean-Yves Migeon wrote: > >> On 03/17/10 17:53, Sad Clouds wrote: >> > 1. Accepting connections >> > >> > On busy web server it is likely that the listening socket will have >> > multiple entries on the comp

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Thu, 18 Mar 2010 00:53:26 +0100 Jean-Yves Migeon wrote: > On 03/17/10 17:53, Sad Clouds wrote: > > 1. Accepting connections > > > > On busy web server it is likely that the listening socket will have > > multiple entries on the completed connection queue (i.e. established > > state). You call

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Jean-Yves Migeon
On 03/17/10 17:53, Sad Clouds wrote: 1. Accepting connections On busy web server it is likely that the listening socket will have multiple entries on the completed connection queue (i.e. established state). You call accept() in a loop until you get EWOULDBLOCK and since these are new connections

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Joerg Sonnenberger
On Wed, Mar 17, 2010 at 12:50:51PM +, Sad Clouds wrote: > Hi is there some undocumented system call that allows a user process to > write to multiple descriptors in one go? No. It's usefulness is questionable and the interface would be messy (e.g. what should it return? how do you know which w

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Jochen Kunz
On Wed, 17 Mar 2010 16:53:11 + Sad Clouds wrote: > This is because it is common for web server to send many small text > files to many concurrent clients. You could fit several of those files > in a socket's TCP send buffer. Only if you have pipelined requests. In case of pipelining you can s

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Thor Lancelot Simon
On Wed, Mar 17, 2010 at 05:55:22PM +, Sad Clouds wrote: > On Wed, 17 Mar 2010 18:28:21 +0100 > Vlad Galu wrote: > > > On Wed, Mar 17, 2010 at 5:22 PM, Sad Clouds > > wrote: > > > On Wed, 17 Mar 2010 16:01:28 + > > > Quentin Garnier wrote: > > > > > >> Do you have a real world use for th

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 18:28:21 +0100 Vlad Galu wrote: > On Wed, Mar 17, 2010 at 5:22 PM, Sad Clouds > wrote: > > On Wed, 17 Mar 2010 16:01:28 + > > Quentin Garnier wrote: > > > >> Do you have a real world use for that?  For instance, I wouldn't > >> call a web server that sends the same data

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 19:07:29 +0100 Vlad Galu wrote: > That's easy to do with sendfile(), the VFS layer should do the caching > for you. Since you already mentioned kevent(), I assume you have > nonblocking I/O in mind, in which case your app won't be considerably > slown down by the send()/sendf

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 17:17:52 + (UTC) chris...@astron.com (Christos Zoulas) wrote: > if you are going to build that, then it is probably easier to include > all the info in the iovec: > > struct iovec2 { > int iov_fd; > int iov_errno; > ssize_t iov_nwritten; > void *iov

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Vlad Galu
On Wed, Mar 17, 2010 at 6:55 PM, Sad Clouds wrote: > On Wed, 17 Mar 2010 18:28:21 +0100 > Vlad Galu wrote: > >> On Wed, Mar 17, 2010 at 5:22 PM, Sad Clouds >> wrote: >> > On Wed, 17 Mar 2010 16:01:28 + >> > Quentin Garnier wrote: >> > >> >> Do you have a real world use for that?  For instan

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Quentin Garnier
On Wed, Mar 17, 2010 at 05:17:52PM +, Christos Zoulas wrote: > In article <20100317131044.f26d5d99.cryintotheblue...@googlemail.com>, > Sad Clouds wrote: > >On Wed, 17 Mar 2010 12:50:51 + > >Sad Clouds wrote: > > > >> Hi is there some undocumented system call that allows a user process >

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Vlad Galu
On Wed, Mar 17, 2010 at 5:22 PM, Sad Clouds wrote: > On Wed, 17 Mar 2010 16:01:28 + > Quentin Garnier wrote: > >> Do you have a real world use for that?  For instance, I wouldn't call >> a web server that sends the same data to all its clients *at the same >> time* realistic. >> > Why? Becaus

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 16:01:28 + Quentin Garnier wrote: > Do you have a real world use for that? For instance, I wouldn't call > a web server that sends the same data to all its clients *at the same > time* realistic. > Why? Because it never happens? I think it happens quite often. Another exa

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Christos Zoulas
In article <20100317131044.f26d5d99.cryintotheblue...@googlemail.com>, Sad Clouds wrote: >On Wed, 17 Mar 2010 12:50:51 + >Sad Clouds wrote: > >> Hi is there some undocumented system call that allows a user process >> to write to multiple descriptors in one go? >> >> What I'm looking for is

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 11:03:03 -0500 Eric Haszlakiewicz wrote: > That it might, but I'm having a hard time seeing how you'd actually > use it. Do you have a delay in each connection to allow for that > connections response to be sent along with any other connections > responses? How do you figure

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Eric Haszlakiewicz
On Wed, Mar 17, 2010 at 03:40:13PM +, Sad Clouds wrote: > On Wed, 17 Mar 2010 16:07:04 +0100 > Manuel Bouyer wrote: > > > On Wed, Mar 17, 2010 at 12:50:51PM +, Sad Clouds wrote: > > > [...] > > > I profiled a simple event driven server, with 5000 simultaneous > > > connections. About 80%

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Quentin Garnier
On Wed, Mar 17, 2010 at 01:10:44PM +, Sad Clouds wrote: > On Wed, 17 Mar 2010 12:50:51 + > Sad Clouds wrote: > > > Hi is there some undocumented system call that allows a user process > > to write to multiple descriptors in one go? Do you have a real world use for that? For instance, I

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 16:07:04 +0100 Manuel Bouyer wrote: > On Wed, Mar 17, 2010 at 12:50:51PM +, Sad Clouds wrote: > > [...] > > I profiled a simple event driven server, with 5000 simultaneous > > connections. About 80% of the time was spent in write() system call. > > What makes you think th

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Manuel Bouyer
On Wed, Mar 17, 2010 at 12:50:51PM +, Sad Clouds wrote: > [...] > I profiled a simple event driven server, with 5000 simultaneous > connections. About 80% of the time was spent in write() system call. What makes you think that a syscall writing the same data to multiple descriptors would be fa

Re: Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
On Wed, 17 Mar 2010 12:50:51 + Sad Clouds wrote: > Hi is there some undocumented system call that allows a user process > to write to multiple descriptors in one go? > > What I'm looking for is a variation of writev() system call, e.g.: > > size_t write2v(int *d, int dcnt, const struct iove

Writing to multiple descriptors with one system call

2010-03-17 Thread Sad Clouds
Hi is there some undocumented system call that allows a user process to write to multiple descriptors in one go? What I'm looking for is a variation of writev() system call, e.g.: size_t write2v(int *d, int dcnt, const struct iovec *iov, int iovcnt); So the application builds an array of file de