[Libevent-users] Re: IOCP writes
From: Toby Douglass [EMAIL PROTECTED] As it is, it's easy to support serial writes with IOCP, it's supporting multiple concurrent writes which is awkward for a convenient-to-use API over IOCP - and this I would think is absolutely required for some uses of UDP sockets. I haven't tried any UDP async socket stuff, not with IOCPs nor anything. Tell us how it goes! Well, it's something I've rarely needed to do - in fact, only for P2P VOIP. However, the example I had in mind (which is similar) is P2P apps, which use writes on a single local UDP socket to send peering information to peers. They would benefit from async writes on that socket. Still, I think I'm missing the point of multiple concurrent writes - wouldn't the output be undefined? Ah, no, because the destination on a UDP socket will be specified for each write. For a TCP socket, your concern is correct, multiple async writes would only be useful if each write was independent of every other write, so the ordering of dispatch didn't matter. (We would also need to assume each send() function call wrote its payload atomically with regard to other send() function calls, which I think is true, although I don't know that it is actually guaranteed). ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: IOCP writes
From: Toby Douglass [EMAIL PROTECTED] I'm adding support for IOCP writes. I've realised, though, that if you issue a write on a socket which currently has an outstanding reading, when the IOCP completes, you don't immediately know which operation has completed. Obviously I used the completion key. I didn't see it immediately because I'm already using that for something else, bah, hambug. I don't think you should be passing the same overlapped structure twice to WSARecv/Send. I'm pretty sure that the overlapped isn't yours until GetQueuedCompletionStatus says so. Try a unique overlapped structure for each overlappable call. -- felix: lightweight threads, HOFs and game scripting. svn co https://svn.sourceforge.net/svnroot/felix/felix/trunk felix ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: IOCP writes
From: Toby Douglass [EMAIL PROTECTED] I've realised, though, that if you issue a write on a socket which currently has an outstanding reading, when the IOCP completes, you don't immediately know which operation has completed. Obviously I used the completion key. Actually, that's wrong. On a related note, did someone say that the event handle in the overlapped structure used by the IO functions (ReadFile, etc) is not actually used? If that's actually untouched, I could use it for my own purposes. The alternative is probably a malloc() per write operation :-/ You shouldn't have to touch that stuff. What are you trying to do? Issuing simultaneous asynchronous reads and writes on socket is a tiny bit rare and special. Are you sure that's what you want to do? If so, great - io completion queues can handle it. Bu-ut if libevent was modelled on the unix select-style interfaces, most of which don't easily support this kind of thing, how is this situation cropping up, assuming that this is just an IOCP port of libevent? The unixy case is trickier - I think only kqueues can handle two separate read/write readiness requests being queued up in separate invocations. With all the other interfaces (epoll, event ports, select) you can request read AND write readiness notification, but you have to do with the one system call (the select interface makes this obvious, the other less so). -- felix: lightweight threads, HOFs and game scripting. svn co https://svn.sourceforge.net/svnroot/felix/felix/trunk felix ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: IOCP writes
Rhythmic Fistman wrote: I don't think you should be passing the same overlapped structure twice to WSARecv/Send. I'm pretty sure that the overlapped isn't yours until GetQueuedCompletionStatus says so. Try a unique overlapped structure for each overlappable call. Don't worry, I'm not re-using overlapped's like that. I've had just the one overlapped struct till now, because I only did reads, and you only have one read outstanding at a time. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: IOCP writes
Rhythmic Fistman wrote: From: Toby Douglass [EMAIL PROTECTED] If that's actually untouched, I could use it for my own purposes. The alternative is probably a malloc() per write operation :-/ You shouldn't have to touch that stuff. What are you trying to do? Keep state on a per read/write basis. Currently I keep state on a per socket basis, but that's not enough to let you know which of an arbitrary number of writes has just completed. However, I'm not going to use the abovementioned rather awful method (which I only considered from lack of alternatives) because I'm going to hijack the OVERLAPPED pointer, which is the right way to do this (or the least wrong way). Issuing simultaneous asynchronous reads and writes on socket is a tiny bit rare and special. I agree, but there are so many possible users people are going to want to do it. Also, it's common enough I think - HTTP with keep-alive, you could want to implement that using async IO for both the read and the write on the socket. Are you sure that's what you want to do? If so, great - io completion queues can handle it. Bu-ut if libevent was modelled on the unix select-style interfaces, most of which don't easily support this kind of thing, how is this situation cropping up, assuming that this is just an IOCP port of libevent? That's a wider question. I'm simply implementing a dead-easy-to-use API on IOCP. The unixy case is trickier - I think only kqueues can handle two separate read/write readiness requests being queued up in separate invocations. With all the other interfaces (epoll, event ports, select) you can request read AND write readiness notification, but you have to do with the one system call (the select interface makes this obvious, the other less so). UNIX, I know athing. Well, not quite true, I know POSIX somewhat, but true enough here for the better IO mechanisms. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: IOCP writes
Rhythmic Fistman wrote: On 2/22/07, Toby Douglass [EMAIL PROTECTED] wrote: Yup. I'll be passing in a struct which contains that overlapped as its first member. When it comes back to be, I'll cast the overlapped pointer to my struct type. I think this means though I need a malloc per write, since I have to create the overlapped structure. That's bad, since the point of this is high throughput. I don't think malloc's going to eat that much into your io-bound code's performance, but if you're worried about it just reuse the same overlapped structure instead of re-mallocing it. I don't want to have to be in a situation where I have to cache my allocations - it's extra code to write and debug and it impacts upon the cleanliness and simplicity of the design. It's a shame, really - with serial async writes, the only malloc you do is up front, once, when the socket is handed over to the API. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Re: IOCP writes
On Thu, Feb 22, 2007 at 12:09:18AM +0100, Rhythmic Fistman wrote: Issuing simultaneous asynchronous reads and writes on socket is a tiny bit rare and special. Are you sure that's what you want to do? If so, great - io completion queues can handle it. Bu-ut if libevent was modelled on the unix select-style interfaces, most of which don't easily support this kind of thing, how is this situation cropping up, assuming that this is just an IOCP port of libevent? Almost all of my use of libevent involves simultaneous reads/writes, for relays which filter streams on-the-fly, in both directions. It was tricky at first, but over the course of several revisions it all settled out. *sigh* Someday I'll have to muster up the energy to change the name, because on this list its just damn confusing, but my buffered-IO implementation is in libevnet: http://www.25thandclement.com/~william/projects/libevnet.html Specifically, see src/bufio/socket.c and the functions bufio_socket_event_run bufio_socket_event_mod bufio_socket_event_del bufio_socket_event_add bufio_socket_poll_handler bufio_sink_poll_handler Cancellations were tricky. Also, what I fail to see in most async libraries is logic to handle re-entrancy from recursive callbacks. That's what the FRAME_PUSH/FRAME_POP calls are all about. evdns.c fails in this respect, last time I looked, as does c-ares; they'll just segfault. My lookup API, built around c-ares, works around the c-ares problem w/ what I consider an abysmal hack; I'm looking to maybe fork the guts into libevnet because my re-entrancy patches were rejected. Anyhow, lookup.c does much, much more than c-ares, ADNS, or evdns, combined. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
