[Libevent-users] Re: IOCP API

2007-02-27 Thread Toby Douglass
I now need to finish the test/example application. When that's done, I'll upload. Might even be this weekend. I've been working away. Got a wierd bug which is holding things up. Prototypes == 1. int socket_iocp_new( void **socket_iocp_state, size_t read_buffer_size, void

[Libevent-users] Re: IOCP writes

2007-02-22 Thread Toby Douglass
From: Toby Douglass [EMAIL PROTECTED] As it is, it's easy to support serial writes with IOCP, it's supporting multiple concurrent writes which is awkward for a convenient-to-use API over IOCP - and this I would think is absolutely required for some uses of UDP sockets. I haven't tried any

Re: [Libevent-users] IOCP writes

2007-02-21 Thread Toby Douglass
I'm adding support for IOCP writes. I've realised, though, that if you issue a write on a socket which currently has an outstanding reading, when the IOCP completes, you don't immediately know which operation has completed. Obviously I used the completion key. Actually, that's wrong. On a

Re: [Libevent-users] IOCP writes

2007-02-21 Thread Toby Douglass
I wrote: On a related note, did someone say that the event handle in the overlapped structure used by the IO functions (ReadFile, etc) is not actually used? If that's actually untouched, I could use it for my own purposes. The alternative is probably a malloc() per write operation :-/ As it

Re: [Libevent-users] IOCP writes

2007-02-21 Thread Toby Douglass
I wrote: As it is, it's easy to support serial writes with IOCP, it's supporting multiple concurrent writes which is awkward for a convenient-to-use API over IOCP - and this I would think is absolutely required for some uses of UDP sockets. I've figured out a decent way of doing it, a

[Libevent-users] Re: IOCP writes

2007-02-21 Thread Toby Douglass
Rhythmic Fistman wrote: I don't think you should be passing the same overlapped structure twice to WSARecv/Send. I'm pretty sure that the overlapped isn't yours until GetQueuedCompletionStatus says so. Try a unique overlapped structure for each overlappable call. Don't worry, I'm not re-using

[Libevent-users] Re: IOCP writes

2007-02-21 Thread Toby Douglass
Rhythmic Fistman wrote: From: Toby Douglass [EMAIL PROTECTED] If that's actually untouched, I could use it for my own purposes. The alternative is probably a malloc() per write operation :-/ You shouldn't have to touch that stuff. What are you trying to do? Keep state on a per read/write

[Libevent-users] Re: IOCP writes

2007-02-21 Thread Toby Douglass
Rhythmic Fistman wrote: On 2/22/07, Toby Douglass [EMAIL PROTECTED] wrote: Yup. I'll be passing in a struct which contains that overlapped as its first member. When it comes back to be, I'll cast the overlapped pointer to my struct type. I think this means though I need a malloc per write

[Libevent-users] IOCP

2007-02-18 Thread Toby Douglass
Okay, getting some work done. You'll be amused to know that I've now moved away from the VC build environment (although obviously still using the MS Platform SDK and compilers) and I'm using gnumake and a makefile :-) I found when I learned kernel development that the process of requiring a

[Libevent-users] IOCP

2007-01-15 Thread Toby Douglass
My graphics card died Saturday afternoon, so I've not been able to finish the IOCP stuff. Next weekend, hopefully. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users

Re: [Libevent-users] Add support for WINDOWS Overlapped IO

2006-12-11 Thread Toby Douglass
Kevin Sanders wrote: On 12/10/06, Toby Douglass [EMAIL PROTECTED] wrote: I've not added support for writes, because I think people generally issue blocking writes, since non-blocking means that if you return from a function which issues a send you have to ensure the lifetime of the buffer

Re: [Libevent-users] Add support for WINDOWS Overlapped IO

2006-12-11 Thread Toby Douglass
Zhu Han wrote: I have made some mistake when I type the above words. I just mean you should issue socket_internal_iocp_read firstly and then let the callbacks to consume the data. However, in your code, you invoke the callbacks before socket_internal_iocp_read. See the following snippet:

Re: [Libevent-users] Add support for WINDOWS Overlapped IO

2006-12-11 Thread Toby Douglass
Kevin Sanders wrote: I haven't looked at you're code yet, sorry I don't know the function names. I'm referring to when the IOCP thread calls GQCS and sees the read completion, it calls a read completion callback I believe? If someone made a blocking call from this callback (the callback is

[Libevent-users] IOCP rewrite

2006-12-06 Thread Toby Douglass
For those of you who are interested, I'm finishing off a rewrite of the IOCP code, such that is handles connect()/accept() behaviour. Which is to say, you init the IOCP state, and all your socket behaviour is then routed through the callback function, which is simply called when an accept,

Re: [Libevent-users] Win32 I/O completion ports code release

2006-12-05 Thread Toby Douglass
Gordon Scott wrote: One technical note; I believe that I/O completion ports cannot handle accept() type behaviour, What do you mean by this? With an IOCP, you can give the IOCP a *connected* socket and the IOCP will call you back when a read occurs. But you cannot pass in an *unconnected*

Re: [Libevent-users] Win32 I/O completion ports code release

2006-12-05 Thread Toby Douglass
The docs say the following functions can be used with an IOCP; ConnectNamedPipe DeviceIoControl LockFileEx ReadDirectoryChangesW ReadFile TransactNamedPipe WaitCommEvent WriteFile Use AcceptEx. The docs sayeth; The following functions can be used to start I/O operations that complete

Re: [Libevent-users] Win32 I/O completion ports code release

2006-12-05 Thread Toby Douglass
Kevin Sanders wrote: Use AcceptEx. Interesting. In the AcceptEx() docs, we find this; As with all overlapped Windows functions, either Windows events or completion ports can be used as a completion notification mechanism. ___ Libevent-users mailing

[Libevent-users] Win32 I/O completion ports code release

2006-12-03 Thread Toby Douglass
Hi. I've written a tiny library which provides a simple front end for IO completion ports. The ultimate purpose of this is to provide excellent socket handling under Win32 in libevent so that the socket problem Tor faces is solved. My code base is written around a resource and error handling

Re: [Libevent-users] Windows build impossible

2006-11-23 Thread Toby Douglass
Kalin Nakov wrote: Hi, I saw that libevent has some .dsp and .dsw files, but the sources are completely unbuildable. First HAVE_CONFIG is not defined in the project, also some WIN32 defines are re-defined in the code, some types are undefined and have to be manually defined, functions are used

Re: [Libevent-users] How does libevent deal with more events than a particular syscall can handle?

2006-11-19 Thread Toby Douglass
Roger Clark wrote: Has there ever been any mention of using IOCP or something on Windows? The Win32 implementation currently uses select() and still imposes the limit, which was mainly why I was asking. I've written IOCP code. It's in the hands of another list reader to get some testing; I'm

Re: [Libevent-users] IO Completion Ports

2006-11-01 Thread Toby Douglass
Kevin Sanders wrote: On 10/31/06, Toby Douglass [EMAIL PROTECTED] wrote: This is with MaxUserPort set to 2 (decimal), although I've not rebooted... Best hope for this to work is rebooting after changing the setting. Yup. Getting between 8200 and 8600 sockets now, with the server

Re: [Libevent-users] IO Completion Ports

2006-11-01 Thread Toby Douglass
Gordon Scott wrote: Unfortunately the code belongs to my employer so I can't share it, but (as Toby says it's late ) I can give you a general gist of the way it works tomorrow. Also if you can provide the code you have I can take a look and see if I see any glaring errors. Thankyou - that

[Libevent-users] GetQueuedCompletionStatus return values question/problem

2006-11-01 Thread Toby Douglass
This is for those of you who've done some IO completion port work. According to my reading of the docs, GetQueuedCompletionStatus has four return status'; --- 1. return value != 0 - all went well, we've just returned from a successful dequeuing 2. return value != 0 and lpOverlapped ==

Re: [Libevent-users] GetQueuedCompletionStatus return values question/problem

2006-11-01 Thread Toby Douglass
Kevin Sanders wrote: Now, the problem is this: ERROR_SUCCESS is #defined as zero! In other words, I don't think it's possible to differentiate between case 3 and 4, because in case 3, *lpNumberOfBytes might be 0 just by chance. Call GetLastError(). I could be wrong, but I don't think that

[Libevent-users] Re: Libevent-users Digest, Vol 13, Issue 8

2006-10-31 Thread Toby Douglass
[EMAIL PROTECTED] wrote: It's pretty spectacular stuff, I had a fairly average xp box doing 10k sockets (through an external interface, no cheatingly going through localhost) Did you get to 10k+ sockets in your first effort, or did you have to tweak the OS config?

Re: [Libevent-users] IO Completion Ports

2006-10-31 Thread Toby Douglass
Ycrux wrote: Hi Gordon and Toby! I'm interested to test your code. Could you please share it with us? Bed time now. Will post code to list tomorrow morning. ___ Libevent-users mailing list Libevent-users@monkey.org

[Libevent-users] yet more sockets under Win32

2006-10-30 Thread Toby Douglass
So, I've written an API to use IO completion ports. It's very simple - far easier than the framework necessary to handle multiple threads and WaitForMultipleObjects(). The API is; new_iocp() delete_iocp() add_socket() Currently, there's one callback function for all sockets. When the callback

[Libevent-users] sockets under Win32

2006-10-25 Thread Toby Douglass
I've almost finished code to handle arbitrary numbers of sockets under Win32 using WaitForMultipleObjects(). I expected to complete with one more weekend of work and then move to test. I've written the code from scratch last weekend. There's a socket object, a listen thread object and a listen