Re: [Libevent-users] Add support for WINDOWS Overlapped IO
On 12/12/06, Kevin Sanders <[EMAIL PROTECTED]> wrote: On 12/11/06, Zhu Han <[EMAIL PROTECTED]> wrote: Ah, that's a good point. We're pretty sure it was 0, which means allow as many as physical processors, but if it was 1, that would explain it. Another clue with this is that if the first thread blocked (in our test we called Sleep(6), one of the other IOCP threads would then be able to pop that sockets completion notification. If the number is 0, the platform is UP and the IOCP thread never blocks at any point, it is possible another thread will never get the chance be waken up. Can you reproduce the CPU affinity problem? -- best regards, hanzhu ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: evtimer complains...
Ya.. I think some better documentation is in order.. I also am thinking that you might want to convert gettime() to evgettime() and export it for use with the library to keep those that might have a system that doesn't have CLOCK_MONOTONIC, which defaults to CLOCK_REALTIME or even defaults to gettimeofday() (all possible scenarios depending on the system), in order to have a common way to manipulate timers that match libevent. I've satisfied my questions.. and used a relative timer in my code. -Phil. Niels Provos wrote: The libevent API has never supported absolute times for timeouts. If they worked before then that was only by accident and never by intention. The switch to clock monotonic was made to support clock resets. So, personally, I am very surprised that absolute times ever worked for you. Is this something that should be documented more clearly in the man page? Niels. On 12/9/06, Phil Oleson <[EMAIL PROTECTED]> wrote: Niels, I've taken a little time time to look through my issue with evtimers. Previously, Scott Lamb questioned my code using absolute time with evtimers, but I see no problem with this. It used to work pre-1.2, and I've tracked the issue down to gettime() using CLOCK_MONOTONIC rather than CLOCK_REALTIME. When I've hacked libevent to use CLOCK_REALTIME, my test code works. To me it seems that using CLOCK_MONOTONIC causes issues with calls to gettimeofday. Was there any particular reason to have CLOCK_MONOTONIC as the arg to clock_gettime()? I would just like to understand this better before I rewrite my application. -Phil. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [Patch] Fix a bug related to DNS timeout counting
Thank you. I have applied the patch to trunk. Niels. On 12/11/06, Nick Mathewson <[EMAIL PROTECTED]> wrote: Hi, all! This patch fixes a bug in nameserver timeouts in evdns.c. Previously, we would count the number of successive requests that had timed out in a row for each nameserver, but we would never actually reset the timeout count to zero. This, obviously, would create problems: see the Tor bugtracker entry at http://bugs.noreply.org/flyspray/index.php?id=326&do=details This patch resets the successive timeout count to zero when: - A nameserver comes up - We receive a reply from a nameserver - We decide to not use the nameserver for a while because of its timeout count. This patch also changes the timeout threshold from 3 to 5 seconds. Probably, it should be easier to adjust. There may also be other issues that cause spurious nameserver timeouts: see the final comment in the bugtracker link above. yrs, -- Nick Mathewson ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] [Patch] Fix a bug related to DNS timeout counting
Hi, all! This patch fixes a bug in nameserver timeouts in evdns.c.
Previously, we would count the number of successive requests that had
timed out in a row for each nameserver, but we would never actually
reset the timeout count to zero. This, obviously, would create
problems: see the Tor bugtracker entry at
http://bugs.noreply.org/flyspray/index.php?id=326&do=details
This patch resets the successive timeout count to zero when:
- A nameserver comes up
- We receive a reply from a nameserver
- We decide to not use the nameserver for a while because of its
timeout count.
This patch also changes the timeout threshold from 3 to 5 seconds.
Probably, it should be easier to adjust. There may also be other
issues that cause spurious nameserver timeouts: see the final comment
in the bugtracker link above.
yrs,
--
Nick Mathewson
=== evdns.c
==
--- evdns.c (revision 11485)
+++ evdns.c (local)
@@ -212,7 +212,7 @@
static int global_max_requests_inflight = 64;
-static struct timeval global_timeout = {3, 0}; // 3 seconds
+static struct timeval global_timeout = {5, 0}; // 5 seconds
static int global_max_reissues = 1; // a reissue occurs when we get some
errors from the server
static int global_max_retransmits = 3; // number of times we'll retransmit a
request which timed out
// number of timeouts in a row before we consider this server to be down
@@ -447,6 +447,7 @@
evtimer_del(&ns->timeout_event);
ns->state = 1;
ns->failed_times = 0;
+ ns->timedout = 0;
global_good_nameservers++;
}
@@ -891,6 +892,7 @@
nameserver_failed(ns, strerror(err));
return;
}
+ ns->timedout = 0;
reply_parse(packet, r);
}
}
@@ -1051,6 +1053,7 @@
req->ns->timedout++;
if (req->ns->timedout > global_max_nameserver_timeout) {
+ req->ns->timedout = 0;
nameserver_failed(req->ns, "request timed out.");
}
pgpPXZxJJbo7L.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
Kevin Sanders wrote: I haven't looked at you're code yet, sorry I don't know the function names. I'm referring to when the IOCP thread calls GQCS and sees the read completion, it calls a read completion callback I believe? If someone made a blocking call from this callback (the callback is running on the IOCP thread) then that would be bad (performance wise). So the IOCP threads aren't totally internal because they run the callbacks. Ah, yes, quite so. Unless these IOCP threads are just triggering an event somewhere that the user threads are waiting on (please say no)? No, they're not. :-) ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
On 12/11/06, Zhu Han <[EMAIL PROTECTED]> wrote: On 12/11/06, Kevin Sanders <[EMAIL PROTECTED]> wrote: > I'm not sure what limit your asking about. Are you talking about the > GQCS milliseconds timeout value? In any case I'm not sure. He was > using his own IOCP code. I mean the value of NumberOfConcurrentThreads when invoke CreateIoCompletionPort. What's the value for it? Ah, that's a good point. We're pretty sure it was 0, which means allow as many as physical processors, but if it was 1, that would explain it. Another clue with this is that if the first thread blocked (in our test we called Sleep(6), one of the other IOCP threads would then be able to pop that sockets completion notification. Kevin ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
On 12/11/06, Toby Douglass <[EMAIL PROTECTED]> wrote: Kevin Sanders wrote: > The lesson here is you don't want the IOCP threads doing anything > except issuing async IO, popping completions and a quick state machine > change (see below) and issue another async IO if needed. If more > processing is needed, put a work item in a queue for another thread to > process. That work thread can call your state machine callback when > it is finished, and that may in turn cause further async IO. True. But you realise in the code I've written the IOCP threads are internal to the library itself - the user doesn't see them or touch them. All they do is call GQCS. If the user did blocking writes, they would be outside the IOCP mechanism and they wouldn't be using the IOCP threads. I haven't looked at you're code yet, sorry I don't know the function names. I'm referring to when the IOCP thread calls GQCS and sees the read completion, it calls a read completion callback I believe? If someone made a blocking call from this callback (the callback is running on the IOCP thread) then that would be bad (performance wise). So the IOCP threads aren't totally internal because they run the callbacks. Unless these IOCP threads are just triggering an event somewhere that the user threads are waiting on (please say no)? Kevin ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
On 12/11/06, Toby Douglass <[EMAIL PROTECTED]> wrote: Ah. Go look at socket_iocp_add_socket(). You'll find the very first call to socket_internal_iocp_read() occurs at the end of *that* function, which gets the read ball rolling. Yeah, you're right! I didn't pay attention to the prior socket_internal_iocp_read. Ye-e-e-s-s...and no. I think I understand you to mean that we could, for example, arrange it so we have 2xCPU threads running GQCS and when one of those threads returns with a complete, we trigger or signal the main event loop thread? the problem with that surely is going to be the large amount of thread switching. You may check out this paper:Multiprocessor Support for Event-Driven Programs. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users -- best regards, hanzhu ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
Niels Provos wrote: > Please, think about that before flaming. I don't think anyone is flaming anyone here right now. It seems to be a pretty scholarly, polite and reasonable discussion. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
Zhu Han wrote: I have made some mistake when I type the above words. I just mean you should issue socket_internal_iocp_read firstly and then let the callbacks to consume the data. However, in your code, you invoke the callbacks before socket_internal_iocp_read. See the following snippet: siss->callback_function( siss->socket, SOCKET_IOCP_SOCKET_RECV_SUCCESS, siss->read_buffer, byte_count, (void *) siss->user_state ); socket_internal_iocp_read( siss ); Ah. Go look at socket_iocp_add_socket(). You'll find the very first call to socket_internal_iocp_read() occurs at the end of *that* function, which gets the read ball rolling. As others have pointed out, Async WRITE is important for high-performance server. Yes, it's clear. I'm going to add it. Note though that with GetQueuedCompletionStatus(), the user has no synchronization work to do. The API handles it all behind the scenes. Single-thread event-driven model just means the event loop is running in one thread's context. There a lot of ways to combine it with the multi-thread worker. Ye-e-e-s-s...and no. I think I understand you to mean that we could, for example, arrange it so we have 2xCPU threads running GQCS and when one of those threads returns with a complete, we trigger or signal the main event loop thread? the problem with that surely is going to be the large amount of thread switching. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
Kevin Sanders wrote: On 12/10/06, Toby Douglass <[EMAIL PROTECTED]> wrote: I've not added support for writes, because I think people generally issue blocking writes, since non-blocking means that if you return from a function which issues a send you have to ensure the lifetime of the buffer you've sent is non-local. If you're doing even a small amount of writing, you're going to have dismal performance (at best) using blocking writes. If you're talking to a peer socket (real world) that is no longer responding, this write may take more than a minute to error out. True. I non-block and select and give it a couple of seconds, but of course even a couple of seconds is still hugely long for some uses. One of my coworkers recently observed, that handles associated with a IOCP seem to have CPU affinity, at least sometimes. In a read completion callback, he posted another read (which is fine and encouraged) and then went off and did a lot of processing which preventing it from calling GQCS for about 20 seconds (very bad). Even though there were 3 other threads waiting on GQCS, they couldn't pop the completion status for the read from the IOCP even though the read had completed. Finally, as soon as the original thread came back around and called GQCS, it popped the completion instead of the other threads which had been waiting the whole time. Oh man...! The lesson here is you don't want the IOCP threads doing anything except issuing async IO, popping completions and a quick state machine change (see below) and issue another async IO if needed. If more processing is needed, put a work item in a queue for another thread to process. That work thread can call your state machine callback when it is finished, and that may in turn cause further async IO. True. But you realise in the code I've written the IOCP threads are internal to the library itself - the user doesn't see them or touch them. All they do is call GQCS. If the user did blocking writes, they would be outside the IOCP mechanism and they wouldn't be using the IOCP threads. Single-thread event-driven seems to me to basically mean state machine. State machines are wonderful things for achieving simple, bug-free code, but they have a cost; they are implicitly single-threaded. This can mean you cannot use them in some situations, because you will inherently block whenever you perform work. I'm not sure I follow this. Are you saying you can't use state machines in a multi-threaded application because they cause threads to block? No - I'm saying state machines inherently serialise, and so if you need parallelism, you have a problem. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Add support for WINDOWS Overlapped IO
On 12/11/06, Kevin Sanders <[EMAIL PROTECTED]> wrote: I'm not sure what limit your asking about. Are you talking about the GQCS milliseconds timeout value? In any case I'm not sure. He was using his own IOCP code. I mean the value of NumberOfConcurrentThreads* *when invoke CreateIoCompletionPort. What's the value for it? Oh I'm not saying this is fact, only something we observed that we can't explain. The handle appeared to have some kind of affinity for that thread in his code, on that machine, that day. I actually forgot about this until this discussion, tomorrow I'll try to reproduce this with my code. Thanks a lot. If you can reproduce it, please let me known. I'll try to reproduce it when some box are available at at hands. I suspect this types of CPU affinity is non-sense since it could badly hurt the performance. -- best regards, hanzhu ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
