Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent

2007-11-03 Thread William Ahern
On Sat, Nov 03, 2007 at 09:15:07PM +0100, Marc Lehmann wrote:
snip
 In that mail, I announced that I will work on the problems I encountered
 in libevent (many of which have been reported and discusssed earlier on
 this list). After analyzing libevent I decided that it wasn't fixable
 except by rewriting the core parts of it (the inability to have multiple
 watchers for the same file descriptor event turned out to be blocking for
 my applications, otherwise I wouldn't have started the effort in the first
 place...).

A good itch, indeed.

 The results look promising so far: I additionally implemented a libevent
 compatibility layer and benchmarked both libraries using the benchmark
 program provided by libevent: http://libev.schmorp.de/bench.html
 
 Here is an incomplete list of what I changed and added (see the full
 list at http://cvs.schmorp.de/libev/README, or the cvs repository at
 http://cvs.schmorp.de/libev/):

Man. More pressure to rename my library from libevnet to something else ;)
 
snip
 * there is full support for fork, you can continue to use the event loop
   in the parent and child (or just one of them), even with quirky backends
   such as epoll.

Curious how you managed to do this. Are you checking the process PID on each
loop?

 * there are two types of timers, based on real time differences and wall
   clock time (cron-like). timers can also be repeating and be reset at
   almost no cost (for idle timeouts used by many network servers). time jumps
   get detected reliably in both directions with or without a monotonic clock.

But then they're not truly real-time, no?

snip
 * event watchers can be added and removed at any time (in libevent,
   removing events that are pending can lead to crashes).

This is news to me. Can you give more detail, maybe with pointers to code?

 * different types of events use different watchers, so you don't have
   to use an i/o event watcher for timeouts, and you can reset timers
   seperately from other types of watchers. Also, watchers are much smaller
   (even the libevent emulation watcher only has about 2/3 of the size of a
   libevent watcher).

libevnet does this for I/O; timer is always set separately from read/write
events. (Point being, its using libevent.)

 * I added idle watchers, pid watchers and hook watchers into the event loop,
   as is required for integration of other event-based libraries, without
   having to force the use of some construct around event_loop.

Needing to do an operation on every loop is arguably very rare, and there's
not much burden in rolling your own. PID watchers, likewise... how many
spots in the code independently manage processes (as opposed to one unit
which can just catch SIGCHLD). Also, curious how/if you've considered Win32
environments.

 * the backends use a much simpler design. unlike in libevent, the code to
   handle events is not duplicated for each backend, backends deal only
   with file descriptor events and a single timeout value, everything else
   is handled by the core, which also optimises state changes (the epoll
   backend is 100 lines in libev, as opposed to 350 lines in libevent,
   without suffering from its limitations).

libevnet optimizes state changes. Logically every I/O request is single-shot
(which is more forgiving to user code), but it actually sets EV_PERSIST and
delays libevent bookkeeping until the [libevnet bufio] callback returns. If
the user code submits another I/O op from its callback (highly likely) then
the event is left unchanged. It's still re-entrant safe because it can
detect further activity up the call chain using some stack message passing
bits (instead of reference counting because I also use mem pools, but I
digress). Again, point being this can be done using libevent as-is.

 As for compatibility, the actual libev api is very different to the
 libevent API (although the design is similar), but there is a emulation
 layer with a corresponding event.h file that supports the event library
 (but no evbuffer, evnds, evhttp etc.).

Well... if you can persuade me of the utility then this Christmas I might
want to investigate writing an evdns-like component. See the lookup
component of libevnet. There are lots of nice things I need in a DNS
resolver that evdns and others are incapable of handling. And I've also
written more HTTP, RTSP, and SOCKS5 parsers than I can remember.

snip
 The obvious plan would be to take the evhttp etc. code from libevent and
 paste it in to libev, making libev a complete replacement for libevent
 with an optional new API. The catch is, I'd like to avoid this, because I
 am not prepared to maintain yet another library, and I am not keen on
 replicating the configure and portability work that went into libevent so
 far.

If you ask me, it would prove more fortuitous to re-write the DNS and HTTP
components then to replace libevent. Reason being because it would be hard
to substantively improve on DNS/HTTP without altering the API, whereas

[Libevent-users] test/*.c files use installed header files not the ones from the package

2007-11-03 Thread Marc Lehmann
during integration of libev into libevent, I found that the test/* files
include the *installed* header files instead of the ones from the package.

this doesn't work if there are any structure changes (obviously the structure
change in this case was going to libev).

fixing the #include event.h statements to use  fixed this.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


[Libevent-users] [PATCH] Allow evhttp server to use alternative event_base

2007-11-03 Thread Paul Fisher
The attached patch allows an evhttp server to properly operate on an 
alternative event_base that is set via a new interface evhttp_base_start(), 
which is also added by this patch. Basically this makes the http.c 
implementation apply the event_base present in the struct evhttp instance 
associated with the evhttp_connection/evhttp_request whenever an event is 
scheduled via event_set/event_add.

A separate patch will need to handle the evhttp clients.


--

paul 
diff -u libevent-1.3e.002/evhttp.h libevent-1.3e.003/evhttp.h
--- libevent-1.3e.002/evhttp.h	2007-10-31 11:49:23.0 -0500
+++ libevent-1.3e.003/evhttp.h	2007-10-31 11:50:14.0 -0500
@@ -66,6 +66,8 @@
 
 /* Start an HTTP server on the specified address and port */
 struct evhttp *evhttp_start(const char *address, u_short port);
+struct evhttp *evhttp_base_start(struct event_base *base, const char *address,
+ u_short port);
 
 /*
  * Free the previously create HTTP server.  Works only if no requests are
diff -u libevent-1.3e.002/http.c libevent-1.3e.003/http.c
--- libevent-1.3e.002/http.c	2007-09-12 17:20:34.0 -0500
+++ libevent-1.3e.003/http.c	2007-10-31 11:55:43.0 -0500
@@ -247,6 +247,14 @@
 }
 
 static void
+evhttp_base_set(struct evhttp_connection *evcon, struct event *ev)
+{
+if( evcon-http_server != 0
+ evcon-http_server-bind_ev.ev_base != 0 )
+event_base_set(evcon-http_server-bind_ev.ev_base, ev);
+}
+
+static void
 evhttp_add_event(struct event *ev, int timeout, int default_timeout)
 {
 	if (timeout != 0) {
@@ -275,6 +283,7 @@
 		event_del(evcon-ev);
 
 	event_set(evcon-ev, evcon-fd, EV_WRITE, evhttp_write, evcon);
+evhttp_base_set(evcon, evcon-ev);
 	evhttp_add_event(evcon-ev, evcon-timeout, HTTP_WRITE_TIMEOUT);
 }
 
@@ -730,6 +739,7 @@
 	}
 	/* Read more! */
 	event_set(evcon-ev, evcon-fd, EV_READ, evhttp_read, evcon);
+evhttp_base_set(evcon, evcon-ev);
 	evhttp_add_event(evcon-ev, evcon-timeout, HTTP_READ_TIMEOUT);
 }
 
@@ -886,6 +896,7 @@
 		event_del(evcon-close_ev);
 	event_set(evcon-close_ev, evcon-fd, EV_READ,
 	evhttp_detect_close_cb, evcon);
+event_base_set(evcon-http_server-bind_ev.ev_base, evcon-close_ev);
 	event_add(evcon-close_ev, NULL);
 }
 
@@ -952,6 +963,7 @@
  cleanup:
 	if (evcon-retry_max  0 || evcon-retry_cnt  evcon-retry_max) {
 		evtimer_set(evcon-ev, evhttp_connection_retry, evcon);
+evhttp_base_set(evcon, evcon-ev);
 		evhttp_add_event(evcon-ev, MIN(3600, 2  evcon-retry_cnt),
 		HTTP_CONNECT_TIMEOUT);
 		evcon-retry_cnt++;
@@ -1474,6 +1486,7 @@
 
 	/* Set up a callback for successful connection setup */
 	event_set(evcon-ev, evcon-fd, EV_WRITE, evhttp_connectioncb, evcon);
+evhttp_base_set(evcon, evcon-ev);
 	evhttp_add_event(evcon-ev, evcon-timeout, HTTP_CONNECT_TIMEOUT);
 
 	evcon-state = EVCON_CONNECTING;
@@ -1539,7 +1552,7 @@
 	if (event_initialized(evcon-ev))
 		event_del(evcon-ev);
 	event_set(evcon-ev, evcon-fd, EV_READ, evhttp_read_header, evcon);
-	
+evhttp_base_set(evcon, evcon-ev);
 	evhttp_add_event(evcon-ev, evcon-timeout, HTTP_READ_TIMEOUT);
 }
 
@@ -1918,7 +1931,8 @@
 }
 
 static int
-bind_socket(struct evhttp *http, const char *address, u_short port)
+bind_socket(struct event_base *base, struct evhttp *http,
+const char *address, u_short port)
 {
 	struct event *ev = http-bind_ev;
 	int fd;
@@ -1933,6 +1947,7 @@
 
 	/* Schedule the socket for accepting */
 	event_set(ev, fd, EV_READ | EV_PERSIST, accept_socket, http);
+event_base_set(base, ev);
 	event_add(ev, NULL);
 
 	event_debug((Bound to port %d - Awaiting connections ... , port));
@@ -1945,7 +1960,7 @@
  */
 
 struct evhttp *
-evhttp_start(const char *address, u_short port)
+evhttp_base_start(struct event_base *base, const char *address, u_short port)
 {
 	struct evhttp *http;
 
@@ -1959,7 +1974,7 @@
 	TAILQ_INIT(http-callbacks);
 	TAILQ_INIT(http-connections);
 
-	if (bind_socket(http, address, port) == -1) {
+	if (bind_socket(base, http, address, port) == -1) {
 		free(http);
 		return (NULL);
 	}
@@ -1967,6 +1982,12 @@
 	return (http);
 }
 
+struct evhttp *
+evhttp_start(const char *address, u_short port)
+{
+return evhttp_base_start(NULL, address, port);
+}
+
 void
 evhttp_free(struct evhttp* http)
 {
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] [PATCH] TAILQ_ENTRY missing in evhttp.h on linux

2007-11-03 Thread Niels Provos
Try

#include sys/queue.h

before including evhttp.h

Niels.

On 10/30/07, Paul Fisher [EMAIL PROTECTED] wrote:



 In using the latest 1.3e on linux, evhttp.h fails to compile because of a
 missing definition of TAILQ_ENTRY in evhttp.h.  This is due to the fact that
 the workaround in event.h is #define'd and #undef'd within event.h and not
 available to evhttp.h when defining struct evhttp_request.  This patch
 obviously fixes it:

  --- libevent-1.3e/evhttp.h  2007-08-25 13:49:22.0 -0500
  +++ libevent-1.3e.002/evhttp.h  2007-10-29 22:32:07.0 -0500
  @@ -108,7 +108,14 @@
* reasonable accessors.
*/
   struct evhttp_request {
  +#if defined(TAILQ_ENTRY)
  TAILQ_ENTRY(evhttp_request) next;
  +#else
  +struct {   \
  +   struct type *tqe_next;  /* next element */  \
  +   struct type **tqe_prev; /* address of previous next element */  \
  +}   next;
  +#endif

  /* the connection object that this request belongs to */
  struct evhttp_connection *evcon;

  ... but it would be nice if this was coordinated with the definition in
 event.h, possibly by simply not #undef'ing it from event.h.  Anyway, if
 there is a preference on how to fix this, I'd be glad to regenerate the
 patch.


  --

  paul
 ___
 Libevent-users mailing list
 Libevent-users@monkey.org
 http://monkey.org/mailman/listinfo/libevent-users


___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] [PATCH] Allow evhttp server to use alternative event_base

2007-11-03 Thread Niels Provos
Hi Paul,

thank you very much for your patch.  Unfortunately, something very
similar is already present in trunk:

/** Create a new HTTP server
 *
 * @param base (optional) the event base to receive the HTTP events
 * @return a pointer to a newly initialized evhttp server structure
 */
struct evhttp *evhttp_new(struct event_base *base);

/**
 * Start an HTTP server on the specified address and port.
 *
 * Can be called multiple times to bind the same http server
 * to multiple different ports.
 *
 * @param address a string containing the IP address to listen(2) on
 * @param port the port number to listen on
 * @return a newly allocated evhttp struct
 * @see evhttp_free()
 */
int evhttp_bind_socket(struct evhttp *http, const char *address, u_short port);


Niels.

On 11/3/07, Paul Fisher [EMAIL PROTECTED] wrote:



 The attached patch allows an evhttp server to properly operate on an
 alternative event_base that is set via a new interface evhttp_base_start(),
 which is also added by this patch. Basically this makes the http.c
 implementation apply the event_base present in the struct evhttp instance
 associated with the evhttp_connection/evhttp_request
 whenever an event is scheduled via event_set/event_add.

  A separate patch will need to handle the evhttp clients.


  --

  paul

 ___
 Libevent-users mailing list
 Libevent-users@monkey.org
 http://monkey.org/mailman/listinfo/libevent-users



___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] [PATCH] TAILQ_ENTRY missing in evhttp.h on linux

2007-11-03 Thread Christopher Layne
On Sat, Nov 03, 2007 at 04:56:07PM -0700, Niels Provos wrote:
 Try
 
 #include sys/queue.h
 
 before including evhttp.h
 
 Niels.

Why is this a usercode issue? Shouldn't evhttp.h be more interested in
handling it's dependencies than non-event parent code? It's similar to the
sys/time.h being a usercode, but event.h dependency as well. I mean why
not make the parent code just handle all includes for event.h and evhttp.h
- down to stdint.h while we're at it? Because that would be ridiculous -
and the child header should handle it's own dependencies. I know Rob Pike
may have thought at one time one shouldn't include other include files -
but that was also 1989.

-cl
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


[Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?

2007-11-03 Thread Marc Lehmann
While debugging a problem of http.c with libev, I found this code
in evhttp_write_buffer:

if (event_pending(evcon-ev, EV_WRITE|EV_TIMEOUT, NULL))
event_del(evcon-ev);

event_set(evcon-ev, evcon-fd, EV_WRITE, evhttp_write, evcon);
evhttp_add_event(evcon-ev, evcon-timeout, HTTP_WRITE_TIMEOUT);

now, event_set initialises a struct event. the problem with the above code
is that it calls event_set on a struct event that is currently in use and
expects it to work.

this doesn't directly contradict the sparse documentation, but is rather weird.

looking at event_set closely, I see that it doesn't initialise the struct event
fully (but it does initialiseev_flags).

I also saw that some parts of the testsuite are indeed expecting that
libevent treats struct events that have never been initialised by an
event_set properly (= ignoring them).

I therefore conclude that there is no function in libevent that actually
initialises struct events.

This strikes me as a rather weird design, especially as it requires the user
of libevent to clear all his struct events before passing it to event_set.

It also is not threadsafe, as event_set overwrites ev_base with current_base
which might not be the correct one.

Note also that some tets programs do not properly clear the event structure
they allocate (test-time for example).

This makes me wonder about these questions:

1. could anybody confirm wether a user must clear the struct event before
   passing it to event_set?
2. shouldn't this rather fundamental requirement be documented *somewhere*?
3. is there some guarantee that one can call event_add/del on cleared memory
   arrays and this will not crash?
4. will the code fragment above not cause an event to be added twice to some
   lists, as event_set clears the flags that event_add uses to detect wether
   an event is already on the list?

In any case, I can manage in the libev emulation layer by also relying
on some has been initialiased before flag, but this is of course no
guarantee that code like the above will ever work (and I suspect it will
not).

In general, this whole design strikes me as rather messy.

(in libev, you always have to call one of the watcher initialiser macros
that do not depend on earlier contents).

Thanks a lot for any insights.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?

2007-11-03 Thread Marc Lehmann
On Sat, Nov 03, 2007 at 05:40:50PM -0700, Niels Provos [EMAIL PROTECTED] 
wrote:
 I think the documentation is very clear on this:

I read that part, but the documentation doesn't mention anything about wether
one can pass in uninitalised memory (as I explained in my mail).

Did you read it?

 The function event_set() prepares the event structure ev to be used in
 future calls to event_add() and event_del().  The event will be
 prepared to call the function specified by the fn argument with an int
 argument indicating the file descriptor, a short argument indicating
 the type of event, and a void * argument given in the arg argument.
 The fd indicates the file descriptor that should be monitored for
 events.  The events can be either EV_READ, EV_WRITE, or both,
 indicating that an application can read or write from the file
 descriptor respectively without blocking.

Ok, where does it say one can event_add, event_set, and then event_add again?

This is what http.c actually does.

And where does it say that one can call event_del on a struct event without
calling event_set before?

This is what other code actually does.

 Perhaps you might like to create a libev mailing list and discuss
 further development of libev there?

It would be off-topic there, as the libev API doesn't suffer from these
problems, and its testsuite isn't as buggy.

 I find your somewhat discourteous insinuation of bugs distracting.

1. I reported a number of bugs in libevent so far. I am not insinuating,
   but reporting bugs, to improve libevent.
2. I rewrote the libevent core part to be faster, much more scalable,
   with less artificial limitations and worked hard to contribute
   this back to libevent.

I don't think reporting bugs (or talking about possible bugs) is
discourteous, nor do I think I was in any way discourteous.

If you don't want to hear about bugs or improvements for libevent, you can
say so, but I think you are treating me rather unfairly, given the amount of
work I did. Even if libev never gets integrated into libevent, fixing the
bugs I found while trying to get it to run should be of interest to you.

Or is it the fact that something came up that is faster and more
featureful (and smaller, too) than libevent that distracts you? Fear not,
as I still intend to contribute, but I realyl don'T want to be insulted
for improving your library.

:(

 Thank you,
  Niels.

omg, a top-poster and full-quoter, too.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?

2007-11-03 Thread Marc Lehmann
On Sun, Nov 04, 2007 at 01:57:31AM +0100, Marc Lehmann [EMAIL PROTECTED] 
wrote:
  I find your somewhat discourteous insinuation of bugs distracting.
 
 1. I reported a number of bugs in libevent so far. I am not insinuating,
but reporting bugs, to improve libevent.

I wanted to mention that I also delivered the fixes to those bugs while
reporting them.

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] [PATCH] TAILQ_ENTRY missing in evhttp.h on linux

2007-11-03 Thread Christopher Layne
On Sat, Nov 03, 2007 at 05:53:42PM -0700, Niels Provos wrote:
 Well, Rob Pike has many great insights.  He also does not like
 conditional compilation.  However, you are right, evhttp.h could
 resolve the dependency on queue.h internally.  I will look into it.

Agreed, Rob Pike is true blue - and I also respect him greatly - but what
he wrote back then (including the grumble about not using include guards)
just doesn't hold today. If one thinks about it in an abstract sense, it's
inverting the child-module dependency maintenance onto the parent module. It
was also somewhat hypocritical in that it was a form of compiler lexical
analyzer pre-optimization in it's own right.

-cl
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?

2007-11-03 Thread Marc Lehmann
On Sun, Nov 04, 2007 at 02:00:15AM +0100, Marc Lehmann [EMAIL PROTECTED] 
wrote:
 On Sun, Nov 04, 2007 at 01:57:31AM +0100, Marc Lehmann [EMAIL PROTECTED] 
 wrote:
   I find your somewhat discourteous insinuation of bugs distracting.
  
  1. I reported a number of bugs in libevent so far. I am not insinuating,
 but reporting bugs, to improve libevent.
 
 I wanted to mention that I also delivered the fixes to those bugs while
 reporting them.

And so that this isn't just idle words, here is a list:

- debian bug #448165 against libevent (the TAILQ_ENTRY problem just reported)
- debian bug #448173 crash bug in evdns_resolve_reverse_ipv6 (with obvious fix)
- the header file problems, causing crashes or worse in the testsuite in ABI
  changes.
- the http.c reported and (by now) verified (no fix, but the function should
  probably just call event_del).

Thats not insinuating, these are clear bugs and clear reports.

I think you need a serious attitude adjustment if you treat actively
contributing people like this :(

Doh :(

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] test/*.c files use installed header files not the ones from the package

2007-11-03 Thread Nick Mathewson
On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos wrote:
 Hi Marc,
 
 I appreciate your insights, but your message has nothing to do with
 libevent.  The make files in libevent use the -I option to provide the
 path to the header files.

I just tried to test this out, as follows.  I built libevent, and then
added #error Foo to the top of the event.h header in the libevent
build directory.

I confirmed that there existed valid headers (without the #error
directive) in /usr/include and /usr/local/include.  Then, I entered
the test directory and typed make clean; make

The compiler gave me:
gcc -DHAVE_CONFIG_H -I. -I..   -I.. -I../compat  -g -O2 -Wall -c test-init.c
In file included from test-init.c:23:
../event.h:2:2: error: #error foo

The test/test-init.c file does indeed #include event.h, but it
looks like thanks to the -I.. flag passed to gcc, it is finding the
copy of event.h in the parent directory.

This seems to confirm to me pretty well that, on my OS at least,
things work okay.

Marc, have you seen this problem you report in the wild?  If so, can
you give me some help reproducing it?  As far as I can tell, Niels is
right above.


yrs,
-- 
Nick Mathewson


pgpTgIjCZtzfF.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent

2007-11-03 Thread Marc Lehmann
On Sat, Nov 03, 2007 at 03:45:39PM -0700, William Ahern [EMAIL PROTECTED] 
wrote:
 A good itch, indeed.

I am currently working on integrating all modules from libevent, so it
becomes a full libevent replacement (and it already runs all of the
testsuite that doesn't require access to internals).

  * there is full support for fork, you can continue to use the event loop
in the parent and child (or just one of them), even with quirky backends
such as epoll.
 
 Curious how you managed to do this. Are you checking the process PID on each
 loop?

I considered that, but I think its too slow (one also needs to be careful
that watchers don't change e.g. epoll state until the getpid check is
done), or at leats I think I don't want that speed hit, no matter what.

Instead, I make it the users job to actually call me after a fork. I provide
three functions:

   void ev_fork_prepare (void);
   void ev_fork_parent (void);
   void ev_fork_child (void);

which you cna simply plug into pthread_atfork and it will work. The reason I
don't myself is that I don't want to require pthreads just for that, but the
perl interface for example does, so perl programs will be safe.

I wrote full support for fork even though its not automatic because it
can be done and is fully supported. With libevent, you can't free the
event base in general (program crashes with an assertion has has been
documented on this list a number of times).

  * there are two types of timers, based on real time differences and wall
clock time (cron-like). timers can also be repeating and be reset at
almost no cost (for idle timeouts used by many network servers). time 
  jumps
get detected reliably in both directions with or without a monotonic 
  clock.
 
 But then they're not truly real-time, no?

Within the limits of technology, they are:

- timers (based on monotonic time) will time out after n seconds (whatever
  was configured), even if the date resets in between (libevent can do that
  only for backward time jumps).

- periodicals will simply be rescheduled, if a periodic timer is scheduled
  to fire at some point then it will not be affected by the time jump,
  it will still fire at that point (its more complicated with periodic
  timers scheduled to repeat, if you schedule a periodic timer to execute
  on every minute than libev will try to schedule it to occur when time()
  % 60 == 0, regardless of any time jumps.

Of course, detecting and correcting this cnanot be done completely
reliable with sub-second precision (there is no API in posix to do that),
but with a monotonic clock, libev should manage quite fine to detect even
very small time jumps caused by ntpd.

(With no monotonic clock its a heuristic, obviously).

  * event watchers can be added and removed at any time (in libevent,
removing events that are pending can lead to crashes).
 
 This is news to me. Can you give more detail, maybe with pointers to code?

That is how I read the code in event.c on first glance. But in fact, it seems
to be safe. I initially thought only removing the current watcher is safe.

(I was in fact fooled by some bugs in the testsuite). Sorry for the
confusion, I was too busy implementing all the other goodies, and right
now I am busy implementing the remaining parts to get 100% event API
support.

  * different types of events use different watchers, so you don't have
to use an i/o event watcher for timeouts, and you can reset timers
seperately from other types of watchers. Also, watchers are much smaller
(even the libevent emulation watcher only has about 2/3 of the size of a
libevent watcher).
 
 libevnet does this for I/O; timer is always set separately from read/write
 events. (Point being, its using libevent.)

Even libevent is somewhat fast if you don't combine timeouts and io
watchers in the same struct event. But it is of course quite the waste.

  * I added idle watchers, pid watchers and hook watchers into the event loop,
as is required for integration of other event-based libraries, without
having to force the use of some construct around event_loop.
 
 Needing to do an operation on every loop is arguably very rare, and there's
 not much burden in rolling your own. 

Its a quality issue. If you have a program that uses libevent and a
library that needs to hook into it, it simply cannot be done. I happen to
have many such cases.

It basiclaly happens as soon as you use libevent as some part of some big
program (such as in form of a perl module :), where many components might
want to hook into the event loop.

With that functionality in place, you can do it. Without it, you simply
fail.

It doesn't matter much, as libev is still faster than libevent even with
all those watcher types.

 PID watchers, likewise... how many spots in the code independently
 manage processes (as opposed to one unit which can just catch
 SIGCHLD).

You could say the same about signals and be right just as well, they are
not as 

Re: [Libevent-users] test/*.c files use installed header files not the ones from the package

2007-11-03 Thread Marc Lehmann
On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos [EMAIL PROTECTED] 
wrote:
 I appreciate your insights, but your message has nothing to do with
 libevent.  The make files in libevent use the -I option to provide the
 path to the header files.

Well, since you thre documentation at me before, let me just say its high
time to actually read up on what those -I switches do to various
compilers

But then, I see you are not really interested in fixing bugs:

 I also noticed that you seem to have found several bugs in libevent.

And I keep finding more the more I look at it.

 It would be nice if you could send patches for them.

I did that for all bugs where I could identify the right solution. Most
bugs are actually fixed by the event core replacement code I wrote, for
the remaining ones I send in detailed instructions how to patch the file.

 I am a little bit dubious about some of the claims such as not being
 able to remove an event while in a callback.

Yes, that seems to be simply wrong, sorry for that. If you keep hitting
and finding bugs (or undocumented features, such as not beign able to have
multiple event watchers for most events, which is probably not a bug, but
certainly something outside of my expectation for an event loop), sometimes
one makes a mistake.

I am, however, not that thrilled anymore about contributing to libevent after
the treatment you give to me, ignoring my patches, and insulting me :(

-- 
The choice of a   Deliantra, the free code+content MORPG
  -==- _GNU_  http://www.deliantra.net
  ==-- _   generation
  ---==---(_)__  __   __  Marc Lehmann
  --==---/ / _ \/ // /\ \/ /  [EMAIL PROTECTED]
  -=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users


Re: [Libevent-users] test/*.c files use installed header files not the ones from the package

2007-11-03 Thread Nick Mathewson
On Sun, Nov 04, 2007 at 03:02:02AM +0100, Marc Lehmann wrote:
 On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos [EMAIL PROTECTED] 
 wrote:
  I appreciate your insights, but your message has nothing to do with
  libevent.  The make files in libevent use the -I option to provide the
  path to the header files.
 
 Well, since you thre documentation at me before, let me just say its high
 time to actually read up on what those -I switches do to various
 compilers
 
 But then, I see you are not really interested in fixing bugs:

I am indeed interested in fixing bugs.  So is Niels.  I am going
through the emails you have sent me in the order I got them, trying to
confirm or disconfirm the bugs, and looking for the fixes you made.

  I also noticed that you seem to have found several bugs in libevent.
 
 And I keep finding more the more I look at it.

  It would be nice if you could send patches for them.
 
 I did that for all bugs where I could identify the right solution. Most
 bugs are actually fixed by the event core replacement code I wrote, for
 the remaining ones I send in detailed instructions how to patch the file.

I think that by patches, Niels means diffs.  I do not see where
those are; please forgive me if I have missed them.

  I am a little bit dubious about some of the claims such as not being
  able to remove an event while in a callback.
 
 Yes, that seems to be simply wrong, sorry for that. If you keep hitting
 and finding bugs (or undocumented features, such as not beign able to have
 multiple event watchers for most events, which is probably not a bug, but
 certainly something outside of my expectation for an event loop), sometimes
 one makes a mistake.

Right.  Anybody can make mistakes.  That's how bugs happen.  But you
can make mistakes too: that means that we need to confirm the bugs
that you send in (unit tests and diffs would be ideal) and confirm
that the fixes fix the bugs and don't break anything else.

I'm sorry that this isn't going as fast as possible.  As mentioned
earlier, I'm at a party in cape cod this weekend, and right now I'm
taking time out to try to help get this stuff going better.

 I am, however, not that thrilled anymore about contributing to libevent after
 the treatment you give to me, ignoring my patches, and insulting me :(

Marc, Niels:  There are ways we could all communicate more politely
and effectively.  Let's take this stuff off-list.  It doesn't help
make a better libevent to argue about it here.

many thanks, and hopes for a better working relationship in the future,
-- 
Nick Mathewson


pgpC9hjwadsCf.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users