Here's the current draft "What's new in Libevent 2.0" document.
============================================================
What's New In Libevent 2.0 so far:
1. About this document
This document describes the key differences between Libevent 1.4 and
Libevent 2.0, from a user's point of view. It was most recently
updated based on features in subversion trunk as of 16 April 2009.
NOTE 1: If any features or fixes get backported from trunk to 1.4,
they should get moved from here into whatsnew-14.txt, since they
will no longer be differences between 1.4 and this version.
NOTE 2: We may have missed some things on this list. Caveat haxxor.
2. New and Improved Event APIs
Many APIs are improved, refactored, or deprecated in Libevent 2.0.
COMPATIBILITY:
Nearly all existing code that worked with should Libevent 1.4 should still
work correctly with Libevent 2.0. However, if you are writing new code,
or if you want to port old code, we strongly recommend using the new APIs
and avoiding deprecated APIs as much as possible.
Binaries linked against Libevent 1.4 will need to be recompiled to link
against Libevent 2.0. This is nothing new; we have never been good at
preserving binary compatibility between releases. We'll try harder in the
future, though: see 2.1 below.
2.1. New header layout for improved compatibility
Libevent 2.0 has a new header layout to make it easier for programmers to
write good, well-supported libevent code. The new headers are divided
into three types.
There are *regular headers*, like event2/event.h. These headers contain
the functions that most programmers will want to use.
There are *backward compatibility headers*, like event2/event_compat.h.
These headers contain declarations for deprecated functions from older
versions of Libevent. Documentation in these headers should suggest what's
wrong with the old functions, and what functions you want to start using
instead of the old ones. Some of these functions might be removed in a
future release. New programs should generally not include these headers.
Finally, there are *structure headers*, like event2/event_struct.h.
These headers contain definitions of some structures that Libevent has
historically exposed. Exposing them caused problems in the past, since
programs that were compiled to work with one version of Libevent would
often stop working with another version that changed the size or layout
of some object. We've moving them into separate headers so that
programmers can know that their code is not depending on any unstable
aspect of the Libvent ABI. New programs should generally not include
these headers unless they really know what they are doing, and are
willing to rebuild their software whenever they want to link it against a
new version of libevent.
Functionality that once was located in event.h is now more subdivided.
The core event logic is now in event2/event.h. The "evbuffer" functions
for low-level buffer manipulation are in event2/buffer.h. The
"bufferevent" functions for higher-level buffered IO are in
event2/bufferevent.h.
COMPATIBILITY:
All of the old headers (event.h, evdns.h, evhttp.h, evrpc.h, and
evutil.h) will continue to work by including the corresponding new
headers. Old code should not be broken by this change.
2.2. New thread-safe, binary-compatibile APIs
Some aspects of the historical Libevent API have encouraged
non-threadsafe code, or forced code built against one version of Libevent
to no longer build with another. The problems with now-deprecated APIs
fell into two categories:
1) Dependence on the "current" event_base. In an application with
multiple event_bases, Libevent previously had a notion of the
"current" event_base. New events were linked to this base, and
the caller needed to explicitly reattach them to another base.
This was horribly error-prone.
Functions like "event_set" that worked with the "current" event_base
are now deprecated but still available (see 2.1). There are new
functions like "event_assign" that take an explicit event_base
argument when setting up a structure. Using these functions will help
prevent errors in your applications, and to be more threadsafe.
2) Structure dependence. Applications needed to allocate 'struct
event' themselves, since there was no function in Libevent to do it
for them. But since the size and contents of struct event can
change between libevent versions, this created binary-compatibility
nightmares. All structures of this kind are now isolated in
_struct.h header (see 2.1), and there are new allocate-and-
initialize functions you can use instead of the old initialize-only
functions. For example, instead of malloc and event_set, you
can use event_new().
(For people who do really want to allocate a struct event on the
stack, or put one inside another structure, you can still use
event2/event_compat.h.)
So in the case where old code would look like this:
#include <event.h>
...
struct event *ev = malloc(sizeof(struct event));
/* This call will cause a stack overrun if you compile with one version
of libevent and link dynamically against another. */
event_set(ev, fd, EV_READ, cb, NULL);
/* If you forget this call, your code will break in hard-to-diagnose
ways in the presence of multiple event bases. */
event_set_base(ev, base);
New code will look more like this:
#include <event2/event.h>
...
struct event *ev;
ev = event_new(base, fd, EV_READ, cb, NULL);
2.3. Overrideable allocation functions
If you want to override the allocation functions used by libevent
(for example, to use a specialized allocator, or debug memory
issues, or so on), you can replace them by calling
event_set_mem_functions. It takes replacements for malloc(),
free(), and realloc().
If you're going to use this facility, you need to call it _before_
Libevent does any memory allocation; otherwise, Libevent may allocate some
memory with malloc(), and free it with the free() function you provide.
You can disable this feature when you are building Libevent by passing
the --disable-malloc-replacement argument to configure.
2.4. Configurable event_base creation
Older versions of Libevent would always got the fastest backend
available, unless you reconfigured their behavior with the environment
variables EVENT_NOSELECT, EVENT_NOPOLL, and so forth. This was annoying
to programmers who wanted to pick a backend explicitly without messing
with the environment.
Also, despite our best efforts, not every backend supports every
operation we might like. Some features (like edge-triggered events, or
working with non-socket file descriptors) only work with some operating
systems' fast backends. Previously, programmers who cared about this
needed to know which backends supported what. This tended to get quite
ungainly.
There is now an API to choose backends, either by name or by feature.
Here is an example:
struct event_config_t *config;
struct event_base *base;
/* Create a new configuration object. */
config = event_config_new();
/* We don't want to use the "select" method. */
event_config_avoid_method(config, "select");
/* We want a method that can work with non-socket file descriptors */
event_config_require_features(config, EV_FEATURE_FDS);
base = event_base_new_with_config(config);
if (!base) {
/* There is no backend method that does what we want. */
exit(1);
}
event_config_free(config);
Supported features are documented in event2/event.h
2.5. Socket is now an abstract type
All APIs that formerly accepted int as a socket type now accept
"evutil_socket_t". On Unix, this is just an alias for "int" as
before. On Windows, however, it's an alias for SOCKET, which can
be wider than int on 64-bit platforms.
2.6. Timeouts and persistent events work together.
Previously, it wasn't useful to set a timeout on a persistent event:
the timeout would trigger once, and never again. This is not what
applications tend to want. Instead, applications tend to want every
triggering of the event to re-set the timeout. So now, if you set
up an event like this:
struct event *ev;
struct timeval tv;
ev = event_new(base, fd, EV_READ|EV_PERSIST, cb, NULL);
tv.tv_sec = 1;
tv.tv_usec = 0;
event_add(ev, &tv);
The callback 'cb' will be invoked whenever fd is ready to read, OR whenever
a second has passed since the last invocation of cb.
2.7. Multiple events allowed per fd
Older versions of Libevent allowed at most one EV_READ event and at most
one EV_WRITE event per socket, per event base. This restriction is no
longer present.
2.8. evthread_* functions for thread-safe structures.
Libevent structures can now be built with locking support. You can
enable this on a per-event-base level by writing functions to implement
mutexes and thread IDs, and passing them to evthread_set_locking_callback
and evthread_set_id_callback. This makes it safe to add, remove,
and activate events on an event base from a different thread.
If you want threading support and you're using pthreads, you can just
call evthread_use_pthreads(). (You'll need to link against the
libevent_pthreads library in addition to libevent_core. These functions are
not in libevent_core)
If you want threading support and you're using Windows, you can just
call evthread_use_windows_threads().
Once locking functions are enabled, every new event_base is created with a
lock. You can prevent a single event_base from being built with a lock
disabled by using the EVENT_BASE_FLAG_NOLOCK flag in its
event_config. If an event_base is created with a lock, it is safe to call
event_del, event_add, and event_active on its events from any thread. The
event callbacks themselves are still all executed from the thread running
the event loop.
To make an evbuffer or a bufferevent object threadsafe, call its
_enable_locking() function.
The HTTP and DNS apis are not currently threadsafe.
To build Libevent with threading support disabled, pass
--disable-thread-support to the configure script.
2.9. Edge-triggered events on some backends.
With some backends, it's now possible to add the EV_ET flag to an event
in order to request that the event's semantics be edge-triggered. Right
now, epoll and kqueue support this.
The corresponding event_config feature is EV_FEATURE_ET; see 2.4 for more
information.
3. Backend-specific improvements.
3.1. kqueue event ordering consistency
TODO(niels)
3.2. Improved notification on Linux
When we need to wake the event loop up from another thread, we use
an epollfd to do so, instead of a socketpair. This is supposed to be
faster.
4. Improvements to evbuffers
Libevent has long had an "evbuffer" implementation to wrap access to an
input or output memory buffer. In previous versions, the implementation
was very inefficient and lacked some desirable features. We've made many
improvements in Libevent 2.0.
4.1. Chunked-memory internal representation
Previously, each evbuffer was a huge chunk of memory. When we ran out of
space in an evbuffer, we used realloc() to grow the chunk of memory. When
data was misaligned, we used memmove to move the data back to the front
of the buffer.
Needless to say, this is a terrible interface for networked IO.
Now, evbuffers are implemented as a linked list of memory chunks, like
most Unix kernels use for network IO. Data is added at the end of the
linked list and removed from the front, so that we don't ever need
realloc huge chunks or memmove the whole buffer contents.
To avoid multiple calls to read and write, we use the readv/writev
interfaces (or WSASend/WSARecv on Windows) to do IO on multiple chunks at
once with a single system call.
COMPATIBILITY NOTE:
The evbuffer struct is no longer exposed in a header. The code here is
too volatile to expose an official evbuffer structure, and there was never
any means provided to create an evbuffer except via evbuffer_new which
heap-allocated the buffer.
If you need access to the whole bufer as a linear chunk of memory, the
EVBUFFER_DATA() function still works. Watch out, though: it needs to copy
the buffer's contents in a linear chunk before you can use it.
4.2. More flexible readline support
The old evbuffer_readline() function (which accepted any sequence of
CR and LF characters as a newline, and which couldn't handle lines
containing NUL characters), is now deprecated. The preferred
function is evbuffer_readln(), which supports a variety of
line-ending styles, and which can return the number of characters in
the line returned.
4.3. Support for file-based IO in evbuffers.
You can now add chunks of a file into a evbuffer, and Libevent will have
your OS use mapped-memory functionality, sendfile, or splice to transfer
the data without ever copying it to userspace. On OSs where this is not
supported, Libevent just loads the data.
There are probably some bugs remaining in this code. On some platforms
(like Windows), it just reads the relevant parts of the file into RAM.
4.4. Support for zero-copy writes in evbuffers.
You can add a piece of memory to an evbuffer without copying it. Instead,
Libevent adds a new element to the evbuffer's linked list of chunks with a
pointer to the memory you supplied.
4.5. Multiple callbacks per evbuffer
Previously, you could only have one callback active on an evbuffer at a
time. In practice, this meant that if one part of Libevent was using an
evbuffer callback to notice when an internal evbuffer was reading or
writing data, you couldn't have your own callback on that evbuffer.
Now, you can now use the evbuffer_add_cb() function to add a callback that
does not interfere with any other callbacks.
The evbuffer_setcb() function is now deprecated.
4.6. New callback interface
Previously, evbuffer callbacks were invoked with the old size of the
buffer and the new size of the buffer. This interface could not capture
operations that simultaneously filled _and_ drained a buffer, or handle
cases where we needed to postpone callbacks until multiple operations were
complete.
Callbacks that are set with evbuffer_setcb still use the old API.
Callbacks added with evbuffer_add_cb() use a new interface that takes a
pointer to a struct holding the total number of bytes drained read and the
total number of bytes written. See event2/buffer.h for full details.
4.7. Misc new evbuffer features
You can use evbuffer_remove() to move a given number of bytes from one
buffer to another.
The evbuffer_search() function lets you search for repeated instances of
a pattern inside an evbuffer.
You can use evbuffer_freeze() to temporarily suspend drains from or adds
to a given evbuffer. This is useful for code that exposes an evbuffer as
part of its public API, but wants users to treat it as a pure source or
sink.
You can have an evbuffer defer all of its callbacks, so that rather than
being invoked immediately when the evbuffer's length changes, they are
invoked from within the event_loop. This is useful when you have a
complex set of callbacks that can change the length of other evbuffers,
and you want to avoid having them recurse and overflow your stack.
5. Bufferevents improvements
Libevent has long included a "bufferevents" structure and related
functions that were useful for generic buffered IO on a TCP connection.
This is what Libevent uses for its HTTP implementation. In addition to
the improvements that they get for free from the underlying evbuffer
implementation above, there are many new features in Libevent 2.0's
evbuffers.
5.1. New OO implementations
The "bufferevent" structure is now an abstract base type with multiple
implementations. This should not break existing code, which always
allocated bufferevents with bufferevent_new().
Current implementations of the bufferevent interface are described below.
See also section TODO(nickm).
5.2. bufferevent_socket_new() replaces bufferevent_new()
Since bufferevents that use a socket are not the only kind,
bufferevent_new() is now deprecated. Use bufferevent_socket_new()
instead.
5.3. Filtered bufferevent IO
You can use bufferevent_filter_new() to create a bufferevent that wraps
around another bufferevent and transforms data it is sending and
receiving. See test/regress_zlib.c for a toy example that uses zlib to
compress data before sending it over a bufferevent.
5.3. Linked pairs of bufferevents
You can use bufferevent_pair_new() to produce two linked bufferevents.
This is like using socketpair, but doesn't require system-calls.
IMPLEMENTATION NOTE: These don't support timeouts properly yet.
5.4. Upcoming bufferevent features (NOT DONE YET)
Nick is writing a bufferevents backend that supports IOCP on Windows.
Supposedly, this will make Windows IO much faster for programs using
bufferevents. We'll have to see; the first implementation will probably
be quite poor.
Nick is writing a bufferevents filter to implement OpenSSL over a
bufferevents.
6. Extras improvements
6.1. DNS
6.1.1. IPv6 nameservers
The evdns code now lets you have nameservers whose addresses are IPv6.
6.1.2: Support for the 0x20 hack
6.1.3: Better security.
TODO(nickm) writeme
7. Infrastructure improvements
7.1. Better unit test framework
We now use a unit test framework that Nick wrote called "tinytest".
The main benefit from Libevent's point of view is that tests which
might mess with global state can all run each in their own
subprocess. This way, when there's a bug that makes one unit test
crash or mess up global state, it doesn't affect any others.
7.2. Better unit tests
Despite all the code we've added, our unit tests are much better
than before. Right now, iterating over the different backends on
my MacBook, I'm getting around 75% test coverage, compared with
less than 45% test coverage in Libevent 1.4.
(Right now, evdns.c has the worst test coverage of any significant
module, at 57%. If we exclude it from consideration, we're at 80%
test coverage, which is fairly good for a networking library.)
_______________________________________________
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
