Just a follow up-
Lots of you got in touch with me and I appreciate your thoughts. After I sent
out the email I made a few minor rewrites to evhttp to support multiple threads
and have successfully been using this new code to run a photo caching server
against production traffic for a few days now.
I also got an email from Brian O'Kelley and Ilya Martynov describing similar
work they had done. I ended up borrowing one of their ideas to improve my
implementation, so hopefully they'll get their code in to svn sometime soon!
In the meantime, if anyone is interested in the diffs I have, feel free to get
in touch with me. No guarantees on correctness or performance but, from my
perspective, so far, so good.
--jason
-----Original Message-----
From: Jason Sobel
Sent: Friday, January 04, 2008 11:28 AM
To: [email protected]
Subject: evhttp and threads
Hi everyone-
A little background: my name is Jason Sobel, I'm an infrastructure engineer at
Facebook and I'm looking at using evhttp as the backbone for Facebook's custom
HTTP servers. I am currently working on two specific projects and we'll almost
certainly end up writing more down the road so this server is something that
will see a lot of use.
I know libevent & threaded applications has been discussed before, with the
prevailing wisdom provided by fellow Facebook engineer Steven Grimm:
http://monkeymail.org/archives/libevent-users/2007-January/000450.html
I've been reading over the evhttp code recently (1.4.1-beta) and, from what I
can see, the approach Steve describes will not work for evhttp. The problem
appears to be that, once a connection is created, the event base cannot be
changed. Thus all connection are trapped on the same event base and you can't
hand them off to other threads to complete. The only workaround I've found is
to hand out some work to a thread, have it put completed requests on a
thread-safe queue, and rely on an evtimer to periodically complete all the
requests on that queue.
Can anyone out there confirm that I'm understanding the code correctly?
I've been toying with an idea for changing the code to more readily support
multiple threads, however, and was hoping to get some feedback. The basic idea
is to change accept_socket:
static void
accept_socket(int fd, short what, void *arg)
{
struct evhttp *http = arg;
struct sockaddr_storage ss;
socklen_t addrlen = sizeof(ss);
int nfd;
if ((nfd = accept(fd, (struct sockaddr *)&ss, &addrlen)) == -1) {
event_warn("%s: bad accept", __func__);
return;
}
if (evutil_make_socket_nonblocking(nfd) < 0)
return;
evhttp_get_request(http, nfd, (struct sockaddr *)&ss, addrlen);
}
Instead of accepting a single evhttp instance my idea is to pass in a variable
number of evhttp structures, each with a different event base. When calling
evhttp_get_request I would somehow (randomly, round-robin, user-defined, etc.)
choose a evhttp to pass on. I *think* that this change will correctly trickle
down throughout the rest of the code and allow multiple connections to run in
parallel on different event bases.
Thanks for taking the time to look over this, I hope to get feedback/comments!
--jason
_______________________________________________
Libevent-users mailing list
[email protected]
http://monkeymail.org/mailman/listinfo/libevent-users
