Wow. What a strange circumstance. That must have been somewhat
puzzling to debug. Your fix seems fine although - I think - it might
not be sufficient. I don't think that anybody (but me and maybe a
couple who read the source code) even know about the high-water mark
feature.
Perhaps there should be a limit by how much a buffer can be increased
at any given time?
Thank you very much for figuring this out,
Niels.
Cc'ing libevent users for more feedback.
On 2/4/06, Christopher Maxwell <[EMAIL PROTECTED]> wrote:
> It has taken quite a strange set of circumstances to do this, but I have
> discovered a new way to exhaust system memory when trying to read from a
> bufferevent.
>
> After externally copying a large file to another filesystem, such that a
> very large portion of the file remains in the system buffer cache (UBC?),
> the automatic calculation by bufferevent_readcb() and evbuffer_read()
> notices the absurdly large number of bytes immediately available to read and
> attempts to resize the evbuffer. This all happens before any callbacks can
> be run to drain the evbuffer. In my case, a file of ~100MB appeared all at
> once, was packaged into another 100MB outbound buffer, then promptly
> exploded.
>
> I could not find any knobs to adjust as watermarks are not checked until
> long after the buffer read. Below is a patch to bufferevent_readcb() that
> constrains the read size to the high watermark, if set. The plan is to
> give the application a fighting chance to drain the buffer, or at least
> disable it for a time if it cannot.
>
> cheers
> --chris
> [EMAIL PROTECTED]
>
> --- evbuffer.c.orig Sun Feb 5 00:21:06 2006
> +++ evbuffer.c Sun Feb 5 00:20:45 2006
> @@ -92,13 +92,17 @@
> int res = 0;
> short what = EVBUFFER_READ;
> size_t len;
> + int howmuch = -1;
>
> if (event == EV_TIMEOUT) {
> what |= EVBUFFER_TIMEOUT;
> goto error;
> }
>
> - res = evbuffer_read(bufev->input, fd, -1);
> + if (bufev->wm_read.high != 0)
> + howmuch = bufev->wm_read.high;
> +
> + res = evbuffer_read(bufev->input, fd, howmuch);
> if (res == -1) {
> if (errno == EAGAIN || errno == EINTR)
> goto reschedule;
>
>
_______________________________________________
Libevent-users mailing list
[email protected]
http://monkey.org/mailman/listinfo/libevent-users
