On Wed, Feb 22, 2017 at 6:36 PM, Jacob Champion <champio...@gmail.com> wrote: > On 02/22/2017 12:00 AM, Stefan Eissing wrote: >> >> Just so I do not misunderstand: >> >> you increased BUCKET_BUFF_SIZE in APR from 8000 to 64K? That is what you >> are testing? > > > Essentially, yes, *and* turn off mmap and sendfile. My hope is to disable > the mmap-optimization by default while still improving overall performance > for most users. > > Technically, Yann's patch doesn't redefine APR_BUCKET_BUFF_SIZE, it just > defines a new buffer size for use with the file bucket. It's a little less > than 64K, I assume to make room for an allocation header: > > #define FILE_BUCKET_BUFF_SIZE (64 * 1024 - 64)
Actually I'm not very pleased with this solution (or the final one that would make this size open / configurable). The issue is potentially the huge (order big-n) allocations which finally may hurt the system (fragmentation, OOM...). So I'm thinking of another way to achieve the same with the current APR_BUCKET_BUFF_SIZE (2 pages) per alloc. The idea is to have a new apr_allocator_allocv() function which would fill an iovec with what's available in the allocator's freelist (i.e. spare apr_memnodes) of at least the given min_size bytes (possibly a max too but I don't see the need for now) and up to the size of the given iovec. This function could be the base of a new apr_bucket_allocv() (and possibly apr_p[c]allocv(), though out of scope here) which in turn could be used by file_bucket_read() to get an iovec of available buffers. This iovec could then be passed to (new still) apr_file_readv() based on the readv() syscall, which would allow to read much more data in one go. With this the scheme we'd have iovec from end to end, well, sort of since mod_ssl would be break the chain but still produce transient buckets on output which anyway will end up in the core_output_filter's brigade of aside heap buckets, for apr_socket_sendv() to finally writev() them. We'd also have more recycled heap buckets (hence memnodes in the allocator) as the core_output_filter retains buckets, all with APR_BUCKET_BUFF_SIZE, up to THRESHOLD_MAX_BUFFER which, if configurable and along with MaxMemFree, would be the real limiter of recycling. So it's also adaptative. Actually it looks like what we need, but I'd like to have feedbacks before I go further the prototype I have so far (quite straightforward apr_allocator changes...). Thoughts? Regards, Yann.
