I am trying to work out how to develop a thread-safe module with two
threads, one thread reading and one thread writing. I'm using mpm-prefork
on apache 2.2.14-5ubuntu8.9 in case that matters. The module is a websocket
proxy.
My main thread (#1) is doing (in essence - error handling removed)
apr_pool_create(&opool, r->pool);
apr_bucket_alloc_t *oallocator = apr_bucket_alloc_create(opool);
apr_bucket_brigade *obb = apr_brigade_create(opool, oallocator);
apr_thread_create (... otherthread ...);
while (1) {
bb = apr_brigade_create(r->pool, r->connection->bucket_alloc);
ap_get_brigade(r->input_filters, bb, AP_MODE_READBYTES,
APR_BLOCK_READ, bufsiz));
apr_brigade_flatten(bb, buffer, &bufsiz);
apr_brigade_destroy(bb);
do_something_with(buffer);
}
The other thread (#2) is doing (in essence)
while (1) {
apr_pollset_poll(recvpollset, timeout, ... );
apr_socket_recv(socket, buf, ...)
do_something_else_with(buf);
ap_fwrite(r->connection->output_filters, obb, ...);
ap_fflush(r->connection->output_filters, obb, ...);
}
I am suffering from very occasional corruption of the bucket brigade which
normally shows up as a corrupted pool pointer or a bogus bucket entry in
thread #1 (for instance a SEGV in apr_brigade_length). Interestingly this
is the relatively quiet input brigade which is only ever touched by the
main apache thread. It's almost as if an allocator is not thread safe.
However, I'm using a separate bucket allocator (see code above) and (at
least in my code) a separate pool.
Could the output filter chain somehow be using the allocator attached to
the request (i.e. thread #1)? If so, how can I stop this? I can't run the
ap_fwrite/ap_flush under a mutex and have the same mutex held during
ap_get_brigade, as the latter blocks (and I can't see how to use the
non-blocking version without spinning).
[apologies for the partial dupe on the apache-users mailing list - I think
I've got nearer the problem since then and this list seems more
appropriate]
--
Alex Bligh
