I did some rough benchmarking (N entries over M threads) on the current
code vs wip-log2 (my naive implementation), which
- allocates an Entry struct on the heap, with 80 chars preallocated
- you write into a streambuf into that, overflowing into a std::string on
the heap
- takes a mutex
This is well-timed.. I just started playing with this code on the flight
over here.
There are a couple pieces to what I've done so far:
- clearly segmented logging code into a new set of classes in src/log,
without dependencies on everything else
- set up config infrastructure to
On Mon, Mar 19, 2012 at 1:53 PM, Tommi Virtanen
tommi.virta...@dreamhost.com wrote:
[mmap'ed buffer discussion]
I always thought mmap'ed circular buffers were an elegant approach for
getting data that survived a process crash, but not paying the
overhead of write(2) and read(2). The main
On 03/22/12 09:38, Colin McCabe wrote:
On Mon, Mar 19, 2012 at 1:53 PM, Tommi Virtanen
tommi.virta...@dreamhost.com wrote:
[mmap'ed buffer discussion]
I always thought mmap'ed circular buffers were an elegant approach for
getting data that survived a process crash, but not paying the
On Thu, Mar 22, 2012 at 11:17, Mark Kampe mark.ka...@dreamhost.com wrote:
(a) I actually favor not simply mmaping the circular buffer,
but having a program that pulls the data out of memory
and writes it to disk (ala Varnish). In addition to doing
huge writes (greatly reducing the
On Thu, 22 Mar 2012, Tommi Virtanen wrote:
If we are logging a lot, buffer management has the potential
to become a bottle-neck ... so we need to be able to allocate
a record of the required size from the circular buffer
with atomic instructions (at least in
On Thu, Mar 22, 2012 at 11:17 AM, Mark Kampe mark.ka...@dreamhost.com wrote:
On 03/22/12 09:38, Colin McCabe wrote:
An mmap'ed buffer, even a lockless one, is a simple beast. Do you
really need a whole library just for that? Maybe I'm just
old-fashioned.
IMHO, surprisingly few things
So, we've been talking about an in-memory buffer that would contain
debug-level messages, and a separate thread/process [1] that would
write a subset of these to disk. Thus, on crash, we'd have a lot of
detail available (the ring buffer entries not yet overwritten),
without paying the
