On Mon, Jul 17, 2006, Henrik Nordstrom wrote: > Instead of the linked list of separately maintained buffers, have you > considered using MemBuf:s?
Nope! I just did this quick hack to mirror stuff I've done in the past under Linux/BSD to try and stuff IPC IO into page-aligned buffers to exploit COW semantics. (Although I didn't follow through with the last bit of the COW semantics - ie, keeping the buffer around until it /had/ been used :) I'll look at re-using the membuf code later today. > > * rework the socket IO scheduling code to try and hold onto a buffer until > > its full or 1 second has passed. It might not really matter but it'll be > > interesting to profile it at very high (>1500/sec) request rates. > > Not sure I agree with this. It should even out anyway under load, and > under low load it's just annoying to have logs delayed. They're only delayed to be sent to the logfile process up to a second. A 32k buffer holds 256 x 128 byte logfile entries; so an actual high throughput will basically ensure things never really get buffered. > > * figure out what to do about rotate (as it stands, this code only re-opens > > the logfiles and therefore pushes logfile maintainence into an external > > script.) > > You flush any pending buffers first, right? This patch doesn't do that yet. It wasn't that important as buffered data doesn't get lost during a rotate. The only thing this patch doesn't address is load shedding where the OS just can't write to the log file(s) fast enough. Its trivial to add, but I wanted to make sure it functioned correctly first. Robert: I do remember your work having that annoying problem of dropping logfile entries. I've got an idea on how to test it (I'll write a script to throw lots of consecutively-numbered requests at my code and make sure they all appear in the logs.) Adrian
