Colm MacCarthaigh wrote:
On Tue, Oct 04, 2005 at 08:37:08AM -0400, Brian Akins wrote:
No, I don't think it's so simple at all. Although it would have to parse
the SQL at init time (how does LogFormat know to ask the MySQL provider
to do this?)
LogFormat doesn't, but the init of the provider could.
that can't be simply untied from the actions at log-time;
in the case of SQL for example you have to be very pedantic about the
escaping, so that a request for "/foo\'; nasty sql;" doesn't kill us.
That what the binding ensures.
And we still need additional per-provider directives to provide the
database, hostname, username and so on information.
No, the init of the provider worries about the parameters. In fact, the
mysql modules may just have seperate directives to set this:
SqlLoggerUsername User
SqlLoggerPassword password
....
If such directives
are required anyway, what effort are we saving? Why not just replace
mod_log_config rather than plug into it.
Because mod_log_config handles a bunch of stuff for us. May be a better
solution would be to use the standard log_config way of replacing init
and writer and replace them with a pluggable one.
Though if all of this text parsing is getting expensive, I wonder would
anyone be interested in a protocol for binary logging from httpd?
Hmmm. interesting. Especially for things like spread, this would make
alot of sense.
CustomLog /logs/site.sock common
You re-implemented syslog :-) I've done the same myself for our hosting
service, we use syslog-ng to do all sorts of weird things with the info.
but syslog can't, on it's own, determine between virtuals (same problems
as piped loggers). I may have to look at syslog-ng. Can you maybe
off-list share some of your techniques?
define "damn busy."
We frequently see over 4000 requests per second being logged. Almost any
time there's a major security update for fedora really.
Okay, you qualify :)
Pipes themselves have little overhead, it's basically shared memory with
a standard IO interface and automatic mutexing. The processes on the
other side certainly need not be cumbersome - and I really like that you
can run a pipe-logger as a different uid, in a chroot and so on, it's a
nice place to sandbox all of that icky SQL parsing and that kind of
thing.
True. The test parsing just seems so "icky" to determine the virtual
host since Apache already knows that.
--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies
