Of course, if you use "reliable piped logging" then if you also write more
than PIPE_BUF bytes (4kiB on Linux) in a log line, it's not guaranteed to
be atomic.
I have been thinking of how to work around that limit, by logging to an
Apache Flume Thrift RPC port. But to avoid the 4kiB limit you'd have to
put the Thrift client inside the httpd server. You'd really want to have a
dedicated thread in each child process for the Thrift client, so that it
can accumulate a batch of messages for the Flume server: otherwise, you'd
be putting a lot of transaction load on Apache Flume. So the httpd threads
would put their log lines on a synchronized queue consumed by the logging
thread. Each child process would have it's own TCP connection to Apache
Flume, so there's none of the atomic message issues you have with >
PIPE_BUF messages using a Pipe.
It'd also be nice if Apache Thrift had a c_apr target in addition to the
c_glib target, so that you don't have to bring in glib to use it.
Anyway, it seemed to be some work to implement, so I think I will just live
with the limit of 4kiB atomic log entries.
Here is the Flume Thrift protocol:
namespace java org.apache.flume.thrift
struct ThriftFlumeEvent {
1: required map <string, string> headers,
2: required binary body,
}
enum Status {
OK,
FAILED,
ERROR,
UNKNOWN
}
service ThriftSourceProtocol {
Status append(1: ThriftFlumeEvent event),
Status appendBatch(1: list<ThriftFlumeEvent> events),
}
On Mon, Apr 18, 2016 at 4:37 PM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:
> That doesn't work because it's no longer atomic, those log lines end up
> fanned out among all the other worker threads doing the same thing. Very
> hard to parse.
>
> On Mon, Apr 18, 2016 at 3:08 PM, Stefan Fritsch <s...@sfritsch.de> wrote:
>
>> On Monday 18 April 2016 10:42:23, Eric Covener wrote:
>> > I'm won over and not tinkering with this FWIW.
>>
>> Maybe add a utility function that splits a message on \n and calls
>> ap_log_rerror for each line?
>>
>
>
