On 21/07 09:06, Alan Carroll wrote: > I haven't seen anything directly log to an external platform. That said, it > is possible to have ATS logging write to a pipe then put a process on the > other end of that pipe that does the logging to the external system.
Yeah, that's pretty much what we do here too. There are a couple of caveats though: (1) the process reading from the pipe needs to be reliable, always running and actually reading, otherwise bad things happen if the pipe gets full, and (2) named pipes can have only one reader at a time. To decouple the task of reading from the pipe and sending logs somewhere we have written a daemon called fifo-log-demux, which is open source and you're very welcome to use/contribute to: https://github.com/wikimedia/operations-software-fifo-log-demux We're using it to inspect and send the same log stream to multiple systems (Prometheus, Kafka, Logstash) and make them available to standard output when an admin needs to look at things live for administrative purposes. Battle-tested with thousands of logs per second. :)
