Re: Structured logging (was Early review of std.logger)
On Tuesday, 15 October 2013 at 15:21:52 UTC, Johannes Pfau wrote: I think one increasingly important point for std.log is 'structured logging'. Structured logging is basically not simply logging textual messages, but also logging additional KEY/VALUE pairs of data. The idea is that logs should not only be readable by humans but also easy to parse and analyze. Structured logging often also includes UUIDs to simplify finding similar errors/problems in a log file. For example: Instead of logging logf(Couldn't login as %s on server %s : Reason: %s, user, server, errorCode); we'd do: log(LOGIN_FAILED_UUID, Couldn't log in, [User: user, Server: server, ErrorCode: errorCode]); The log can then be queried for all events with 'LOGIN_FAILED_UUID' uuid, Server=... ErrorCode=42, etc. +1 I like the idea of structured logging. I really need to log some complex data, not only string. I usually use `to!string(value)` to convert complex data to string. About syntax - we can use something like this: log!(user, server, errorCode)(LOGIN_FAILED_UUID, Couldn't log in); user, server and errorCode is variables passed to template. We can get variable value as usually and variable name via `__traits(identifier, variable)`, it should work after DMD 2.064. For example, Vide.d use this idea, see `render` function: http://vibed.org/docs We can store complex data as string via `to!string(value)` cast until we can use real backend implementation like SystemDs.
Re: Structured logging (was Early review of std.logger)
On Friday, 18 October 2013 at 15:29:09 UTC, ilya-stromberg wrote: About syntax - we can use something like this: log!(user, server, errorCode)(LOGIN_FAILED_UUID, Couldn't log in); That will be template instance bloat disaster for something used as commonly as log function (comparing to a typical few dozen `render` calls per web application at most)
Re: Structured logging (was Early review of std.logger)
On Friday, 18 October 2013 at 16:01:30 UTC, Dicebot wrote: On Friday, 18 October 2013 at 15:29:09 UTC, ilya-stromberg wrote: About syntax - we can use something like this: log!(user, server, errorCode)(LOGIN_FAILED_UUID, Couldn't log in); That will be template instance bloat disaster for something used as commonly as log function (comparing to a typical few dozen `render` calls per web application at most) Yes, you are right. I forgot about this.
Re: Structured logging (was Early review of std.logger)
On 10/15/2013 05:20 PM, Johannes Pfau wrote: I think one increasingly important point for std.log is 'structured logging'. Structured logging is basically not simply logging textual messages, but also logging additional KEY/VALUE pairs of data. The idea is that logs should not only be readable by humans but also easy to parse and analyze. Structured logging often also includes UUIDs to simplify finding similar errors/problems in a log file. For example: Instead of logging logf(Couldn't login as %s on server %s : Reason: %s, user, server, errorCode); we'd do: log(LOGIN_FAILED_UUID, Couldn't log in, [User: user, Server: server, ErrorCode: errorCode]); The log can then be queried for all events with 'LOGIN_FAILED_UUID' uuid, Server=... ErrorCode=42, etc. As a nice benefit structured logging can also log the function, module and line of the source file that logged the message, Exceptions can be written to the log in a nice way, etc. SystemDs journal [1] is a structured replacement for syslog and is already being used on some linux distributions (ArchLinux, Fedora). It's likely that the journal will replace syslog on linux so at least for server-side software structured logging support is becoming important. The GNOME guys are also working on a log viewer for systemd's journal [2]. std.log certainly doesn't need to provide a backend for the journal right now. But some questions regarding structured logging need to taken into account: * Will structured logging be integrated with the normal logging at all? * Will the public API somehow collide with the proposed std.log API? * Should the API for log backends already support structured logging or is it possible to add this later on? Most important: * What to do about structured logging calls when logging to a simple text logger? Ignore the additional fields or serialize them? What about the UUID? Even if we don't have a backend implementation it'd be nice to have the public API available. Otherwise new projects cannot use structured logging with std.log right now and the user code needs to be rewritten once std.log does support structured logging. [1] http://0pointer.de/blog/projects/journalctl.html [2] https://mail.gnome.org/archives/gnome-announce-list/2013-September/msg00097.html Currently there is no structured logging implemented in std.logger. That been said, you can add it. The method Logger.logf is a variadic template. You can simple create your own Logger Class and overwrite that method and implemented your structured logging approach there. The only pitfall is that you need a ref of type WhateverYourLoggerClassIsCalled and call logf on that or you fall back to the default logf implementation provided by the Logger class. Another way would be to parse the message passed to logMessage and go from there. Long story short. Global default Logger means structured logging requires parsing. All other logger require correct reference type of logger.
Re: Structured logging (was Early review of std.logger)
On Tuesday, 15 October 2013 at 15:53:34 UTC, Robert Schadek wrote: That been said, you can add it. The method Logger.logf is a variadic template. You can simple create your own Logger Class and overwrite that method and implemented your structured logging approach there. The only pitfall is that you need a ref of type WhateverYourLoggerClassIsCalled and call logf on that or you fall back to the default logf implementation provided by the Logger class. Another way would be to parse the message passed to logMessage and go from there. Long story short. Global default Logger means structured logging requires parsing. All other logger require correct reference type of logger. I don't think it is a good design to conflate logging mode with output mode in a single logger class. I'd prefer to see log() as variadic structured logger and logf() as it is now (making conditional logger `logIf()`)
