Excerpts from Juan Antonio Osorio's message of 2017-11-04 00:11:47 +0200: > > On 3 Nov 2017 19:59, "Doug Hellmann" <[email protected]> wrote: > > > > Excerpts from Cédric Jeanneret's message of 2017-11-01 14:54:34 +0100: > > > Dear Stackers, > > > > > > While working on my locally deployed Openstack (Pike using TripleO), I > > > was a bit struggling with the logging part. Currently, all logs are > > > pushed to per-service files, in the standard format "one line per > > > entry", plain flat text. > > > > > > It's nice, but if one is wanting to push and index those logs in an ELK, > > > the current, default format isn't really good. > > > > > > After some discussions about oslo.log, it appears it provides a nice > > > JSONFormatter handler¹ one might want to use for each (python) service > > > using oslo central library. > > > > > > A JSON format is really cool, as it's easy to parse for machines, and it > > > can be on a multi-line without any bit issue - this is especially > > > important for stack traces, as their output is multi-line without real > > > way to have a common delimiter so that we can re-format it and feed it > > > to any log parser (logstash, fluentd, …). > > > > > > After some more talks, olso.log will not provide a unified interface in > > > order to output all received logs as JSON - this makes sens, as that > > > would mean "rewrite almost all the python logging management > > > interface"², and that's pretty useless, since (all?) services have their > > > own "logging.conf" file. > > > > > > That said… to the main purpose of that mail: > > > > > > - Default format for logs > > > A first question would be "are we all OK with the default output format" > > > - I'm pretty sure "humans" are happy with that, as it's really > > > convenient to read and grep. But on a "standard" Openstack deploy, I'm > > > pretty sure one does not have only one controller, one ceph node and one > > > compute. Hence comes the log centralization, and with that, the log > > > indexation and treatments. > > > > > > For that, one might argue "I'm using plain files on my logger, and > > > grep-ing -r in them". That's a way to do things, and for that, plain, > > > flat logs are great. > > > > > > But… I'm pretty sure I'm not the only one wanting to use some kind of > > > ELK cluster for that kind of purpose. So the right question is: what > > > about switching the default log format to JSON? On my part, I don't see > > > "cons", only "pros", but my judgment is of course biased, as I'm "alone > > > in my corner". But what about you, Community? > > > > > > - Provide a way to configure the output format/handler > > > While poking around in the puppet modules code, I didn't find any way to > > > set the output handler for the logs. For example, in puppet-nova³ we can > > > set a lot of things, but not the useful handler for the output. > > > > > > It would be really cool to get, for each puppet module, the capability > > > to set the handler so that one can just push some stuff in hiera, and > > > Voilà, we have JSON logs. > > > > > > Doing so would allow people to chose between the default (current) > > > output, and something more "computable". > > > > Using the JSON formatter currently requires setting a logging > > configuration file using the standard library configuration format > > and fully specifying things like log levels, handlers, and output > > destination. Would it make sense to add an option in oslo.log to > > give deployers an easier way to enable the JSON formatter? > > > > This would actually be very useful.
Great! Let me know if you would like to help figuring out where to do that in the library code. Doug __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
