Proposed a patch to be able to enable the JSON formatter via an oslo.log configuration parameter:
https://review.openstack.org/#/c/517882/ On Mon, Nov 6, 2017 at 9:43 AM, Cédric Jeanneret < cedric.jeanne...@camptocamp.com> wrote: > On 11/06/2017 08:36 AM, Juan Antonio Osorio wrote: > > Giving this a bit more thought; I'm slightly more inclined on merely > adding > > the JSON formatter option to the standard oslo.log configuration options. > > Fine for me. > > > > > This is because we right now have the ability to pass oslo.cfg options > via > > the CLI, and we would loose that with the advanced logging configuration > > file. The aforementioned option is something that we're using for > > containerized openstack services. > > OK - I'll check on my own if I can get something nice with the > logging.conf file. But that won't be for "tomorrow", as I'm not > full-time on openstack (unfortunately :(). Both solutions have their > pros and cons in the end. > > > > > I'll look into adding the ability to turn that handler on/off from > oslo.log. > > Ping me if I can help :). And big thanks for that coming change! > > > > > > > > > On Mon, Nov 6, 2017 at 8:34 AM, Juan Antonio Osorio <jaosor...@gmail.com > > > > wrote: > > > >> > >> > >> On Mon, Nov 6, 2017 at 8:04 AM, Cédric Jeanneret < > >> cedric.jeanne...@camptocamp.com> wrote: > >> > >>> On 11/04/2017 07:08 AM, Doug Hellmann wrote: > >>>> Excerpts from Juan Antonio Osorio's message of 2017-11-04 00:11:47 > >>> +0200: > >>>>>> On 3 Nov 2017 19:59, "Doug Hellmann" <d...@doughellmann.com> wrote: > >>>>>> > >>>>>> Excerpts from Cédric Jeanneret's message of 2017-11-01 14:54:34 > +0100: > >>>>>>> Dear Stackers, > >>>>>>> > >>>>>>> While working on my locally deployed Openstack (Pike using > TripleO), > >>> I > >>>>>>> was a bit struggling with the logging part. Currently, all logs are > >>>>>>> pushed to per-service files, in the standard format "one line per > >>>>>>> entry", plain flat text. > >>>>>>> > >>>>>>> It's nice, but if one is wanting to push and index those logs in an > >>> ELK, > >>>>>>> the current, default format isn't really good. > >>>>>>> > >>>>>>> After some discussions about oslo.log, it appears it provides a > nice > >>>>>>> JSONFormatter handler¹ one might want to use for each (python) > >>> service > >>>>>>> using oslo central library. > >>>>>>> > >>>>>>> A JSON format is really cool, as it's easy to parse for machines, > >>> and it > >>>>>>> can be on a multi-line without any bit issue - this is especially > >>>>>>> important for stack traces, as their output is multi-line without > >>> real > >>>>>>> way to have a common delimiter so that we can re-format it and feed > >>> it > >>>>>>> to any log parser (logstash, fluentd, …). > >>>>>>> > >>>>>>> After some more talks, olso.log will not provide a unified > interface > >>> in > >>>>>>> order to output all received logs as JSON - this makes sens, as > that > >>>>>>> would mean "rewrite almost all the python logging management > >>>>>>> interface"², and that's pretty useless, since (all?) services have > >>> their > >>>>>>> own "logging.conf" file. > >>>>>>> > >>>>>>> That said… to the main purpose of that mail: > >>>>>>> > >>>>>>> - Default format for logs > >>>>>>> A first question would be "are we all OK with the default output > >>> format" > >>>>>>> - I'm pretty sure "humans" are happy with that, as it's really > >>>>>>> convenient to read and grep. But on a "standard" Openstack deploy, > >>> I'm > >>>>>>> pretty sure one does not have only one controller, one ceph node > and > >>> one > >>>>>>> compute. Hence comes the log centralization, and with that, the log > >>>>>>> indexation and treatments. > >>>>>>> > >>>>>>> For that, one might argue "I'm using plain files on my logger, and > >>>>>>> grep-ing -r in them". That's a way to do things, and for that, > plain, > >>>>>>> flat logs are great. > >>>>>>> > >>>>>>> But… I'm pretty sure I'm not the only one wanting to use some kind > of > >>>>>>> ELK cluster for that kind of purpose. So the right question is: > what > >>>>>>> about switching the default log format to JSON? On my part, I don't > >>> see > >>>>>>> "cons", only "pros", but my judgment is of course biased, as I'm > >>> "alone > >>>>>>> in my corner". But what about you, Community? > >>>>>>> > >>>>>>> - Provide a way to configure the output format/handler > >>>>>>> While poking around in the puppet modules code, I didn't find any > >>> way to > >>>>>>> set the output handler for the logs. For example, in puppet-nova³ > we > >>> can > >>>>>>> set a lot of things, but not the useful handler for the output. > >>>>>>> > >>>>>>> It would be really cool to get, for each puppet module, the > >>> capability > >>>>>>> to set the handler so that one can just push some stuff in hiera, > and > >>>>>>> Voilà, we have JSON logs. > >>>>>>> > >>>>>>> Doing so would allow people to chose between the default (current) > >>>>>>> output, and something more "computable". > >>>>>> > >>>>>> Using the JSON formatter currently requires setting a logging > >>>>>> configuration file using the standard library configuration format > >>>>>> and fully specifying things like log levels, handlers, and output > >>>>>> destination. Would it make sense to add an option in oslo.log to > >>>>>> give deployers an easier way to enable the JSON formatter? > >>>>>> > >>>>> > >>>>> This would actually be very useful. > >>>> > >>>> Great! Let me know if you would like to help figuring out where to do > >>>> that in the library code. > >>> > >>> There's already some (not really documented) feature allowing oslo.log > >>> to load a configuration file. In fact, there's already one existing in > >>> the keystone tree (/etc/keystone/logging.conf - altougn not loaded) - > we > >>> might base something "casual" and "generic" on that one. > >>> > >>> I think all wsgi/python services should configure the logging through > >>> that file so that it's clearer and cleaner. But that part is maybe a > bit > >>> too big and "aggressive" :). And the logging configuration isn't that > >>> friendly, to be honest, at least I have some issues with its doc ;). > >>> > >>> But I think we might come to something nice and cool. It would allow > >>> anyone to push their own log "formatter", in the end. > >>> So you (oslo.log) wouldn't need to work with format output requests > >>> anymore, just provide two basics (plain and json), and let users play > >>> with the logging configuration (and even output things in XML if they > >>> want) ;). > >>> > >>> Regarding oslo.log: apparently, adding the following entry in the > >>> service configuration file should do it: > >>> log-config-append¹ > >>> > >>> Can anyone confirm that? > >>> > >> > >> It seems to be the case at least from the docs in the option [1]. But if > >> we use this file (in TripleO and puppet) we really need to make it > >> backwards compatible. Would folks be OK with taking it into use? I guess > >> what it would take would be to document better the usage of this > advanced > >> configuration. Taking it into use doesn't look that bad; If you look at > the > >> file where the options are being set (_options.py), you will see that > there > >> are several options that get ignored once we start using this file [2]. > To > >> see all the options that get ignored you can look at the instances of > >> _IGNORE_MESSAGE. So, if we would start taking that into use, we would > need > >> to change the parameters of the oslo::log resource in puppet [3] to also > >> configure an equivalent option to that advanced logging file. > >> > >> [1] https://github.com/openstack/oslo.log/blob/master/oslo_log/_ > >> options.py#L44 > >> [2] https://github.com/openstack/oslo.log/blob/master/oslo_log/_ > >> options.py#L32 > >> [3] https://github.com/openstack/puppet-oslo/blob/master/ > manifests/log.pp > >> > >>> > >>> > >>> ¹ https://github.com/openstack/oslo.log/blob/master/oslo_log/_ > >>> options.py#L44 > >>> > >>>> > >>>> Doug > >>>> > >>>> ____________________________________________________________ > >>> ______________ > >>>> OpenStack Development Mailing List (not for usage questions) > >>>> Unsubscribe: openstack-dev-requ...@lists.op > >>> enstack.org?subject:unsubscribe > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>> > >>> > >>> > >>> ____________________________________________________________ > >>> ______________ > >>> OpenStack Development Mailing List (not for usage questions) > >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: > unsubscrib > >>> e > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > >>> > >> > >> > >> -- > >> Juan Antonio Osorio R. > >> e-mail: jaosor...@gmail.com > >> > >> > > > > > > > > > > ____________________________________________________________ > ______________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: > unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Juan Antonio Osorio R. e-mail: jaosor...@gmail.com
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev