On 07/14/2018 07:29 AM, Sergey Glazyrin wrote:
Hello guys!
We are migrating our product to kolla-ansible and as far as probably
you know, it uses fluentd to control logs, etc. In non containerized
openstack we use rsyslog to send data to logstash.
Why not use rsyslog in containerized openstack too?
Why not use rsyslog to mutate/unify the records? Why use logstash? Note
that rsyslog can send records to elasticsearch, and the latest rsyslog
8.36 has enhanced the elasticsearch plugin to do client cert auth as
well as handle bulk index retries more efficiently.
We get data from syslog events. It looks like it's impossible to use
syslog in kolla-ansible. Unfortunately external_syslog_server option
doesn't work. Is there anyone who was able to use it ? But, nevermind,
we may use fluentd BUT.. we have one problem - different data format
for each service/container.
So, probably the most optimal solution is to use default logging idea
in kolla-ansible. (to be honest, I am not sure... but I've no found
better option). But even with default logging idea in kolla - ansible
we have one serious problem. Fluentd has different data format for
each service, for instance, you may see this link with explanation how
its designed in kolla-ansible
https://github.com/openstack/kolla-ansible/commit/3026cef7cfd1828a27e565d4211692f0ab0ce22e
there are grok patterns which parses log messages, etc
so, we managed to put data to elasticsearch but we need to solve two
problems:
1. unify data format for log events. We may solve it using logstash to
unify it before putting it to elasticsearch (or should we change
fluentd configs in our own version of kolla-ansible repository ? )
For instance, we may do it using this logstash plugin
https://www.elastic.co/guide/en/logstash/2.4/plugins-filters-mutate.html#plugins-filters-mutate-rename
What's your suggestion ?
--
Best, Sergey
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
