sbezverk, yep. Event though I prefer to use Logstash. But fluentd is OK to me.
On Mon, Nov 28, 2016 at 8:31 AM, Serguei Bezverkhi (sbezverk) < sbezv...@cisco.com> wrote: > I would vote for fluentd because: 1 – it has been around since 2011 so it > is hard to call it green. 2 – There is constant development of new > features/filters, 3 – As it was mentioned at Kubecon by fluentd people, > they are deeply committed to Open Source community so rumors that they > would go private does not sound reliable. We use it in kolla kubernetes and > so far we could share only positive feedback. > > > > Serguei > > > > *From:* Steven Dake (stdake) > *Sent:* Sunday, November 27, 2016 3:46 PM > > *To:* OpenStack Development Mailing List (not for usage questions) < > openstack-dev@lists.openstack.org> > *Subject:* Re: [openstack-dev] [kolla] the alternative of log processing > tool > > > > Jeffrey, > > > > Logstash-forwarder is deprecated upstream, so we can’t rely on that. > Elastic's replacement is filebeat. > > > > I’m not sure which one meets the requirements – filebeat or fluentd. In > kolla-kubernetes fluentd is being used, and is well maintained. Both > implementations are pretty green IMO. Not sure if fluentd also does log > processing. I think its crucial to pick a component that just does log > forwarding since that is the part that was deprecated. > > > > Our system has no log stash at all in it, and I’d like to keep it that > way. Logstash is unnecessary for our use case. What we want is > forwarder->es->cabana. Whatever forwarder is chosen, recommend picking the > best of the two choices. I’d start with defining best as “does it solve > the same problem as Heka does in our current implementation” then sprinkle > throughput and minimal cpu and network utilization on top. If we can’t > make a decision from there, not sure I have any further suggestions as I am > not writing the code. > > > > Regards > > -steve > > > > > > *From: *Jeffrey Zhang <zhang.lei....@gmail.com> > *Reply-To: *"OpenStack Development Mailing List (not for usage > questions)" <openstack-dev@lists.openstack.org> > *Date: *Sunday, November 27, 2016 at 9:40 AM > *To: *"OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > *Subject: *Re: [openstack-dev] [kolla] the alternative of log processing > tool > > > > So filebeat is working with Logstash right? We need split the logs into > pieces by using logstash. IMU, Filebeat do not a variety of processing > plugins, like Logstash[0]. > > > > [0] https://www.elastic.co/guide/en/logstash/current/filter-plugins.html > > > > On Sun, Nov 27, 2016 at 11:30 PM, Ian Cordasco <sigmaviru...@gmail.com> > wrote: > > File beat is maintained be elastic and a part of their product line just > like ELK. It's a fantastic tool and quite flexible given its age and size > of codebase > > > > On Nov 26, 2016 11:59 PM, "Jeffrey Zhang" <zhang.lei....@gmail.com> wrote: > > Heka is marked deprecated in Kolla during Newton cycle[0]. And Now we have > a > > blueprint for this[1]. Two alternatives, fluentd[3] and Filebeat. > > > > For Filebeat, it is just a replacement of logstash-forward[2]. It is not > intent > > to replace the Logstash at all. > > > > > Filebeat is based on the Logstash Forwarder source code and replaces > Logstash > > > Forwarder as the method to use for tailing log files and forwarding them > to > > > Logstash. > > > > Fillebeat is a log transport tool rather than log processing too. I do not > > treat it as an alternative at all. > > > > To be honest, I'd like back to Logstash, and Logstash 5.x is released with > high > > performance improvement[4]. > > > > > In our performance testing, we've seen consistent throughput increases > > > across multiple configurations. In some cases, we observed up to 75% > > > increase in events processed through Logstash. > > > > another benefit to using Logstash is the whole ELK stack is maintained by > one > > community/company. It is well tested and easy to upgrade the whole stack > at the > > same time. Using other tools may force us on certain elasticsearch release. > > > > So, I think we have to alternative tools. > > > > * Fluentd > > * Logstash > > > > IMO, we need to make the decision and at least prepare the migration > solution now. > > > > [1] https://blueprints.launchpad.net/kolla/+spec/heka-deprecation > > [2] https://www.elastic.co/guide/en/beats/filebeat/current/ > migrating-from-logstash-forwarder.html > > [3] http://www.fluentd.org/ > > [4] https://www.elastic.co/blog/logstash-5-0-0-released > > > > -- > > Regards, > > Jeffrey Zhang > > Blog: http://xcodest.me > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > Regards, > > Jeffrey Zhang > > Blog: http://xcodest.me > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Regards, Jeffrey Zhang Blog: http://xcodest.me
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev