I've been the guardian of our underpowered graylog cluster, and I think he's wanting the same thing I want, which is an alert when any random one of our 14,000+ devices begins sending unusual, or massive amounts of syslog, not just one specific device.
Presently I have to use a dashboard with views of the most common keywords such devices spew, and watch that dashboard myself. I have a few streams with the most common causes, but every day there is a new one. Adding a separate stream for each physical device is certainly unworkable; adding a stream for each error message that 50+ brands/models of devices can emit is playing a game of whack-a-mole. One simple thing I would love to do automatically is something I currently do manually: create a pie chart with table of the top 5 senders for the past (hour, 5 minutes, whatever), and find one particular host I know to be stable and rather steady in its logging... and then eyeball every host in the pie chart/table that is producing more than that specific host. I particularly notice any host that is producing more than 5% of all messages in the past X minutes. The typical device is less than 2%, and most are less than .01%. (We have a lot of devices sending small amounts each, under normal conditions.) There are a number of wifi access points, switches, and routers we use, and it seems every one of them has at least one error state they can enter into, in which they produce a spew of unusual error messages, reboot themselves, spew more error messages, reboot, ad infinitum. One such device can easily generate over a million lines of junk an hour, and unless I'm watching a dashboard at the right moment, or unless our NOC happens to notice the failure of the device (which for a single AP serving a single apartment, this can be a while) the one device can easily overload our under-powered graylog cluster. On Thu, Sep 8, 2016 at 5:32 AM, Jochen Schalanda <joc...@graylog.com> wrote: > Hi, > > that's certainly possible. > > Simply create a stream containing only the messages of that single system > (e. g. by checking the value of the "source" message field) and create a > stream alert which will go off if the stream contains more than X messages > within the last 60 minutes. > > - http://docs.graylog.org/en/2.1/pages/streams.html > - http://docs.graylog.org/en/2.1/pages/streams/alerts.html# > message-count-condition > > > Cheers, > Jochen > > On Wednesday, 7 September 2016 19:04:56 UTC+2, ironmanmk42 wrote: >> >> Graylog 1.3.2 (for now and looking to implement graylog 2.1) = >> >> Is it possible to setup a stream to alert if the number of messages from >> a single sources exceeds a count? >> I have some misbehaving apps on hosts which suddenly send over a million >> syslogs in say an hour or two because of a faulty app. >> It would be great to have a stream which can alert with the source and >> message count over last 1 hour if say > 1million. >> >> Thanks, >> > -- > You received this message because you are subscribed to the Google Groups > "Graylog Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to graylog2+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/graylog2/c53d5502-3240-4254-90de-84aceba9d018%40googlegroups.com > <https://groups.google.com/d/msgid/graylog2/c53d5502-3240-4254-90de-84aceba9d018%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- No matter what we think of Linux versus FreeBSD, etc., the one thing I really like about Linux is that it has Microsoft worried. Anything that kicks a monopoly in the pants has got to be good for something. - Chris Johnson -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/CAL5rfGUof%2B1Ho1bKdNrHZk2_PNoAzB3BoUsT44SRN9D5OrFOig%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
