Re: [graylog2] Suggestion: Stream Schedules
Hey, please use our ideas portal to check for existing or submit new ideas. Thank you! https://www.graylog.org/product-ideas/ Regards, Bernd Nicholas Meacoe [Wed, Aug 05, 2015 at 01:19:19AM -0700] wrote: >Hello, > >I've been looking for something, with very similar use cases to yours, as >we often have thousands of errors before market data feeds kick in at about >7am in the UK, so would be good to have the stream on a schedule to enforce >times of operation, i.e. when we actually care about being notified. > >Currently our ideas are to: >a) Stop/Start via the REST API >b) Process a different field and use greater/less than stream rules, but >this isn't ideal. > >Has there been any progress made on the feature request in Graylog? > >Kind regards, >Nick > >On Monday, November 17, 2014 at 7:22:41 PM UTC, Zi Dvbelju wrote: >> >> Just wanted to follow up - another benefit of this would be alert >> schedules (i.e. alerting day shift/night shift). >> >> On Friday, November 14, 2014 1:02:40 PM UTC-5, Zi Dvbelju wrote: >>> >>> Hey Lennart, >>> >>> Two example scenarios: >>> >>> 1) I'm monitoring a job that runs at 8AM. It finishes anywhere between >>> 8:10-8:15AM. I'm using nxlog (parsing a custom log file) and sending to >>> graylog to verify/alert if the job has failed. It's inefficient to have >>> that stream active for the entire day (wasting CPU resources) when it's >>> only required during an estimated window of time. >>> >>> 2) I monitor database backups with graylog (success messages). A script >>> queries elasticsearch every 10 minutes to see if the backup completed >>> successfully. Once it has been confirmed that the backup completed, >>> immediately start compressing the backup for off-site storage (this >>> eliminates needing to schedule compression jobs). The backup completion >>> window is a ~two hour range at most. >>> >>> I realize there are better solutions for each of these scenarios, but >>> centrally managing all alerts is a wonderful thing. Having a schedule for >>> specific streams could be useful in many different scenarios. >>> >>> Thanks! >>> Zi >>> >>> >>> >>> >>> >>> On Friday, November 14, 2014 12:01:17 PM UTC-5, lennart wrote: Hey Zi, thanks for the suggestion! Can you elaborate your use case for this? Thanks, Lennart On Fri, Nov 14, 2014 at 4:01 PM, Zi Dvbelju wrote: > I have a quick suggestion for streams - implement optional schedules during > which a stream can be active/paused. Would be an incredibly nice feature! > > Keep up the good work, absolutely loving Graylog2. > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to graylog2+u...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >>> > >-- >You received this message because you are subscribed to the Google Groups >"Graylog Users" group. >To unsubscribe from this group and stop receiving emails from it, send an >email to graylog2+unsubscr...@googlegroups.com. >To view this discussion on the web visit >https://groups.google.com/d/msgid/graylog2/8ffe47d5-8a3b-40a8-a207-1040673077e3%40googlegroups.com. >For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/20150819101319.GC22616%40tumbler.torch.local. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Suggestion: Stream Schedules
Hello, I've been looking for something, with very similar use cases to yours, as we often have thousands of errors before market data feeds kick in at about 7am in the UK, so would be good to have the stream on a schedule to enforce times of operation, i.e. when we actually care about being notified. Currently our ideas are to: a) Stop/Start via the REST API b) Process a different field and use greater/less than stream rules, but this isn't ideal. Has there been any progress made on the feature request in Graylog? Kind regards, Nick On Monday, November 17, 2014 at 7:22:41 PM UTC, Zi Dvbelju wrote: > > Just wanted to follow up - another benefit of this would be alert > schedules (i.e. alerting day shift/night shift). > > On Friday, November 14, 2014 1:02:40 PM UTC-5, Zi Dvbelju wrote: >> >> Hey Lennart, >> >> Two example scenarios: >> >> 1) I'm monitoring a job that runs at 8AM. It finishes anywhere between >> 8:10-8:15AM. I'm using nxlog (parsing a custom log file) and sending to >> graylog to verify/alert if the job has failed. It's inefficient to have >> that stream active for the entire day (wasting CPU resources) when it's >> only required during an estimated window of time. >> >> 2) I monitor database backups with graylog (success messages). A script >> queries elasticsearch every 10 minutes to see if the backup completed >> successfully. Once it has been confirmed that the backup completed, >> immediately start compressing the backup for off-site storage (this >> eliminates needing to schedule compression jobs). The backup completion >> window is a ~two hour range at most. >> >> I realize there are better solutions for each of these scenarios, but >> centrally managing all alerts is a wonderful thing. Having a schedule for >> specific streams could be useful in many different scenarios. >> >> Thanks! >> Zi >> >> >> >> >> >> On Friday, November 14, 2014 12:01:17 PM UTC-5, lennart wrote: >>> >>> Hey Zi, >>> >>> thanks for the suggestion! Can you elaborate your use case for this? >>> >>> Thanks, >>> Lennart >>> >>> On Fri, Nov 14, 2014 at 4:01 PM, Zi Dvbelju wrote: >>> > I have a quick suggestion for streams - implement optional schedules >>> during >>> > which a stream can be active/paused. Would be an incredibly nice >>> feature! >>> > >>> > Keep up the good work, absolutely loving Graylog2. >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "graylog2" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an >>> > email to graylog2+u...@googlegroups.com. >>> > For more options, visit https://groups.google.com/d/optout. >>> >> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/8ffe47d5-8a3b-40a8-a207-1040673077e3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Suggestion: Stream Schedules
Just wanted to follow up - another benefit of this would be alert schedules (i.e. alerting day shift/night shift). On Friday, November 14, 2014 1:02:40 PM UTC-5, Zi Dvbelju wrote: > > Hey Lennart, > > Two example scenarios: > > 1) I'm monitoring a job that runs at 8AM. It finishes anywhere between > 8:10-8:15AM. I'm using nxlog (parsing a custom log file) and sending to > graylog to verify/alert if the job has failed. It's inefficient to have > that stream active for the entire day (wasting CPU resources) when it's > only required during an estimated window of time. > > 2) I monitor database backups with graylog (success messages). A script > queries elasticsearch every 10 minutes to see if the backup completed > successfully. Once it has been confirmed that the backup completed, > immediately start compressing the backup for off-site storage (this > eliminates needing to schedule compression jobs). The backup completion > window is a ~two hour range at most. > > I realize there are better solutions for each of these scenarios, but > centrally managing all alerts is a wonderful thing. Having a schedule for > specific streams could be useful in many different scenarios. > > Thanks! > Zi > > > > > > On Friday, November 14, 2014 12:01:17 PM UTC-5, lennart wrote: >> >> Hey Zi, >> >> thanks for the suggestion! Can you elaborate your use case for this? >> >> Thanks, >> Lennart >> >> On Fri, Nov 14, 2014 at 4:01 PM, Zi Dvbelju wrote: >> > I have a quick suggestion for streams - implement optional schedules >> during >> > which a stream can be active/paused. Would be an incredibly nice >> feature! >> > >> > Keep up the good work, absolutely loving Graylog2. >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "graylog2" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to graylog2+u...@googlegroups.com. >> > For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Suggestion: Stream Schedules
Hey Lennart, Two example scenarios: 1) I'm monitoring a job that runs at 8AM. It finishes anywhere between 8:10-8:15AM. I'm using nxlog (parsing a custom log file) and sending to graylog to verify/alert if the job has failed. It's inefficient to have that stream active for the entire day (wasting CPU resources) when it's only required during an estimated window of time. 2) I monitor database backups with graylog (success messages). A script queries elasticsearch every 10 minutes to see if the backup completed successfully. Once it has been confirmed that the backup completed, immediately start compressing the backup for off-site storage (this eliminates needing to schedule compression jobs). The backup completion window is a ~two hour range at most. I realize there are better solutions for each of these scenarios, but centrally managing all alerts is a wonderful thing. Having a schedule for specific streams could be useful in many different scenarios. Thanks! Zi On Friday, November 14, 2014 12:01:17 PM UTC-5, lennart wrote: > > Hey Zi, > > thanks for the suggestion! Can you elaborate your use case for this? > > Thanks, > Lennart > > On Fri, Nov 14, 2014 at 4:01 PM, Zi Dvbelju > wrote: > > I have a quick suggestion for streams - implement optional schedules > during > > which a stream can be active/paused. Would be an incredibly nice > feature! > > > > Keep up the good work, absolutely loving Graylog2. > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "graylog2" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to graylog2+u...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Suggestion: Stream Schedules
Hey Zi, thanks for the suggestion! Can you elaborate your use case for this? Thanks, Lennart On Fri, Nov 14, 2014 at 4:01 PM, Zi Dvbelju wrote: > I have a quick suggestion for streams - implement optional schedules during > which a stream can be active/paused. Would be an incredibly nice feature! > > Keep up the good work, absolutely loving Graylog2. > > -- > You received this message because you are subscribed to the Google Groups > "graylog2" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to graylog2+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Suggestion: Stream Schedules
I have a quick suggestion for streams - implement optional schedules during which a stream can be active/paused. Would be an incredibly nice feature! Keep up the good work, absolutely loving Graylog2. -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
