Re: [Simple-evcorr-users] Problem with action2

2024-04-11 Thread Risto Vaarandi
hi Tom, the PairWithWindow rule works as follows (see also the documentation of the PairWithWindow rule in https://simple-evcorr.github.io/man.html#lbAP): step1) if the incoming event matches the pattern defined with the 'pattern' field, the rule either (a) starts a new event correlation

[Simple-evcorr-users] SEC tutorial has been updated

2023-12-26 Thread Risto Vaarandi
hi all, SEC tutorial has been updated and you can access the new version at https://simple-evcorr.github.io/SEC-tutorial.pdf. Many thanks to Jim Van Meggelen for suggestions on how to improve the tutorial! Happy Boxing Day to all SEC users :) risto

Re: [Simple-evcorr-users] Storing a sequence counter in a context

2023-09-24 Thread Risto Vaarandi
//www.clearlycore.com > > > *Asterisk: The Definitive GuideFIFTH EDITION NOW AVAILABLE TO DOWNLOAD:* > https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf > > -- > > *From: *"Risto Vaarandi" > *To: *"Jim Van Megge

Re: [Simple-evcorr-users] Storing a sequence counter in a context

2023-09-23 Thread Risto Vaarandi
hi Jim, the solutions from my previous post were provided without knowing the root cause of the issue. If the main reason for the problem is the nature of the timestamps in the logs (that is, they are provided with an accuracy of a second), I would recommend using a different logging scheme with

Re: [Simple-evcorr-users] Storing a sequence counter in a context

2023-09-22 Thread Risto Vaarandi
hi Jim, let me provide some suggestions how to accomplish this task. First, you could utilize the context based approach that you have described in your post. In that case, the numeral that has been retrieved from the context with the 'pop' action needs to be incremented, and you can do it with

Re: [Simple-evcorr-users] SEC tutorial

2023-08-11 Thread Risto Vaarandi
001 (DID) > +1-877-253-2716 (Canada) > +1-866-644-7729 (USA) > +1-416-425-6111 x6001 > jim.vanmegge...@clearlycore.com > http://www.clearlycore.com > > > *Asterisk: The Definitive GuideFIFTH EDITION NOW AVAILABLE TO DOWNLOAD:* > https://cdn.oreillystatic.com/pdf/Asterisk_T

[Simple-evcorr-users] SEC-2.9.2 released

2023-06-03 Thread Risto Vaarandi
hi all, SEC-2.9.2 has been released which is available from the SEC home page. You can also download it through a direct download link: https://github.com/simple-evcorr/sec/releases/download/2.9.2/sec-2.9.2.tar.gz . Here is the changelog for the new version: --- version 2.9.2 * starting from

Re: [Simple-evcorr-users] Duplicate suppression and rearming

2023-03-16 Thread Risto Vaarandi
g now! > > > > Thanks a lot for the help, it was very precious! > > > > > > Best regards, > > M > > > > *Da:* Risto Vaarandi > *Inviato:* mercoledì 15 marzo 2023 19:20 > *A:* Spelta Edoardo > *Cc:* simple-evcorr-users@lists.sourceforge

Re: [Simple-evcorr-users] Duplicate suppression and rearming

2023-03-15 Thread Risto Vaarandi
> > >1. As long as we exclude the longer window expiration case (so we just >want to suppress events for 10 secs), why you suggest to use two Single >rules instead of a SingleWithSuppress ? > > > > I’m referring to these rules: > > type=Single > ptype=RegExp > pattern=event_(\w+) >

Re: [Simple-evcorr-users] Duplicate suppression and rearming

2023-03-14 Thread Risto Vaarandi
> Absolutely amazing! > it is great that I was able to help :) > > > @Risto Vaarandi I need some time to understand > deeply what you propose and if it’s applicable to hundreds of conditions, > but it sounds really promising !! > As a side note: this technique is a

Re: [Simple-evcorr-users] Duplicate suppression and rearming

2023-03-14 Thread Risto Vaarandi
should be suppressed, > since they are separated by less than 30 minutes, but the event at minute > 61 should trigger an action, since the suppression can't last for longer > than 1 hour > > Any ideas ? > > M. > > > > *Da:* Risto Vaarandi > *Inviato:* martedì 14 marz

Re: [Simple-evcorr-users] Duplicate suppression and rearming

2023-03-14 Thread Risto Vaarandi
hi Mugugno, let me clarify your scenario a bit, considering the diagram from your post: T1---T27---T30---T57-T60--T61 Eventsuppr supprsuppr suppr suppr Event Do you want

Re: [Simple-evcorr-users] SEC tutorial

2022-12-02 Thread Risto Vaarandi
> > SEC helping me reliably more than 6 years without any issues. > > It’s simple, reliable and well-documented. > > > > Thank you, > > Dusan > > > > *Od: *Risto Vaarandi > *Odoslané: *štvrtok 1. decembra 2022 11:25 > *Komu: *simple-evcorr-users@lists

Re: [Simple-evcorr-users] SEC tutorial

2022-12-01 Thread Risto Vaarandi
716 (Canada) > +1-866-644-7729 (USA) > +1-416-425-6111 x6001 > jim.vanmegge...@clearlycore.com > http://www.clearlycore.com > > > *Asterisk: The Definitive GuideFIFTH EDITION NOW AVAILABLE TO DOWNLOAD:* > https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf > >

[Simple-evcorr-users] SEC tutorial

2022-12-01 Thread Risto Vaarandi
hi all, during the last few weeks, I have written a new SEC tutorial paper which can be accessed through this web link: https://raw.githubusercontent.com/simple-evcorr/tutorial/main/SEC-tutorial.pdf Links to this tutorial and the relevant Github repository (

Re: [Simple-evcorr-users] add string to the event store of a context only if it doesn't exist

2022-11-06 Thread Risto Vaarandi
hi Sean, there are several ways to approach this problem and perhaps I will outline two possible ways below. Before adding a new ID to the context, one could search all existing IDs in the context with the SEC 'while' action, looping over all IDs one by one, and comparing each previously stored

Re: [Simple-evcorr-users] context w/ input file name embedded

2022-11-05 Thread Risto Vaarandi
hi Sean, Risto, > > > > Man I feel like an idiot. That’s what I get for copy / pasteing stuff > around. I removed the ; in the test setup and it’s working like I thought > it would. > > > > My real setup seems to be good now as well. > > > > Thanks a million. > It's great I was able to help

Re: [Simple-evcorr-users] context w/ input file name embedded

2022-11-05 Thread Risto Vaarandi
hi Sean, I was quite puzzled why the ruleset you have posted is not working, and after testing it several times and looking at the rules, I think I found the reason. When you look at the 'context' field of the second rule, there is an extra semicolon at the end of the context name (when the rule

[Simple-evcorr-users] new features introduced in SEC-2.9.1

2022-05-04 Thread Risto Vaarandi
hi all, this email provides an introduction to new features in the 2.9.1 version. Starting from the 2.9.0 version (released last year), EventGroup rules are supporting event group patterns which allow for matching specific event sequences within predefined time windows. For example, suppose you

[Simple-evcorr-users] SEC-2.9.1 released

2022-05-04 Thread Risto Vaarandi
hi all, SEC-2.9.1 has been released which is available from the SEC home page and through the following download link: https://github.com/simple-evcorr/sec/releases/download/2.9.1/sec-2.9.1.tar.gz Here is the changelog for the new version: --- version 2.9.1 * added support for 'egtoken*'

Re: [Simple-evcorr-users] Parsing Asterisk log files for downstream reporting - so far so good!

2022-04-01 Thread Risto Vaarandi
> +1-866-644-7729 (USA) > +1-416-425-6111 x6001 > jim.vanmegge...@clearlycore.com > http://www.clearlycore.com > > > *Asterisk: The Definitive GuideFIFTH EDITION NOW AVAILABLE TO DOWNLOAD:* > https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf > > ---

Re: [Simple-evcorr-users] Parsing Asterisk log files for downstream reporting - so far so good!

2022-03-31 Thread Risto Vaarandi
hi Jim, if you want to match the "ActivationHelp" event and react to the earliest "User entered" event, provided that these events share the same caller ID, you could use the following rule: type=PAIR desc=IVR caller $4 offered activation or statement inquiry ptype=RegExp action= write - $4

Re: [Simple-evcorr-users] Parsing Asterisk log files for downstream reporting - so far so good!

2022-03-31 Thread Risto Vaarandi
m > > > *Asterisk: The Definitive GuideFIFTH EDITION NOW AVAILABLE TO DOWNLOAD:* > https://cdn.oreillystatic.com/pdf/Asterisk_The_Definitive_Guide.pdf > > -- > > *From: *"Risto Vaarandi" > *To: *"Jim Van Meggelen" > *Cc: *simp

Re: [Simple-evcorr-users] Parsing Asterisk log files for downstream reporting - so far so good!

2022-03-31 Thread Risto Vaarandi
hi Jim, I do have couple of things in mind that might help addressing this issue, but before coming up with any suggestions, may I ask some questions? As I understand, the phone call is uniquely identified by the numeral that follows the C character (C-4037 in your example) which is present

[Simple-evcorr-users] update in the sec rule repository

2021-11-20 Thread Risto Vaarandi
hi all, just a small note -- the JSON parsing example in the sec rule repository has been updated: https://github.com/simple-evcorr/rulesets/tree/master/parsing-json Instead of the perl JSON wrapper module which will call some backend module, the updated example directly employs fast and

Re: [Simple-evcorr-users] An assist with a rule

2021-07-08 Thread Risto Vaarandi
> > Beautiful..thank you SO much Risto. I love the fact that you not only > provide something I can use right away, but took the time to explain how it > works. I learn more about SEC every time I email the list. > > Thanks again! > > James > It is great to hear that the examples from my post

Re: [Simple-evcorr-users] An assist with a rule

2021-07-08 Thread Risto Vaarandi
hi James, yes, you can employ the EventGroup rule for addressing this task, and let me provide two slightly different solutions below. The first and somewhat simpler solution looks like this: type=EventGroup ptype=RegExp pattern=^\d+\.\d+ \S+ ([\d.]+) \d+ ([\d.]+) 445 \d+\.\d+ pipelsass

Re: [Simple-evcorr-users] reopening inputfile inconsistently fails

2021-05-26 Thread Risto Vaarandi
hi Brian, let me provide my comments below in inline fashion: > > I've seen a log rotation where the input file did not get re-opened, and am > working on troubleshooting that. > > For the SEC process that failed, sending a SIGUSR2 failed, but sending a > SIGABRT worked. > (both sent as the

Re: [Simple-evcorr-users] SEC-2.9.0 released

2021-05-24 Thread Risto Vaarandi
ove it from there when it's uploaded to Debian. The package > also installs cleanly to current stable. > > -Jaakko > On Wed, May 12, 2021 at 8:41 PM Risto Vaarandi > wrote: > > > > hi all, > > > > SEC-2.9.0 has been released which is available from the SEC ho

[Simple-evcorr-users] SEC-2.9.0 released

2021-05-12 Thread Risto Vaarandi
hi all, SEC-2.9.0 has been released which is available from the SEC home page (use the following link for direct download: https://github.com/simple-evcorr/sec/releases/download/2.9.0/sec-2.9.0.tar.gz). Here is the changelog for the new version: * added support for 'cmdexec', 'spawnexec',

[Simple-evcorr-users] SEC-2.9.alpha2 released

2021-04-06 Thread Risto Vaarandi
hi all, SEC-2.9.alpha2 has been released which can be downloaded from: https://github.com/simple-evcorr/sec/releases/download/2.9.alpha2/sec-2.9.alpha2.tar.gz The download link has also been provided in the SEC home page. Compared to the 2.9.alpha1 version, this version introduces a number of

[Simple-evcorr-users] 20th birthday of SEC

2021-03-23 Thread Risto Vaarandi
hi all, on March 23 2001, SEC version 1.0 was released into public domain. I would like to take the opportunity to thank all SEC users for creative discussions in this mailing list during the last two decades. I would also like to thank all people who have suggested new features or supplied

Re: [Simple-evcorr-users] executing multiple actions

2021-03-16 Thread Risto Vaarandi
hi Stuart, I just saw a post with almost the same question as the previous one (perhaps it was posted before my answer reached your mailbox), and my apologies if information in this email is redundant. > > Correction -- this also produces the same error > > But this does not: > # - Radius

Re: [Simple-evcorr-users] executing multiple actions

2021-03-16 Thread Risto Vaarandi
hi Stuart, if you want to specify multiple actions for the 'action' field of the rule, semicolon should indeed be used as a separator. However, the 'action' keyword with an equal sign should appear just once in the beginning of the rule field definition. Therefore, the example rule from your post

[Simple-evcorr-users] new features in 2.9.alpha1 version

2021-03-13 Thread Risto Vaarandi
hi all, this email provides a more detailed description of major new features in SEC-2.9.alpha1. Firstly, one can use 'egptype' and 'egpattern' fields in EventGroup rule that specify an additional event group matching condition to conventional threshold conditions. The 'egptype' and 'egpattern'

[Simple-evcorr-users] SEC-2.9.alpha1 released

2021-03-12 Thread Risto Vaarandi
hi all, SEC-2.9.alpha1 has been released which is available for download from SEC home page (link for direct download: https://github.com/simple-evcorr/sec/releases/download/2.9.alpha1/sec-2.9.alpha1.tar.gz). This version is an alpha version of the upcoming 2.9 major release, and it introduces a

[Simple-evcorr-users] an update to SEC FAQ

2021-02-28 Thread Risto Vaarandi
hi all, for your information, the SEC FAQ has been updated with an example about matching input lines in UTF-8 and other encodings: https://simple-evcorr.github.io/FAQ.html#23 kind regards, risto ___ Simple-evcorr-users mailing list

Re: [Simple-evcorr-users] Use global variables

2020-12-16 Thread Risto Vaarandi
hi Agustin, Currently, there are no variables which could be set from one rule, and be accessible in *all* fields of other rules. However, the same action list variable can be accessed in all rules, but the use of action list variables is limited to action* rule fields only. kind regards, risto

Re: [Simple-evcorr-users] How can I trigger all outstanding context actions on SEC_SHUTDOWN?

2020-12-16 Thread Risto Vaarandi
...one additional note -- SEC official documentation on the 'while' action has other relevant examples about processing context event stores (you can find them in the end of the "ACTIONS, ACTION LISTS AND ACTION LIST VARIABLES" section of SEC man page:

Re: [Simple-evcorr-users] How can I trigger all outstanding context actions on SEC_SHUTDOWN?

2020-12-15 Thread Risto Vaarandi
> > For better testing, it would be cool if SEC's idea of the current time could > be derived from the timestamps in the log file instead of wall-clock time, so > that context actions happen at the right time relative to log messages > (rather than 30 seconds after the program ends! :-), but

Re: [Simple-evcorr-users] How can I trigger all outstanding context actions on SEC_SHUTDOWN?

2020-12-15 Thread Risto Vaarandi
hi Penelope, since 'obsolete' is a SEC action, it can not be called in Perl, but you rather need some sort of loop written in the SEC rule language. Fortunately, SEC supports the 'while' action that executes an action list as long as the given action list variable evaluates true in boolean

Re: [Simple-evcorr-users] using variables learned in rule A in rule B's perlfunc: possible?

2020-10-18 Thread Risto Vaarandi
hi Michael, thanks a lot for sharing examples from your rulebase! I am sure they will be helpful for people who have to tackle similar tasks in the future, and will be searching the mailing list for relevant examples. kind regards, risto Risto- > > > > Thank you for taking time to respond so

Re: [Simple-evcorr-users] using variables learned in rule A in rule B's perlfunc: possible?

2020-10-17 Thread Risto Vaarandi
hi Michael, there are a couple of ways to address this problem. Firstly, instead of using sec match variables, one can set up Perl's native variables for sharing data between rules. For example, the regular expression pattern of the first rule can be easily converted into perlfunc pattern, so

Re: [Simple-evcorr-users] Reset all rules

2020-10-08 Thread Risto Vaarandi
hi Agustin, if you want to reset the entire state of SEC (not just event correlation operations, but also contexts, action list variables and other data), you can use 'sigemul HUP' action. This action will emulate the reception of the HUP signal which is used to reset all internal state of SEC.

Re: [Simple-evcorr-users] Multiple Correlation Question

2020-08-06 Thread Risto Vaarandi
> Thanks for the answer. I am looking for window based detection, simple it > is going to be something like SIEM log correlation. Within 10 min event A,B > and C must occur and this three event must be in order (first A, then B > last C) > > Thanks > Suat Toksoz > > On

Re: [Simple-evcorr-users] Multiple Correlation Question

2020-08-05 Thread Risto Vaarandi
hi Suat, are you interested in some rule examples about detecting event sequences, or are you investigating opportunities for creating a new rule type for matching sequences of events? Many event sequences can be handled by combining existing rules and contexts, so a new rule type might not be

[Simple-evcorr-users] sec-2.8.3 released

2020-05-02 Thread Risto Vaarandi
hi all, SEC version 2.8.3 has been released, and here is the change log for the new version: * added support for collecting rule performance data, and the --ruleperf and --noruleperf command line options. * improved dump file generation in JSON format (some numeric fields that were reported as

Re: [Simple-evcorr-users] log files existence and accessibility

2020-04-08 Thread Risto Vaarandi
t; > Richard > > st 8. 4. 2020 o 12:05 Risto Vaarandi > napísal(a): > >> hi Richard, >> >> if you want to find input files which can not be opened because of >> permission issues, and want to conduct all checks specifically from SEC >> process without fo

Re: [Simple-evcorr-users] log files existence and accessibility

2020-04-08 Thread Risto Vaarandi
hi Richard, if you want to find input files which can not be opened because of permission issues, and want to conduct all checks specifically from SEC process without forking anything, I would recommend to set up an 'lcall' action that runs all checks from a Perl function. Since this function is

Re: [Simple-evcorr-users] RV: IP correlation with EventGroup

2020-04-06 Thread Risto Vaarandi
hi John, Hi Risto: > > ... > > > > >However, if you would like to suppress the output message that is > generated > >on 3rd input event and rather generate an output message "Events A , B and > >C observed for IP 1.1.1.1" on 5th input event, it is not possible to > >achieve that goal with

Re: [Simple-evcorr-users] RV: IP correlation with EventGroup

2020-04-06 Thread Risto Vaarandi
> > > However, if you would like to suppress the output message that is > generated on 3rd input event and rather generate an output message "Events > A , B and C observed for IP 1.1.1.1" on 5th input event, it is not possible > to achieve that goal with EventGroup (or any other) rules, since

Re: [Simple-evcorr-users] RV: IP correlation with EventGroup

2020-04-06 Thread Risto Vaarandi
hi Agustin, > Hi Risto, > > Thank you very much for your help. > I have another question related to this problem. > > Suppose we have the next entry in less than 60 seconds: > EVENT_TYPE_A 1.1.1.1 <--- the beginning of input for SEC > EVENT_TYPE_A 2.2.2.2 > EVENT_TYPE_B 1.1.1.1 >

Re: [Simple-evcorr-users] IP correlation with EventGroup

2020-04-05 Thread Risto Vaarandi
hi Agustin, I have tried the rule from your e-mail, and I am able to get the output you are expecting: /usr/bin/sec --conf=test4.sec --input=- SEC (Simple Event Correlator) 2.8.2 Reading configuration from test4.sec 1 rules loaded from test4.sec No --bufsize command line option or --bufsize=0,

Re: [Simple-evcorr-users] IP correlation with EventGroup

2020-04-05 Thread Risto Vaarandi
hi Agustin, and thanks for feedback! Instead of developing one rule which addresses all scenarios, it is better to write a separate rule for each case. For example, for the first case EVENT_TYPE_A && EVENT_TYPE_B the rule would look like this: type=EventGroup2 ptype=RegExp pattern=EVENT_TYPE_A

Re: [Simple-evcorr-users] IP correlation with EventGroup

2020-04-04 Thread Risto Vaarandi
hi Agustin, Hi Risto, > My name is Agustín, I'm working with the SEC and I have a problem that I > can't solve. > I have different events such as: > EVENT_TYPE_A FROM 1.1.1.1 > EVENT_TYPE_A FROM 2.2.2.2 > EVENT_TYPE_B FROM 1.1.1.1 > EVENT_TYPE_B FROM 2.2.2.2 > EVENT_TYPE_C FROM 1.1.1.1 >

Re: [Simple-evcorr-users] SEC CPU utilization

2020-04-03 Thread Risto Vaarandi
> > I mentiond as an offhand remark to Risto a profile mode that would > count not only every rule that lead to an action, but every time the > rule executed its regular expression. Having some sort of profile mode > (not to be run in production) would help identify these sorts of > issues. > > I

Re: [Simple-evcorr-users] SEC CPU utilization

2020-04-02 Thread Risto Vaarandi
hi Richard, > We were doing log monitoring migration from HPOM to open-source monitoring > tool, and using SEC for duplicate events flow reduction before passing to > monitoring agent, in the manner as HPOM agent with built-in correlations > was used, so the design of rules and correlations is

Re: [Simple-evcorr-users] action-list checking if log file is already open by SEC

2020-04-02 Thread Risto Vaarandi
mance? > > Richard > > št 20. 2. 2020 o 21:23 Risto Vaarandi > napísal(a): > >> hi Richard, >> >> I think this scenario is best addressed by creating a relevant SEC >> context when 'addinput' action is called. In fact, handling such scenarios >> is o

[Simple-evcorr-users] updates to SEC FAQ

2020-04-01 Thread Risto Vaarandi
hi all, SEC FAQ has received couple of updates: *) Q24 (https://simple-evcorr.github.io/FAQ.html#24) that describes the use of 'addinput' and 'dropinput' actions has been updated with a second example about tracking log files with timestamps in file names, *) new entry Q27

Re: [Simple-evcorr-users] Interesting command line behaviour

2020-03-04 Thread Risto Vaarandi
hi James, you are observing this behavior since --detach option involves changing working directory to root directory (that's a standard part of turning the process into a daemon). Actually, when you look into debug messages from SEC, there is also a message about directory change in the

Re: [Simple-evcorr-users] action-list checking if log file is already open by SEC

2020-02-20 Thread Risto Vaarandi
hi Richard, I think this scenario is best addressed by creating a relevant SEC context when 'addinput' action is called. In fact, handling such scenarios is one of the purposes of contexts, and here is an example rule which illustrates this idea: type=single ptype=regexp pattern=start monitoring

Re: [Simple-evcorr-users] How to introduce new match variable

2020-02-19 Thread Risto Vaarandi
hi Dusan, you can find my comments below: > > I try to add new variable using “context” and :> operator also using “lcall” action but no luck. > Any idea how to achieve this? > > This is what I have produced so far: > > Config file: dusko.sec > > rem=Rule 1 >

Re: [Simple-evcorr-users] how not to keep monitored files permanently open (not only) on NFS

2020-02-07 Thread Risto Vaarandi
hi Richard, In this context I am also curious, what would be the effect of using > --check-timeout / --poll-timeout, if the log file will be closed or remain > open during timeout... I am trying to find a way, how to use SEC in "close > after read" mode - used to use this mode in previous log

Re: [Simple-evcorr-users] how not to keep monitored files permanently open (not only) on NFS

2020-02-04 Thread Risto Vaarandi
hi Richard, I have never used SEC for monitoring files on NFS file systems, but I can provide few short comments on how input files are handled. After SEC has successfully opened an input file, it will be kept open permanently. When input file is removed or renamed, input file is still kept open

Re: [Simple-evcorr-users] SEC rules performance monitoring and tuning

2020-02-04 Thread Risto Vaarandi
ar version of SEC. > > Richard > > ut 28. 1. 2020 o 12:40 Risto Vaarandi > napísal(a): > >> hi Richard, >> >> as I understand from your post, you would like to create SEC dump files >> periodically, in order to monitor performance of SEC based on thes

Re: [Simple-evcorr-users] SEC rules performance monitoring and tuning

2020-01-28 Thread Risto Vaarandi
hi Richard, as I understand from your post, you would like to create SEC dump files periodically, in order to monitor performance of SEC based on these dump files. Let me first provide some comments on performance related question. Essentially, creation of dump file involves a pass over all major

Re: [Simple-evcorr-users] SEC + AI (machine learning)

2020-01-23 Thread Risto Vaarandi
hi Richard, Next step would be integrating AI (machine learning) with SEC somehow, so > that user won't need to configure correlations statically, but they would > configure and self-optimize automatically. (There still could be some input > needed from the user, but system would be also able to

Re: [Simple-evcorr-users] using original message in TValue rule action

2020-01-20 Thread Risto Vaarandi
hi Richard, there are several pattern types like TValue and SubStr which have been designed for fast matching and which do not support match variables (including $0). Handling of match variables involves additional computational cost, since after successful match, all variables in rule definition

Re: [Simple-evcorr-users] Public Docker image for SEC

2019-12-16 Thread Risto Vaarandi
hi Andres, so far, official sec distribution has not had a docker image, since sec is packaged for common linux and bsd distributions, and it doesn't have many dependencies (just standard perl is needed without any exotic modules). That has made sec very easy to deploy. I had a quick look into the

Re: [Simple-evcorr-users] RegExp modifiers

2019-12-09 Thread Risto Vaarandi
hi Richard, Kontakt Richard Ostrochovský () kirjutas kuupäeval E, 9. detsember 2019 kell 01:57: > Hello colleagues, > > I was searching for the answer here: > https://simple-evcorr.github.io/man.html > https://sourceforge.net/p/simple-evcorr/mailman/simple-evcorr-users/ > and haven't found the

Re: [Simple-evcorr-users] generate SEC configurations automatically

2019-12-02 Thread Risto Vaarandi
> > > This SEC admin, as I see it, is still also from user perspective, tightly > bound with SEC technology and user needs to know, how SEC rules and their > compositions work internally. I am imagining something, that is > technology-agnostic, just describing logic of correlations, and from that

Re: [Simple-evcorr-users] generate SEC configurations automatically

2019-12-01 Thread Risto Vaarandi
hi Richard and John, Hi Richard: > > In message > , > Richard_Ostrochovsk writes: > >this post loosely follows this one: > >https://sourceforge.net/p/simple-evcorr/mailman/message/36867007/. > > > >Being monitoring consultant and developer, I have > >an idea to hide complexity of SEC

Re: [Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-28 Thread Risto Vaarandi
hi Richard, just one followup thought -- have you considered sec native multi-line patterns such as RegexpN for handling multi-line logs? Of course, there are scenarios where the value of N (max number of lines in a multi-line event) can be very large and is difficult to predict, and for such

Re: [Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-27 Thread Risto Vaarandi
hi Richard, Risto, thank you for your pre-analysis about multi-lines with regexp, and > also for suggestions about multi-files yet more sophisticated solution. > > My comments are also inline: > > st 27. 11. 2019 o 15:07 Risto Vaarandi > napísal(a): > >> hi Richard, &

Re: [Simple-evcorr-users] "multi-line" and multi-file logs - out of box

2019-11-27 Thread Risto Vaarandi
hi Richard, these are interesting questions and you can find my comments inline: Hello guys, > > ... > > My question is, if you see, how some of this things could be accomplished > in more generic way, without special configurations of correlation rules. > It would be great having SEC

Re: [Simple-evcorr-users] Pass variable to second rule - json pattern

2019-11-26 Thread Risto Vaarandi
hi Andres, the %user action list variable gets indeed overwritten if multiple deployments for different services are ongoing simultaneously. However, you can utilize DEPLOY_STARTED_ context for storing the user name for the given service (provided that you are not using this context already for

Re: [Simple-evcorr-users] Close input file if there are now events after specified duration

2019-10-04 Thread Risto Vaarandi
hi Alberto, if the input file has been provided to SEC with --input command line option, there is currently no way to close it. (It is only possible to increase the status polling time frame for input files with --check-timeout option, in order to reduce the system load if there is a large number

Re: [Simple-evcorr-users] Accessing A Perl Hash From Pattern1 In Pattern 2

2019-10-04 Thread Risto Vaarandi
hi David, I would second to Rock's recommendation to use regular expressions in the Pair rule. Firstly, two PerlFunc patterns implement only regular expression matching and there isn't anything additional (such as arithmetic operations) which would require the use of Perl. Therefore, it is easier

Re: [Simple-evcorr-users] Maintaining events while modifying rules

2019-09-09 Thread Risto Vaarandi
hi David, restarting SEC means that all event correlation operations executed by previous instance will be lost. Although it is possible to write custom rules for storing SEC contexts to disk at shutdown and load them at restart (e.g., see http://simple-evcorr.github.io/FAQ.html#15), there is no

Re: [Simple-evcorr-users] (no subject)

2019-09-04 Thread Risto Vaarandi
method works exactly as >> expected. >> >> This resolves many of my other queries. Thank you for prompt response. >> >> Regards, >> Santhosh >> >> >> On Fri, Aug 30, 2019, 20:59 Risto Vaarandi >> wrote: >> >>> hi Santhosh, >>&

Re: [Simple-evcorr-users] spawn timeout and exit code

2019-09-03 Thread Risto Vaarandi
code: (\d+)$ context=TESTPROG desc=catch the exit code of command action=logonly Command has terminated with exit code $1 I have also updated the relevant FAQ entry with an example involving the 'timeout' tool: http://simple-evcorr.github.io/FAQ.html#20 kind regards, risto Kontakt Risto Vaaran

Re: [Simple-evcorr-users] (no subject)

2019-08-30 Thread Risto Vaarandi
6353 for outside: > 187.189.195.208 <http://187.189.195.208:8443/>/24057 to > > identity: 172.18.124.136/161 duration 0:02:01 bytes 313" > > > regards, > Santhosh > > On Wed, Aug 28, 2019 at 4:18 AM Risto Vaarandi > wrote: > >> hi Santhosh, >> &

Re: [Simple-evcorr-users] Help calling perl to get hostname

2019-08-29 Thread Risto Vaarandi
hi Clayton, Also, for completeness, here’s what worked for our user. > > This rule detects a BGP routing tunnel going down. The rule then waits for > a matching "Up" event from the same host with the same neighbor IP. Once > detected, it then measures the total time the route was down and

Re: [Simple-evcorr-users] Help calling perl to get hostname

2019-08-28 Thread Risto Vaarandi
hi Clayton, for testing purposes, I have separated the problematic part of the rule into a simple Single rule. As you can see, the rule calls the 'eval' action for resolving an IP address and then writes it to standard output with 'write' action: type=single ptype=regexp pattern=neighbor

Re: [Simple-evcorr-users] (no subject)

2019-08-27 Thread Risto Vaarandi
hi Santhosh, Kontakt Santhosh Kumar () kirjutas kuupäeval T, 27. august 2019 kell 04:55: > Hello Risto > > > I’ve been running tests on SEC for a while and stuck with below points. > I’m not familiar with Perl though I tried to find a solution from sec mail > bucket but no luck, please suggest

Re: [Simple-evcorr-users] spawn timeout and exit code

2019-07-29 Thread Risto Vaarandi
hi Pedro, these are interesting questions. As for fetching the exit code from spawn, SEC does that and produces a warning message if it is non-zero, but there is no equivalent to bash $? variable. Firstly, command lines are executed asynchronously by spawn, and secondly, many such command lines

Re: [Simple-evcorr-users] EVENTGROUP RULE and match variables

2019-06-28 Thread Risto Vaarandi
hi Jia, thanks for an interesting question! SEC match variables are set to new values after each pattern match and they don't have any persistence over several matches. Since there is only one capture group in each regular expression of your example rule, all four patterns are setting the $1

[Simple-evcorr-users] sec-2.8.2 released

2019-06-02 Thread Risto Vaarandi
hi all, today, SEC-2.8.2 has been released which can be downloaded from: https://github.com/simple-evcorr/sec/releases/download/2.8.2/sec-2.8.2.tar.gz Here is the changelog for the new version: --- version 2.8.2 * added support for 'varset' action. * fixed a bug where reference to

Re: [Simple-evcorr-users] Correlation Upon Aggregation

2019-05-13 Thread Risto Vaarandi
hi Santhosh, since you are using SingleWithSuppress rule for aggregation, is my understanding correct that the term "aggregation" means generating a syslog message on the first matching event, suppressing the following matching events during 300 seconds? If so, you don't need the PairWithWindow

Re: [Simple-evcorr-users] Question on rule

2019-03-18 Thread Risto Vaarandi
> action = pipe '%s' /usr/local/bin/sendEmail > > Can I add a second action to this? Thank you! > > James > > On 2019-03-18 08:11, James Lay wrote: > > Wow thanks so much RistoI love the way you actually explain what's > > going on...really appreciate it! > &

Re: [Simple-evcorr-users] Question on rule

2019-03-16 Thread Risto Vaarandi
hi James, for addressing this problem, you could try the following EventGroup rule: type=EventGroup ptype=RegExp pattern=^\S+\s+\S+\s+((?:\d{1,3}\.){3}\d{1,3})\s+\d+\s+(?:\d{1,3}\.){3}\d{1,3}\s+88\s+AS\s+(\S+)\s+(\S+)\s+F\s+KDC_ERR_PREAUTH_FAILED context=!WORKSTATION_$1_LOGIN_FAILURE_$2 &&

[Simple-evcorr-users] an update to sec FAQ

2019-01-30 Thread Risto Vaarandi
hi all, sec FAQ has been updated with a new entry about setting up a control file or fifo for issuing commands to sec: http://simple-evcorr.github.io/FAQ.html#26 The rule example under the new entry should be particularly useful if the OS platform does not support signals natively (e.g.,

Re: [Simple-evcorr-users] list of suppress events

2018-12-20 Thread Risto Vaarandi
lps, risto Kontakt Kagan, Eli () kirjutas kuupäeval N, 20. detsember 2018 kell 20:38: > Thanks Risto. I’ll give it a try. > > > > Are you keeping user defined perl variables in a separate namespace? > > > > -- ek > > > > *From:* Risto Vaarandi > *Sent:* T

Re: [Simple-evcorr-users] list of suppress events

2018-12-18 Thread Risto Vaarandi
hi Eli, if you would like to have regular expressions stored in an external file and load them at startup and restarts, you could use the following ruleset. The first rule loads patterns from a file when sec is started or has received HUP or ABRT signal. The rule assumes that each line contains a

Re: [Simple-evcorr-users] help

2018-12-06 Thread Risto Vaarandi
hi Graeme, your posting is apparently empty -- can you re-post your question? risto Kontakt Graeme Danielson () kirjutas kuupäeval N, 6. detsember 2018 kell 05:05: > > > > > -- Graeme Danielson tel:+64-21-611345 <+64-21-611345> UTC+13 > > > > Good planets are hard to find - please think of the

Re: [Simple-evcorr-users] SingleWithThreshold reference current input line

2018-11-08 Thread Risto Vaarandi
hi Dusan, the problem lies in the fact that when SingleWithThreshold rule starts a counting operation, match variables in the 'action' field receive their values from the first event which triggered that operation (that is done for staying consistent with substitution of variables in other

Re: [Simple-evcorr-users] Suppress rule and continue filed support

2018-10-14 Thread Risto Vaarandi
h a new entry: http://simple-evcorr.github.io/FAQ.html#25 kind regards, risto Thank you, >> >> Dusan >> -- >> *Od:* Risto Vaarandi >> *Odoslané:* piatok, 12. októbra 2018 11:25 >> *Komu:* dusan.so...@hotmail.sk >> *Kópia:* sim

Re: [Simple-evcorr-users] Suppress rule and continue filed support

2018-10-12 Thread Risto Vaarandi
ation accordingly. risto I like idea to add this as separate FAQ entry. > > > > Thank you, > > Dusan > -- > *Od:* Risto Vaarandi > *Odoslané:* piatok, 12. októbra 2018 11:25 > *Komu:* dusan.so...@hotmail.sk > *Kópia:* simple-evcorr-us

Re: [Simple-evcorr-users] Suppress rule and continue filed support

2018-10-12 Thread Risto Vaarandi
> Thanks for this great piece of software and I really appreciate your > support and help. > I am happy that you like sec and have found it useful :-) kind regards, risto > Dusan > > -- > *Od:* Risto Vaarandi > *Odoslané:* štvrtok, 11. októ

Re: [Simple-evcorr-users] Suppress rule and continue filed support

2018-10-11 Thread Risto Vaarandi
> Hello SEC Users, > > > hi Dusan, Base on SEC documentation *Suppress* rules doesn’t support “continue” field > like other rules. > > My understanding is that if suppress rule match event the search for > matching rules ends in the *current* configuration file. > That's correct, the Suppress

  1   2   3   4   5   6   >