On Sat, 19 Oct 2013, Risto Vaarandi wrote: >> >> There have been some ideas about how to make SEC operate in >> non-realtime mode. They would involve some extensive changes to how >> SEC's main loop and time processing works. Because of the difficulty >> of changes nobody has tried to do them. >> > > Indeed, and this problem has also been discussed in the context of taking > timestamps from incoming events during real-time mode, in order to use them > during event correlation. At first glance it looks quite simple, but > digging deeper will reveal fairly complex issues. Suppose you have an event > which comes in with one minute old timestamp. It would be fairly > straightforward to consume this event if it only triggers an external > program, and we are willing to tolerate 1 minute delay. Also, if we only > let the event influence currently ongoing counting operations, it becomes > just a matter of incrementing event counters (and triggering an action if > threshold was reached). > The problem becomes much harder if we take into account that in SEC event > correlation entities (rules, operations, contexts, natural and synthetic > events etc.) can influence each other, and the (non)presence of some entity > can direct event correlation process to a completely different path. For > example, if an event with 1 minute old timestamp suddenly appears which > should have created a context for disabling some rules, we must rematch the > rulebase against the input of the last minute for achieving as much > precision as possible. Unfortunately, that would be extremely expensive and > the results of some mistakenly executed actions could still not be fully > altered.
I don't think that it's sane to try and do this. Just use the timestamp from the current message to set expiration times and to check against expiration times. This does mean that you have to deal with different timezones, normalizing them first, but beyond that I think it's very reasonable to say that SEC requires that your times be consistent, and that it will make reasonable attempts at doing the right thing if they aren't, but re-processing an arbitrary amount of old log data (most of which SEC will no longer have access to) if an old timestamp arrives is not reasonable. David Lang ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
