2011/9/10 Joe Prosser <[email protected]>: > If I have performance stats like these: > > Performance statistics: > ============================================================ > Run time: 185641 seconds > User time: 172700.86 seconds > System time: 1852.9 seconds > Child user time: 6.56 seconds > Child system time: 9.26 seconds > Processed input lines: 14345391 > > Do I have a problem?
One thing I noticed that for some reason, this SEC instance is consuming a lot of CPU time -- for 185641 seconds it has worked, it has consumed 174553.76 seconds of CPU time. In other words, for 94% of the execution time it has used a CPU, which is a sign that it has more computations to do than it can handle (in my own installations, this figure has rarely exceeded 20-30%). Interestingly enough, the event rate per second is 14345391 / 185641 = 77 which isn't a huge load, since on reasonable hardware SEC should be able to handle at least 500-1000 events per second. This is leading me to think that you might either have a lot of rules (perhaps several thousand?), or there could be a regular expression in one of your rules that has a high computational complexity and could be simplified. Are your expressions fairly simple, or do they have some advanced constructs? One thing to look out for are nested quantifiers like ( +)*, ( *)+ and others -- they are known to have exponential complexity... However, if you have large amount of rules, can you segment them somehow with Jump rules? hope this helps, risto > > On Sat, Sep 10, 2011 at 3:12 PM, Risto Vaarandi > <[email protected]> wrote: >> Just out of curiosity -- if SIGUSR1 signal is sent to the sec process >> for creating the dump file (tmp/sec.dump), what does the "Performance >> statistics" section look like in the dump? Look at the "Run time", >> "User time", "System time", "Processed input lines" readings -- the >> figures tell you how many lines per second sec has processed and how >> much CPU time has been used for this. If these readings are very high, >> the input needs to be divided somehow between multiple sec instances, >> or irrelevant events (if there are any) need to be filtered out with >> some preprocessing tool (e.g., logpp). >> regards, >> risto >> >> 2011/9/9 Joe Prosser <[email protected]>: >>> Hi Folks, >>> I have an extremely busy sec process running with 17675 active >>> contexts. I'm finding that I need to kill -ABRT the process every >>> hour or so or else it hogs the CPU and lags in reading the input file. >>> I know the number of active contexts has been growing, but its been >>> running for over a year with at least 15k of them. >>> >>> Has anyone else had this experience or know what else can be done? >>> >>> Cheers, >>> -Joe >>> >>> ------------------------------------------------------------------------------ >>> Why Cloud-Based Security and Archiving Make Sense >>> Osterman Research conducted this study that outlines how and why cloud >>> computing security and archiving is rapidly being adopted across the IT >>> space for its ease of implementation, lower cost, and increased >>> reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/ >>> _______________________________________________ >>> Simple-evcorr-users mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users >>> >> > ------------------------------------------------------------------------------ Malware Security Report: Protecting Your Business, Customers, and the Bottom Line. Protect your business and customers by understanding the threat from malware and how it can impact your online business. http://www.accelacomm.com/jaw/sfnl/114/51427462/ _______________________________________________ Simple-evcorr-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
