The hacker requests are in the hundreds, but they are still only a tiny percentage.
I ran with PROGRESSFREQ, and it turns out that on our server, Analog is taking about 80-100 seconds to read every million lines. It looks to be CPU-bound (75-85% CPU running at low priority). Removing my REQALIAS REGEXP lines (about 15 of them) speeded it up to about 50s/ML, so it looks like my aliases account for about half the slowdown. (Our database-driven site has a lot of scripts, so I really want to strip out some of the excess parameters). I could get further, to below 30s/ML if I took out FILELOWMEM 1 -- but then some of my attempts ran out of memory, so no good for now. With REGEXP but without FILELOWMEM the results were around 40s/ML. So ... definitely should add memory. CPU would be nice, but not a simple upgrade for that. But I'm going to adjust the reporting time to around 2 AM (from midnight) to make sure that I get a quiet time of day. Jean-Christian reported 30 minutes for 8 GB. If I assume 100 bytes per line (about the average for us), then it's 1800 seconds for 80 ML, or a little over 20s/ML. That makes sense given that he's running on a fast P4 system with lots of memory, surely without FILELOWMEM, and quite possibly without too many REGEXP lines. Something to strive for, I suppose ... Tom -----Original Message----- From: Henk Schrik [mailto:[EMAIL PROTECTED]] Sent: Friday 28 June 2002 19:36 To: [EMAIL PROTECTED] Subject: RE: [analog-help] Analog performance Or thinking about what you want to record in your logfiles in the first place, say, redefining your logfile, setting up your logfile differently. Look at the amount of mistrials you will find in your logfiles by those automatic hackers, mistrials caused by /default.ida /script/....... /cmd.exe /root.exe etc. etc. Fore some time it might be interesting for system managers to see what happens to their servers, to also analyse the 'dirt' in the logfiles, but after a while it gets annoying, if you realize that not having this dirt in your logfile might save you 1/3 of the size of your logfiles. Unless leaving it out in the analog process doesn't cost extra processing time........ Henk Schrik tel. (31)(0)6 53612294 e-mail: [EMAIL PROTECTED] website: http://henk.schrik.nl or http://henk.schrik.org +------------------------------------------------------------------------ | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at | http://www.mail-archive.com/analog-help@lists.isite.net/ | http://lists.isite.net/listgate/analog-help/archives/ | http://www.tallylist.com/archives/index.cfm/mlist.7 +------------------------------------------------------------------------ +------------------------------------------------------------------------ | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at | http://www.mail-archive.com/analog-help@lists.isite.net/ | http://lists.isite.net/listgate/analog-help/archives/ | http://www.tallylist.com/archives/index.cfm/mlist.7 +------------------------------------------------------------------------
