>"Robert Brenstein" <[EMAIL PROTECTED]> wrote: > >> It would be nice if Analog reported always the progress lines >> from the PROGRESSFREQ command to the screen even if the ERRFILE >> directs debugging output to a file. I am processing logs with >> close to 1,000,000 entries and am capturing DNS resolves and >> corrupt lines into the file but by doing so I seem to have no >> means to watch program's progress. I am using Mac version 5.21 >> if that matters. > >I've been using Analog for years, and I never even knew there was a >PROGRESSFREQ directive until now, because Analog is so fast that I've >never needed it. (I just did a quick test, and analysed a logfile of >1,000,000 entries in less than 3 minutes, on a 600Mhz PIII. Of course, >that's without doing DNS translation - I'm not even going to try it with >Analog doing DNS translation, because I know it will take hours. But I can >resolve the 25,000 hosts in that log file in about 10 or so minutes using >a helper app. Have you tried to use DNSTrans with your logs?)
What you are saying is true but even without DNS resolution (using just cache) a typical run lasts 7-8 mins. I failed to mention that while a single-month log is about 1 mln entries but I run the 12-months analyses. You say it takes you 3 mins for a 1-mln-entries file (1 month). If you ran the same over 12 months to produce an annual report that would be 3x12 = 36 mins. Probably less since there is some start/end overhead, so let's less say 24 mins (since I run these in 7-8 mins, I am not doing so bad with my G3/300). The problem is that some of the analyses are running out of memory (I have only 256-mb machine), so I have to effectively produce only a single "report" in each run. This means that I have to Analog quite a few times to get what I need. While Analog is usually quite good in noticing low-memory situation and exiting cleanly, I get ocassional hangups and freezes (I do use other programs in the meantime) and I can't easily say whether the program is doing some analyses (I observe some hangups when it is producing the actual report) or whether there is a problem. I am only now getting to the organization and host level analyses, so I may need to activate some of the lowmem options which is my understanding will extend the run times even further. >I don't know of any way to do what you're actually asking for, because I >presume that Analog simply switches the value of STDOUT when you set the >ERRFILE directive. > >Aengus Yes, I don't see a command to do it, hence I am asking to change the behavior as the default or as an option. I realize this requires some changes in the code but it should not be a major rewrite and the (notably small) group of users like me would greatly appreciate that. You have to agree that writing progress info to a disk file makes no sense. Robert +------------------------------------------------------------------------ | This is the analog-help mailing list. To unsubscribe from this | mailing list, go to | http://lists.isite.net/listgate/analog-help/unsubscribe.html | | List archives are available at | http://www.mail-archive.com/analog-help@lists.isite.net/ | http://lists.isite.net/listgate/analog-help/archives/ | http://www.tallylist.com/archives/index.cfm/mlist.7 +------------------------------------------------------------------------
