[analog-help] Running out of memory

2000-05-31 Thread Josue Aguilar
Hi.!! I have a problem, frecuently when I'm running analog appear this error: ./analog: Fatal error: Ran out of memory: cannot continue: exiting Then I must add this lines in the analog.cfg: HOSTLOWMEM 3 USERLOWMEM 3 FILELOWMEM3 REFLOWMEM

Re: [analog-help] Running out of memory

2000-05-31 Thread josef . dyma
Hi Josue, how about some more info about how big log files are you analyzing etc...? Thanks, Josef D. On Wed, 31 May 2000, Josue Aguilar wrote: Hi.!! I have a problem, frecuently when I'm running analog appear this error: ./analog: Fatal error: Ran out of memory: cannot continue: exiting

Re: [analog-help] Running out of memory

2000-05-31 Thread Joshua P. Cooley
[EMAIL PROTECTED] wrote: how about some more info about how big log files are you analyzing etc...? Also the OS you are running under would help. If you're running under OpenVMS you may be exceeding your account quotas or the pagefile may not be large enough for what you are trying to do.

Re: [analog-help] Running out of memory

2000-05-31 Thread Jeremy Wadsack
Josue Aguilar wrote: Hi.!! I have a problem, frecuently when I'm running analog appear this error: ./analog: Fatal error: Ran out of memory: cannot continue: exiting Then I must add this lines in the analog.cfg: HOSTLOWMEM 3 USERLOWMEM 3 FILELOWMEM3 REFLOWMEM 3 But I don't know why

RE: Re: [analog-help] Running out of memory

2000-05-31 Thread Josue Aguilar
Hi.! well my log files have an average size of 700Mb (each day counting all servers). Thanks.!!! Hi Josue, how about some more info about how big log files are you analyzing etc...? Thanks, Josef D. On Wed, 31 May 2000, Josue Aguilar wrote: Hi.!! I have a problem, frecuently when I'm

Re: [analog-help] Running Out Of Memory

2000-04-10 Thread Stephen Turner
On Mon, 10 Apr 2000, Jeremy Wadsack wrote: Stangely enough (and maybe Stephen can shed light on this) is that the domain report, which was included in all runs, had different number with all LOWMEMs at 3 than with HOSTLOWMEM 2. Shouldn't the count be the same if the report is going to be

Re: [analog-help] Running Out Of Memory

2000-04-10 Thread Jeremy Wadsack
I just wanted to follow-up on this thread for anyone else looking at processing large logfiles. In testing Analog on the stats files, *LOWMEM 3 was able to run within the memory space of the server. We subsequently ran each of the LOWMEM options at 2, one at a time. Then used a script to collect

Re: [analog-help] Running Out Of Memory

2000-04-05 Thread Stephen Turner
On Tue, 4 Apr 2000, Tristan Lawrence wrote: I think Analog is a wonderful program, but for whatever reason it uses an absolutely ridiculous amount of memory doing large reports like this. My script unquestionably takes a long time to run (hours) but is dramatically more memory-efficient

Re: [analog-help] Running Out Of Memory

2000-04-05 Thread Tristan Lawrence
Hi Stephen I think that is it. This site (nypost.com) typically has 5-10 million hits a day-on over 2.5 million distinct files. I can't imagine how many distinct files in a month but it must be a huge number. So Analog would need gigabytes and gigabytes of RAM to keep track of all of those

[analog-help] Running Out Of Memory

2000-04-04 Thread Fiorela Gonzalez
I am running analog on a machine that has 14G of memory, disk space a couple of tarabytes. I am trying to run a monthly report (just for march) and i keep running out of memory. I get this message from the my error log. And Yes I have set hostlowmem to 3 and filelowmem to 3. I also have cache

Re: [analog-help] Running Out Of Memory

2000-04-04 Thread Tristan Lawrence
I have attached a perl script I used to get around a similar problem (although I only had 1.5 GB of memory not 14 Gb!!!) It runs each day seperately and combines the output. Tristan Fiorela Gonzalez wrote: I am running analog on a machine that has 14G of memory, disk space a couple of

Re: [analog-help] Running Out Of Memory

2000-04-04 Thread Jeremy Wadsack
Tristan, You have to be careful with this script: The problem is that each month only contains the collected stats of each day. This means the data at the bottom of each time report may not be accurate (in fact all the numbers could be inaccurate). When you produce the reports, Analog cuts off

Re: [analog-help] Running Out Of Memory

2000-04-04 Thread Tristan Lawrence
Good point, and thanks for your comments-not to mention the lucid example! I'm not sure that would be a problem, though. As I recall (I wrote this a while ago) the script collects all of the stats for a particular day; it doesn't limit them at all. The filtering takes place when it outputs.

Re: [analog-help] Running Out Of Memory

2000-04-04 Thread Jeremy Wadsack
Tristan Lawrence wrote: I'm not sure that would be a problem, though. As I recall (I wrote this a while ago) the script collects all of the stats for a particular day; it doesn't limit them at all. The filtering takes place when it outputs. Well, Analog does filtering by default and the

Re: [analog-help] Running Out Of Memory

2000-04-04 Thread Tristan Lawrence
Jeremy Wadsack wrote: Well, Analog does filtering by default and the script doesn't 'turn off' the floors. The default requests floor compiled in (admittedly I have an older version of Analog) is only 20 requests, and as you noted below I override the directory floor. Regardless, an excellent

RE: [analog-help] Running Out Of Memory

2000-04-04 Thread Fiorela Gonzalez
analog on cache files this big? Fiorella -Original Message- From: Tristan Lawrence [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 04, 2000 3:10 PM To: [EMAIL PROTECTED] Subject: Re: [analog-help] Running Out Of Memory I have attached a perl script I used to get around a similar problem