Tristan,
You have to be careful with this script: The problem is that each month only
contains the collected stats of each day. This means the data at the bottom of
each time report may not be accurate (in fact all the numbers could be
inaccurate). When you produce the reports, Analog cuts off the bottom of the
report at the floor. Some items may never make the reports that should, others
could be miscounted. Here are two examples to highlight it:
Say that the bottom of the request report, every day of the month is at about 1.5
million requests. A particular file, sales.html, has about 1.4 million requests
per day. It never makes the cut off. Now, assume there's a lot of different
activity on the site. Many of the pages get 2 million requests some days and 200
request others. At the end of the period, the top 200 requests in your stats could
show some of these latter items, which may have only totaled 3 million requests,
yet sales.html, with a total of 43.4 million requests is never listed (though it
should be) 'cause it was never in any daily report.
Next, assume you've been running a service at program/login.cgi. At the end of the
month, your ad campaign finally runs and tells people about that file. While the
rest of the month this was only getting 800 thousand requests a day, in the first
day after the campaign runs, this reaches 10 million requests. The next two days
(before the end of the month) this logs 8 million requests a day. The last three
days it is listed in the daily reports and totals in your report at 26 million, so
it doesn't make the month-end report (which cuts off at 30 million). But really,
it's been getting hits all along. The actual amount should be 48.4 million. That's
a big difference to marketing!
So, I'm not saying don't use the script. Just be certain you understand the
limitations and implications. This may be the only solution for huge logfiles like
you both have. Of course might be able to use something like Marco's script (or a
set of equivalent Analog configs) in the separate, concurrent thread "Unique user
counts per request" to report on parts of the logs that you need.
HTH,
Jeremy Wadsack
Wadsack-Allen Digital Group
Tristan Lawrence wrote:
> I have attached a perl script I used to get around a similar problem
> (although I only had 1.5 GB of memory not 14 Gb!!!)
>
> It runs each day seperately and combines the output.
>
> Tristan
>
> Fiorela Gonzalez wrote:
> >
> > I am running analog on a machine that has 14G of memory, disk space a couple
> > of tarabytes. I am trying to run a monthly report (just for march) and i
> > keep running out of memory. I get this message from the my error log. And
> > Yes I have set hostlowmem to 3 and filelowmem to 3. I also have cache files.
> > A daily takes about 30 min to 45 min to run. A weekly takes about 2 hours to
> > run (this is with hostlowme 2 and filelowmem 2. So monthly usually dies at
> > around 3 hours or so. Any suggestions?
> >
> > Fiorella Gonzalez
> > Xdrive.com
> > Intranet/Stat Manager
>
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/analog-help@lists.isite.net/
------------------------------------------------------------------------
