Re: (RADIATOR) Logfile size
On 6/3/99 at 7:43 AM Mike McCauley wrote: >Hi Ryan, > >On Jun 2, 11:51am, ryanm wrote: >> Subject: (RADIATOR) Logfile size >> Hello everyone, >> >> I am curious what size my daily logfiles will be with 1+ logins >> a day. I am doing some capicity planning and want to add a couple >> disks to store logging info on. I would appreciate any average >> sizes you have. I have looked at the entries in the detail log > >That logic seems quite reasonable. I have seen similar numbers elsewhere. I >would certainly advocate compression too. IMHO, compression has a major drawback: it make the files difficult to search in. If you want to give your users a way to check their connection statistics online, this is a problem. Note that even without compression, making an fgrep in a 250MB file to get the statistics for a given login out of it can be a killer for the server, specially of you have lots of users making statistics requests through the web. But... without going as far as putting everything in a database (yeah, I know I can be a PITA with my database stuff :)), there's a way to compress data and let it 'searchable': just pass it through a filter and put everything in a more compact format (for example, change plain text dates to a Unix timestamp, ...). I think part of this can already be done by a proper configuration of the logging options of Radiator, but I can't advice on that as I don't use text logging. Stephan * ** Stephan Forseilles - Operations Director ** Belgacom Skynet SA/NV ** Email [EMAIL PROTECTED] ** Tel: +32 2 706 13 11 Fax: +32 2 726 93 11 * ÿ Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
Re: (RADIATOR) Logfile size
On 6/2/99 at 4:54 PM Jason Godsey wrote: >-rw-r- 1 root wheel 15470156 Feb 28 23:59 detail.1999.02.gz >-rw-r- 1 root wheel 17094556 Mar 31 23:59 detail.1999.03.gz >-rw-r- 1 root wheel 17061311 Apr 30 23:59 detail.1999.04.gz >-rw-r- 1 root wheel 257358389 May 31 23:59 detail.1999.05 >-rw-r- 1 root wheel 14702947 Jun 2 16:46 detail.1999.06 > >We have 2.5k dialup users. Looks like about 8megs/day. Looks like you'll >need much more than 2 20gig drives to keep a years worth :) (you'll need >that for just 2 months) Each month will eat nearly 30megs if our numbers >scale. If you use a DB (datawarehouse type) you can purge all unnecessary data and only keep what's vital in a compacted format. We have 120K diakup users and have something like 6GB for 7 months. Stephan * ** Stephan Forseilles - Operations Director ** Belgacom Skynet SA/NV ** Email [EMAIL PROTECTED] ** Tel: +32 2 706 13 11 Fax: +32 2 726 93 11 * ÿ Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
Re: (RADIATOR) Logfile size
-rw-r- 1 root wheel 15470156 Feb 28 23:59 detail.1999.02.gz -rw-r- 1 root wheel 17094556 Mar 31 23:59 detail.1999.03.gz -rw-r- 1 root wheel 17061311 Apr 30 23:59 detail.1999.04.gz -rw-r- 1 root wheel 257358389 May 31 23:59 detail.1999.05 -rw-r- 1 root wheel 14702947 Jun 2 16:46 detail.1999.06 We have 2.5k dialup users. Looks like about 8megs/day. Looks like you'll need much more than 2 20gig drives to keep a years worth :) (you'll need that for just 2 months) Each month will eat nearly 30megs if our numbers scale. - Jason Godsey On Wed, 2 Jun 1999, ryanm wrote: > Hello everyone, > > I am curious what size my daily logfiles will be with 1+ logins > a day. I am doing some capicity planning and want to add a couple > disks to store logging info on. I would appreciate any average > sizes you have. I have looked at the entries in the detail log > and 1 login/logout is roughyl 800 bytes. I multiplies this out > by 1 and got 8,000,000 so I am assuming roughly 10 Megs a day?? > I plan on archiving these for up to a year for various reasons so > was hoping to get 2 20 gig Disks to do this. I also plan on using > some form of compression scheme. > > Thanks for any info anyone can get back to me, > > Thanks again, > > Ryan > > === > Archive at http://www.thesite.com.au/~radiator/ > To unsubscribe, email '[EMAIL PROTECTED]' with > 'unsubscribe radiator' in the body of the message. > === Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
Re: (RADIATOR) Logfile size
With roughly 3700 calls per day, our logfile (trace level 4) is roughly 6-6.5 meg, with the details file being slightly smaller, at about 5-5.5 meg per day. All our accounting we log into a database, but we also use the standard logfiles just as a backup. So for a month, these two log files account for about 300+ meg (getting bigger every month). Since we are running Radiator under NT, I just use WinZip once a month to archive the logs, which brings them down to about 25 meg per month, which is not bad compression! Regards, Anton Sparrius Chief Operations Officer --- Smarter Wayhttp://www.smart.net.au Email[EMAIL PROTECTED] Phone (03) 9846 1711 Melb 1800-240-829 Sydn 1800-888-761 - Original Message - From: ryanm <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, June 03, 1999 1:51 AM Subject: (RADIATOR) Logfile size > Hello everyone, > > I am curious what size my daily logfiles will be with 1+ logins > a day. I am doing some capicity planning and want to add a couple > disks to store logging info on. I would appreciate any average > sizes you have. I have looked at the entries in the detail log > and 1 login/logout is roughyl 800 bytes. I multiplies this out > by 1 and got 8,000,000 so I am assuming roughly 10 Megs a day?? > I plan on archiving these for up to a year for various reasons so > was hoping to get 2 20 gig Disks to do this. I also plan on using > some form of compression scheme. > > Thanks for any info anyone can get back to me, > > Thanks again, > > Ryan > > === > Archive at http://www.thesite.com.au/~radiator/ > To unsubscribe, email '[EMAIL PROTECTED]' with > 'unsubscribe radiator' in the body of the message. === Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
Re: (RADIATOR) Logfile size
Hi Ryan, On Jun 2, 11:51am, ryanm wrote: > Subject: (RADIATOR) Logfile size > Hello everyone, > > I am curious what size my daily logfiles will be with 1+ logins > a day. I am doing some capicity planning and want to add a couple > disks to store logging info on. I would appreciate any average > sizes you have. I have looked at the entries in the detail log > and 1 login/logout is roughyl 800 bytes. I multiplies this out > by 1 and got 8,000,000 so I am assuming roughly 10 Megs a day?? > I plan on archiving these for up to a year for various reasons so > was hoping to get 2 20 gig Disks to do this. I also plan on using > some form of compression scheme. That logic seems quite reasonable. I have seen similar numbers elsewhere. I would certainly advocate compression too. Cheers. > > Thanks for any info anyone can get back to me, > > Thanks again, > > Ryan > > === > Archive at http://www.thesite.com.au/~radiator/ > To unsubscribe, email '[EMAIL PROTECTED]' with > 'unsubscribe radiator' in the body of the message. >-- End of excerpt from ryanm -- Mike McCauley [EMAIL PROTECTED] Open System Consultants Pty. LtdUnix, Perl, Motif, C++, WWW 24 Bateman St Hampton, VIC 3188 Australia http://www.open.com.au Phone +61 3 9598-0985 Fax +61 3 9598-0955 Radiator: the most portable, flexible and configurable RADIUS server anywhere. SQL, proxy, DBM, files, LDAP, NIS+, password, NT, Emerald, Platypus, Freeside, TACACS+, PAM, external, etc etc on Unix, Win95/8, NT, Rhapsody === Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
Re: (RADIATOR) Logfile size
On 6/2/99 at 11:51 AM ryanm wrote: >Hello everyone, > >I am curious what size my daily logfiles will be with 1+ logins >a day. I am doing some capicity planning and want to add a couple >disks to store logging info on. I would appreciate any average >sizes you have. I have looked at the entries in the detail log >and 1 login/logout is roughyl 800 bytes. I multiplies this out >by 1 and got 8,000,000 so I am assuming roughly 10 Megs a day?? >I plan on archiving these for up to a year for various reasons so >was hoping to get 2 20 gig Disks to do this. I also plan on using >some form of compression scheme. Ryan, If they grow that big, I'd advice you to go to database logging. Currently we log temporarily to a MySQL and we have a deamon continously emptying that DB and feeding everything into a data warehouse (Sybase based). This lets us keep stats for more than a year (we make 1 table per month, something like 7 or 800 megs/month), have always handy summaries tables at hand (updated by the deamon), only keep data we want (there are some irrelevant data in a radius accounting packet dump), get rid of the STARTs and ALIVEs (make session reconstruction), ... Also, this deamon lets us make some nifty things such as for exemple send an email to all customers of a given POP several seconds after they connect (when it cleans the START), send our newsletter, and so on. File-based logging will never give you the scalability you'd like to have. Big files tend to make systems slow, are unmanageable, difficult to search, difficult to backup (ever tried to backup a growing file? Works far better with a DB :))... Speaking of a database, there's almost no size limit. You just have to throw disks at it and don't have to worry about filesystem problems (if you go for any serious DB engine, I definitely prefer to let them format the FS and use the raw device, far faster). Just my two cents. Stephan * ** Stephan Forseilles - Operations Director ** Belgacom Skynet SA/NV ** Email [EMAIL PROTECTED] ** Tel: +32 2 706 13 11 Fax: +32 2 726 93 11 * ÿ Archive at http://www.thesite.com.au/~radiator/ To unsubscribe, email '[EMAIL PROTECTED]' with 'unsubscribe radiator' in the body of the message.
