That doesn't stop me from wondering. I think my server would pass out from exhaustion.
(Your post is another keeper, I'd say) William -----Original Message----- From: Daniel Chenault [mailto:[EMAIL PROTECTED]] Sent: Thursday, January 17, 2002 12:02 AM To: Exchange Discussions Subject: Re: MDBDATA Log Files The issue is not compressing; it is assumed this would be a lengthy job and would be done off-hours. The problem is with manipulating the compressed file. When a request to open a file is passed to NTFS and the file system finds that the file is compressed it will decompress as much of that file as possible into memory. This is all well and good if you're working on a Word document that will get some changes (usually to the end of the file) and be saved back to looked at another time for a total size of maybe a few megs. But for a 12G database that is constantly being read and written to at many different locations, and constantly, this unraveling of the compressed file will kill performance as the CPU gets maxed out uncompressing multiple locations of this file into memory(1). The disks get thrashed as well while the system asks for more and more free physical memory forcing page faults(2)(3). Overall, compressing a database file being accessed hundreds, of not thousands, of times a minute with a file size well beyond physical memory (if you're uncompressing the file into VM you're losing even more time) is just not a good idea. The log files, on the other hand, are a different story. Each one exists in memory being written to an address with a pre-allocated size of 5M(4). As that memory allocation is used up, the entire chunk is written to a file and the file handle is closed(5). Exchange will not read this file again except in certain circumstances(6) so once closed the file will stay closed. Thus one would think that compressing the log files would be a good move to save disk space. Or would it? Most organizations I know who do backups(7) do them nightly. Since this wipes the logs off the drive we need only look at some numbers for a single day's worth of transactions. Assume an organization that moves, say, ten thousand messages a day through a given server. Each message has an average size of 50K. Allowing 5% for overhead (which is a number I picked just to keep the numbers even) this yields an average of 1,075,253,760 or just a bit over 1gig. The NTFS encryption algorithm is relatively efficient(8) and will largely depend on the data being compressed (9) A file of nothing but 01010101 repeating over and over will compress wonderfully (10) but although it is possible for there to be blocks of such patterns overall the file's data structure will be complex and somewhat random. At best I'd say expect 30% compression. Even giving it the benefit of the doubt and saying 40% yields a diskspace savings of about 400M. And brother, if you're sweating 400M on your server, you need to add another drive to the array or increase your budget. (1) unless you have a server with 12G, that is (2) writing pages of memory to the swapfile (3) disk access is high overhead, reading the file into memory such that more disk access is forced doubles the overhead resulting in a noticeably performance hit (4) I don't know why 5M was chosen; I guess it was just as good as 4 or 6 megs (5) for the purists: the current edb.log file is renamed to edb<the next sequential hex number> and the new log file is named edb.log (6) all of them having to do with disaster recovery and, at that point, diskspaceis the least of your worries (7) ahem (8) it's a trade-off between efficiency and performance (9) if you are curious you may wish to run your own tests since each installation and deployment will have its own traffic flow characteristics (10) like PKZip the algorithm largely depends on repeating pattern substitution(11) (11) meaning the software attempts to find repeating pattersn of multiple subpatterns (12) and replace it with a single pattern (12) such as the word "the" which can be replaced with one character resulting in savings of 66.6%(13) (13) this is a purposely simple example to illustrate the mechanism(14) (14) Hi Sherry! ----- Original Message ----- From: "Lefkovics, William" <[EMAIL PROTECTED]> To: "Exchange Discussions" <[EMAIL PROTECTED]> Sent: Wednesday, January 16, 2002 8:31 PM Subject: RE: MDBDATA Log Files > I wonder how long it would take to compress say a 12GB priv.edb? > > William > > > -----Original Message----- > From: Daniel Chenault [mailto:[EMAIL PROTECTED]] > Sent: Wednesday, January 16, 2002 1:46 PM > To: Exchange Discussions > Subject: Re: MDBDATA Log Files > > > compress log files, yes. compress databases, no. Just clarifying > > ----- Original Message ----- > From: "Lefkovics, William" <[EMAIL PROTECTED]> > To: "Exchange Discussions" <[EMAIL PROTECTED]> > Sent: Wednesday, January 16, 2002 2:00 PM > Subject: RE: MDBDATA Log Files > > > > You could, in an emergency, compress some of the log files with NTFS > > compression. They compress fairly well. > > > > This should not be the practice though. > > > > William > > > > > > -----Original Message----- > > From: Taylor, Skip [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, January 16, 2002 12:02 PM > > To: Exchange Discussions > > Subject: RE: MDBDATA Log Files > > > > > > Thanks for the vote of confidence. I planned on using PerfWiz to make the > > move, circular logging is Not enabled and I schedule a full backup every > > night. I missed one day of backing up due to a failed tape causing the > > drive to fill. I don't have much room for error because of the small > > partition. > > > > Thanks, > > > > Skip Taylor, MCSE > > Network Administrator > > Jordan, Jones, & Goulding > > > > > > -----Original Message----- > > From: Leeann McCallum [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, January 16, 2002 2:44 PM > > To: Exchange Discussions > > Subject: RE: MDBDATA Log Files > > > > > > Good point. I forgot about perfwiz. Call me paranoid, but I'd still do a > > full backup before running perfwiz. > > > > -----Original Message----- > > From: Soysal, Serdar [mailto:[EMAIL PROTECTED]] > > Sent: Thursday, 17 January 2002 8:42 a.m. > > To: Exchange Discussions > > Subject: RE: MDBDATA Log Files > > > > > > If you use perfwiz to move them, it will be ok [1]. Especially since when > > perfwiz shuts down Exchange services to run, all logs will be committed to > > the database anyways. > > > > But yes, a backup ASAP is in good order. I don't recommend deleting the > > logfiles until a good backup [2]. > > > > S. > > > > [1] I give Skip enough credit to assume that when he meant "move the > > logfiles" he was indeed talking about PerfWiz. [2] Unless Skip is using > > circular logging [3], in which case it doesn't really matter. [3] *GASP* > > > > -----Original Message----- > > From: Leeann McCallum [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, January 16, 2002 2:31 PM > > To: Exchange Discussions > > Subject: RE: MDBDATA Log Files > > > > > > I wouldn't move them. Exchange expects to find them in the place they > are. > > Get a full backup done asap. Do it now > > > > -----Original Message----- > > From: Taylor, Skip [mailto:[EMAIL PROTECTED]] > > Sent: Thursday, 17 January 2002 8:30 a.m. > > To: Exchange Discussions > > Subject: RE: MDBDATA Log Files > > > > > > I do a full backup every night but had a bad tape so the log files did not > > get deleted and the server came to a halt. I only have a 2gb partition > for > > these files. I'll see about moving them to a different partition. I just > > did not know what was considered normal. > > > > Thanks to all who replied, > > > > Skip Taylor, MCSE > > Network Administrator > > Jordan, Jones, & Goulding > > > > _________________________________________________________________ List posting FAQ: http://www.swinc.com/resource/exch_faq.htm Archives: http://www.swynk.com/sitesearch/search.asp To unsubscribe: mailto:[EMAIL PROTECTED] Exchange List admin: [EMAIL PROTECTED]
