Re: [vchkpw] OT - Preferred File Systems
Patrick Ale wrote: On 12/23/07, Christopher Chan <[EMAIL PROTECTED]> wrote: Patrick Ale wrote: On 12/22/07, Christopher Chan <[EMAIL PROTECTED]> wrote: PSU = Power Supply Unit, and you need at least two of them when using XFS IMO (and according to SGI self). A UPS is also good but it only helps you against power delivery issues. It doesn't help against people working in your server racks and accidently yanking cables or a PSU blowing away. Now that is a good point. Okay, two redundant PSU plugged into an UPS. Battery-backed caches are caches (mostly on RAID controllers) that will hold the data for an X amount of minutes while the power delivery is interupted, allowing the disks to sync up. The OP was asking for a filesystem for a RAM-based drive for a qmail queue. No BBU cache involved. His RAID system will be used for the mail store and it most probably has BBU cache but then again that may or may not protect against data/metadata sitting in the kernel cache. !DSPAM:47710b3a310548132029000!
Re: [vchkpw] OT - Preferred File Systems
On 12/23/07, Christopher Chan <[EMAIL PROTECTED]> wrote: > Patrick Ale wrote: > > On 12/22/07, Christopher Chan <[EMAIL PROTECTED]> wrote: > > PSU = Power Supply Unit, and you need at least two of them when using XFS IMO (and according to SGI self). A UPS is also good but it only helps you against power delivery issues. It doesn't help against people working in your server racks and accidently yanking cables or a PSU blowing away. Battery-backed caches are caches (mostly on RAID controllers) that will hold the data for an X amount of minutes while the power delivery is interupted, allowing the disks to sync up. Patrick !DSPAM:476e3187310546557812175!
Re: [vchkpw] OT - Preferred File Systems
Patrick Ale wrote: On 12/22/07, Christopher Chan <[EMAIL PROTECTED]> wrote: I'd say XFS. But with the restriction you have battery-backed caches and redundant PSUs. XFS is so fast cause of it's caching mechanism. A power failure without battery-backed caches or redundant PSU will make you cry. And after you cry chances are very big you'll cry a lot more due to corruption. I think you meant having a UPS system. The I-RAM is a DRAM media drive and as such it has zero caches and we want to protect against the contents in main system RAM not making it. !DSPAM:476e0fa6310541573590211!
Re: [vchkpw] OT - Preferred File Systems - I-RAM Drive
Jeff Koch wrote: By the way, to give you an idea of the speed of the i-ram drive with the XFS file system we tar-zipped the entire /usr directory into an 811MB archive. It took 54 seconds to untar-unzip it on a 4GB I-Ram drive and 141 seconds on a Seagate 750 GB SATA drive with the ext3 filesystem in the same machine. The CPU is a Core-Duo 6400 with 4GB RAM. ...how about apples to apples? Like when time it takes with the i-ram when it is formatted with xfs and when it is formatted with ext3? Straight file copies are even faster. Duplicating the same 811MB archive on the I-Ram took 13 seconds on the I-Ram drive and 43 seconds on the Seagate. My plan is to use the I-RAM for the following directories; var/qmail/queue var/qmail/simscan var/log Don't you want a more permanent record of logs? maybe /tmp let me know if you guys think of any other directories that would benefit from the speedup. Also, since the i-ram's battery backup only lasts a few hours we added some startup scripts to rc.local that try mounting the i-ram and then test for the existence of some key files. If they don't exist or the i-ram can't be mounted we then we assume the RAM got erased and use parted to re-create the partitions and mkfs to add the xfs filesystem. Then we mount the i-ram drive and copy over and untar the directories that we backed up upon shutdown (and also backup every few hours). That is assuming a complete power loss. The battery backup does not come into play if the server is turned off but still connected to the mains. The I-RAM will draw from power so long as the power supply is connected and not switched off. !DSPAM:476e0ebd310543618740901!
[vchkpw] OT - Preferred File Systems - I-RAM Drive
By the way, to give you an idea of the speed of the i-ram drive with the XFS file system we tar-zipped the entire /usr directory into an 811MB archive. It took 54 seconds to untar-unzip it on a 4GB I-Ram drive and 141 seconds on a Seagate 750 GB SATA drive with the ext3 filesystem in the same machine. The CPU is a Core-Duo 6400 with 4GB RAM. Straight file copies are even faster. Duplicating the same 811MB archive on the I-Ram took 13 seconds on the I-Ram drive and 43 seconds on the Seagate. My plan is to use the I-RAM for the following directories; var/qmail/queue var/qmail/simscan var/log maybe /tmp let me know if you guys think of any other directories that would benefit from the speedup. Also, since the i-ram's battery backup only lasts a few hours we added some startup scripts to rc.local that try mounting the i-ram and then test for the existence of some key files. If they don't exist or the i-ram can't be mounted we then we assume the RAM got erased and use parted to re-create the partitions and mkfs to add the xfs filesystem. Then we mount the i-ram drive and copy over and untar the directories that we backed up upon shutdown (and also backup every few hours). I think this approach should substantially speedup the mailserver and allow it to handle greater capacity. At 09:20 AM 12/22/2007, you wrote: Ed McLain wrote: As for recoveries after a hardware failure, I've only had to do 3 or 4. On one of them we had a buggy version of xfs_repair, and that caused some weirdness, but we had done a full dd before the restore to a secondary disk.. After upgrading xfs_repair we got back everything with no corruption that we could find.. Now, that's not to say that a man page didn't have null's in it, but everything we wanted was there and in tact. Man pages? You had existing files corrupted? Now that is something I have not had with ext3. As for XFS, I did lose one filesystem but I cannot pin it down to XFS code with certainty because that happened after a crash although I have not lost any ext3 filesystem due to a crash yet. In any case, my previous mail was about files that were created just prior to a crash or a power cut, not existing files. Existing files should not get corrupted. If a filesystem cannot guarantee integrity of existing files both in a data and metadata sense, then I'd say that is a candidate for 'untouchable'. When you are dealing with a mail queue, as the OP was asking about, you do want data integrity because once the mail has been queued, the sending side will deleted its copy as you have now assumed responsibility for delivery. This really means that only filesystems that do full journaling can meet such a criteria. If you do not mind losing whatever was very recently queued in the event of a crash/power cut, then go for XFS. Best Regards, Jeff Koch, Intersessions !DSPAM:476d7f49310544027043058!
Re: [vchkpw] OT - Preferred File Systems
On 12/22/07, Christopher Chan <[EMAIL PROTECTED]> wrote: I'd say XFS. But with the restriction you have battery-backed caches and redundant PSUs. XFS is so fast cause of it's caching mechanism. A power failure without battery-backed caches or redundant PSU will make you cry. And after you cry chances are very big you'll cry a lot more due to corruption. Patrick !DSPAM:476d693a310541648582167!
Re: [vchkpw] OT - Preferred File Systems
Ed McLain wrote: As for recoveries after a hardware failure, I've only had to do 3 or 4. On one of them we had a buggy version of xfs_repair, and that caused some weirdness, but we had done a full dd before the restore to a secondary disk.. After upgrading xfs_repair we got back everything with no corruption that we could find.. Now, that's not to say that a man page didn't have null's in it, but everything we wanted was there and in tact. Man pages? You had existing files corrupted? Now that is something I have not had with ext3. As for XFS, I did lose one filesystem but I cannot pin it down to XFS code with certainty because that happened after a crash although I have not lost any ext3 filesystem due to a crash yet. In any case, my previous mail was about files that were created just prior to a crash or a power cut, not existing files. Existing files should not get corrupted. If a filesystem cannot guarantee integrity of existing files both in a data and metadata sense, then I'd say that is a candidate for 'untouchable'. When you are dealing with a mail queue, as the OP was asking about, you do want data integrity because once the mail has been queued, the sending side will deleted its copy as you have now assumed responsibility for delivery. This really means that only filesystems that do full journaling can meet such a criteria. If you do not mind losing whatever was very recently queued in the event of a crash/power cut, then go for XFS. !DSPAM:476d1d36310541813613882!
Re: [vchkpw] OT - Preferred File Systems
We have 4GB's of RAM on the I-RAM which should be more than enough for the qmail queue and log files. I'll check out JFS. Thanks At 06:49 PM 12/21/2007, you wrote: Jeff Koch wrote: We're setting up a new mailserver with a Gigabyte i-ram drive to handle the qmail queue, and qmail logs. The i-ram drive uses standard DDR modules, has a battery backup and is lightning fast - much faster than solid state drives. We'll use SATA RAID for the maildirs and remainder of the server. I'd like to get some input on the best filesystem for the i-ram drive. EXT3 is out because writes are slowed to the speed of a hard drive. I'm leaning towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs is easily corrupted. xfs seems the best. You mean using ext3 on the i-ram gets you the speed of a hard drive? try running ext3 with data=journal (full journal mode) and with noatime. xfs may have the best performance but its data integrity is the worst of all the filesystems available for Linux besides being one of the buggiest. It is the filesystem with the largest codebase there is and it is a filesystem that was ported to Linux with lots of workarounds to work with the Linux kernel. xfs only journals metadata so you will have an intact filesystem but you will get in files after a crash or after power loss. No, the i-ram will not make a difference for xfs will use main memory for data caching. Data will not get a chance to hit the i-ram in the event of a crash or a power cut. reiserfs assumes perfect media which is why it is easily corrupted. The i-ram will probably not give you that problem and besides, this is the qmail queue you are talking about. It probably will not store that much mail at all. I am amazed that you feel comfortable with just 1GB. I would have thought that maybe using the i-ram as an external journal for an ext3 filesystem on SATA disks in full journal mode would have been better. You can also give jfs a shot. It is robust and comes second in most filesystem tests whether they are reads, writes or writing time at least according to Bruce Guenter's maildir suitability benchmark. (google fsbench) Best Regards, Jeff Koch, Intersessions !DSPAM:476c614c310543838827298!
Re: [vchkpw] OT - Preferred File Systems
Hrmm.. Haven't had any of those issues. We run XFS religiously on all data and hd intensive apps and in 8 years I don't recall once ever having XFS crap out on us without a hardware failure. As for recoveries after a hardware failure, I've only had to do 3 or 4. On one of them we had a buggy version of xfs_repair, and that caused some weirdness, but we had done a full dd before the restore to a secondary disk.. After upgrading xfs_repair we got back everything with no corruption that we could find.. Now, that's not to say that a man page didn't have null's in it, but everything we wanted was there and in tact. At the moment we use GFS as our main filesystem for storing email home directories and the like, simply because we are running a multi node active/active cluster, but the system was running XFS before hand with a secondary disk dedicated to the journal. That system ran for 4 years, sometimes with loads pegging up to 20 (qmail-scanner mostly), but never developed any major corruption through hardware failures or somebody randomly yanking the power cords out. I will agree with your other email though as well, before I used XFS I used JFS and it was wonderful as well. -- Ed McLain From: Christopher Chan <[EMAIL PROTECTED]> Reply-To: Date: Fri, 21 Dec 2007 17:57:09 -0600 To: Conversation: [vchkpw] OT - Preferred File Systems Subject: Re: [vchkpw] OT - Preferred File Systems Ed McLain wrote: > XFS.. I'm addicted and will freely admit it. > > We ran some benchmarks a while back between EXT3, XFS, JFS, and ReiserFS with > both qmail and mysql. While MySQL didn't care much, qmail was able to > deliver between 10,000 and 20,000 messages per minute more on XFS than either > other option. Also, having had to recover data from all filesystems listed > due to hardware or other failures, XFS has been the most reliable and easiest > to recover. Reiser and EXT3 were the worst. Just my .02. > XFS has the most aggressive data caching schemes of all the filesystems available for Linux. Are you sure it is the most reliable and the most recoverable? I have always found files stuffed with nulls on a xfs filesystem after a crash or a power cut. Never a completely corrupt filesystem but definitely zero data recovery for freshly created files just prior to crashing. That is notwithstanding xfs shutting itself down for no apparent reason (the box did not crash, no disk errors in logs, etc) which may be resolved by remounting or a reboot but in some cases, a repair was needed. !DSPAM:476c5625310545200813374!
Re: [vchkpw] OT - Preferred File Systems
Ed McLain wrote: XFS.. I'm addicted and will freely admit it. We ran some benchmarks a while back between EXT3, XFS, JFS, and ReiserFS with both qmail and mysql. While MySQL didn't care much, qmail was able to deliver between 10,000 and 20,000 messages per minute more on XFS than either other option. Also, having had to recover data from all filesystems listed due to hardware or other failures, XFS has been the most reliable and easiest to recover. Reiser and EXT3 were the worst. Just my .02. XFS has the most aggressive data caching schemes of all the filesystems available for Linux. Are you sure it is the most reliable and the most recoverable? I have always found files stuffed with nulls on a xfs filesystem after a crash or a power cut. Never a completely corrupt filesystem but definitely zero data recovery for freshly created files just prior to crashing. That is notwithstanding xfs shutting itself down for no apparent reason (the box did not crash, no disk errors in logs, etc) which may be resolved by remounting or a reboot but in some cases, a repair was needed. !DSPAM:476c52db310542446110226!
Re: [vchkpw] OT - Preferred File Systems
Jeff Koch wrote: We're setting up a new mailserver with a Gigabyte i-ram drive to handle the qmail queue, and qmail logs. The i-ram drive uses standard DDR modules, has a battery backup and is lightning fast - much faster than solid state drives. We'll use SATA RAID for the maildirs and remainder of the server. I'd like to get some input on the best filesystem for the i-ram drive. EXT3 is out because writes are slowed to the speed of a hard drive. I'm leaning towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs is easily corrupted. xfs seems the best. You mean using ext3 on the i-ram gets you the speed of a hard drive? try running ext3 with data=journal (full journal mode) and with noatime. xfs may have the best performance but its data integrity is the worst of all the filesystems available for Linux besides being one of the buggiest. It is the filesystem with the largest codebase there is and it is a filesystem that was ported to Linux with lots of workarounds to work with the Linux kernel. xfs only journals metadata so you will have an intact filesystem but you will get in files after a crash or after power loss. No, the i-ram will not make a difference for xfs will use main memory for data caching. Data will not get a chance to hit the i-ram in the event of a crash or a power cut. reiserfs assumes perfect media which is why it is easily corrupted. The i-ram will probably not give you that problem and besides, this is the qmail queue you are talking about. It probably will not store that much mail at all. I am amazed that you feel comfortable with just 1GB. I would have thought that maybe using the i-ram as an external journal for an ext3 filesystem on SATA disks in full journal mode would have been better. You can also give jfs a shot. It is robust and comes second in most filesystem tests whether they are reads, writes or writing time at least according to Bruce Guenter's maildir suitability benchmark. (google fsbench) !DSPAM:476c50fd310541151219808!
Re: [vchkpw] OT - Preferred File Systems
[EMAIL PROTECTED] wrote: I've heard good things about xfs, but no personal experience. However, I would STRONGLY DISCOURAGE you from running reiserfs for this. I've seen, and dealt with, resierfs corrupting filesystems and it's a nightmare. I do not have a good opinion of resierfs after real world experience with it. -- Mike Garrison What about things like XFS delays allocation and ordering of meta-data and data flush to disk. XFS may choose not to write data to the disk when it is received, so loss of data during crash is more likely. Creation and deletion of directory entries is a much slower metadata operation than other file systems !DSPAM:476c42e6310547862416751!
RE: [vchkpw] OT - Preferred File Systems
I can speak from experience. I used to use only ReiserFS, until I had a corruption on a 60GB Maildir mailstore. A tree rebuild was required. This took a whopping 17 hours on a high end Apple XServe RAID Array. Needless to say that after this I've migrated away from ReiserFS to XFS and have been very happy. Anytime I've had a problem with and XFS file system that required xfs_repair to be run, it was very swift. Now all this was with ReiserFS 3. I have no experience with ReiserFS 4. HTH, Tren > -Original Message- > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] > Sent: Friday, December 21, 2007 2:08 PM > To: vchkpw@inter7.com > Cc: [EMAIL PROTECTED] > Subject: Re: [vchkpw] OT - Preferred File Systems > > > I'd like to get some input on the best filesystem for the i-ram > drive. > > EXT3 > > is out because writes are slowed to the speed of a hard drive. I'm > leaning > > towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs > is > > easily corrupted. xfs seems the best. > > > > Anyone have any comments of suggestions. > > I've heard good things about xfs, but no personal experience. However, > I > would STRONGLY DISCOURAGE you from running reiserfs for this. I've > seen, > and dealt with, resierfs corrupting filesystems and it's a nightmare. I > do > not have a good opinion of resierfs after real world experience with > it. > > -- > Mike Garrison > > > !DSPAM:476c3b87310541291091484!
Re: [vchkpw] OT - Preferred File Systems
> I'd like to get some input on the best filesystem for the i-ram drive. > EXT3 > is out because writes are slowed to the speed of a hard drive. I'm leaning > towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs is > easily corrupted. xfs seems the best. > > Anyone have any comments of suggestions. I've heard good things about xfs, but no personal experience. However, I would STRONGLY DISCOURAGE you from running reiserfs for this. I've seen, and dealt with, resierfs corrupting filesystems and it's a nightmare. I do not have a good opinion of resierfs after real world experience with it. -- Mike Garrison !DSPAM:476c3934310541077512779!
Re: [vchkpw] OT - Preferred File Systems
XFS.. I'm addicted and will freely admit it. We ran some benchmarks a while back between EXT3, XFS, JFS, and ReiserFS with both qmail and mysql. While MySQL didn't care much, qmail was able to deliver between 10,000 and 20,000 messages per minute more on XFS than either other option. Also, having had to recover data from all filesystems listed due to hardware or other failures, XFS has been the most reliable and easiest to recover. Reiser and EXT3 were the worst. Just my .02. -- Ed McLain From: Jeff Koch <[EMAIL PROTECTED]> Reply-To: Date: Fri, 21 Dec 2007 10:24:18 -0600 To: , <[EMAIL PROTECTED]> Conversation: [vchkpw] OT - Preferred File Systems Subject: [vchkpw] OT - Preferred File Systems We're setting up a new mailserver with a Gigabyte i-ram drive to handle the qmail queue, and qmail logs. The i-ram drive uses standard DDR modules, has a battery backup and is lightning fast - much faster than solid state drives. We'll use SATA RAID for the maildirs and remainder of the server. I'd like to get some input on the best filesystem for the i-ram drive. EXT3 is out because writes are slowed to the speed of a hard drive. I'm leaning towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs is easily corrupted. xfs seems the best. Anyone have any comments of suggestions. Best Regards, Jeff Koch, Intersessions !DSPAM:476bec2f310541617020193!
Re: [vchkpw] OT - Preferred File Systems
Jeff Koch wrote: > > > We're setting up a new mailserver with a Gigabyte i-ram drive to handle > the qmail queue, and qmail logs. The i-ram drive uses standard DDR > modules, has a battery backup and is lightning fast - much faster than > solid state drives. We'll use SATA RAID for the maildirs and remainder > of the server. > > I'd like to get some input on the best filesystem for the i-ram drive. > EXT3 is out because writes are slowed to the speed of a hard drive. I'm > leaning towards reiserfs or xfs. However, I've read (wikipedia) that > reiserfs is easily corrupted. xfs seems the best. > > Anyone have any comments of suggestions. > > > > > Best Regards, > > Jeff Koch, Intersessions > > > > > Personally, I haven't had major issues with ReiserFS. It had some problems when it first came out, but has become quite the stable filesystem since then. In fact, I once had to use an early version of it when an RH 7.x system kept hitting a race condition with ext3 (I couldn't upgrade to the patched version of the kernel, the system was running software that had a license key tied to the software versions - evil). The main thing to look out for is keeping uncompressed copies of reiserfs-formatted files (disk images) on a reiserfs drive. That totally breaks things when you run an fsck. Honestly though, unless you need the journaling capabilities, look at the non-journaled systems like ext2 as well. In most situations, your queue won't need to be journaled, and neither will your logs. !DSPAM:476beb90310541644812091!
[vchkpw] OT - Preferred File Systems
We're setting up a new mailserver with a Gigabyte i-ram drive to handle the qmail queue, and qmail logs. The i-ram drive uses standard DDR modules, has a battery backup and is lightning fast - much faster than solid state drives. We'll use SATA RAID for the maildirs and remainder of the server. I'd like to get some input on the best filesystem for the i-ram drive. EXT3 is out because writes are slowed to the speed of a hard drive. I'm leaning towards reiserfs or xfs. However, I've read (wikipedia) that reiserfs is easily corrupted. xfs seems the best. Anyone have any comments of suggestions. Best Regards, Jeff Koch, Intersessions !DSPAM:476be8bd310544004610834!
