Re: Too slow
Hello, I turned the mail partition now to xfs and it's terribly fast. WOW!!! did you use an ext3 partition with dir_index before? I'm just asking because we are at a similar point and need to make a decision. All benchmarks I know of don't show that XFS actually performs faster than a modern ext3. Actually, with many concurrent reads and writes, ext3 seem to perform better, but only according to benchmarks. I cannot tell from an own installation. A switch from ext3 to XFS would be a big step for us, because we have to take care for about 13,000 mailboxes. So I want to be really sure if it is the right step. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
On Wednesday 25 October 2006 20:20, Marten Lehmann wrote: did you use an ext3 partition with dir_index before? I'm just asking because we are at a similar point and need to make a decision. All benchmarks I know of don't show that XFS actually performs faster than a modern ext3. Actually, with many concurrent reads and writes, ext3 seem to perform better, but only according to benchmarks. I cannot tell from an own installation. A switch from ext3 to XFS would be a big step for us, because we have to take care for about 13,000 mailboxes. So I want to be really sure if it is the right step. XFS may be faster _until_ it craps out. I have had two (not one) instances of a '.' directory entry being removed from a particular directory and fsck.xfs not being able to repair it caused the need for reinstallation. Doing anything in the directory caused the FS to be unmounted due to detecting an inconsistency in the FS. This happened to my /var partition on both occasions. Granted, it happened on a laptop during a suspend resume both times, but it does not instill confidence that it is not robust enough to stand up to this, not to mention unexpected power offs and the like. Also, the XFS developers are no longer paid by SGI to do XFS from what I understand, so that's another strike. Ciao, wt -- Warren Turkal Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
I turned the mail partition now to xfs and it's terribly fast. WOW!!! did you use an ext3 partition with dir_index before? I'm just asking We've just had some experience with filesystems ourselves. Previously we've used reiserfs exclusively for the last 5 years. All up, it's been really good to us. There are two really, really important things with reiserfs though: 1. You MUST have hardware that doesn't lie about it's write cache. When the filesystem tells the device driver to sync to disk, and the disk says it's done, it must be done (http://community.livejournal.com/lj_dev/670215.html - see the Disk cache issues) 2. Your hardware must be IO reliable, it must never report any write or read IO errors at the sector level Both of these issues are really to do with your hardware and device driver. If you've got good hardware and drivers, you should never see either. I believe that every report out there with people saying reiserfs screwed my partition totally is because of one of these 2 things. In the 5 years we've been using it, we've had dozens of different kernel crashes, power lost, etc, and not one corruption due to any of it. The only corruptions we've had are due to IO errors on the external RAID device itself (RAID 6, two simultaneous failed drives, and a third drive started reporting errors which were returned as IO errors to the kernel). In this respect, IBM seem to really have gotten their SCSI hardware right. Recently, a number of new machines we got we decided to try out ext3 again with dir_index. The results have been horrible with huge server loads. Using a spare partition, we're juggling users back to reiserfs partitions. The result is significantly lower loads for the exact same user set. I'd rate the general pros/cons of *linux* filesystems as: * ext3 pros: most widely used; excellent recovery tools; full data journaling available; best in the face of flakey hardware or disk caches that lie cons: performance just isn't that good in a large active user base * reiserfs pros: performs well with large active user base configuration, full data journaling available cons: recovery tools generally work, but have been known to crash and can be slow on large partitions; large mount time (will be fixed in 2.6.19), apparently some concurrecny issues with taking the BKL * xfs pros: fast on large files, good concurrency cons: no data journaling, only meta-data; not really stable when bugs like this occur that even a xfs_repair wouldn't fix! (http://oss.sgi.com/projects/xfs/faq.html#dir2) All the other filesystems I'd label as less used, which means that it's more likely bugs to appear and wouldn't recommend for a production environment. What about zfs? It's solaris only, and I believe there's still horrible performance in the face of fsync() calls, which cyrus does a LOT of. My 2 cents. Rob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
On Apr 7, 2006, at 2:58 PM, Sascha Bieler wrote: Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? I assume deleting is expunging the messages and not just flagging the messages as deleted.I don't have a hint for your setup, but cyrus-imapd 2.3 has a delayed expunge function that (among several things )speeds up the apparent speed of expunging by removing the message from the message index (I think) but not performing the actual delete. To the user the messages is expunged.In off peak hours, cyr_expire can be run to remove all the expunged messages from disk.-Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Too slow
Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Too slow
Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Too slow
Unfortunatly it's not better with 2.3.1. It takes about 3 Seconds to purge the for deletion flagged mails. Don't know why it's as slow now. Maybe it's got to do with the SMP-Kernel??? From: Patrick Radtke [mailto:[EMAIL PROTECTED] Sent: Friday, April 07, 2006 9:12 PM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow On Apr 7, 2006, at 2:58 PM, Sascha Bieler wrote: Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? I assume deleting is expunging the messages and not just flagging the messages as deleted. I don't have a hint for your setup, but cyrus-imapd 2.3 has a delayed expunge function that (among several things )speeds up the apparent speed of expunging by removing the message from the message index (I think) but not performing the actual delete. To the user the messages is expunged. In off peak hours, cyr_expire can be run to remove all the expunged messages from disk. -Patrick Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
Don't use ext3? Honestly ext2/3 is fine and dandy, but gentoo does +S the /var/imap folder, so everything has to call a sync() That is slow on ext2/3, perhaps reiserfs or xfs would be better for you? On Fri, 7 Apr 2006 20:58:57 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html !DSPAM:4436ca8945071336712104! -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Too slow
If I mount all partitions with async it's not getting better... :-( mmmh... -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Friday, April 07, 2006 10:30 PM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow Don't use ext3? Honestly ext2/3 is fine and dandy, but gentoo does +S the /var/imap folder, so everything has to call a sync() That is slow on ext2/3, perhaps reiserfs or xfs would be better for you? On Fri, 7 Apr 2006 20:58:57 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html !DSPAM:4436ca8945071336712104! -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
So buy a new drive, format it with reiser or xfs or whatever and do some tests of your own. I know from personal experience after running it with xfs, it was alot faster then with ext3. That or reinstall? Why would you keep your mail on your main partition anywho? If your drive fails your mail fails. :( That's bad. On Fri, 7 Apr 2006 22:57:19 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: If I mount all partitions with async it's not getting better... :-( mmmh... -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Friday, April 07, 2006 10:30 PM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow Don't use ext3? Honestly ext2/3 is fine and dandy, but gentoo does +S the /var/imap folder, so everything has to call a sync() That is slow on ext2/3, perhaps reiserfs or xfs would be better for you? On Fri, 7 Apr 2006 20:58:57 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates !DSPAM:4436e6a064921804284693! -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Too slow
Yes, indeed. That's why I bought an RAID5 Controller and do a backup every night. I have an extra partition for var, so I just have to copy all, boot from disc, format and recopy... That's simple too, but unfortunatly I got no more power today to do this... ;-) Have a good night and nice weekend! Thank u for helping and I'll let u know. Oh one thing I'd like to know: Do u think xfs is better than reiser? Greetings Sascha ___ Radio Gong 2000 GmbH Co. KG Sascha Bieler Technischer Leiter Franz-Joseph-Strasse 14 80801 München -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Saturday, April 08, 2006 12:29 AM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow So buy a new drive, format it with reiser or xfs or whatever and do some tests of your own. I know from personal experience after running it with xfs, it was alot faster then with ext3. That or reinstall? Why would you keep your mail on your main partition anywho? If your drive fails your mail fails. :( That's bad. On Fri, 7 Apr 2006 22:57:19 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: If I mount all partitions with async it's not getting better... :-( mmmh... -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Friday, April 07, 2006 10:30 PM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow Don't use ext3? Honestly ext2/3 is fine and dandy, but gentoo does +S the /var/imap folder, so everything has to call a sync() That is slow on ext2/3, perhaps reiserfs or xfs would be better for you? On Fri, 7 Apr 2006 20:58:57 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates !DSPAM:4436e6a064921804284693! -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
I personally prefer xfs over reiserfs, however I don't think it's better, because each file-system has it's give and take. No matter which 'guide' you may find they should tell you the cost and benefit of every file-system. On Sat, 8 Apr 2006 00:52:51 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Yes, indeed. That's why I bought an RAID5 Controller and do a backup every night. I have an extra partition for var, so I just have to copy all, boot from disc, format and recopy... That's simple too, but unfortunatly I got no more power today to do this... ;-) Have a good night and nice weekend! Thank u for helping and I'll let u know. Oh one thing I'd like to know: Do u think xfs is better than reiser? Greetings Sascha ___ Radio Gong 2000 GmbH Co. KG Sascha Bieler Technischer Leiter Franz-Joseph-Strasse 14 80801 München -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Saturday, April 08, 2006 12:29 AM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow So buy a new drive, format it with reiser or xfs or whatever and do some tests of your own. I know from personal experience after running it with xfs, it was alot faster then with ext3. That or reinstall? Why would you keep your mail on your main partition anywho? If your drive fails your mail fails. :( That's bad. On Fri, 7 Apr 2006 22:57:19 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: If I mount all partitions with async it's not getting better... :-( mmmh... -Original Message- From: Scott M. Likens [mailto:[EMAIL PROTECTED] Sent: Friday, April 07, 2006 10:30 PM To: Sascha Bieler Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Too slow Don't use ext3? Honestly ext2/3 is fine and dandy, but gentoo does +S the /var/imap folder, so everything has to call a sync() That is slow on ext2/3, perhaps reiserfs or xfs would be better for you? On Fri, 7 Apr 2006 20:58:57 +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Greetings Sascha Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates !DSPAM:4437006883861002057823! -- What does one want when one is engaged in the sexual act? That everything around you give you its utter attention Think only of you, care only for you... Every man wants to be a tyrant when he fornicates Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Too slow
--On April 7, 2006 8:58:57 PM +0200 Sascha Bieler [EMAIL PROTECTED] wrote: Hi there, I am running cyrus-imapd-2.1.12 on a Gentoo Linux with kernel 2.6.15. Got a Pentium 4 Xeon 2,8 GHz and 2 GB Ram. The cyruspartition is ext3 on a SCSI RAID 5. hdparm -tT /dev/sda says: /dev/sda: Timing cached reads: 2316 MB in 2.00 seconds = 1158.00 MB/sec Timing buffered disk reads: 80 MB in 3.01 seconds = 26.58 MB/sec Everythings working just fine, but when I want to delete an email it's so slowly... Has anyone a hint for me? Depends a LOT on your I/O load. 27MB/sec is pretty slow, and that's reads, RAID5 writes are often as slow as half, depending on your hardware, plus that's before the filesystem is in the way. Large directories slow ext3 down quite a bit. Expunge operations on our large mail backends are often pretty slow too, but aren't too bad, though we're using a lot bigger hardware and servers. Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
