Re: Using a RAMDISK for /var/qmail/queue thoughts ?
David Dyer-Bennet wrote: Um, most reporting measured results from optimizing high-traffic qmail-based mail servers have found that disk activity on the queue disk is the first limit they hit. How about, if the first delivery fails, pass it off to a server with some disks. Why not pre-process with qmail-remote before queueing? -- David Nicol 816.235.1187 [EMAIL PROTECTED] Five seconds of light is a lot of data.
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
On Fri, Jan 26, 2001 at 05:46:51PM -0600, David L. Nicol wrote: David Dyer-Bennet wrote: Um, most reporting measured results from optimizing high-traffic qmail-based mail servers have found that disk activity on the queue disk is the first limit they hit. How about, if the first delivery fails, pass it off to a server with some disks. Why not pre-process with qmail-remote before queueing? qmail-remote is way too late as most of the I/O load is putting the mail in the queue, not getting it off. Besides, the symptom isn't delivery failure, it's slowness. Regards.
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
On Fri, Jan 26, 2001 at 11:56:54PM +, Mark Delany wrote: On Fri, Jan 26, 2001 at 05:46:51PM -0600, David L. Nicol wrote: David Dyer-Bennet wrote: Um, most reporting measured results from optimizing high-traffic qmail-based mail servers have found that disk activity on the queue disk is the first limit they hit. How about, if the first delivery fails, pass it off to a server with some disks. Why not pre-process with qmail-remote before queueing? qmail-remote is way too late as most of the I/O load is putting the mail in the queue, not getting it off. Besides, the symptom isn't delivery failure, it's slowness. Actually. I was little hasty on the first point. I now see what you're getting at. One question is, how far do you let qmail-remote go before deciding it will work. It it's past the MAIL FROM/RCPT TO part, then why not complete the delivery rather than incur the double load and double latency? If you mean to try and completely do the remote delivery prior to placing it in the queue, then this is the passthru idea that people have suggest previously. It potentially has merit with an amount of added complexity. One problem is that a busy system, such as a mailing list system may be at full concurrencyremote for extended periods of time, in which case, new submissions should not attempt qmail-remote delivery so you're back to square one. Another problem is that the mail has to be stored somewhere while qmail-remote attempts delivery. Well, unless you want the submitting client to wait - that may create a lot of confusing latency for, eg, people sitting on a PC using Eudora. If the mail is stored somewhere, you're starting to get back to a disk queue. But it's not necessarily a silly idea. I believe that sendmail tries to do something like this in certain circumstances. A monolithic design has an advantage in this regard. Doing this is a nice compartmentalized way with the current qmail wouldn't be a lot of fun. Regards.
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
On Wed, Dec 13, 2000 at 04:20:19PM +, Greg Cope wrote: Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably fast, but you still have to worry about things like swamping the todo and on top of that you may have to worry about filling up your queue disc. You can get QMail into a situation where it's completely wedged until you manually remove some files from the ram disc to give it enough space to continue delivering mail. Sean -- I never thought I'd live in a country where physical violence would be used to disenfranchise voters. Have you heard about Bush supporters rioting? Sean Reifschneider, Inimitably Superfluous [EMAIL PROTECTED] tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
Sean Reifschneider wrote: On Wed, Dec 13, 2000 at 04:20:19PM +, Greg Cope wrote: Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably fast, but you still have to worry about things like swamping the todo and on top of that you may have to worry about filling up your queue disc. You can get QMail into a situation where it's completely wedged until you manually remove some files from the ram disc to give it enough space to continue delivering mail. Thanks for that: Queue disk filling is an issue - I was thinking of a 256 meg ram disk - and small jobs - i.e try to leave a lot of room - i.e. if the av message size is 10 K, not inject more than 25000 messages at once - although this leaves little headroom. Ok given that I can: control job size control job inject fequency (by looking at the queue size / space left) control injection speed (avoid / stop todo swaping) Would this still be a good idea ? As a 256 meg dim is 108 UK pounds sterling - or less than a SCSI card ... Greg Sean -- I never thought I'd live in a country where physical violence would be used to disenfranchise voters. Have you heard about Bush supporters rioting? Sean Reifschneider, Inimitably Superfluous [EMAIL PROTECTED] tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
RE: Using a RAMDISK for /var/qmail/queue thoughts ?
-Original Message- From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 13, 2000 12:12 PM 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably I was thinking about doing this awhile back. I made a ramdisk and used both bonnie and bonnie2 to do some benchmarking on it, and I actually got worse performance on the Ramdisk. You still have the overhead of the file system, and you have no hardware DMA controller with a Ramdisk. It was slower in my experience. The box I tested it on had 1GB of Ram, and a 512MB ramdisk. No swap was being used during the tests. Jay -Original Message- From: Sean Reifschneider [mailto:[EMAIL PROTECTED]] Sent: Wednesday, December 13, 2000 12:12 PM To: Greg Cope Cc: qmail mailing list Subject: Re: Using a RAMDISK for /var/qmail/queue thoughts ? On Wed, Dec 13, 2000 at 04:20:19PM +, Greg Cope wrote: Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? 10ns is much faster than 5.3ms... It works, I've done it, it's reasonably fast, but you still have to worry about things like swamping the todo and on top of that you may have to worry about filling up your queue disc. You can get QMail into a situation where it's completely wedged until you manually remove some files from the ram disc to give it enough space to continue delivering mail. Sean -- I never thought I'd live in a country where physical violence would be used to disenfranchise voters. Have you heard about Bush supporters rioting? Sean Reifschneider, Inimitably Superfluous [EMAIL PROTECTED] tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
On Wed, Dec 13, 2000 at 06:43:25PM +, Greg Cope wrote: Would this still be a good idea ? As a 256 meg dim is 108 UK pounds sterling - or less than a SCSI card ... I can't say... I used such a setup on a system with 1GB RAM sending out 1+million e-mails of the sort you are. It was more painful to manage, but worked. If you need the performance, then there's not much choice. Sean -- I thought Bush and Gore were both bad choices. However, Bush seems to be doing his best to prove he's the greatest of two evils. Sean Reifschneider, Inimitably Superfluous [EMAIL PROTECTED] tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
Thus spake Greg Cope ([EMAIL PROTECTED]): Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? Why would you expect a speed increase at all? And even if there were one, would anyone notice? Who looks at his email every millisecond and would even notice the improvement? I would suspect that your mail service, like everyone else's, is not limited by disk throughput, but by network throughput. Or are you delivering all those emails locally? Felix
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
On Wed, Dec 13, 2000 at 08:31:48PM +0100, Felix von Leitner wrote: Thus spake Greg Cope ([EMAIL PROTECTED]): Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? Why would you expect a speed increase at all? Er, perhaps because a disk seek/fsync is slower to a real disk than it is to a ramdisk? And even if there were one, would anyone notice? Sure. When you deliver thousands or millions of emails a day. And plenty of people do that right now. Who looks at his email every millisecond and would even notice the improvement? Er, I believe the discussion is about queue processing, not MUA reading? I would suspect that your mail service, like everyone else's, is not limited by disk throughput, but by network throughput. "suspect" being the operative word here. Mostly qmail thruput on large systems *is* spindle bound. A lot of these sort of systems are housed at co-los where there is a *lot* of available bandwidth. I have seen plenty of qmail systems run out of spindle first so I don't know what led you to conclude that "..like everyone else's, is not limited by disk throughput". Regards.
Re: Using a RAMDISK for /var/qmail/queue thoughts ?
Felix von Leitner [EMAIL PROTECTED] writes on 13 December 2000 at 20:31:48 +0100 Thus spake Greg Cope ([EMAIL PROTECTED]): Has anyone any empirical evidence for the speed increases I may expect (as opposed to a fast EIDI (ATA 66, 8.5ms seek) or SCSI system (eg 10k, 5.3 ms seek 25mb/s) ? Why would you expect a speed increase at all? And even if there were one, would anyone notice? Who looks at his email every millisecond and would even notice the improvement? I would suspect that your mail service, like everyone else's, is not limited by disk throughput, but by network throughput. Or are you delivering all those emails locally? Um, most reporting measured results from optimizing high-traffic qmail-based mail servers have found that disk activity on the queue disk is the first limit they hit. -- David Dyer-Bennet / Welcome to the future! / [EMAIL PROTECTED] SF: http://www.dd-b.net/dd-b/ Minicon: http://www.mnstf.org/minicon/ Photos: http://dd-b.lighthunters.net/
