Jeff Mayzurk wrote:
>
> I wrote:
>
> >> By the way, does anyone have any interest in comparing notes on really high
> >> volume qmail configs? I'm looking for performance in the range of 200-250k
> >> remote deliveries per hour. We're halfway there with relatively few
>
> And Greg Cope replied:
>
> >I've a similar issue (high performance required).
> >
> >We deceided to go for multiple smallish intel boxes due to cost for
> >emails per hour ratios work in favour of intel on Linux (or FreeBSD but
> >thats a different flame ;-).
> >
> >Lirking on this list I've seen people achieve between 30k to 150k and
> >hour. The average appears to be 50k an hour.
>
> We're getting 100k-110k per hour on a single Sun Ultra2/2167, 512MB RAM,
> Solaris 2.7, and the queue directories mounted from a NetApp F740. This is
> with an "isolated" queue, i.e., only one message in the queue and qmail-send
> processing it as quickly as possible.
Low spec box then ;-).
Out of interest does the Netfilter have a large / battery backed cache
to decrease the I/O / disk bottle neck ?
Also does your system only send one message - the ones I deal with are
all individual (both in content and message headers).
> We have concurrencyremote set to 500 and the processor run queue is usually
> around 10 deep. The machine is 100% busy, with about 30% of CPU time spent in
> user code and the rest in the kernel. (More on this below.) We're going to
> swap in some faster processors and see how much more work we can get done.
>
> >If bandwidth is not an issue then next bottle neck is the queue and disk
> >speed. As far as a I can see once you've moved from a simple DISK, you
> >can either:
> >
> >a) Avoide the queue and talk direct to qmail-remote (and take care of
> >bounces / process mgt yourself).
> >
> > This appears to be where people are getting above 100k and hour.
> >
> >b) Buy lots of fast disks - RAID 0+1 (due to write performance RAID 5
> >appears a bad idea).
>
> We're seeing very little disk I/O. At peak we're doing about 30 read ops/s and
> 30 write op/s, or around 300-500KB/s in both read and write. Service times are
> near-zero and there's no blocking on disks. Again, we're using NetApp filers.
Could this be due to a nice large cache ?
> >If you've any nice patches I would be very interested - but my spin on
> >the issue is KISS - i.e we will scale by adding small nodes - not
> >expensive boxen, and also going with simple set-ups for the sanity of
> >the people whom will run this.
> >
> >I would be very instered in anyone thoughts on theses issues.
>
> The biggest issue I have with qmail's current architecture is that it's doing
> a fork/exec for each delivery--and this is really expensive on some platforms.
> Short of a threaded qmail-remote (or qmail-send/qmail-remote hybrid), a
> pre-forked pool of qmail-remotes waiting on a common socket would probably be
> a significant improvement. In short, Apache-style process management.
Sounds like a whole new SMTP server to me .....
Preforking sounds better than threading - although, from a resiliance
point of view.
Would some sort of wrapper cum process mamanager not be a simple hybrid
solution ?
I know others have been down the route, and many end up coding a whole
new sever, which is great if you can support it - I am not ready to go
there as I tend to install the boxes and others have to admin them
(usually an admin / web developer style person - not a C / email
specialist).
You may also consider caching DNS mx record requests in IPC memory as
well.
> Ideally communication between qmail-send and qmail-remote would be done over
> IP, permitting a distributed configuration with multiple small nodes running
> qmail-remotes and a single queue server running qmail-send. Otherwise,
> distributing across multiple small boxes running completely independent
> instances of qmail is problematic.
I can understand that - i.e how to distribute jobs to other boxes /
processes.
It appears there are two approaches:
a) use a vanila (+ bigtodo and concurrency remote patches) qmail on
low-end boxes and x nodes of them. Use another process to dispatch jobs
based on XZY
b) Code a custom system that just includes all of your points, a job
distribution system to spread load to other boxes, and DNS look-up
caches etc ....
I would still be insterested in any ideas that intergrate with qmail
especially if they are simple ;-)
Greg
>
> -Jeff
