On Fri, Dec 10, 1999 at 09:12:01AM -0800, John White wrote:
> Did you see a discussion in the last 30 days about RAID levels?
>
> Is your /var/qmail/queue on the RAID1? Big write I/O reduction
> if you are.
IMHO this is NOT the problem. The problem is a IMHO design flaw in
qmail-send's processing loop.
It uses concurrency* to its full extend if and only if there a no new
messages to process.
If you have a bulk inject of lets say 10000 messages qmail-send
transfers them from the unpreprocessed queue to the processed queue.
Due to the design of the processing loop it does - within one loop - at most
process one unpreprocessed message and one for delivery. As soon as
all messages from the unprocessed queue are processed (processing is
kinda slow; Russell Nelsons big-todo-patch helps on some systems for
speedup and doesn't on some others, it's FS type dependant)
qmail "speeds up" and starts concurrency* number of deliveries and
becomes really fast. However if one of those messages bounces, it is
"reinjected" into the unpreprocessed queue and slows down qmail again.
This causes qmail to deliver in "waves".
I have graphically documented this (avail at
<URL:http://www.lamer.de/maex/creative/software/qmail/deliver-stats.gif>
and also asked on the list whether there is a workaround for this,
but it looks like there isn't.
As you can see from the above figure, as soon as qmail-send has an
empty unprocessed queue it starts (in my case) 240 concurrent (120 local
and 120 remote) deliveries. These are being processed really fast, but
they produce bounces, which slow down the (remote/local) deliveries for
some time.
During that time only few new deliveries are started until the unprocessed
queue ist empty again. Then qmail-send again starts up to 120 deliveries
and stays there, unless bounces happen and then slows down again.
If you look at the time frame 150-180 of the image you can see a case
where only one or two bounces happend, these were processed fast and
it is up at 120 again.
One way to somewhat overcome this on a "bulk only" mailserver, where it's
not too important that you have quick notice on new messages is to
chmod 000 /var/qmail/queue/lock/trigger
That way qmail has no communication with qmail-queue and checks for
unprocessed messages only each 20 minutes.
Thus the bounces don't come in the way of deliveries, so deliveries are
highly concurrent.
However this is no solution for a "default" mailserver.
Any comments on this analysis?
\Maex
--
SpaceNet GmbH | http://www.Space.Net/ | Stress is when you wake
Research & Development | mailto:[EMAIL PROTECTED] | up screaming and you
Joseph-Dollinger-Bogen 14 | Tel: +49 (89) 32356-0 | realize you haven't
D-80807 Muenchen | Fax: +49 (89) 32356-299 | fallen asleep yet.
