Hi
Looking for some hints on a performance problem i have with postfix and looping
mail through a content filter and it slowly feedback back out via the maildrop.
It would seem that pickup process is trickle feeding the maildrop back into the
active queue. I conscious that pickup is single thread but for comparison on
reboot of the match postfix will seemingly take the maildrop items real fast
and smtp send them out. But if I batch send the items in via postfix input
queue (from an external smtp server) it goes slow.
Scenario:
Stress test were I send 30-60k of messages to a single domain (not the best
test but its suits my requirements).
Jame SMTP server -> sends to postfix (25 connections) -> postfix setup with a
content filter that is a python script (see below) -> reinjects using send mail
-> pickup trickles messages into active -> the relay (smtp) to another mail
server but very slowly, i.e 1 a second or there about.
Again. If I reboot the box (restarting postfix doesn't seem to do it) the relay
sending goes out at 20 connections and really fast.
Snippet from the python script:
# pass message to python script which calls webservice
/usr/bin/python /etc/postfix/custom_scripts/audit/mime.py
/etc/postfix/custom_scripts/audit/messages/rawmessage_$$.msg "$@"
# || { echo Python script did not execute; exit $EX_UNAVAILABLE; }
# send message back to queue - strip the postfix instance name
/usr/sbin/sendmail -G -i -f "$1" "${@:3}"
</etc/postfix/custom_scripts/audit/messages/rawmessage_$$.msg
I don't believe the above is an issue. Its the fact I'm sendmail it to maildrop
and pickup is being slow.
CPU utilisation is moderate to low when pickup is being slow. IO wait is
averaging 20% so i don't think its a deliberate bottle neck. maildrop directory
entries are in the order of 20-30k.
default_destination_concurrency_limit = 50 - i've tried bumping this. Works on
the fresh restart but seems to have no bearing ones the postfix has been
running, i.e. concurrency doesn't ramp up.
in_flow_delay = 0, didn't seem to have any influence either.
My thoughts:
- on a reboot (but why not restart) pickup is detecting the maildrop items and
then pushing them fast into the active/incoming queue.
- when running normally and after delivery via sendmail pickup is not detecting
the items and only picking up at a rate limit - why?
Any ideas or suggestions would be appreciated.
Thanks
Raymond
