Re: speeding dkim filtering
Hi, The incoming queue was big and increased continually ( from 2000 to up to 5 messages) The active queue was full. The traffic was not to too heavy ( 5,000- messages per 10 minutes). Trivial-rewrite is using LDAP lookups (locally) to route messages. System CPU, disk or network were NOT starved. The qmgr was getting enough CPU, RAM (i do not know for disk I/O inputs). What I also noticed is a lot of qmgr error log lines: Jan 19 08:01:41 fe2 postfix/qmgr[9644]: 5DF7D12AAF4: to=s...@foo.org, relay=none, delay=4825, delays=4825/0.11/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]: Connection refused) and also cleanup error log lines: Jan 19 00:19:22 fe2 postfix/cleanup[22033]: 17A48126FA4: milter-reject: END-OF-MESSAGE from srv[a.b.c.d]: 4.7.1 Service unavailable - try again later; from=i...@foo.com to=a...@foo.org proto=SMTP helo=foo.com and also Kernel logs warnings (kernel: VFS: file-max limit 16384 reached) Can you explain the qmgr log I had (qmgr wants to connect to what at 127.0.0.1 ?) ? Can you explain the cleanup log I had (why milter-reject in a cleanup log line ? while milter uses the smtpd daemon = smtpd_milters = inet:localhost:10030 ) ? Finally I increased the sys file-max parameter to 32768 and things went back to a normal situation. Any similar cases ?(I looked at Postfix archives but without success) Thank you. Alain - Original Message - From: test Victor Duchovni victor.ducho...@morganstanley.com Date: Tuesday, January 20, 2009 8:14 pm Subject: Re: speeding dkim filtering To: postfix-users@postfix.org On Tue, Jan 20, 2009 at 07:25:20PM +0100, postfix wrote: I have milter/dkim filter installed on a rhel4 linux server. I noticed a delay between dkim-filter and qmgr processes when the traffic becomes important. Jan 20 12:35:04 fe2 dkim-filter[3380]: 9E463127A68 DKIM-Signature header added Jan 20 12:43:14 fe2 postfix/qmgr[20888]: 9E463127A68: from=jairo.ab...@foo.com, size=11787, nrcpt=1 Is it possible to speed up dkim filter? It is not clear that the delay you are reporting is the result of the DKIM milter. How big is your incoming queue? Is your system CPU, disk or network starved? The queue manager serially imports mail from the incoming queue. Messagesare elgible for import as soon as cleanup processing is complete. What we don't know here is when cleanup (including milter) processing completed.We also don't know whether trivial-rewrite is using any high-latency lookup tables, whether the queue manager is not getting enough CPU or disk I/O, whether the active queue is full, ... You need to identify the cause of the delay first. -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. To unsubscribe from the postfix-users list, visit http://www.postfix.org/lists.html or click the link below: mailto:majord...@postfix.org?body=unsubscribe%20postfix-users If my response solves your problem, the best way to thank me is to not send an it worked, thanks follow-up. If you must respond, please put It worked, thanks in the Subject so I can delete these quickly.
Re: speeding dkim filtering
On Fri, Jan 23, 2009 at 07:24:08PM +0100, postfix wrote: The incoming queue was big and increased continually ( from 2000 to up to 5 messages) Symptom. The active queue was full. Cause. The traffic was not to too heavy ( 5,000- messages per 10 minutes). Trivial-rewrite is using LDAP lookups (locally) to route messages. Potential problem if LDAP lookup latency is high, because this kills queue-manager throughput, and there is only one queue manager. System CPU, disk or network were NOT starved. The qmgr was getting enough CPU, RAM (i do not know for disk I/O inputs). Yes, but what about latency? How quick are those LDAP lookups? Jan 19 08:01:41 fe2 postfix/qmgr[9644]: 5DF7D12AAF4: to=s...@foo.org, relay=none, delay=4825, delays=4825/0.11/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]: Connection refused) Well, you have a content filter that is unresponsive. What is it? Jan 19 00:19:22 fe2 postfix/cleanup[22033]: 17A48126FA4: milter-reject: END-OF-MESSAGE from srv[a.b.c.d]: 4.7.1 Service unavailable - try again later; from=i...@foo.com to=a...@foo.org proto=SMTP helo=foo.com Well, your milter is broken. Is this pre-filter mail or after filter re-injection. and also Kernel logs warnings (kernel: VFS: file-max limit 16384 reached) That's really bad, your process limits, ... are too large for the kernel's default resource limits (16K descriptors is way too low for a modern server, you should raise those unless running with very low RAM). Can you explain the qmgr log I had (qmgr wants to connect to what at 127.0.0.1 ?) ? Can you explain the cleanup log I had (why milter-reject in a cleanup log line ? while milter uses the smtpd daemon = smtpd_milters = inet:localhost:10030 ) ? Finally I increased the sys file-max parameter to 32768 and things went back to a normal situation. Your system is hosed in multiple ways. I can't help with quite this much breakage. You need on site help, or someone very patient with a lot of time who can help you off list. -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. To unsubscribe from the postfix-users list, visit http://www.postfix.org/lists.html or click the link below: mailto:majord...@postfix.org?body=unsubscribe%20postfix-users If my response solves your problem, the best way to thank me is to not send an it worked, thanks follow-up. If you must respond, please put It worked, thanks in the Subject so I can delete these quickly.
speeding dkim filtering
Hi I have milter/dkim filter installed on a rhel4 linux server. I noticed a delay between dkim-filter and qmgr processes when the traffic becomes important. Jan 20 12:35:04 fe2 dkim-filter[3380]: 9E463127A68 DKIM-Signature header added Jan 20 12:43:14 fe2 postfix/qmgr[20888]: 9E463127A68: from=jairo.ab...@foo.com, size=11787, nrcpt=1 Is it possible to speed up dkim filter? By default I have : postfix mail_version = 2.3.13 milter9629 1 1 Jan19 ? 00:13:05 /usr/sbin/dkim-filter -x /etc/dkim-milter/dkim-filter.conf lsof -i TCP:10030 | grep dkim-filt | wc -l 53 Thanks for your help Alain
