Re: How can I debug a timing out milter
ram: > > On Mon, 2008-09-22 at 13:03 -0400, Wietse Venema wrote: > > ram: > > > My milter is quiet simple. It just does a bsearch on a in-memory array , > > > to find if the recipient has blacklisted / whitelisted the sender and > > > takes action accordingly > > > > > > The array now has approx 200k elements, which should be nothing for > > > 4GBRam box > > > > What measures did you take to avoid errors (race conditions, > > deadlock, etc.) due to concurrent access to this data structure? > > > > The array is constant once the milter is started. To refresh the array I > restart the milter every hour > Do I have to still take care of deadlocks ? Not is queries are delayed until the array stops changing. However there is a possibility of disruption after the milter is stopped. > Now I think this was due to some network issue. The ethernet links were > on half-duplex for this machine. I have forced a full duplex and now the > problem is apparently disappeared. ( fingers crossed ) Wietse
Re: How can I debug a timing out milter
On Mon, 2008-09-22 at 13:03 -0400, Wietse Venema wrote: > ram: > > My milter is quiet simple. It just does a bsearch on a in-memory array , > > to find if the recipient has blacklisted / whitelisted the sender and > > takes action accordingly > > > > The array now has approx 200k elements, which should be nothing for > > 4GBRam box > > What measures did you take to avoid errors (race conditions, > deadlock, etc.) due to concurrent access to this data structure? > The array is constant once the milter is started. To refresh the array I restart the milter every hour Do I have to still take care of deadlocks ? Now I think this was due to some network issue. The ethernet links were on half-duplex for this machine. I have forced a full duplex and now the problem is apparently disappeared. ( fingers crossed )
Re: How can I debug a timing out milter
ram: > My milter is quiet simple. It just does a bsearch on a in-memory array , > to find if the recipient has blacklisted / whitelisted the sender and > takes action accordingly > > The array now has approx 200k elements, which should be nothing for > 4GBRam box What measures did you take to avoid errors (race conditions, deadlock, etc.) due to concurrent access to this data structure? Wietse
Re: How can I debug a timing out milter
On Mon, 2008-09-22 at 08:08 -0400, Wietse Venema wrote: > ram: > > I have implemented a custom whitelist/blacklist with a milter. This > > milter has been working smoothly for a nearly 2 years now on multiple > > machines > > > > But now On 1 machine even if the load is very low and there is ample > > free memory once the number of smtpd processes reaches 300 ( I have set > > limit to 650 ) some milter processes start timing out. Some smtp > > connections only are affected most still go thru > > All Postfix SMTP servers connect to the same milter process. This > means that the milter can see hundreds of concurrent connections > from Postfix at the same time. > > Apparently, some milters stop working properly under conditions of > high concurrency. Perhaps you can configure the milter to reserve > space for more. > My machine has enough memory. Is that what is meant by space How do I "reserve space for more" milters. I have been STFW-ing but with no results My milter is quiet simple. It just does a bsearch on a in-memory array , to find if the recipient has blacklisted / whitelisted the sender and takes action accordingly The array now has approx 200k elements, which should be nothing for 4GBRam box Thanks Ram
Re: How can I debug a timing out milter
ram: > I have implemented a custom whitelist/blacklist with a milter. This > milter has been working smoothly for a nearly 2 years now on multiple > machines > > But now On 1 machine even if the load is very low and there is ample > free memory once the number of smtpd processes reaches 300 ( I have set > limit to 650 ) some milter processes start timing out. Some smtp > connections only are affected most still go thru All Postfix SMTP servers connect to the same milter process. This means that the milter can see hundreds of concurrent connections from Postfix at the same time. Apparently, some milters stop working properly under conditions of high concurrency. Perhaps you can configure the milter to reserve space for more. Wietse > I restart and then everything works fine again for some time > > How can I debug this ? > > Thanks > Ram > > > > >
How can I debug a timing out milter
I have implemented a custom whitelist/blacklist with a milter. This milter has been working smoothly for a nearly 2 years now on multiple machines But now On 1 machine even if the load is very low and there is ample free memory once the number of smtpd processes reaches 300 ( I have set limit to 650 ) some milter processes start timing out. Some smtp connections only are affected most still go thru I restart and then everything works fine again for some time How can I debug this ? Thanks Ram
