Thanks for the reply Daniel.
We have had delay problems with SBL, RCFI, DYNABLOCK, NJABL - even SPAMCOP gets overloaded on occasion. Once we start getting network test delays the problems cascade very quickly and load averages escalate to the stratosphere.
I question your statement that these DNSRBL can handle the load. Our mailservers are handling over 10K messages per hour - but to be conservative assume there are a million SA boxes checking 1.0K messages per hour. Is it reasonable to assume that each DNSRBL can handle a billion queries an hour?
I also believe that SA should provide a mechanism for logging network test response times. We need to be able to quickly turn off tests that create delays.
At 12:06 AM 8/17/2004, Daniel Quinlan wrote:
Jeff Koch <[EMAIL PROTECTED]> writes:
> In our experience we have traced the problem to the large number of RBL's, > DNSBL's, and other network tests (SPF, SURL, SORB) that seem to be > multiplying in SA.
DNSBLs are the fastest network tests in reasonably recent SA versions. SPF (in 3.0), the MX check (in 2.6x), and distributed checksum tests (DCC, Razor2, Pyzor) are generally the slow network tests.
> Not counting DCC and RAZOR there must be over a dozen that are now > consulted for every email. If you consider the hundreds of thousands > of boxes now using SA and the billions of emails processed by SA these > RBL's are just getting overloaded.
Millions of boxes, and not really.
> Has anybody thought about that? And is anybody considering response time
> and load capacity of an RBL before adding it to SA? No wonder we're getting
> time-outs on some of these tests and our servers are overloading with SA
> processes sitting around waiting for RBL results.
Yes, we've thought about it and yes, we test DNSBL speed before adding them and check with the people running them before publishing with them. All obvious stuff.
> I think before any other network tests are added we need some logging added
> to SA that would show the response speed of each network test. Then we'd be
> able to quickly turn off those that are timing-out instead of the trial and
> error method we use now.
If a DNSBL is timing it, SA doesn't wait on it. It's the other tests that are problematic. Hopefully, the plugin architecture will help improve DCC, Razor2, and Pyzor by the time 3.1 rolls around. SPF is going to be a tough cookie to speed up.
Daniel
-- Daniel Quinlan http://www.pathname.com/~quinlan/
Best Regards,
Jeff Koch, Intersessions
