Re: [spamdyke-users] Recipient blacklist vs. RDNS checks
Very nice explanation Sam. Thanks for all you do. -- -Eric 'shubes' On 02/14/2012 06:53 PM, Sam Clippinger wrote: > Yes and no. From a purely academic standpoint, it takes less work/time for > spamdyke to reject a blacklisted recipient than to perform the DNS tests > because searching a file is faster than sending and receiving network data > (assuming the file isn't huge). And yes, spamdyke re-reads all of its files > (config files, whitelist, blacklist, graylist) for every incoming connection. > Because the OS caches disk access, this doesn't incur much actual overhead. > > However, several factors make this a non-issue. First, your DNS server is > caching the results for the frequent senders, so there's actually very little > traffic being generated for those queries. Second, spamdyke runs its filters > in a specific order (listed in the FAQ) in order to disqualify a connection > as quickly as possible. This is because qmail must remain running as long as > there is a chance the message will be accepted. As soon as spamdyke is sure > the message will be rejected, it tells qmail to quit and continues talking to > the remote server by itself. From a performance standpoint, closing the > process and freeing the memory is a bigger win than the file/DNS comparison. > > Third, and most importantly, spamdyke is going to run the DNS queries whether > you add the recipients to your blacklist or not. In order to try to reject a > message as soon as possible, spamdyke runs its filters as soon as the > required information is available: rDNS tests are run as soon as spamdyke > starts, MX checks are run as soon as the sender is given, etc. However, even > if those tests are positive, spamdyke refrains from sending a rejection until > it's sure the message cannot possibly be accepted. For example, if you use a > recipient whitelist, spamdyke can't reject a message until it sees the > recipient address -- otherwise it might reject a message too early when the > recipient is actually on the whitelist. The recipient is identified pretty > late in the SMTP protocol, so spamdyke may > have to hold its rejection for a while for safety. (In reality, "a while" > is typically hundredths of a second.) > > So by the time the recipient address is given and spamdyke /could/ check the > recipient blacklist, it's already done the DNS work. If the DNS tests > triggered a filter, the recipient blacklist won't be checked at all. So > there's really no point in using your spamdyke rejection messages to create a > recipient blacklist -- it'll never be used anyway. > > Caveat: the third point above doesn't apply if configuration directories are > in use. In that scenario, spamdyke doesn't run any tests until the recipient > address is given, so it can first load the config files from the correct > configuration directory(s). When that happens, the recipient blacklist is > checked before the DNS tests are run. > > Overall, my advice is: don't worry about it. If your server is so heavily > loaded that a few milliseconds of processing time are critical, you should > upgrade the hardware or get a second server (or both). > > -- Sam Clippinger > > > > > On Feb 14, 2012, at 4:58 PM, Angus McIntyre wrote: > >> Watching the logs on my new mail server, I'm having the pleasure of seeing >> spamdyke knocking lots of incoming spam on the head. >> >> In most cases, the incoming messages are getting taken out by RBL_MATCH, >> SENDER_NO_MX or RDNS_MISSING rules. A lot of the messages would eventually >> fail anyway because they're being sent to non-existent recipients. >> >> My question is, should I bother adding those non-existent recipients to >> the recipient blacklist file? Does Spamdyke do less work/take less time to >> reject a message if it finds the recipient in a blacklist than if it has >> to do an RBL or RDNS check? >> >> I imagine that simple string-matching should be faster and more efficient >> than doing a network-check (RBL or RDNS), but it probably depends on the >> order in which Spamdyke does the checks, and whether it re-reads the >> blacklist file for each message it processes. >> >> Any recommendations? >> >> Angus >> >> ___ >> spamdyke-users mailing list >> spamdyke-users@spamdyke.org >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Recipient blacklist vs. RDNS checks
Yes and no. From a purely academic standpoint, it takes less work/time for spamdyke to reject a blacklisted recipient than to perform the DNS tests because searching a file is faster than sending and receiving network data (assuming the file isn't huge). And yes, spamdyke re-reads all of its files (config files, whitelist, blacklist, graylist) for every incoming connection. Because the OS caches disk access, this doesn't incur much actual overhead. However, several factors make this a non-issue. First, your DNS server is caching the results for the frequent senders, so there's actually very little traffic being generated for those queries. Second, spamdyke runs its filters in a specific order (listed in the FAQ) in order to disqualify a connection as quickly as possible. This is because qmail must remain running as long as there is a chance the message will be accepted. As soon as spamdyke is sure the message will be rejected, it tells qmail to quit and continues talking to the remote server by itself. From a performance standpoint, closing the process and freeing the memory is a bigger win than the file/DNS comparison. Third, and most importantly, spamdyke is going to run the DNS queries whether you add the recipients to your blacklist or not. In order to try to reject a message as soon as possible, spamdyke runs its filters as soon as the required information is available: rDNS tests are run as soon as spamdyke starts, MX checks are run as soon as the sender is given, etc. However, even if those tests are positive, spamdyke refrains from sending a rejection until it's sure the message cannot possibly be accepted. For example, if you use a recipient whitelist, spamdyke can't reject a message until it sees the recipient address -- otherwise it might reject a message too early when the recipient is actually on the whitelist. The recipient is identified pretty late in the SMTP protocol, so spamdyke may have to hold its rejection for a while for safety. (In reality, "a while" is typically hundredths of a second.) So by the time the recipient address is given and spamdyke /could/ check the recipient blacklist, it's already done the DNS work. If the DNS tests triggered a filter, the recipient blacklist won't be checked at all. So there's really no point in using your spamdyke rejection messages to create a recipient blacklist -- it'll never be used anyway. Caveat: the third point above doesn't apply if configuration directories are in use. In that scenario, spamdyke doesn't run any tests until the recipient address is given, so it can first load the config files from the correct configuration directory(s). When that happens, the recipient blacklist is checked before the DNS tests are run. Overall, my advice is: don't worry about it. If your server is so heavily loaded that a few milliseconds of processing time are critical, you should upgrade the hardware or get a second server (or both). -- Sam Clippinger On Feb 14, 2012, at 4:58 PM, Angus McIntyre wrote: > Watching the logs on my new mail server, I'm having the pleasure of seeing > spamdyke knocking lots of incoming spam on the head. > > In most cases, the incoming messages are getting taken out by RBL_MATCH, > SENDER_NO_MX or RDNS_MISSING rules. A lot of the messages would eventually > fail anyway because they're being sent to non-existent recipients. > > My question is, should I bother adding those non-existent recipients to > the recipient blacklist file? Does Spamdyke do less work/take less time to > reject a message if it finds the recipient in a blacklist than if it has > to do an RBL or RDNS check? > > I imagine that simple string-matching should be faster and more efficient > than doing a network-check (RBL or RDNS), but it probably depends on the > order in which Spamdyke does the checks, and whether it re-reads the > blacklist file for each message it processes. > > Any recommendations? > > Angus > > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
[spamdyke-users] Recipient blacklist vs. RDNS checks
Watching the logs on my new mail server, I'm having the pleasure of seeing spamdyke knocking lots of incoming spam on the head. In most cases, the incoming messages are getting taken out by RBL_MATCH, SENDER_NO_MX or RDNS_MISSING rules. A lot of the messages would eventually fail anyway because they're being sent to non-existent recipients. My question is, should I bother adding those non-existent recipients to the recipient blacklist file? Does Spamdyke do less work/take less time to reject a message if it finds the recipient in a blacklist than if it has to do an RBL or RDNS check? I imagine that simple string-matching should be faster and more efficient than doing a network-check (RBL or RDNS), but it probably depends on the order in which Spamdyke does the checks, and whether it re-reads the blacklist file for each message it processes. Any recommendations? Angus ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
