Re: [spamdyke-users] Greylisting effectiveness?
Then why am I not getting hammered with spam? Is it the failed-reverse-lookup that is saving me? On 7/9/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: > Overall, I suspect Eric suspects what I also believe -- graylisting isn't > effective any more. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
I don't know, I'm just going from my gut feeling here. Like Eric, I don't have a script to measure this either. I just ran a few quick greps on my own server's logs for today and found that out of 192 unique senders who were graylisted, 145 successfully delivered at least one message (76%). The number of rejections due to the DNS filters and local blacklists were 1218, successful deliveries were 190 (16%). On one of my customer's servers (configured very differently from mine), I see 2141 graylisted with 1618 successful (76%). DNS filters blocked 2039 but those senders somehow successfully delivered 1381 anyway (68%). Another server (with yet another configuration) shows 1560 graylisted with 1411 successes (90%). DNS filters blocked 5937 but those senders successfully delivered 4392 (74%). What does all that mean? I have no idea -- remember what Mark Twain said about statistics. I didn't do anything to match senders to recipients, check if the messages were actually spam, allow for frequent senders or mailing lists, check if the rejections came before or after the successes, etc. (For that matter, I'm not even completely sure my search commands were written correctly.) Also, since the DNS filters kick in before graylisting does, it's impossible to say how the graylisting percentage would change if I turned off all the DNS filters. Until those factors are accounted for, the numbers don't actually mean anything. Hopefully Eric's script will allow for all that (assuming he's writing one). :) -- Sam Clippinger On Jul 9, 2012, at 4:57 PM, BC wrote: > > > Then why am I not getting hammered with spam? Is it the > failed-reverse-lookup that is saving me? > > > On 7/9/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >> Overall, I suspect Eric suspects what I also believe -- graylisting isn't >> effective any more. > > > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 07/10/2012 04:48 PM, BC wrote: > > > How interesting. Well, whatever the reason I still only very > occasionally get any spam, yet when I look at the maillog there are > countless attempts to send me span each day. One in particular that > is amusing is to one email address I used exactly ONE time 10 years > ago. There are hundreds of attempts to send me email to that address, > every day. > > So spamdyke is still tops in my mind and I look forward to Eric's > findings. I've disabled graylisting on a few domains that are sensitive to timely delivery. They haven't complained about any increase in spam. You might try doing the same to see the effect. I expect that the various rDNS filters, along with blacklists, are doing an adequate job. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
I found that greylisting is still a significant deterrent. A rough examination of the total number of greylist files to the number of empty ones says that, after all the whitelist and blacklist operations, about 25% of the graylisted emails didn't get through. I regularly identify graylisted items that should be whitelisted, so that may bias the results.Gary-- Sent from my HP TouchPadOn Jul 10, 2012 8:17 PM, Eric Shubert wrote: On 07/10/2012 04:48 PM, BC wrote: > > > How interesting. Well, whatever the reason I still only very > occasionally get any spam, yet when I look at the maillog there are > countless attempts to send me span each day. One in particular that > is amusing is to one email address I used exactly ONE time 10 years > ago. There are hundreds of attempts to send me email to that address, > every day. > > So spamdyke is still tops in my mind and I look forward to Eric's > findings. I've disabled graylisting on a few domains that are sensitive to timely delivery. They haven't complained about any increase in spam. You might try doing the same to see the effect. I expect that the various rDNS filters, along with blacklists, are doing an adequate job. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 07/10/2012 05:34 PM, g...@genashor.com wrote: > A rough examination of the total number of greylist files to the number > of empty ones says that, after all the whitelist and blacklist > operations, about 25% of the graylisted emails didn't get through. Can you elaborate on this a little. All graylist files are empty ttbomk. I'm probably missing something. This does make me think, though, that perhaps a difference between created date/time and modified date/time would indicate one or more graylisted items which passed. IOW, if the created date/time is equal to the modified date/time, this would indicate a graylisted message that was blocked (so long as the date/time was significantly enough in the past, say a day old). Would this be correct? -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
Actually, graylist files are created empty when the first rejection is done. If the sender tries again and the connection is allowed, spamdyke puts the IP address and rDNS name of the remote server into the file. So comparing the number of zero-byte files to non-zero-byte files would give a number of how many successful deliveries were made after graylisting. -- Sam Clippinger On Jul 10, 2012, at 10:24 PM, Eric Shubert wrote: > On 07/10/2012 05:34 PM, g...@genashor.com wrote: >> A rough examination of the total number of greylist files to the number >> of empty ones says that, after all the whitelist and blacklist >> operations, about 25% of the graylisted emails didn't get through. > > Can you elaborate on this a little. All graylist files are empty ttbomk. > I'm probably missing something. > > This does make me think, though, that perhaps a difference between > created date/time and modified date/time would indicate one or more > graylisted items which passed. IOW, if the created date/time is equal to > the modified date/time, this would indicate a graylisted message that > was blocked (so long as the date/time was significantly enough in the > past, say a day old). Would this be correct? > > -- > -Eric 'shubes' > > > > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 7/11/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: > I've disabled graylisting on a few domains that are sensitive to timely > delivery. They haven't complained about any increase in spam. You might > try doing the same to see the effect. > > I expect that the various rDNS filters, along with blacklists, are doing > an adequate job. I'm not using any external blacklists, just what spamdyke does internally. Shall I risk it and see? The maillog shows a LOT of "greylisted" attempts that are never repeated. A LOT!!! ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 07/11/2012 10:40 AM, BC wrote: > > On 7/11/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >> I've disabled graylisting on a few domains that are sensitive to timely >> delivery. They haven't complained about any increase in spam. You might >> try doing the same to see the effect. >> >> I expect that the various rDNS filters, along with blacklists, are doing >> an adequate job. > > I'm not using any external blacklists, just what spamdyke does internally. > > Shall I risk it and see? > > The maillog shows a LOT of "greylisted" attempts that are never > repeated. A LOT!!! > I use: dns-blacklist-entry=zen.spamhaus.org dns-blacklist-entry=bl.spamcop.net It's very rare that these give a false positive. I would try them to see how they perform for you. -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 7/11/12 1:50 PM, Eric Shubert wrote: > On 07/11/2012 10:40 AM, BC wrote: >> On 7/11/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >>> I've disabled graylisting on a few domains that are sensitive to timely >>> delivery. They haven't complained about any increase in spam. You might >>> try doing the same to see the effect. >>> >>> I expect that the various rDNS filters, along with blacklists, are doing >>> an adequate job. >> I'm not using any external blacklists, just what spamdyke does internally. >> >> Shall I risk it and see? >> >> The maillog shows a LOT of "greylisted" attempts that are never >> repeated. A LOT!!! >> > I use: > dns-blacklist-entry=zen.spamhaus.org > dns-blacklist-entry=bl.spamcop.net > > It's very rare that these give a false positive. I would try them to see > how they perform for you. > I concur with your choices, to round off the list, I use these these which also have a very low false-positive result: b.barracudacentral.org zen.spamhaus.org dyna.spamrats.com ix.dnsbl.manitu.net I find barracudacentral to be a bit more robust than spamcop. Barracuda networks uses this in their own highly rated appliances. Zen is good because it tends to get spammers on the list quicker, but isn't as robust as barracudacentral. I've also found that right-hand side filtering (rhs-blacklist-file) is very effective. My list is: dbl.spamhaus.org urired.spameatingmonkey.net fresh15.spameatingmonkey.net The last one is good. It rejects email from domains that have been created within the last 15 days. You can use the 10 day list instead if you want. Lots of spam comes from throwaway domains. Once they start getting a high rate of rejection, they change the domain name. Waiting 15 days is usually enough for these to get listed on the other blacklists. I use an internal caching DNS server as a DNS forwarder for spamdyke's dns requests. This way I only need to query outside once, and subsequent spam bursts from the same server are rejected by local lookups to the cache. This dramatically lowers my pound rate on the above servers and gets subsequent spam rejected very quickly. I used to use dnscache, but I'm currently testing unbound as a replacement. Gary ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
That spameatingmonkey.net list is a great tip! There used to be one called "Day Old Bread" that did that same thing but it's been offline for a while and I had never found a replacement. -- Sam Clippinger On Jul 11, 2012, at 1:15 PM, Gary Gendel wrote: > On 7/11/12 1:50 PM, Eric Shubert wrote: >> On 07/11/2012 10:40 AM, BC wrote: >>> On 7/11/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: I've disabled graylisting on a few domains that are sensitive to timely delivery. They haven't complained about any increase in spam. You might try doing the same to see the effect. I expect that the various rDNS filters, along with blacklists, are doing an adequate job. >>> I'm not using any external blacklists, just what spamdyke does internally. >>> >>> Shall I risk it and see? >>> >>> The maillog shows a LOT of "greylisted" attempts that are never >>> repeated. A LOT!!! >>> >> I use: >> dns-blacklist-entry=zen.spamhaus.org >> dns-blacklist-entry=bl.spamcop.net >> >> It's very rare that these give a false positive. I would try them to see >> how they perform for you. >> > I concur with your choices, to round off the list, I use these these > which also have a very low false-positive result: > > b.barracudacentral.org > zen.spamhaus.org > dyna.spamrats.com > ix.dnsbl.manitu.net > > I find barracudacentral to be a bit more robust than spamcop. Barracuda > networks uses this in their own highly rated appliances. Zen is good > because it tends to get spammers on the list quicker, but isn't as > robust as barracudacentral. > > I've also found that right-hand side filtering (rhs-blacklist-file) is > very effective. My list is: > > dbl.spamhaus.org > urired.spameatingmonkey.net > fresh15.spameatingmonkey.net > > The last one is good. It rejects email from domains that have been > created within the last 15 days. You can use the 10 day list instead if > you want. Lots of spam comes from throwaway domains. Once they start > getting a high rate of rejection, they change the domain name. Waiting > 15 days is usually enough for these to get listed on the other blacklists. > > I use an internal caching DNS server as a DNS forwarder for spamdyke's > dns requests. This way I only need to query outside once, and > subsequent spam bursts from the same server are rejected by local > lookups to the cache. This dramatically lowers my pound rate on the > above servers and gets subsequent spam rejected very quickly. I used to > use dnscache, but I'm currently testing unbound as a replacement. > > Gary > > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
Am 10.07.2012 um 01:08 schrieb Sam Clippinger: > > I just ran a few quick greps on my own server's logs for today [...] Just for the record I did a little math on my greylist cleanup log files of this year. As for all stats it's value lies in the eye of the beer^h^hholder: I have an average delete of ~36 greylist files older than 7 days every day. At the same time my script deletes around 121 empty files, i.e. greylist files being empty and not younger than one day. It's, as one can see, not a high volume MTA, but seems to indicate there's still good reasons - at least for my domains - to do greylisting. Additionally I have to admit the variance is pretty huge. Smallest "emtpy" deletes in 2012 is 5, biggest 1295. Smallest "too old" is 3, biggest 92. Maybe I find a way to constantly monitor and stat-count MTA logs too, which could additionally give some numbers about other blocking reasons. -- Regards, Peter ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
Hello! Ok here some stats from some of our server with following setup: Spamdyke idle-timeout-secs=300 reject-identical-sender-recipient sender-blacklist-file=/var/qmail/control/blacklist_senders recipient-blacklist-file=/var/qmail/control/blacklist_recipients recipient-whitelist-file=/var/qmail/control/whitelist_recipients recipient-whitelist-file=/var/qmail/control/whitelist_recipients_cp ip-in-rdns-keyword-blacklist-file=/var/qmail/control/ip-in-rdns-keyword-blacklist-file ip-in-rdns-keyword-whitelist-file=/var/qmail/control/ip-in-rdns-keyword-whitelist-file ip-blacklist-file=/var/qmail/control/blacklist_ip ip-whitelist-file=/var/qmail/control/whitelist_ip reject-empty-rdns reject-unresolvable-rdns reject-missing-sender-mx rdns-whitelist-file=/var/qmail/control/whitelist_rdns Qmail Vpopmail Greylisting via qmail-spp Plugin Script Greylisting Keys are (Sender, Sender-IP - Recipient,Recipient-IP). Greylisting Time 300s Greylisting database clean-up once a day: Greylist-Timeout 12000s (greylisted older than 3h) Whitelist-Timeout 3110400s (no mails within 36 days) So it's a little bit different from using spamdyke's greylisting which may open another viewpoint for the topic. I have counted 51086 entries overall. The count of entries with only one single (initial greylisted) connection is 4056. All other table entries have counted one another mail at least. Therefor, only about 8% of the connections coming through spamdyke have been blocked by greylisting. Best, Hartmut On 11 Jul 12, Peter Palmreuther wrote: > Am 10.07.2012 um 01:08 schrieb Sam Clippinger: > > > > I just ran a few quick greps on my own server's logs for today [...] > > Just for the record I did a little math on my greylist cleanup log files of > this year. > As for all stats it's value lies in the eye of the beer^h^hholder: > > I have an average delete of ~36 greylist files older than 7 days every day. > At the same time my script deletes around 121 empty files, i.e. greylist > files being empty and not younger than one day. > > It's, as one can see, not a high volume MTA, but seems to indicate there's > still good reasons - at least for my domains - to do greylisting. > Additionally I have to admit the variance is pretty huge. Smallest "emtpy" > deletes in 2012 is 5, biggest 1295. Smallest "too old" is 3, biggest 92. > > Maybe I find a way to constantly monitor and stat-count MTA logs too, which > could additionally give some numbers about other blocking reasons. > -- > Regards, > > Peter > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users > ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting Effectiveness
We are getting away from the original thought of this thread... I get 1 spam per day, maybe. So I have no interest in using an outside blacklist checker. In my case it would merely be adding to background internet traffic clutter unnecessarily. Here is a line from my maillog: DENIED_RDNS_MISSING from: cd...@hotmail.com to: wcfgynhh90...@yahoo.com.tw origin_ip: 27.41.147.251 origin_rdns: (unknown) auth: (unknown) encryption: (none) reason: (empty) I have SCADS of lines like this (I have no idea who the to: or from: folks are - I only host one domain on my box). In my mind, this implies that the RDNS_MISSING function of spamdyke is keeping the OVERWHELMING majority of the spam out of my box. Am I misinterpreting this? On 7/12/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: > I use: > dns-blacklist-entry=zen.spamhaus.org > dns-blacklist-entry=bl.spamcop.net > > It's very rare that these give a false positive. I would try them to see > how they perform for you. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 7/12/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: > I use an internal caching DNS server as a DNS forwarder for spamdyke's > dns requests. This way I only need to query outside once, and > subsequent spam bursts from the same server are rejected by local > lookups to the cache. This dramatically lowers my pound rate on the > above servers and gets subsequent spam rejected very quickly. I used to > use dnscache, but I'm currently testing unbound as a replacement. Is this to say that you used to use djbdns for your caching DNS server but you are going to something else? ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 7/12/12 1:18 PM, BC wrote: > On 7/12/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >> I use an internal caching DNS server as a DNS forwarder for spamdyke's >> dns requests. This way I only need to query outside once, and >> subsequent spam bursts from the same server are rejected by local >> lookups to the cache. This dramatically lowers my pound rate on the >> above servers and gets subsequent spam rejected very quickly. I used to >> use dnscache, but I'm currently testing unbound as a replacement. > Is this to say that you used to use djbdns for your caching DNS server > but you are going to something else? Yes. I'm playing with unbound www.unbound.net > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting Effectiveness
Well, remember the filters run in a specific order. Graylisting is one of the very last filters to run -- it only gets a chance to reject connections that have already passed every other filter. So it's very possible some of the connections rejected by the missing rDNS filter would also have been stopped by graylisting, which would make graylisting's effectiveness appear higher. Ditto for the other tests like DNS blackholes, earlytalkers, etc. The only way to know for sure would be to disable every other filter and see what happens to the rejection rate. -- Sam Clippinger On Jul 12, 2012, at 12:15 PM, BC wrote: > > We are getting away from the original thought of this thread... > > I get 1 spam per day, maybe. So I have no interest in using an > outside blacklist checker. In my case it would merely be adding to > background internet traffic clutter unnecessarily. > > > Here is a line from my maillog: > > DENIED_RDNS_MISSING from: cd...@hotmail.com to: > wcfgynhh90...@yahoo.com.tw origin_ip: 27.41.147.251 origin_rdns: > (unknown) auth: (unknown) encryption: (none) reason: (empty) > > I have SCADS of lines like this (I have no idea who the to: or from: > folks are - I only host one domain on my box). In my mind, this > implies that the RDNS_MISSING function of spamdyke is keeping the > OVERWHELMING majority of the spam out of my box. > > Am I misinterpreting this? > > > > On 7/12/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >> I use: >> dns-blacklist-entry=zen.spamhaus.org >> dns-blacklist-entry=bl.spamcop.net >> >> It's very rare that these give a false positive. I would try them to see >> how they perform for you. > > > ___ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting effectiveness?
On 07/12/2012 10:36 AM, Gary Gendel wrote: > On 7/12/12 1:18 PM, BC wrote: >> On 7/12/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: >>> I use an internal caching DNS server as a DNS forwarder for spamdyke's >>> dns requests. This way I only need to query outside once, and >>> subsequent spam bursts from the same server are rejected by local >>> lookups to the cache. This dramatically lowers my pound rate on the >>> above servers and gets subsequent spam rejected very quickly. I used to >>> use dnscache, but I'm currently testing unbound as a replacement. >> Is this to say that you used to use djbdns for your caching DNS server >> but you are going to something else? > Yes. I'm playing with unbound www.unbound.net > FWIW, I use PowerDNS now. (pdns-recursor package for CentOS) -- -Eric 'shubes' ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
Re: [spamdyke-users] Greylisting Effectiveness
Right. But the bottom line is that spamdyke is still doing a fabulous job of blocking spam by whatever filter is doing it. Thanks. On 7/13/2012 11:00 AM, spamdyke-users-requ...@spamdyke.org wrote: > Well, remember the filters run in a specific order. Graylisting is one of > the very last filters to run -- it only gets a chance to reject connections > that have already passed every other filter. So it's very possible some of > the connections rejected by the missing rDNS filter would also have been > stopped by graylisting, which would make graylisting's effectiveness appear > higher. Ditto for the other tests like DNS blackholes, earlytalkers, etc. > > The only way to know for sure would be to disable every other filter and see > what happens to the rejection rate. ___ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users
