Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On 03/28/2013 06:52 PM, Sarah Caswell wrote: Hi all, I had a question about greylisting (with spamd) in production. I've successfully run spamd on firewalls (as a frontend to either barracuda or SpamAssassin) and have really liked the reduction in SPAM volume. Unfortunately my employer's wife does not like the delays that this introduces into our mail delivery, since she uses email for quick turn-around communication. The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? Or will this mostly be a DIY effort on my part? Any thoughts/insights/experiences would be greatly appreciated. I used to use an rsync'ed uceprotect.net whitelists. then feed them into manual-white table of my pf. works as a charm. -- With best regards, Gregory Edigarov
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On Mar 28 12:52:46, s.casw...@protocol6.com wrote: Hi all, I had a question about greylisting (with spamd) in production. I've successfully run spamd on firewalls (as a frontend to either barracuda or SpamAssassin) and have really liked the reduction in SPAM volume. Unfortunately my employer's wife does not like the delays that this introduces into our mail delivery, since she uses email for quick turn-around communication. The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? For e.g. google, I did dig -t txt _netblocks.google.com | grep spf and put the following in /etc/mail/nospamd 173.194.0.0/16 209.85.128.0/17 74.125.0.0/16 12.31.165.64/27 208.48.95.16/28 216.34.181.0/24 I left out 64.18.0.0/20 and 207.126.144.0/20 as they were sending me a lot of spam. These lists could change of course, but I never got to automatizing it. Jan
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On 2013-03-28, Sarah Caswell s.casw...@protocol6.com wrote: Are there any up-to-date lists that already track the MTAs of these large mail providers? Yes, dnswl.org, but you have to pay if you need rsync access, otherwise you can just do DNS lookups.
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On Thu, 28 Mar 2013 21:16:38 -0600 Steve Williams st...@williamsitconsulting.com wrote: It's designed to work with sites that use spf records, and it doesn't know about ip6, not an issue in my case If you are interested in my script, feel free to contact me off list SPF for spamd. I'd be interested if you don't mind. Thanks
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
Hi Sarah,
On 3/28/2013 10:52 AM, Sarah Caswell wrote:
I had a question about greylisting (with spamd) in production.
I've successfully run spamd on firewalls (as a frontend to either
barracuda or SpamAssassin) and have really liked the reduction in SPAM
volume.
Unfortunately my employer's wife does not like the delays that this
introduces into our mail delivery, since she uses email for quick
turn-around communication.
The main problem occurs with senders like Gmail, yahoo, hotmail, etc.
...i.e. all the senders that have large farms of smtp servers from
which they can retry delivery after initial greylisting delay.
I know this means I'm not doing proper whitelisting of those major
sender domains, but I'm at a loss on how to best construct and
maintain such a whitelist.
Are there any up-to-date lists that already track the MTAs of these
large mail providers?
Or will this mostly be a DIY effort on my part?
Any thoughts/insights/experiences would be greatly appreciated.
I understand this email is about spamd(8), but I would like to mention
milter-greylist though, I think it is worth for people searching the net
for such a problem. You can build a ruleset to decide what you want to
whitelist/greylist/blacklist. The documentation is good enough,
sometimes a little bit terse, but you basically end up with a ruleset
like this which is pretty straightforward to understand and powerfull
enough:
% #
% # System config boilerplate skipped
% ...
%
% #
% # Some definitions.
% list my network addr { \
% 127.0.0.1/8 \
% 10.0.0.0/8 \
% 1.2.3.4 \ # MX 1
% 5.6.7.8 \ # MX 2
% }
% # provided by default
% list broken mta addr { \
% 12.5.136.141/32\ # Southwest Airlines (unique sender)
% 12.5.136.142/32\ # Southwest Airlines
% ...
% }
% dnsrbl SPL-XBL zen.spamhaus.org 127.0.0.0/29
% dnsrbl PBL zen.spamhaus.org 127.0.0.10/31
%
% #
% # Now the ruleset
% racl whitelist list my network
% racl whitelist list broken mta
% racl whitelist domain freebsd.org
% racl whitelist domain openbsd.org
% racl blacklist dnsrbl SPL-XBL msg Sorry, refused by SPL/XBL
% racl whitelist spf pass
% racl blacklist spf fail msg Sorry, your IP is refused by SPF
% racl greylist dnsrbl PBL delay 30m autowhite 3d msg Please retry later
minutes, you are caught by PBL
% racl greylist default delay 5m autowhite 15d
As a bonus, you can synchronize the greylist and auto-built whitelist
between your different MXes.
--
Jeremie Le Hen
Scientists say the world is made up of Protons, Neutrons and Electrons.
They forgot to mention Morons.
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
This helps me quite some. https://alexander.beard.se/spub/6dd1a43187c19431d087c91e403b756b/index.html No man page (yet) but the code should should reveal the secrets of most options and switches. Simple start: # spamwhite -nv Two cron jobs: spamwhite every few minutes (it's cheap). spamwhite -a once per day or so to pick up changes. Not as cheap. /Alexander On 03/28/13 17:52, Sarah Caswell wrote: Hi all, I had a question about greylisting (with spamd) in production. I've successfully run spamd on firewalls (as a frontend to either barracuda or SpamAssassin) and have really liked the reduction in SPAM volume. Unfortunately my employer's wife does not like the delays that this introduces into our mail delivery, since she uses email for quick turn-around communication. The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? Or will this mostly be a DIY effort on my part? Any thoughts/insights/experiences would be greatly appreciated. :-) Sarah
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
Hi there, I currently use and recommend spf2cidr by Todd Fries: https://github.com/toddfries/spf2cidr It populates a file with the IP addresses from SPF records from a list of domains specified. This file is then used on a pf table. cheers, --rodolfo
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On 03/29/13 16:22, Alexander Hall wrote: This helps me quite some. https://alexander.beard.se/spub/6dd1a43187c19431d087c91e403b756b/index.html And now with a little license. https://alexander.beard.se/spub/21e51b4ac875d82d8e10dbf67a79cfa9/index.html /Alexander
[Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
Hi all, I had a question about greylisting (with spamd) in production. I've successfully run spamd on firewalls (as a frontend to either barracuda or SpamAssassin) and have really liked the reduction in SPAM volume. Unfortunately my employer's wife does not like the delays that this introduces into our mail delivery, since she uses email for quick turn-around communication. The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? Or will this mostly be a DIY effort on my part? Any thoughts/insights/experiences would be greatly appreciated. :-) Sarah -- To speak another language is to possess another soul - Charlemagne
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
Sarah Caswell s.casw...@protocol6.com writes: The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? I think you would need to construct it by hand. I very occasionally update my /etc/mail/nospamd, and then mostly by looking for relevant domains' published spf records. for example for gmail, [Thu Mar 28 18:49:27] peter@deeperthought:~$ host -ttxt gmail.com gmail.com descriptive text v=spf1 redirect=_spf.google.com [Thu Mar 28 18:49:37] peter@deeperthought:~$ host -ttxt _spf.google.com _spf.google.com descriptive text v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ?all [Thu Mar 28 18:52:02] peter@deeperthought:~$ for foo in _netblocks.google.com _netblocks2.google.com _netblocks3.google.com ; do host -ttxt $foo ; done _netblocks.google.com descriptive text v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:209.85.128.0/17 ip4:66.102.0.0/20 ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ip4:173.194.0.0/16 ?all _netblocks2.google.com descriptive text v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ?all _netblocks3.google.com descriptive text v=spf1 ?all and so forth. Not all domains publish SPF, and you may need to work around a certain bitrot factor. And spend some time poring over your spamd log to weed out the non-obvius ones. Then again, I just decided to share mine, which is the product of just the process I've described. It's up at http://www.bsdly.net/~peter/nospamd free to use, corrections welcome (will be rewarded with a personal thank you message ;)). - Peter -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ Remember to set the evil bit on all malicious network traffic delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: [Question] Building whitelists so that spamd greylisting can work without users perceiving delivery delays...
On 3/28/2013 10:52 AM, Sarah Caswell wrote: Hi all, I had a question about greylisting (with spamd) in production. I've successfully run spamd on firewalls (as a frontend to either barracuda or SpamAssassin) and have really liked the reduction in SPAM volume. Unfortunately my employer's wife does not like the delays that this introduces into our mail delivery, since she uses email for quick turn-around communication. The main problem occurs with senders like Gmail, yahoo, hotmail, etc. ...i.e. all the senders that have large farms of smtp servers from which they can retry delivery after initial greylisting delay. I know this means I'm not doing proper whitelisting of those major sender domains, but I'm at a loss on how to best construct and maintain such a whitelist. Are there any up-to-date lists that already track the MTAs of these large mail providers? Or will this mostly be a DIY effort on my part? Any thoughts/insights/experiences would be greatly appreciated. :-) Sarah Hi, Years ago I was faced with the same frustration on my own system. I ended up writing a shell/awk script that I run 2x a day. Basically, you build up a list of trusted hosts and whitelist them. Whenever I got delayed mail that I noticed, I would add the hostname to the trusted list and my script would automatically whitelist them the next time it ran (or when I ran it manually). It may not be perfect, but it's worked flawlessly for probably 4 years now. It's designed to work with sites that use spf records, and it doesn't know about ip6, not an issue in my case If you are interested in my script, feel free to contact me off list The output for google.com is: #--- # google.com #--- # Got 5 elements in [v=spf1 include:_spf.google.com ip4:216.73.93.70/31 ip4:216.73.93.72/31 ~all] # queueing for spf lookup: [_spf.google.com] 216.73.93.70/31 216.73.93.72/31 # == # Recursing for additional spf records # == #--- # _spf.google.com #--- # Got 5 elements in [v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ?all ] # queueing for spf lookup: [_netblocks.google.com] # queueing for spf lookup: [_netblocks2.google.com] # queueing for spf lookup: [_netblocks3.google.com] # == # Recursing for additional spf records # == #--- # _netblocks.google.com #--- # Got 12 elements in [v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:209.85.128.0/ 17 ip4:66.102.0.0/20 ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ip4:173.194.0.0/16 ?all] 216.239.32.0/19 64.233.160.0/19 66.249.80.0/20 72.14.192.0/18 209.85.128.0/17 66.102.0.0/20 74.125.0.0/16 64.18.0.0/20 207.126.144.0/20 173.194.0.0/16 #--- # _netblocks2.google.com #--- # Got 8 elements in [v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 i p6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ?all] # UNKNOWN: [ip6:2001:4860:4000::/36] # UNKNOWN: [ip6:2404:6800:4000::/36] # UNKNOWN: [ip6:2607:f8b0:4000::/36] # UNKNOWN: [ip6:2800:3f0:4000::/36] # UNKNOWN: [ip6:2a00:1450:4000::/36] # UNKNOWN: [ip6:2c0f:fb50:4000::/36] #--- # _netblocks3.google.com #--- # Got 2 elements in [v=spf1 ?all] # Returning from recursion # Returning from recursion
