Re: blacklists
On Sa, 11.12.2004, 01:18, Michael Loftis wrote: ... Actually as it turns out I can't send mail directly to you, something about my @modwest.com address is apprently offensive to your mailserver... 550 Sender access denied for [EMAIL PROTECTED] -- I sent a second one from my home address but it won't get delivered or bounced for a while since I'm not likely being blocked at your firewall ;) JAMM (I think it's much too agressive to use it) said: Result for 216.220.25.60: Sent a notice to the ISP that owns the block containing this IP. They have not yet provided the information requested. Christian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 11:18:16PM -0700, Michael Loftis wrote: --On Friday, December 10, 2004 16:43 +1100 Craig Sanders [EMAIL PROTECTED] wrote: DoS is a huge exaggeration. a few smtpd processes waiting to timeout does not constitute a DoS. neither does a few dozen. I had about 800 waiting around in just a few minutes on the one server I began testing it on, but this is a large installation. And this isn't peak time...It's holding at around 1000 blocked hosts, most of them for blacklist infractions. i certainly wouldn't recommend running it on a large installation. i'm surprised you even tried. i run it on my home system at the moment. i wouldn't run it at work. i experiment with lots of things on my home system that i wouldn't even think of doing at work. some of them, very few, actually turn out to be worthwhile and safe enough to use at work. try dropping only SYN smtp packets if you still want to experiment with it, adding --syn to the end of the iptables args in the scripts. that should stop the hanging processes. But when you've got a lot of mail (and a number of customer domains that just tend to attract junk) it's easy to get a lot of processes hanging around. unfortunately, my domain seems to attract a lot of junk. i've had my domain for over 10 years, and kept the same email address all along. and i've been joe-jobbed many times over the last decade by spammers who don't like me (or my anti-spam methods, or the fact that i share them openly), and i've had thousands of bogus, non-existant addresses in my domain added to spam lists also by spammers who don't like me. the current crop of spammers probably don't even notice or care, but in the early days of spam it was different. spammers got very offended and took it personally...which, of course, was excellent incentive to keep on blocking them :) i pissed off quite a few in the very early days, when spammers didn't hide their identities and hadn't yet learned not to use their own address. one of the things i wrote was a script which i could bounce spam to. it would then parse the sender addresses and add it to a database of spammersand sent copies of each spam to a random subset of the database. that infuriated them and amused me no end. my intention was to annoy them at least as much as their MMF or green card or whatever spam had annoyed me. unfortunately that stopped being a viable tactic fairly quickly, and it certainly wouldn't scale to anything like the spam load of today (back then 1 or 2 spams every few days was a lot. now i wouldn't even notice it). craig ps: anyone know if MMF spams still happen? i haven't seen one for years. could be my body checks rules block them all, or maybe they've just given up since 419 scams are more lucrative. -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: EHLO/HELO [was blacklists]
On Friday 10 December 2004 00:39, Mark Bucciarelli [EMAIL PROTECTED] wrote: I've recently turned on EHLO/HELO validation and am encouraged by how effective it is. WIth RBL's (spamcop and dnsbl) and SpamAssassin 3, only 88% of spam was stopped. So far, it's 100%. (This is a _very_ small What exactly do you mean by EHLO/HELO validation? In my postfix configuration I have: smtpd_helo_restrictions = permit_mynetworks, reject_invalid_hostname, reject_non_fqdn_hostname, reject_unknown_sender_domain I tried out reject_unknown_hostname but had to turn it off, too many machines had unknown hostnames. For example a zone foo.com has a SMTP server named postfix1 and puts postfix1.foo.com in the EHLO command but has an external DNS entry of smtp.foo.com. Such a zone is moderately well configured and there are too many such zones to block them all. The other helo restrictions get enough non-spam traffic. Using reject_unknown_hostname would get close to blocking 100% of spam, but that's because it would block huge amounts of non-spam email. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: EHLO/HELO [was blacklists]
On Fri, Dec 10, 2004 at 11:08:53PM +1100, Russell Coker wrote: I tried out reject_unknown_hostname but had to turn it off, too many machines had unknown hostnames. For example a zone foo.com has a SMTP server named postfix1 and puts postfix1.foo.com in the EHLO command but has an external DNS entry of smtp.foo.com. Such a zone is moderately well configured and there are too many such zones to block them all. The other helo restrictions get enough non-spam traffic. actually, it's not moderately well configured. it's trivial to add a DNS entry for postfix1.foo.com (preferably an A record and not a CNAME - doesn't matter for HELO/EHLO but it does matter for $myorigin). it's even more trivial to make the postfix server announce itself with a real hostname, one that actually exists in the DNS - smtp.foo.com would be perfect. that's all it takes to get past reject_unknown_hostname. it's unusual to see this level of cluelessness with someone running a unix MTA - i thought it was reserved to Exchange and Groupwise users. Using reject_unknown_hostname would get close to blocking 100% of spam, nowhere near that much. it helps a little, but it's not even remotely close to the final solution to spam. but that's because it would block huge amounts of non-spam email. that's not the case in my experience (but that depends on exactly what kind of mail traffic is received). but it's your server, you get to choose what rules are on it. craig ps: yes, this is another rule i use at home but not at work. there are lots of windows MTAs out there run by the clueless. fortunately, at home i don't need or have to communicate with them, but at work there are many people who might. -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: EHLO/HELO [was blacklists]
[CC'ing Bill Taroli who has been helping me with this on courier-user] On Friday 10 December 2004 07:08, Russell Coker wrote: On Friday 10 December 2004 00:39, Mark Bucciarelli [EMAIL PROTECTED] wrote: I've recently turned on EHLO/HELO validation and am encouraged by how effective it is. WIth RBL's (spamcop and dnsbl) and SpamAssassin 3, only 88% of spam was stopped. So far, it's 100%. (This is a _very_ small What exactly do you mean by EHLO/HELO validation? The courier man page just says verify the hostname provided in the ESTMP EHLO/HELO statement. From reading the code, here's what it does: (0) If connecting IP addresses is in the checkhelo whitelist -- PASS (1) If SPF HELO checking is on and lookup matches connecting IP -- PASS (2) If HELO host name is a numeric IP and it matches connecting IP -- PASS (3) Lookup MX records for HELO hostname. If one matches connecting IP -- PASS (4) Lookup A records for hostname. If one matches connecting IP -- PASS Otherwise, return 517 HELO $hostname does not match $remote-ip If there is an RFC1035_MX_HARDERR or RFC1035_MX_BADDNS when looking up the MX record, return a 517. If the MX or A DNS lookup fails, return a 417. In my postfix configuration I have: smtpd_helo_restrictions = permit_mynetworks, reject_non_fqdn_hostname, reject_unknown_sender_domain I tried out reject_unknown_hostname but had to turn it off, too many machines had unknown hostnames. I find it interesting that postfix defaults the response code to 450 instead of a 5XX for this failure. This is along the lines that I have been thinking. For example a zone foo.com has a SMTP server named postfix1 and puts postfix1.foo.com in the EHLO command but has an external DNS entry of smtp.foo.com. Such a zone is moderately well configured and there are too many such zones to block them all. The other helo restrictions get enough non-spam traffic. Using reject_unknown_hostname would get close to blocking 100% of spam, but that's because it would block huge amounts of non-spam email. So I guess the questions are: (1) Given a log entry (hostname and connecting IP) of an EHLO reject, can I reliably figure out if the host was valid? (2) Can I do this quickly enough that my whitelist will be updated before their MTA stops retrying and customers start complaining? (3) Will the whitelist stabilize enough over time to make this worth it. (4) Would it be possible to build a secure data pool where a group of like-minded and trusted admins could share whitelisted connecting IP's. Regards, Mark
Re: blacklists
--On Friday, December 10, 2004 22:48 +1100 Craig Sanders [EMAIL PROTECTED] wrote: On Thu, Dec 09, 2004 at 11:18:16PM -0700, Michael Loftis wrote: --On Friday, December 10, 2004 16:43 +1100 Craig Sanders [EMAIL PROTECTED] wrote: DoS is a huge exaggeration. a few smtpd processes waiting to timeout does not constitute a DoS. neither does a few dozen. I had about 800 waiting around in just a few minutes on the one server I began testing it on, but this is a large installation. And this isn't peak time...It's holding at around 1000 blocked hosts, most of them for blacklist infractions. i certainly wouldn't recommend running it on a large installation. i'm surprised you even tried. Well, we're very anti-spam, and willing ot try anything to help...I had to disable it after we got around ~8K rules in the tables on that box, that ended up causing the system CPU time to go through the roof. Though it was very effective. :) i run it on my home system at the moment. i wouldn't run it at work. I've made a few modifications already, including making everything persistent and making it purge SEEN entries after not seeing a host for 24hrs (this also effectively caps any block time to being 24hrs). I might just set it so that it only watches our MailScanner and blocks the IPs it reports as sending virii. That would probably help to shrink the number of reports a lot, and help with my virus load. That'd be a good site-wide table to share (we use central mysql maps a lot). i experiment with lots of things on my home system that i wouldn't even think of doing at work. some of them, very few, actually turn out to be worthwhile and safe enough to use at work. Same here. try dropping only SYN smtp packets if you still want to experiment with it, adding --syn to the end of the iptables args in the scripts. that should stop the hanging processes. Yeah. Last night when I wrote back my brain was a bit mushy, couldn't think of the right option so I just said it should probably be changed :) unfortunately, my domain seems to attract a lot of junk. i've had my domain for over 10 years, and kept the same email address all along. and i've been joe-jobbed many times over the last decade by spammers who don't like me (or my anti-spam methods, or the fact that i share them openly), and i've had thousands of bogus, non-existant addresses in my domain added to spam lists also by spammers who don't like me. the current crop of spammers probably don't even notice or care, but in the early days of spam it was different. spammers got very offended and took it personally...which, of course, was excellent incentive to keep on blocking them :) I'm glad you share them. Spammers are criminals, pure and simple, they're stealing our time, resources, and our users time and resources and money. They have no place in the world. Heck, I'm all for capitol(or is it al? I can't remember) punishment for spammers. (see short disclaimer.h at the bottom of this message) i pissed off quite a few in the very early days, when spammers didn't hide their identities and hadn't yet learned not to use their own address. one of the things i wrote was a script which i could bounce spam to. it would then parse the sender addresses and add it to a database of spammersand sent copies of each spam to a random subset of the database. that infuriated them and amused me no end. my intention was to annoy them at least as much as their MMF or green card or whatever spam had annoyed me. unfortunately that stopped being a viable tactic fairly quickly, and it certainly wouldn't scale to anything like the spam load of today (back then 1 or 2 spams every few days was a lot. now i wouldn't even notice it). Now that's a heck of a tactic LOL :) too bad I didn't' think of it back then...although I had a firewall setup for the longest time at home that automatically did counter-recon of offenders, and if it determined common open holes would get in and attempt shut the system off. It was always satisfying to watch a zombie get halfway through a portscan, then just disappear. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Fri, Dec 10, 2004 at 11:20:28AM -0700, Michael Loftis wrote: i certainly wouldn't recommend running it on a large installation. i'm surprised you even tried. Well, we're very anti-spam, and willing ot try anything to help...I had to disable it after we got around ~8K rules in the tables on that box, that ended up causing the system CPU time to go through the roof. Though it was very effective. :) i guess what it needs is something that can merge addresses into a CIDR range. e.g. if it sees x.y.z.1 through to .8, it should merge them all into a /29. delete the individual /32 rules and replace them with the /29. managing that would get fairly complicated, especially when it has to later merge a few /29s and /32s into larger blocks. but it should be doable. if i get time, i'll think about how that might be implemented. the simpler thing is to just uncomment the /24 stuff that's already in there, and block the entire /24 rather than just the offending host. heavy-handed but it should reduce the number of entries in iptables. hmmm. one useful optimisation here would be to use Net::DNS to find out whether the IP is in a DUL - and if it is, then block the entire /24. i reject all mail direct from dynamic/dialups anyway so it wouldn't hurt anything. (and remember to add your own dialup pools to @whitelist :) another thing it should do is add the rules to a SPAMMERS table, rather than to the main INPUT table. that would be a trivial change, only a few minutes work. ...done. new version now published: # v1.8 - changed to use SPAMMERS table rather than INPUT. # use --syn when adding iptables rules, to avoid hanging smtpd processes # # the SPAMMERS table should be set up like this (BEFORE this script is run): # # # create SPAMMERS table # iptables -F SPAMMERS # iptables -X SPAMMERS # iptables -N SPAMMERS # # # send all INPUT FORWARD packets to the SPAMMERS table # iptables -I INPUT -j SPAMMERS # iptables -I FORWARD -j SPAMMERS # # FORWARD rule needed only on gateway/router boxes, not normal hosts. # # you could optionally create a SPAMDROP table too, which logged the packet # with a SPAMMERS prefix before dropping itbut that kind of defeats the # purpose of this script which is to remove spammer noise from the logs. I've made a few modifications already, including making everything persistent and making it purge SEEN entries after not seeing a host for 24hrs (this also effectively caps any block time to being 24hrs). I might just set it so that it only watches our MailScanner and blocks the IPs it reports as sending virii. That would probably help to shrink the number of reports a lot, and help with my virus load. That'd be a good site-wide table to share (we use central mysql maps a lot). diff -u ??? one of the things i wrote was a script which i could bounce spam to. it would then parse the sender addresses and add it to a database of spammersand sent copies of each spam to a random subset of the database. that infuriated them and amused me no end. my intention Now that's a heck of a tactic LOL :) oh yes, i forgot the most amusing thing about it. it not only sent it to a subset of the spammer database, it also used random addresses out of that db as the envelope and header sender addresses, so that they'd complain at each other. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Friday, December 10, 2004 17:01 -0700 Michael Loftis [EMAIL PROTECTED] wrote: --On Saturday, December 11, 2004 10:50 +1100 Craig Sanders [EMAIL PROTECTED] wrote: diff -u ??? I'll attach privately the diff's from your version ( CVS) Actually as it turns out I can't send mail directly to you, something about my @modwest.com address is apprently offensive to your mailserver... 550 Sender access denied for [EMAIL PROTECTED] -- I sent a second one from my home address but it won't get delivered or bounced for a while since I'm not likely being blocked at your firewall ;) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Saturday, December 11, 2004 10:50 +1100 Craig Sanders [EMAIL PROTECTED] wrote: diff -u ??? I'll attach privately the diff's from your version ( CVS) Now that's a heck of a tactic LOL :) oh yes, i forgot the most amusing thing about it. it not only sent it to a subset of the spammer database, it also used random addresses out of that db as the envelope and header sender addresses, so that they'd complain at each other. So it's your fault they figured out the forged MAIL FROM trick! Bad craig, no donut! ;) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: EHLO/HELO [was blacklists]
On Friday 10 December 2004 09:36, Mark Bucciarelli wrote: (1) If SPF HELO checking is on and lookup matches connecting IP -- PASS [..] Otherwise, return 517 HELO $hostname does not match $remote-ip Sorry to reply to myself, but this sequence is more complicated if SPF checking is turned on and the SPF lookup fails. You can choose to softfail (417) or hardfail (517). I wanted to set the record straight. Regards, Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
Am 2004-12-09 19:04:49, schrieb Richard Zuidhof: Michelle Konzack wrote: cbl.abuseat.org is included in xbl.spamhaus.org so it is not needed to use both. If you use sbl.spamhaus.org I do not see why not to use sbl-xbl.spamhaus.org had never FP bl.spamcop.net as well. Both have some collateral damage and false I had used spamcop.net but gotten to much FP. Richard Greetings Michelle -- Linux-User #280138 with the Linux Counter, http://counter.li.org/ Michelle Konzack Apt. 917 ICQ #328449886 50, rue de Soultz MSM LinuxMichi 0033/3/8845235667100 Strasbourg/France IRC #Debian (irc.icq.com) signature.pgp Description: Digital signature
Re: blacklists
On Fri, Dec 10, 2004 at 05:01:33PM -0700, Michael Loftis wrote: So it's your fault they figured out the forged MAIL FROM trick! Bad craig, no donut! ;) no, many of them already knew that. it was obvious anyway. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thursday 09 December 2004 00:42, Michael Loftis wrote: See my just prior response on this thread, most people don't understand bounces. Yes it could be argued a bounce needs to be reformatted so the humans can read it easier, but it's not my system generating the bounce message, I'm just telling the attempter. Whether it's permanent or temporary doesn't matter. We reject a huge amount of mail with RBL's. A very small percentage of it is false positive. A very small percentage of those false positives end up as complaints with our help desk. However a significant portion of those who complain are business people, company directors, IT support staff etc, and if they are complaining to us they are either one of our customers or they are transacting with one of our customers. (This does not mean they understand the bounce message). By treating these complaints with a little care we have managed to swing (or keep) a number of large business clients, or at least establish a relationship with them which will ensure they remain our clients for a long while. We have a simple policy for handling RBL related complaints. - If the rejecting list was Spamcop, we whitelist the IP and e-mail the sender to say their mail will be accepted. Spamcop is not an exact science, whitelisting a few bona-fida mail servers is quicker than explaining our motivation for using it in the first place. - For any other list we send a standard messages to the sender and copy to the original intended recipient explaining that their mail is being sent from an open relay. The message has some technical info and links and a suggestion to forward the message to a technically competent administrator. (This involves copy and pasting two e-mail addresses and the IP of listed server into a pre-formated e-mail.) Answer: find their spammers, and squash them. This puts the support load on the offender, not on me. Which is where it should be. If they user sender gets frustrated enough and finally calls us, emails us via alternate channels (maybe via the recipient) or whatever, People who conduct business with e-mail notice very quickly when their mail disappears into a black hole. They get very frustrated very quickly and are not shy to change ISP's, even if they never understand the problem nor have any idea who is at fault. We have found that a little informed feedback in this situation is very cost effective marketing. But back to the subject. I prefer a 5xx permanent error or that results in an immediate bounce message. Regards Ian -- Ian Forbes ZSD http://www.zsd.co.za Office: +27 21 683-1388 Fax: +27 21 674-1106 Snail Mail: P. O. Box 46827, Glosderry, 7702, South Africa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
EHLO/HELO [was blacklists]
On Tuesday 07 December 2004 17:55, Michael Loftis wrote: --On Wednesday, December 08, 2004 08:47 +1100 Craig Sanders [EMAIL PROTECTED] wrote: Now I reject by 554 code... should I change to 4xx? if it suits your needs. i wouldn't. I have to agree with that statement. For us it suits our needs very well. I don't mind handling the extra retry traffic if it means legitimate mail on a 'grey/pink' host is just temporarily rejected or delayed while they clean up, in fact this is far more desireable for us. Complaints of 'lost' mail went up when we were using permanent fatal codes as an experiment. Yes legitimate hosts get blacklisted, but legitimate hosts will retry, and if they don't well, it's their problem, not ours. We're telling them 454 listed on spamciop see URL of whatever (I'm obviously paraphrasing) I've been following this thread with great interest. I'm wondering if the same 4XX technique could apply to EHLO/HELO checks--with automatic whitelisting thrown in. If spammers never retry, couldn't you watch the logs and when you see a retry, add that IP to EHLO/HELO whitelist? (And generate a report so you can check up on this later.) Folks on the courier-user list have reported that the EHLO/HELO whitelist becomes quite stable after a while. I've recently turned on EHLO/HELO validation and am encouraged by how effective it is. WIth RBL's (spamcop and dnsbl) and SpamAssassin 3, only 88% of spam was stopped. So far, it's 100%. (This is a _very_ small sample--one email account for one day, but the change is dramatic from my perspective.) And what's to stop spammers from starting to retry? Does it double their cost of doing business? If I then require a second retry, does it triple their cost? If I want to hack the courier backport package to force an invalid EHLO to get a 4XX instead of the hardcoded 517, are these the correct steps (taken from Debian Quick Reference, Ch. 3): apt-get source courier dpkg-source courier.dsc cd courier-0.47 ... edit source dpkg-buildpackage -rfakeroot -us -uc su -c dpkg -i courier-mta.deb Is that correct? How do I change the newly-built package name, and what do I change it to so apt-get update/upgrade will find a new release uploaded to backports.org? Regards, Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
Michelle Konzack wrote: I have tested in sequenz: sbl-xbl.spamhaus.org cbl.abuseat.org relays.ordb.org opm.blitzed.org list.dsbl.org dul.dnsbl.sorbs.org blackholes.mail-abuse.org dialups.mail-abuse.org But now I have removed all exept the first two. spamhaus catchs more then 50% of my spams and abuseat around 10%. The rest is done by Blacklists and spamassassin. cbl.abuseat.org is included in xbl.spamhaus.org so it is not needed to use both. If you use sbl.spamhaus.org I do not see why not to use bl.spamcop.net as well. Both have some collateral damage and false positives but still prove to be very useful. I can also advise to use list.dsbl.org. To see some statistics on the hit rate of various blacklists see: http://cgi.monitor.nl/popstats.html http://www.sdsc.edu/~jeff/spam/cbc.html I you use Spamassassin be sure to use SURBL. That is a very effective blacklist with virtually no false positives. Richard -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 07:04:49PM +0100, Richard Zuidhof wrote: To see some statistics on the hit rate of various blacklists see: http://cgi.monitor.nl/popstats.html http://www.sdsc.edu/~jeff/spam/cbc.html or if you run postfix and want to compare RBLs against the client IPs in your mail.log, then download http://taz.net.au/postfix/scripts/compare-rbls.pl notes: 1. you will also need to download openlogfile.pl from the same place and put it in the same directory as compare-rbls.pl 2. i wrote it several years ago, so it has several very old and now defunct RBLs listed in it. change the @dnsrbls array to list only the ones you want to check. for example, change this: my @dnsrbls = qw(blackholes.mail-abuse.org relays.mail-abuse.org dialups.mail-abuse.org relays.osirusoft.com inputs.orbz.org outputs.orbz.org or.orbl.org relays.ordb.org); to this: my @dnsrbls = qw(cn-kr.blackholes.us taiwan.blackholes.us brazil.blackholes.us hongkong.blackholes.us list.dsbl.org sbl-xbl.spamhaus.org dul.dnsbl.sorbs.net dnsbl.sorbs.net Dynablock.njabl.org relays.ordb.org ); BTW, except for dnsbl.sorbs.net (which i don't use because i don't like their de-listing policy - but i do use their DUL list), these are the RBLs i am currently using. 3. i just updated the script to use the @dnsrbls array as shown abovebut it's still useful to know how to configure it. 4. it is very slow. it has do to one DNS lookup per RBL per IP address seen. this is fairly slow, anyway, and it uses Net::DNS, which is not noted for it's speed. if you want to trial it on a small subset, do something like this: tail -1000 /var/log/mail.log /tmp/small.log compare-rbls.pl /tmp/small.log | less craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Thursday, December 09, 2004 12:22 +1100 Craig Sanders [EMAIL PROTECTED] wrote: On Thu, Dec 09, 2004 at 11:27:27AM +1100, Russell Coker wrote: On Thursday 09 December 2004 01:12, Craig Sanders [EMAIL PROTECTED] wrote: the log file noise issue is important to me - i've recently started monitoring mail.log and adding iptables rules to block smtp connections from client IPs that commit various spammish-looking crimes against my system. Interesting. Do you plan to package it for Debian? nope, it's just a trivial script - and one that's probably dangerous to use if you don't understand what it's doing, and i don't plan on documenting it beyond comments in the script itself. in short, it's a toy for me. if you want to see it, look in http://taz.net.au/postfix/scripts/ it's called watch-maillog.pl there's a bunch of other postfix related scripts in there. One little note about that script, the DROP needs to be changed since basically you're DoSing yourself by hanging a bunch of connections because you suddenly start dropping their inbound packets while still 'in-flight' as it were. postfix's default timeouts are about 300s, so you'll want to turn those down (300s seems too generous to me for most of them anyway) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 10:22:24PM -0700, Michael Loftis wrote: if you want to see it, look in http://taz.net.au/postfix/scripts/ it's called watch-maillog.pl One little note about that script, the DROP needs to be changed since basically you're DoSing yourself by hanging a bunch of connections DoS is a huge exaggeration. a few smtpd processes waiting to timeout does not constitute a DoS. neither does a few dozen. because you suddenly start dropping their inbound packets while still 'in-flight' as it were. postfix's default timeouts are about 300s, so you'll want to turn those down (300s seems too generous to me for most of them anyway) aside from the DoS exaggeration, that is true, but i don't careor more accurately, i care more about spammer noise in my logs and the bandwidth that spammers waste. i have more than enough smtpd processes, ram, and cpu power available to cope with a few (or even several dozen) smtpds waiting to time out. i can also cope with the eventual dropped connection messages in the logs - instead of vaguely annoying me like the spam rejects do, they give me a feeling of satisfaction that i have in some small way slowed down the spamware by silently dropping their packets. the first workable fix i can think of is to DROP only smtp packets with SYN set, rather than all smtp packets. alternatively, i could extract the PID of the smtpd process and send it a HUP at the same time as i created the iptables rule. if it ever bothered me, i'd do one or the otherbut, as i said, it's not something i care much about. craig ps: watch-maillog.pl is a toy that i wrote for my own amusement. if you like it, run it or adapt it for your own needs. if you don't, then ignore it. i don't claim that it's good software or even that it's useful. i wrote it more as a proof of concept than anything else. pps: it also monitors TLS connection failures and adds them to /etc/postfix/tls_per_site (which doesn't seem to be really necessary now, but they were quite common a few years ago, mainly due to a particularly broken version of communigate) and it does basic pop-before-smtp (dovecot only because that's what i run). these two features are actually useful :) -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Friday, December 10, 2004 16:43 +1100 Craig Sanders [EMAIL PROTECTED] wrote: DoS is a huge exaggeration. a few smtpd processes waiting to timeout does not constitute a DoS. neither does a few dozen. I had about 800 waiting around in just a few minutes on the one server I began testing it on, but this is a large installation. And this isn't peak time...It's holding at around 1000 blocked hosts, most of them for blacklist infractions. because you suddenly start dropping their inbound packets while still 'in-flight' as it were. postfix's default timeouts are about 300s, so you'll want to turn those down (300s seems too generous to me for most of them anyway) aside from the DoS exaggeration, that is true, but i don't careor more accurately, i care more about spammer noise in my logs and the bandwidth that spammers waste. i have more than enough smtpd processes, ram, and cpu power available to cope with a few (or even several dozen) smtpds waiting to time out. I suppose my advisory is more for larger sites, on my home system I'd have never noticed it nor even thought about it. But when you've got a lot of mail (and a number of customer domains that just tend to attract junk) it's easy to get a lot of processes hanging around. i can also cope with the eventual dropped connection messages in the logs - instead of vaguely annoying me like the spam rejects do, they give me a feeling of satisfaction that i have in some small way slowed down the spamware by silently dropping their packets. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wednesday 08 December 2004 09:55, Michael Loftis [EMAIL PROTECTED] wrote: I have to agree with that statement. For us it suits our needs very well. I don't mind handling the extra retry traffic if it means legitimate mail on a 'grey/pink' host is just temporarily rejected or delayed while they clean up, in fact this is far more desireable for us. Complaints of 'lost' mail went up when we were using permanent fatal codes as an experiment. Yes legitimate hosts get blacklisted, but legitimate hosts will retry, and if they don't well, it's their problem, not ours. We're telling them 454 listed on spamciop see URL of whatever (I'm obviously paraphrasing) How would I configure Postfix to do this? Craig, why do you think it's undesirable to do so? -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wed, Dec 08, 2004 at 07:51:13PM +1100, Russell Coker wrote: On Wednesday 08 December 2004 09:55, Michael Loftis [EMAIL PROTECTED] wrote: I have to agree with that statement. For us it suits our needs very well. I don't mind handling the extra retry traffic if it means legitimate mail on a 'grey/pink' host is just temporarily rejected or delayed while they clean up, in fact this is far more desireable for us. How would I configure Postfix to do this? probably maps_rbl_reject_code = 450 Craig, why do you think it's undesirable to do so? because i dont want the extra retry traffic. i want spammers to take FOAD as an answer, and i dont want to welcome them with a pleasant please try again later message. i think it is a sin to be polite or pleasant to a spammer :) even on my little home system, at the end of an adsl line, i reject nearly 10,000 spams per day (and climbing all the time). i would expect that to at least double or triple if i 4xx-ed them rather than 5xx, depending on how much came from open relays or spamhaus rather than dynamic/DUL. i can see why some might want to do this, but not me. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
Michael Loftis wrote: --On Monday, December 06, 2004 09:34 +0100 Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] wrote: Various AOL mailservers, the Debian mailservers, and other servers sending out lots of regular mail get listed in spamcop regularly, so my recommendation (and that of spamcop.net themselves, btw) is not to use bl.spamcop.net for blacklisting. Use it in spamassassin to score points. Odd, since we don't see this. And when it does happen to 'big' mail senders it's never AOL for one (they're on the whitelist). And it's totally automatic so if they do end up on it's usually for less than a day. And how to deal with legitimate email sent via webmail (eg. yahoo) where the IP of the sender is inside a RBL, typicall for dsl or dialup ip-classes? This is part of the headers from a mail i received: Mon, 29 Nov 2004 19:12:38 +0100 (CET) Received: from web60309.mail.yahoo.com (web60309.mail.yahoo.com [216.109.118.120]) by -snip- (Postfix) with SMTP id 474D8249E74 for snip; Mon, 29 Nov 2004 19:12:38 +0100 (CET) Received: (qmail 47653 invoked by uid 60001); 29 Nov 2004 18:12:36 - Message-ID: [EMAIL PROTECTED] Received: from [217.226.195.183] by web60309.mail.yahoo.com via HTTP; Mon, 29 Nov 2004 19:12:36 CET Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Status: No, hits=4.2 tagged_above=0.0 required=4.5 tests=BAYES_50, RCVD_IN_DSBL, RCVD_IN_NJABL, RCVD_IN_SORBS, RCVD_IN_SORBS_HTTP, RCVD_IN_SORBS_SOCKS if i would have used rbl checks in postfix instead of spamassim i would never receive that mail, right? the tracked ip is of course 217.226.195.186 and not the yahoo ip 216.109.118.120. Or i didn't understand? :( -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Wednesday, December 08, 2004 10:32 +0100 daniele becchi [EMAIL PROTECTED] wrote: if i would have used rbl checks in postfix instead of spamassim i would never receive that mail, right? the tracked ip is of course 217.226.195.186 and not the yahoo ip 216.109.118.120. Or i didn't understand? :( First RBL checks inside of postfix *ONLY* operate ont he host connecting to you, NOT on anything else. Second, I recommend using a 4xx (temprorary failure) code too, third, I'm implying that you use the up-front to augment SpamAssassin, not replace it. It GREATLY reduces the amount of load on mailservers handilg large quantities of email to use reliable, well maintained, and well run lists aligning closely to your sites policies for SPAM as early as possible to save you the CPU, Disk and associated costs of carrying the crap. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wednesday 08 December 2004 20:32, daniele becchi [EMAIL PROTECTED] wrote: Odd, since we don't see this. And when it does happen to 'big' mail senders it's never AOL for one (they're on the whitelist). And it's totally automatic so if they do end up on it's usually for less than a day. And how to deal with legitimate email sent via webmail (eg. yahoo) where the IP of the sender is inside a RBL, typicall for dsl or dialup ip-classes? This is part of the headers from a mail i received: Yahoo server IP address space should not be in a dialup class. If that happens then notify the person maintaining the dialup-list that you use that they have an inaccuracy. Mon, 29 Nov 2004 19:12:38 +0100 (CET) Received: from web60309.mail.yahoo.com (web60309.mail.yahoo.com [216.109.118.120]) by -snip- (Postfix) with SMTP id 474D8249E74 for snip; Mon, 29 Nov 2004 19:12:38 +0100 (CET) Received: (qmail 47653 invoked by uid 60001); 29 Nov 2004 18:12:36 - Message-ID: [EMAIL PROTECTED] Received: from [217.226.195.183] by web60309.mail.yahoo.com via HTTP; Mon, 29 Nov 2004 19:12:36 CET Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Status: No, hits=4.2 tagged_above=0.0 required=4.5 tests=BAYES_50, RCVD_IN_DSBL, RCVD_IN_NJABL, RCVD_IN_SORBS, RCVD_IN_SORBS_HTTP, RCVD_IN_SORBS_SOCKS if i would have used rbl checks in postfix instead of spamassim i would never receive that mail, right? the tracked ip is of course 217.226.195.186 and not the yahoo ip 216.109.118.120. Or i didn't understand? :( Most people use DNSBLs for the IP address that's the source of the port 25 connection and don't use them on the addresses in the Received headers. Such use won't have a problem with this. Yahoo don't seem to police their TOS well so they tend to get on black-lists (among other things they don't even have a functional abuse address). So if you want email from yahoo you probably have to white-list them anyway. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wednesday 08 December 2004 20:16, Craig Sanders [EMAIL PROTECTED] wrote: Craig, why do you think it's undesirable to do so? because i dont want the extra retry traffic. i want spammers to take FOAD as an answer, and i dont want to welcome them with a pleasant please try again later message. i think it is a sin to be polite or pleasant to a spammer :) I agree that we don't want to be nice to spammers. But there is also the issue of being nice in the case of false-positives. The extra traffic shouldn't be that great (the message body and headers are not being transmitted). When a legit user accidentally gets into a black-list their request to get the black-list adjusted can often be processed within the time that their mail server is re-trying the message. even on my little home system, at the end of an adsl line, i reject nearly 10,000 spams per day (and climbing all the time). i would expect that to at least double or triple if i 4xx-ed them rather than 5xx, depending on how much came from open relays or spamhaus rather than dynamic/DUL. 30,000 rejections per day is only one every three seconds. Not a huge load. I am not trying to convince you to change your system (I'm not entirely convinced to change mine at this time). -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 12:00:42AM +1100, Russell Coker wrote: On Wednesday 08 December 2004 20:16, Craig Sanders [EMAIL PROTECTED] wrote: Craig, why do you think it's undesirable to do so? because i dont want the extra retry traffic. i want spammers to take FOAD as an answer, and i dont want to welcome them with a pleasant please try again later message. i think it is a sin to be polite or pleasant to a spammer :) I agree that we don't want to be nice to spammers. But there is also the issue of being nice in the case of false-positives. if it's a false positive, the sender will get a bounce from their MTA and they can fix the problem or route around it. IMO, that's far nicer to legit senders than them not knowing that their mail isn't being delivered because it's stuck in their MTA's queue rather than bouncing back to them - the former means it's probably 5 days before they know there is a problem, while the latter gives them instant feedback. The extra traffic shouldn't be that great (the message body and headers are not being transmitted). it's still MY bandwidth being used by spamming vermin, even if it's not much (i begrudge those bastards even a single bit) and it still generates huge amounts of noise in my mail.log files. the log file noise issue is important to me - i've recently started monitoring mail.log and adding iptables rules to block smtp connections from client IPs that commit various spammish-looking crimes against my system. some crimes get blocked for 60 seconds, some for 10 minutes, some for an hour. each time the same IP address is seen committing a crime, the time is doubled. i am doing this not because i'm worried that spammers will get their junk through my anti-spam rules but because a) i don't want their noise in my mail.log, and b) it was an interesting programming project that amused me for a few days of part time perl hacking. When a legit user accidentally gets into a black-list their request to get the black-list adjusted can often be processed within the time that their mail server is re-trying the message. similarly, they can resend the message themselves when they know the problem has been fixed, WITHOUT flooding my logs with crap i don't want to see AND they'll have had immediate feedback about the problem with their mail system. everyone wins. if it's important, they'll resend it. if the sender doesn't think it's important enough to bother resending, then why should i care? even on my little home system, at the end of an adsl line, i reject nearly 10,000 spams per day (and climbing all the time). i would expect that to at least double or triple if i 4xx-ed them rather than 5xx, depending on how much came from open relays or spamhaus rather than dynamic/DUL. 30,000 rejections per day is only one every three seconds. Not a huge load. the important factor here is that it's one every 3 seconds that I DON'T WANT. i don't want the ~10,000 per day that i currently get and i see no reason to take any action that will increase that number. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wednesday 08 December 2004 15:00, Russell Coker wrote: I agree that we don't want to be nice to spammers. But there is also the issue of being nice in the case of false-positives. I think, that a permanent error is the best response for a false-positive. The sender will then receive a bounce message and he will know that his communication has not been received by the other party. He can then make another plan - like sending a fax or changing ISP. If you give a temporary error, it will depend on the rules in the mail server one-hop before the one generating the false positive on when the sender will get a bounce or a warning - this could take hours or even days. The server where the mail gets queued is likely to be an outgoing relay server for a large ISP. The sender would have no access to such a server and little opportunity to talk to the persons responsible for managing it so he would have a tricky job establishing what happened to the mail. Support staff's nightmare: I sent an e-mail but it just never arrived ...(long story about how important that particular message was)... Did you get a bounce or an error message? No, the mail just vanished - what are you going to do to fix your servers! Regards Ian -- Ian Forbes ZSD http://www.zsd.co.za Office: +27 21 683-1388 Fax: +27 21 674-1106 Snail Mail: P. O. Box 46827, Glosderry, 7702, South Africa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wed, 2004-12-08 at 15:30 +0100, Michelle Konzack wrote: sbl-xbl.spamhaus.org opm.blitzed.org But now I have removed all exept the first two. spamhaus catchs more then 50% of my spams and abuseat around 10%. The rest is done by Blacklists and spamassassin. Spamhaus XBL incorporates e.g. Blitzed OPM and others. That's why the composite blocking list catches so much. But with the size the possiblity of false positives increases. Regards, Philipp Kern -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wed, 2004-12-08 at 23:56 +1100, Russell Coker wrote: Yahoo server IP address space should not be in a dialup class. If that happens then notify the person maintaining the dialup-list that you use that they have an inaccuracy. This is incorrect when you look at the headers. Received: from [217.226.195.183] by web60309.mail.yahoo.com via HTTP; Mon, 29 Nov 2004 19:12:36 CET Content-Type: text/plain; charset=iso-8859-1 SpamAssassin looks at all the headers. If this is a good choice or not is debatable. The MTA would only judge by the IP that connects to him which was in fact a Yahoo IP. This IP [217.226.195.183] is a valid dialup out of Germany which Yahoo adds as a trace from where the webmail request came. Regards, Philipp Kern -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Thursday, December 09, 2004 01:12 +1100 Craig Sanders [EMAIL PROTECTED] wrote: if it's a false positive, the sender will get a bounce from their MTA and they can fix the problem or route around it. IMO, that's far nicer to legit senders than them not knowing that their mail isn't being delivered because it's stuck in their MTA's queue rather than bouncing back to them - the former means it's probably 5 days before they know there is a problem, while the latter gives them instant feedback. You massively over estimate the intelligence of email users. Far far over-estimate it. When we were doing 5xx returns, about a dozen bounces a day were reported as SPAM to my abuse address. Further more, most people do not understand bounce messages, at all. For us it's far cheaper to let the mailservers deal with it, and delay and bounce later, than involve a human for all the people who just don't get it, but get unlisted before they even READ the bounce. When a legit user accidentally gets into a black-list their request to get the black-list adjusted can often be processed within the time that their mail server is re-trying the message. similarly, they can resend the message themselves when they know the problem has been fixed, WITHOUT flooding my logs with crap i don't want to see AND they'll have had immediate feedback about the problem with their mail system. everyone wins. If they ever get the bounce message, increasingly I'm seeing the owrrying trend that bounce messages from MTAs end up as SPAM or just /dev/null-ed. if it's important, they'll resend it. if the sender doesn't think it's important enough to bother resending, then why should i care? If the sender even understands that it wasn't sent at all. Many, if not most, don't. Your average @aol.com-er doesn't. For a busines it might be acceeptable, for an ISP doing 5xx is hard to justify, atleast in our case. Anyway there certainly ARE merits to both sides, and I can and do understand and see your point. I don't like the log chatter but that's easy to deal with compared to our L1 support time when we 5xx. People get stupidly angry sometimes too 'you have no right to bounce my mail!' that's like saying the post office has to deliver all bombs, but logic has nothing to do with most people :) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Wednesday, December 08, 2004 16:04 +0200 Ian Forbes [EMAIL PROTECTED] wrote: On Wednesday 08 December 2004 15:00, Russell Coker wrote: I agree that we don't want to be nice to spammers. But there is also the issue of being nice in the case of false-positives. I think, that a permanent error is the best response for a false-positive. The sender will then receive a bounce message and he will know that his communication has not been received by the other party. He can then make another plan - like sending a fax or changing ISP. See my just prior response on this thread, most people don't understand bounces. Yes it could be argued a bounce needs to be reformatted so the humans can read it easier, but it's not my system generating the bounce message, I'm just telling the attempter. Whether it's permanent or temporary doesn't matter. ... I sent an e-mail but it just never arrived ...(long story about how important that particular message was)... Did you get a bounce or an error message? No, the mail just vanished - what are you going to do to fix your servers! Answer: find their spammers, and squash them. This puts the support load on the offender, not on me. Which is where it should be. If they user sender gets frustrated enough and finally calls us, emails us via alternate channels (maybe via the recipient) or whatever, we'll gladly explain the problem, and helpfully suggest they contact their ISP, if that's not working we suggest they switch ISPs, if they're local we also take the time to offer our own ISP service ;) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wed, Dec 08, 2004 at 07:41:12PM +0100, Philipp Kern wrote: Received: from [217.226.195.183] by web60309.mail.yahoo.com via HTTP; Mon, 29 Nov 2004 19:12:36 CET Content-Type: text/plain; charset=iso-8859-1 SpamAssassin looks at all the headers. If this is a good choice or not is debatable. The MTA would only judge by the IP that connects to him which was in fact a Yahoo IP. i turn off DUL checking in SpamAssassin. i use DULs in postfix RBL checks, but it makes no sense to do DUL checking in SA on mail received from a real MTA - almost all mail will originate from a dialup/dynamic IP. in local.cf, that looks like this: # ignore DUL score RCVD_IN_DYNABLOCK 0.0 craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Wed, Dec 08, 2004 at 03:38:36PM -0700, Michael Loftis wrote: --On Thursday, December 09, 2004 01:12 +1100 Craig Sanders [EMAIL PROTECTED] wrote: if it's a false positive, the sender will get a bounce from their MTA and they can fix the problem or route around it. IMO, that's far nicer to legit senders than them not knowing that their mail isn't being delivered because it's stuck in their MTA's queue rather than bouncing back to them - the former means it's probably 5 days before they know there is a problem, while the latter gives them instant feedback. You massively over estimate the intelligence of email users. Far far over-estimate it. no, it's just that i don't give a damn about the dumb ones. if they're too dumb to figure it out, then they're too dumb to be using the net. i'd rather provide help useful information to the smart ones than spoonfeed dumb users. no matter how much you spoonfeed dumb users, they'll still cause you problems and cost you time money. best of all, providing useful information also gives a potentially educational experience to all, with a small but real chance of starting them on the path towards being a smart user. (in short, my unshakably firm belief about tech support has always been that education is far better than spoonfeeding. education works in the long run, spoonfeeding is just begging for an eternal problem) When we were doing 5xx returns, about a dozen bounces a day were reported as SPAM to my abuse address. never happened to me. if it ever did, i'd probably just ignore it. at most, i'd reply this is not spam, it's a bounce, and include a URL to a page with information about how mail works. i certainly wouldn't waste much time on it. Further more, most people do not understand bounce messages, at all. yes, i've seen that thousands of times. the common occurence is some idiot user forwarding you the bounce message that clearly states user unknown or similar and asking you why did this bounce? if i'm in a bad mood, i'll reply and say something like 'User unknown' means that the user is unknown. or if i'm feeling particularly helpful, i may expand on that and give them some suggestions on what to do or try. either way, in any reply i'll *always* point out that the answer to their question was right there in front of them - all they had to do was make the trivial effort to read it. If they ever get the bounce message, increasingly I'm seeing the owrrying trend that bounce messages from MTAs end up as SPAM or just /dev/null-ed. you're not responsible for the stupid things that other people do. Anyway there certainly ARE merits to both sides, and I can and do understand and see your point. I don't like the log chatter but that's easy to deal with compared to our L1 support time when we 5xx. People get stupidly angry sometimes too 'you have no right to bounce my mail!' that's like saying the post office has to deliver all bombs, but logic has nothing to do with most people :) yes, i've seen that too. the obvious response is that the post office can't deliver misaddressed mail either. if you send a snail-mail letter with the wrong address on it, the post office can't magically correct it. if you're lucky, you'll get it returned back to you. most likely, it will just sit on the hall-stand of the wrong address along with all the other misaddressed mail and mail for people who used to live there that they receive. just because they're angry doesn't make them right. if they won't listen to reason, then ignore them. you've done your best, that's all that can be asked of you. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thursday 09 December 2004 01:12, Craig Sanders [EMAIL PROTECTED] wrote: the log file noise issue is important to me - i've recently started monitoring mail.log and adding iptables rules to block smtp connections from client IPs that commit various spammish-looking crimes against my system. some crimes get blocked for 60 seconds, some for 10 minutes, some for an hour. each time the same IP address is seen committing a crime, the time is doubled. i am doing this not because i'm worried that spammers will get their junk through my anti-spam rules but because a) i don't want their noise in my mail.log, and b) it was an interesting programming project that amused me for a few days of part time perl hacking. Interesting. Do you plan to package it for Debian? -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 11:27:27AM +1100, Russell Coker wrote: On Thursday 09 December 2004 01:12, Craig Sanders [EMAIL PROTECTED] wrote: the log file noise issue is important to me - i've recently started monitoring mail.log and adding iptables rules to block smtp connections from client IPs that commit various spammish-looking crimes against my system. Interesting. Do you plan to package it for Debian? nope, it's just a trivial script - and one that's probably dangerous to use if you don't understand what it's doing, and i don't plan on documenting it beyond comments in the script itself. in short, it's a toy for me. if you want to see it, look in http://taz.net.au/postfix/scripts/ it's called watch-maillog.pl there's a bunch of other postfix related scripts in there. you may also like qvmenu.pl, a curses-based postfix queue browser that i wrote. it allows you to pipe queued messages into less or urlview, select multiple messages and delete, hold, unhold or re-queue them (i.e. a wrapper around postsuper). pretty simple to add new features...as with most of my scripts, i write for readability rather than optimised speed. i work on this occasionally, and add new stuff as i need ite.g. there's a half-implemented bounce to sa-spam/sa-ham alias featurethat's because i have a header_checks rule to HOLD all mail with an SA score over 10. this is the reason i wrote qvmenu.pl in the first place. false-positives i just unhold. spam, i usually view urlview them to get fodder for my SA body_checks rules, then delete them. i also want to be able to bounce them to sa-spam or sa-ham (sa-learn aliases on my system). i'll finish that off when i get time. because it calls mailq to get the queue listing, it's probably too slow to use on any system with thousands of messages in the queue. i've used it on systems with hundreds, and found it to be OK. actually, i doubt if it would be much faster even if it trawled the queue directories itself - mailq isn't exactly inefficient and bloated. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Thu, Dec 09, 2004 at 11:27:27AM +1100, Russell Coker wrote: On Thursday 09 December 2004 01:12, Craig Sanders [EMAIL PROTECTED] wrote: the log file noise issue is important to me - i've recently started monitoring mail.log and adding iptables rules to block smtp connections i also wrote another trivial script which fetches a named blackholes.us text file and creates iptables rules to match. not sure if this is a worthwhile experiment - if for no other reason than the fact that iptables doesn't seem to cope well with thousands of rules in a chain (could probably work around that with a chain per country...but i'm probably not going to bother since i'm pretty sure that this is NOT a good thing to do). i'm currently running with korea.blackholes.us completely filtered out as a test. (korea is where most of my spam attempts come from). so far it has blocked over 16000 packets from korea. since all *I* ever get from there is spam and probes from script-kiddies and viruses, that's a Good Thingtm. it probably wouldn't be a good thing anywhere other than on my home gateway. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Monday, December 06, 2004 09:34 +0100 Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] wrote: Various AOL mailservers, the Debian mailservers, and other servers sending out lots of regular mail get listed in spamcop regularly, so my recommendation (and that of spamcop.net themselves, btw) is not to use bl.spamcop.net for blacklisting. Use it in spamassassin to score points. Odd, since we don't see this. And when it does happen to 'big' mail senders it's never AOL for one (they're on the whitelist). And it's totally automatic so if they do end up on it's usually for less than a day. We don't (obviously) use a 5xx response because the failure isn't permanent. In any event we watch pretty carefully for problems, and aren't seeing any in the year (or more) that we've been using bl.spamcop.net. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Monday, December 06, 2004 09:34 +0100 Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] wrote: Various AOL mailservers, the Debian mailservers, and other servers sending out lots of regular mail get listed in spamcop regularly, so my recommendation (and that of spamcop.net themselves, btw) is not to use bl.spamcop.net for blacklisting. Use it in spamassassin to score points. Sorry about double replies, but also, you have to remember we're an ISP, not a small shop. By doing this we cut our incoming mail in *HALF* or less. That's because there is that much spam out there. Spamcop is fair, well run, and FULLY dynamic. Not subjet to human policies, or delay in listing, or unlisting. They are also not subject to arbitrary policy decisions that many of the other lists are subject to because they are automated. And because it is based on submitted spam, seen email, and honeypots or traps it is very accurate, far more so than many of the others, with far less collateral damage. When you're handling LOTS of email every day, we're in excess of about 3/4 a million envelopes a day, and far more recipients than that -- and that's just the main mail servers, doesn't include the Mailman server, you're going to do everything you can to not have to handle total junk, and to keep SA, and virus scanning, two VERY expensive processes, from hitting the email. We estimate we'd roughly have to double the amount of mail servers just in order to handle the SPAM that nobody wants. I'm sorry, but for us the tiny amount of collateral damage, is perfectly acceptable. Metrics have it at, and probably far below, 1%. If a false positive is encountered the 4xx code will cause the legitimate mailservers to just retry later, and usually the length of time that they're retry is more than adequate to get them unlisted IF the administrators are responsive to their SPAM problem. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
Hello Michael, You mean you reject spam only temporarily? By setting maps_rbl_reject_code in postfix to 4xx? What value exactly? My question is - does the spam software (or whatever is used for sending majority of spams) try to re-send it? How often and for how long? Now I reject by 554 code... should I change to 4xx? thanks Tuesday, December 7, 2004, 21:56:43, you wrote: ML I'm sorry, but for us the tiny amount of collateral damage, is ML perfectly acceptable. Metrics have it at, and probably far below, ML 1%. If a false positive is encountered the 4xx code will cause the ML legitimate mailservers to just retry later, and usually the length ML of time that they're retry is more than adequate to get them ML unlisted IF the administrators are responsive to their SPAM ML problem. -- bYE, Marki -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Tue, Dec 07, 2004 at 10:18:28PM +0100, Marek Podmaka wrote: My question is - does the spam software (or whatever is used for sending majority of spams) try to re-send it? most (if not all) spamware and viruses won't. open relays and spamhaus sites and other real MTAs will. How often impossible to say, depends on many factors on the sending host - how it is configured, what software it is running, how busy it is, how many items in the queue, etc etc.could be every few seconds, could be several hours between delivery attempts. many mail servers implement an exponential backoff strategy. they will try again very quickly to start with and, on each failure, increase the delay (e.g. doubling the time) between attempts until it reaches a maximum delay. and for how long? until the queue lifetime expires (usually 5 days, although it could be anything). Now I reject by 554 code... should I change to 4xx? if it suits your needs. i wouldn't. craig -- craig sanders [EMAIL PROTECTED] (part time cyborg) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Tuesday, December 07, 2004 22:18 +0100 Marek Podmaka [EMAIL PROTECTED] wrote: Hello Michael, You mean you reject spam only temporarily? By setting maps_rbl_reject_code in postfix to 4xx? What value exactly? We use either 450 or 454, don't remember the value exactly. And I'm not sure what the 'Correct' RFC value would be since there's no obvious choice. My question is - does the spam software (or whatever is used for sending majority of spams) try to re-send it? How often and for how long? Now I reject by 554 code... should I change to 4xx? Rarely, if ever, does the spam ratware retry, and even if it does it's FAR less resource intensive to just keep saying 4xx to RCPT commands rather than taking the mail, reading it, sending it through Virus Scanning and/or Spam Assassin/spam scanning. Far, far, far faster. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
--On Wednesday, December 08, 2004 08:47 +1100 Craig Sanders [EMAIL PROTECTED] wrote: Now I reject by 554 code... should I change to 4xx? if it suits your needs. i wouldn't. I have to agree with that statement. For us it suits our needs very well. I don't mind handling the extra retry traffic if it means legitimate mail on a 'grey/pink' host is just temporarily rejected or delayed while they clean up, in fact this is far more desireable for us. Complaints of 'lost' mail went up when we were using permanent fatal codes as an experiment. Yes legitimate hosts get blacklisted, but legitimate hosts will retry, and if they don't well, it's their problem, not ours. We're telling them 454 listed on spamciop see URL of whatever (I'm obviously paraphrasing) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Sunday 05 December 2004 11.31, Michael Loftis wrote: we use bl.spamcop.net up front at the SMTP level before taking any messages from a site. Hmm. Various AOL mailservers, the Debian mailservers, and other servers sending out lots of regular mail get listed in spamcop regularly, so my recommendation (and that of spamcop.net themselves, btw) is not to use bl.spamcop.net for blacklisting. Use it in spamassassin to score points. -- vbi -- featured product: vim - http://vim.org -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On Monday 06 December 2004 19:34, Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] wrote: Various AOL mailservers, the Debian mailservers, and other servers sending out lots of regular mail get listed in spamcop regularly, so my recommendation (and that of spamcop.net themselves, btw) is not to use bl.spamcop.net for blacklisting. Use it in spamassassin to score points. Received: from johnny.adanco.com (151.adsl.as8758.net [212.25.16.151]) by murphy.debian.org (Postfix) with ESMTP id B42442DED6 for [EMAIL PROTECTED]; Mon, 6 Dec 2004 02:34:01 -0600 (CST) Received: from humphrey.adanco.local (humphrey.adanco.local [172.18.10.16]) by johnny.adanco.com (Postfix) with ESMTP id 24E4B2C6D for [EMAIL PROTECTED]; Mon, 6 Dec 2004 09:34:01 +0100 (CET) The Debian servers correctly preserve the Received: path. This is used by Spamcop to assign blame to the correct server. Above are the original Received: headers from your message to the list, if your message was reported to Spamcop then it would send a complaint to [EMAIL PROTECTED] about IP address 212.25.16.151. If your message was reported to spamcop it would not list a Debian server, it would list 212.25.16.151. I doubt that Debian servers get listed regularly. I use the spamcop DNSBL and it doesn't get in the way of Debian mailing lists. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
Marek Podmaka wrote: which blacklists do you use to block spam emails on production boxes? I use relays.ordb.org and list.dsbl.org and now I have read about Spamhaus SBL and XBL on their website. What are your experiences with it? From what I understand XBL is a *private* list not for public use, I have heard that it cuts a rather wide path through uunet for example. Spamhaus is about as upstanding as they come. IIRC think they even provide corporate support if your needs are that significant. You might want to try plugging those addresses (spamhaus.org, relays.ordb.org and XBL into google groups news.admin.net-abuse.email (http://groups.google.com.au/groups?hl=enlr=client=firefox-aq=XBLbtnG=Searchmeta=group%3Dnews.admin.net-abuse.email) and see what comes out. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: blacklists
On So, 5.12.2004, 10:54, Marek Podmaka sagte: Hello debian-isp, which blacklists do you use to block spam emails on production boxes? I use relays.ordb.org and list.dsbl.org and now I have read about Spamhaus SBL and XBL on their website. What are your experiences with it? Have a look at http://www.dnsstuff.com/tools/ip4r.ch?ip=1.2.3.4 and check it. Christian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
