RE: Innovative Host Blacklisting Idea
At 12:42 PM 6/15/2007, Brent Kennedy wrote: How did you setup your spamtrap address with postfix.. Do you have them delivered after they are scanned by spamassassin or do you scan them and send them on from there? If you bypass SA, how are you doing that? For the spamtraps, I have an address hidden from human view on our web pages but obvious to bots. I also looked at the 550 rejects we were sending and picked several names that it seemed everyone was trying to send to. These were then all entered as aliases for my spam folder using Workgroup Manager from the OS X desktop. Also, when some site insists on an e-mail address, I give them one that goes straight to the spam folder. sa-learn is called from cron once an hour, I modified the script to learn and then deleted the messages in both my spam and ham folders. If you don't mind, what tarpit settings are you using? # added 12/15/6 per Pterobyte's app. note disable_vrfy_command = yes smtpd_helo_required = yes smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_hostname, reject_invalid_hostname, permit smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_sender, permit smtpd_data_restrictions = permit_mynetworks, reject_unauth_pipelining, permit #soft error limit added 1-8-6 by GJ Durand to slow down spam senders smtpd_soft_error_limit = 1 smtpd_error_sleep_time = 20 smtpd_client_connection_count_limit = 5 # hard error limit changed by GJ Durand, 5-31-5 to allow our mail backup # to send more messages. The default for this is 20. # lowered to 100 on 3-13-6 # changed to default on 3-14-6 since prxy.net is now filtering 550 errors smtpd_hard_error_limit = 20 -- Jerry Durand, Durand Interstellar, Inc. www.interstellar.com tel: +1 408 356-3886, USA toll free: 1 866 356-3886 Skype: jerrydurand
RE: Innovative Host Blacklisting Idea
How did you setup your spamtrap address with postfix.. Do you have them delivered after they are scanned by spamassassin or do you scan them and send them on from there? If you bypass SA, how are you doing that? If you don't mind, what tarpit settings are you using? I am using the following: smtp_error_sleep_time = 3s smtp_soft_error_limit = 1 smtp_hard_error_limit = 15 smtp_junk_command_limit = 50 smtp_recipient_overshoot_limit = 500 smtp_recipient_limit = 300 Thanks! -Brent -Original Message- From: Jerry Durand [mailto:[EMAIL PROTECTED] Sent: Friday, June 15, 2007 12:32 PM To: users@spamassassin.apache.org Subject: Re: Innovative Host Blacklisting Idea On Jun 15, 2007, at 9:06 AM, [EMAIL PROTECTED] wrote: > A simpler approach might be to blacklist senders that try multiple > non-existent recipients, regardless of mx priority > In Postfix I tarpit after the first bad recipient and eventually disconnect. That's cut things down quite a bit. > BTW: at one time I was quite happy with some pre-filtering on my > private mail (which is fetchmail ultimately feeding to SA) until I > found that SA would no longer recognize some spam in the bayes > section. So, if capacity permits, it might be a good idea to feed (a > random sampling of) pre-filtered spam to sa-learn I have a few spamtrap addresses that feed directly to sa-learn. Seems to work pretty well. Now to deal with the companies that send out billing, etc. through a third party that uses the original company's return address but third- party servers. I even had to explain SPF to an anti-virus company, not sure it they got it.
Re: Innovative Host Blacklisting Idea
Richard Frovarp wrote: I've heard Exchange and Notes/Domino in the past. I don't know if there is any truth to this or not. I swear Domino did/does it so that they can claim faster queue clearing times. In any case, be aware that caching of your involved MX and A records can have drastic effects on where a server will attempt to deliver your mail. If for any reason it has a cached A record for one of your lower pref MXes, but none for your higher pref MXes, many will just attempt to deliver to the lower pref MX rather than doing additional queries for your higher pref MXes' A records. You see this happen more often when the name server that is authoritative for the domain's MX record isn't also authoritative for the A records listed in that MX record. Daryl
Re: Innovative Host Blacklisting Idea
Marc Perkel wrote: Richard Frovarp wrote: Marc Perkel wrote: Terry Soucy wrote: In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. Terry, of my 8 MX records 4 are spam traps. The are the highest numbered MX. I have 3 real servers online right now on lower numbered MX records so no legit email should got to the 4 upper MX records. The hits over the last 9 hours are as follows: 65521, 74854, 26132 and 27076 hits This indicates to me that the spam bots are hitting random MX records. Of those 1511 have connected 10 times or more to one of these 4 addresses. The question is, how can you prove that those hits are bots? I've seen references that indicate different legitimate mailers don't always follow the correct order of MX records. Interesting. What legitimate servers don't follow MX order? I've heard Exchange and Notes/Domino in the past. I don't know if there is any truth to this or not.
Re: Innovative Host Blacklisting Idea
Jerry Durand schrieb: I have a few spamtrap addresses that feed directly to sa-learn. Seems to work pretty well. I do almost the same, but i first check email coming into the spamtraps and require a score of 2 before learning it to avoid poisening my bayes in case a real ham should come in. arni
Re: Innovative Host Blacklisting Idea
Marc Perkel wrote: > I'm trying out a new idea for blacklisting hosts. I have several email > servers for processing spam. These servers service my lowered numbered > MX records. I also have several dummy mx records that are higher > numbered than my real servers. So in theory no one should ever hit the > higher numbered servers. Especially when the IP addresses are on the > same server as the lower numbered MX. > > But as most of you know spammers don't play by the rules and they try > hitting the higher MX records first thinking there's less spam filtering > there. So what I'm doing is counting hits by IP address. At the moment > they have to hit it 75 times to get blacklisted. And it's all spammers > and spam bots. > > Who thinks this is interesting? When it works I think it will work great. That is what you are seeing right now while setting this up and monitoring it. In this time it is hard to imagine it not working right. I expect you to have great statistics from it. However the real problem is handling problems in the automated system when things do not work right. It is handling 100% of the time all of the problem cases that might arise. But thinking about problems and simulating problems is hard. The real world is very much more inventive and tireless in producing unexpected corner cases. Even if statistically the occurrence is very low these things can cause severe distress to us and so we are going to be very cautious of this type of approach. Bob
Re: Innovative Host Blacklisting Idea
On Jun 15, 2007, at 9:06 AM, [EMAIL PROTECTED] wrote: A simpler approach might be to blacklist senders that try multiple non-existent recipients, regardless of mx priority In Postfix I tarpit after the first bad recipient and eventually disconnect. That's cut things down quite a bit. BTW: at one time I was quite happy with some pre-filtering on my private mail (which is fetchmail ultimately feeding to SA) until I found that SA would no longer recognize some spam in the bayes section. So, if capacity permits, it might be a good idea to feed (a random sampling of) pre-filtered spam to sa-learn I have a few spamtrap addresses that feed directly to sa-learn. Seems to work pretty well. Now to deal with the companies that send out billing, etc. through a third party that uses the original company's return address but third- party servers. I even had to explain SPF to an anti-virus company, not sure it they got it.
Re: Innovative Host Blacklisting Idea
Richard Frovarp wrote: Marc Perkel wrote: Terry Soucy wrote: In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. Terry, of my 8 MX records 4 are spam traps. The are the highest numbered MX. I have 3 real servers online right now on lower numbered MX records so no legit email should got to the 4 upper MX records. The hits over the last 9 hours are as follows: 65521, 74854, 26132 and 27076 hits This indicates to me that the spam bots are hitting random MX records. Of those 1511 have connected 10 times or more to one of these 4 addresses. The question is, how can you prove that those hits are bots? I've seen references that indicate different legitimate mailers don't always follow the correct order of MX records. Interesting. What legitimate servers don't follow MX order?
Re: Innovative Host Blacklisting Idea
[EMAIL PROTECTED] schrieb: BTW: at one time I was quite happy with some pre-filtering on my private mail (which is fetchmail ultimately feeding to SA) until I found that SA would no longer recognize some spam in the bayes section. So, if capacity permits, it might be a good idea to feed (a random sampling of) pre-filtered spam to sa-learn Wolfgang Whats the problem with spamassassin and fetchmail? I'm using it myself and I only get complaints that 127.0.0.1 doesnt have a reverse dns. arni
Re: Innovative Host Blacklisting Idea
>> >> >> >> >> >> I'm trying out a new idea for blacklisting hosts. I have >> >> several email >> >> servers for processing spam. These servers service my lowered >> >> numbered >> >> >> > >> > As others said, not a good idea. >> > >> > Don't bother BL isting them, if they hit your dummy mx record, they die, >> > don't retry, and have in effect blacklisted themselves. >> > >> > >> >> What I see happening is that they are hitting MX randomly. So some times >> they hit a good server and sometimes they hit the trap. Once they have >> hit the trap several times then they are blacklisted in my hostkarma >> blacklist and if they hit a real server they are rejected at connect time. >> >> On my servers less than 1% of all email attempts make it as far as spam >> assassin. This reduces it further. >> >> A simpler approach might be to blacklist senders that try multiple non-existent recipients, regardless of mx priority BTW: at one time I was quite happy with some pre-filtering on my private mail (which is fetchmail ultimately feeding to SA) until I found that SA would no longer recognize some spam in the bayes section. So, if capacity permits, it might be a good idea to feed (a random sampling of) pre-filtered spam to sa-learn Wolfgang
Re: Innovative Host Blacklisting Idea
Marc Perkel wrote: Terry Soucy wrote: In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. Terry, of my 8 MX records 4 are spam traps. The are the highest numbered MX. I have 3 real servers online right now on lower numbered MX records so no legit email should got to the 4 upper MX records. The hits over the last 9 hours are as follows: 65521, 74854, 26132 and 27076 hits This indicates to me that the spam bots are hitting random MX records. Of those 1511 have connected 10 times or more to one of these 4 addresses. The question is, how can you prove that those hits are bots? I've seen references that indicate different legitimate mailers don't always follow the correct order of MX records.
Re: Innovative Host Blacklisting Idea
Terry Soucy wrote: In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. Terry, of my 8 MX records 4 are spam traps. The are the highest numbered MX. I have 3 real servers online right now on lower numbered MX records so no legit email should got to the 4 upper MX records. The hits over the last 9 hours are as follows: 65521, 74854, 26132 and 27076 hits This indicates to me that the spam bots are hitting random MX records. Of those 1511 have connected 10 times or more to one of these 4 addresses.
Re: Innovative Host Blacklisting Idea
Terry Soucy wrote: In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. But legit email would not hit these higher mx records so I doubt I'll have a problem with false positives. I think you're mistaken about this. To assume that legitimate mail servers won't use legitimate methods of delivering mail in the instance of service unavailability, IMHO, is a mistake. I think you're missing an important fact. The lowest 4 MX records point to legitimate servers. The highest 4 MX records point to the spamtrap which is on the lowest MX server. And it takes a lot of hits to get listed.
Re: Innovative Host Blacklisting Idea
In the testing we have done here, less than 1% of connections to our low priority MX actually cycled around to one of the higher priority MX systems to deliver the message. I'm still not sure if this is a growing pattern yet, but it could be a sign of spambots catching on. Whether or not they hit a *randon* MX record is kind of difficult to determin. As already mentioned, I would *love* to see this information. >> But legit email would not hit these higher mx records so I doubt I'll >> have a problem with false positives. I think you're mistaken about this. To assume that legitimate mail servers won't use legitimate methods of delivering mail in the instance of service unavailability, IMHO, is a mistake. __ Terry Soucy, Systems Analyst Integrated Technology Services University of New Brunswick, Fredericton Campus http://www.unbf.ca/its Voice: 506.447.3018Fax: 506.453.3590 E-mail: [EMAIL PROTECTED] **ITS is a scent-reduced workplace - www.unbf.ca/its/policies**
Re: Innovative Host Blacklisting Idea
On Fri, 15 Jun 2007, Marc Perkel wrote: Shane Williams wrote: Unless you have some other reliable source of statistics regarding how various entities choose MX records, I'd expect blacklisting this way is likely to garner significant false positives. It appears that some spammers hit the highest mx first and some spammers hit random mx records. But legit email would not hit these higher mx records so I doubt I'll have a problem with false positives. It appears that way based on what? If you have some data that demonstrates this pattern, please share. -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT iSchool =--+--- All syllogisms contain three lines | [EMAIL PROTECTED] Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Innovative Host Blacklisting Idea
Shane Williams wrote: On Fri, 15 Jun 2007, Marc Perkel wrote: What I see happening is that they are hitting MX randomly. So some times they hit a good server and sometimes they hit the trap. Once they have hit the trap several times then they are blacklisted in my hostkarma blacklist and if they hit a real server they are rejected at connect time. On my servers less than 1% of all email attempts make it as far as spam assassin. This reduces it further. The fact that you're seeing random connections is out of line with your own assertion that spammers "don't play by the rules and they try hitting the higher MX records first thinking there's less spam filtering there." The two most likely conclusions of this are that a) Spammers don't behave the way you think they behave and/or b) spammers do behave the way you presume they do, but you're catching legit servers that pick an MX randomly rather than going with lowest first. Either way, it suggests there's a flaw in the original suppositions that led you to employ this method of blacklisting. Unless you have some other reliable source of statistics regarding how various entities choose MX records, I'd expect blacklisting this way is likely to garner significant false positives. It appears that some spammers hit the highest mx first and some spammers hit random mx records. But legit email would not hit these higher mx records so I doubt I'll have a problem with false positives.
Re: Innovative Host Blacklisting Idea
On Fri, 15 Jun 2007, Marc Perkel wrote: What I see happening is that they are hitting MX randomly. So some times they hit a good server and sometimes they hit the trap. Once they have hit the trap several times then they are blacklisted in my hostkarma blacklist and if they hit a real server they are rejected at connect time. On my servers less than 1% of all email attempts make it as far as spam assassin. This reduces it further. The fact that you're seeing random connections is out of line with your own assertion that spammers "don't play by the rules and they try hitting the higher MX records first thinking there's less spam filtering there." The two most likely conclusions of this are that a) Spammers don't behave the way you think they behave and/or b) spammers do behave the way you presume they do, but you're catching legit servers that pick an MX randomly rather than going with lowest first. Either way, it suggests there's a flaw in the original suppositions that led you to employ this method of blacklisting. Unless you have some other reliable source of statistics regarding how various entities choose MX records, I'd expect blacklisting this way is likely to garner significant false positives. -- Public key #7BBC68D9 at| Shane Williams http://pgp.mit.edu/| System Admin - UT iSchool =--+--- All syllogisms contain three lines | [EMAIL PROTECTED] Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Innovative Host Blacklisting Idea
Michael Scheidell wrote: -Original Message- From: Marc Perkel [mailto:[EMAIL PROTECTED] Sent: Friday, June 15, 2007 3:19 AM To: users@spamassassin.apache.org Subject: Innovative Host Blacklisting Idea I'm trying out a new idea for blacklisting hosts. I have several email servers for processing spam. These servers service my lowered numbered As others said, not a good idea. Don't bother BL isting them, if they hit your dummy mx record, they die, don't retry, and have in effect blacklisted themselves. What I see happening is that they are hitting MX randomly. So some times they hit a good server and sometimes they hit the trap. Once they have hit the trap several times then they are blacklisted in my hostkarma blacklist and if they hit a real server they are rejected at connect time. On my servers less than 1% of all email attempts make it as far as spam assassin. This reduces it further.
Re: Innovative Host Blacklisting Idea
Raymond Dijkxhoorn wrote: Hi! servers for processing spam. These servers service my lowered numbered MX records. I also have several dummy mx records that are higher numbered than my real servers. So in theory no one should ever hit the higher numbered servers. Especially when the IP addresses are on the same server as the lower numbered MX. But as most of you know spammers don't play by the rules and they try hitting the higher MX records first thinking there's less spam filtering there. So what I'm doing is counting hits by IP address. At the moment they have to hit it 75 times to get blacklisted. And it's all spammers and spam bots. Who thinks this is interesting? Yeah really cool idea, if your smtp is too busy to accept connections and people start sending on your second ip, they get blacklisted after some time, really cute. Since you dont accept there either. I think its a stupid idea! I have several servers on several lower numbered MX records and this is on the same computer as my lowest mx. If the load levels get high it quits recording hits.
Re: Innovative Host Blacklisting Idea
Daryl C. W. O'Shea wrote: Marc Perkel wrote: I'm trying out a new idea for blacklisting hosts. I have several email servers for processing spam. These servers service my lowered numbered MX records. I also have several dummy mx records that are higher numbered than my real servers. So in theory no one should ever hit the higher numbered servers. Especially when the IP addresses are on the same server as the lower numbered MX. Nobody except for users of Domino, Blackberry, and who knows how many other business mail platforms that send mail to whatever MX they feel like. Who thinks this is interesting? Apparently you do. Sorry Marc, couldn't resist. :) This is pretty old news though. You've even brought it up yourself at least once, but probably five times, before. I've brought up the idea of using high numbered fake MX records several times and it's very effective. What's new here is that I'm powering my public hostkarma blacklist database in part by the IP addresses that make multiple attempts to send email to high numbers mx records when low numbered mx records are available. In the last 7 hours I get 145000 hits that I've recorded. And checking the dnsstuff lookup a lot of these IP addresses aren't listed with anyone but me.
RE: Innovative Host Blacklisting Idea
> -Original Message- > From: Marc Perkel [mailto:[EMAIL PROTECTED] > Sent: Friday, June 15, 2007 3:19 AM > To: users@spamassassin.apache.org > Subject: Innovative Host Blacklisting Idea > > > I'm trying out a new idea for blacklisting hosts. I have > several email > servers for processing spam. These servers service my lowered > numbered As others said, not a good idea. Don't bother BL isting them, if they hit your dummy mx record, they die, don't retry, and have in effect blacklisted themselves. _ This email has been scanned and certified safe by SpammerTrap(tm). For Information please see http://www.spammertrap.com _
Re: Innovative Host Blacklisting Idea
Hi! servers for processing spam. These servers service my lowered numbered MX records. I also have several dummy mx records that are higher numbered than my real servers. So in theory no one should ever hit the higher numbered servers. Especially when the IP addresses are on the same server as the lower numbered MX. But as most of you know spammers don't play by the rules and they try hitting the higher MX records first thinking there's less spam filtering there. So what I'm doing is counting hits by IP address. At the moment they have to hit it 75 times to get blacklisted. And it's all spammers and spam bots. Who thinks this is interesting? Yeah really cool idea, if your smtp is too busy to accept connections and people start sending on your second ip, they get blacklisted after some time, really cute. Since you dont accept there either. I think its a stupid idea! Bye, Raymond.
Re: Innovative Host Blacklisting Idea
Marc Perkel wrote: I'm trying out a new idea for blacklisting hosts. I have several email servers for processing spam. These servers service my lowered numbered MX records. I also have several dummy mx records that are higher numbered than my real servers. So in theory no one should ever hit the higher numbered servers. Especially when the IP addresses are on the same server as the lower numbered MX. Nobody except for users of Domino, Blackberry, and who knows how many other business mail platforms that send mail to whatever MX they feel like. Who thinks this is interesting? Apparently you do. Sorry Marc, couldn't resist. :) This is pretty old news though. You've even brought it up yourself at least once, but probably five times, before. Daryl
Innovative Host Blacklisting Idea
I'm trying out a new idea for blacklisting hosts. I have several email servers for processing spam. These servers service my lowered numbered MX records. I also have several dummy mx records that are higher numbered than my real servers. So in theory no one should ever hit the higher numbered servers. Especially when the IP addresses are on the same server as the lower numbered MX. But as most of you know spammers don't play by the rules and they try hitting the higher MX records first thinking there's less spam filtering there. So what I'm doing is counting hits by IP address. At the moment they have to hit it 75 times to get blacklisted. And it's all spammers and spam bots. Who thinks this is interesting?
