Re: Apache SpamAssassin and Spammers 1st Amendment Rights
I think this argument is sort of odd. Here is my take: You have a right to say what you want. I have a right to ignore you. Spam filtering allows me to exercise my right to ignore you. jay plesset, IT director. D. P. Design On 11/20/2020 3:59 PM, Eric Broch wrote: It's a given people on this side of the argument don't like spam, your conclusion being correct, it still comes down to preference. They prefer sending spam you prefer they didn't. They, ERRONEOUSLY, justify sending spam using a political argument (*their protected right), our side is rejecting politics and its origin, religion; so, it still comes down to preference, and ultimate authority rests in man. It comes down to, "Who is to say?" I argue, and I think the original post argues against their position. I also argue that the political (based in the religious) needs to be brought bear to refute them. I agree with the original post that they improperly use the 1st Amendment for justification but for the wrong reasons. *Note: According to the founding documents of the u.S. rights come from the Creator. On 11/20/2020 2:45 PM, Rob McEwen wrote: On 11/20/2020 4:37 PM, Eric Broch wrote: It seems spammers are using political arguments to justify their actions. I'll give them credit, at least they're trying to justify what they do by something greater than (outside of) themselves, albeit wrongly. It seems people on this side of the argument want to jettison politics (and religion) and have no justification (only personal preference) for what they do. Curious! At the core spammers seem more logically consistent than those who oppose them. I have extremely large amounts of spams on file in my spamtrap spam collection from all various political viewpoints, political parties, and moral/ethical/religious viewpoints - MANY of them think that THEIR greater good justifies spamming, and ironically their beliefs are often in 100% contradiction to OTHER spammers who have opposite beliefs, but likewise think that their spam is justified by THEIR "greater good". Thankfully, it isn't my job to determine who is justified and, instead, I believe that NONE of them are justified in sending spam - spam is about *consent* - NOT *content*.
Re: UTF-8 rule generator script Re: UTF-8 rules, what am I missing?
On 09/27/2014 01:16 PM, John Hardin wrote: > On Fri, 26 Sep 2014, Adi wrote: >> I don't know if SA converts the text on the fly. > > In my experience it does not. There's been some discussion of charset > normalization, but I don't think that's been implemented yet, so SA is > still seeing whatever bytes are in the raw message. normalize_charset is documented at least since 3.3.2. I found some list traffic expressing concerns about performance problems, but I've turned it on on (low-to-medium-volume) mail servers I'm responsible for and haven't seen problems. (We get about 25K incoming messages a day at work.) Haven't made extensive use of it, though, and I just recently figured out that my failed attempts to do so were because the rule files themselves weren't being interpreted as UTF-8 (so I need to use Darxus' preprocessing scripts or something similar). Seems like it would be a huge convenience if either (1) turning on normalize_charset forced interpretation of rule files as UTF-8, (2) there were a similar setting to specify the encoding of rule files, or (3) there were a way on a file-by-file basis to say what charset the rules in the file were in (which is probably best since it would facilitate custom rule sharing across sites). That's off the top of my head with no thought so it may be dumb. :-) Jay
Re: Ready to throw in the towel on email providing...
On 7/29/2014 9:33 AM, Ted Mittelstaedt wrote: On 7/28/2014 4:17 PM, Jay Plesset wrote: My church decided to go with O-365, without even evaluating any alternatives. We have an unemployed IT person that talked the staff into this, even though I've offered to implement a "real" e-mail solution multiple times, and even provide hardware to run it on. Apparently they didn't understand if the guy was an unemployed IT person there was a reason he was unemployed! Agreed. "free" was the biggest draw, then "no administration". *sigh*. But, the "no administration" isn't true at all. There's still administration. Does Microsoft provide Office 365 free to churches? I know that they had ridiculously cheap server license pricing (through their Charity Pricing program) but I didn't know they had got to Free with Office 365? That's what they told me. I said, "Free for now at least. . . " I did a lot of work for my families church a decade ago in the volunteer area. Both on the building committee and IT work for them. I learned after a year that if your goal is to have people who don't understand or appreciate what you do for them, and shit all over what you do for them, volunteer for a church. Oh, yeah. My wife and I built a new website for them. Last summer, the staff didn't bother with updating the calendar, and come fall, they said, "we forgot how". The other thing about churches is that the staff runs more than they should, and really, truly doesn't understand the reason for a website, marketing, etc. jay There's a reason most churches constantly solicit for volunteers. A church is the only place that a professional tradesperson can volunteer his services and during the job be told that he's doing it wrong, by people who have never held a wrench, paintbrush, pipe threader, network cable, you name it. I actually saw one time a couple come in and paint a large room in the church, used very good paint, excellent coverage, masked off everything, etc. and when they left the room looked like a pro had done it - no paint runs or drips where they weren't supposed to be etc. Then 2 weeks later the church paid to have a professional come in and paint the room - again - same color - same paint. When I asked why, I was told "we had the painters scheduled for that room, they should have asked us before painting in there" This is the kind of politics you run into with church volunteering. Ted jay plesset IT, dp-design.com On 7/28/2014 3:49 PM, Ian Zimmerman wrote: On Mon, 28 Jul 2014 12:57:38 -0400 "David F. Skoll" wrote: David> 1) Gmail is actually pretty good at filtering spam. I can't David> speak for MSFT since I don't use it. David> 2) Especially in North America, companies are short-sighted and David> go for quick fixes and things that look cheap up-front without David> considering the long-term costs. David> 3) Especially in North America, people don't see the value in David> learning technology. They want simple, spoon-fed solutions and David> they love the word "oursourcing". Sorry if (2) and (3) are not David> PC, but the slag against North Americans is based on my personal David> experience. :) And hey, I'm Canadian so I can dis my own crowd... David> 4) Most non-technical small businesses equate "Mail Server" with David> "Microsoft Exchange", and Microsoft has steadily been making David> Exchange more and more of a PITA to administer. Each new version David> of Exchange breaks things and requires learning new procedures. David> Combine that with (3) and we see that MSFT is using on-premise David> Exchange as a trojan horse to get people on O-365. The huge pool David> of "managed service providers" that recommend MSFT solutions is David> by-and-large staffed by incompetents who are only too happy to David> shove their customers onto O-365 and collect kickbacks every David> month. Good summary, but I think you forgot (5): They have prettier icons. I am not 100% kidding, either. --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com
Re: Ready to throw in the towel on email providing...
My church decided to go with O-365, without even evaluating any alternatives. We have an unemployed IT person that talked the staff into this, even though I've offered to implement a "real" e-mail solution multiple times, and even provide hardware to run it on. "free" was the biggest draw, then "no administration". *sigh*. jay plesset IT, dp-design.com On 7/28/2014 3:49 PM, Ian Zimmerman wrote: On Mon, 28 Jul 2014 12:57:38 -0400 "David F. Skoll" wrote: David> 1) Gmail is actually pretty good at filtering spam. I can't David> speak for MSFT since I don't use it. David> 2) Especially in North America, companies are short-sighted and David> go for quick fixes and things that look cheap up-front without David> considering the long-term costs. David> 3) Especially in North America, people don't see the value in David> learning technology. They want simple, spoon-fed solutions and David> they love the word "oursourcing". Sorry if (2) and (3) are not David> PC, but the slag against North Americans is based on my personal David> experience. :) And hey, I'm Canadian so I can dis my own crowd... David> 4) Most non-technical small businesses equate "Mail Server" with David> "Microsoft Exchange", and Microsoft has steadily been making David> Exchange more and more of a PITA to administer. Each new version David> of Exchange breaks things and requires learning new procedures. David> Combine that with (3) and we see that MSFT is using on-premise David> Exchange as a trojan horse to get people on O-365. The huge pool David> of "managed service providers" that recommend MSFT solutions is David> by-and-large staffed by incompetents who are only too happy to David> shove their customers onto O-365 and collect kickbacks every David> month. Good summary, but I think you forgot (5): They have prettier icons. I am not 100% kidding, either.
Re: Current best-practices around normalize_charset?
On Wed, 2014-03-12 at 19:04 -0700, Ivo Truxa wrote: > Your message is a few months old, but I see no answer, and stumbled upon it > when writing an enhanced version of the normalize_charset feature, so > thought that I could perhaps help. Thanks! I'm glad to hear of your experiences. > [R]egardless whether > you use normalizing or not, as long as you need to match non-ASCII patterns, > you need to write rules also in Unicode anyway, because you cannot reject > Unicode messages. Indeed! And even if you only want to accept messages in English (or some other ASCII-supported language), nowadays it's not at all uncommon for messages to have dingbats or printer's quotation marks in them -- or one of your correspondents might be sitting at a relative's computer or in an internet cafe somewhere and the subject line might get the Chinese equivalent of "Re:" prepended to it, or the body might have a disclaimer in French appended. > Another possibility may be normalizing, instead to UTF, to plain 7bit > US-ASCII. The currently proposed patch for ASCII normalizing transliterates > also non-Latin alphabets. The patch was proposed to the dev list, so > impatient and courageous users might want to try it on a non-production > server, but be warned that it is not any official code (at least not now), > and currently very little tested. Interesting idea! I searched in the spamassassin-dev archives but I don't think I found the right patch; could you point me at it? How do you handle non-alphabetic scripts (like CJK, where a character may have multiple pronunciations both within and between languages)? Seems like just normalizing them to U+ might be better than trying to transcribe them. (And that would let a brave or foolhardy mail administrator write rules to match patterns seen in, say, Chinese-language spam even without knowing Chinese, or even without knowing what language the spam was in.) Anyway, glad to hear that normalize_charset hasn't been causing you problems, and for us, normalizing to UTF8 is almost certainly what we want if it's reasonably safe. Jay -- Jay Sekora Linux system administrator and postmaster, The Infrastructure Group MIT Computer Science and Artificial Intelligence Laboratory
Re: dependency hell
| If you can get e-mail across this not-quite-air-gap, wouldn't it be far | more effective to put your anti-spam gateway on the *internet side* of the | gap? [snips] i have one there already. this is to implement the management-required local stuff that won't be done by the (purchased) spam filter. j. -- Jay Scott 512-835-3553g...@arlut.utexas.edu Head of Sun Support, Sr. System Administrator Applied Research Labs, Computer Science Div. S224 University of Texas at Austin
Re: dependency hell]
Sorry. Haven't been able to work on this for several weeks. (I'm the OP.) The machine runs RH linux (5.4, IIRC) installed via kickstart, using a "stock" configuration -- no special efforts to include any perl packages. So it's just a basic configurtion, perl-wise. However, it turns out that there is a CPAN mirror inside our firewall (nice of them to tell me about it -- which, in fact, they didn't; I found it by snooping around). Someone did send me a list of the dependencies they knew about (which aren't in the INSTALL file), but, that's __supposed__ to be moot, if this CPAN mirror is all that it should be. We shall see -- though, right now, I don't know when we shall see, since I'm still called away to do this other thing. After all the time I spent chasing dependencies it seems like I ought to be able to find time to test out this mirror, but About the only thing we can get past the "air gap" (not a true air gap, but it's the shortest way to describe it) is email. Management has all these grandfathered requirements about stuff they must have _and_ stuff I can't do (e.g.,no RBLs) _and_ (so it seems to me) a real problem with certain kinds of spam (read blue pills), so, bottom line is, I'm reinventing a lot of wheels. Don't get me started. Thanks to all who replied. I should have said so earlier, but j. - Forwarded message from Karsten Br?ckelmann - Date: Tue, 29 Oct 2013 22:45:40 +0100 From: Karsten Br?ckelmann To: users@spamassassin.apache.org Subject: Re: dependency hell X-Mailer: Evolution 2.22.1.1 On Tue, 2013-10-29 at 13:27 -0500, Jay G. Scott wrote: > I have a machine on which I'd like to run spamassassin. > But it's behind an air gap. It's not on the internet. > I've been downloading missing perl packages a handful > at a time, but I despair of the list ever coming to an > end. > 2. Or does somebody have this list of dependencies > already? See the INSTALL file. It lists required and optional Perl Modules SA depends on. Dependencies of these SA dependencies are outside our scope. CPAN and (distro) package management systems handle these. I notice you didn't (yet) answer the questions about your distribution and how you installed Linux in the first place. However, even without telling us -- you should be able to extract the complete dependency tree out of your distro's package management. In case you are permitted to tell -- I'm also curios about the reason for these strict requirements, and what you're going to use SA for in such an environment. -- char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4"; main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}} - End forwarded message - -- Jay Scott 512-835-3553g...@arlut.utexas.edu Head of Sun Support, Sr. System Administrator Applied Research Labs, Computer Science Div. S224 University of Texas at Austin
dependency hell
I have a machine on which I'd like to run spamassassin. But it's behind an air gap. It's not on the internet. I've been downloading missing perl packages a handful at a time, but I despair of the list ever coming to an end. 1. I _might_ (and might not) be able to put a similar machine outside the air gap. If I install spamassassin on it, is there any way to log what extra packages spamassassin brings in to satisfy dependencies? If I knew what was brought in, I could get all the dependencies at once. 2. Or does somebody have this list of dependencies already? 3. Or, should I do item (1) above and then tar up the perl tree? Is it going to go to a different perl tree? FWIW the box is (or will be) running linux. (I'm ready to give up on this, frankly.) j. -- Jay Scott 512-835-3553g...@arlut.utexas.edu Head of Sun Support, Sr. System Administrator Applied Research Labs, Computer Science Div. S224 University of Texas at Austin
Re: UTF-8 Spam rules
On 09/16/2013 10:12 AM, Kevin A. McGrail wrote: Anyone have some examples of rules designed to catch words by content in UTF-8 encoded messages? I'm doing some work on improving this. Are you trying to match UTF-8 encoded messages as a stream of bytes, or are you using normalize_charset? (And if the latter, how is it working for you? I asked on this list a while back whether the advice I'd seen that normalize_charset is dangerous resource-wise was still valid, and didn't get any replies.) I guess I don't have anything to offer other than that I really want to see what you come up with, too. :-) Jay
Current best-practices around normalize_charset?
Hi. We're running SpamAssassin 3.3.1, and pursuant to some advice I've seen in archives of this list and spamassassin-dev (e.g., http://osdir.com/ml/spamassassin-dev/2009-07/msg00156.html), I am *not* using normalize_charset. Unfortunately, this makes filtering text in binary encodings almost impossible, since even if you can come up with a word you want to match, word boundaries aren't at byte boundaries, so if I were to try to write rules byte-by-byte, I'd need several possible match strings, and I wouldn't be able to match the first or last character of the phrase I want to match (which for, say, Chinese, where words tend to be one or two characters long, is a big problem). That's on top of the alternative patterns needed to represent non-Unicode encodings, of course. Anyway, my question is, is that advice still valid (for 3.3.1, which is packaged for Debian Squeeze, or for latest stable)? And if so, what do people tend to do to write rules for East Asian character sets (or, for that matter, for Western character sets encoded in binary to make them harder to filter)? The traffic on the bug report quoted in the above message is kind of ambiguous. (I will note that ok_languages and ok_locales are pretty useless here, at least for site-wide use, since we have users with correspondence in pretty much any language we've ever seen spam in.) Jay -- Jay Sekora Linux system administrator and postmaster, The Infrastructure Group MIT Computer Science and Artificial Intelligence Laboratory
Re: New virus outbreak with malformed payload
yes, saw both the scanner ones and the new ones, too. jay plesset IT, dp-design.com On 6/21/2013 10:40 AM, David F. Skoll wrote: Hi, We're seeing a huge rash of viruses with malformed payloads. They're supposed to contain a ZIP file, but the MIME part supposedly containing the ZIP file simply contains: Error[Base64] Sample: http://pastebin.com/fkjf9LHR Yesterday, they were "Scanned Copy" spams from an HP printer. Today they are "Invoice Notification for June 2013" spams. Annoyingly, the envelope sender is no-re...@intuit.com which has an SPF permerror... FAIL. $ spfquery --id intuit.com --ip 192.168.1.1 permerror intuit.com ... spf-ext-a.intuit.com: Maximum DNS-interactive terms limit (10) exceeded intuit.com ... spf-ext-a.intuit.com: Maximum DNS-interactive terms limit (10) exceeded Received-SPF: permerror (intuit.com ... spf-ext-a.intuit.com: Maximum DNS-interactive terms limit (10) exceeded) identity=mailfrom; envelope-from=intuit.com *sigh* Anyone else seeing tons of these? Regards, David.
Re: Curious phenomenon with 9-repetitions of each spam...
If each message is indeed a separate message, then no sane MTA could find them the "same" message. Each will have a unique message ID, and will have different envelope addresses. I certainly would not use an MTA that would combine such. jay plesset Oracle Messaging Server support. On 9/8/2011 2:53 PM, John Hardin wrote: On Thu, 8 Sep 2011, Bowie Bailey wrote: On 9/8/2011 2:26 PM, Steve wrote: In any case, as it turns out, none of this helps me store a single inbound spam once - rather than duplicate it for each address in the envelope... which, to my thinking, remains a sane objective... Agreed. Although you would think that a sane MTA would see that all aliases resolve to a single destination and just deliver the message once. Agreed, but that's probably an issue for the Postfix list...
Re: __PILL_PRICE Problems
On Sun, 2011-03-20 at 10:50 -0400, Matt Elson wrote: > I'm having the problem on an Intel 32-bit Linux machine running 5.8.8 > with the same version of re2c, so it looks like the common thread is > Intel 32 bit + re2c. I'll see if I can throw up 64 bit machine to test > further. We saw the problem on an x86_64 machine, running Perl 5.10.0. # uname -a Linux [redacted] 2.6.26-2-amd64 #1 SMP Tue Mar 9 22:29:32 UTC 2010 x86_64 GNU/Linux # re2c -version re2c 0.13.5 # spamassassin -V SpamAssassin version 3.3.1 running on Perl version 5.10.0 # dpkg -l perl-base spamassassin Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad) ||/ Name VersionDescription +++-==-==- ii perl-base 5.10.0-19lenny minimal Perl system ii spamassassin 3.3.1-1~bpo50+ Perl-based spam filter using text analysis (it's a Debian Lenny machine, with SpamAssassin from Lenny backports.) # grep model.name /proc/cpuinfo | uniq model name : Intel(R) Xeon(R) CPU5160 @ 3.00GHz So it doesn't seem to be exclusive to 32-bit x86 architecture. On the other hand, we had another machine running the same Debian version, also 64-bit, also a Xeon (though different speed), with the same versions of perl-base and spamassassin, and also using compiled rules, which did *not* see the problem. So it's puzzling. At first I commented out the rules and recompiled, but then I discovered that adding meta __PILL_PRICE_1 (0) meta __PILL_PRICE_2 (0) meta __PILL_PRICE_3 (0) to /etc/spamassassin/local.cf as suggested by Karsten Bräckelmann worked fine --Jay
Re: Should Emails Have An Expiration Date
How about something that doesn't depend on the SENDER setting something? I've set my system up to automatically "empty the trash" after 30 days, and dump the "spam" folder after 2 weeks. I could easily set up an "archive" folder for my users and automatically "expire" their inbox at whatever time period I want If they want to keep something forever, move it to the "archive" folder.. jay plesset IT, dp-design.com Sr. Support Engineer, Oracle On 2/28/2011 1:51 PM, Matt wrote: Looking at top 8 newest messages from my personnel email account: Newsletter Magazine Renwal Offer Ebook Update Notice Travel Deal of Week Sales Flyer with weekly specials Reply to forum thread Anouther Newsletter Custommer Service Response. Etc. Hmm. All of these could really expire at 30 day mark except custommer service response in my opinion. Even if they expired at 365 days its better then sitting there forever. I can not honestly think of any reason to keep any of these past 30 days. If personnel messages never expire thats fine but all this other crap can AFAIC. On personnel messages perhaps give sender option of choosing option of 30days, 12months or never and default to never. Seems like new email clients default to leaving mail on server rather then downloading and deleting. Thats fine tell every email user is using 10G+ for email server space. Server space is not free and backups take time and even more space. Plus this all slows down POP3 etc as everytime you check email it must return a list of messages and when there are thousands of messages to look at that this can really load down a server. I imagine this would be like return receipts. Yeah its there but that does not mean all clients or servers are going to honor it.
Re: [OT?] Web Form Spam
I've been getting 2 or 3 of these daily. The mail address typically matches the "name" put in, it's always a gmail address, and so far, it's always been a bad mail address. It's more an annoyance than a problem, my mailing program sends out a confirm, and when it bounces, I remove the bogus entry from the db. jay plesset IT, dp-design.com Jason Bertoch wrote: On 1/29/2010 12:44 PM, te...@cnysupport.com wrote: Really, I was just trying to figure out what the point would be for someone to fill out the form with obviously invalid data. My guess is that it's a spammer's bot looking for a broken web form to abuse.
Re: Email / Inbox Speed Problems
Ted Mittelstaedt wrote: Jay Plesset wrote: Ted Mittelstaedt wrote: What is the point of a quota system that does not limit the received mail? And if it does limit it then we get irate calls from people complaining that sally sue sent them a message and got it returned. Of course, sally sue never reads the error message and tells our user that their e-mail box is too large - or if she did, then irate user thinks it's our problem. Um, well, that's not exactly how it works. System messages and "guranteed delivery" messages always get through. Messages that will take a user over quota are held for a configurable "grace" period, and the user is warned that they are over quota at a configurable repeat rate. Messages are returned to the sender after a configurable hold period. there are plenty of knobs for you to turn. . . I can understand that, and in a corporate environment where you have more control over the userbase (and the users are much more inclined to listen to you, after all it's not their money on the line) I am sure it would work well. Of course, if I was using a -standards- based method of handling mail in such an environment (ie: NOT MS Exchange) then I wouldn't be using POP3 in the first place, I'd be using IMAP and I'd also setup a set of shared e-mail folders accessible from the IMAP client. I'd also probably run some scripts that warned me when people were letting their inbox get too large, so I could go train them in how to drag the mail messages they want to save into private or shared folders on the server. But, that's my style - other admins might go out and buy software to do this. Ultimately it works the same way. This discussion really illustrates the disconnect between people who write e-mail systems for a living and what ISP's need. While I've not looked at the Sun comm suite your talking about, I'm sure it's not that much different from many other commercial e-mail systems I've been pitched over the years from people wanting to make my life easier as an ISP admin (in exchange for some money, of course) Just to be clear, the software I'm offering is designed not to "replace Exchange", but for ISP's or large corp accounts. One of the customers I'm assigned to support has 100 "store" systems, each with 500,000 mailboxes and typically sees 30,000 simultaneous imap connections. We often see systems with a million mailboxes. You like webmail? Our webmail interface also talks to our Calendar Server, our IM server, and should shortly include gateways into other IM systems. It's all pretty open, based on standard protocols, and no, there isn't a gui admin interface. Maybe later. The MTA has been around for 25 years, previously called, "PMDF". Yes, we'd like you to license it, and pay for support. You can download and use at no cost. . . jay The problem though is when I've drilled into them, I've always found issues like this. Those systems are written first as competitors to Exchange, and make a boatload of assumptions about the users, and the admin's skill level. Usually they assume the users are smarter and the admins are dumber. That's about right for the corporate networks I've admined. But ISPs don't survive unless the admin is a lot smarter - because the users in general are a lot dumber. Oh, there's exceptions - but most of the time it's customers who work in office environments and come home and want the same level of support they get at the office. Those people are in a minority. The majority of customers quite obviously don't understand very much, and with a surprising number of them they don't even understand the accepted nomenclature. If I had a nickel for every time I've told a user "OK now open your web browser" and gotten back "what's a web browser" I'd be a rich man. I've learned to refer to web browsers with phrases like "go to google" or "click on the Internet". This is the level of skill we deal with regularly. After all, it's not the new-technology embracers who are calling in for ISP support. It's the people who were left behind years ago, who are only on the Internet because the rest of their family won't spend the time to communicate with them unless they are on facebook or e-mail. At least once a week I and the other admins get someone who we just shake our head over and wonder why in the world this person is even wasting their money and time with a computer at all - they are like the old grandmother who never drives on the highway and never drives faster than 45Mph who owns a Lamborghini. It's really a sad thing, to be honest. Not to mention the user thinks their inbox is -on their mac- not on our mailserver, since of course they
Re: Email / Inbox Speed Problems
right, and you are wrong. For us to win at the game we must educate the users, and the most ignorant of the users will only open their minds for knowledge for a very short time, before it snaps closed like a steel trap, and they will never believe there's a problem unless they see it for themselves. After all, just think of your average conservative Republican's reaction to Global Warming. It's not something they can see and their brains are (apparently) incapable of imagination so they cannot imagine that Global Warming is real, that's why they make silly arguments like "global warming must not be happening because we are having a pretty cold winter" It's the same principle in operation here. Well, it's the devil you know vs the one you don't. I was offereing a solution that doesn't slow down. If you don't think it would help you, then you don't have to look at it. jay Ted Jay Plesset wrote: Many of my users use the various quota settings in Messaging Server. You can set quotas on message number and/or mailbox size. Notifications are sent to the user, even if they're over quota. . . You can set quota individually, by "class of service", or globally. Yes, it'll run on the same hardware you're running now. On Redhat 4 or 5, or Solaris. jay Ted Mittelstaedt wrote: Jay Plesset wrote: Geez, unless your users are into the millions of messages, maybe you need a more scalable mail server. My day job is support of the Sun comms suite. I only get these when there are litterally tens of millions of messages in an inbox. Where we generally get these problems is when users are running MacOS X and using the included free Apple Mail as a POP3 client, because one of the DEFAULTS of that client is to leave a copy of the mail message on the server. The typical scenario is that we get one of these users who runs it this way for a couple months, then one day their relative starts e-mailing them 50MB pictures of their latest vacation, and once their e-mail box exceeds 800MB in size, popper (qpopper) starts getting really slow in downloading the message ID list and their client starts running like a dog. There's probably many ways I could fix it, from replacing qpopper to going to faster disks or more powerful hardware, or running a nightly script that squawks about the bad citizens, but I frankly don't feel compelled to allocate all of our POP3 users a gigabyte of disk space for their mailbox, and if did fix it then I'd have to setup quotas on /var/mail Doing it this way penalizes only the users who engage in the objectionable behavior, and it penalizes them in such a way that it doesn't cause them to lose mail, or cause the server to reject incoming mail messages to them, or causes mail they have to be truncated. And it also doesn't do it in a way that is sudden - the user just starts noticing things getting slower and slower and slower over time - so they have plenty of time to contact us at their leisure. I suppose that one of these days the author of qpopper will rewrite the search logic in the qpopper program to fix this and then I'll have to find some other way to gently enforce this. Ted jay Ted Mittelstaedt wrote: Sean Leinart wrote: -Original Message- From: Sean Leinart [mailto:slein...@fscarolina.com] Sent: Friday, October 23, 2009 2:04 PM To: TJ Russ Cc: allison.ays...@lonesource.com; Spamassassin Mailing List Subject: Email / Inbox Speed Problems Hi TJ, Looking over your Inbox situation, you suffer from the same problem as most here do. You have too much email stored on the server. Can you give me a rundown of the folders that can be eliminated in your Inbox, we can archive them off then delete them from your folders that are online, this will help a great deal. Thank you, Sean Leinart Network Systems Engineer First Service Carolina Inc. Raleigh, North Carolina United States slein...@fscarolina.com 919-832-5553 DOH!! List, please disregard the erroneous CC: post to the list. I had to look twice since it was the identical problem to what we deal with every week around here. Ted
Re: Email / Inbox Speed Problems
Many of my users use the various quota settings in Messaging Server. You can set quotas on message number and/or mailbox size. Notifications are sent to the user, even if they're over quota. . . You can set quota individually, by "class of service", or globally. Yes, it'll run on the same hardware you're running now. On Redhat 4 or 5, or Solaris. jay Ted Mittelstaedt wrote: Jay Plesset wrote: Geez, unless your users are into the millions of messages, maybe you need a more scalable mail server. My day job is support of the Sun comms suite. I only get these when there are litterally tens of millions of messages in an inbox. Where we generally get these problems is when users are running MacOS X and using the included free Apple Mail as a POP3 client, because one of the DEFAULTS of that client is to leave a copy of the mail message on the server. The typical scenario is that we get one of these users who runs it this way for a couple months, then one day their relative starts e-mailing them 50MB pictures of their latest vacation, and once their e-mail box exceeds 800MB in size, popper (qpopper) starts getting really slow in downloading the message ID list and their client starts running like a dog. There's probably many ways I could fix it, from replacing qpopper to going to faster disks or more powerful hardware, or running a nightly script that squawks about the bad citizens, but I frankly don't feel compelled to allocate all of our POP3 users a gigabyte of disk space for their mailbox, and if did fix it then I'd have to setup quotas on /var/mail Doing it this way penalizes only the users who engage in the objectionable behavior, and it penalizes them in such a way that it doesn't cause them to lose mail, or cause the server to reject incoming mail messages to them, or causes mail they have to be truncated. And it also doesn't do it in a way that is sudden - the user just starts noticing things getting slower and slower and slower over time - so they have plenty of time to contact us at their leisure. I suppose that one of these days the author of qpopper will rewrite the search logic in the qpopper program to fix this and then I'll have to find some other way to gently enforce this. Ted jay Ted Mittelstaedt wrote: Sean Leinart wrote: -Original Message- From: Sean Leinart [mailto:slein...@fscarolina.com] Sent: Friday, October 23, 2009 2:04 PM To: TJ Russ Cc: allison.ays...@lonesource.com; Spamassassin Mailing List Subject: Email / Inbox Speed Problems Hi TJ, Looking over your Inbox situation, you suffer from the same problem as most here do. You have too much email stored on the server. Can you give me a rundown of the folders that can be eliminated in your Inbox, we can archive them off then delete them from your folders that are online, this will help a great deal. Thank you, Sean Leinart Network Systems Engineer First Service Carolina Inc. Raleigh, North Carolina United States slein...@fscarolina.com 919-832-5553 DOH!! List, please disregard the erroneous CC: post to the list. I had to look twice since it was the identical problem to what we deal with every week around here. Ted
Re: Email / Inbox Speed Problems
Geez, unless your users are into the millions of messages, maybe you need a more scalable mail server. My day job is support of the Sun comms suite. I only get these when there are litterally tens of millions of messages in an inbox. jay Ted Mittelstaedt wrote: Sean Leinart wrote: -Original Message- From: Sean Leinart [mailto:slein...@fscarolina.com] Sent: Friday, October 23, 2009 2:04 PM To: TJ Russ Cc: allison.ays...@lonesource.com; Spamassassin Mailing List Subject: Email / Inbox Speed Problems Hi TJ, Looking over your Inbox situation, you suffer from the same problem as most here do. You have too much email stored on the server. Can you give me a rundown of the folders that can be eliminated in your Inbox, we can archive them off then delete them from your folders that are online, this will help a great deal. Thank you, Sean Leinart Network Systems Engineer First Service Carolina Inc. Raleigh, North Carolina United States slein...@fscarolina.com 919-832-5553 DOH!! List, please disregard the erroneous CC: post to the list. I had to look twice since it was the identical problem to what we deal with every week around here. Ted
Improving a spam report?
Greetings, Below I have offered the content of my spam score report generated by Spam Assassin. We are Kintera subscribers. Problem is I don't know how to make changes in the text that will result in a better score. Could you send me someplace to learn what different scores mean and how to make them better. I will be happy if I can get below 2.0. For example, how do I get the body of the text out of the objectionable HTML format? Thanks, Jay >From my Report: ...Your spam score is: 2.4 points Score Details: pts rule name description -- -- 0.2 HTML_MESSAGE BODY: HTML included in message 0.3 HTML_FONT_BIG BODY: HTML has a big font 0.1 HTML_FONTCOLOR_UNSAFE BODY: HTML font color not in safe 6x6x6 palette 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts 0.5 HTML_TITLE_UNTITLED BODY: HTML title contains "Untitled" 0.7 HTML_50_60 BODY: Message is 50% to 60% HTML The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor. Jay Langley, International Leadership Institute Director of Web Services Office 770 832 1244 Ext. 19 Training Leaders - Changing Lives Daily Embracing ILI's Eight Core Values <http://www.iliteam.org/site/c.deIILQOpGlF/b.3226917/k.97B7/Core_Values. htm>
Re: Just a general question
At home. 1 domain, 5 users. At work? I do tech support for Sun mail servers. . . . . . . jay John Rudd wrote: Jonathan M Metts wrote: Count me in. 1 domain, 1 user. Why? Just because I can. Evan Platt wrote: At 01:06 PM 3/23/2007, Gary V wrote: I've been on this mail list only for a few months now, and am wondering if I am the smallest guy here. No, you're not. Oh me me me! 1 domain, 1 user. :) At home: 1 domain, 2 users At work: 3 domains, 25,000 users
Re: SPF is hopelessly broken and must die!
Marc Perkel wrote: Justin Mason wrote: Marc Perkel writes: [EMAIL PROTECTED] wrote: Sounds good, I found this an interesting read about why SPF is ineffective: http://en.hakin9.org/products/articleInfo/102 Excellent article. SPF catches no spam - but does create false positives. It's less than useless. It's dangerous. Marc -- Please pay attention to what Matt wrote yesterday. Repeat: SPF is *NOT* for catching spam. It works great at what we use it for in SpamAssassin -- as an authentication mechanism, to detect legit ham and whitelist it. This is what you use authentication mechanisms for: similarly, DK, DKIM, and many other proposed standards are for authentication, not for reputation. It *does* work well for that, in our experience. If you want to rail against SPF as a bad anti-spam technology, perhaps a personal blog would be a more appropriate venue? --j. Two things Jason, First - I agree with you that SPF is totally useless at detecting spam. I would say it is also useless at detecting ham. Second - tell it to everyone here who is suggesting that SPF is a spam solution of some sort. SPF really has no useful function at all. Preventing Joe Jobs. Past that, you're right. However, that's a very useful function in and of itself. If you don't like it, don't use it, but for god's sake please take your zealotry elsewhere. You'd fit right in over in nanae. -- Jay Chandler Network Administrator, Chapman University 714.628.7249 / [EMAIL PROTECTED] Today's Excuse: The Dilithium Crystals need to be rotated.
Re: Simple mail from Dynamic IP listed as spam
Martin von Gagern wrote: Jason Little wrote: Just fire it through your ISP's mail server. They should be able to relay for you without a problem. Jason Why would things look any different if I use my ISP instead of my IMP (Mail Provider)? I'll check this week, but I'm not conviced yet. And how about those ideas about restricting the IP where a mail from a certain domain comes from? That was the main reason why I chose to relay to my IMP instead of my ISP, because they can authenticate me as the owner of that address, which my ISP knows nothing about. It's not as if I were trying to reach the final destination in all cases, I'm relaying to the MSA responsible for my sender's domain. Only in my attached example of a test mail to myself that is the same host, as it would be for all recipients using the same mail provider. Martin Not entirely sure what you're saying in the last two paragraphs, but it looks different from the receiving MTA. It's thought process, approximately: Who's trying to connect to me? It's a static IP with correct DNS reversals? Okay, works for me. Versus: Who's trying to connect to me? A dynamic IP address? Looks spammy to me. REJECT! -- -- Jay Chandler Network Administrator, Chapman University 714.628.7249 / [EMAIL PROTECTED] Today's Excuse: sounds like a Windows problem, try calling Microsoft support
Re: Breaking up the Bot army - we need a plan
John Rudd wrote: John D. Hardin wrote: This doesn't mean SPF is crap. As SPF currently exists, it is crap. Let's not forget that the primary purpose of SPF was/is to cut down on spammers forging legitimate domains. In that, it's been less craptacular than some approaches. No one solution is going to be the silver bullet against the spam problem. -- Jay Chandler Network Administrator, Chapman University 714.628.7249 / [EMAIL PROTECTED] Today's Excuse: The mainframe needs to rest. It's getting old, you know.
Re: Braindeath in the Navy
It never fails to amaze me now many mail server admins ask for ways to break the RFC's in the interest of "security". I do tech support on mail servers, and get requests to configure out server for this kind of thing weekly. . . jay Philip Prindeville wrote: Well, I tried to contact some people responsible for the servers below that what they were doing was broken, including citing chapter and verse where in RFC-2822 in syntax of the Received: lines was spec'd out: Received: from Gate2-sandiego.nmci.navy.mil (gate2-sandiego.nmci.navy.mil [138.163.0.42]) by mail.redfish-solutions.com (8.13.8/8.13.7) with ESMTP id kAGNLZHp020689 for <[EMAIL PROTECTED]>; Thu, 16 Nov 2006 16:21:40 -0700 Received: from nawesdnims03.nmci.navy.mil by Gate2-sandiego.nmci.navy.mil via smtpd (for mail.redfish-solutions.com [71.36.29.88]) with ESMTP; Thu, 16 Nov 2006 23:21:40 + Received: (private information removed) Received: (private information removed) Received: (private information removed) Received: (private information removed) Received: (private information removed) and which fields it requires (like the semi-colon followed by the timestamp coming after a comment field) [cf: RFC 2822, section 3.6.7: received= "Received:" name-val-list ";" date-time CRLF name-val-list = [CFWS <http://tools.ietf.org/html/rfc2822#ref-CFWS>] [name-val-pair *(CFWS name-val-pair)] including the definition of CFWS in 3.2.3.] It just boggles my mind why anyone would go through that much trouble to deliberately damage a header line, rather than just delete it. Well, maybe they'll get a whiff of the errs of their ways in the Hall of Spam Shame... -Philip
Re: [OT] Filter Server Specs
Clifton Royston wrote: On Fri, Oct 27, 2006 at 02:42:49PM +, Duane Hill wrote: Currently, we are looking to install a server that will be doing content filtering for our main e-mail server. I thought I would toss this out to everyone to get some feedback on if the server would be adequate. The server is a Dell PowerEdge 6850 with the following: - Four 2.6 GHz/800Mhz/4mb Cache Dual-Core Intel Zeon 7110M processors - Eight GB DDR2 400Mhz ram - Four 300GB, 3Gbps, SAS, 10K RPM Hard Drives running Raid-5 on a PERC5/i controller Our main e-mail server services over 500 domains with an account total of around 40,000. The current filter server we have can not do any content filtering outside of itself (i.e. the MTA) because of CPU load (i.e. SpamAssassin). Any message scanning where the message size is over 1.5K will kill the CPU. The current filter server we have in place is rejecting an average 2.4 million per day with just the common blacklisting and some other things that are set in place. I *think* this should handle your load. Personally from my years of ISP experience, I'd strongly favor going the road of multiple identical servers in parallel rather than putting all your eggs in one basket. E.g. use two 4 CPU servers rather than one 8 CPU (4x dualcore) server. The difference is that if it comes up just short, or if load jumps up again, it's easier to add a 3rd server and cut it into the mail path than to upgrade a server which is handling all your filtering. You also don't need fast hard drives on a filtering server; it's almost all gonna be pushing the CPU and RAM. Totally agreed! I support mail servers for a living. . . . The other thing I would like to know is what kind of an operating system would one install on this new server? This'll get you into a religious war for sure... I would favor FreeBSD latest (6.x), but any version of Linux with a good package system and a recent 2.6 kernel is a good choice - maybe better than FreeBSD at using 8 CPUs. Reasonable possibilities include CentOS, Gentoo, Debian. I'm not a big Linux head, others may have stronger opinions on that front. Have a look at Solaris 10. It's free, and very well tested. SA runs very, very will on it. It handles multi cpu well, and gets patched well. jay plesset sr. support engineer, sun microsystems. -- Clifton
Re: I'm thinking about suing Microsoft
You have to explicitly choose that option. Are you suggesting we shouldn't be able to choose that? I'm not a big fan of trusting MS patches, as they tend to break things periodically...On Oct 27, 2006, at 8:47 AM, Michael Beckmann wrote:I think there is a problem where a version of XP downloads the security patches automatically, but does not install them. This does not lead to increased security, because most users are gnorant of security patches and would never install them manually.Michael--On Montag, 23. Oktober 2006 16:46 -0400 "Rose, Bobby" <[EMAIL PROTECTED]> wrote: But windows patches are free. Even if you are using an illegal copy ofwindows, you can still manually download and install the patches. It'sMicrosoft Update where they mostly have the genuine windows verificationcode. Even Redhat forces you to pay subscriptions for their autoupdatemanagement stuff.-Original Message-From: Marc Perkel [mailto:[EMAIL PROTECTED]]Sent: Monday, October 23, 2006 3:59 PMTo: JoCc: Duane Hill; users@spamassassin.apache.orgSubject: Re: I'm thinking about suing MicrosoftPopularity is a factor. But the real vulnerability is that Windows canbe more secure if it has the patches. If Linux for example restrictedit's seurity patches to only licensed users they would have the sameproblem. I'm not saying either that MS should be compelled to distributeany upgrades for free. Just secutiry fixes. -- Jay ChandlerNetwork Administrator, Chapman University714-628-7249 / [EMAIL PROTECTED]"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter Da Silva in a.s.r.
per-user whitelists under MailScanner?
If this is the wrong place to ask this, I apologize in advance.Right now, I'm running an older version of SpamAssassin, with user_prefs in each user's .spamassassin folder.Is there any way to migrate this to MailScanner and still use per-user whitelisting (and ideally other settings), or do I have to run SA as a separate program? -- Jay ChandlerNetwork Administrator, Chapman University714-628-7249 / [EMAIL PROTECTED]"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter Da Silva in a.s.r.
RE: How to disable autolearn for FuzzyOcr?
-Original Message- From: Giampaolo Tomassoni [mailto:[EMAIL PROTECTED] Sent: Monday, October 16, 2006 5:26 AM To: users@spamassassin.apache.org Subject: R: How to disable autolearn for FuzzyOcr? >> My apologies if this question has already been discussed here. I have a >> feeling it was but I could not find anything in archives. >> Question: >> Is there a way to disable autolearn if the spam triggers FUZZY_OCR? >> These spams usually contain lots of legitimately looking text and I worry >> about the possibility of Bayes poisoning. >As far as I know, FuzzyOcr doesn't use bayes: it relies on its own database >to store image hashes. >Giampaolo I think what the original poster was asking was how to make the gibberish bodies not get Bayes scanned, so as to not pollute the database with text that isn't spammy. -- Jay Chandler Network Administrator, Chapman University 714.628.7249 / [EMAIL PROTECTED] Ethernet, n. What one uses to catch the Etherbunny.
Re: Ideas
On Oct 10, 2006, at 4:53 PM, Clifton Royston wrote:On Tue, Oct 10, 2006 at 04:31:54PM -0400, Robert Swan wrote: OMG, listen. We setup regular mail server for companies (mostly exchange servers). Once we setup the mail server I want to send an e-mail from that new mail server to [1][EMAIL PROTECTED]. I want that email run through all the Spamassasin tests then sent back to me with all the rules that were triggered etc in the body.. this domain and SPAM server would be used only for this purpose. So it could not be used as a relay or anything like that... Yes, but replying to sender is a terrible idea. Tremendous amountsof spam get sent to random addresses with a real person's addressforged into the header; with your planned setup, spam from thoseaddresses to your server would get mailed back to these innocentparties. To give you an idea, I had to permanently cancel some of the contactaddresses at my wife's professional organization because they had beenforged in spam runs over a period of weeks; her mailbox was gettinganywhere from dozens to hundreds of bounces from a single forgedcontact address. The idea of being able to get back a scored copy of a mail is fine inprinciple, but you need to work out something where it forwards it to afixed address at your server or something of the kind. That way if itgets spammed, it harms nobody but your server. -- CliftonQuite. I've blacklisted addresses that bounce improperly addressed spam to me. Doing this intentionally is a horrible idea.-- Jay ChandlerNetwork Administrator, Chapman University714-628-7249 / [EMAIL PROTECTED]"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter Da Silva in a.s.r.
Re: Only Local Mail
Jess Mooers wrote: I have 2 email addresses that I would like only local domains to be able to send messages to. Is there a way to do this with SpamAssassin 3.1.1? SA is really the wrong tool for this, you should look at setting up a filter via your Mail Server... Jay begin:vcard fn:Jay Lee n:Lee;Jay org:Philadelphia Biblical University;Information Technology Dept. email;internet:[EMAIL PROTECTED] title:Network / Systems Administrator x-mozilla-html:TRUE version:2.1 end:vcard
Re: The Future of Email is SQL
"fast enough" is a value judgement. Fast enough may be ok, if you have a few hundred or even a few thousand users, saving small mailboxes. In a large scale system, where you have a million users, each of which has thousands of messages, I doubt any current database, SQL or other will have that kind of performance. I regularly use a mail server capable of handling that kind of load. It's free, and will eventually be open sourced. Sun Java System Messaging Server. Runs on Solaris, Soaris X86, Linux. Uses individual files for each message. jay plesset sr. tech support engineer. Sun Microsystem. Marc Perkel wrote: After considerable experimenting and thinking things through I thought I'd start a thread on the future of email to start planting the seeds of where MTA development needs to go. I'm convinced that someday soon we will all realize that MBOX and MAILDIR are obsolete technologies and that the future is going to be SQL based storage. First - before everyone starts screaming about speed comparisons, I'm not going to go there. Every storage technology has it's advantages and disadvantages but I'm just going to say that SQL based mail storage is fast enough. The advantages of SQL has to do with power and not with speed. Those who would choose it would do so because they want to do new things that you can do with a database and can't do without one. SQL has several advantages. You don't have t deal with the quirks of the underlying file system or OS. It takes care of all the locking issues and indexing and makes it so that multiple applications can seamlessly access the data. With an SQL backend email can be stored from the MTA, read from and IMAP client that accesses the same database, and the spam filtering engine will have access to the stored email as well. To give you some examples of what could be done . Suppose a spammer sends 1000 phishing spams to your users and then you figure out that the 1000 spams already delivered is spam. With a database you can do a query to retroactively delete spam that was already delivered to the mailboxes. This could also be used to retroactively delete viruses already delivered. Spam filtering programs can lookup existing email in existing folders and compare it with new email already deliverd to help determine more accurately if a message is spam or not. For example, if the host server has a reputation for 100% ham then it can deliver new email without running it through Spam Assassin. If programs like Spamassassin can access existing email in existing folders it can evaluate new email using tricks no one has yet considered. SQL databases allow for multiple masters and slaves and replication that lets you create a cluster that never fails under any conditions. It would be far easier to create a system that is always on and always backed up. An SQL backend allows you to use a wide variety of tools, programming languages, operating systems in order for you to easily integrate more easily than non database systems. And - this is important - once you have a database then new things that no one has yet thought of will be possible and new things we've never heard of will be developed because the new power will lend to the development of more tricks than you can do without database power. My point here is - think outside the box. I'm going to be lobbying IMAP server developers to include SQL backends. exim could pipe data into a local delivery agent, or it can have features written to write directly to the SQL backend. Thoughts . ? -- ## List details at http://www.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://www.exim.org/eximwiki/
Re: Latest sa-stats from last week
Bowie Bailey wrote: Michael Monnerie wrote: On Mittwoch, 10. Mai 2006 17:27 Bowie Bailey wrote: So you are saying that I should not feed Bayes with the unsolicited marketing garbage that I get because it looks like something that could have been requested? If it's a newsletter from a seemingly legit company I don't feed it to bayes. I try to unsubscribe from them. If they still send me, I write some rule to filter them. If some customer then rants, I tell them that said company doesn't work nicely - and he should make a filter to get e-mail from that company out of the SPAM folder again. If it comes to an account that does not subscribe to newsletters (webmaster, sales, etc), it is spam by definition and is fed to Bayes. Remember: 10 good SPAM and HAM are better than 200 where 5% are wrong. Wrong for who? If it looks like marketing, 99% of the time, I don't want it. And for most of the accounts that I deal with, this goes up to 100%. Not true for my customers, tho. Yes, some manual filters can catch those. If it's stupid SPAM, then bayes. My philosophy with Bayes has always been to skip the ham/spam definitions and go with a wanted/unwanted model. This way Bayes learns to filter out the emails you don't want even if some of them may technically be ham. (Obviously, I would not be able to do this on a site-wide installation) But as you said your bayes is not quite accurate, so it seems not to work really. Wouldn't it be better to have a highly accurate bayes, and setup some filters for you personally? If a BAYES_99 would be always SPAM for you, you could give it 4.5 or 5 points, and probably filter more SPAM than now? If I look at my personal database, the spam percentage shown in the stats is lower than I'd like, but I wouldn't say it's not accurate. I very rarely see a true false positive or negative with Bayes and I watch my account closely. I do see a few ham with BAYES_99 and spam with BAYES_00, but that's usually simply because those were either spam that only hit BAYES_99 or ham (usually from this list) that tripped a few extra rules. But then again, I think less than half of my users are even taking advantage of the spam markup. Since I don't do any blocking or sorting on the server, it is up to them to use MUA rules to sort or delete the spam once my server has marked it. I do the same, just wrote a nice document for Outlook 2003 describing how to filter SPAM. I've done the same for both Outlook Express and Thunderbird. The Thunderbird setup is a single checkbox. :) It would be nice if updates.spamassassin.org wasn't using mirrors on non-standard ports, sa-update is trying to use http://buildbot.spamassassin.org.nyud.net:8090/updatestage/ which means I'd have to open a port on my firewall just to get updates, sigh... Jay
Re: OT: anyone know how to do server-side MS-Exchange filters?
John D. Hardin wrote: On Thu, 11 May 2006, Jason Haar wrote: Has anyone done this, and if so, what sort of tools allow it? A Linux mail relay in front of the Exchange server. :) That wouldn't allow messages to be put in a subfolder instead of inbox, just to do the header tagging. Not having used Exchange I can't answer intellegently on whether or not it supports server side sorting. However, if it doesn't you could use something like Maia Mailguard and a Postfix frontend to the exchange server to quarantine and report the spam, users would be able to configure and safely view and "free" tagged spam messages via a web interface. It also can send regular reports to the users on what spam they've gotten, senders and subject, etc. Website is: http://www.renaissoft.com/maia/
Re: spam getting autolearn=ham problem
The message you sent directly to me hit the following: * 0.5 HTML_40_50 BODY: Message is 40% to 50% HTML * 0.1 HTML_MESSAGE BODY: HTML included in message * 1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level * above 50% * [cf: 100] * 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/) * 3.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50% * [cf: 100] * 10 URIBL_SBL Contains an URL listed in the SBL blocklist * [URIs: missusoandforever.org] * 4.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist * [URIs: missusoandforever.org] Of course, the scores are heavily inflated by my own personal rules (I don't recommend doing this unless you know what you're doing) but the point is, your SA doesn't seem to be firing on certain things it should, do you have the DNS BL's working? Are you using Razor or DCC? Are you on the latest 3.1.1? Jay
Re: spam getting autolearn=ham problem
Bazooka Joe wrote: X-Spam-Status: No, score=1.0 required=3.0 tests=BAYES_60 autolearn=ham version=3.0.4 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on agwebinc.com I have required of 3 which you can see and i have the milter rejecting email w/ score more than 7 On 5/10/06, Matt Kettler <[EMAIL PROTECTED]> wrote: Bazooka Joe wrote: > more and more i am seeing spam marked as autolearn=ham > > I was wondering the best way to correct this? Depends.. Really you first need to figure out why it this happened before you take any action at all. Can you post a X-Spam-Status header for one of the messages? Have you modified the required_score, or any of the learning thresholds in your config? In general there are only a few rules that can cause a message to be tagged as spam, but do not count toward the computation of score for learning purposes. *_IN_BLACKLIST, AWL, BAYES_*, and GTUBE are the most noteworthy ones. You can set bayes_auto_learn_threshold_nonspam in local.cf to be 0 or a negative number, then autolearn=ham won't kick in unless it's below a certain score (not sure if this counts bayes or not). But yes, the real question is why are no rules triggering... Is DNS working? Are you using the blacklist rules, etc? What does the spam look like? Jay
Re: My only problem with URIBL_BLACK
wrote: | But. | | There are some spammers who run "subscribe to" mailing lists. | | I got spam at home the other day from ediets.co.uk, for example. | | I call this stuff "subscription spam" and would block most of it anyway. | | Cheers, | | Phil Easier said than done when you have a paying customer who wants this specific mailing. Have you tried lowering the score of the spamassassin rules that are getting hit? Jay
Re: Which Operating Systems Do You Use and Why?
Interesting answers. I'm using Solaris 10/X86. Sun Java Enterprise Messaging Server. Integration is built in. easy to set up. Dead stable, but,then I work for Sun. jay Bowie Bailey wrote: Ask List wrote: We can not seem to come to an agreement on the best operating system to run spam assassin. So we have decided to post this question to the mailing list so we can have other opinions. I realize everyone will have a different opinion on the subject and some will have none at all, linux is linux and unix is unix. So I would like to hear users experiences using different operating systems. Pros/Cons/Problems/Headaches/etc. The operating systems I'm most interested in are Debian, Ubuntu, Gentoo, Slackware, FreeBSDs, and OpenSolaris. Hopefully this doesn't start a flame-war, but it is likely to become a large thread in any case. Ah well... here we go! :) I have been using RedHat and Fedora, but am now in the process of transferring my servers over to CentOS. It is a direct rebuild of RedHat Enterprise Linux, so it has stability and a slower upgrade cycle which is very nice for a server. I have run Courier-MTA, Apache, Bind, SpamAssassin, ClamAV, Samba, etc and it has been very easy to deal with and extremely stable.
HREF based rule idea...
Has any thought been given to creating a rule that looks for "forged" links? Here's one I got today in a phishing scam: http://www.createtokill-clan.de/onlineshop/catalog/images/admin/chase.com/index.htm";> http://www.chase.com/verification.asp So how hard would it be to create a rule that triggers if the href (http://www.createtokill-clan.de...) doesn't match the url that is displayed (http://www.chase.com...) or at least contain the same domain? I realize this is mostly done with phishing scams but it's not unheard of for spammers to use this technique too. I've not seen a SA rule that triggers on this specifically. Any thoughts? Jay begin:vcard fn:Jay Lee n:Lee;Jay org:Philadelphia Biblical University;Information Technology Department email;internet:[EMAIL PROTECTED] title:Network / Systems Administrator version:2.1 end:vcard
Re: Sorta OT - was: RE: Out of Office AutoReply
No decent MTA should be returning OOO messages to a mailing list. Any such should be considered buggy, and fixed. I know that the MTA I use (Sun JES Messaginag Server) doesn't return OOO messages to a group. It only returns OOO messages when the addressed "to" matches the entry in the user's mail or mailAlternateAddress. jay Loren Wilton wrote: Differentiating between personal accounts and company email systems, how do you all classify OOO messages? Personally if they are a reply to a mailing list I consider them spam, but generally not a spam that should be reported, merely one that should be quietly dropped. (There are exceptions.) Why do I consider them not reportable? Because: a) It is reasonable in some companies to subscribe from mailing lists at work b) Some companies REQUIRE that you have an OOO message if you are OOO. Some companies set them up automatically, or the person's boss does the day after the user goes on vacation. c) Not all people run Unix mail clients, and thus many either don't know how to do an OOO that will only respond in-company, or that won't respond to mailing list messages d) Most people (sigh) use MS mail "tools" (as I am) and the ffing MS idiots have never even considered the *possibility* that someone might want a different auto-response to a list message than a personal message. Or to a spam. The result is OOO messages, even if the person would like that to not happen. So I have a moderately decent filter rule in OE that catches most of them and quietly deletes them. Seems a reasonable compromise for things that most people really can't control. Now, there are eggregious cases that are reportable. Like the idiots in customer service at some companies that signed the "customer comments" mailbox up to a bunch of mailing lists, so anytime a message is posted the company sends out a "thank you for your inquiry about our wonderful products; someone will get back to you in several days". Or the autoresponders that autorespond to their own OOO messages with another OOO message. Loren
Re: Stopping Rules
Chris L. Franklin said: > Thanks but we do run my servers as I posted above (minus the Non DNS > compliant part). Blacked listed user and Domains my server to not accept > messages from. Whitelisted users and domain DO NOT get passed though SA > WE DO NOT use negitive scoring. > We Stop 99.2% of all spam and get less the %0.82 miss marked emails. > We Subject mark at 5 points, and We report a "550" error" on all emails > with a score of 8 or more during the smtp transaction. (Yes we Do SA > scanning during the smtp transaction. Aka we stop spam at the door.) If you are rejecting mail during the SMTP session than you have no way of verifying you are at %0.82 false positive rate. How do you know I'm not sending you a legit message that's being rejected at the SMTP level unless I bother to contact you via other means? (something few senders bother with) 0.82% seems very high to me also, nearly 1 in 100 message is marked wrong? Maybe your users are more tolerant of false positives and just want all spam blocked but this is not the case for most organizations. Many organizations demand an extremely low to non-existant FP percentage while being more tolerant of the occassional false negative. To each his own I guess, but I agree with the first respondant that your missing out by turning off negative scoring... Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Spamassassin vs spamd
shane mullins wrote: Is anyone here running spamd? We use Spamassassin 3.0.4 and several SARE rules. Now that our primary MX server handles about 20k emails a day, cpu usage stays over 90 % and load average is between 5 and 6. I was wondering how much faster spamd is? Much, much faster. It's really the only option when processing this much mail. Switch and watch your load drop dramatically. Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: score based on MX's IP?
Mike Jackson wrote: Perhaps this is too much to ask of SpamAssassin, but... My server receives a piece of spam that's undeliverable. It looks up the MX for the sender's address, and finds that the IP is 127.0.0.1. It then complains that there's a configuration problem because it's not set up to handle mail for that domain. What I'd like to do is build a SpamAssassin rule that would assign points against messages sent from senders with those 127.0.0.1 MXes. Granted, it won't do any good in these undeliverable/bounce scenarios, but I'm sure there's spams getting through to legit addresses from them as well, and those are what I'd like to put a stop to. Is that possible (without writing a plugin to do it myself)? 1) Why is your MTA accepting mail that is undeliverable? 2) It would be better to block these MXs at the server level, many MTAs are capable of blocking based on the declared helo or dns lookup of the connecting server. Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: SA 304/spamc milter question
Dr Robert Young wrote: We want to do some testing of our email system with, and without, SA intercepting the mails. Currently, we have SA 304 installed and running with sendmail, using the milter-spamc "hook". I just want to verify that if one manually "shuts down" the spamd daemon, that the emails would be eventually "passed" along as "unchecked" email after any appropriate "timeouts" are encountered ? I basically want to avoid having to recompile sendmail to remove the "milter" lines currently in the system. I'm not sure where you question is. To test this out, disable spamd... Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Nigerian scam not catched by 3.10?
Menno van Bennekom wrote: I installed 3.10 on my testserver to compare some scores with my current 3.03 version. I only have the default checks. Some spam was not marked in 3.10 because checks like NIGERIAN_BODY* didn't get off. It seems that everything with 'NIGERIAN' in it is removed from /usr/share/spamassassin/*.cf in version 3.10. Any idea why? These checks were really important to me, I get a lot of Nigerian scams especially via hotmail. They're there, names just changed. Look for ADVANCE_FEE_ rules. These still hit Nigerian style scams for me regular as well as more generic scams. I did bump the scores for these rules up somewhat to help them along... Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Postfix/SA/Exchange 2000 'NDR attack' exploit spam and other bad things
Wow. I knew I didn't like Exchange.. . I run Sun's Messaging Server 6.2. SA integrates right into it, with hooks provided by Sun. Addresses are first verified, even before the sending system gets to the "data" part of the conversation. If the address is bogus, they get a 550 5.1.1 unkown alias rejection, right there. Then the message goes to SA for processing... Sun Java Messaging Server runs on many OS's, and is a free download, to try. They'd like you to pay for a license... jay Greg Allen wrote: I have recently been working on the Exchange 2000 NDR attack issue. For those who are not aware of this issue, I will explain. It seems there is a certain group of desperate idiot spammers that believe that bouncing off good Exchange 2000 servers with non-delivery reports is a good way to deliver spam. They send tons of email at your Exchange 2000 server, with a different reply addresses forged for each email. The spam recipient apparently sees an NDR from your server, with spam attached. Your server did the delivery. (ooops) Moronic idea, must look like hell to the spam recipient, but apparently it is being done out there. There is also apparently little to nothing that can be done for the exchange server. There are a few third party items that I am looking into, but the real fix (supposedly) is to upgrade to Exchange 2003. See here: http://support.microsoft.com/?kbid=886208 The thing that apparently is the tip off for this issue is tons of queued up email to spam domains in your Exchange queues. The difficult part, it that it is hard to tell the difference between NDR attacks on your Exchange server as opposed to some idiot just using your domain for his reply address in a spam run. It has about the same affect as far as I can tell with the queues. Ok, that is the background... Now onto the problem as I see it. Let's say I do the fix with 2003 (which I have already done). So, recipient verification is now enabled on Exchange 2003. One small problem however. If I have SpamAssassin kill emails at lets say...20 points spam score, the email recipient never gets verified on my front end Postfix/SA server. I am receiving all the various bogus email addresses and sending them to the trash can where they belong. What would be better though, is for Postfix/SA to allow recipient verification to Exchange before Postfix/SA starts going to work at all. I would rather not make recipient files on the postfix server. Seems like there should be a better way. It would seem that ideally, the error "User unknown (in reply to RCPT TO command)" (or whatever) should be allowed to happen before SA starts testing the email. I could just let the high score emails go through without killing it, and that would probably work correctly as far as recipient verification goes with the Exchange 2003 server, but I would rather not do that. The legit users would see a flood of more ***spam*** tagged emails than they are used to seeing. So, I guess my question would be, does anyone know of a way to allow a natural recipient validation check downstream to the Exchange 2003 server before SA starts working, so that SA does not start testing on all these bogus email addresses. Again, I am looking for some solution that does not involve creating recipient verification maps on the Postfix server. Thanks in advance for any ideas.
Re: Spamassassin scoring bad after years of service......
Lorin G. Tremblay wrote: Was wondering if anyone had any clue to why spamassassin would start to score spam badly and let almost any spam throught. Tehy was no change in the hardware of software, it just started to score spam really badly, but had workes for atleast a full year without any glitch! Unfortunately, we are not psychic and cannot determine what the problem might be with the amount of information you gave us. What version are you running? Are you using AWL, Whitelisting, Autolearning, SQL Based Prefs? What platform are you on? What type of spam doesn't get caught anymore and what does the SA report say? We need more details to help you. Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Problem loading ClamAV plugin
Usually, I'm pretty good at following instructions. I have done so, far as I can tell. SA works fine. ClamAV works, in that clamd starts, listens on the correct port, and clamdscan works fine. but. . . spamassassin --lint throws this: # /usr/local/bin/spamassassin --lint failed to create instance of plugin ClamAV: Can't locate object method "new" via package "ClamAV" (perhaps you forgot to load "ClamAV"?) at (eval 46) line 1. Failed to run CLAMAV SpamAssassin test, skipping: (Can't locate object method "check_clamav" via package "Mail::SpamAssassin::PerMsgStatus" at /usr/local/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/PerMsgStatus.pm line 2312. ) lint: 1 issues detected. please rerun with debug enabled for more information. # ls What I did: Install current stable version clamav, 0-.86.2. compiles and seems happy. freshclam is happy, too. install File::Scan::ClamAV through cpan copy the files, clamav.cf clamav.pm to the /etc/mail/spamassassin directory, and made them readable by the user that spamassassin is running as. The doc on installing the plugin has nothing beyond this: http://wiki.apache.org/spamassassin/ClamAVPlugin Have I missed something obvious? Googling hasn't helped... thank you! jay plesset mail admin for D. P. Design day job: Tech Support (Messaging Server, Sun Microsystems)
Re: Early Questions
Mark Williams wrote: I have just installed spamassassin v3.0.4 in a test environment (which is a mirror of the live environment) and have a number of questions, which I can not see within the manuals/support documentation. Firstly, this is my configuration: Server: Linux (RH9.0), with spamassassin installed from spamassassin.org web site using "make" etc (not RPM's). This machine then runs both IMAP and POP3 for clients. MTA is sendmail Surely your not going live with a distribution as old and unsupported as RedHat 9! Do you want to become a spam zombie? I urge you strongly to look at moving up to RedHat Enterprise Linux 4, CentOS 4 or a recent Fedora release. Also, you really should stick with the RPMS, it makes management and future upgrades much smoother. Client(s): Windows XP. All running Windows XP and MS Outlook 2000. All users connct to POP3 Server (on Linux machine) and use PST files to download their e-mail(s). General: Setup is such that spamassassin is site wide (not per user) - as per management request. All working fine at the moment - just about to "switch on bayes" Questions: (q1) Given that this is a site-wide installation, how do I get the requisite 200 e-mails (spam/ham) for spamassassin to work with? Where should I put these (an individual mailbox)? Use bayes autolearning so that you don't have to bother to much. Also setup some aliases like [EMAIL PROTECTED] and [EMAIL PROTECTED] where users can forward wrongly classified mail for you to reclassify. Don't try to use someone else's bayes db and don't use just your personal email since it won't match the bayes characteristics of the entire company. Note that you can also modify the number of spam and ham messages the bayes db needs before it starts scoring with these two rules in local.cf: bayes_min_ham_num 100 bayes_min_spam_num 50 be careful about setting it to low though, the less bayes knows about your org's email characteristics the more likely false positives are. Jay
Re: How to shut down
Steven Dickenson wrote: On Jul 12, 2005, at 1:19 PM, Chris Santerre wrote: Thinking of you, Tom Cruise You owe me for the can of soda I just sprayed on my desk. Good times... How to shut down the spamassassin? so it doesnt run ?? What operating system are you running SA on? How is it being called within your mail path? We can't help you if you don't help us. I think you meant "Help me, help you!" Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Distinguishing between mail that is "almost certainly" or "probably" spam
Tim Litwiller wrote: this is the way I've been doing it in procmail - then I don't have to count *'s # --- # Spamassassin - certainly spam # --- :0 H: * ^X-Spam-Status: +(yes|no), +score=\/[^. ]* * ? (( ${MATCH} > 14 )) /dev/null # --- # Spamassassin - probably spam # --- :0 H: * ^X-Spam-Status: Yes.* \;Junk/new I've never understood what procmail users have against human readable code :-) I just use maildrop: if (/^X-Spam-Level: \*\*\*\*\*\*\*\*/:h) { #do something with obvious spam } else { if (/^X-Spam-Level: \*\*\*\*\*/:h) { #do something else with likely spam } } Very simple and easy to understand. My live server actually runs somewhat more complex where the users "obvious spam" score is stored in a database and retreived for comparison at local delivery time rather than being hard coded, but anyway, I get the ability to have multiple spam categories without source code modification to SpamAssassin. Jay hmm after pasting that in I wonder if there is any chance that the catches large No scores also? It's just easier for most filtering languages to look at the stars, that's why there there. The yes/no only gives you a black/white world, the score number is easy for humans to read but hard for programming languages. Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Distinguishing between mail that is "almost certainly" or "probably" spam
Richard Duran wrote: Hello, I'm not sure if this belongs in the dev-list or not, but we have made some minor changes to SA in order for us to allow our users to create separate filters for mail that we consider to be "almost certainly" spam, versus mail that is "probably" spam. Just filter based on X-Spam-Level headers. If 8 is certainly spam then have your server side filter or client filter look for 8 *s, then look for 5 *s for probably spam. Very simple, no code changes needed. Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
ALL_TRUSTED score change
What file do I need to edit to change the score on ALL_TRUSTED? Thank you
Re: Reporting scams to fraudwatchinternational
Kris Deugau said: >> If you use a competent email client you will be offered the option >> of keeping a local copy, which saves the redundant recipient. > > Some people deliberately turn this off. I'm not sure why. (I can > *sort* of understand it for mailing list mail, but not for "direct" > mail.) > >> Further, you should never assume that other recipients do not >> see BCCs. That it entirely up to the settings of the recipient's email >> client. > > If your MUA is actually adding a "real" header with BCC: information, > it's broken. BCC isn't supposed to be a header in the usual sense; it's a > way to tell your mail client to add extra SMTP RCPT TO: commands when > sending the message. The recipients should NEVER see those extra > recipients. > > The only way someone might find out about BCC'ed recipients is if they > are the server admin (or have access to the mail logs) and are willing to > spend the effort to wade through the logs tracking the message ID to see > who got a copy. And that only applies in the case where the sender's SMTP > server is also the destination; and partially applies if there are > multiple recipients at a remote domain. If a remote domain only has one > recipient in the list, they will NOT see any information regarding other > recipients. I've also seen broken mail servers that add headers based on the "rcpt to:" so you should assume that recipients bcc or not on the same remote server may be able to discover each other. But if you're confident your mail server/client isn't doing something stupid then there should be no way for [EMAIL PROTECTED] to discover the message was BCCed to [EMAIL PROTECTED] Jay -- Jay Lee Network / Systems Administrator Information Technology Dept. Philadelphia Biblical University --
Re: Bayes issue
Thank you, even though over 3,000 emails have gone through I only have: debug: bayes: Not available for scanning, only 133 spam(s) in Bayes DB < 200 that Bayes I am not training Bayes manually so Bayes just hasn't collected enough messages to train on. Thanks - Original Message - From: <[EMAIL PROTECTED]> To: Sent: Thursday, April 21, 2005 11:01 AM Subject: RE: Bayes issue Jay Ehrhart wrote: > The Bayes score is not being used in the overall spam score. ... > I did a rm bayes_* and it removed the files. > I have had over 3,000 emails through since I did the rm ... > How do I get it to start using the Bayes score again? http://wiki.apache.org/spamassassin/BayesNotWorking This can be checked by running "spamassassin -D --lint" and keeping an eye out for this line: debug: bayes: Not available for scanning, only 0 spam(s) in Bayes DB < 200 Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer perl -e"map{y/a-z/l-za-k/;print}shift" "Jjhi pcdiwtg Ptga wprztg,"
Bayes issue
The Bayes score is not being used in the overall spam score. My MailScanner/SpamAssassin has been working fine. I wanted to wipe out the Bayes files and have them recreate and learn again. I did a rm bayes_* and it removed the files. I restarted MailScanner and the files were recreated and the time stamp is current and the size is slowly building. I can see in the log file "autolearn=spam" and "BAYES_99 1.89" and other Bayes_ scores in the log. I ran sa-learn -dump -D and spamassassin --lint -D and everything looks fine. I have had over 3,000 emails through since I did the rm on the bayes files. The problem is the Bayes score is not being used in the SpamAssassin scoring. Before the flush I would see the Bayes score first and other the scores under it. How do I get it to start using the Bayes score again? I have retarted MailScanner mulitple times.
Re: Confused about HELO_DYNAMIC_*
Matt Kettler wrote: At 10:43 PM 3/1/2005, Jay Levitt wrote: Why would the HELO_DYNAMIC_* rules trigger on these headers? Surely it's ok to have a dynamic IP as the *source* of a message, just not in a relay..? It looks like it might be a trust path issue.. are the brandeis.edu hosts trusted? If so, SA would be correct in deciding a dynamic node from attbi.com dropped mail off directly. Nope, they're not - I had no trusted_networks or internal_networks defined. What do the *.home.jay.fm hosts resolve as when the machine running SA does a DNS lookup? are they reserved IP's? If so, you'll have trust path issues and need to manualy define trusted_networks. Yep, they're 192.168/16. According to the man page for Mail::SpamAssassin::Conf, that should be automatically trusted due to the DNS checks... is that not correct? I'll try setting it manually. Jay
Confused about HELO_DYNAMIC_*
Why would the HELO_DYNAMIC_* rules trigger on these headers? Surely it's ok to have a dynamic IP as the *source* of a message, just not in a relay..? Return-Path: <[EMAIL PROTECTED]> Received: from server.home.jay.fm ([unix socket]) by linux.home.jay.fm (Cyrus v2.2.8) with LMTPA; Sun, 27 Feb 2005 23:25:34 -0500 X-Sieve: CMU Sieve 2.2 Received: from blanca.unet.brandeis.edu (blanca.unet.brandeis.edu [129.64.99.169]) by server.home.jay.fm (8.13.1/8.13.1) with ESMTP id j1S4PWlk011698 for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:33 -0500 Received: from blanca.unet.brandeis.edu (localhost.localdomain [127.0.0.1]) by blanca.unet.brandeis.edu (8.13.1/8.13.1) with ESMTP id j1S4PUer006126 for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:32 -0500 Received: (from [EMAIL PROTECTED]) by blanca.unet.brandeis.edu (8.13.1/8.13.1/Submit) id j1S4PUhv006125 for [EMAIL PROTECTED]; Sun, 27 Feb 2005 23:25:30 -0500 Received: from h00c04f2d101a.ne.client2.attbi.com (h00c04f2d101a.ne.client2.attbi.com [66.30.139.164]) by webmail.grad.brandeis.edu (IMP) with HTTP for <[EMAIL PROTECTED]>; Sun, 27 Feb 2005 23:25:30 -0500 Message-ID: <[EMAIL PROTECTED]> Date: Sun, 27 Feb 2005 23:25:30 -0500 From: [EMAIL PROTECTED] To: Jay Levitt <[EMAIL PROTECTED]> Subject: Re: Wow, now I really don't know what to say References: <[EMAIL PROTECTED]> In-Reply-To: <[EMAIL PROTECTED]> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.2.6 X-Spam-Score: 5.555 (*) BAYES_00,HELO_DYNAMIC_ATTBI,HELO_DYNAMIC_IPADDR,J_CHICKENPOX_21,NO_REAL_NAME X-Scanned-By: MIMEDefang 2.43
Re: SA 3.01 eventually stops noticing DNSBLs
Jay Levitt wrote: A quick test shows that indeed, an awful lot of domains are repeatedly failing in lookup_ns, but that different domains fail at different times - the domains that repeatedly fail right now were fine last night in the SA logs. So it looks like this is something (intermittment) to do with the resolver on my system, or perhaps the caching nameserver, and nothing to do with SA. I'll keep digging and report back what I find. If anyone has any tips, of course, feel free to let me know. I spoke too soon. Turns out I'd accidentally left "recurse=>0" in the test harness. No wonder it was failing so often. I discovered Net::DNS::Resolver::errorstring, and put some more logging into SA, and the problem is really simple: my caching-only nameserver times out when looking up NS records for a site that's not in the cache. Not entirely surprising, with a 3-second timeout in SA. And my site is infinitely small (just me), so it's going to be fairly common that one of the well-known sites is not in cache. SA realizes this, and tries to loop, in Dns.pm's is_dns_available, but the loop is coded wrong, because either a success or a failure breaks out of the loop! A timeout in lookup_ns will result in $result defined, but containing no records, and that triggers the "failed horribly" clause, setting $IS_DNS_AVAILABLE to zero until mimedefang eventually cycles the child process. I *think* the bug fix is just to remove that whole else clause from is_dns_available, but as a Perl novice I'd certainly like someone to double-check that. And, you know, now that I look at it, it seems like is_dns_available uses lookup_ns to test general DNS availability, but lookup_ns has its own caching that would seem to defeat the point of the test if a site is ever hit twice! Jay
Re: SA 3.01 eventually stops noticing DNSBLs
Jeff Chan wrote: On Wednesday, February 23, 2005, 8:38:31 AM, Jay Levitt wrote: I tried to create a test harness to see if I can replicate this outside of SA, but for some reason, even though I double-checked the code I copied from Dns.pm, I'm getting weird results - it's always giving me the root nameservers, instead of the name servers for each of the domains. This is true with recurse => 0, recurse => 1, or recurse left out entirely as it is in Dns.pm. I'm no Perl whiz; can anyone see my mistake? Off the top of my head, that sounds like a DNS configuration error. Do you have a recent root hints file? That got updated a couple times over the past couple years IIRC. Nope, that's not it - I should clarify that this same code does get the right NS servers when it's running in SA, just not standalone (and I'm using the same login). So I'm doing something wrong Perl-wise; I just don't know what... Jay
Re: SA 3.01 eventually stops noticing DNSBLs
Jeff Chan wrote (quoting Jay Levitt): Nope, that's not it. I've been throwing debug code in bit by bit. (More accurately, I've been re-copying the dbg statements as "warns", because while there's plenty of useful output, there are just too many un-categorized dbg statements to leave debug enabled... sigh.) Looks like every once in a while, the lookup_ns sanity-checks that SA does on well-known domains are returning with zero NS records. Still not sure why that happens yet, or exactly what is going on, but that does understandably lead SA to disable DNSBL processing for a while. Hmm, that sounds like something that may deserve a bugzilla. Can anyone else replicate that behavior? Is your Net::DNS completely current and happy? Yep, 0.48. Have you checked all of your: /etc/resolv.conf $HOME/.resolv.conf ./.resolv.conf for the user mimedefang or SA runs as to make sure they're all correct and all the name servers on them resolve the RBLs correctly? Yep. The only *resolv.conf file on the system is /etc/resolv.conf. Also when you say "At some point, SA seems to stop doing lookups on the DNSBLs" what is the time scale? Does "At some point" mean at some times of day, after several months of operation and all the time now, for a few hours at a time, for every 6th message, etc.? After it's been running for a few hours, the lookup_ns check (which does a sanity check to make sure we can resolve the nameservers of a well-known domain) seems to fail. Or, rather, it returns, but with 0 entries in the array. This causes SA to stop doing any RBL lookups for some period of time. I tried to create a test harness to see if I can replicate this outside of SA, but for some reason, even though I double-checked the code I copied from Dns.pm, I'm getting weird results - it's always giving me the root nameservers, instead of the name servers for each of the domains. This is true with recurse => 0, recurse => 1, or recurse left out entirely as it is in Dns.pm. I'm no Perl whiz; can anyone see my mistake? Code follows: - #!/usr/bin/perl no strict; no warnings; require Net::DNS; require Net::DNS::Resolver; use strict; use warnings; my @EXISTING_DOMAINS = qw{ adelphia.net akamai.com apache.org cingular.com colorado.edu comcast.net doubleclick.com ebay.com gmx.net google.com intel.com kernel.org linux.org mit.edu motorola.com msn.com sourceforge.net sun.com w3.org yahoo.com }; my $res = Net::DNS::Resolver->new ( recurse => 0, retry => 1, retrans => 0, dnsrch => 0, defnames => 0, tcp_timeout => 3, udp_timeout => 3, persistent_tcp => 1, persistent_udp => 1 ); die unless defined $res; for(;;) { my @domains = @EXISTING_DOMAINS; my $domain = splice(@domains, rand(@domains), 1); print "trying '$domain'...\n"; lookup_ns($domain); } sub lookup_ns { my ($self, $dom) = @_; my $query = $res->search($dom, 'NS'); if ($query) { foreach my $rr ($query->answer) { print "type=", $rr->type, ", nsdname=", $rr->nsdname, "\n"; } } else { print "ERROR! no query\n"; } } 1;
Re: SA 3.01 eventually stops noticing DNSBLs
Kelson wrote: Jay Levitt wrote: I have SA 3.01 running under mimedefang 2.43 with sendmail 8.13.1. At some point, SA seems to stop doing lookups on the DNSBLs; spam gets through that is listed in multiple BLs; if I check manually with spamassassin -t, it detects the BL entry, even if I run it moments after the spam was received. I don't see anything obvious in the logs. What can I do to troubleshoot this? Make sure MIMEDefang hasn't created a new /etc/mail/sa-mimedefang.cf on an upgrade. That happened to my server a while back -- We were just using /etc/mail/spamassassin/local.cf, and upgraded MD, and MD saw there was no sa-mimedefang.cf, so it created it with the defaults -- and the defaults disable DNSBLs. Nope, that's not it. I've been throwing debug code in bit by bit. (More accurately, I've been re-copying the dbg statements as "warns", because while there's plenty of useful output, there are just too many un-categorized dbg statements to leave debug enabled... sigh.) Looks like every once in a while, the lookup_ns sanity-checks that SA does on well-known domains are returning with zero NS records. Still not sure why that happens yet, or exactly what is going on, but that does understandably lead SA to disable DNSBL processing for a while. Jay
Re: SA 3.01 eventually stops noticing DNSBLs
Jeff Chan wrote: On Friday, February 18, 2005, 8:35:35 PM, Jay Levitt wrote: I have SA 3.01 running under mimedefang 2.43 with sendmail 8.13.1. At some point, SA seems to stop doing lookups on the DNSBLs; spam gets through that is listed in multiple BLs; if I check manually with spamassassin -t, it detects the BL entry, even if I run it moments after the spam was received. One thing to check is whether your name resolution is truly correct. Are you running Net::DNS 0.48? I was running 0.47; just upgraded to 0.48. Was there some known bug in 0.47 that could cause this? The Changes for 0.48 don't mention anything that looks relevant. Is it installed and upgraded in a consistent way (i.e. always rpms or always CPAN or always tarballs)? Using different upgrade methods can confuse things. Always CPAN. Did you see the recent thread about the various resolve.conf's used by Net::DNS? Are they all correct for the user SpamAssassin runs as? Just checked.. there is only one resolv.conf on the system, in /etc/resolv.conf, and it correctly points to my own machine, which runs a caching named (actually caching for the world, authoritative for my own domain). It's important to note that the DNS lookups DO work for a while after starting mimedefang; it's just at some point after days/weeks that it stops trying (or stops succeeding). Any tips as to where I could put debugging code? Should SA already be writing something to a log file? Jeff C.
SA 3.01 eventually stops noticing DNSBLs
I have SA 3.01 running under mimedefang 2.43 with sendmail 8.13.1. At some point, SA seems to stop doing lookups on the DNSBLs; spam gets through that is listed in multiple BLs; if I check manually with spamassassin -t, it detects the BL entry, even if I run it moments after the spam was received. I don't see anything obvious in the logs. What can I do to troubleshoot this? Jay Levitt
Re: OT Boincing Spam
Timeout should not be a problem. My SA seems to take 3 to 6 seconds to scan a message. SMTP timeout should be 10 minutes, for any server that's compliant with rfc. jay John Andersen wrote: On Friday 24 December 2004 06:59 pm, [EMAIL PROTECTED] wrote: Recently, I have set up my account to reject with a 554 SMTP error code anything that spamassassin flags as spam, using the default threshold of 5.0, >From your web page: "Bodytest" support - allows you to run filters like spamassassin and clamscan on the body of a mail message before replying to the final "." of the SMTP DATA command. (See the edinplace(1) man page and the bodytest description in the avenger(1) man page.) This would imply that you hold the connection open from the sender till SA has had a look at the mail, (which may entail several network based hits in the process of checking surbl etc). Does this not entail some rather large number of open connections on the mail server, some of which might time out when SA is working hard? Also does avenger sit ahead of sendmail or is it called by sendmail? (Who is listening on 25? Avenger or sendmail/qumail?
Re: spamd dns problems
Does your local server also do reverse lookups? Jon Dossey wrote: As per Matthew Romanek's ([EMAIL PROTECTED]) recommendations, I re-pointed my resolver to a different nameserver (from resolving locally), and can successfully scan a message in a little under 2.5 seconds (2.3 - 2.4 seconds). I already upgraded to perl 5.8.5 and Net::DNS 0.48, which didn't resolve the problem. Does anyone have any idea why it fails when attempting to resolve off the local nameserver? The resolver works perfectly otherwise. Any input appreciated. Thanks, .jon __ "The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from all computers."
Re: Brightmail
Richard, my day job is tech support for Sun mail systems. I support the integration with both SpamAssassin and Brightmail. Both do a very good job. Brightmail is commercial software, and is sold with a contract that automatically updates it, often. Many customers are more comfortable with this approach than they are with open source software, like SA. Brighmail is now owned by the Symantic folk, and also can be purchased with full integration with their virus scanning package. Personally, I use SA on my system, for my wife's company. I had some difficulty getting everything installed, compiled, and integrated, but once it's in, it works very, very well, here. Brightmail indeed seems to live up to their claims for effectiveness and performance. SA may be somewhat lower in performance, but I can't claim to have benchmarked it. Since SA depends on outside resources for some tests, it must be slower at least at times, while Brightmail simply updates an internal database to refer to. jay Gray, Richard wrote: Brightmail seems to be getting a lot of good press on the SPAM front. So I'm wondering, why do people running large mail systems choose SA over corporate offerings. Is it cost? Is it configurability, or performance? Can anyone shed any light on how Brightmail achieves the rather impressive statistics it is quoting, or do you think it is just smoke and mirrors? Is it possible to reproduce the other features without spending the cash? --- This email from dns has been validated by dnsMSS Managed Email Security and is free from all known viruses. For further information contact [EMAIL PROTECTED]
Re: New Hardware
You might also look at Solaris X86. I've just brought up such a box, and am impressed with the performance relative to Linux on the same box. jay Jeff Chan wrote: On Tuesday, November 30, 2004, 4:28:35 AM, Ronan Ronan wrote: Hey list, I am in the quite sureal situation of being given a blank cheque by my boss to buy 2 new servers for SA. They were so impressed with the upgrade to v3 + SURIBLS et al that when i said that our current setup was hitting load max they found some cash for me... :D We are in a university environment with over 100,000 mails daily. What Im currently looking at is either 2 Sun v150s or 2 dual-opterons probably with a gig each, and the standard 80+gigs. Which one will be better suited to SA? I know SA is more cpu/ram than disk IO so im leaning more toward the AMD approach. The reason there are 2 machines of each is because im gonna implement fail over using heartbeat. Does it make a difference the Solaris / Linux route? Will SA benefit from the dual processor option? Any other factors I should consider? In general, I'd recommend Linux on AMD. Unix type operating systems often benefit from multiprocessing, especially recent Linux/BSD/etc kernels that have deeper support for multiple processors built in. I'm sure other folks have some more ideas. BTW were you able to get your local mirroring of the SURBL zones working well? Jeff C.
Per user blacklist
I have setup SpamAssassin 2.64, and qmail-scanner 1.23 on FreeBSD with perl-5.8.4 and have been using them separately with great success. However, I have decided to use qmail-scanners ability to run SpamAssassin as the mail is processed. And, I have this working to a point (i.e. the mail is flagged correctly accoring to rules), but I cannot get the blacklist to be recongized. Spamd is running as root. Spamc is called from the qmail-scanner-queue.pl script with -u qscand. qscand is the user whose rules I would like to have used. In debug, I see the following logmsg: handle_user: unable to find user '[EMAIL PROTECTED]'! logmsg: Still running as root: user not specified with -u, not found, or set to root. Fall back to nobody. I understand not being able to find the user [EMAIL PROTECTED] since this server is simply a relay for scanning, etc. before the e-mail is delivered to the Exchange server. My thinking was that with the -u option whenever a user is not found, the rules in the /home/qscand/.spamassassin directory would be used. The other thing that is puzzling is that I have added, for testing purposes, my e-mail address to the blacklist for nobody, and once the e-mail is received, I am not being identified as a blacklisted sender. If I add the blacklist entry to local.cf, I am properly identified as a blacklisted send. How do I force SpamAssassin to use a particular user's rules, as opposed to nobody, when the user is not found? Any ideas what I might be doing wrong with the blacklists? Thanks for all your help. Jay
Re: Rule problem (.exe attachments)
[EMAIL PROTECTED] wrote: Jay Hall wrote: I am experiencing a problem with one of my rules that I cannot seem to find. I have the following rules defined. rawbody __RAW_EXE_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_VBS_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_COM_ATTACHMENT/filename=\".*\.com\"/i rawbody __RAW_PIF_ATTACHMENT/filename=\".*\.pif\"/i rawbody __RAW_CMD_ATTACHMENT/filename=\".*\.cmd\"/i rawbody __RAW_BAT_ATTACHMENT/filename=\".*\.bat\"/i meta ATTACHMENT_RULES (__RAW_EXE_ATTACHMENT || __RAW_VBS_ATTACHMENT || __RAW_COM_ATTACHMENT || __RAW_PIF_ATTACHMENT || __RAW_CMD_ATTACHMENT || __RAW_BAT_ATTACHMENT) score ATTACHMENT_RULES 25.00 Any attachments listed above will be properly identified as and the tests run with the exception of an EXE attachment. A filename with an .exe extension is not flagged. I have added an additional rule that checks for an .exe attachment, that is not part of the meta rule, and I receive the same results. This leads me to believe there is something wrong with my test for .exe attachments. I am running SA 2.64, spamd, and it is invoked from q-mail. Any suggestions would be greatly appreciated. Thanks in advance for your assistance. Jay Hall How about trying: rawbody ATTACHMENT_RULES /filename=\"?.*\.(?:exe|vbs|com|pif|cmd|bat|cpl|scr)\"?\s*$/i score ATTACHMENT_RULES 25.00 Note: added .cpl and .scr added end-of-line test $ to avoid false positives on things like "example.com contract.doc" made quotes optional [EMAIL PROTECTED] 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer perl -e"map{y/a-z/l-za-k/;print}shift" "Jjhi pcdiwtg Ptga wprztg," I went back through the e-mail logs this evening, and e-mails with an exe attachment were being scored correctly until last night about 7:00 pm. Is it possible there is something wrong with one of the bayes files? Thanks for your help. Jay
Re: Rule problem (.exe attachments)
[EMAIL PROTECTED] wrote: Jay Hall wrote: I am experiencing a problem with one of my rules that I cannot seem to find. I have the following rules defined. rawbody __RAW_EXE_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_VBS_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_COM_ATTACHMENT/filename=\".*\.com\"/i rawbody __RAW_PIF_ATTACHMENT/filename=\".*\.pif\"/i rawbody __RAW_CMD_ATTACHMENT/filename=\".*\.cmd\"/i rawbody __RAW_BAT_ATTACHMENT/filename=\".*\.bat\"/i meta ATTACHMENT_RULES (__RAW_EXE_ATTACHMENT || __RAW_VBS_ATTACHMENT || __RAW_COM_ATTACHMENT || __RAW_PIF_ATTACHMENT || __RAW_CMD_ATTACHMENT || __RAW_BAT_ATTACHMENT) score ATTACHMENT_RULES 25.00 Any attachments listed above will be properly identified as and the tests run with the exception of an EXE attachment. A filename with an .exe extension is not flagged. I have added an additional rule that checks for an .exe attachment, that is not part of the meta rule, and I receive the same results. This leads me to believe there is something wrong with my test for .exe attachments. I am running SA 2.64, spamd, and it is invoked from q-mail. Any suggestions would be greatly appreciated. Thanks in advance for your assistance. Jay Hall How about trying: rawbody ATTACHMENT_RULES /filename=\"?.*\.(?:exe|vbs|com|pif|cmd|bat|cpl|scr)\"?\s*$/i score ATTACHMENT_RULES 25.00 Note: added .cpl and .scr added end-of-line test $ to avoid false positives on things like "example.com contract.doc" made quotes optional [EMAIL PROTECTED] 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer perl -e"map{y/a-z/l-za-k/;print}shift" "Jjhi pcdiwtg Ptga wprztg," I changed the rules as you suggested, but e-mails with exe attachments are still not being marked as SPAM. However, others are. Following are the headers from an e-mail sent with an exe attachment. To: [EMAIL PROTECTED] Subject: EXE Test 1 - exe Content-Type: multipart/mixed; boundary="050409040702070007040104" X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on mnea-hq.mnea.org X-Spam-Level: X-Spam-Status: No, hits=-4.9 required=5.0 tests=BAYES_00 autolearn=ham version=2.64 Return-Path: [EMAIL PROTECTED] X-OriginalArrivalTime: 29 Sep 2004 22:12:44.0170 (UTC) FILETIME=[71AA06A0:01C4A671] If I am reading the headers correctly, it appears the attachment tests were not done in this case. The file attached to the message was vncviewer.exe. What additional information should I be looking for to troubleshoot this problem? Thanks for your help. Jay
Rule problem (.exe attachments)
I am experiencing a problem with one of my rules that I cannot seem to find. I have the following rules defined. rawbody __RAW_EXE_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_VBS_ATTACHMENT/filename=\".*\.exe\"/i rawbody __RAW_COM_ATTACHMENT/filename=\".*\.com\"/i rawbody __RAW_PIF_ATTACHMENT/filename=\".*\.pif\"/i rawbody __RAW_CMD_ATTACHMENT/filename=\".*\.cmd\"/i rawbody __RAW_BAT_ATTACHMENT/filename=\".*\.bat\"/i meta ATTACHMENT_RULES (__RAW_EXE_ATTACHMENT || __RAW_VBS_ATTACHMENT || __RAW_COM_ATTACHMENT || __RAW_PIF_ATTACHMENT || __RAW_CMD_ATTACHMENT || __RAW_BAT_ATTACHMENT) score ATTACHMENT_RULES 25.00 Any attachments listed above will be properly identified as and the tests run with the exception of an EXE attachment. A filename with an .exe extension is not flagged. I have added an additional rule that checks for an .exe attachment, that is not part of the meta rule, and I receive the same results. This leads me to believe there is something wrong with my test for .exe attachments. I am running SA 2.64, spamd, and it is invoked from q-mail. Any suggestions would be greatly appreciated. Thanks in advance for your assistance. Jay Hall
Re: Spammers using my server
Thank you very much. The spammer was using an exploit in Formmail.cgi which I use on my web site which has now been disabled. They crafted a message, inserted it into the formmail on the web page which delivered it to sendmail for delivery. Normally it would have gone to the local email account but they were able to set an outside email address so sendmail began delivering the emails. Thanks - Original Message - From: "Lucas Albers" <[EMAIL PROTECTED]> To: "Justin Mason" <[EMAIL PROTECTED]> Cc: "Jay Ehrhart" <[EMAIL PROTECTED]>; Sent: Friday, September 24, 2004 1:41 PM Subject: Re: Spammers using my server > As a another good step, just SA scan ALL incoming and outgoing mail. > > Run a vulnerability scan against your server, nessus or sara against your > machine to find what is being exploited. > > -- > Luke Computer Science System Administrator > Security Administrator,College of Engineering > Montana State University-Bozeman,Montana > > >
Spammers using my server
This morning I had over 7000 emails in my Linux server's outbound queue which I deleted. My firewall log shows over 20,000 emails went out with a SunTrust bank announce saying to login and enter your username and password. I do not see the emails coming in like I would in a relay. How can I stop this or how are they doing this? My firewall using a SMTP proxy and only allows my domain in. I run MailScanner on my Red Hat 3.0 mail server with Sendmail. The box has the lastest patches from Red Hat. I have Sendmail setup to accept only my domain email. The non-deliverable reports are coming from my Linux apache user. Non-deliverables usually come from root. I am running apache on the server with forms. The forms software is the latest version and patches. Can anybody help on this? Thanks, Jay