Re: What to do about False Positives on messages I am sending?
Jon Ribbens wrote: Loren Wilton <[EMAIL PROTECTED]> wrote: Taking a look at that and offering my opinions: Thanks for taking the time to have a look at it. Apart from inline images though, the other points either don't apply to our emails, or don't appear to be contributing to the SpamAssassin score. In all honesty, I have to ask: does this mail NEED to be html? Other than flashy colors and imbedded images what does it buy you that the text message wouldn't convey? Unfortunately html, embedded images, align right, and flashy colors all end up making the thing look like a typical drug spam. I know that flashy colors and imbedded images are important if you are sending these to CEOs or other people that never learned to read. Unfortunately yes, this mail absolutely 100% does need to be HTML. A significant proportion of the target audience includes management types, sales and marketing types, etc and the presentation is at least as important as the content, if not more so. Personally I don't have anything against HTML emails if they have a text equivalent as well, and it's somewhat irritating that it's this precise feature that's one of the things SpamAssassin dislikes :-/ It's not so much about liking or disliking. It's about using tell tale signs to detecting probable spam. Jo
Re: What to do about False Positives on messages I am sending?
Loren Wilton <[EMAIL PROTECTED]> wrote: > Taking a look at that and offering my opinions: Thanks for taking the time to have a look at it. Apart from inline images though, the other points either don't apply to our emails, or don't appear to be contributing to the SpamAssassin score. > In all honesty, I have to ask: does this mail NEED to be html? Other than > flashy colors and imbedded images what does it buy you that the text > message wouldn't convey? Unfortunately html, embedded images, align right, > and flashy colors all end up making the thing look like a typical drug spam. > > I know that flashy colors and imbedded images are important if you are > sending these to CEOs or other people that never learned to read. Unfortunately yes, this mail absolutely 100% does need to be HTML. A significant proportion of the target audience includes management types, sales and marketing types, etc and the presentation is at least as important as the content, if not more so. Personally I don't have anything against HTML emails if they have a text equivalent as well, and it's somewhat irritating that it's this precise feature that's one of the things SpamAssassin dislikes :-/
Re: What to do about False Positives on messages I am sending?
From: "Rob Anderson" <[EMAIL PROTECTED]> Jon Ribbens <[EMAIL PROTECTED]> 12/20/06 03:16PM >>> Adam Lanier <[EMAIL PROTECTED]> wrote: That's why I asked to see a sample message. We could probably give some pointers on what is triggering SA. Set your TRUSTED_NETWORKS and that'll help. That's one of the few negative scoring rules. As he pointed out the first time around, the problem isn't when HE receives these. He is SENDING these, and is concerned when other people receive them. Trusted networks being right isn't going to help that case. (Besides, there's no particular indication that his trusted-networks is wrong. I'm assuming he sent this to himself, so trigging all-trusted is appropriate.) Loren
Re: What to do about False Positives on messages I am sending?
I have attached a sample message to this email. Note, it's just an example. This message does not trigger at the 5.0 level, but I know messages like this are being blocked by some of our customers. It does get a higher score than I would like it to (i.e. 0.0 ;-) ), and certainly the rules its triggering make little sense to me. Taking a look at that and offering my opinions: 1.Avoid text-align: right. Common spammer trick used to obfuscate drug spams. 2.Avoid excessively long lines in the HTML. Typical sign of spammers that can't quite figure out how to format a message. 3.Avoid excessive whitespace on the front of HTML lines. A sign of certain forms of table-layout phish mails. (You mail didn't have this, I'm just pointing it out.) 4.Avoid inline images if possible. 5.Avoid downloaded images even more. In all honesty, I have to ask: does this mail NEED to be html? Other than flashy colors and imbedded images what does it buy you that the text message wouldn't convey? Unfortunately html, embedded images, align right, and flashy colors all end up making the thing look like a typical drug spam. I know that flashy colors and imbedded images are important if you are sending these to CEOs or other people that never learned to read. But if you are sending this to the sysop to tell him how well his web site works, wouldn't it be just as useful to simply send the report in ascii? That would avoid virtually all of the potential spam signs. Loren
Re: What to do about False Positives on messages I am sending?
>>> Jon Ribbens <[EMAIL PROTECTED]> 12/20/06 03:16PM >>> Adam Lanier <[EMAIL PROTECTED]> wrote: > That's why I asked to see a sample message. We could probably give some > pointers on what is triggering SA. I have attached a sample message to this email. Note, it's just an example. This message does not trigger at the 5.0 level, but I know messages like this are being blocked by some of our customers. It does get a higher score than I would like it to (i.e. 0.0 ;-) ), and certainly the rules its triggering make little sense to me. > Set your TRUSTED_NETWORKS and that'll help. That's one of the few negative scoring rules. Rob
Re: What to do about False Positives on messages I am sending?
On Wed, Dec 20, 2006 at 06:44:48PM +, Jon Ribbens wrote: > I did that. The problem that needs fixing is SpamAssassin. It is > triggering on things that are nothing to do with spam (for example, > RFC-compliant use of multipart/related). Your main issue is that spammers are making their mails look more like ham, and that sometimes leads to more FPs. There's already discussions going on about things like EXTRA_MPART_TYPE and such. For example: https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5226 https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5110 etc. -- Randomly Selected Tagline: "My teacher says I need cupcakes. Cupcakes to learn." - Lisa Simpson, "Simpsons Safari" pgp60HEyvMo95.pgp Description: PGP signature
Re: What to do about False Positives on messages I am sending?
Noel Jones <[EMAIL PROTECTED]> wrote: > So why not find which rules are triggered by your message I already did - see my original post at the start of this thread. > Can't be too hard, spammers do it all the time. That's my point - why should I have to behave like a spammer in order to avoid getting classed as one? > If you're not sending spam, why does it look like spam? Find out and > fix the problem! I did that. The problem that needs fixing is SpamAssassin. It is triggering on things that are nothing to do with spam (for example, RFC-compliant use of multipart/related).
Re: What to do about False Positives on messages I am sending?
On Wed, 2006-12-20 at 11:38 -0600, Noel Jones wrote: > On 12/20/06, Jon Ribbens <[EMAIL PROTECTED]> wrote: > > "John D. Hardin" <[EMAIL PROTECTED]> wrote: > > > ...sign up with a service like Habeas or Bonded Sender and put their > > > headers in your messages? > > > > I suppose we could do. Does anyone know how much that costs? > > > > It still seems wrong to me though that SpamAssassin is penalising mail > > that doesn't look like spam, and encouraging people to make their ham > > look like spam. > > > > So why not find which rules are triggered by your message and just > rewrite your message so it doesn't trigger those rules? Can't be too > hard, spammers do it all the time. > > I find very few legit messages - even legit marketing mail - get > tagged as spam. > > If you're not sending spam, why does it look like spam? Find out and > fix the problem! > That's why I asked to see a sample message. We could probably give some pointers on what is triggering SA. signature.asc Description: This is a digitally signed message part
Re: What to do about False Positives on messages I am sending?
On 12/20/06, Jon Ribbens <[EMAIL PROTECTED]> wrote: "John D. Hardin" <[EMAIL PROTECTED]> wrote: > ...sign up with a service like Habeas or Bonded Sender and put their > headers in your messages? I suppose we could do. Does anyone know how much that costs? It still seems wrong to me though that SpamAssassin is penalising mail that doesn't look like spam, and encouraging people to make their ham look like spam. So why not find which rules are triggered by your message and just rewrite your message so it doesn't trigger those rules? Can't be too hard, spammers do it all the time. I find very few legit messages - even legit marketing mail - get tagged as spam. If you're not sending spam, why does it look like spam? Find out and fix the problem! -- Noel Jones
Re: What to do about False Positives on messages I am sending?
"John D. Hardin" <[EMAIL PROTECTED]> wrote: > ...sign up with a service like Habeas or Bonded Sender and put their > headers in your messages? I suppose we could do. Does anyone know how much that costs? It still seems wrong to me though that SpamAssassin is penalising mail that doesn't look like spam, and encouraging people to make their ham look like spam.
Re: What to do about False Positives on messages I am sending?
On Tue, 19 Dec 2006, Kelson wrote: > John D. Hardin wrote: > > Do they still subtract points from the score? That's the relevant > > factor. > > The headers? No. Unless you're running a really old SpamAssassin. No, the fact that the sender has registered with either Habeas or Bonded Sender. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter da Silva in a.s.r --- 6 days until Christmas
Re: What to do about False Positives on messages I am sending?
John D. Hardin wrote: Do they still subtract points from the score? That's the relevant factor. The headers? No. Unless you're running a really old SpamAssassin. -- Kelson Vibber SpeedGate Communications
Re: What to do about False Positives on messages I am sending?
On Tue, Dec 19, 2006 at 10:46:22AM -0800, John D. Hardin wrote: > Do they still subtract points from the score? That's the relevant > factor. Yes, they do. Just sharing that it doesn't involve modifying the message anymore. :) -- Randomly Selected Tagline: "Hey, you know what'd cheer you up? You should get yourself a puppy." -Amy "A puppy? Nibbler loved to eat puppies" -Leela pgpNq0XYbtmjJ.pgp Description: PGP signature
Re: What to do about False Positives on messages I am sending?
On Tue, 19 Dec 2006, Theo Van Dinter wrote: > On Tue, Dec 19, 2006 at 09:59:40AM -0800, John D. Hardin wrote: > > ...sign up with a service like Habeas or Bonded Sender and put their > > headers in your messages? > > FWIW, neither of those put headers in the message (Habeas stopped > doing that years ago). They're both DNS reputation services. Argh. Do they still subtract points from the score? That's the relevant factor. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter da Silva in a.s.r --- 6 days until Christmas
Re: What to do about False Positives on messages I am sending?
On Tue, Dec 19, 2006 at 09:59:40AM -0800, John D. Hardin wrote: > ...sign up with a service like Habeas or Bonded Sender and put their > headers in your messages? FWIW, neither of those put headers in the message (Habeas stopped doing that years ago). They're both DNS reputation services. -- Randomly Selected Tagline: Have I ever claimed to be sane? pgpDLPzXM4hPz.pgp Description: PGP signature
Re: What to do about False Positives on messages I am sending?
On Tue, 19 Dec 2006, Jon Ribbens wrote: > I work at a company with an automated on-line system. This system > sends emails to people. Spam Assassin appears to be triggering > very strongly, and incorrectly, on our messages. > Any advice would be gratefully received! ...sign up with a service like Habeas or Bonded Sender and put their headers in your messages? Just an idea. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED] key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- "Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never does quite what I want. I wish Christopher Robin was here." -- Peter da Silva in a.s.r --- 6 days until Christmas
Re: What to do about False Positives on messages I am sending?
On Tue, 2006-12-19 at 16:58 +, Jon Ribbens wrote: > > But that's all a bit philosophical and beside the point of my > question, which is: should I change our emails, and if so, in what > way - or do SpamAssassin's default settings as provided on > updates.spamassassin.org need changing? Perhaps you could post a sample message so we could take a look at the structure? signature.asc Description: This is a digitally signed message part
Re: What to do about False Positives on messages I am sending?
Sietse van Zanen <[EMAIL PROTECTED]> wrote: > Do you have your trusted_networks, internal_networks and all_trusted set > up correctly? > > With these three options you should be able to exclude messages sent > from your IP address. Yes, the problem is not that *our* SpamAssassin installation is flagging our mail as spam, it's that (some of) our *customers'* spam filters are flagging our mail as spam, and I assume that many of their spam filters use SpamAssassin. Obviously it's hard to get all of them to add our mail server IPs to their whitelists ;-) > BTW, you are sending bulk mail (same mail, many recipients) and bulk > mail isn't necessarily spam of course. Actually the mails I was talking about aren't bulk mail, because they are different mails with different recipients. They are automated, but not in the sense of "fill in the name and address into a template and send 1,000 copies" but in the sense of "do some work which can take up to several hours to perform, then send 1 personal (contains individual report results) email to 1 person to report completion". But that's all a bit philosophical and beside the point of my question, which is: should I change our emails, and if so, in what way - or do SpamAssassin's default settings as provided on updates.spamassassin.org need changing? Cheers Jon
RE: What to do about False Positives on messages I am sending?
If you look at politicians you will surely see that saying: "you shouldn't ..." wih a straight face is not that hard at all. :-) Do you have your trusted_networks, internal_networks and all_trusted set up correctly? With these three options you should be able to exclude messages sent from your IP address. BTW, you are sending bulk mail (same mail, many recipients) and bulk mail isn't necessarily spam of course. Ultimately you could even separate outgoing and incoming mail, by using multiple MTA's. Then you can use the outgoing MTA without SA, so it saves you some resources too. -Sietse -Original Message- From: Jon Ribbens [mailto:[EMAIL PROTECTED] Sent: Tuesday, December 19, 2006 5:10 PM To: users@spamassassin.apache.org Subject: What to do about False Positives on messages I am sending? I work at a company with an automated on-line system. This system sends emails to people. Spam Assassin appears to be triggering very strongly, and incorrectly, on our messages. FWIW, no we are not spammers, in fact the emails I'm talking about aren't even a mailing list. They're emails generated in response to a (confirmed) registered user performing an action on the system (each email goes to a single recipient, not bulk). A couple of examples of the tests being triggered include: EXTRA_MPART_TYPE This one appears to be penalising people who comply with the RFCs. multipart/related *requires* the 'type' parameter that is being flagged as 'spammy'. TVD_FW_GRAPHIC_NAME_MID This one appears to be penalising people who put images in the email with sensible names. HTML_IMAGE_ONLY_12 HTML_SHORT_LINK_IMG_2 These two appear to be penalising people who send short messages. I have read the AvoidingFpsForSenders page, and I am already doing most of what it says. I'm not encouraged by the first point: "The rules catch spam. If your email isn't spam, you shouldn't be matching the rules." I don't see how you can claim this with a straight face, given the rule examples I've mentioned above. One of the later bits of advice, "If you're using HTML emails, include a text part" is precisely what is triggering your own "spam-detecting" EXTRA_MPART_TYPE rule! I could work around these problems - I could break the RFC rules to avoid EXTRA_MPART_TYPE, I could obfuscate the image filenames to avoid TVD_FW_GRAPHIC_NAME, I could pad the message with invisible junk to avoid HTML_IMAGE_ONLY etc. But that would be ridiculous - that's what spammers do! Am I supposed to disguise my non-spam messages as spam in order to prevent SpamAssassin calling them spam? Any advice would be gratefully received! On the plus side, I should point out that we have recently implemented SpamAssassin on our incoming email and it's cut down the spam on the 'catchall' mailbox from approximately 3,000 a day to more like 10, so it's being very helpful in that respect ;-) Cheers Jon
