Re: [ANNOUNCE] Apache SpamAssassin 3.4.2 available

[Dianne Skoll]

On Mon, 17 Sep 2018 15:22:48 -0400
"Kevin A. McGrail"  wrote:

[snip]

> Good to know. Did the Makefile.PL gracefully tell you that your
> Makemaker was too old?

It did indeed, which made the fix very simple.  Thanks for your hard work!

Regards,

Dianne.

Re: [ANNOUNCE] Apache SpamAssassin 3.4.2 available

[Dianne Skoll]

On Mon, 17 Sep 2018 13:22:32 -0400
"Kevin A. McGrail"  wrote:

> I'd be pretty shocked if you have to do very much to that src rpm for
> 3.4.1 to get 3.4.2 working.

I ran into one gotcha on (ancient) Debian 5; the version of
ExtUtils::MakeMaker was too old.  Installing from CPAN did the trick.  I'd
imagine something similar might happen on ancient Red Hat Enterprise Linux 5.

Regards,

Dianne.

Re: MIME Tools and Diane Skoll question

[Dianne Skoll]

On Mon, 17 Sep 2018 09:46:19 + (UTC)
Pedro David Marco  wrote:

> To my remember Dianne Skoll was in his list...

Hello...

> Does anyone know whether she is still maintaining MIME::Tools ?

Yep, I am.

Regards,

Dianne.

OFF-TOPIC: Re: Just to lighten your day?

[Dianne Skoll]

On Wed, 2 May 2018 15:32:50 -0500 (CDT)
David B Funk  wrote:

[...]

> The first three terminations weren't good enough, so we're going to
> do it one more time. And if -that- one doesn't do it, we'll proceed
> to the final ultimate termination...

As in "I'm not dead yet!" from Spamalot? :)

Regrads (dammti...),

Dianne.

Re: Dropping mail

[Dianne Skoll]

On April 29, 2018 11:11:18 PM EDT, Linda Walsh  wrote:

>   Except users who have their own rules are not likely
>doing it in the context of the initial choice of whether or
>not to accept the email onto the server.

They do in our system.
 
>   I.e. it "should" never be the case that user-defined
>filters are run in the MTA's initial receive context as the MTA
>just received (or is in process of receiving) an email coming
>in on a privileged port (like port 25) which would imply a
>privileged context (most likely root).

No; see the milter API and library.

>   Delays are not the same as dropped email.

I have seen sending MTAs give up after an absurdly small number of retries.

Regards,

Dianne.

Re: Dropping mail

[Dianne Skoll]

On Fri, 27 Apr 2018 15:18:28 -0500 (CDT)
David B Funk  wrote:

> If you have that many different classes of recipients, just set the
> number of allowed recipients/transaction to one and be done with it.

That will cause mail failures.  It's not *supposed* to, but I know
from experience it will.  Some MTAs limit the number of retries to a
ridiculously low number like 10 or 20 and then give up.

Regards,

Dianne.

Re: Dropping mail

[Dianne Skoll]

On Fri, 27 Apr 2018 14:39:43 -0500 (CDT)
David B Funk  wrote:

[snip]

> Define two classes of recipients:
>class A == all users who want everything
>class B == all users who want "standard" filtering

This works if you have a limited number of classes, but in some cases
users can make their own rules and settings so the number of classes
can be the same as the number of RCPTs.

Even in the two-class case, there's still a delay for the subsequent
class(es).

Regards,

Dianne.

Dropping mail

[Dianne Skoll]

Hi,

I have reluctantly come to the conclusion that in some cases, it is
necessary to silently drop spam rather than reject it.  This is the
situation:

An email comes in for two recipients in one SMTP trasaction (ie,
a MAIL, two RCPTs and then DATA).

One recipient's rules say to accept.  The other recipient's says to reject.

You can't reject post-DATA because then it looks like both recipients
received the mail.

You can accept and create a failure message for one recipient, but then
you risk generating backscatter.

You can tempfail all but the first RCPT to force the message to be
split up into individual messages per recipient, allowing you to accept
or reject individually.  But this will delay mail and possibly cause it
not to be delivered if there are many recipients and the sending relay
is impatient.

So I reluctantly conclude that in all but the smallest of installations,
dropping the mail for the recipient whose rules say to do so is the
best thing to do.

There have been SMTP extensions proposed to combat this.  I recall an
extension that had you issue RCPTs until one of the RCPTs was
accepted, then DATA, then additional RCPTs with a "also send the
foregoing to this one" keyword so you could have per-recipient data
filtering, but of course spammers could not be obliged to use the
extension. :(

Regards,

Dianne.

Re: dropping other's email(s) as a "best practice" for hosted email? (was: "anyone recognize these headers? ...")

[Dianne Skoll]

On Thu, 26 Apr 2018 13:41:05 -0700
L A Walsh  wrote:

> To my way of thinking, dropping someone else's email,
> telling the sender the email is being rejected for having
> spam-like characteristics and telling the recipient nothing
> seems like it might have legal liability for the for the
> user potentially missing vital email.

It all depends on the contract between the service provider and the
customer.

If the service provider puts something in to the effect that it will
make best-effort attempts to deliver mail but is not responsible for
lost mail, then I doubt there's any legal liability.

For example, Google's Terms of Service say the following (in all-caps)

 Other than as expressly set out in these terms or additional terms,
 neither Google nor its suppliers or distributors make any specific
 promises about the services. For example, we don’t make any
 commitments about the content within the services, the specific
 functions of the services, or their reliability, availability, or
 ability to meet your needs. We provide the services “as is”.

 Some jurisdictions provide for certain warranties, like the implied
 warranty of merchantability, fitness for a particular purpose and
 non-infringement. To the extent permitted by law, we exclude all
 warranties.

They also have a limitation of liability clause that limits their liability
to the amount paid to use the services.

Regards,

Dianne.

Re: Check for valid MX of sender and rspamd testing

[Dianne Skoll]

On Mon, 9 Apr 2018 09:56:20 -0500
David Jones  wrote:

> On 04/09/2018 09:44 AM, Reindl Harald wrote:
> > you simply don't want connect to every innocent MX which inbound
> > mail is forged because for the sake of god you are attacking the
> > victim of spoofed mails and you are easily part of a distributed
> > DOS when your few connections back are only a small part

Also, if an innocent domain's MX server just happens to be down
when you check, you could get a FP.

Checking for the existence of a sane MX record is good practice.
I'm not so sure about actually trying to connect to said MX, even if
you take basic precautions to minimize connections.

Regards,

Dianne.

Re: bypass milter but not the test.

[Dianne Skoll]

On Wed, 4 Apr 2018 17:16:04 +0200
Tom Hendrikx  wrote:

> Sounds like a mimedefang question, not a spamassassin one. But did you
> restart mimedefang after adding the rule?

You can force MIMEDefang to reread everything without restarting it:

md-mx-ctrl reread

Regards,

Dianne.

Re: OT: Congratulations Dianne

[Dianne Skoll]

Thank you everyone.  I hope this leads to good things for email filtering.

> Sorry, but what is AppRiver, and what is Roaring Penguin, and who is
> Dianne?

Answers to those questions are all a Google query away.

It is off-topic for Spamassassin, I grant you, and hence the OT: tag.
Thank you again everyone, but I think this thread should end.

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Tue, 3 Apr 2018 11:21:35 -0400
Rob McEwen  wrote:

> Thanks for all you do! And good luck with that. But there are a few 
> potential problems. When I analyzed Google's shortners about a month 
> ago, I found that a VERY large percentage of the most malicious 
> shortened URLs were a situation where the spammers were generating a 
> unique shortner for each individual message/recipient-address.

We found that too, but in most cases, they generated the unique URLs
by adding query parameters to the same base URL, sort of like this:

http://malware.net/?id=znsjdsjau
http://malware.net/?id=aosu94e
etc...

and then shortening them.

So if you blacklist just the base URL, you cover those all off,
assuming you expand out shortened URLs as part of your processing, of course.

> Meanwhile, in my analysis I did about a month ago, about 80% of
> Google's shortners found in egregious spams (that did this one-to-one 
> shorter-to-recipient tactic)... were all banging on one of ONLY a
> dozen different spammers' domains. Therefore, doing a lookup on these
> and then checking the domain found at the base of the link it
> redirects to... is a more effective strategy for these - whereas, for
> THESE 80% of egregious google shortners, a full URL lookup is
> worthless, consuming resources without a single hit.

Yep, that's what we found too.

Regards,

Dianne.

OT: Frequency vs. Period (was Re: The "goo.gl" shortner...)

[Dianne Skoll]

On Tue, 3 Apr 2018 11:09:38 -0300
Leandro  wrote:

> This means, for example, your system do 10 queries at same second,
> then the query frequency is 100ms.

In SI units, frequency has the unit s^(-1) and period has the unit s,
where s stands for "second"

So 100ms is the period, and 10/s is the frequency.  Basic dimensional
analysis.

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Mon, 26 Feb 2018 00:07:54 -0600 (CST)
David B Funk  wrote:

> So my bet is that the spammers are crafty enough to check things like
> browser referrer, cookies, etc to detect/differentiate a browser vs a
> link-checker.

Yep.  You need to fake your User-Agent (not hard) and put a limit on
the number of redirections you follow or you can end up in trouble.

I picked a likely-looking agent out of our logs and use this:

$ua->agent('Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; 
Trident/7.0; SLCC2;');

Might need revising later because it's pretty ancient!

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Wed, 21 Feb 2018 12:41:05 -0700
Amir Caspi  wrote:

> On that note -- regardless of what OTHER HW/SW solutions might do,
> since this is a SpamAssassin mailing list ... is there any facility
> to implement this in SA?

Someone earlier posted a link to https://github.com/smfreegard/DecodeShortURLs

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Wed, 21 Feb 2018 16:48:40 +
Anthony Cartmell  wrote:

> If you mean following URLs in messages, you do need to be aware that
> this can break one-time login links.

Big time.  That's why you only want to do it for URLs that are
absolutely known to be shortened URLs.  You have to keep a list of
known URL-shorteners.  It's hard/impossible to get a complete list,
but pretty easy to get a list of the big ones that cover 90% of all
shortened URLs you're likely to see.

Regards,

Dianne.

Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))

[Dianne Skoll]

On Wed, 21 Feb 2018 16:35:27 +
Karol Augustin  wrote:

> I think the point here might be that if Google acted promptly on abuse
> spammers would stop using shorteners.

True, that might happen.  OTOH, I see about as many spams with bit.ly
shorteners as goo.gl shorteners which is not what one might expect if
bit.ly were really that much more proactive than goo.gl.

Regards,

Dianne.

Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))

[Dianne Skoll]

On Wed, 21 Feb 2018 11:29:00 -0500
Rob McEwen  wrote:

> Nevertheless, it is a shame to have to shift more of the burden onto 
> spam filters to do more work (some of which requires MORE latency) -
> in order to partly mitigate Google's failure to prevent/correct the
> abuse.

Yes, I agree.  On the other hand, IMO a spam filter should block messages
that point to a taken-down shortened URL.  Although such messages may
not be downright dangerous, they are still annoying and are still spam.
There's no way of avoiding the work.

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Wed, 21 Feb 2018 11:00:48 -0500
Rob McEwen  wrote:

> > [Expanding shorteners] been part of our practice for about a year now.

> Excellent! I wish others would be as innovative and on top of things
> as you are! Unfortunately, your statement doesn't alter my point you
> were replying to, even one tiny bit.

Really?  This isn't rocket science.  If I thought of it, I'm sure dozens
if not hundreds of others have thought of it and implemented it.

Regards,

Dianne.

Re: Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))

[Dianne Skoll]

On Wed, 21 Feb 2018 10:58:17 -0500
Rob McEwen <r...@invaluement.com> wrote:

> On 2/21/2018 10:37 AM, Dianne Skoll wrote:
> > The concern voiced in another email about overloading Google's
> > infrastructure is quite charming and quaint.  

> My concern was NEVER about overloading google.

I guess I misinterpreted: "...such automated lookups could also put a
huge extra burden on Google's servers..." from Message-Id
<b229b274-ff75-0109-9510-346b078d9...@invaluement.com>

> My concern was about Google auto-blocking or throwing a captcha at
> very high volume and automated lookups. That is a HUGE difference.

That's a concern, but you can mitigate it slightly by caching the
results of URL expansion.  Also, I suspect most URL-shorteners
expect to be bombarded with requests and are engineered accordingly.
I have seen a CAPTCHA thrown up from the search page, but I've never
seen a goo.gl HEAD request reply with anything other than 404 or 301.

Regards,

Dianne.

Re: action_drop_with_warning called outside of filter context

[Dianne Skoll]

Hi,

> mimedefang.pl[10245]: w1K87JOB027594: action_drop_with_warning called
> outside of filter context

> then the attachment was not dropped.
> here is my filter:

Please read mimedefang-filter man page very carefully.

Regards,

Dianne.

Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response)

[Dianne Skoll]

On Wed, 21 Feb 2018 02:30:40 -0500
Rob McEwen  wrote:

> (a) it might not "scale" for high volume mail flows and DNSBLs who,
> like invaluement, process dozens (or more) spams per second.

We use HEAD requests to expand known URL-shorteners on a cluster that
peaks around 60 msgs/s

> (b) and this isn't going to suddenly become a feature inside of many 
> types of spam filtering hardware and software overnight... that could 
> even take years,

It's been part of our practice for about a year now.

Regards,

Dianne.

Expanding shortened URLs (was Re: The "goo.gl" shortner is OUT OF CONTROL (+ invaluement's response))

[Dianne Skoll]

On Tue, 20 Feb 2018 23:38:53 -0700
"@lbutlr"  wrote:

> As I suspected, it is possible to get the goo.gl target URL without
> loading the site, though using curl is probably not realistic in this
> specific case.

We do a HEAD request and it works on most URL shorteners.

The concern voiced in another email about overloading Google's
infrastructure is quite charming and quaint.

Regards,

Dianne.

Re: URIBL_BLOCKED

[Dianne Skoll]

On Thu, 15 Feb 2018 16:06:40 +0100
Matus UHLAR - fantomas  wrote:

> >Or if you like using your ISP's servers, most DNS server software
> >lets you forward by default but make exceptions for specific
> >domains.  

> although possible, this does not make sense IMHO.

It makes a lot of sense, IMO.  I'm not H like the rest of you.

> you would need to keep track of DNSBLs you need to access directly,
> while they can change with SA rules without your knowledge.

IMO, it makes no sense to run a mail server without having complete
knowledge of which DNSBLs you use.

Regards,

Dianne.

Re: URIBL_BLOCKED

[Dianne Skoll]

On Wed, 14 Feb 2018 14:05:54 -0800 (PST)
John Hardin  wrote:

> This detail always gets glossed over: set up a local NON-FORWARDING 
> resolver.

> If you set up a local resolver and it just forwards requests to your
> ISP's DNS servers, you have not materially changed the problem.

Or if you like using your ISP's servers, most DNS server software lets
you forward by default but make exceptions for specific domains.

Regards,

Dianne.

Re: New idea for stopping spam

[Dianne Skoll]

On Fri, 26 Jan 2018 11:49:07 -0800
Ted Mittelstaedt  wrote:

[snip]

> Do you think this approach might work?

Not any better than Bayes.  All your "spam archetype" examples are
already easy to stop; we whack them all handily with Bayes.  The
annoying ones are more like:

Subject: hi
Subject: 'sup
Subject: Order #12345

etc.

Regards,

Dianne.

Re: Penalty for no/bad SPF

[Dianne Skoll]

On Thu, 25 Jan 2018 05:19:38 -0500
Bill Shirley  wrote:

> I'm all for tightening up standards compliance with email, but what I
> would see if this would happen is a request from my customers saying:
> Bob'semails (b...@bad-spf.com) are going to the spam folder; whitelist
> him please Bob's email admin won't ever know that his SPF is failing.

This matches our experience.  We perform a very mild form of SPF enforcement:
If a message fails or softfails SPF, then we ignore any sender or domain
whitelist.  One of our top support questions is how to allow the whitelist
anyway. :(

Regards,

Dianne.

Re: Penalty for no/bad SPF

[Dianne Skoll]

On Wed, 24 Jan 2018 14:20:57 -0800 (PST)
John Hardin  wrote:

> > At this point, I would be willing to penalize sites with bad SPF
> > records (syntactically invalid; more than one different SPF record
> > attached to the same domain, etc.)  Those people really deserve
> > penalties because they've messed up.  

> Does that include "+all" or authorizing more than a class-b space
> through any method, which I'd characterize as "malicious" rather than
> "messed up"?

+all is malicious for sure.  More than a Class-B might just be bad
planning AKA Microsoft's outbound IP address list.

However, a malicious actor can use the "exists:" mechanism to simulate
+all in a way that can't easily be proven by an SPF evaluator. :(

I would like to see the exists: mechanism tossed.

Regards,

Dianne.

Re: Penalty for no/bad SPF

[Dianne Skoll]

At this point, I would be willing to penalize sites with bad SPF
records (syntactically invalid; more than one different SPF record
attached to the same domain, etc.)  Those people really deserve
penalties because they've messed up.

I would not be willing to penalize sites with *no* SPF at all just
yet.  Maybe eventually.

Regards,

Dianne.

Re: Penalty for no/bad SPF

[Dianne Skoll]

On Wed, 24 Jan 2018 19:01:28 +
Vincent Fox  wrote:

> SPF is a zombie legacy that someone should shoot in
> the head.

+1

> Maybe then we could design something that
> is useful for what we all desire, which is properly
> authenticating senders.

We cannot authenticate senders and keep SMTP as useful as it is now.
The whole beauty of email is that some stranger can contact you out of the
blue.  And that mailing lists can send mail on your behalf.

We can work on people not being able to spoof "well-known" senders; for
that, I'd much prefer DKIM+DMARC over SPF.  SPF was a mistake and we should
all admit that.

Regards,

Dianne.

Danger of using your real name (was Re: From name containing a spoofed email address)

[Dianne Skoll]

On Sat, 20 Jan 2018 00:33:32 -0500
"Bill Cole"  wrote:

> On 19 Jan 2018, at 20:02 (-0500), jdow wrote:

> > After your first time being a victim of cyberstalking you'll soon
> > enough wish your "from" line was as generic as mine. People who put
> > their full name in the From: line haven't been mugged yet. I spent
> > a year learning about this 1985-1986.

Wow, that's awful!

[snip]

> OTOH, if I were a woman or looked less like a biker-bar bouncer or
> had a history I'd rather not have widely known, I'd almost surely
> evaluate my risk differently. This is a hard problem that no one has
> yet solved.

One minute of googling "d...@roaringpenguin.com" will produce a lot
of information including the fact that I'm a member of a fairly
vulnerable group.  The info is out there, so at this point I've
stopped worrying about it.  The good news is that I do not live in the
United States, whereas most of the people I provoke online do :), so I
have some physical distance to protect me.

> > As a byproduct of this habit of mine, when I see a "To: John" or
> > other name than mine it's automatically spam, especially when it
> > cannot even get the gender right.

Yeah, "Dear Dfs," is a dead giveaway for me.

Regards,

Dianne.

Re: Turn OFF SA spam filtering but keep ON header examination

[Dianne Skoll]

On Thu, 18 Jan 2018 16:01:13 -0500
Chip  wrote:

> I'm tied to a Cpanel/WHM VPS which can't be changed.

That's a problem.  It's like having someone require you to play
Hungarian Rhapsody while wearing mittens.  I mean sure... maybe it's
possible, but why would you try?

Is there no possibility of a filtering machine in front of that box?
Or as others have said, use procmail which AFAIK works fine with Exim.

Regards,

Dianne.

Re: Link following leads to redirect

[Dianne Skoll]

On Wed, 27 Dec 2017 19:21:32 +0100
Reindl Harald  wrote:

> > At most, I would do a HEAD on a URL and not a GET.  HEAD is
> > probably safer and will usually tell you if the link is a redirect  

> no, for the web application it's typically transparent because the
> whole purpose of HEAD is that you get the whole headers as you would
> do with a GET request without the body

That's true.  I was mistaken about "safer".

Regards,

Dianne.

Re: Link following leads to redirect

[Dianne Skoll]

On Wed, 27 Dec 2017 12:47:00 -0500
Alex  wrote:

> It [fetching URLs] would also probably lead to inadvertently
> unsubscribing people from mailing lists.

Yes, if the lists use badly-written mailing list software.

At most, I would do a HEAD on a URL and not a GET.  HEAD is probably safer
and will usually tell you if the link is a redirect.  You also want
to fake the user-agent to be a common Windows browser because some
malware servers look at the User-Agenet and return a 404 if they think
the client is not a real Web browser.

Even a HEAD can be dangerous; there's an Internet "security" [sic] company
out there that shall remain nameless; these geniuses view HEAD requests
as attacks and report you to your ISP.  It took me 2+ weeks to sort out
their BS "abuse" complaints.

> I'd like to think some intelligence could be built into such a system,
> and know many of the spam companies like Symantec and Mimecast are
> doing this to differing degrees.

I know of one company that collects URLs and has a central server farm that
analyzes them (ie, the URL fetching is done on a completely different
set of machines than the spam filtering.)  They have all kinds of heuristics
and special-case code to make it relatively safe.

Regards,

Dianne.

Re: Link following leads to redirect

[Dianne Skoll]

On Wed, 27 Dec 2017 07:50:38 -0800 (PST)
John Hardin  wrote:

> > Hi, is there anything available that can follow a link to either
> > test it itself for its reputation or RBL, or somehow add points to
> > an email that contains a link that just redirects?

> That's unfortunately a way to trigger tracking bugs.

Additionally, it's not clear to me you'd want your mail server's IP
to appear in the web logs of potentially dodgy websites.  The last thing
you need is police showing up with a warrant because your IP has visited
a criminal site.

Regards,

Dianne.

Re: Rule to detect mailsploit

[Dianne Skoll]

On Wed, 06 Dec 2017 14:37:28 +0100
Benny Pedersen  wrote:

> http://www.postfix.org/postconf.5.html#message_strip_characters

That won't work because the doc says:

Note 1: this feature does not recognize text that requires MIME
decoding. It inspects raw message content, just like header_checks and
body_checks.

and the exploit uses MIME-encoding to hide the NULs.

Regards,

Dianne.
ader_checks and
body_checks.

and the exploit uses MIME-encoding to hide the NULs.

Regards,

Dianne.

Re: How to view bayesian database in legible text

[Dianne Skoll]

On Thu, 9 Nov 2017 17:02:52 +
Gary Smith  wrote:

> I could be absolutely wrong but isn't bayes a hash of the string
> parts which is part of the performance of bayes?

Part right, part wrong.  The SpamAssassin Bayes implementation does
hash tokens.  But it doesn't improve performance measurably.

Anyway, there's no practical way to reverse the hash to get
the original tokens.  It can be done with a lot of work, and it'd
be uncertain because SA doesn't keep the entire hash... only a truncated
version thereof.

Regards,

Dianne.

Re: Weird new malware

[Dianne Skoll]

Hi,

In case anyone wants an actual sample: https://pastebin.com/raw/R3b0UHsB

Regards,

Dianne.

Re: Weird new malware

[Dianne Skoll]

On Wed, 8 Nov 2017 11:49:38 -0800 (PST)
Jim Dunphy  wrote:

> header J_BAD_CONTYPE Content-Type !~ 
> /^(application|audio|image|message|multipart|text|video|x-)/i

For messages that lack a content type header, I guess you need the
[if-unset:] tag at the end:  [if-unset: text/plain]

I know those messages are pretty scarce nowadays, but they do still
exist.

Regards,

Dianne.

Re: Weird new malware

[Dianne Skoll]

On Wed, 8 Nov 2017 11:02:16 -0500
Rob McEwen  wrote:

> This seems to be catching most of them:

> Subject: Invoice [A-Z]{2,3}\d{7}\b

Yes, that'll work.  Maybe a better approach is a combo rule that looks
in the headers for Content-Type: .*art/mixed but NOT multipart/mixed

I don't know offhand how to create such a rule in SpamAssassin, but I imagine
a meta rule could take care of it.

Regards,

Dianne.

Weird new malware

[Dianne Skoll]

Hi,

Heads-up: We're seeing weird new malware with a subject that looks like

   Invoice XXX

where XXX is two or three random upper-case letters and n is a series
of digits.  What's weird is that the Content-Type: header looks like this:

Content-Type: multXXXart/mixed

where the XXX is the same as in the subect.  That is, a message
with subject "Invoice UUI8187685" has Content-Type "multUUIart/mixed".  This
is fooling our MIME parser because it doesn't see the container as a
multipart.  Does any client software?

Anyway, might want to make rules for this.

Regards,

Dianne.

Re: Toys-r-us SPF record

[Dianne Skoll]

On Wed, 27 Sep 2017 14:24:52 -0500
David Jones  wrote:

> Sure.  It's not specific to toyrus.com but typically subdomains like 
> *.toysrus.com won't include Office 365 so they are candidates for 
> whitelist_auth entries.

Ah, gotcha.  I was missing that point.

Regards,

Dianne.

Re: Toys-r-us SPF record

[Dianne Skoll]

On Wed, 27 Sep 2017 13:24:16 -0500
David Jones  wrote:

> Alex  brought something interesting to my 
> attention that I think is worth passing along.  The SPF record for 
> email.toysrus.com is pretty sloppy.  It includes other SPF records
> that contain Office 365's servers.

I don't think that's specific to Toys-R-Us.  Anybody who uses Office 365
for outbound mail will have those SPF records included (if they use SPF
at all).

The question is whether or not Microsoft allows O-365 users to spoof their
domain when sending outbound mail.  If they do, then it's a problem because
anyone can buy the minimum order of O-365 and spoof any other O-365 customer
with an SPF "pass".

Regards,

Dianne.

Re: Would anyone be interested in a SA enhancing service?

[Dianne Skoll]

On Fri, 22 Sep 2017 10:43:57 -0700 (PDT)
John Hardin  wrote:

> > Are you suggesting people send you their email for scanning? If so,
> > I'd say nobody in their right mind should be interested, for
> > obvious reasons of privacy. It would also be illegal where I live.

> He was only proposing the subject. Essentially it sounds like a
> subjectBL service.

The subject initially, but eventually everything.

A subject BL wouldn't be that useful.  It would catch a lot of the
obvious porn stuff, but not so much the ones that say "Hi" or "Hey".

Regards,

Dianne.

Re: Would anyone be interested in a SA enhancing service?

[Dianne Skoll]

On Fri, 22 Sep 2017 08:40:50 -0700
Marc Perkel  wrote:

> The next level would be sending the message headers and eventually -
> the full message.

Why would anyone do that as opposed to just purchasing your commercial
spam filtering service?  If someone wants to offload spam-detection,
they might as well offload the entire problem to someone else and not
worry about it.  Especially given this statement:

> Would need someone to write a simple plugin - not a perl guy - but
> how hard can that be? Would eventually need to be encrypted though.

I think most admins would have significant privacy concerns.

Regards,

Dianne.

Re: In anyone else getting 325KB spams from cont...@cron-job.org?

[Dianne Skoll]

Hi, again,

Aha...

https://cron-job.org/en/spam-statement/

They are victims of a joe-job.

Regards,

Dianne.

Re: In anyone else getting 325KB spams from cont...@cron-job.org?

[Dianne Skoll]

On Thu, 14 Sep 2017 11:27:27 -0700
"Loren Wilton"  wrote:

> Other than being obvious spam, they seem to be set up as though they
> were legitimate commercial mailing list stuff, often containing
> things like contact-id and the like in the links.

> Is anyone else seeing these?

A small number.  The cont...@cron-job.org address is only in the From:
header; the envelope recipients look randomly-generated and sometimes
from unrelated domains.

Should be easy to block.  Just block the cron-job.org domain.

Regards,

Dianne.

Re: Identifiying PDF phish docs

[Dianne Skoll]

On Tue, 22 Aug 2017 20:19:06 -0400
Alex  wrote:

> > Take a look at podofopdfinfo.  It can extract URLs from PDF docs
> > and you can trigger on those.  

> Thank you. It didn't work on this one :-(

It worked for me:

$ podofopdfinfo pdf-phish.pdf 
Document Info
-
File: pdf-phish.pdf
PDF Version: 1.5
[,,, much output deleted ...]

Annotation 0
[,,, much output deleted ...]

Link Target: 1
Action URI: http://dabanlar.com/west/scan.html

My verstion of podofopdfinfo is 0.9.3 as packaged on Debian Jessie.

> I also don't see a way to use it with amavisd.

Right.  I use MIMEDefang which is a little more flexible in how you
glue the bits and pieces together.

> "strings" was able to extract the URL.

That works this time, but generally speaking, PDF documents may be
compressed in which case "strings" won't be useful.

I reported the URL to Google as fraudulent.

Regards,

Dianne.

Re: Identifiying PDF phish docs

[Dianne Skoll]

On Tue, 22 Aug 2017 14:55:01 -0400
Alex  wrote:

> I know there was a PDF OCR plugin of some sort, but I don't recall it
> being all that effective. Ideas greatly appreciated.

Take a look at podofopdfinfo.  It can extract URLs from PDF docs and you
can trigger on those.

Regards,

Dianne.

Re: Operators Blacklist Survey

[Dianne Skoll]

On Tue, 15 Aug 2017 12:46:59 -0500
Shivram Krishnan  wrote:

> Rule-based systems like spamassassin make room for false positives
> from any one of the rules. For instance , a blacklist can have a
> false positive, but there may be other rules which may not agree with
> the blacklist. An ensemble of such rules allows make spamassassin to
> be more accurate.

I didn't know any of that!  Thanks!  Wow!  Consider my mind blown.

> In case of non-rule based systems like firewall, an inaccurate
> blacklist can prove costly when the firewall drops legitimate traffic
> based on inaccurate blacklists. I was reading about graylists on
> cisco firewalls...

That's not a standard use of the term "graylist"

> where the network operators could use the graylists to generate
> alerts to the operator to act upon. A network operator can treat a
> third-party blacklist as a graylist and generate alerts. Is this
> common?

Not if sysadmins want a life.  If I did something like that on our
systems, I'd be getting multiple alerts per second.

Regards,

Dianne.

Re: Operators Blacklist Survey

[Dianne Skoll]

On Tue, 15 Aug 2017 12:02:23 -0500
Shivram Krishnan  wrote:

> Thanks for the response Bill. I have got a couple of responses from
> this group, which agree with what you are saying - they have their
> own custom techniques to prevent spam and reduce false positives.  If
> thats the case, who uses third-party generated blacklists?

I think you'll find a lot of people use them.  My instincts tell me the
userbase falls into three sets of administrators:

1) Admins of large organizations that can afford reputable lists like Spamhaus,
etc. and use them.

2) Admins of tiny mail servers who are highly aggressive and use
blacklists like kids popping candy and who don't care overly-much
about false positives.

3) Admins of small to medium organizations who use commercial
anti-spam filters or commercial email hosts that make use of
blacklists by default, and who probably don't really understand the
ramifications of using blacklists.

My $0.02: Blacklists can be useful, but I would never reject based solely
on an IP being blacklisted.  Also, I don't use third-party blacklists, though
I do use a set of DNSBLs that my company controls.

Regards,

Dianne.

Re: I know what I want to do ... BUT ...

[Dianne Skoll]

On Fri, 11 Aug 2017 15:27:52 +
hospice admin  wrote:

> text = "v=spf1 exists:%{i}._spf.xyz.com ~all"
> What I'd like to do is turn this into an RBL check, but
> eval:check_rbl('Evil-ESP','_spf.xyz.com')

I understand what you're trying to do, but have you thought about the
implications?  You are allowing an evil (your word) organization to be
an RBL for you.  They may very well put every possible IPv4 address in
their exists: SPF record which would not bode well for your spam
filtering...

If you still want to do it, I suspect it'd have to be done outside of
SpamAssassin in the glue code you're using.

Regards,

Dianne.

Re: HTML (was Re: Sender needs help with false positive)

[Dianne Skoll]

On Tue, 08 Aug 2017 20:01:52 +0200
Benny Pedersen  wrote:

> why does the OP need to tell sendgrid his users passwords ?

That is indeed a very good question. :)

It's not as if this is some sort of mass-mailing or marketing-oriented
email that needs to be tracked.

Regards,

Dianne.

HTML (was Re: Sender needs help with false positive)

[Dianne Skoll]

On Tue, 8 Aug 2017 08:00:04 -0500
David Jones  wrote:

> I absolutely agree but it's possible that this part is out of his 
> control.  Sendgrid might be receiving a plain text email from the
> normal source and adding HTML to get that image in there for
> tracking.

If you can't determine the content of your own messages, time to find
another provider, I think.  Surely Sendgrid lets you control this sort
of thing?

Regards,

Dianne.

Re: Sender needs help with false positive

[Dianne Skoll]

On Tue, 8 Aug 2017 07:36:01 -0500
David Jones  wrote:

> The origin of the email and the path it takes makes a big difference
> in how it's filtered.

Sure, but doing a plain-text message with no HTML will immediately knock
2.2 points off the score.  That's a pretty cheap and easy win.

Regards,

Dianne.

Re: Sender needs help with false positive

[Dianne Skoll]

On Mon, 7 Aug 2017 19:28:04 -0400
"Jacek Osuchowski"  wrote:

> This is an email I sent to IsNotSpam.com. They list the whole thing
> when testing for spam. I am getting a lot of complains from our
> customers that our emails are not received. Our domain is not
> blacklisted anywhere so I suspect it is the spam filtering (as
> IsNotSpam tool indicates). Is there anything in the email we send
> that could trigger flagging as a spam. THANK YOU

Don't send HTML.  Just send a plain-text message.

That'll knock 2.2 points off the score and bring it to 3.6.

Simple fix, no?

Regards,

Dianne.

Password reset strategies (was Re: Sender needs help with false positive)

[Dianne Skoll]

[Just replying to one aspect of the original message.]

On Mon, 7 Aug 2017 18:26:00 -0500
David Jones  wrote:

> First, it's a bad idea for a number of reasons to send passwords via 
> email.  Most modern "lost password" mail loops use a unique URL that 
> expires after a short period of time.

As long as both methods expire, both methods require answering a
prearranged question (or some out-of-band method of authentication),
and both methods require immediate changing of the password, a link is
no more secure than sending the temporary password.  In fact, a link may
eventually lead to *less* security as it's easier to phish people if
legitimate messages include a link rather than not including a link.
Encouraging people not to click links in messages like legitimate
password recovery emails is a Good Thing, IMO, as it'll make them less
likely to click links in fake ones.

I realize I'm tilting at windmills.

Regards,

Dianne.

Re: Direct download link detection - new variant

[Dianne Skoll]

On Wed, 26 Jul 2017 08:28:52 -0700 (PDT)
John Hardin  wrote:

> ...all of which is, sadly, whack-a-mole.

However, there are few to no alternatives to whack-a-mole for this
spam run.  The messages are pretty bland.

We've been diligently adding the URLs to our phishing list and we seem
to have caught most of them; there are only a couple of hundred or so
URLs.

Regards,

Dianne.

Re: Direct download link detection - new variant

[Dianne Skoll]

On Wed, 26 Jul 2017 17:15:43 +0200
Michael Storz  wrote:

[...]

> /boundary="-{4}=_NextPart_000_[0-9A-F]{4}_[0-9A-F]{8}\.[0-9A-F]{8}"/

You may get FPs.  See for example 
https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails==sk105578

I am guessing that boundary is generated by a library that's also used
for legitimate purposes.

Regards,

Dianne.

Re: Direct download link detection - new variant

[Dianne Skoll]

On Tue, 25 Jul 2017 08:36:22 -0400
Dianne Skoll <d...@roaringpenguin.com> wrote:

> All of the URLs match this pattern:

> /\/[A-Z]{4}\d{6}\/$/

We see a new variant with the subject "Your Virgin Media bill is ready" and
URLs that match:

uri__RP_D_00108_03 /\/\d{12}\/[A-Z]{6}\/?$/

Regards,

Dianne.

Re: Direct download link detection

[Dianne Skoll]

On Tue, 25 Jul 2017 13:15:33 +0100
RW  wrote:

> https://pastebin.com/p7EnFNf7

We've seen lots of those and collected a few dozen unique URLs for our
URL blacklists.  I added a swath of them to the APER project in this
commit:

https://sourceforge.net/p/aper/code/11830/

All of the URLs match this pattern:

/\/[A-Z]{4}\d{6}\/$/

(The leading/trailing Perl slash delimiters are included, but not part of
the pattern.)

so I think a URL rule can catch these.  Doing a HEAD on the URL
gives Content-Type: application/msword.  I think it's most likely safe
to do HEAD requests, but definitely not GET as others have mentioned.

Regards,

Dianne.

Re: ramsonware URI list

[Dianne Skoll]

My only comment on this is that shell scripting is a completely inappropriate
language to use for this.  Use a real language like Perl, Python, Ruby, or
whatever.

Regards,

Dianne.

Re: "bout u" campaign

[Dianne Skoll]

On Wed, 12 Jul 2017 21:04:37 -0400
Alex  wrote:

> Has anyone else experienced a spam campaign with any one of the
> following subjects:

> - sometimes enjoy it wild, how bout you?
> - sometimes like it ruff, what bout you?
> - sumtimes enjoy it ruff, wat bout you?

144 hits, all of them except one on Tuesday, 11 July.  All
whacked very handily by Bayes.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Thu, 15 Jun 2017 10:44:03 -0400
Dianne Skoll <d...@roaringpenguin.com> wrote:

> Hi, Kevin,

I did not realize I was replying to the list. :P  Newbie mistake...

Anyway, this is for the list.  Feature idea is up at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7436

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

Hi, Kevin,

> Diane, I'd appreciate if you would synopsize things to the best of
> what you see from the conversation and add it as a feature request in 
> bugzilla bug.  https://bz.apache.org/SpamAssassin/

Sure.  I've forgotten my Bugzilla password, so just waiting for the
reset token to arrive and then I'll add the feature idea.

> If you have even a rough framework patch to start, that will help too.

Unfortunately not... it's all just an idea at the moment.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Wed, 14 Jun 2017 13:38:49 +0100
RW  wrote:

> The way I suggested has advantages:

>  - there's no need to clean-out expired rules manually

A minor advantage at best.  And nothing stops you from keeping your
expiring rules in a separate .cf file so they can be auto-purged.

>  - it's clear which rules are permanent, temporary and expired

No advantage at all; it's clear with the tflags expiry= method.

>  - grep can easily separate the three classes

No advantage for your method.

>  - expired body rules aren't compiled by sa-compile

So what?  Does that really matter given that probably only a tiny
percentage of rules will have expiries?

>  - it doesn't complicate the core modules.

:) Have you read the code?  SpamAssassin is pretty complex as it is.
This is a very minor addition.

The benefits of a standard, supported, maintained way to do expiry far
outweigh everything else.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Wed, 14 Jun 2017 00:52:15 +0100
RW  wrote:

> If you want it to work that way it can be done in an external script
> in about 10 lines.

*SIGH*

Yes.  I'm perfectly aware of that.

My point is that we can have hundreds of sysadmins writing hacky little
scripts that all do things slightly differently and making up directory
hierarchies of their own that mean nothing to anyone else, OR we could
have a standard, supported, maintained way to do it in SpamAssassin.

> If you want a rule, or rules, to expire on a specific day put it in a
> file that starts with "YYMMDD-" in a separate directory. The script
> just needs to deal with the expired files, warn about any badly named
> files and generate a file containing a list of include lines.

Why not do it properly, once, so everyone can benefit?  And not put the
burden on sysadmins who have to document all these little hacks for their
successors?

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Tue, 13 Jun 2017 23:10:25 +0100
RW  wrote:

> Then why not write a script to parse your logs and determine when that
> happens.

Because that's more work, and I'm lazy, just like all true sysadmins.

> > What if we did something like:
> > expire MYRULE_FOO 2017-09-01

> This seems sub-optimal to me.

How so?  If you make a rule that you *know* will be effective for a pretty
limited timespan, setting the expiry at the time of creation seems more
efficient to me than having to remember to go back and expire it.

Of course, if you prefer to do the work, then you don't have to use
the expiration feature.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Tue, 13 Jun 2017 14:38:21 -0500
Noel  wrote:

> Maybe expired rules could automatically score as 0.01 rather than
> invalid.   Then log a warning to remind the admin.

No, I don't like that.  As others mentioned, that does nothing for dependent
rules.  I think a sensible use case for this would be:

1) It's meant for site rules, not the core rules or any other set of
automatically-updated or widely-distributed rules.

2) If a rule has an expiry set and then is used to build a meta rule,
then the expiry is ignored and the parser issues a warning or even a
fatal error.  I'm partial to the fatal error because warnings are usually
ignored. :)

3) Expired rules are removed from the list if already in memory or completely
ignored at parse time.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Tue, 13 Jun 2017 11:56:57 -0400
"Kevin A. McGrail"  wrote:

> Brilliant idea but how to keep that information from spammers?

Would it matter?  Especially for private site rules.  I wouldn't advocate
this for centrally-distributed rules, which are in any event expired out by
removing the rules.

Regards,

Dianne.

Re: Feature idea: Expiring rules

[Dianne Skoll]

On Tue, 13 Jun 2017 08:59:27 -0700 (PDT)
John Hardin  wrote:

> Dependencies.

Yes, that would mess things up.  Probably shouldn't be able to expire
rules that others depend on.  The parser could check for that and make
them non-expiring (with a warning.)

Regards,

Dianne.

Feature idea: Expiring rules

[Dianne Skoll]

Hi,

Something I and possibly others might find useful would be rules that
expire.  Quite often, we might make some very specific rules to handle
a particular spam run and they lose their effectiveness pretty quickly.
What if we did something like:

expire MYRULE_FOO 2017-09-01

or maybe

tflags MYRULE_FOO expire=2017-09-01

Then the file parser would ignore expired rules, and if expired rules
have already been parsed into memory from before they expired, the run-time
would skip them.

Thoughts?

Regards,

Dianne.

Re: Somewhat OT: DMARC and this list

[Dianne Skoll]

On Fri, 19 May 2017 12:00:29 -0700
Alan Hodgson  wrote:

> This is actually one of the few mailing lists that a DMARC p=reject
> domain can send anything to. Assuming they DKIM-sign their mail, of
> course.

Yep.

> I would argue that setting a DMARC p=reject policy without working
> DKIM is fundamentally broken idea on the sender's part.

Seconded.  The gluing of SPF onto DMARC is a mess.

Regards,

Dianne.

Re: Somewhat OT: DMARC and this list

[Dianne Skoll]

On Fri, 19 May 2017 20:43:39 +0200
Benny Pedersen  wrote:

> some maillists break DKIM, forkus on that first, not last !

Thank you for not adding any value to the conversation.  The
domain in question is not using DKIM.

Regards,

Dianne.

Somewhat OT: DMARC and this list

[Dianne Skoll]

Hi,

Tons of list traffic keeps getting quarantined because of DMARC.  For
example, a recent message from David Jones :

DMARC policy for domain ena.com suggests Rejection as
DMARC_POLICY_REJECT, but quarantined due to rule settings

$ host -t txt _dmarc.ena.com
_dmarc.ena.com descriptive text "v=DMARC1\; p=reject\; sp=reject\; 
rua=mailto:dm...@ena.net\;;

(In this instance, we've overridden the DMARC policy and converted it
to quarantine instead of reject, so I was able to retrieve the email, but...)

I'm pretty sure Mailman can do DMARC-munging.  Can ezmlm do the equivalent
of Mailman's "ALLOW_FROM_IS_LIST" feature?

Regards,

Dianne.

Blocking administrivia (was Re: Unsubscribe)

[Dianne Skoll]

On Wed, 5 Apr 2017 12:26:56 -0700
Bret Miller  wrote:

[...]

> I would never send an unsubscribe to a list because I know it's bad 
> etiquette, just like I know that typing in all caps is "shouting".
> But people do it all the time without meaning to offend anyone.

I wonder if technology exists that could detect and stop such messages?

Wouldn't such technology be amazing?  If only someone would come up with
it.

:P

Regards,

Dianne.

Re: Matching To and Received addresses

[Dianne Skoll]

On Tue, 28 Mar 2017 19:49:10 +0100
Markus  wrote:

> Honestly didn't even think of mailing lists such as this, nor BCC
> (don't deal with BCC emails very much to be honest).

> Though, would you not be able to test against the bottom most
> Received header compared to the To: header?

The "for..." clause is optional and a lot of MTAs don't add it.
Almost all MTAs will refuse to add it if it's for more than
one local recipient.

Regards,

Dianne.

Re: Matching To and Received addresses

[Dianne Skoll]

On Tue, 28 Mar 2017 19:04:44 +0100
Markus  wrote:

> How likely is it to be in legitimate mail? Highly unlikely (if ever),
> so you'd be pretty safe outright rejecting mail that behaves this
> way, to be honest.

You'd reject every single message in this mailing list if you did that.

Regards,

Dianne.

Re: New whitelisting trick using from and spf

[Dianne Skoll]

On Tue, 7 Mar 2017 00:04:59 +
David Jones  wrote:

> >Er... well.  The envelope-from is not any more trustworthy than
> >the header From:.  But it *is* the thing the SPF spec say to check,
> >and *not* the header From:.

> It should be way more trustworthy since it is where bounces go.

You assume that someone who is willing to forge a sender address
(whether envelope or header) is going to be fastidious about receiving
bounces? :)

> Many MTAs can do DNS checks (make sure it exists in DNS) plus
> DBL checks against the envelope domain.  Regular user mailboxes
> where compromised accounts come from usually don't/can't spoof
> the envelope-from.  It's definitely more reliable which is why the
> SPF spec chose to use it.

No, that's not true.  It's no more "reliable" than anything else.  In
fact, in the entire SMTP transaction, there's only one set of email
addresses that are reliable, and those are the RCPT To: addresses.

SPF chose to use envelope sender not because it's more reliable, but
(I suspect) so as not to break mailing lists.

Anyway: We see millions of spams per day.  Tons of them have spoofed
envelope sender addresses and tons have spoofed From: header addresses.

Regards,

Dianne.

Re: New whitelisting trick using from and spf

[Dianne Skoll]

On Mon, 6 Mar 2017 23:22:00 +
David Jones  wrote:

[...]

> Not good.  SPF should be checked against the envelope-from
> address which is more trustworthy.

Er... well.  The envelope-from is not any more trustworthy than
the header From:.  But it *is* the thing the SPF spec say to check,
and *not* the header From:.

Regards,

Dianne.

Re: New whitelisting trick using from and spf

[Dianne Skoll]

On Mon, 6 Mar 2017 11:58:25 -0600 (CST)
David B Funk  wrote:

> But that won't help you when the scammers set the user visible from
> as "acco...@paypai.com" or some other variant (with the actual
> address part as  or something else.

I recall someone bringing this up on the DMARC discussion list. :)
http://lists.dmarc.org/pipermail/dmarc-discuss/2012-February/000206.html

> user-agents (such as OutHouse) by default only show the "comment"
> part of the address and hide the actual <> address part, making it
> easy for scammers to fool the non-tech savvy users.

Way back, I proposed recommending that MUAs that show DMARC results also
show the domain name in brackes, similar to what slashdot.org does
for URLS.

So if you have an email with this header:

From: "acco...@paypal.com" 

The MUA should display the From: field as:

acco...@paypal.com [example.org]

That suggestion was deemed out of scope for the DMARC standard, so... bleh.

I really don't know a good way around this.  Maybe if an email passes
DMARC but has a comment field that looks like an email address, the
comment fieldd should be suppressed?  ("Looks like" should probably
be as generic as "contains an @ sign")

Regards,

Dianne.

Re: New whitelisting trick using from and spf

[Dianne Skoll]

On Sun, 5 Mar 2017 10:38:09 -0800
Marc Perkel  wrote:

> If the from address is whitelisted AND the SPF of the from address is 
> good - I pass the email.

And that's exactly how SPF is supposed to work.  You shouldn't whitelist
domains willy-nilly because they can be spoofed, and you shouldn't allow
SPF "pass" messages through without spam-scanning because spammers set
up SPF records, but the combination of a domain you trust and an SPF "pass"
is a definite ham indicator.

We also ignore end-user-created whitelists on a domain or sender if there's an
SPF "fail" or "softfail".

Regards,

DIanne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Fri, 24 Feb 2017 16:26:38 -0500
Alex  wrote:

> We've actually had false-positives due to how the list is built into
> rules. In other words, "i...@ca.com" is still on the list from 2011.
> They're also not bounded by default, so noi...@ca.com and
> morei...@ca.com would also be caught, for example.

We use MIMEDefang's Perl integration, so we don't compile the list into
SA rules.  We use a database lookup instead, which does not suffer
from the partial-match problem, and we don't load anything older than 6
months into the database.

> How do you build the phishing URLs list into rules similar to how the
> addresses2spamassassin.pl does for the phishing emails?

Same idea; we do it in Perl integration code around SpamAssassin with
a database lookup.

Regards,

Dianne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Fri, 24 Feb 2017 18:07:50 +
RW  wrote:

> > OK.  Any FPs, though?  That's the other half of the test.

> No, but it's pretty unlikely there would be. 

Actually, it's very likely there will be a lot of FPs, but it's also
very likely that any given user of the list won't see them.  That's
because when someone's email address gets compromised and then the
system administrator clears it up, the only recipients to suffer
false-positives are those with whom the sender would normally
correspond.

We have seen a few of these cases happen.

> It seems like a lot of hassle for little benefit.

The APER doesn't catch all that much, nor do the known-phishing URLs catch
much, but every little bit helps.

As a data point, one of our installations scanned 4 million messages
yesterday.  Of those, only 262 hit our known-phishing URL list (which
uses APER and additional sources) and 155 hit APER's known-phishing
email address list.

But maybe those few hundred were really worth stopping because they
prevented phishing attacks.  Who knows?

Regards,

Dianne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Wed, 22 Feb 2017 20:14:33 +
RW  wrote:

> FWIW I ran that list against 3k spams received from late 2015
> onwards. I got 2 hits on 2 separate addesses both timestamped with
> 2012.

OK.  Any FPs, though?  That's the other half of the test.

Regards,

Dianne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Wed, 22 Feb 2017 08:45:07 +
Vincent Fox  wrote:

> Come on, look at the datestamps on the addresses in that list!
> Plenty from 2009.

The reason they datestamp the addresses is so that sites making use of
the list can determine on their own when data is stale enough to ignore.

I do agree that they should auto-remove anything older than about 90
days since some sites using the list probably ignore the datestamp.
I will bring it up with the admins.

Regards,

Dianne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Mon, 20 Feb 2017 14:21:08 -0500
Alex  wrote:

> Maybe we're using something different. This is the link I was using to
> download the phishing addresses until the other day, when it became a
> dead link:

> https://aper.svn.sourceforge.net/svnroot/aper/phishing_reply_addresses

That URL works for me.  However, I am currently pulling the SVN repo from
svn://svn.code.sf.net/p/aper/code (also can use 
http://svn.code.sf.net/p/aper/code)

It looks like the list of addresses has not been updated since 2017-02-16, but
the list of phishing URLs has an entry dated 2017-02-20.

Regards,

Dianne.

Re: Google anti-phishing code project

[Dianne Skoll]

On Sun, 19 Feb 2017 12:21:14 -0500
Alex  wrote:

> https://code.google.com/archive/p/anti-phishing-email-reply/
> It appears to no longer be active, as some time yesterday.

It's still active.  The most recent commit is dated today, and I still
have commit privileges.

Regards,

Dianne.

Re: Filtering outbound mail

[Dianne Skoll]

On Thu, 16 Feb 2017 10:07:46 +
David Jones  wrote:

> Would it make sense for me to setup/manage my own custom
> rules for checking the To: header or could the FreeMail plugin
> be extended to add new rules like FREEMAIL_TO?

The To: header may not contain useful information.  I don't think
the usual spam-filtering techniques are appropriate for blocking
internal abusers; I think you want to apply some sort of rate-limiting
that blocks senders (possibly domains and IP addresses) that exceed some
number of recipients per hour.

It's not trivial to set this up, unfortunately.

Regards,

Dianne.

Re: New type of monstrosity

[Dianne Skoll]

Ruga  wrote:

> RFC-822 is the e-mail standard, without "group addresses". What we do
> complies with the standard.

You are wrong.  Wrong, wrong, wrong, wrong.

Take a look at RFC-822: https://www.ietf.org/rfc/rfc0822.txt

Go to Section 6. ADDRESS SPECIFICATION.  Look at Section 6.1.

Here's a copy/paste:

 address =  mailbox  ; one addressee
 /  group; named list

 group   =  phrase ":" [#mailbox] ";"


Oh look!  The group address specification!  In RFC 822!  Amazing!

Ruga, my dear fellow, (or lady... I can't tell), stop digging yourself
in deeper.

Regards,

Dianne.

Aiieee, stop it! (was Re: RFC compliance pedantry (was Re: New type of monstrosity))

[Dianne Skoll]

On Thu, 09 Feb 2017 08:21:28 -0500
Ruga  wrote:

[nonsense]

I thought I'd take this opportunity to remind everyone of my Perl package
http://search.cpan.org/~dskoll/Mail-ThreadKiller-1.0.1/lib/Mail/ThreadKiller.pm

Regards,

Dianne.

Re: RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On Thu, 09 Feb 2017 03:44:24 -0500
Ruga  wrote:

> Proper snail mail and e-mail have addresses. Those who do not, are
> quickly archived in the trashcan. This is what we do, and it works.

We get it.

I'm overcome with delight that you are implementing the mail policy
that you like.  It warms my heart... you have no idea.

But please don't claim you're doing it in the name of RFC compliance.

Regards,

Dianne.

Re: RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On February 9, 2017 3:41:32 AM EST, Ruga  wrote:

>Let see who can read amon us.

You spelled "among" incorrectly.

>What is your highest level of formal education?

Um?  None of your business?

Master's degree, if you must know.

-- Dianne 

Re: RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On Wed, 08 Feb 2017 09:01:35 -0500
Ruga  wrote:

> How odd, in a mailing list of spam fighters someone really wants me
> to accept junk mail.

Wow.  You really don't know how to read, do you?  What was unclear
about my statement:

Hey, you do you. You can do whatever you want with your mail, but
claiming it's in the name of RFC compliance is alternatively factual.

> In the snail mail box, we put in the trashcan everything that does
> not carry a recipient address. Guess what? We do the same with
> e-mail. And we are happy about it.

You can do whatever you want.  But don't spread misinformation about
standards.  We have to deal with enough crappy noncompliant software.
We don't need Internet vigilantes on a witch-hunt against software
that actually *does* comply with standards.

Regards,

Dianne.

Re: RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On Wed, 08 Feb 2017 07:16:48 -0500
Ruga  wrote:

> It is precisely because I am responsible for other persons that I
> make such rules based upon the RFC standard,

No, you don't.  You make the rules based on your misreading of RFC 822.

RFC 822 permits this header:

To: undisclosed recipients:;

End of story.

Regards,

Dianne.

Re: RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On Tue, 07 Feb 2017 18:33:49 -0500
Ruga  wrote:

> I follow the actual RFC standard, not the proposed revisions.

No you don't.  You follow your misunderstanding of the actual standard.
RFC822 permits group syntax.  It's right in the ABNF.  Learn to read
carefully.

Here's a hint, taken directly from https://www.ietf.org/rfc/rfc0822.txt

 address =  mailbox  ; one addressee
 /  group; named list
 group   =  phrase ":" [#mailbox] ";"

> The To From and Cc fields are defined by a grammar AND a natural
> language description. Such fields MUST hold addresses,

Wrong.

> Bear in mind the state of corruption we live in.

I know.  It's awful that would-be pedants don't even read properly.

> On the subject length, although the RFC standard did not foresee the
> abuse, it did speak about the intended purpose of the field. If it
> does not fit the one line of 78 (ASCII) characters, it bounches back
> to the sender.

Well, you know, for someone who only follows STANDARDS, you're making stuff
up.  There's no mention whatsoever of line length limits in the STANDARD
RFC 822.  Those are only in the proposed revisions, which you disdain.
Or are you selective about picking and choosing from proposed revisions?

Oh and by the way, I certainly hope your mail server does not speak
ESMTP.  That's not a standard, you know.

> I understand that sloppy e-mail software allows
> spammers to send the library of congress inside a subject field, but
> rest assured that I such abuses do not survive my filters, even if
> Trump himself will allow for it with a presidential decree.

Hey, you do you.  You can do whatever you want with your mail, but claiming
it's in the name of RFC compliance is alternatively factual.

Regards,

Dianne.

RFC compliance pedantry (was Re: New type of monstrosity)

[Dianne Skoll]

On Tue, 07 Feb 2017 02:57:06 -0500
Ruga  wrote:

> > To: undisclosed recipients: ;

> The To header is not RFC compliant.

Yes it is.  RFC 5322 even gives the header Cc: undisclosed recipients: ;
as an example in Appendix A.1.3, Group Addresses.

> The Subject header exceeds the
> maximum line length, being another RFC constraints.

Nope.  It's around 720 characters, less than the 998 maximum.  And
while it's true that it's not wrapped at 78 characters, that's a SHOULD
requirement, not a MUST requirement.

Anyway, we see a few of these every so often.  Bayes usually disposes
of them quite handily.

Regards,

Dianne.

The nice thing about standards (was Re: Legit Yahoo mail servers list)

[Dianne Skoll]

On Mon, 30 Jan 2017 09:06:34 -0500
Rob McEwen  wrote:

> On 1/30/2017 8:54 AM, Matus UHLAR - fantomas wrote:
> > they do and it has been mentioned:
> > https://help.yahoo.com/kb/SLN23997.html

Cool.  So Yahoo uses an HTML page that's a pain to process by
computer.  Microsoft has 
https://support.content.office.net/en-us/static/O365IPAddresses.xml,
which at least is XML.  And Google, so far as I can see, can be mined by
recursively expanding _spf.google.com.

Yay standards...

Regards,

Dianne.

Re: Legit Yahoo mail servers list

[Dianne Skoll]

On Mon, 30 Jan 2017 13:40:26 +
David Jones  wrote:

> My goal in whitelisting Yahoo servers is to make sure these
> messages get to MailScanner where they are not whitelisted
> and are scores based more on content by Spamassassin rather
> than sender reputation (DNSBLs).

OK, understood now.

I would always err on the side of more flexible filtering rather than
conserving server resources, and I'd use a filter flexible enough to a
avoid an RBL lookup on an SPF "pass" for yahoo.com.  But I understand
that others have different optimization goals.

Regards,

Dianne.