Re: HTTPS_HTTP_MISMATCH and explanation

[RW]

On Mon, 26 Sep 2016 09:16:43 -0400
Joe Quinn wrote:

> On 9/26/2016 8:54 AM, RW wrote:
> > Informational rules do that, but IIRC __RULES are simply a special
> > case.
> >  
> Hmm, you're probably right on that point. I can't find anything in
> the source that behaves that way, but the documentation claims that's
> how it works and I also don't see anything to support being scored
> 0.001 either.

It's a long time since I looked it, but I think the way it works is that
these are the only rules that don't have a default score. The scores are
undefined so they don't contribute to the result, but they aren't
actually 0, so they still run.

Re: HTTPS_HTTP_MISMATCH and explanation

[Joe Quinn]

On 9/26/2016 8:54 AM, RW wrote:

Informational rules do that, but IIRC __RULES are simply a special
case.

Hmm, you're probably right on that point. I can't find anything in the 
source that behaves that way, but the documentation claims that's how it 
works and I also don't see anything to support being scored 0.001 either.

Re: HTTPS_HTTP_MISMATCH and explanation

[RW]

On Mon, 26 Sep 2016 08:26:27 -0400
Joe Quinn wrote:

> On 9/25/2016 9:25 PM, Sean Greenslade wrote:
> > On Sun, Sep 25, 2016 at 07:57:37PM -0400, Alex wrote:  
> >> I think the rule still has a use, perhaps in a meta or something.  
> > I believe (though don't quote me on this) that a zero-weight rule
> > will still be checked if it's used as part of a metarule.
> >
> > --Sean  
> A rule that's weighted exactly zero will never fire. The way __RULES
> get around this is by being scored 0.001.

Informational rules do that, but IIRC __RULES are simply a special
case.

Re: HTTPS_HTTP_MISMATCH and explanation

[Joe Quinn]

On 9/25/2016 9:25 PM, Sean Greenslade wrote:

On Sun, Sep 25, 2016 at 07:57:37PM -0400, Alex wrote:

I think the rule still has a use, perhaps in a meta or something.

I believe (though don't quote me on this) that a zero-weight rule will
still be checked if it's used as part of a metarule.

--Sean
A rule that's weighted exactly zero will never fire. The way __RULES get 
around this is by being scored 0.001.

Re: HTTPS_HTTP_MISMATCH and explanation

[Sean Greenslade]

On Sun, Sep 25, 2016 at 07:57:37PM -0400, Alex wrote:
> I think the rule still has a use, perhaps in a meta or something.

I believe (though don't quote me on this) that a zero-weight rule will
still be checked if it's used as part of a metarule.

--Sean

Re: HTTPS_HTTP_MISMATCH and explanation

[Alex]

Hi,

> I had to add a rule weight for this rule to get it to trigger, so
> evidently the latest ML rules weights have disabled this rule.
>
> Anyway, this is what triggered it:
>
>>https://www.google.com/url?q=3Dhttps%=
>> 3A%2F%2Fglobal.gotomeetinA.com%2Fjoin%2F726265509sa=3DDusd=3D2=
>> p;usg=3DAFQjCNHlVtBtL2J4tx-l3Ej-YPjED9EKjA" target=3D"_blank">https://globa=
>> l.gotomeetinB.com/join/726265509
>
> (Note that I altered the domains so that I could tell which triggered
> the rule.)

Thanks so much for your efforts.

> Now I've given a more thorough analysis to that code (I'm not a perl
> programmer by any means), I realized what it's actually doing is
> comparing the _domain_ of the text to the domain of the anchor. If the
> two don't match (e.g. the link text is for gotomeeting.com but the link
> goes to google.com), the rule triggers. The http/https is misleading.

I thought there were already other rules that perform similar
functions, although I can't remember their names right now. Maybe
there is a perl programmer with ideas and interest in improving
this...

> Unfortunately, it's all too common nowadays for emails to include a link
> click-through redirect domain hidden in the anchor tag. I personally
> hate this, but it can't really be considered a sign of spam anymore
> since too many legitimate emails do it.

This definitely is a common practice.

> I would probably zero-weight this rule.

I think the rule still has a use, perhaps in a meta or something.

Re: HTTPS_HTTP_MISMATCH and explanation

[Sean Greenslade]

On Sun, Sep 25, 2016 at 04:51:20PM -0400, Alex wrote:
> On Sun, Sep 25, 2016 at 4:41 PM, Sean Greenslade
>  wrote:
> > On Sun, Sep 25, 2016 at 03:54:53PM -0400, Alex wrote:
> >> > If you want to see what that rule's code looks like, here's a link:
> >> >
> >> > https://fossies.org/dox/Mail-SpamAssassin-3.4.1/classMail_1_1SpamAssassin_1_1Plugin_1_1HTTPSMismatch.html
> >> >
> >> > It's possible there is a bug in that rule. If you send it through
> >> > SpamAssassin with debug enabled, the rule should print out the domain
> >> > pairs that trigger it. Maybe try that, and see if what it outputs makes
> >> > sense.
> >>
> >> I should have mentioned that I tried that - it doesn't.
> >
> > If you don't mind sending the entire email, I'm curious now.
> 
> http://pastebin.com/XcGzedNk
> 
> This one also hits URIBL_SBL and a local BAD_TLD rule, so it may have
> been marked as spam anyway, but those two rules have also since been
> adjusted.
> 
> Thanks

I had to add a rule weight for this rule to get it to trigger, so
evidently the latest ML rules weights have disabled this rule.

Anyway, this is what triggered it:

>https://www.google.com/url?q=3Dhttps%=
> 3A%2F%2Fglobal.gotomeetinA.com%2Fjoin%2F726265509sa=3DDusd=3D2=
> p;usg=3DAFQjCNHlVtBtL2J4tx-l3Ej-YPjED9EKjA" target=3D"_blank">https://globa=
> l.gotomeetinB.com/join/726265509

(Note that I altered the domains so that I could tell which triggered
the rule.)

Now I've given a more thorough analysis to that code (I'm not a perl
programmer by any means), I realized what it's actually doing is
comparing the _domain_ of the text to the domain of the anchor. If the
two don't match (e.g. the link text is for gotomeeting.com but the link
goes to google.com), the rule triggers. The http/https is misleading.

Unfortunately, it's all too common nowadays for emails to include a link
click-through redirect domain hidden in the anchor tag. I personally
hate this, but it can't really be considered a sign of spam anymore
since too many legitimate emails do it. 

I would probably zero-weight this rule.

--Sean

Re: HTTPS_HTTP_MISMATCH and explanation

[RW]

On Sun, 25 Sep 2016 12:52:57 -0700
jdow wrote:


> These days even mixed links, some https and some simple http, is
> suspicious.

There's nothing in this for a spammer, it's just a sign of carelessness.

Re: HTTPS_HTTP_MISMATCH and explanation

[Alex]

On Sun, Sep 25, 2016 at 4:41 PM, Sean Greenslade
 wrote:
> On Sun, Sep 25, 2016 at 03:54:53PM -0400, Alex wrote:
>> > If you want to see what that rule's code looks like, here's a link:
>> >
>> > https://fossies.org/dox/Mail-SpamAssassin-3.4.1/classMail_1_1SpamAssassin_1_1Plugin_1_1HTTPSMismatch.html
>> >
>> > It's possible there is a bug in that rule. If you send it through
>> > SpamAssassin with debug enabled, the rule should print out the domain
>> > pairs that trigger it. Maybe try that, and see if what it outputs makes
>> > sense.
>>
>> I should have mentioned that I tried that - it doesn't.
>
> If you don't mind sending the entire email, I'm curious now.

http://pastebin.com/XcGzedNk

This one also hits URIBL_SBL and a local BAD_TLD rule, so it may have
been marked as spam anyway, but those two rules have also since been
adjusted.

Thanks

Re: HTTPS_HTTP_MISMATCH and explanation

[Sean Greenslade]

On Sun, Sep 25, 2016 at 03:54:53PM -0400, Alex wrote:
> > If you want to see what that rule's code looks like, here's a link:
> >
> > https://fossies.org/dox/Mail-SpamAssassin-3.4.1/classMail_1_1SpamAssassin_1_1Plugin_1_1HTTPSMismatch.html
> >
> > It's possible there is a bug in that rule. If you send it through
> > SpamAssassin with debug enabled, the rule should print out the domain
> > pairs that trigger it. Maybe try that, and see if what it outputs makes
> > sense.
> 
> I should have mentioned that I tried that - it doesn't.

If you don't mind sending the entire email, I'm curious now.

--Sean

Re: HTTPS_HTTP_MISMATCH and explanation

[Alex]

Hi,

On Sun, Sep 25, 2016 at 3:46 PM, Sean Greenslade
 wrote:
> On Sun, Sep 25, 2016 at 03:39:20PM -0400, Alex wrote:
>> I think it must be something more than that. I've included the HTML
>> component of an FP I received, and I don't see any occurrences of an
>> https link where the text component is just http, or even vice-versa.
>>
>> http://pastebin.com/BNM9sLRL
>>
>> The HTML is a bit hard to read. Let me know if you want the whole
>> email (which is even harder, consider it's encoded, so you'd have to
>> actually run it through SA).
>
> If you want to see what that rule's code looks like, here's a link:
>
> https://fossies.org/dox/Mail-SpamAssassin-3.4.1/classMail_1_1SpamAssassin_1_1Plugin_1_1HTTPSMismatch.html
>
> It's possible there is a bug in that rule. If you send it through
> SpamAssassin with debug enabled, the rule should print out the domain
> pairs that trigger it. Maybe try that, and see if what it outputs makes
> sense.

I should have mentioned that I tried that - it doesn't.

Re: HTTPS_HTTP_MISMATCH and explanation

[jdow]

On 2016-09-25 12:39, Alex wrote:

Hi,

On Sun, Sep 25, 2016 at 3:29 PM, Sean Greenslade
 wrote:

On Sun, Sep 25, 2016 at 03:12:00PM -0400, Alex wrote:

Hi, I'm seeing quite a few FPs with HTTPS_HTTP_MISMATCH and its score
of 2.0. Isn't that kind of high for a rule that doesn't even have a
description?

Can someone explain what the rule does, and consider whether its score
should be adjusted?

Thanks,
Alex


From my quick glance over the code, it looks like that rule is meant to
trigger when a link presents its text as an https://... link, however
the actual link is to an http://... URL. Like this:

http://spammersite.com/virus;>https://www.email-service.com/login

The only place I would imagine false positives arising from this rule
would be if an email sender uses some sort of automatic link replacement
(e.g. for click-through tracking) that doesn't support https. And I
personally am inclined to agree that an email that mis-represents
insecure links as secure should be considered suspisious.

Contact the senders of the flagged emails and ask them to fix their
systems. Spam or not, that is a real problem.


I think it must be something more than that. I've included the HTML
component of an FP I received, and I don't see any occurrences of an
https link where the text component is just http, or even vice-versa.

http://pastebin.com/BNM9sLRL

The HTML is a bit hard to read. Let me know if you want the whole
email (which is even harder, consider it's encoded, so you'd have to
actually run it through SA).

Thanks,
alex


These days even mixed links, some https and some simple http, is suspicious. 
It's easy to include https links to page elements and an http link to the real 
intended payload. With http you know nothing about who sent it. With https you 
know either who sent it or who has an insecure system. Conceptually you can turn 
your pet lawyer loose on 'em. (That being the "nicest" thing I think should 
happen to them.)


{o.o}

Re: HTTPS_HTTP_MISMATCH and explanation

[jdow]

Yeah, it should have a much higher score.

{O.O}

On 2016-09-25 12:12, Alex wrote:

Hi, I'm seeing quite a few FPs with HTTPS_HTTP_MISMATCH and its score
of 2.0. Isn't that kind of high for a rule that doesn't even have a
description?

Can someone explain what the rule does, and consider whether its score
should be adjusted?

Thanks,
Alex

Re: HTTPS_HTTP_MISMATCH and explanation

[Sean Greenslade]

On Sun, Sep 25, 2016 at 03:39:20PM -0400, Alex wrote:
> I think it must be something more than that. I've included the HTML
> component of an FP I received, and I don't see any occurrences of an
> https link where the text component is just http, or even vice-versa.
> 
> http://pastebin.com/BNM9sLRL
> 
> The HTML is a bit hard to read. Let me know if you want the whole
> email (which is even harder, consider it's encoded, so you'd have to
> actually run it through SA).

If you want to see what that rule's code looks like, here's a link:

https://fossies.org/dox/Mail-SpamAssassin-3.4.1/classMail_1_1SpamAssassin_1_1Plugin_1_1HTTPSMismatch.html

It's possible there is a bug in that rule. If you send it through
SpamAssassin with debug enabled, the rule should print out the domain
pairs that trigger it. Maybe try that, and see if what it outputs makes
sense.

--Sean

Re: HTTPS_HTTP_MISMATCH and explanation

[Alex]

Hi,

On Sun, Sep 25, 2016 at 3:29 PM, Sean Greenslade
 wrote:
> On Sun, Sep 25, 2016 at 03:12:00PM -0400, Alex wrote:
>> Hi, I'm seeing quite a few FPs with HTTPS_HTTP_MISMATCH and its score
>> of 2.0. Isn't that kind of high for a rule that doesn't even have a
>> description?
>>
>> Can someone explain what the rule does, and consider whether its score
>> should be adjusted?
>>
>> Thanks,
>> Alex
>
> From my quick glance over the code, it looks like that rule is meant to
> trigger when a link presents its text as an https://... link, however
> the actual link is to an http://... URL. Like this:
>
> http://spammersite.com/virus;>https://www.email-service.com/login
>
> The only place I would imagine false positives arising from this rule
> would be if an email sender uses some sort of automatic link replacement
> (e.g. for click-through tracking) that doesn't support https. And I
> personally am inclined to agree that an email that mis-represents
> insecure links as secure should be considered suspisious.
>
> Contact the senders of the flagged emails and ask them to fix their
> systems. Spam or not, that is a real problem.

I think it must be something more than that. I've included the HTML
component of an FP I received, and I don't see any occurrences of an
https link where the text component is just http, or even vice-versa.

http://pastebin.com/BNM9sLRL

The HTML is a bit hard to read. Let me know if you want the whole
email (which is even harder, consider it's encoded, so you'd have to
actually run it through SA).

Thanks,
alex

Re: HTTPS_HTTP_MISMATCH and explanation

[Sean Greenslade]

On Sun, Sep 25, 2016 at 03:12:00PM -0400, Alex wrote:
> Hi, I'm seeing quite a few FPs with HTTPS_HTTP_MISMATCH and its score
> of 2.0. Isn't that kind of high for a rule that doesn't even have a
> description?
> 
> Can someone explain what the rule does, and consider whether its score
> should be adjusted?
> 
> Thanks,
> Alex

>From my quick glance over the code, it looks like that rule is meant to
trigger when a link presents its text as an https://... link, however
the actual link is to an http://... URL. Like this:

http://spammersite.com/virus;>https://www.email-service.com/login

The only place I would imagine false positives arising from this rule
would be if an email sender uses some sort of automatic link replacement
(e.g. for click-through tracking) that doesn't support https. And I
personally am inclined to agree that an email that mis-represents
insecure links as secure should be considered suspisious.

Contact the senders of the flagged emails and ask them to fix their
systems. Spam or not, that is a real problem.

--Sean