Re: Link following leads to redirect

2017-12-28 Thread RW 
On Thu, 28 Dec 2017 11:18:10 -0500
Olivier Coutu wrote:

> On 2017-12-27 17:07, RW wrote:

> > There is a  DecodeShortURLs plugin that follows known shortners like
> > bit.ly, tinyurl etc, and  adds the final URI to the list.
 
> I was thinking of trying DecodeShortURLs, but all these
> considerations are making me think twice.

AFAIK the only problem with DecodeShortURLs is the extra round trips.
It only uses known redirects, so it never actually accesses spammer
controlled servers or anything that might have side-effects.

Re: Link following leads to redirect

2017-12-28 Thread RW 
On Wed, 27 Dec 2017 22:07:23 +
RW wrote:


> I do use it but I have put much effort into determining how well it
> works.

s/have/haven't/

Re: Link following leads to redirect

2017-12-28 Thread Anthony Cartmell 
Yes, following links in email messages breaks "one-time login" links for things 
like forgotten password messages. If the mail scanning requests the link then 
the human recipient can't.

I've seen this problem, and it took a while to work out what was going on... 


On 27 December 2017 18:33:21 GMT+00:00, Pedro David Marco 
 wrote:
> 
>... there are also "one time links", that vanish once
>visisted/downloaded. 
>
>
>PedroD

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Link following leads to redirect

2017-12-27 Thread RW 
On Wed, 27 Dec 2017 10:42:02 -0500
Alex wrote:

> Hi, is there anything available that can follow a link to either test
> it itself for its reputation or RBL, or somehow add points to an email
> that contains a link that just redirects?
> 
> http://d9na.abidjanjeu.com/neabi0HYOsudvVB09j2GCA9rjE4ldYHQs1hHd7lpAHJn9%2B%2Fb994oe5aUUN8Kea%2F48EZmtBFyJ1VKJvAHYB2LithFy1w%3D%3Ddjat38r
> 
> I realize redirects are not unique to spam, but it's commonly used as
> a way to point to a single actual site used by spammers.

There is a  DecodeShortURLs plugin that follows known shortners like
bit.ly, tinyurl etc, and  adds the final URI to the list.

I do use it but I have put much effort into determining how well it
works.

Re: Link following leads to redirect

2017-12-27 Thread Alex 
On Wed, Dec 27, 2017 at 1:52 PM, Dianne Skoll  wrote:
> On Wed, 27 Dec 2017 19:21:32 +0100
> Reindl Harald  wrote:
>
>> > At most, I would do a HEAD on a URL and not a GET.  HEAD is
>> > probably safer and will usually tell you if the link is a redirect
>
>> no, for the web application it's typically transparent because the
>> whole purpose of HEAD is that you get the whole headers as you would
>> do with a GET request without the body
>
> That's true.  I was mistaken about "safer".

I think also what many of the antispam vendors do is wrap each URL in
their own redirect, which is then expanded and evaluated at click
time.

Re: Link following leads to redirect

2017-12-27 Thread Dianne Skoll 
On Wed, 27 Dec 2017 19:21:32 +0100
Reindl Harald  wrote:

> > At most, I would do a HEAD on a URL and not a GET.  HEAD is
> > probably safer and will usually tell you if the link is a redirect  

> no, for the web application it's typically transparent because the
> whole purpose of HEAD is that you get the whole headers as you would
> do with a GET request without the body

That's true.  I was mistaken about "safer".

Regards,

Dianne.

Re: Link following leads to redirect

2017-12-27 Thread Pedro David Marco 
 
 ... there are also "one time links", that vanish once visisted/downloaded. 


PedroD

Re: Link following leads to redirect

2017-12-27 Thread Dianne Skoll 
On Wed, 27 Dec 2017 12:47:00 -0500
Alex  wrote:

> It [fetching URLs] would also probably lead to inadvertently
> unsubscribing people from mailing lists.

Yes, if the lists use badly-written mailing list software.

At most, I would do a HEAD on a URL and not a GET.  HEAD is probably safer
and will usually tell you if the link is a redirect.  You also want
to fake the user-agent to be a common Windows browser because some
malware servers look at the User-Agenet and return a 404 if they think
the client is not a real Web browser.

Even a HEAD can be dangerous; there's an Internet "security" [sic] company
out there that shall remain nameless; these geniuses view HEAD requests
as attacks and report you to your ISP.  It took me 2+ weeks to sort out
their BS "abuse" complaints.

> I'd like to think some intelligence could be built into such a system,
> and know many of the spam companies like Symantec and Mimecast are
> doing this to differing degrees.

I know of one company that collects URLs and has a central server farm that
analyzes them (ie, the URL fetching is done on a completely different
set of machines than the spam filtering.)  They have all kinds of heuristics
and special-case code to make it relatively safe.

Regards,

Dianne.

Re: Link following leads to redirect

2017-12-27 Thread Alex 
Hi,

On Wed, Dec 27, 2017 at 11:15 AM, Dianne Skoll  wrote:
> On Wed, 27 Dec 2017 07:50:38 -0800 (PST)
> John Hardin  wrote:
>
>> > Hi, is there anything available that can follow a link to either
>> > test it itself for its reputation or RBL, or somehow add points to
>> > an email that contains a link that just redirects?
>
>> That's unfortunately a way to trigger tracking bugs.
>
> Additionally, it's not clear to me you'd want your mail server's IP
> to appear in the web logs of potentially dodgy websites.  The last thing
> you need is police showing up with a warrant because your IP has visited
> a criminal site.

It would also probably lead to inadvertently unsubscribing people from
mailing lists.

I'd like to think some intelligence could be built into such a system,
and know many of the spam companies like Symantec and Mimecast are
doing this to differing degrees.

Re: Link following leads to redirect

2017-12-27 Thread Dianne Skoll 
On Wed, 27 Dec 2017 07:50:38 -0800 (PST)
John Hardin  wrote:

> > Hi, is there anything available that can follow a link to either
> > test it itself for its reputation or RBL, or somehow add points to
> > an email that contains a link that just redirects?

> That's unfortunately a way to trigger tracking bugs.

Additionally, it's not clear to me you'd want your mail server's IP
to appear in the web logs of potentially dodgy websites.  The last thing
you need is police showing up with a warrant because your IP has visited
a criminal site.

Regards,

Dianne.

Re: Link following leads to redirect

2017-12-27 Thread John Hardin 

On Wed, 27 Dec 2017, Alex wrote:


Hi, is there anything available that can follow a link to either test
it itself for its reputation or RBL, or somehow add points to an email
that contains a link that just redirects?

http://d9na.abidjanjeu.com/neabi0HYOsudvVB09j2GCA9rjE4ldYHQs1hHd7lpAHJn9%2B%2Fb994oe5aUUN8Kea%2F48EZmtBFyJ1VKJvAHYB2LithFy1w%3D%3Ddjat38r

I realize redirects are not unique to spam, but it's commonly used as
a way to point to a single actual site used by spammers.


That's unfortunately a way to trigger tracking bugs.

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Watch... Wallet... Gun... Knee...-- Denny Crane
---
 272 days since the first commercial re-flight of an orbital booster (SpaceX)

Link following leads to redirect

2017-12-27 Thread Alex 
Hi, is there anything available that can follow a link to either test
it itself for its reputation or RBL, or somehow add points to an email
that contains a link that just redirects?

http://d9na.abidjanjeu.com/neabi0HYOsudvVB09j2GCA9rjE4ldYHQs1hHd7lpAHJn9%2B%2Fb994oe5aUUN8Kea%2F48EZmtBFyJ1VKJvAHYB2LithFy1w%3D%3Ddjat38r

I realize redirects are not unique to spam, but it's commonly used as
a way to point to a single actual site used by spammers.