Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-28 Thread John Levine via mailop 
It appears that Gellner, Oliver via mailop  said:
>> That occasionally happens, but since I whitelist any /24 that has 
>> successfully retried, it doesn't happen very much.
>
>I see, this looks like a better approach. Unfortunately many greylisting 
>implementations only whitelist senders for a few days or even
>less. Or they fail to sync connections across different MTAs / regions. 

There's an unfortunate tendency for people to imagine that if X is a
useful anti-spam measure, doing more X is better, which is usually wrong. 
Greylisting everything is worse than just greylisting unknown hosts, and
a 5 second greet pause delay works as well as 5 minutes.

My greylister remembers hosts for 90 days, on the theory that if they
haven't sent mail for three months, they probably won't send mail
ever. It also has a single daemon shared among all the MTAs to track
the retries and whitelist.

I recently added a greet pause for hosts subject to greylisting and I
have to admit it works pretty well. Most of the hosts that get the
pause are early talkers. It appears that nearly all hosts that aren't
also retry. If that continues to be true, I'll just ditch the retries
since the greet pause is less disruptive.

R's,
John
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-28 Thread Gellner, Oliver via mailop 
> On 24.06.2023 at 19:38 John Levine via mailop wrote:

> According to Gellner, Oliver via mailop :
>>
>>It matters if the greylisting takes place before or after RBL / domain
>>reputation checks. If the greylisting comes first, I could imagine that the 
>>connections from most of those bots would have been blocked anyway (with a 
>>5xx response code).

> No, this is after DNSBL checks, which catch about 85% of attempted 
> connections. Those get, ah, special handling.

>>and thereby introduces noticeable breakage into email communication. As
>>a commercial ESP I wouldn’t set it up. It’s annoying for the users when
>>they repeatedly have to wait for registration emails, order
>>confirmation with payment details, when they are on the phone with
>>someone and want to discuss a document, which just doesn’t arrive at
>>the other end, and so on. Let alone cases where messages aren’t
>>delivered at all,

> That occasionally happens, but since I whitelist any /24 that has 
> successfully retried, it doesn't happen very much.

I see, this looks like a better approach. Unfortunately many greylisting 
implementations only whitelist senders for a few days or even less. Or they 
fail to sync connections across different MTAs / regions. Example from the last 
hours (redacted):

2023-06-28 06:33 170.10.152.242 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 07:45 207.211.30.242 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 08:01 91.220.42.201 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 09:04 205.139.110.141 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 09:55 205.139.110.221 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 10:13 62.140.10.22 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 10:40 91.220.42.211 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 10:41 195.130.217.241 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 10:55 91.220.42.241 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'
2023-06-28 11:10 207.211.30.221 response: 4.1.0 451 'Internal resource 
temporarily unavailable - https://community.mimecast.com/docs/DOC-1369#451'

It goes on like this every day, all day long. The second attempt then succeeds, 
but this is just braindead.

--
BR Oliver


dmTECH GmbH
Am dm-Platz 1, 76227 Karlsruhe * Postfach 10 02 34, 76232 Karlsruhe
Telefon 0721 5592-2500 Telefax 0721 5592-2777
dmt...@dm.de * www.dmTECH.de
GmbH: Sitz Karlsruhe, Registergericht Mannheim, HRB 104927
Geschäftsführer: Christoph Werner, Martin Dallmeier, Roman Melcher

Datenschutzrechtliche Informationen
Wenn Sie mit uns in Kontakt treten, beispielsweise wenn Sie an unser 
ServiceCenter Fragen haben, bei uns einkaufen oder unser dialogicum in 
Karlsruhe besuchen, mit uns in einer geschäftlichen Verbindung stehen oder sich 
bei uns bewerben, verarbeiten wir personenbezogene Daten. Informationen unter 
anderem zu den konkreten Datenverarbeitungen, Löschfristen, Ihren Rechten sowie 
die Kontaktdaten unserer Datenschutzbeauftragten finden Sie 
hier.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-26 Thread John R Levine via mailop 

Do you have any idea how many of those would be tripped up by a
Postfix-style banner delay?


Good question. I've been meaning to add a greet pause but haven't yet
gotten around to it.


I got around to it and now do a greet pause before I greylist.  Most of 
the hosts on the Spamhaus BLs are early talkers but that's not a surprise 
and I wouldn't waste effort greylisting them.  Instead I accept the mail, 
reject at the end of data, and put it in the spamtrap to collect 
statistics.


After it runs for a while I'll see what the numbers look like

Regards,
John Levine, jo...@taugh.com, Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-24 Thread John Levine via mailop 
According to Gellner, Oliver via mailop :
>
>> On 24.06.2023 at 06:56 John Levine via mailop wrote:
>>
>> It appears that Al Iverson via mailop  said:
>>> What if we just got to the heart of the matter and admitted that
>>> greylisting is useless 2023?
>>
>> Because it's still quite useful if you do it sensibly.  Here's what my
>> logs say for the past two years, by number of IPs
>>
>> greylist but never retried: 13573
>> greylist and did retry: 9299
>>
>> whitelisted or previously retried: 6519
>>
>> More than half of the addresses I greylist never retry, and spot
>> checks of the logs show that's overwhelmingly bots.
>
>It matters if the greylisting takes place before or after RBL / domain 
>reputation checks. If the greylisting comes first, I could imagine that
>the connections from most of those bots would have been blocked anyway (with a 
>5xx response code).

No, this is after DNSBL checks, which catch about 85% of attempted
connections. Those get, ah, special handling.

>and thereby introduces noticeable breakage into email communication. As a 
>commercial ESP I wouldn’t set it up. It’s annoying for the users
>when they repeatedly have to wait for registration emails, order confirmation 
>with payment details, when they are on the phone with someone and
>want to discuss a document, which just doesn’t arrive at the other end, and 
>so on. Let alone cases where messages aren’t delivered at all,

That occasionally happens, but since I whitelist any /24 that has successfully 
retried,
it doesn't happen very much.

-- 
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-24 Thread John Levine via mailop 
It appears that Bill Cole via mailop  said:
>>  greylist but never retried: 13573
>>  greylist and did retry: 9299
>>
>>  whitelisted or previously retried: 6519
>>
>> More than half of the addresses I greylist never retry, and spot
>> checks of the logs show that's overwhelmingly bots.
>
>Do you have any idea how many of those would be tripped up by a 
>Postfix-style banner delay?

Good question. I've been meaning to add a greet pause but haven't yet
gotten around to it.

R's,
John
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-24 Thread Bill Cole via mailop 

On 2023-06-24 at 00:49:46 UTC-0400 (24 Jun 2023 00:49:46 -0400)
John Levine via mailop 
is rumored to have said:


It appears that Al Iverson via mailop  said:

What if we just got to the heart of the matter and admitted that
greylisting is useless 2023?


Because it's still quite useful if you do it sensibly.  Here's what my
logs say for the past two years, by number of IPs

 greylist but never retried: 13573
 greylist and did retry: 9299

 whitelisted or previously retried: 6519

More than half of the addresses I greylist never retry, and spot
checks of the logs show that's overwhelmingly bots.


Do you have any idea how many of those would be tripped up by a 
Postfix-style banner delay?




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-24 Thread Gellner, Oliver via mailop 

> On 24.06.2023 at 06:56 John Levine via mailop wrote:
>
> It appears that Al Iverson via mailop  said:
>> What if we just got to the heart of the matter and admitted that
>> greylisting is useless 2023?
>
> Because it's still quite useful if you do it sensibly.  Here's what my
> logs say for the past two years, by number of IPs
>
> greylist but never retried: 13573
> greylist and did retry: 9299
>
> whitelisted or previously retried: 6519
>
> More than half of the addresses I greylist never retry, and spot
> checks of the logs show that's overwhelmingly bots.

It matters if the greylisting takes place before or after RBL / domain 
reputation checks. If the greylisting comes first, I could imagine that the 
connections from most of those bots would have been blocked anyway (with a 5xx 
response code).

I’m writing „most“, as surely greylisting does block spam messages which would 
otherwise not have been caught, but as with every antispam measure one has to 
weigh its benefits and costs, especially false positives. Unfortunately 
greylisting tends to have a lot of false positives and thereby introduces 
noticeable breakage into email communication. As a commercial ESP I wouldn’t 
set it up. It’s annoying for the users when they repeatedly have to wait for 
registration emails, order confirmation with payment details, when they are on 
the phone with someone and want to discuss a document, which just doesn’t 
arrive at the other end, and so on. Let alone cases where messages aren’t 
delivered at all, as can be seen in this thread or in the one just a few days 
old: „Strange mail delivery from microsoft“ (although the temporary errors in 
this thread might be due to maintenance work or some software errors, I‘m not 
sure).

—
BR Oliver



dmTECH GmbH
Am dm-Platz 1, 76227 Karlsruhe * Postfach 10 02 34, 76232 Karlsruhe
Telefon 0721 5592-2500 Telefax 0721 5592-2777
dmt...@dm.de * www.dmTECH.de
GmbH: Sitz Karlsruhe, Registergericht Mannheim, HRB 104927
Geschäftsführer: Christoph Werner, Martin Dallmeier, Roman Melcher

Datenschutzrechtliche Informationen
Wenn Sie mit uns in Kontakt treten, beispielsweise wenn Sie an unser 
ServiceCenter Fragen haben, bei uns einkaufen oder unser dialogicum in 
Karlsruhe besuchen, mit uns in einer geschäftlichen Verbindung stehen oder sich 
bei uns bewerben, verarbeiten wir personenbezogene Daten. Informationen unter 
anderem zu den konkreten Datenverarbeitungen, Löschfristen, Ihren Rechten sowie 
die Kontaktdaten unserer Datenschutzbeauftragten finden Sie 
hier.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] greylisting, SendGrid is deleting your mail

2023-06-23 Thread John Levine via mailop 
It appears that Al Iverson via mailop  said:
>What if we just got to the heart of the matter and admitted that
>greylisting is useless 2023? 

Because it's still quite useful if you do it sensibly.  Here's what my
logs say for the past two years, by number of IPs

 greylist but never retried: 13573
 greylist and did retry: 9299

 whitelisted or previously retried: 6519

More than half of the addresses I greylist never retry, and spot
checks of the logs show that's overwhelmingly bots.

I suspect that Sendgrid's problem is that they gush so much spam that
mail systems have cranked the spam filters way up, so they run into
delivery problems that better behaved mail systems don't have. I have
a bunch of special rules that whitelist a modest set of known good
return addresses and report most of the rest as spam to them, for what
little good that does.

R's,
John



___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop