Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Tue, 2023-02-28 at 12:31 +0100, Jaroslaw Rafa via mailop wrote:
> 
> I do greylisting, and that's how I found out about the immediate
> retries. I run postgrey with default setting which is 5 minutes, and
> I often see in logs multiple retries within those 5 minutes, with
> first ones being really almost immediate (15-30 seconds). I have no
> other MXes, so I can't observe retries to other MXes, but I often see
> retries FROM different IP address than the original attempt, which
> count as new entries for greylisting, as greylisting works on triplet
> (client IP address, sender, recipient).
> 

ONE immediate retry can be helpful. Since you mention postfix, the
"accidental greylisting" caused by its deep protocol tests can be
mitigated by sharing the postscreen cache across MX, even if one of
them is simply an alias:

  https://www.postfix.org/POSTSCREEN_README.html#after_220

In those situations, the immediate retry will go through. (And FWIW I
would suggest lowering the timeout on postgrey; spambots don't come
back after a few seconds.)

That doesn't help when the retries all come from different IPs though.
That is indeed stupid. It's not surprising however given that most of
those providers will loan you a consistent IP for a small fee.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Mon, 2023-02-27 at 21:38 -0700, Luke wrote:
> 
> First, we send a lot of email; 9 billion messages on black friday
> alone. Each message generates an average of 8 events (deferrals,
> opens, clicks, spam reports, unsubscribes etc). Storage cost is a
> small part of the reason for minimizing MTA chatter. It's not a big
> enough factor for us to make irresponsible decisions in this area to
> save money, but it's a factor.

I don't know if they're separate systems, but nobody is complaining
about the missing marketing emails. It's the password resets, two-
factor auths, email verifications, etc. disappearing that get noticed.


> Second factor (and the main factor) is our customers. We send 32
> deferral events over the course of 72 hours to our customers' data
> warehouses for each message...So do we read the RFC to our customers
> and tell them to pound sand? Or do we take some time and see if we
> can help make their experience a little better *and* save us some
> money at the same time?

It sounds like what they're unhappy with is the mountain of unwanted
data, not the retries themselves. And these sound like marketing
complaints that are being used to make decisions for transactional
customers.

I know your transactional customers don't want this, because they call
me and ask why they're not getting the emails from their website. The
answer is that a lot of people aren't getting mail from your website,
and you're losing users because they can't complete the registration
process because SendGrid deletes the confirmation emails.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Julian Bradfield via mailop]

On 2023-02-28, Jaroslaw Rafa via mailop  wrote:
> Dnia 28.02.2023 o godz. 11:10:05 Julian Bradfield via mailop pisze:

>> Maybe worth pointing that people do greylisting, and with
>> greylisting it's helpful to retry quite soon. Immediately isn't
>> useful, but within five minutes is. (I personally have one-minute
>> greylisting set.)
>
> I do greylisting, and that's how I found out about the immediate retries. I
> run postgrey with default setting which is 5 minutes, and I often see in
> logs multiple retries within those 5 minutes, with first ones being really
> almost immediate (15-30 seconds). I have no other MXes, so I can't observe
> retries to other MXes, but I often see retries FROM different IP address
> than the original attempt, which count as new entries for greylisting, as
> greylisting works on triplet (client IP address, sender, recipient).

That too. Which is why I greylist not on individual addresses, but on
/24s (or on /48s for IPv6).

Of course there are still senders with larger mail farms, and
some with widely scattered mail farms, but there's only so much I can
be bothered to do ..
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jaroslaw Rafa via mailop]

Dnia 28.02.2023 o godz. 11:10:05 Julian Bradfield via mailop pisze:
> On 2023-02-28, Jaroslaw Rafa via mailop  wrote:
> > Another nonsense thing for me is that some senders - again, mostly the big
> > ones - retry almost *immediately* (often from a different IP address) if
> > they encounter a 4xx, and after a few such unsuccessful retries (within only
> > a few minutes) they eventually give up. I wonder what had in mind someone
> > who designed it that way? In a very broad sense, 4xx means some transient
> > problem on the receiving side that will eventually get fixed, and then the
> > mail will be accepted. But fixing any problem requires time. Did someone
> > who designed this behavior expect that the problem will "miraculously"
> > disappear if they retry immediately a few times?
> 
> Maybe worth pointing that people do greylisting, and with
> greylisting it's helpful to retry quite soon. Immediately isn't
> useful, but within five minutes is. (I personally have one-minute
> greylisting set.)

I do greylisting, and that's how I found out about the immediate retries. I
run postgrey with default setting which is 5 minutes, and I often see in
logs multiple retries within those 5 minutes, with first ones being really
almost immediate (15-30 seconds). I have no other MXes, so I can't observe
retries to other MXes, but I often see retries FROM different IP address
than the original attempt, which count as new entries for greylisting, as
greylisting works on triplet (client IP address, sender, recipient).

And so they often try, say, 5 different IP addresses within 5 minutes, all
tempfailed of course (because from greylisting point of view each one is a
completely new delivery attempt because of different IPs), and then retry
only after a hour or so (when the mail gets finally delivered) - but
sometimes don't retry at all...

Postgrey by default whitelists several senders known to behave this way, but
not all...
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Julian Bradfield via mailop]

On 2023-02-28, Jaroslaw Rafa via mailop  wrote:
> Another nonsense thing for me is that some senders - again, mostly the big
> ones - retry almost *immediately* (often from a different IP address) if
> they encounter a 4xx, and after a few such unsuccessful retries (within only
> a few minutes) they eventually give up. I wonder what had in mind someone
> who designed it that way? In a very broad sense, 4xx means some transient
> problem on the receiving side that will eventually get fixed, and then the
> mail will be accepted. But fixing any problem requires time. Did someone
> who designed this behavior expect that the problem will "miraculously"
> disappear if they retry immediately a few times?

Maybe worth pointing that people do greylisting, and with
greylisting it's helpful to retry quite soon. Immediately isn't
useful, but within five minutes is. (I personally have one-minute
greylisting set.)
I often see immediate retries to the next MX, which of course is a
reasonable thing to do (and often bypasses my greylist, owing to the
way I do it - I view this as a feature:).

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jaroslaw Rafa via mailop]

Dnia 27.02.2023 o godz. 21:38:01 Luke via mailop pisze:
> Second factor (and the main factor) is our customers. We send 32 deferral
> events over the course of 72 hours to our customers' data warehouses for
> each message that reaches max queue time. That's 32 events they are
> consuming and storing as well. The customers, not being RFC-reading (and
> RFC-writing) mail operators like the group here, ask us why we keep
> retrying for so long on messages we know will never deliver. To the
> customer these deferral events are just a nuisance.

Do I understand correctly that you send 32 deferral notices within 72 hours
*per each message*? That seems nonsense to me.
The standard behavior of the good old sendmail (inherited by many other MTAs
as default setting) was to send *one* deferral notice if the message was
not delivered within the first 4 hours, and then only send a final bounce
when the message reaches max queue time.
Why did you modify it so heavily?

As for the not retrying on 4xx: If it is marketing mail only, I agree that
basically nobody gets hurt (or even quite the opposite ;)) if some of the
mail doesn't get delivered.

It's worse when it comes to transaction mail (like password reset links,
receipts from Internet shops etc.) that is not retried. And I have seen
quite a few of these, from big senders, *not* being retried on 4xx (although
if I remember correctly, not from Sendgrid). This is much worse.

Another nonsense thing for me is that some senders - again, mostly the big
ones - retry almost *immediately* (often from a different IP address) if
they encounter a 4xx, and after a few such unsuccessful retries (within only
a few minutes) they eventually give up. I wonder what had in mind someone
who designed it that way? In a very broad sense, 4xx means some transient
problem on the receiving side that will eventually get fixed, and then the
mail will be accepted. But fixing any problem requires time. Did someone
who designed this behavior expect that the problem will "miraculously"
disappear if they retry immediately a few times?
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Luke via mailop]

Those possible (and likely) explanations for a few of the rejections are
actually really insightful. It sounds like we agree that there are cases
where 4xx should not be retried. That's progress! The question around the
downside of queuing them up for a while is essentially 2-part.

First, we send a lot of email; 9 billion messages on black friday alone.
Each message generates an average of 8 events (deferrals, opens, clicks,
spam reports, unsubscribes etc). Storage cost is a small part of the reason
for minimizing MTA chatter. It's not a big enough factor for us to make
irresponsible decisions in this area to save money, but it's a factor.

Second factor (and the main factor) is our customers. We send 32 deferral
events over the course of 72 hours to our customers' data warehouses for
each message that reaches max queue time. That's 32 events they are
consuming and storing as well. The customers, not being RFC-reading (and
RFC-writing) mail operators like the group here, ask us why we keep
retrying for so long on messages we know will never deliver. To the
customer these deferral events are just a nuisance. Making matters worse,
they incorrectly believe that *our* MTA is to blame. That we should be
handling this better. So do we read the RFC to our customers and tell them
to pound sand? Or do we take some time and see if we can help make their
experience a little better *and* save us some money at the same time?

Luke

On Mon, Feb 27, 2023 at 8:24 PM Michael Orlitzky via mailop <
mailop@mailop.org> wrote:

> On Mon, 2023-02-27 at 18:53 -0700, Luke via mailop wrote:
> > For better or worse, *not *retrying *some *4xx is even easier to
> > justify.
> > Here are a few massive examples.
> >
> > 421 4.7.1 Message permanently deferred:
> >
> > 452 Sender Rejected:
>
> Keeping in mind that the RFC says not to use the text in your decision,
> these two sound the most reasonable. Skipping for brevity.
>
>
> > 450 4.1.1 Recipient Address Rejected: this one is tough because with
> > some MXs retries result in delivery, in others, it's a dead end.
> > Dynamic rule handling per receiving MX would be awesome, but it would
> > require machine learning to accomplish at scale.
>
> This could  postfix's unverified_recipient_reject_code. When (for
> example) postfix sits in front of an Exchange server and when a full
> list of valid recipients is not available, postfix can launch a
> background probe to see if the recipient is valid in Exchange before it
> will accept a message destined for it (otherwise, you get backscatter).
> If postfix is up but Exchange is down, it'll give you this rejection.
>
>
> > 451 Relay Not Permitted: this one can mean a lot of things and comes
> > in a few flavors, but when the data shows retrying never results in a
> > delivery, we permfail it.
>
> This happens when somebody's domain expires and the MX record gets
> pointed somewhere it shouldn't. This is the right response because when
> they finally find the guy who knows the guy who knows the guy who
> receives the email that resets the password at the registrar three days
> later, it's nice to have all their mail start flowing as if nothing
> ever happened.
>
>
> > Unlike the rare 5xx we decide to retry, these 4xx that shouldn't be
> > retried are far more numerous and come from large-ish reputable
> > receivers. Believe me, simply retrying 4xx and failing 5xx would make
> > our jobs a lot easier. Unfortunately the real world isn't so simple
>
> Not pounding on servers that have told you to go away with a 5xx has
> obvious benefits for both sides though. What's the downside to leaving
> the 4xx in the queue for a while? No one is going to fault you for
> retrying a message they told you to retry, and even 1% of legitimate
> mail is a lot to toss out without a good reason.
>
> Finally, while it might be 1% of *your* mail it's not 1% of *our* mail.
> Michael Scott said Wayne Gretzky said that you lose 100% of the
> messages you don't retry to recipients who would have accepted them.
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Mon, 2023-02-27 at 18:53 -0700, Luke via mailop wrote:
> For better or worse, *not *retrying *some *4xx is even easier to
> justify.
> Here are a few massive examples.
> 
> 421 4.7.1 Message permanently deferred:
> 
> 452 Sender Rejected:

Keeping in mind that the RFC says not to use the text in your decision,
these two sound the most reasonable. Skipping for brevity.


> 450 4.1.1 Recipient Address Rejected: this one is tough because with
> some MXs retries result in delivery, in others, it's a dead end.
> Dynamic rule handling per receiving MX would be awesome, but it would
> require machine learning to accomplish at scale.

This could  postfix's unverified_recipient_reject_code. When (for
example) postfix sits in front of an Exchange server and when a full
list of valid recipients is not available, postfix can launch a
background probe to see if the recipient is valid in Exchange before it
will accept a message destined for it (otherwise, you get backscatter).
If postfix is up but Exchange is down, it'll give you this rejection.


> 451 Relay Not Permitted: this one can mean a lot of things and comes
> in a few flavors, but when the data shows retrying never results in a
> delivery, we permfail it.

This happens when somebody's domain expires and the MX record gets
pointed somewhere it shouldn't. This is the right response because when
they finally find the guy who knows the guy who knows the guy who
receives the email that resets the password at the registrar three days
later, it's nice to have all their mail start flowing as if nothing
ever happened.


> Unlike the rare 5xx we decide to retry, these 4xx that shouldn't be
> retried are far more numerous and come from large-ish reputable
> receivers. Believe me, simply retrying 4xx and failing 5xx would make
> our jobs a lot easier. Unfortunately the real world isn't so simple

Not pounding on servers that have told you to go away with a 5xx has
obvious benefits for both sides though. What's the downside to leaving
the 4xx in the queue for a while? No one is going to fault you for
retrying a message they told you to retry, and even 1% of legitimate
mail is a lot to toss out without a good reason.

Finally, while it might be 1% of *your* mail it's not 1% of *our* mail.
Michael Scott said Wayne Gretzky said that you lose 100% of the
messages you don't retry to recipients who would have accepted them.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Luke via mailop]

For better or worse, *not *retrying *some *4xx is even easier to justify.
Here are a few massive examples.

421 4.7.1 Message permanently deferred: less than 1% delivered rate after
32 retries. This is across 100s of millions of messages, thousands of
unique senders, and thousands of IPs, looking back over 6 months of data.
Retrying is a waste of resources on both sides.

452 Sender Rejected: we get this from thousands of distinct receivers. Same
results as the above. Less than 1% delivery rate after 32 retries over a
diverse sampling of senders.

450 4.1.1 Recipient Address Rejected: this one is tough because with some
MXs retries result in delivery, in others, it's a dead end. Dynamic rule
handling per receiving MX would be awesome, but it would require machine
learning to accomplish at scale.

451 Relay Not Permitted: this one can mean a lot of things and comes in a
few flavors, but when the data shows retrying never results in a delivery,
we permfail it.

Unlike the rare 5xx we decide to retry, these 4xx that shouldn't be retried
are far more numerous and come from large-ish reputable receivers. Believe
me, simply retrying 4xx and failing 5xx would make our jobs a lot easier.
Unfortunately the real world isn't so simple

Luke

On Mon, Feb 27, 2023 at 5:15 PM Michael Orlitzky via mailop <
mailop@mailop.org> wrote:

> On Mon, 2023-02-27 at 15:45 -0700, Luke via mailop wrote:
> >
> > With that said, given the responses in this thread, we will be taking
> > a close look at the few rules we have in place where we retry 5xx and
> > see if A.) the rules are still being hit at all, and B.) are these
> > retries still resulting in reliable and consistent delivery after a
> > small number of attempts.
> >
>
> That's a good idea anyway (thank you), but the original issue wasn't
> about an extra retry or two for a 5xx, but the lack of any retry at all
> for a 4xx.
>
> That happens all day every day and is what makes mail go missing.
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Mon, 2023-02-27 at 15:45 -0700, Luke via mailop wrote:
> 
> With that said, given the responses in this thread, we will be taking
> a close look at the few rules we have in place where we retry 5xx and
> see if A.) the rules are still being hit at all, and B.) are these
> retries still resulting in reliable and consistent delivery after a
> small number of attempts.
> 

That's a good idea anyway (thank you), but the original issue wasn't
about an extra retry or two for a 5xx, but the lack of any retry at all
for a 4xx.

That happens all day every day and is what makes mail go missing.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Luke via mailop]

I can't go into the specifics in great detail, but I want to make it clear
that these are rare edge cases. We aren't out there retrying 5xx every
chance we get because we don't care about being a good citizen. These are
small scale, often short-lived exceptions where retrying a very specific
5xx results in immediate and reliable delivery.

One example: when a dozen customers complain about us permanently failing
responses like *554 Temporarily Deferred, try again in 5 minutes *(Yeah,
that's an actual response seen in the wild), and we flip it to a retry and
watch the mail deliver consistently on the second or third attempt, we are
inclined to leave it as a retry. It seems like getting more mail delivered
to our customers' end recipients is a small price to pay for violating
the spirit of an RFC in a few small, and well vetted edge cases.

With that said, given the responses in this thread, we will be taking a
close look at the few rules we have in place where we retry 5xx and see if
A.) the rules are still being hit at all, and B.) are these retries still
resulting in reliable and consistent delivery after a small number of
attempts.

Luke

On Mon, Feb 27, 2023 at 2:01 PM Michael Orlitzky via mailop <
mailop@mailop.org> wrote:

> On Mon, 2023-02-27 at 04:55 +, Denny Watson via mailop wrote:
> >
> > All I have said (which you had conveniently redacted) is that RFC5321
> > leaves the door open to process bounces differently should the
> > sending MTA be able to determine the reason for non-delivery.[1]
> >
> > ...
> >
> > [1] See my original email on this subject about section 4.5.4.1 of
> > RFC5321 and why rejected mail can have a different queuing strategy
> > based on the sender's interpretation of the 4xx response the it had
> > received.
>
> I didn't redact it to make my argument seem stronger, I redacted it
> because it's irrelevant. That's not what SendGrid is doing. And you
> quoted the RFC out of context anyway. Here's the full text of the first
> two paragraphs of that section for anyone too lazy to look it up:
>
>   4.5.4.1.  Sending Strategy
>
>   The general model for an SMTP client is one or more processes that
>   periodically attempt to transmit outgoing mail.  In a typical
>   system, the program that composes a message has some method for
>   requesting immediate attention for a new piece of outgoing mail,
>   while mail that cannot be transmitted immediately MUST be queued and
>   periodically retried by the sender.  A mail queue entry will include
>   not only the message itself but also the envelope information.
>
>   The sender MUST delay retrying a particular destination after one
>   attempt has failed. In general, the retry interval SHOULD be at
>   least 30 minutes; however, more sophisticated and variable strategies
>   will be beneficial when the SMTP client can determine the reason for
>   non-delivery.
>
> It clearly states that messages must be retried. The "variable
> strategies" you latched onto is in the context of how often the sender
> should retry.
>
> I don't care if SendGrid takes your money and pipes your mail to
> /dev/null but we can stop pretending that they're playing 4d chess with
> RFC5321 now.
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[John Levine via mailop]

It appears that Kelly Molloy via mailop  said:
>On Fri, Feb 24, 2023 at 7:14 PM Matt Palmer via mailop
> wrote:
>> That's something to talk to your ESP about.  They're in charge of retrying.
>
>Christine *is* the ESP.

She's at Shopify, but Sendgrid is the ESP.

Perhaps she can confirm with Sendgrid that they're not retrying 4xx and so
are losing customer mail.

Given how much mail I get from Sendgrid that should not have been sent in the
first place, I'm not inclined to push too hard on that point.

R's,
John
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Mon, 2023-02-27 at 04:55 +, Denny Watson via mailop wrote:
> 
> All I have said (which you had conveniently redacted) is that RFC5321
> leaves the door open to process bounces differently should the
> sending MTA be able to determine the reason for non-delivery.[1]
> 
> ...
> 
> [1] See my original email on this subject about section 4.5.4.1 of 
> RFC5321 and why rejected mail can have a different queuing strategy 
> based on the sender's interpretation of the 4xx response the it had 
> received.

I didn't redact it to make my argument seem stronger, I redacted it
because it's irrelevant. That's not what SendGrid is doing. And you
quoted the RFC out of context anyway. Here's the full text of the first
two paragraphs of that section for anyone too lazy to look it up:

  4.5.4.1.  Sending Strategy

  The general model for an SMTP client is one or more processes that
  periodically attempt to transmit outgoing mail.  In a typical 
  system, the program that composes a message has some method for 
  requesting immediate attention for a new piece of outgoing mail,
  while mail that cannot be transmitted immediately MUST be queued and 
  periodically retried by the sender.  A mail queue entry will include 
  not only the message itself but also the envelope information.

  The sender MUST delay retrying a particular destination after one
  attempt has failed. In general, the retry interval SHOULD be at
  least 30 minutes; however, more sophisticated and variable strategies
  will be beneficial when the SMTP client can determine the reason for
  non-delivery.

It clearly states that messages must be retried. The "variable
strategies" you latched onto is in the context of how often the sender
should retry.

I don't care if SendGrid takes your money and pipes your mail to
/dev/null but we can stop pretending that they're playing 4d chess with
RFC5321 now.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Kelly Molloy via mailop]

On Fri, Feb 24, 2023 at 7:14 PM Matt Palmer via mailop
 wrote:
> That's something to talk to your ESP about.  They're in charge of retrying.

Christine *is* the ESP.

Kids these days.
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Denny Watson via mailop]

On 2/26/2023, Michael Orlitzky via mailop wrote:

On Sun, 2023-02-26 at 19:43 +, Denny Watson via mailop wrote:

This appears to be a specially built MTA, and with the (probable)
consent of its users, has policy in place to not resend after 4xx under
specific conditions.  I.e. the sending MTA is interpreting specific 4xx
responses (and more likely the text) to mean a permanent failure.

I do not see a problem with elevating a 4xx to mean 5xx where the users
of the system understand the policy as it does not appear to impact the
receiving system.


This is not whatever subtle and nuanced situation you are imagining
though. The users don't know, and the response code/text makes no
difference.


I'm going to have to take your word for it as;

* I have never read Sendgrid's contracts, nor their user documentation.
* I do not know if this is what Sendgrid is actually doing, and if they 
are, that they are not also attempting to send at least one more time.

* I do not know if this is a user configurable setting.

All I have said (which you had conveniently redacted) is that RFC5321 
leaves the door open to process bounces differently should the sending 
MTA be able to determine the reason for non-delivery.[1]  I'll add also 
that RFC5321 specifically says under section 4.2.1;


4yz  Transient Negative Completion reply
The command was not accepted, and the requested action did not occur. 
However, the error condition is temporary, and the action may be 
requested again.  The sender should return to the beginning of the 
command sequence (if any).  It is difficult to assign a meaning to 
"transient" when two different sites (receiver- and sender-SMTP agents) 
must agree on the interpretation.  Each reply in this category might 
have a different time value, but the SMTP client SHOULD try again.  A 
rule of thumb to determine whether a reply fits into the 4yz or the 5yz 
category (see below) is that replies are 4yz if they can be successful 
if repeated without any change in command form or in properties of the 
sender or receiver (that is, the command is repeated identically and the 
receiver does not put up a new implementation).


Please note "However, the error condition is temporary, and the action 
_MAY_ be requested again" (emphases added) and "[...] but the SMTP 
client SHOULD try again." Note the use of SHOULD and not MUST.


Again, without further detail on actual operation and/or an 
understanding of what the users of the system have been told, I can not 
comment specifically about Sendgrid's MTAs today.[2]


[1] See my original email on this subject about section 4.5.4.1 of 
RFC5321 and why rejected mail can have a different queuing strategy 
based on the sender's interpretation of the 4xx response the it had 
received.
[2] I have twice pulled one of their folks aside to inform them that 
their MTAs were not fully compliant with the RFCs, and they corrected 
the problems swiftly.  But because I do not have evidence here, I can 
not specifically state that this is or is not the case.


--
Denny Watson
Lead Investigator
The Spamhaus Project

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Peddemors via mailop]

Luke, you might want to share SendGrid's logic on that. to the list. if 
you keep sending to 5xx too many times, that will be 'bad' for your 
reputation.


When/where do you retry? How often, for how long.. under what circumstances?

On 2023-02-25 19:15, Luke via mailop wrote:
I can assure you sendgrid retries 4xx. We also don't retry 4xx. We also 
retry 5xx. We also don't retry 5xx. How is it 2023 and people still 
think 4xx means retry and 5xx means don't retry?


Luke

On Sat, Feb 25, 2023, 9:21 AM Michael Orlitzky via mailop 
mailto:mailop@mailop.org>> wrote:


On Fri, 2023-02-24 at 15:57 -0500, Christine Borgia via mailop wrote:
 > It is transient, but they are blocks and are not retried. Weird,
right?
 >

SendGrid doesn't retry on 4xx and haven't for many years.

___
mailop mailing list
mailop@mailop.org 
https://list.mailop.org/listinfo/mailop



___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop



--
"Catch the Magic of Linux..."

Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.

604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Sun, 2023-02-26 at 19:43 +, Denny Watson via mailop wrote:
> This appears to be a specially built MTA, and with the (probable) 
> consent of its users, has policy in place to not resend after 4xx under 
> specific conditions.  I.e. the sending MTA is interpreting specific 4xx 
> responses (and more likely the text) to mean a permanent failure.
> 
> I do not see a problem with elevating a 4xx to mean 5xx where the users 
> of the system understand the policy as it does not appear to impact the 
> receiving system.

This is not whatever subtle and nuanced situation you are imagining
though. The users don't know, and the response code/text makes no
difference.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Al Iverson via mailop]

> This appears to be a specially built MTA, and with the (probable)
> consent of its users, has policy in place to not resend after 4xx under
> specific conditions.  I.e. the sending MTA is interpreting specific 4xx
> responses (and more likely the text) to mean a permanent failure.
>
> I do not see a problem with elevating a 4xx to mean 5xx where the users
> of the system understand the policy as it does not appear to impact the
> receiving system.  In fact one could take section 4.5.4.1 of RFC5321 to
> opening the door for just that; "[...] however, more sophisticated and
> variable strategies will be beneficial when the SMTP client can
> determine the reason for non-delivery."

Agree. Also, I think it's unreasonable to assume that people will
never try new things when it comes to bounce processing or MTA queue
management. RFCs could never evolve if folks were prohibited from ever
trying anything different than what's written in the RFCs today.

Many large sending email platforms have had unique MTA handling rules
in place for years. And there are a lot of reasons smart platforms
HAVE to, going all the way back to (pulls note out of greybeard,
checks) stuff like Cisco PIX firewalls causing email bombing -- not a
4xx/5xx handling issue, but an example of how going back for 20+
years, weird shit has sometimes happened that required workarounds.
Those workarounds are not always based on a sender's intent to get
away with something. Often the intent is to do better, be better-- not
to be sneaky.

-- 

Al Iverson / Deliverability blogging at www.spamresource.com
Subscribe to the weekly newsletter at wombatmail.com/sr.cgi
DNS Tools at xnnd.com / (312) 725-0130 / Chicago (Central Time)
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Denny Watson via mailop]

On 2/24/2023, Christine Borgia via mailop wrote:

It is transient, but they are blocks and are not retried. Weird, right?


I have an opinion on this;

This appears to be a specially built MTA, and with the (probable) 
consent of its users, has policy in place to not resend after 4xx under 
specific conditions.  I.e. the sending MTA is interpreting specific 4xx 
responses (and more likely the text) to mean a permanent failure.


I do not see a problem with elevating a 4xx to mean 5xx where the users 
of the system understand the policy as it does not appear to impact the 
receiving system.  In fact one could take section 4.5.4.1 of RFC5321 to 
opening the door for just that; "[...] however, more sophisticated and 
variable strategies will be beneficial when the SMTP client can 
determine the reason for non-delivery."


I do have issues where that 5xx is treated as a 4xx (with the exception 
of the correction to the minimum/maximum problem, first pointed out in 
RFC2821.  See RFC5321 section 4.5.3.1.10) as this has an impact on the 
receiving system.


--
Denny Watson
Lead Investigator
The Spamhaus Project

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jaroslaw Rafa via mailop]

Dnia 25.02.2023 o godz. 19:15:14 Luke via mailop pisze:
> How is it 2023 and people still think
> 4xx means retry and 5xx means don't retry?

Because it does. As simple as that.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On 2023-02-25 19:15:14, Luke via mailop wrote:
> I can assure you sendgrid retries 4xx. We also don't retry 4xx. We also
> retry 5xx. We also don't retry 5xx. How is it 2023 and people still think
> 4xx means retry and 5xx means don't retry?
> 

It's literally unexplainable if you've never read the part of the RFC
with the word "retry" in the title:

  mail that cannot be transmitted immediately MUST be queued and
  periodically retried by the sender
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Rathbun via mailop]

On Sun, 26 Feb 2023 10:34:11 +, Laura Atkins via mailop
 wrote:

>If people intend to be interoperable, there is NO variation in what a 4xx and 
>a 5xx mean.
>
>4xx means: this message can be queued and retried at a future date.
>
>5xx means: this message cannot be retried without human intervention.
>
>Those interactions are defined in RFC 821 and it’s successors. [1]
>
>The variation / confusion / problems / conflict comes when we try and use 
>these very clear *transactional* signals to interpret what to do with a 
>different and future message to the same address. 
>

Then there's the Y! "4xx -- deferred, but retrying anything from this IP will
never succeed" message.  I speculate that this is in place to encourage the
spamming system to retain the messages to help blow out its queue space and
slow things down considerably.  Think of it as implicit rate limiting, while
increasing the senders' costs.

mdr
-- 
  If I laugh when a guy goes flying on a banana peel, is that a
  schadenfreudian slip?
   -- Zippy

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Slavko via mailop]

Ahoj,

Dňa Sun, 26 Feb 2023 10:34:11 + Laura Atkins via mailop
 napísal:

> If people intend to be interoperable, there is NO variation in what a
> 4xx and a 5xx mean.
> 
> 4xx means: this message can be queued and retried at a future date.
> 
> 5xx means: this message cannot be retried without human intervention.
> 
> Those interactions are defined in RFC 821 and it’s successors. [1]

Yes, but unfortunately it seems, that some people are more smart than
others and they know better than we, what our return codes wants to
tell...

For now it seems, that one can generate these codes (for some
companies) from RNG output :-P

regards

-- 
Slavko
https://www.slavino.sk


pgpA3CALULQgi.pgp
Description: Digitálny podpis OpenPGP
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Laura Atkins via mailop]

If people intend to be interoperable, there is NO variation in what a 4xx and a 
5xx mean.

4xx means: this message can be queued and retried at a future date.

5xx means: this message cannot be retried without human intervention.

Those interactions are defined in RFC 821 and it’s successors. [1]

The variation / confusion / problems / conflict comes when we try and use these 
very clear *transactional* signals to interpret what to do with a different and 
future message to the same address. 

Bounce handling in the transaction should follow the RFC. If the SMTP response 
starts with a 4, queue it and retry. If the SMTP response starts with a 5, stop 
trying to deliver it. 

Address handling and list hygiene is undefined in the SMTP spec. It’s something 
we’ve tried to cobble together with every group doing a different thing over 
the years. It’s a mess and it is complicated and there probably should be 
documentation of what to do with it. [2]

laura 

[1] I’m still annoyed by “4xy: this message will never deliver” that a major 
ISP uses. If the message will never deliver, give it a 5xx. 

[2] Yes, this is my current project - figuring out a way to make bounce 
handling less of a maze of twisty little codes all alike and making it a 
clearer communication channel. 

> On 26 Feb 2023, at 05:12, Evan Burke via mailop  wrote:
> 
> 
> There's a lot of variation in what 4xx and 5xx mean in practice. This means 
> there's room for some senders to spend time investigating and understanding 
> what each receiving domain's 4xx and 5xx (and subcategories thereof) actually 
> mean for address delivery over time, and for designing rules and policies to 
> handle them based on observed outcomes - IF those are a more reliable 
> indicator than the general guidance from RFCs.
> 
> That doesn't mean Luke is ignoring null MXes.
> 
> On Sat, Feb 25, 2023 at 8:00 PM Benny Pedersen via mailop  > wrote:
>> Luke via mailop skrev den 2023-02-26 04:15:
>> > I can assure you sendgrid retries 4xx. We also don't retry 4xx. We
>> > also retry 5xx. We also don't retry 5xx. How is it 2023 and people
>> > still think 4xx means retry and 5xx means don't retry?
>> 
>> what books do you read ?
>> 
>> maybe rfc 7505 is soon or later for all ?
>> ___
>> mailop mailing list
>> mailop@mailop.org 
>> https://list.mailop.org/listinfo/mailop
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop

-- 
The Delivery Experts

Laura Atkins
Word to the Wise
la...@wordtothewise.com 

Email Delivery Blog: http://wordtothewise.com/blog  






___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Evan Burke via mailop]

There's a lot of variation in what 4xx and 5xx mean in practice. This means
there's room for some senders to spend time investigating and understanding
what each receiving domain's 4xx and 5xx (and subcategories thereof)
actually mean for address delivery over time, and for designing rules and
policies to handle them based on observed outcomes - IF those are a more
reliable indicator than the general guidance from RFCs.

That doesn't mean Luke is ignoring null MXes.

On Sat, Feb 25, 2023 at 8:00 PM Benny Pedersen via mailop 
wrote:

> Luke via mailop skrev den 2023-02-26 04:15:
> > I can assure you sendgrid retries 4xx. We also don't retry 4xx. We
> > also retry 5xx. We also don't retry 5xx. How is it 2023 and people
> > still think 4xx means retry and 5xx means don't retry?
>
> what books do you read ?
>
> maybe rfc 7505 is soon or later for all ?
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Benny Pedersen via mailop]

Luke via mailop skrev den 2023-02-26 04:15:

I can assure you sendgrid retries 4xx. We also don't retry 4xx. We
also retry 5xx. We also don't retry 5xx. How is it 2023 and people
still think 4xx means retry and 5xx means don't retry?


what books do you read ?

maybe rfc 7505 is soon or later for all ?
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Luke via mailop]

I can assure you sendgrid retries 4xx. We also don't retry 4xx. We also
retry 5xx. We also don't retry 5xx. How is it 2023 and people still think
4xx means retry and 5xx means don't retry?

Luke

On Sat, Feb 25, 2023, 9:21 AM Michael Orlitzky via mailop 
wrote:

> On Fri, 2023-02-24 at 15:57 -0500, Christine Borgia via mailop wrote:
> > It is transient, but they are blocks and are not retried. Weird, right?
> >
>
> SendGrid doesn't retry on 4xx and haven't for many years.
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jaroslaw Rafa via mailop]

Dnia 24.02.2023 o godz. 09:51:02 Mark Milhollan via mailop pisze:
> Did you read the link?  Does your ESP have a postmaster account?  If
> they don't send enough messages ("a large volume") to Gmail they
> won't provide any information about reputation, except that that
> itself indicates that you haven't (and can't) collect enough
> reputation to be other than low aka poor.

Which is nonsense. If someone sends too low volume of messages to calculate
reputation, their reputation is "unknown" which BY NO MEANS SHOULD MEAN the
same as "low/poor". Unknown is unknown, you cannot assume a known value for
something that is unknown.

And mails from sources of unknown reputation should be accepted until enough
mail is collected to actually determine reputation. If *then* it turns out
to be "low/poor", then Gmail may stop accepting mail.

Otherwise, nobody *new* would be ever able to *start* sending mail to Gmail,
because everybody starts as unknown.
-- 
Regards,
   Jaroslaw Rafa
   r...@rafa.eu.org
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Michael Orlitzky via mailop]

On Fri, 2023-02-24 at 15:57 -0500, Christine Borgia via mailop wrote:
> It is transient, but they are blocks and are not retried. Weird, right?
> 

SendGrid doesn't retry on 4xx and haven't for many years.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Simon Greenwood via mailop]

On 25/02/2023 12:32, Benny Pedersen via mailop wrote:

Simon Greenwood via mailop skrev den 2023-02-24 22:09:


This (which I didn't know) adds a whole different aspect to the issue
- much has been said about how email is now centralised and is almost
impractical to run as a small operator level, but if a company like
Shopify and indeed Sendgrid can't assure mail delivery because the
largest mail operators will reject mail sporadically based on
non-specific criteria, and more that someone in Christine's position
doesn't have someone they can call at Google or Microsoft or wherever,
then there's a bigger problem.


sendgrid do mistakes aswell :)

inbound is not outbound, all inbound ips could be recieving mails from 
custommers, thats fine, but question is then:


do sendgrid keep the inbound mail on this ip when sending to google ?, 
does google see mails from all ips at sendmail pr sendgrid custommer ?


who knows why this is failing ? :)

please if sendgrid is reading my silly mails, do not share ips pr 
sendgrid custommer, each custommer can have payed pr ip non shared, 
make more money, to make better service pr custommer, this advice is 
100% free from me


Dedicated IPs are a paid addon of all commercial relay providers and it 
isn't just Sendgrid who have this problem. For that matter, handling 
inbound mail is usually a paid service, so a domain's MX records don't 
necessarily match their sending IPs but this doesn't matter if the SPF 
record is configured correctly. I would assume that Shopify do have 
dedicated IPs, but shared among their customers, which still results in 
a similar problem if customers spam or at least are perceived as spamming.


Simon


___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Benny Pedersen via mailop]

Matt Palmer via mailop skrev den 2023-02-25 01:01:

[Fixed TOFU]

On Fri, Feb 24, 2023 at 03:57:00PM -0500, Christine Borgia via mailop 
wrote:
On Fri, Feb 24, 2023 at 1:09 PM Benny Pedersen via mailop 


wrote:

> Christine Borgia via mailop skrev den 2023-02-24 17:17:
>
> >>> 421 4.7.0 [149.72.90.158 15] Our system has detected that this
>
> > If someone could reach out to me, I'd greatly appreciate it. I'm not
> > sure what to advise this customer except to say that email may not
> > work for their business model.
>
> note 421 4.7.0 ?
>
> this is softfails not hardfails, so mail in queue will try next 5 days
> to deliver it, if that expire it would be returned to sender

It is transient, but they are blocks and are not retried. Weird, 
right?


That's something to talk to your ESP about.  They're in charge of 
retrying.


in this case esp is sendgrid, with will not as i see it accept msg if 
google tempfails it ? :=)


in postfix its proxy content filter, that is not the problem of 
tempfails for the sender to sendgrid, same will happend if i stop fuglu 
here


or it could simple be sendgrid validate recipients but never succed with 
it on google ?


time to wake up for sendgrid :)
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Benny Pedersen via mailop]

Simon Greenwood via mailop skrev den 2023-02-24 22:09:


This (which I didn't know) adds a whole different aspect to the issue
- much has been said about how email is now centralised and is almost
impractical to run as a small operator level, but if a company like
Shopify and indeed Sendgrid can't assure mail delivery because the
largest mail operators will reject mail sporadically based on
non-specific criteria, and more that someone in Christine's position
doesn't have someone they can call at Google or Microsoft or wherever,
then there's a bigger problem.


sendgrid do mistakes aswell :)

inbound is not outbound, all inbound ips could be recieving mails from 
custommers, thats fine, but question is then:


do sendgrid keep the inbound mail on this ip when sending to google ?, 
does google see mails from all ips at sendmail pr sendgrid custommer ?


who knows why this is failing ? :)

please if sendgrid is reading my silly mails, do not share ips pr 
sendgrid custommer, each custommer can have payed pr ip non shared, make 
more money, to make better service pr custommer, this advice is 100% 
free from me

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Benny Pedersen via mailop]

Christine Borgia via mailop skrev den 2023-02-24 21:57:

It is transient, but they are blocks and are not retried. Weird,
right?


proff from logs is ?

others say this ip is not google but sendgrid

simple "mailq" will show it


___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Brandon Long via mailop]

That error message is not overly accurate in this case, unfortunately.  The
fact that it was a temp fail means its throttling, which can definitely
happen when there is infrequent sending, so the behavior is "unusual"... it
can be very common for a low to high sending change to be due to domain
reaping on expiration, or compromise.

As for "maybe Google gets it wrong"... well, doh.  Every spam result is
potentially inaccurate, especially given the motto of spam being in the eye
of the beholder, so no opinion is going to match multiple receivers 100%.

The only thing you can do is statistical analysis to validate your
effectiveness, but the laws of large numbers mean that even small failures
can have a large effect... and the false positives are not equally
distributed either.

Brandon

On Fri, Feb 24, 2023 at 8:20 AM Christine Borgia via mailop <
mailop@mailop.org> wrote:

> Hello! Long time no see. I have a customer whose email campaign was
> heavily blocked yesterday by Gmail with the error:
>
>
> 421 4.7.0 [149.72.90.158 15] Our system has detected that this message is
> suspicious due to the very low reputation of the sending domain. To best
> protect our users from spam, the message has been blocked. Please visit
> https://support.google.com/mail/answer/188131 for more information.
> v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp
>
> I would love to speak to someone at Gmail about this because I don't think
> the customer should have a very low reputation. Of course, I could be
> wrong, and Gmail is seeing something I'm not seeing. The customer has
> double opt-in, low complaints, good opens, lots of purchases, and an active
> social media presence with a lot of followers/fans. I think the issue is
> that they only send every couple of months. They are a fashion retailer
> that announces their upcoming event via email (the email we saw blocked)
> and their inventory sells out during the event.
>
> If someone could reach out to me, I'd greatly appreciate it. I'm not sure
> what to advise this customer except to say that email may not work for
> their business model.
>
> Thx!
>
> --
> *Christine Borgia*
> Email Deliverability Manager
> [image: Shopify]
> 
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jarland Donnell via mailop]

In defense of Google on that, Christine works for Shopify. Shopify is a 
huge spam outlet. If you want to flood someone's inbox with junk, find 
thousands of shopify sites and sign them up for their newsletters. Zero 
double opt in procedures, and if you happen to get a response to abuse 
complaints their response is roughly equivalent to "Deal with it."


Shopify doesn't deserve deliverability.

On 2023-02-24 18:19, Bill Cole via mailop wrote:

On 2023-02-24 at 16:09:26 UTC-0500 (Fri, 24 Feb 2023 21:09:26 +)
Simon Greenwood via mailop 
is rumored to have said:

[...]
This (which I didn't know) adds a whole different aspect to the issue 
- much has been said about how email is now centralised and is almost 
impractical to run as a small operator level, but if a company like 
Shopify and indeed Sendgrid can't assure mail delivery because the 
largest mail operators will reject mail sporadically based on 
non-specific criteria, and more that someone in Christine's position 
doesn't have someone they can call at Google or Microsoft or wherever, 
then there's a bigger problem.


Sendgrid can't deliver mail because they are grossly incompetent at 
avoiding spammers as customers.


But yes, it is a HUGE problem that the largest mailbox providers have 
essentially no support for senders except for various inaccurate 
"knowledge bases" and systems like SNDS and JMRP that don't work as 
advertised. (At least Google doesn't pretend that they want to help...) 
The fact that Christine Borgia has to put out a public SOS here to get 
random bits of advice on deliverability to Google is an indictment of 
what has become of email. It is clearly no longer possible to know how 
to do email well.

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Bill Cole via mailop]

On 2023-02-24 at 16:09:26 UTC-0500 (Fri, 24 Feb 2023 21:09:26 +)
Simon Greenwood via mailop 
is rumored to have said:

[...]
This (which I didn't know) adds a whole different aspect to the issue 
- much has been said about how email is now centralised and is almost 
impractical to run as a small operator level, but if a company like 
Shopify and indeed Sendgrid can't assure mail delivery because the 
largest mail operators will reject mail sporadically based on 
non-specific criteria, and more that someone in Christine's position 
doesn't have someone they can call at Google or Microsoft or wherever, 
then there's a bigger problem.


Sendgrid can't deliver mail because they are grossly incompetent at 
avoiding spammers as customers.


But yes, it is a HUGE problem that the largest mailbox providers have 
essentially no support for senders except for various inaccurate 
"knowledge bases" and systems like SNDS and JMRP that don't work as 
advertised. (At least Google doesn't pretend that they want to help...) 
The fact that Christine Borgia has to put out a public SOS here to get 
random bits of advice on deliverability to Google is an indictment of 
what has become of email. It is clearly no longer possible to know how 
to do email well.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Matt Palmer via mailop]

[Fixed TOFU]

On Fri, Feb 24, 2023 at 03:57:00PM -0500, Christine Borgia via mailop wrote:
> On Fri, Feb 24, 2023 at 1:09 PM Benny Pedersen via mailop 
> wrote:
> 
> > Christine Borgia via mailop skrev den 2023-02-24 17:17:
> >
> > >>> 421 4.7.0 [149.72.90.158 15] Our system has detected that this
> >
> > > If someone could reach out to me, I'd greatly appreciate it. I'm not
> > > sure what to advise this customer except to say that email may not
> > > work for their business model.
> >
> > note 421 4.7.0 ?
> >
> > this is softfails not hardfails, so mail in queue will try next 5 days
> > to deliver it, if that expire it would be returned to sender
>
> It is transient, but they are blocks and are not retried. Weird, right?

That's something to talk to your ESP about.  They're in charge of retrying.

- Matt

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Simon Greenwood via mailop]

On 24/02/2023 19:19, Rob McEwen via mailop wrote:
Did you read the link?  Does your ESP have a postmaster account?  If 
they don't send enough messages ("a large volume") to Gmail they 
won't provide any information about reputation, except that that 
itself indicates that you haven't (and can't) collect enough 
reputation to be other than low aka poor.  SORBS certainly doesn't 
like that IP address.

/mark


Mark,
The PTR record (and matching FCrDNS) clearly shows that this is an 
official Shopify sending IP. Any system that blocks these, with rare 
exception, probably isn't very smart? ...or doesn't mind having many 
egregious false positives, where users see legit/desired messages in 
the spam folder? And I don't totally fault the SORBS listing because 
SORBS is purposely very aggressive, and that is why most spam filters 
that use SORBS - only use it for adding a point or two to a scoring 
system, such as SpamAssassin - and SORBS is very beneficial when used 
that way! But blocking these based on some spams getting through 
Shopify, or based on a SORBS listing - isn't very smart.
The bottom line is that Google appears to be doing horrible spam 
filtering in this particular instance. Meanwhile, you should take a 
look at Christine Borgia's resume at LinkedIn. For just one example, 
she was THE Postmaster at AOL for several years.

https://www.linkedin.com/in/christineborgia/
So I think she knows/understands much more than what you give her 
credit for - and here's the thing - even if what you said about 
"collect enough information" is true - it's just not very smart for 
the spam filter of a hosting platform THIS large to treat such an IP 
as an "island" - without somewhat factoring in the other IPs which 
share common domains at the end of the PTR records (which pass FCrDNS 
and where they dynamic PTR records). In other words, a smarter system 
would factor into their analysis of one single legit Shopify outbound 
IP, the behavior/history of other IPs from this SAME sender, but of 
course not going too far with that, since an occasional individual IP 
can get severely compromised.


So, in this case, it certainly APPEARS that Google did NONE of that 
factoring in of other Shopify sending IPs, and treated this like an 
island?


Also, even if it could be shown that a shopify user was sending some 
spam out of that IP, unless that was an egregious situation - this is 
by far most likely a stupid Google spam-filtering mistake that's 
causing many legit/desired emails to be blocked, with probably far 
more such blockage of legit/desired emails than any spam this is 
blocking, if any.


Quite frankly, I'm constantly amazed at how often I see assumptions 
from so many - that assume Google is always right and the other entity 
is wrong - even when all the facts point to thy opposite.


This (which I didn't know) adds a whole different aspect to the issue - 
much has been said about how email is now centralised and is almost 
impractical to run as a small operator level, but if a company like 
Shopify and indeed Sendgrid can't assure mail delivery because the 
largest mail operators will reject mail sporadically based on 
non-specific criteria, and more that someone in Christine's position 
doesn't have someone they can call at Google or Microsoft or wherever, 
then there's a bigger problem.


Simon

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Christine Borgia via mailop]

It is transient, but they are blocks and are not retried. Weird, right?

On Fri, Feb 24, 2023 at 1:09 PM Benny Pedersen via mailop 
wrote:

> Christine Borgia via mailop skrev den 2023-02-24 17:17:
>
> >>> 421 4.7.0 [149.72.90.158 15] Our system has detected that this
>
> > If someone could reach out to me, I'd greatly appreciate it. I'm not
> > sure what to advise this customer except to say that email may not
> > work for their business model.
>
> note 421 4.7.0 ?
>
> this is softfails not hardfails, so mail in queue will try next 5 days
> to deliver it, if that expire it would be returned to sender
>
> follow the url in that tempfail to get support is best advise from me
>
> clean up time here
>
> https://multirbl.valli.org/lookup/149.72.90.158.html
>
> that sayed i dont belive google uses outside rbl lists
>
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jarland Donnell via mailop]

On 2023-02-24 12:38, Andrew C Aitchison via mailop wrote:

On Fri, 24 Feb 2023, Alessandro Vesely via mailop wrote:


On Fri 24/Feb/2023 18:41:34 +0100 Christine Borgia via mailop wrote:


I also should have mentioned we use shared IPs so there is no issue 
with volume from our servers, however volume from the domain is 
definitely spikey. They only send every 2-3 months when they promote 
a new event.



I think you should try and send at least ~100 mails every day to keep 
an IP warm.  Otherwise you're a ghost...


So now you have to send spam to avoid being thought to be a spammer ?


Microsoft is the same way. It's too common now that you need a continual 
and reasonable volume of email from an IP to keep a high reputation. 
It's difficult to publicly point to the result in the places that matter 
because they're all closed and internal, but it's very noticeable and MS 
will tell you it's true if you talk to the right people. This is why I 
cut our sending pool down to a /24 from a /22 a while back, spreading 
out the volume too thin caused increased spam folder delivery 
(determined by testing and volume of user complaints).

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Mark Milhollan via mailop]

On Fri, 24 Feb 2023, Rob McEwen wrote:

Quite frankly, I'm constantly amazed at how often I see assumptions from so 
many - that assume Google is always right and the other entity is wrong - even 
when all the facts point to the opposite.


I never said Google was "right", but if you want to deliver to them you 
have to jump through their hoops.  If you don't have enough volume to 
qualify for them to provide data -- Google's vague metric -- it turns 
into a guessing game, so you have to look for other indicators such as 
having any RBL hits which the IP address quoted does (SORBS listings 
within the last 28 days).



/mark
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Jay Hennigan via mailop]

On 2/24/23 10:40, Simon Greenwood via mailop wrote:

61 entries in SORBS, last January 23rd .

This should be noted to your ESP


$ whois 149.72.90.158
NetRange:   149.72.0.0 - 149.72.255.255
CIDR:   149.72.0.0/16
NetName:SENDGRID-149-72-0-0-16

Good luck with that.
 --
Jay Hennigan - j...@west.net
Network Engineering - CCIE #7880
503 897-8550 - WB6RDV

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Rob McEwen via mailop]

>which pass FCrDNS and where they dynamic PTR records

oops -  typo - I meant to say, "and where they're NOT dynamic"

Rob McEwen, invaluement
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Rob McEwen via mailop]

Did you read the link?  Does your ESP have a postmaster account?  If they don't send 
enough messages ("a large volume") to Gmail they won't provide any information 
about reputation, except that that itself indicates that you haven't (and can't) collect 
enough reputation to be other than low aka poor.  SORBS certainly doesn't like that IP 
address.
/mark


Mark,
The PTR record (and matching FCrDNS) clearly shows that this is an 
official Shopify sending IP. Any system that blocks these, with rare 
exception, probably isn't very smart? ...or doesn't mind having many 
egregious false positives, where users see legit/desired messages in the 
spam folder? And I don't totally fault the SORBS listing because SORBS 
is purposely very aggressive, and that is why most spam filters that use 
SORBS - only use it for adding a point or two to a scoring system, such 
as SpamAssassin - and SORBS is very beneficial when used that way! But 
blocking these based on some spams getting through Shopify, or based on 
a SORBS listing - isn't very smart.
The bottom line is that Google appears to be doing horrible spam 
filtering in this particular instance. Meanwhile, you should take a look 
at Christine Borgia's resume at LinkedIn. For just one example, she was 
THE Postmaster at AOL for several years.

https://www.linkedin.com/in/christineborgia/
So I think she knows/understands much more than what you give her credit 
for - and here's the thing - even if what you said about "collect enough 
information" is true - it's just not very smart for the spam filter of a 
hosting platform THIS large to treat such an IP as an "island" - without 
somewhat factoring in the other IPs which share common domains at the 
end of the PTR records (which pass FCrDNS and where they dynamic PTR 
records). In other words, a smarter system would factor into their 
analysis of one single legit Shopify outbound IP, the behavior/history 
of other IPs from this SAME sender, but of course not going too far with 
that, since an occasional individual IP can get severely compromised.


So, in this case, it certainly APPEARS that Google did NONE of that 
factoring in of other Shopify sending IPs, and treated this like an 
island?


Also, even if it could be shown that a shopify user was sending some 
spam out of that IP, unless that was an egregious situation - this is by 
far most likely a stupid Google spam-filtering mistake that's causing 
many legit/desired emails to be blocked, with probably far more such 
blockage of legit/desired emails than any spam this is blocking, if any.


Quite frankly, I'm constantly amazed at how often I see assumptions from 
so many - that assume Google is always right and the other entity is 
wrong - even when all the facts point to the opposite.


Rob McEwen, invaluement

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Simon Greenwood via mailop]

61 entries in SORBS, last January 23rd .

This should be noted to your ESP, but you may be in a bit of cleft stick 
in this situation - changing IPs or  upgrading to a exclusive IP would 
still require reputation to be maintained, but shared IPs in a lot of 
ESPs now suffer from this kind of issue.


Simon

On 24/02/2023 17:51, Mark Milhollan via mailop wrote:

On Fri, 24 Feb 2023, Christine Borgia wrote:


I have a customer whose email campaign was heavily
blocked yesterday by Gmail with the error:

421 4.7.0 [149.72.90.158 15] Our system has detected that this 
message is

suspicious due to the very low reputation of the sending domain. To best
protect our users from spam, the message has been blocked. Please visit
https://support.google.com/mail/answer/188131 for more information.
v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp


Did you read the link?  Does your ESP have a postmaster account? If 
they don't send enough messages ("a large volume") to Gmail they won't 
provide any information about reputation, except that that itself 
indicates that you haven't (and can't) collect enough reputation to be 
other than low aka poor.  SORBS certainly doesn't like that IP address.



/mark
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Andrew C Aitchison via mailop]

On Fri, 24 Feb 2023, Alessandro Vesely via mailop wrote:


On Fri 24/Feb/2023 18:41:34 +0100 Christine Borgia via mailop wrote:


I also should have mentioned we use shared IPs so there is no issue with 
volume from our servers, however volume from the domain is definitely 
spikey. They only send every 2-3 months when they promote a new event.



I think you should try and send at least ~100 mails every day to keep an IP 
warm.  Otherwise you're a ghost...


So now you have to send spam to avoid being thought to be a spammer ?

--
Andrew C. Aitchison  Kendal, UK
   and...@aitchison.me.uk
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Mark Milhollan via mailop]

On Fri, 24 Feb 2023, Christine Borgia wrote:


I have a customer whose email campaign was heavily
blocked yesterday by Gmail with the error:

421 4.7.0 [149.72.90.158 15] Our system has detected that this message is
suspicious due to the very low reputation of the sending domain. To best
protect our users from spam, the message has been blocked. Please visit
https://support.google.com/mail/answer/188131 for more information.
v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp


Did you read the link?  Does your ESP have a postmaster account?  If 
they don't send enough messages ("a large volume") to Gmail they won't 
provide any information about reputation, except that that itself 
indicates that you haven't (and can't) collect enough reputation to be 
other than low aka poor.  SORBS certainly doesn't like that IP address.



/mark
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Benny Pedersen via mailop]

Christine Borgia via mailop skrev den 2023-02-24 17:17:


421 4.7.0 [149.72.90.158 15] Our system has detected that this



If someone could reach out to me, I'd greatly appreciate it. I'm not
sure what to advise this customer except to say that email may not
work for their business model.


note 421 4.7.0 ?

this is softfails not hardfails, so mail in queue will try next 5 days 
to deliver it, if that expire it would be returned to sender


follow the url in that tempfail to get support is best advise from me

clean up time here

https://multirbl.valli.org/lookup/149.72.90.158.html

that sayed i dont belive google uses outside rbl lists


___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Alessandro Vesely via mailop]

On Fri 24/Feb/2023 18:41:34 +0100 Christine Borgia via mailop wrote:


I also should have mentioned we use shared IPs so there is no issue with volume 
from our servers, however volume from the domain is definitely spikey. They 
only send every 2-3 months when they promote a new event.



I think you should try and send at least ~100 mails every day to keep an IP 
warm.  Otherwise you're a ghost...



Best
Ale
--





___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Christine Borgia via mailop]

Thanks Simon and Ken! I am not aware of them having any abuse outside of
our platform. I have no evidence of it.

I also should have mentioned we use shared IPs so there is no issue with
volume from our servers, however volume from the domain is definitely
spikey. They only send every 2-3 months when they promote a new event.

I noticed that they had a lot of deferrals about an unusual rate of
incoming mail when they started sending. Our ESP adapted and started
throttling the customer but perhaps not enough because then the blocks
started. The mail is now all getting delivered again, slowly, which in this
case is fine.

I'm going to talk to our ESP about throttling faster/more when they see
these Gmail deferrals to keep them from turning into blocks. That sounds
like it could work.

Thanks all! ~Christine

On Fri, Feb 24, 2023 at 11:57 AM Simon Greenwood via mailop <
mailop@mailop.org> wrote:

> Hello -
>
> Check the domain with MXToolbox if you haven't all ready - it's not
> impossible that the domain or servers are on a blacklist.
>
> Also check that SPF and DKIM/DMARC records are good and set up for your
> sending service and that reverse lookups are correct.
>
> Reputation in this case applies to email and not the standing of the
> domain, site or another considerations like that. It could be something as
> simple as Gmail not being used to seeing a large amount of mail from the
> sending servers - keeping a low level of traffic passing through them seems
> to improve reputation.
>
> Simon
> On 24/02/2023 16:17, Christine Borgia via mailop wrote:
>
> Hello! Long time no see. I have a customer whose email campaign was
> heavily blocked yesterday by Gmail with the error:
>
>
> 421 4.7.0 [149.72.90.158 15] Our system has detected that this message is
> suspicious due to the very low reputation of the sending domain. To best
> protect our users from spam, the message has been blocked. Please visit
> https://support.google.com/mail/answer/188131 for more information.
> v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp
>
> I would love to speak to someone at Gmail about this because I don't think
> the customer should have a very low reputation. Of course, I could be
> wrong, and Gmail is seeing something I'm not seeing. The customer has
> double opt-in, low complaints, good opens, lots of purchases, and an active
> social media presence with a lot of followers/fans. I think the issue is
> that they only send every couple of months. They are a fashion retailer
> that announces their upcoming event via email (the email we saw blocked)
> and their inventory sells out during the event.
>
> If someone could reach out to me, I'd greatly appreciate it. I'm not sure
> what to advise this customer except to say that email may not work for
> their business model.
>
> Thx!
>
> --
> *Christine Borgia*
> Email Deliverability Manager
> [image: Shopify]
> 
>
> ___
> mailop mailing listmailop@mailop.orghttps://list.mailop.org/listinfo/mailop
>
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>
___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Simon Greenwood via mailop]

Hello -

Check the domain with MXToolbox if you haven't all ready - it's not 
impossible that the domain or servers are on a blacklist.


Also check that SPF and DKIM/DMARC records are good and set up for your 
sending service and that reverse lookups are correct.


Reputation in this case applies to email and not the standing of the 
domain, site or another considerations like that. It could be something 
as simple as Gmail not being used to seeing a large amount of mail from 
the sending servers - keeping a low level of traffic passing through 
them seems to improve reputation.


Simon

On 24/02/2023 16:17, Christine Borgia via mailop wrote:
Hello! Long time no see. I have a customer whose email campaign was 
heavily blocked yesterday by Gmail with the error:



421 4.7.0 [149.72.90.158 15] Our system has detected that this
message is suspicious due to the very low reputation of the
sending domain. To best protect our users from spam, the
message has been blocked. Please visit
https://support.google.com/mail/answer/188131 for more
information.
v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp  


I would love to speak to someone at Gmail about this because I don't 
think the customer should have a very low reputation. Of course, I 
could be wrong, and Gmail is seeing something I'm not seeing. The 
customer has double opt-in, low complaints, good opens, lots of 
purchases, and an active social media presence with a lot of 
followers/fans. I think the issue is that they only send every couple 
of months. They are a fashion retailer that announces their upcoming 
event via email (the email we saw blocked) and their inventory sells 
out during the event.


If someone could reach out to me, I'd greatly appreciate it. I'm not 
sure what to advise this customer except to say that email may not 
work for their business model.


Thx!

--
*Christine Borgia*
Email Deliverability Manager
Shopify 




___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop

Re: [mailop] Gmail blocking of good customer

[Ken Simpson via mailop]

Hi Christine,
I wonder if the domain has been abused elsewhere?

Regards,
Ken

On Fri, Feb 24, 2023 at 8:21 AM Christine Borgia via mailop <
mailop@mailop.org> wrote:

> Hello! Long time no see. I have a customer whose email campaign was
> heavily blocked yesterday by Gmail with the error:
>
>
> 421 4.7.0 [149.72.90.158 15] Our system has detected that this message is
> suspicious due to the very low reputation of the sending domain. To best
> protect our users from spam, the message has been blocked. Please visit
> https://support.google.com/mail/answer/188131 for more information.
> v14-20020a05620a440e00b0073ba7884843si8095361qkp.8 - gsmtp
>
> I would love to speak to someone at Gmail about this because I don't think
> the customer should have a very low reputation. Of course, I could be
> wrong, and Gmail is seeing something I'm not seeing. The customer has
> double opt-in, low complaints, good opens, lots of purchases, and an active
> social media presence with a lot of followers/fans. I think the issue is
> that they only send every couple of months. They are a fashion retailer
> that announces their upcoming event via email (the email we saw blocked)
> and their inventory sells out during the event.
>
> If someone could reach out to me, I'd greatly appreciate it. I'm not sure
> what to advise this customer except to say that email may not work for
> their business model.
>
> Thx!
>
> --
> *Christine Borgia*
> Email Deliverability Manager
> [image: Shopify]
> 
> ___
> mailop mailing list
> mailop@mailop.org
> https://list.mailop.org/listinfo/mailop
>


-- 

Ken Simpson

CEO, MailChannels



Facebook   |  Twitter   |
LinkedIn  |  Help Center


Our latest case study video: watch here!

___
mailop mailing list
mailop@mailop.org
https://list.mailop.org/listinfo/mailop