FYI that the missing domain in the tempfail message should now be fixed (thanks to Wei)! Please let me know if you are not seeing it.
On Thu, Feb 15, 2024 at 11:50 PM Stefano Bagnara via mailop < mailop@mailop.org> wrote: > On Fri, 16 Feb 2024 at 05:02, Evan Burke <evan.s.bu...@gmail.com> wrote: > >> It sounds like you're looking for a technical solution to a non-technical >> problem. This bounce code is quite rare, occurring for a fraction of a >> percent of the senders in my dataset. When it happens, it's a pretty strong >> sign the sender has a spam problem and needs to do some cleanup; adding >> rate limiting on your end won't change that. >> > > No, your conclusions don't match my experience. It's weird somehow people > think the only cause for issues is spam :-) > > Turns out the issue occours also to very good senders the first time they > authenticate their flow with their own domain, so the first time their > email pass from being signed with the ESP dkim to being signed with both > the ESP dkim and their own domain dkim (and being aligned). > > So, it is not about unsolicited, but about "new flow" that Google does not > recognize (they probably don't get the double dkim signature to keep track > of the reputation in this change). > > In fact the very same senders after a first time they get this error > starts sending without any issue: if it was about "a spam problem" the > issue would occour every time (or multiple times). > > This is happening to us a lot because in the past months (after the > Yahoogle announce) we forced many of our customers in the 3-20K daily email > to authenticate so to pass DMARC. Previously they passed SPF and DKIM but > with a shared ESP domain, and not DMARC because there was no alignment, but > this worked fine (and still works fine, even for big senders, even if we > don't know how long: maybe this is because of the established good > reputation of our shared domain). > > Considering the error from Google is not recognized as "special" our mta > retries sending to every google mx every 10 minutes and the whole email > (the error occours at ".") is transmitted 5 times per attempt. > So sometimes a single email is transmitted hundreds of times before it > finally gets delivered. > > We also have indicators that even if the emails are delayed for hours at > the end they land the inbox and not the spam folder (open rates at gmail > are in the 40%). > > Now we implemented a special MTA code that will recognize the specific > error and will delay for 1 hour every email with the same auth domain of > the email that received the error and we are fine: this way we still have > delays in sending that email flow, but we have almost zeroed > retransmissions (temp fails): this means many millions temp fail. And it > was a technical solution to a technical problem :-) > > Also, Google acknowledged the missing domain indication in the refuse > message is not expected: who knows, maybe the fact they are not able to > handle the reputation transition from the shared DKIM to the additional > sender dkim is somehow related to this. Another technical problem. > > Stefano > > >> On Tue, Feb 13, 2024 at 9:30 AM Stefano Bagnara via mailop < >> mailop@mailop.org> wrote: >> >>> On Tue, 13 Feb 2024 at 18:09, Matus UHLAR - fantomas <uh...@fantomas.sk> >>> wrote: >>> >>>> On 05.02.24 14:56, Stefano Bagnara via mailop wrote: >>>> >we are a small ESP and every email sent from our system has SPF+DKIM >>>> >authentication from our system and most email also have a second DKIM >>>> >signature (one signature with our domain, one with the domain of the >>>> >sender). >>>> >>>> is bago.org that domain? >>>> >>> >>> No, it's not :-) bago.org is my personal domain and does not send bulk >>> emails. >>> >>> A Google person already contacted me and identified the logs with the >>> missing dkim domain indication. I guess they consider it (the missing >>> domain indication in the error) a bug but I had no updates since then. >>> >>> Considering the email are signed with 2 different DKIM signatures they >>> told me which one the error is about, and it is the "customer one" and this >>> is enough for me. >>> >>> Unfortunately when I move a customer from "not authenticated" to >>> "authenticated" seems like Google is not able to recognize the consistent >>> DKIM signature on my own domain and starts rate limiting the new additional >>> DKIM signature for the customer. This is creating delay issues for some >>> customer in the first days after the authentication, but I understood >>> there's nothing I can do to improve this. >>> >>> >>>> - it does not have DMARC, perhaps this is you problem? >>>> >>> >>> No >>> >>> - it has some redundant SPF records: >>>> >>> >>> I'm not aware of issues with redundant SPF records, as long as I stay in >>> the 10 lookup: what are you talking about? >>> >>> >>>> I'm just not sure if I should advise you to drop >>>> "include:spf.mailvox".it or >>>> replace "include:spf.void.it" with "include:_spf.google.com" >>>> >>> >>> No, I need that because spf.void.it may change in future. >>> >>> >>>> note that void.it also does not have DMARC and mailvox.it, while >>>> having >>>> SPF record for "spf.mailvox.it" does not have SPF for "mailvox.it". >>>> >>> >>> Everything is expected. >>> mailvox.it does not send emails, only subdomains of mailvox.it sends >>> email and they have SPF records. >>> >>> BTW none of those domains are involved in the issue I have with google >>> :-D >>> >>> And here are my attempt to answer my own questions: >>> >>>> My questions: >>>> 1) is it expected that the error does not report the DKIM domain but >>>> only [ 36]? >>>> >>> >>> It is a bug from Google. >>> >>> >>>> 2) how much is the "rate limit"? Does the rejected attempt count in >>>> this rate? >>>> >>> >>> Starts as low as 1000/2000 emails per hour and looks like an hourly >>> limit. >>> >>> >>>> 3) when, how often, how long does Google expects we retry the delivery >>>> after a 421-4.7.28 ? >>> >>> >>> I don't know but I changed the retry schedule for those flows so to wait >>> 1 hour when I see that error. >>> I also think this is not about "Unsolicited rate" but simply about "rate >>> limiting" a dkim domain they don't recognize: it is unfortunate that they >>> don't keep trusting the other DKIM domain when they see 2 DKIM signatures >>> and one is well known to them, but given this seems to happen once per >>> customer we'll work around it. >>> >>> Stefano >>> >>> >>>> >>>> >>>> >Since the past november we started seeing some >>>> UnsolicitedRateLimitError >>>> >temporary error and while some of them refer to an SPF domain ( "... >>>> from >>>> >your SPF domain [$domain$ 35] ..." ) we also see a lot of "... >>>> from >>>> >your DKIM domain [ 36]. ..." where no domain is in brackets, but >>>> only >>>> >the number 36. >>>> >>>> Google started rolling increased requirements since last year, perhaps >>>> they applied >>>> stricter requirements for your domain sooner. >>>> >>>> >>>> >The email being tempfailed are signed with multiple keys for multiple >>>> >domains, so I don't know what to "rate limit" (and how). >>>> > >>>> >Here is the full error received at the "." after sending the full body: >>>> >> 421-4.7.28 Gmail has detected an unusual rate of unsolicited mail >>>> >originating >>>> >> 421-4.7.28 from your DKIM domain [ 36]. To protect our users >>>> from >>>> >spam, >>>> >> 421-4.7.28 mail sent from your domain has been temporarily rate >>>> limited. >>>> >For >>>> >> 421-4.7.28 more information, go to >>>> >> 421-4.7.28 >>>> https://support.google.com/mail/?p=UnsolicitedRateLimitError >>>> >to >>>> >> 421 4.7.28 review our Bulk Email Senders >>>> > >>>> >Given it is a temporary failure our system keeps retrying very often, >>>> >resending the full message dozens times before it's accepted. >>>> > >>>> >Our Google Postmaster Tools (for our main domain signing every email) >>>> >doens't show anything problematic: >>>> >- Spam rate is < 0.2% >>>> >- IP reputation good for every IP >>>> >- Domain reputation is good >>>> >- "Feedback Loop" is completely empty (we have a token for each sender >>>> but >>>> >our senders are small: what's the volume required to show up here?) >>>> >- Authentication is 100% for SPF/DKIM/DMARC >>>> >- Cryptography is 100% TLS >>>> >- Delivery errors is shaky, most days is zero, but sometimes is 80%, >>>> >because of the tempfails above >>>> > >>>> >My questions: >>>> >1) is it expected that the error does not report the DKIM domain but >>>> only >>>> >[ 36]? >>>> >>>> Hard to say, but you should know sending domain. >>>> >>>> >2) how much is the "rate limit"? Does the rejected attempt count in >>>> this >>>> >rate? >>>> >>>> they mention ~5000 mails a day >>>> >>>> https://support.google.com/a/answer/14229414#zippy=%2Cwhat-is-a-bulk-sender >>>> >>>> >>>> This number may change, I guess they don't want anyone avoid their >>>> limits by >>>> splitting mail flow... >>>> >>>> >>>> >3) when, how often, how long does Google expects we retry the delivery >>>> >after a 421-4.7.28 ? >>>> >>>> I have no idea, would apply standard delay from 5 minutes to few hours. >>>> >>>> -- >>>> Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ >>>> Warning: I wish NOT to receive e-mail advertising to this address. >>>> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. >>>> M$ Win's are shit, do not use it ! >>>> >>> >>> >>> -- >>> Stefano Bagnara >>> Apache James/jDKIM/jSPF >>> VOXmail/Mosaico.io/VoidLabs >>> _______________________________________________ >>> mailop mailing list >>> mailop@mailop.org >>> https://list.mailop.org/listinfo/mailop >>> >> > > -- > Stefano Bagnara > Apache James/jDKIM/jSPF > VOXmail/Mosaico.io/VoidLabs > _______________________________________________ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop >
_______________________________________________ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
