Re: [mailop] Google unsolicited mail rejected with 421
Most likely the reasoning behind something like this is dkim replay attacks, where a message with a single recipient is re-sent to a large number of recipients. Of course mailing lists should be exempted, the challenge is if you reply to both the list and the person directly, the reply to the person didn't go through the list and can't be easily exempted. Oh, and you have to be careful how you make the exemption since you don't want the dkim reply spammers to figure it out and exempt themselves, there isn't a registry of legitimate mailing lists. Obviously if this is affecting a large number of folks as false positives, it could use some tuning... but I think the set of rules like this started years ago at this point, but new rules and tuning and just the general dynamics of the system will change the performance and effect. As for preferencing Google Groups, at least when I was on the team, Google Groups went in through the same smtp-in door as any other mail, and was treated the same as any other mail... in general, the only differences there was that it is a known mailing list host and as a large one had some special handling to differentiate between different groups... and this same type of thing was used for most of the large mailing list operators that the team was aware of. Open source mailing lists were also usually handled specially where possible, definitely out of proportion to their actual mail volume or affected users. At the time, debian mailing lists were also specially handled due to their poor email hygiene practices, in particular their opposition to using SPF because they didn't want to run a central MSA for people to use their debian.org email addresses. The issue with any special handling is that it adds tech debt to the system, makes it more complicated, and the handling can become out of date over time... I don't know if debian has started using SPF (a quick look says no), or if they don't, whether the system still has any special handling for them, since removing that special handling or having it break probably wouldn't move the needle on any metrics for the system as a whole. I wouldn't be surprised if we still have yahooogroups.com listed as a large scale mailing list system... even though it's gone or if groups.io is listed since it's probably grown large enough that it could be. Brandon On Sun, Mar 17, 2024 at 6:04 AM Benny Pedersen via mailop wrote: > Jaroslaw Rafa via mailop skrev den 2024-03-17 13:38: > > Dnia 16.03.2024 o godz. 13:08:52 Benny Pedersen via mailop pisze: > >> > >> bingo its why its tempfailed, gmail should redesign how to handle > >> maillists where message-id can come to inbound on gmail, should not > >> count on message-id abuse counts > > > > Well... from Google's point of view, it seems like a pretty effective > > mechanism to force people to move to *their* mailing list service, > > instead > > of running mailing list themselves... > > > > A monopoly wants to be a monopoly. > > sure any stupid user can forward mails that is received on maillist to > there own private gmail account, more or less its this > > stop forwarding mails as maillist subscriber is better, in generic world > is lots better when no mail is forwarded > > srs would not fix anything anyway > > setup forwarder with sasl client in mta to delivery to freemail provider > solves it > > > > ___ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Jaroslaw Rafa via mailop skrev den 2024-03-17 13:38: Dnia 16.03.2024 o godz. 13:08:52 Benny Pedersen via mailop pisze: bingo its why its tempfailed, gmail should redesign how to handle maillists where message-id can come to inbound on gmail, should not count on message-id abuse counts Well... from Google's point of view, it seems like a pretty effective mechanism to force people to move to *their* mailing list service, instead of running mailing list themselves... A monopoly wants to be a monopoly. sure any stupid user can forward mails that is received on maillist to there own private gmail account, more or less its this stop forwarding mails as maillist subscriber is better, in generic world is lots better when no mail is forwarded srs would not fix anything anyway setup forwarder with sasl client in mta to delivery to freemail provider solves it ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Dnia 16.03.2024 o godz. 13:08:52 Benny Pedersen via mailop pisze: > > bingo its why its tempfailed, gmail should redesign how to handle > maillists where message-id can come to inbound on gmail, should not > count on message-id abuse counts Well... from Google's point of view, it seems like a pretty effective mechanism to force people to move to *their* mailing list service, instead of running mailing list themselves... A monopoly wants to be a monopoly. -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am 16.03.2024 um 13:08:52 Uhr schrieb Benny Pedersen via mailop: > Marco Moock via mailop skrev den 2024-03-16 12:46: > > Am 14.03.2024 um 10:28:13 Uhr schrieb Julian Bradfield via mailop: > > > >> Their latest daftness (latest in my noticing it, anyway) is > >> rate-limiting on the basis of too many recipients for a single > >> message-id, where "too many" varies from 6 to 30. You'd think > >> they'd never heard of organization mailing lists. > > > > That seems to be the case here too. > > If I reply to somebody a gmail directly (not to the list) it gets > > through. > > bingo its why its tempfailed, gmail should redesign how to handle > maillists where message-id can come to inbound on gmail, should not > count on message-id abuse counts The current situation is even worse when mailing list subscribers forward their stuff to gmail. That will result in many, many "unsolicited" mails because those servers will try it a few times because of the tempfail. A rather crappy solution by Google. -- Gruß Marco Send spam to 1710590932mu...@cartoonies.org ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Marco Moock via mailop skrev den 2024-03-16 12:46: Am 14.03.2024 um 10:28:13 Uhr schrieb Julian Bradfield via mailop: Their latest daftness (latest in my noticing it, anyway) is rate-limiting on the basis of too many recipients for a single message-id, where "too many" varies from 6 to 30. You'd think they'd never heard of organization mailing lists. That seems to be the case here too. If I reply to somebody a gmail directly (not to the list) it gets through. bingo its why its tempfailed, gmail should redesign how to handle maillists where message-id can come to inbound on gmail, should not count on message-id abuse counts ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am 14.03.2024 um 10:28:13 Uhr schrieb Julian Bradfield via mailop: > Their latest daftness (latest in my noticing it, anyway) is > rate-limiting on the basis of too many recipients for a single > message-id, where "too many" varies from 6 to 30. You'd think they'd > never heard of organization mailing lists. That seems to be the case here too. If I reply to somebody a gmail directly (not to the list) it gets through. -- kind regards Marco Send spam to 1710408493mu...@cartoonies.org ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On 2024/03/14 10:28, Julian Bradfield via mailop wrote: > Their latest daftness (latest in my noticing it, anyway) is > rate-limiting on the basis of too many recipients for a single > message-id, where "too many" varies from 6 to 30. You'd think they'd > never heard of organization mailing lists. Same problem for the openbsd.org mailing lists: "Gmail has detected this message exceeded its quota for sending messages with the same Message-ID. To best protect our users, the message has been temporarily rejected" ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am Thu, 14 Mar 2024 10:04:42 +0100 schrieb Marco Moock via mailop : > Although, I send only a very small amount of mail to Google. Do they > use that to calculate the rate? I got that error again. I participated in some mailing lists with gmail subscribers. One of those subscribers has a forward to Google and I got an email from their MTA that Google temp rejected it there too. Does every attempt count here for Google's calculation? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On Thu, 14 Mar 2024, Johann Klasek via mailop wrote: On Thu, Mar 14, 2024 at 12:03:46PM +0100, Marco Moock via mailop wrote: Am 14.03.2024 schrieb Julian Bradfield via mailop : On 2024-03-14, Marco Moock via mailop wrote: sendmail tried to deliver it 20 times during the night - this morning I deleted the mail from mqueue. That's a fairly aggressive retry strategy. That is the default in sendmail. Is there any standard that defines the retry rates or at least a best practise? The exim default is (description from the documentation): * * F,2h,15m; G,16h,1h,1.5; F,4d,6h This causes any temporarily failing address to be retried every 15 minutes for 2 hours, then at intervals starting at one hour and increasing by a factor of 1.5 until 16 hours have passed, then every 6 hours up to 4 days. If an address is not delivered after 4 days of temporary failure, it is bounced. The time is measured from first failure, not from the time the message was received. It depends on the average queue size and contention of the queue. With many entry a queue runner interval might be exhausted easily and retry streches over multiple intervals. I would regard a 10 to 15 minute queue runner interval as acceptable. If the queue is nearly empty the retry happens 4 to 6 times a hour. That's not very aggressive meanwhile. Exim's queue runner doesn't try to deliver every mail on every run through. I hadn't expected that other queue runners would. Our inbound queuerunner operates in permanent queue running mode which could lead to a retry every minute if the queue is nearly empty. This is more or less "aggressive" ... An *inbound* queue runner sounds like a special case - I would expect it to be more "aggressive". -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On Thu, Mar 14, 2024 at 12:17:39PM +0100, Slavko via mailop wrote: > D??a 14. 3. o 12:03 Marco Moock via mailop napísal(a): > > > Is there any standard that defines the retry rates or at least a best > > practise? > > RFC 5321, sect. 4.5.4.1: > > In general, the retry interval SHOULD be at least 30 minutes... A recommendation from the year 2008, one might re-consider this under today's standards. Beside this, MTA queue runners implement some dynamically calculated backoff strategy based on time interval stretching (usually exponential) or priority based, or a combination of these strategies. Johann ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On Thu, Mar 14, 2024 at 12:03:46PM +0100, Marco Moock via mailop wrote: > Am 14.03.2024 schrieb Julian Bradfield via mailop : > > > On 2024-03-14, Marco Moock via mailop wrote: > > > sendmail tried to deliver it 20 times during the night - this > > > morning I deleted the mail from mqueue. > > > > That's a fairly aggressive retry strategy. > > That is the default in sendmail. > > Is there any standard that defines the retry rates or at least a best > practise? It depends on the average queue size and contention of the queue. With many entry a queue runner interval might be exhausted easily and retry streches over multiple intervals. I would regard a 10 to 15 minute queue runner interval as acceptable. If the queue is nearly empty the retry happens 4 to 6 times a hour. That's not very aggressive meanwhile. Our inbound queuerunner operates in permanent queue running mode which could lead to a retry every minute if the queue is nearly empty. This is more or less "aggressive" ... Johann ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Dňa 14. 3. o 12:03 Marco Moock via mailop napísal(a): Is there any standard that defines the retry rates or at least a best practise? RFC 5321, sect. 4.5.4.1: In general, the retry interval SHOULD be at least 30 minutes... -- Slavko https://www.slavino.sk/ ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am 14.03.2024 schrieb Julian Bradfield via mailop : > On 2024-03-14, Marco Moock via mailop wrote: > > sendmail tried to deliver it 20 times during the night - this > > morning I deleted the mail from mqueue. > > That's a fairly aggressive retry strategy. That is the default in sendmail. Is there any standard that defines the retry rates or at least a best practise? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On 2024-03-14, Marco Moock via mailop wrote: > sendmail tried to deliver it 20 times during the night - this morning > I deleted the mail from mqueue. That's a fairly aggressive retry strategy. If there's something about that message gmail doesn't like, then retrying that often might be enough to reinstate the limit each time. Generally best to back off the retry rate after a couple of hours. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am 14.03.2024 schrieb Julian Bradfield via mailop : > They don't reject with 5xx because they're not rejecting that message, > they are rate-limiting you or the network you're on. > I get this often, because one user forwards their mail to > gmail, including all the spam. Google rejects the spam, and from time > to time also rate-limits me. Which I don't care about, because I send > very little mail, and it goes through on the next run. sendmail tried to deliver it 20 times during the night - this morning I deleted the mail from mqueue. Other mails to Google now go through, so it seems it is not a complete limitation of my IP address/ISP. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On 2024-03-14, Marco Moock via mailop wrote: > Hello! > > Yesterday I replied somebody directly on debian-users (he uses a crappy > mailer and sends to the author and the mailing list...). > > Gmail doesn't like this mail, but rejects it with a tempfail. I've now > deleted it from mqueue. > > Mar 14 06:54:17 srv1.xyz sm-mta[498019]: 42DK6aqc496761: > to=, delay=09:47:40, xdelay=00:00:03, mailer=esmtp, > pri=5370980, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.7.28, > reply=421 4.7.28 Gmail has detected an unusual rate of unsolicited mail > originating, stat=Deferred: 421-4.7.28 Gmail has detected an unusual > rate of unsolicited mail originating > > It was only this message. Why don't they reject them with 5xx when they > treat the mail as unsolicited? > It was only this mail, my server wasn't abused by spammers. > > Although, I send only a very small amount of mail to Google. Do they > use that to calculate the rate? They don't reject with 5xx because they're not rejecting that message, they are rate-limiting you or the network you're on. I get this often, because one user forwards their mail to gmail, including all the spam. Google rejects the spam, and from time to time also rate-limits me. Which I don't care about, because I send very little mail, and it goes through on the next run. Google is brain-dead about "unsolicited" mail - whatever machine learning it's doing, is crap. Their latest daftness (latest in my noticing it, anyway) is rate-limiting on the basis of too many recipients for a single message-id, where "too many" varies from 6 to 30. You'd think they'd never heard of organization mailing lists. Julian. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
Am 14.03.2024 schrieb Stefano Bagnara via mailop : > On Thu, 14 Mar 2024 at 10:09, Marco Moock via mailop > wrote: > > Mar 14 06:54:17 srv1.xyz sm-mta[498019]: 42DK6aqc496761: > > to=, delay=09:47:40, xdelay=00:00:03, mailer=esmtp, > > pri=5370980, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.7.28, > > reply=421 4.7.28 Gmail has detected an unusual rate of unsolicited > > mail originating, stat=Deferred: 421-4.7.28 Gmail has detected an > > unusual rate of unsolicited mail originating > > The full message from Gmail gives you more hints about the problem: it > may be a rate limiting for the DKIM signing domain, for the SPF > domain, for an URL included in the email, for the sender and so on. IIRC it was the DKIM signing domain, but I can't see the full message in the sendmail log. > You need the full message to make a better guess, but, for example, if > Google is receiving a lot of spam with a given URL domain in the body > it may start rate limiting that content from every source, including > you. Is there a test address at google where I can send such messages to test it? ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Google unsolicited mail rejected with 421
On Thu, 14 Mar 2024 at 10:09, Marco Moock via mailop wrote: > Mar 14 06:54:17 srv1.xyz sm-mta[498019]: 42DK6aqc496761: > to=, delay=09:47:40, xdelay=00:00:03, mailer=esmtp, > pri=5370980, relay=alt4.gmail-smtp-in.l.google.com., dsn=4.7.28, > reply=421 4.7.28 Gmail has detected an unusual rate of unsolicited mail > originating, stat=Deferred: 421-4.7.28 Gmail has detected an unusual > rate of unsolicited mail originating The full message from Gmail gives you more hints about the problem: it may be a rate limiting for the DKIM signing domain, for the SPF domain, for an URL included in the email, for the sender and so on. E.g. the full message could be like this: 421-4.7.28 Gmail has detected an unusual rate of unsolicited mail originating 421-4.7.28 from your DKIM domain [#redacted# 36]. To protect 421-4.7.28 our users from spam, mail sent from your domain has been temporarily 421-4.7.28 rate limited. For more information, go to 421-4.7.28 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.28 review our Bulk Email Senders Guidelines. #redacted# - gsmtp The SPF tempfail error starts with the same words, too. The URL one have a different prehamble. I don't know how many other errors you can get, but if you want to investigate you need the full error. > It was only this message. Why don't they reject them with 5xx when they > treat the mail as unsolicited? They give you a tempfail because it is a rate limiting: so if you try later it usually will work. > It was only this mail, my server wasn't abused by spammers. > > Although, I send only a very small amount of mail to Google. Do they > use that to calculate the rate? You need the full message to make a better guess, but, for example, if Google is receiving a lot of spam with a given URL domain in the body it may start rate limiting that content from every source, including you. Stefano -- Stefano Bagnara Apache James/jDKIM/jSPF VOXmail/Mosaico.io/VoidLabs ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
