As Brandon points out, it's not that simple. Some nice examples from one customer's bounce logs today - "552 Transaction <uid>@<domain.net> failed, remote said "550 No such user" (#5.1.1)" Obviously, we don't want to resend to this address again, but... is it a 552 (mailbox exceeded allocation), a 550 (mailbox unavailable doesn't necessary mean no such user) or a 5.1.1 (bad destination address). A 552 shouldn't match with a x.1.1. They're two different bounce reasons that are mutually exclusive, but we see stuff like this all the time. If it's a 552, then we could retry this address again later because the mailbox is full. Most likely it's a 5.1.1 and the address is completely invalid so we suppress it.
Then there are the "554 Invalid mailbox: <uid>@<domain2>.com" and the "554 rejecting banned content" bounces. 554 is a valid response for an invalid mailbox although 550 and 553 are more common there. Going strictly by 554, you would think retrying would be fine because it's transaction-specific, but not suppressing the first will get you blocked whereas the second should be retried once you've resolved the root cause. "550 mailbox unavailable" sometimes means the address is invalid and other times means you aren't allowed to access the mailbox due to technical or reputation issues. The mailbox is either completely unavailable or the mailbox is only unavailable to you or the mailbox can't be internally accessed. Or the "550 5.7.1 <u...@domain3.com>... Relaying denied" which may mean that the mail is being forwarded and the forward is failing or that you are not allowed to access this mailbox at this time except for when it means that the address is invalid. Is "550 5.1.1 unknown or illegal alias" an invalid address or is it a directory failure? (This ISP passes a 4.4.3 for directory failures, but many others don't or can't because of system architecture limitations.) On Fri, Dec 18, 2015 at 11:40 AM, Brandon Long <bl...@google.com> wrote: > 4xx vs 5xx is only part of the story. We're talking about whether the > next message will succeed. > > Sure, a 4xx response probably mostly means go ahead and try the next > message, but a 5xx doesn't mean that no messages will be accepted for this > account, just this one wasn't accepted. > > And agreed, some of the extended status codes can be interpreted in this > way fairly easily, but others... > > Brandon > On Dec 18, 2015 7:35 AM, "Ian Eiloart" <i...@sussex.ac.uk> wrote: > >> >> > On 18 Dec 2015, at 01:55, Karen Balle <kba...@rcn.com> wrote: >> > >> > Brandon, >> > >> > I really like this idea, even as a mental exercise. There would be a >> lot of usefulness in being able to clearly dictate from the enhanced codes >> when our MTAs should stop, back off, or go into an extended timeout. >> >> Well, that should be pretty clear from the 4xx vs 5xx status. If it’s >> 4xx, you should queue the message for later delivery, otherwise generate a >> bounce. I guess if the recipient wants to indicate that you should reduce >> your sending rate, that’s something that needs a new status, but you could >> use "4.3.1 Mail system full", or "X.4.5 Mail system congestion" because >> the sensible response would be to back off. >> >> >> >> > Perhaps something like having the enhanced code be tied to the primary >> reason for an organization not accepting email would be useful? I can't >> see where we could (or should) come up with a system where every reason an >> email isn't being accepted is communicated in a single NDR. Filters are >> getting horribly complicated and there are often multiple reasons why a >> sender's mail isn't being accepted. >> >> >> > From an ESP perspective, being able to tell a customer that Gmail >> clearly said in a bounce 1) It's you, not them. >> >> There are plenty of codes that say this. All of these are errors that >> could be fixed by the sender (not their MTA) >> >> X.1.1 Bad destination mailbox address >> X.1.2 Bad destination system address >> X.1.3 Bad destination mailbox address syntax >> X.1.6 Destination mailbox has moved, No forwarding address >> X.1.7 Bad sender's mailbox address syntax >> X.1.8 Bad sender's system address >> X.2.3 Message length exceeds administrative limit [per mailbox] >> X.3.4 Message too big for system >> X.5.3 Too many recipients >> >> > 2) Your content/domain rep/IP rep is horrible. >> >> X.7.1 Delivery not authorized, message refused >> The sender is not authorized to send to the destination. >> >> But, there’s plenty of scope for extension here: with three digits >> available for . Perhaps it’s time to look at extending the set of enhanced >> error codes available. A 20th anniversary celebration! >> RFC 1893 Mail System Status Codes January 1996 >> RFC 3463 Enhanced Mail System Status Codes January 2003 >> >> > 3) and you should feel bad >> >> X.7.10 Go directly to Jail, do not pass GO! ;^) >> >> >> There’s a registry, and a review might be a good idea. >> >> https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml#smtp-enhanced-status-codes-2 >> >> > would cut through a lot of the pushback we get now. >> >> >> > I could see a lot of use for anyone with a shared IP infrastructure, as >> well. If most of the DSNs from shared IP space came back with an enhanced >> code pointing to IP reputation but a handful of customers have more >> specific content or sender-related codes, it gives us amazing insight into >> the troublemakers trying to hide on the shared IPs. How much benefit do >> you think most mailbox providers would get if ESPs were able to more >> reliably adapt around having that extra certainty in our decision-making >> process? >> > >> > Couple of ideas for it >> > Mailbox specific - invalid account, recently closed account, mailbox >> full, recipient-configured rejection rules >> > ISP specific - server/network saturation (which we should treat with a >> full stop on traffic from all our senders rather than an extended retry for >> a single sender with traffic shaping relating to reputation), LDAP failure, >> server farm under attack by backhoe or glowing molerats >> > IP specific - traffic shaping for excessive volume for current >> reputation (very handy for the holidays and ramp ups), blocked for >> complaints, blocked for excessive invalid users/spamtraps, blocked for >> phishing/malware, traffic spread across too many IPs, snowshoe, your >> provider sucks, get off my lawn >> > Sender specific - could be broken down for domain, subdomain, or >> localpart combinations if we're feeling really ambitious but that could be >> providing too much feedback that could be abused. Marketing vs >> transactional is often split this way, though. Helpful for customers using >> their mailbox provider's domain. Would allow ESPs to set rules to stop >> mailing for specific senders immediately and definitely ties in with >> compliance standards. >> > Authentication and configuration failures - DMARC, no rDNS, missing A >> or MX records (I had a fight with a customer this week about setting those >> up) >> > Content specific - Again, could be easily abused but there are some >> ways it could be helpful. Redirector who doesn't handle their spam, >> linking to commonly phished sites, bad landing page rep (possibly >> malicious), Normally good sender who included third party content on a send. >> > >> > Karen >> > >> > On Wed, Dec 16, 2015 at 11:35 PM, Brandon Long <bl...@google.com> >> wrote: >> > We have automated systems, especially for Google groups, which are >> basically doing the same thing, and it's a pain. >> > >> > I agree that the SMTP status code is mostly useless from this >> prospective, and the extended ones aren't much better. >> > >> > It might be nice if there was a concept of whether a failure is message >> specific, sender specific, etc... But anything like that would be gamed >> both for antispam and for spammers. >> > >> > Still, may be worth a thought experiment, a replacement for the >> extended status codes or even a bcp about how they should be used. >> > >> > On Dec 16, 2015 6:03 PM, "Karen Balle" <kba...@rcn.com> wrote: >> > Since there's been a lot of drift on this topic and I'm not talking >> about IPv6, DNSSEC, or delivery to Gmail.... >> > >> > For NDRs and DSNs to be truly useful to an ESP, we tend to break them >> down into more buckets than two. >> > >> > Hard bounce/invalid (do not retry, generally a 5xx) >> > Soft bounce (temporary and related to a specific email address, >> generally a 4xx, try again later during the current send, Yahoo uses 4xx >> where we would expect a 5xx such as with some of the traffic shaping >> deferrals) >> > Technical (DNS failures and the like, try again later, generally 5xx >> but depends on the reason) >> > Spam or block (stop for this send, try this recipient on future sends, >> mix of 5xx and 4xx) >> > Unknown (parser doesn't have an appropriate string, manual review and >> update of systems required) >> > >> > The primary challenge from a technical perspective is making sure that >> any changes in friendly language in an NDR or DSN is updated on the bounce >> parser. We do tend to MOSTLY ignore 5xx and 4xx, but that is more a >> function of how those are used by mailbox providers. The desired end >> result is that we send all email in a manner that is acceptable to each >> mailbox provider based on their (technical and personal) feedback and >> desired rate limits. Extended codes are useful, when you know how a >> provider applies them, but again, there is such variation in the how codes >> are applied and when that it's all but impossible for someone not steeped >> in the intricacies of RFCs to interpret them properly. >> > >> > Not all customers have access to people who spend a lot of time going >> over bounces and making them actionable. A lot of our job as an ESP is >> taking the technical and making it understandable to someone who doesn't >> have the training and background on RFCs and "proper" technical jargon. >> I've spent the last 18 years or so going over NDRs and DSNs and some still >> throw me. Filters are also increasingly more complex and we see the same >> bounce used for multiple purposes, so cross-referencing which bounces >> happen where is becoming more important to understanding the root issue for >> a customer. >> > >> > >> > >> > Cheers, >> > Karen >> > >> > On Fri, Dec 11, 2015 at 9:06 AM, Ian Eiloart <i...@sussex.ac.uk> wrote: >> > I wonder why they don’t use the terminology from the RFCs: "reject", >> "defer", "non-delivery notification", "delay notification"? >> > >> > As it is, when you say "Hardbounce", I don’t know whether you’re >> referring to an SMTP 5yz reply (a rejection) by the receiving MTA or an rfc >> 3461 "failed" DSN sent to the sender to alert them to the problem. >> > >> > Similarly, I don’t know whether "softbounces" means an rfc5321 >> temporary failure reply , or an rfc 3461 "delayed" DSN. Neither of which >> mean quite what you said, since the intention is that the sending MTA will >> retry for a period. The sending MTA can’t use other means. If the original >> sender tries again, then the recipient will get duplicate messages when the >> fault is resolved. >> > >> > If the ESP is good, the the sender will have access to a support team, >> so all faults should be actionable. >> > >> > >> > > On 11 Dec 2015, at 11:59, David Hofstee <da...@mailplus.nl> wrote: >> > > >> > > That’s why, in ESP parlance, there are two sorts of bounces, to keep >> it simple: >> > > - Hardbounces: The recipient address does not work. Contact >> through other means. >> > > - Softbounces: This email did not arrive. Try again later or >> contact through other means. If this keeps happening the address >> effectively does not work. >> > > >> > > What else is there to do? The sender might be curious to what the >> reason is (if the bounce is even telling you that) but it does not change >> the outcome or the call to action. The only things that are actionable for >> a sender, in a message, is when the content gets unacceptable (spam, too >> big). >> > > >> > > We have about 30 categories to explain what type of error occurred >> (available on request). >> > > >> > > Met vriendelijke groet, >> > > >> > > >> > > David Hofstee >> > > Deliverability Management >> > > MailPlus B.V. Netherlands (ESP) >> > > >> > > Van: mailop [mailto:mailop-boun...@mailop.org] Namens Brandon Long >> > > Verzonden: donderdag 10 december 2015 22:23 >> > > Aan: Dave Warren >> > > CC: mailop >> > > Onderwerp: Re: [mailop] Delivery to gmail via IPv6 >> > > >> > > I was just discussing this issue with our support folks, and we're >> looking at improving ours for this reason. We also see a remarkable number >> of our NDRs marked as spam, especially by those whose primary language >> isn't English. >> > > >> > > We'll definitely still include the technical information, but it'll >> be further down. >> > > >> > > And we'll do our best with handling the remote server's error >> messages, but ugh. It's too bad that the extended status codes are still >> so generic... though, as our tech writer and ux folks pointed out, really, >> all the user cares about is "did I do something wrong? Is there something >> else I can do?" >> > > >> > > Brandon >> > > >> > > On Thu, Dec 10, 2015 at 12:18 PM, Dave Warren <da...@hireahit.com> >> wrote: >> > > And unfortunately the friendlier they are, the less useful they >> usually are. >> > > >> > > The ugly ones are the only ones that are useful, but for whatever >> reason, it's beyond MTA developers to start with friendly messages with a >> "Troubleshooting information below" that contains a useful transcript. >> > > >> > > As a techie, I appreciate the info, but the reality is that unless >> you expect the sender to take some action, transient error messages aren't >> usually useful. We've scaled back the transient failures that we send, at >> most you get a single transient and single permanent error, and even still, >> I question the value of the transient error since the user can't actually >> do anything (and nor does forwarding it to support@ help). Of course, we >> also allow users to view the SMTP queue for all messages involving their >> account for those who care, so that might skew my viewpoint. >> > > >> > > I'm not a fan of the current trend of using permanent error codes in >> SMTP for what might well be transient errors (DNS problems, for example), >> but at the same time, as a sender, I don't see any value in retrying more >> than 24 hours. >> > > >> > > It's tough to balance user expectations though. >> > > >> > > -- >> > > Dave Warren >> > > http://www.hireahit.com/ >> > > http://ca.linkedin.com/in/davejwarren >> > > >> > > On 2015-12-10 10:43, Franck Martin wrote: >> > > It also has to do with people not understanding DSN. Seriously they >> are ugly and hard to find the relevant information in them... >> > > >> > > >> > > >> > > >> > > _______________________________________________ >> > > mailop mailing list >> > > mailop@mailop.org >> > > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> > > >> > > _______________________________________________ >> > > mailop mailing list >> > > mailop@mailop.org >> > > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> > >> > -- >> > Ian Eiloart >> > Postmaster, University of Sussex >> > +44 (0) 1273 87-3148 >> > >> > _______________________________________________ >> > mailop mailing list >> > mailop@mailop.org >> > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> > >> > >> > >> > -- >> > Love is strong yet delicate. >> > It can be broken. >> > To truly love is to understand this. >> > To be in love is to respect this. >> > ~ Stephen Packer ~ >> > >> > >> > _______________________________________________ >> > mailop mailing list >> > mailop@mailop.org >> > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> > >> > >> > >> > >> > -- >> > Love is strong yet delicate. >> > It can be broken. >> > To truly love is to understand this. >> > To be in love is to respect this. >> > ~ Stephen Packer ~ >> > >> > _______________________________________________ >> > mailop mailing list >> > mailop@mailop.org >> > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> >> -- >> Ian Eiloart >> Postmaster, University of Sussex >> +44 (0) 1273 87-3148 >> >> -- Love is strong yet delicate. It can be broken. To truly love is to understand this. To be in love is to respect this. ~ Stephen Packer ~
_______________________________________________ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
