While every attempt is made to deliver mail as quickly as possible, any sufficiently large distributed system will occasionally have issues.
We monitor the delivery latency within our systems, and act when they go bad. At times, there have been deliberate choices made to the design which allowed for extended queuing when we had datacenter outages, instead of insuring writes hit multiple datacenters... for example. It's also possible for individual mailboxes to be overloaded, due to the store & forward nature of email, it's not always possible to push back at smtp time, and so internal queuing occurs. Historically, it was also possible for individual mailboxes to have issues that could require fixing... there's still some issues due to locking of the entire mailbox under some scenarios, though it seems unlikely to result in an 8h delay. Every once in a while there's also a message-of-death or other issue, where something about an individual message makes our system unhappy or causes extensive cpu usage. And that's without adding in additional complexities that some companies have for their email routing, which can add a very large number of hops or even administrative quarantine. All of this would be visible in the Received headers of the message... though only the successful attempts, not the failed ones that resulted in retries, we considered at one point how we could add that information as well, but that never went anywhere. For workspace customers, there is the email log search which allows you to see where a message is and retry attempts. At one point I started a proposal for something simpler for individual mailboxes, but it got complicated quickly and didn't make the prioritization cutoff. Folks definitely tend to think of email communications now being as fast as instant messaging, and they dislike when there are delays and no indication of such... the fact that we'll retry a message internally for up to 3-5d is completely out of touch with what user expectations are... but bounces are also terrible. Outside of making sure that a high percentile of messages are delivered in reasonable times, I'm not sure anyone has come up with a better solution. 8h would be a fairly large statistical outlier, that's for sure. Brandon On Wed, Apr 20, 2022 at 3:57 AM Cyril - ImprovMX via mailop < mailop@mailop.org> wrote: > Hi everyone ! > > I'm reaching out in the hope to find someone in the same situation, and > that I can receive some input on why this happened. > > We forward a lot of emails including to Gmail, and a user reached out to > us because their email arrived in their inbox with a delay of 8 hours! > > That user was able to send us the .eml, so I was able to peak at the > headers. What I noticed is that the Date present in the email was in UTC > all the way up to us, but when it left our server and landed at Gmail, it > changed, but ... oddly. > > The "Date" header was "Thu, 14 Apr 2022 18:29:43 +0000" and during the > processing in the sender, and in our servers, it remained the same with a > few seconds changing, but that's all. > > We forwarded it to "gmail-smtp-in.l.google.com", and the next date > present (in the "Received" header) became "Thu, 14 Apr 2022 19:24:28 -0700 > (PDT)" > > I don't mind the timezone changing, but the hour only changed for one > hour, while the timezone changed by an offset of 7 hours! The two changes > resulted in a difference of 8 hours, which is exactly when the user saw his > email in his inbox. > > What happened here ? > > Our logs clearly show that we forwarded the email in under 2 seconds and > we didn't mess with the dates (or anything, for that matter). > > Does anyone here experience this before? Do you have any explanations > about what happened? > > I'd love to reach out to Gmail to ask them about this, but, well, it's > Gmail. > > Thank you for your help! > > Best, > Cyril > _______________________________________________ > mailop mailing list > mailop@mailop.org > https://list.mailop.org/listinfo/mailop >
_______________________________________________ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
