Messages to the list are being duplicated again. Some of the messages are from Gmail, but not all (e.g. some are from kostyrka.org and cnlohr.com). I can try to help debug the problem from the Gmail end if an admin on the list machine can send me verbose SMTP logs; please see below for details. Harald or Sean, can one of you send detailed logs? Anyone else?
Some Gmail message IDs with duplicated messages today: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] Thanks, Marco ----- Forwarded message from Marco Barreno <[EMAIL PROTECTED]> ----- From: Marco Barreno <[EMAIL PROTECTED]> Date: Fri, 13 Jul 2007 14:36:57 -0700 To: "Wolfgang S. Rupprecht" <[EMAIL PROTECTED]> Cc: community@lists.openmoko.org Subject: Re: gmail.com problems and this list Hi everyone, Short version: It seems the duplicated messages are not just a Gmail problem: mails are also being duplicated from starband.net and possibly others as well, so it seems the problem is likely something on the openmoko side that only happens to cause duplicates with some sending domains and not others. I'ts also not consistent; some messages from Gmail and starband.net are duplicated while others aren't. I've checked with the Gmail team, and the duplicated messages from Gmail are happening because Gmail doesn't receive the SMTP OK from the openmoko list server, so it retransmits because it thinks the messages don't get delivered. To debug further, an admin on the list machine needs to enable verbose SMTP logging if it's not already enabled and provide information about if/when the OK is sent. Here are some example message IDs for which the logs would be useful: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] If verbose logging isn't already enabled, however, enabling it and then sending logs for any future duplicated messages would work too. Detailed version: On Mon, Jul 09, 2007 at 11:14:03PM -0700, thus spake Wolfgang S. Rupprecht: > At least one of the opemoko machines at 88.198.124.203 opens an smtp > connection back to the sending domain to verify the sender address of > any incoming message. The smtp-back machine has messed up DNS. The > claimed rDNS for that IP is "openmoko.org" but the forward DNS check > for that "openmoko.org" doesn't list 88.198.124.203 as a valid > address. If the sending machine is checking for spammers claiming a > bunk DNS name will reject 88.198.124.203's SMTP verify. The opemmoko > machine will then interpret that failed smtp verify attempt as the > verified address not existing and will decline the initial incoming > transfer. Even ignoring the messed-up DNS for the moment, doing an SMTP callback is not a good way to check existence of an email account. For one thing, whenever a spammer spoofs a From address in an innocent domain, that domain can get bombarded with callbacks. For another thing, spammers sometimes use RCPT commands themselves to check existence of email addresses, so ISPs are likely to consider many such requests to be abusive behavior. In the case of a large ISP (such as Gmail), if a lot of email is sent to a destination that does callbacks, the frequency of callbacks can trigger abuse prevention measures. See http://en.wikipedia.org/wiki/Callback_verification for more drawbacks. (In particular, note: "If a server receives a lot of spam, it will do a lot of callbacks and if those addresses are invalid, the server will look very similar to a spammer who is doing a dictionary attack to harvest addresses. This in turn might get the server blacklisted elsewhere.") For these reasons, Gmail strongly recommends avoiding callbacks and instead checking SPF records (Sender Policy Framework, provided via DNS) to verify that the mail is indeed coming from Google (i.e. the SMTP connection is coming from an IP authorized to send email on behalf of Google). However, in this case it does not seem that the problem was caused by the callbacks. The openmoko server accepted the mail (which it wouldn't do if the callback failed to verify the sender) and the DATA command was sent, but Gmail never received the next SMTP OK acknowledgment before timing out. So one of four things was happening: 1) openmoko never sent the OK -- problem on openmoko end 2) openmoko waited too long before sending the OK and SMTP timed out -- seems to indicate problem on the openmoko end 3) the OK was lost in transit -- network difficulties, neither side at fault 4) the OK was received by Gmail -- problem at the Gmail end Since at least one other domain is showing the same behavior, it seems 1 or 2 is most likely, but we can't say for sure without more information. One interesting fact is that the openmoko server started sending mail back to Gmail recipients (after list expansion) before Gmail received the SMTP OK. Perhaps this means the list server waits until after list expansion to send the OK? That could cause it to time out pretty easily. It would help diagnosis if someone on the list machine could send the verbose SMTP logs (if they were recorded). The logs posted by Harald Welte don't give enough information: what we really need to see is whether (and if so, when) the SMTP OK was sent for any of the failed/duplicated messages. Here are three example message IDs: [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] If the logging needs to have more verbosity enabled, however, the logs for any duplicated message after such enabling would work just as well. By the way, the Received header chains posted by David Ford don't necessarily show that the problem is within Gmail; there could be any number of legitimate reasons why the resending isn't originating at the last hop out of the Google network. Yes, Gmail is sending the mail multiple times, but that's because it thinks the openmoko server did not receive the message on earlier attempts. I hope someone with admin access to the list machine can help debug this problem so we can get it fixed, wherever the bug is located. Thanks, Marco _______________________________________________ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community ----- End forwarded message ----- _______________________________________________ OpenMoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community