Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
On Tue, Oct 08, 2019 at 03:55:10PM +0200, Benoit Panizzon via mailop wrote: > Yet another one, reported a work report with full salary detail from > his employer, not aware that Microsoft would forward that sensitive data > to our abuse desk. This one, at least, smells like it might be a GDPR risk, at a bare minimum. - Matt ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
On Tue, Oct 08, 2019 at 12:01:20PM -0700, Luis E. Muñoz via mailop wrote: > On 8 Oct 2019, at 6:55, Benoit Panizzon via mailop wrote: > > 3: Try to make it more obvious in the documentation of that junk > > folder, that moving emails there will lead to a complaint to the > > senders ISP. > > I've always believed that "junk" is too subtle – although English is not my > first language. I suppose lawyers had a significant say in the selection of > this term. English is my first language, and I can confirm that "junk" is a terrible word to use to refer to UBE. I have a shed full of what is commonly referred to as "junk", and not only do I not want to get rid it, I would be saddened if I never got any more of it (although my wife would, on the whole, not share that emotion). - Matt ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
I've noticed when using Gmail that I can trash a message, or I can report it as spam and (sometimes) choose between "report" and "report and unsubscribe." But I don't think I can "trash and unsubscribe," and I don't think there's a way (other than scrolling through the message looking for a unsubscribe link) to just unsubscribe without reporting as spam. If that's correct, maybe it helps to explain why people "people still use report spam as 'I don't want to receive this any more'." On Tue, Oct 8, 2019 at 2:13 PM Brandon Long via mailop wrote: > At one point when I was complaining about the Gmail UI for that, I did a > survey, and several competing products used the exact opposite > iconography for spam/trask, > just leading to more user confusion... and us making sure to use something > very different for report spam. No idea if it helped or not for our > reports, people still use report spam as "I don't want to receive this any > more". > > Brandon > > On Tue, Oct 8, 2019 at 12:03 PM Luis E. Muñoz via mailop < > mailop@mailop.org> wrote: > >> >> >> On 8 Oct 2019, at 6:55, Benoit Panizzon via mailop wrote: >> >> > 3: Try to make it more obvious in the documentation of that junk >> > folder, that moving emails there will lead to a complaint to the >> > senders ISP. >> >> I've always believed that "junk" is too subtle – although English is >> not my first language. I suppose lawyers had a significant say in the >> selection of this term. >> >> I've run into people who are obviously confused between "junk" and >> "trash" folders. >> >> Best regards >> >> -lem >> >> ___ >> mailop mailing list >> mailop@mailop.org >> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop >> > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > -- === Russell Clemings === ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
On 10/8/19 3:05 PM, Brandon Long via mailop wrote: … people still use report spam as "I don't want to receive this any more". I'd like to see MUAs get smart enough to question what needs to be done when people indicate "I don't want to receive this any more". If the message passes all contemporary hygiene tests and has the RFC 8058 list-unsubscribe headers, consider asking "Do you want to delete this message? / Unsubscribe from this (legitimate) mailing list? / Report this message as spam?". -- Grant. . . . unix || die smime.p7s Description: S/MIME Cryptographic Signature ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
On Tue, Oct 8, 2019 at 1:51 PM Bill Cole via mailop wrote: > Not exactly garbage: if it exists, it needs a '@' and the legal content > is slightly less permissive than the 'addr-spec' definition (i.e. email > addresses.) Also, it must be unique, so using a real fully qualified > hostname (i.e. one that no one else will use) is useful. OR: use none, > and have your gateway generate one. > > https://tools.ietf.org/html/rfc5322#section-3.6. > Yeah--I should have been clearer when I said 'garbage'. I was pretty sure vaguely-readable line-noise separated by an @ qualifies. :) -A ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
> Not exactly garbage: if it exists, it needs a '@' and the legal content is > slightly less permissive than the 'addr-spec' definition (i.e. email > addresses.) some sources (aliexpress!) generate message-id lacking '@' (also '<' and '>') so removed that as a hard-requirement when filtering here ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
At one point when I was complaining about the Gmail UI for that, I did a survey, and several competing products used the exact opposite iconography for spam/trask, just leading to more user confusion... and us making sure to use something very different for report spam. No idea if it helped or not for our reports, people still use report spam as "I don't want to receive this any more". Brandon On Tue, Oct 8, 2019 at 12:03 PM Luis E. Muñoz via mailop wrote: > > > On 8 Oct 2019, at 6:55, Benoit Panizzon via mailop wrote: > > > 3: Try to make it more obvious in the documentation of that junk > > folder, that moving emails there will lead to a complaint to the > > senders ISP. > > I've always believed that "junk" is too subtle – although English is > not my first language. I suppose lawyers had a significant say in the > selection of this term. > > I've run into people who are obviously confused between "junk" and > "trash" folders. > > Best regards > > -lem > > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
I looked through the logs. No duplicate message IDs over a 7-day period. -A On Tue, Oct 8, 2019 at 1:33 PM Chris Wedgwood wrote: > > The second tech arrived at the conclusion that it was the Message-Id > > header. Messages that were delivered had an externally-resolvable domain > > as part of the Message-Id header. Messages that were 'disappeared' had > our > > internal domain (i.e. whatever.local) as part of the Message-Id. > > > > I renamed one of the copiers to use our external domain and suddenly it > > worked perfectly. > > is there any chance you are reusing message-ids (bad randomness?) > > iirc gmail deduplicates per-user per-message-id (some people rely on > this) > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
On 8 Oct 2019, at 15:57, Aaron C. de Bruyn via mailop wrote: [...] Maybe I'm getting old and forgetful, but I seem to recall the RFCs saying the Message-Id was nothing more than a tracking identifier and could be complete garbage. Not exactly garbage: if it exists, it needs a '@' and the legal content is slightly less permissive than the 'addr-spec' definition (i.e. email addresses.) Also, it must be unique, so using a real fully qualified hostname (i.e. one that no one else will use) is useful. OR: use none, and have your gateway generate one. https://tools.ietf.org/html/rfc5322#section-3.6. Can someone at Google shed some light on this? Note that I am NOT Google. -- Bill Cole b...@scconsult.com or billc...@apache.org (AKA @grumpybozo and many *@billmail.scconsult.com addresses) ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
So, nothing disappears, and the logs should definitely have something... though, it might not reach the GSuite logs if the SMTP conversation doesn't reach a point where we can identify the relevant customer (ie, RCPT TO). Also, the only message we give with 451 4.5.0 doesn't start with OK, its "SMTP protocol violation, see RFC 2821". We only return this when there's an SMTP protocol violation, ie sending commands when you're not allowed. Mostly this is because the sender is trying to use PIPELINING, but it's disabled. The only message that is "OK - gsmtp" is our 250 OK response when we accept a message. If you can send me the details and the case id #, I can take a look. As for message-ids, they do have a format that's required, and we do replace them when they're not properly formatted, and that replacement can be a signal to the spam engine, as can the domain of the message-id. Everything about the message is fair game for the spam pipeline. Brandon On Tue, Oct 8, 2019 at 12:59 PM Aaron C. de Bruyn via mailop < mailop@mailop.org> wrote: > We have a bunch of satellite offices that relay through our corporate mail > gateway. The mail gateway handles things like scan-to-email from copiers, > email from our internal fax gateway, etc... > > Starting Monday morning-ish Google has been one of a few things apparently > at random: > * Accepting the message from our mail gateway and delivering it to Inbox > or Spam > * Accepting the message from our mail gateway and disappearing it (nothing > in GSuite logs either) > * Rejecting the message with "451 4.5.0 OK - gsmtp" > > After doing a bunch of testing and finding nothing wrong, I chatted with > GSuite. > > The first tech kept insisting I sign in to gmail and get him mail headers > from the messages that were magically disappearing. I finally convinced > him to escalate me. > > The second tech arrived at the conclusion that it was the Message-Id > header. Messages that were delivered had an externally-resolvable domain > as part of the Message-Id header. Messages that were 'disappeared' had our > internal domain (i.e. whatever.local) as part of the Message-Id. > > I renamed one of the copiers to use our external domain and suddenly it > worked perfectly. > > The moment it was fixed, the tech said the problem was solved and closed > the case. > > Maybe I'm getting old and forgetful, but I seem to recall the RFCs saying > the Message-Id was nothing more than a tracking identifier and could be > complete garbage. > > Can someone at Google shed some light on this? I don't think the correct > fix for this is to spend days updating copiers and fax gateways at tens of > locations...and I'm suspicious that the Message-Id domain is the root cause. > > -A > ___ > mailop mailing list > mailop@mailop.org > https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop > ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Google and the Message-Id Header
> The second tech arrived at the conclusion that it was the Message-Id > header. Messages that were delivered had an externally-resolvable domain > as part of the Message-Id header. Messages that were 'disappeared' had our > internal domain (i.e. whatever.local) as part of the Message-Id. > > I renamed one of the copiers to use our external domain and suddenly it > worked perfectly. is there any chance you are reusing message-ids (bad randomness?) iirc gmail deduplicates per-user per-message-id (some people rely on this) ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
[mailop] Google and the Message-Id Header
We have a bunch of satellite offices that relay through our corporate mail gateway. The mail gateway handles things like scan-to-email from copiers, email from our internal fax gateway, etc... Starting Monday morning-ish Google has been one of a few things apparently at random: * Accepting the message from our mail gateway and delivering it to Inbox or Spam * Accepting the message from our mail gateway and disappearing it (nothing in GSuite logs either) * Rejecting the message with "451 4.5.0 OK - gsmtp" After doing a bunch of testing and finding nothing wrong, I chatted with GSuite. The first tech kept insisting I sign in to gmail and get him mail headers from the messages that were magically disappearing. I finally convinced him to escalate me. The second tech arrived at the conclusion that it was the Message-Id header. Messages that were delivered had an externally-resolvable domain as part of the Message-Id header. Messages that were 'disappeared' had our internal domain (i.e. whatever.local) as part of the Message-Id. I renamed one of the copiers to use our external domain and suddenly it worked perfectly. The moment it was fixed, the tech said the problem was solved and closed the case. Maybe I'm getting old and forgetful, but I seem to recall the RFCs saying the Message-Id was nothing more than a tracking identifier and could be complete garbage. Can someone at Google shed some light on this? I don't think the correct fix for this is to spend days updating copiers and fax gateways at tens of locations...and I'm suspicious that the Message-Id domain is the root cause. -A ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
On 8 Oct 2019, at 6:55, Benoit Panizzon via mailop wrote: 3: Try to make it more obvious in the documentation of that junk folder, that moving emails there will lead to a complaint to the senders ISP. I've always believed that "junk" is too subtle – although English is not my first language. I suppose lawyers had a significant say in the selection of this term. I've run into people who are obviously confused between "junk" and "trash" folders. Best regards -lem ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Gmail marking email from me as spam
On Tue, Oct 8, 2019 at 12:51 AM Alessandro Vesely via mailop < mailop@mailop.org> wrote: > On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote: > > Also, it's hard to optimize for the servers that send us one message a > day. > > > If it sends a message a day, it cannot be spam (by the B in UBE). > We use the overall spam label for things that often aren't that bulk, including 419 scams and spearphishing, for example. There are specific rules for those, but the overall system still protects against them. Brandon > > > > I've argued before that we should have better handling for the smallest > > servers (whitelist the first 5 messages/day for low volume IPs, for > > example), > > > Yes, please :-) > > There don't seem to be big advantages for Google in killing small > servers. It > might more convenient to support them, for the sake of the mail ecosystem. > > > > but the total volume compared to the effort against the major spam > > campaigns, it's hard to get that high enough on the priority list. We > did > > make some changes for that for smtp time blocking, but it doesn't move > any > > of our numbers because the number of messages affected is tiny... and > when > > you're talking about IPv6, even small numbers like that can result in > large > > enough holes for spam campaigns. > > > IPv4 allows to store a single DB record per IP address, even on a PC disk. > As I stated, we have the data, it's just not useful data. And a transactional global quota system for every address is more expensive than a single disk. Given we've had unknown or low volume hosts ramp to millions of message attempts in under 30s, none of this stuff ends up being that trivial. Anyways, we have the system, but there are resource limits even for us. Brandon ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Gmail marking email from me as spam
On Tue, 08 Oct 2019 10:16:48 -0400, Bill Cole via mailop wrote: >"Bulk" isn't about the delivery path, it is about how mail is composed >and targeted. And, properly understood, the 'B' in "UBE" is not "bulk", it is "broadcast". So, if more than one person received a substantially identical email, unsolicited, it is spam. We figured that out in the Early Dialup Cretaceous Era (1996), when somebody would claim that "bulk" was at least 50,000 and they had only sent 30,000, so we couldn't terminate their account. mdr -- "There are no laws here, only agreements." -- Masahiko ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Gmail marking email from me as spam
On 8 Oct 2019, at 3:48, Alessandro Vesely via mailop wrote: On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote: Also, it's hard to optimize for the servers that send us one message a day. If it sends a message a day, it cannot be spam (by the B in UBE). This isn't true. Are you really unfamiliar with the "snowshoe" tactic used by spammers? It has been common for well over a decade. Originally, showshoers would get disjointed /24 blocks from one provider, then they graduated to /27s spread across multiple providers, now they stand up a few VMs in multiple zones of multiple self-serve 'cloud' providers. They spread the distribution work across many source IPs so that none seems to be sending "bulk" mail. "Bulk" isn't about the delivery path, it is about how mail is composed and targeted. I've argued before that we should have better handling for the smallest servers (whitelist the first 5 messages/day for low volume IPs, for example), Yes, please :-) There don't seem to be big advantages for Google in killing small servers. Google is a mostly automated advertising company supported a massive IT operation. Every small server represents eyeballs they can't readily capture. ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
Hi Chris I have exactly the same issue. I have found a hotmail user who made rule to 'save' all emails from a whole list of 'known friends' sender to the 'junk' folder. Causing an immediate Spam Complaint from Microsoft every time one of our customers sends that hotmail user an email. The hotmail user does not understand what she did wrong. I had another hotmail user deleting multiple year old emails by moving them all to the junk folder. Causing us to receive several complaint sent by one customer that recipient was in contact with one year and more ago. Yet another one, reported a work report with full salary detail from his employer, not aware that Microsoft would forward that sensitive data to our abuse desk. And also a nuisance are real spam emails which our users (possibly after disabling spam filtering on their account with us) forward to their hotmail account and then report as spam. I have notified the Microsoft Abuse Desk about those often reoccurring 'false' complaints. I have notified the sender of those complaints. No reaction. The solution would be easy: 1: Define a cut-off date. Don't send reports if a user moves an email older than say 14 days to the spam folder. 2: If a user moves an email to the spam folder, throw an POP-UP or something similar to him and make him confirm he does want to report that emails as spam. 3: Try to make it more obvious in the documentation of that junk folder, that moving emails there will lead to a complaint to the senders ISP. 4: Better recognition of forwarded emails: If the origin IP Address of the reported spam is in the same range as the MX for the original recipient in the To: Header and the Received: header, and possibly SRS signed From: header also all hint to a forwarding situation, please consider the Received: of the forwarder to be trusted and report it to the ISP of the Received: before that header. So if anyone on that list has a better connection to Microsoft and could hint them to those issues, that would be great. I don't suggest Microsoft should stop reporting spam. It is a great help. I would say about half of the reports we get are legit and help us block phished accounts in a timely manner. But some attempts could be done to lower the ratio of false positives. Mit freundlichen Grüssen -Benoît Panizzon- -- I m p r o W a r e A G-Leiter Commerce Kunden __ Zurlindenstrasse 29 Tel +41 61 826 93 00 CH-4133 PrattelnFax +41 61 826 93 01 Schweiz Web http://www.imp.ch __ ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
[mailop] Erroneous Hotmail spam/junk JMR email due to recipient error, where's the operator feedback loop?
Today I had to deal with an erroneous Hotmail JMR notification via my SNDS registration - the email was an order confirmation to a customer, it appears they use the Junk/Spam button as a Delete button. Once I receive this email, if I can demonstrate that it wasn't actually a complaint but was reported in error by the recipient, is there any 'right of reply' mechanism to establish that the complained-about email was actually legitimate? There's nothing on the Hotmail JMR email to indicate a means for the operator to respond or contest the complaint, and the SNDS portal shows no statistics for the sending server (insufficient email volume to display stats). I'm concerned junk complaints like this will bias mail filtering algorithms into considering legitimate email as junk in future. While the feedback loop to the operator worked well in this instance, it fails when the operator needs to contact someone to indicate the complaint is not valid. Chris ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
[mailop] Microsoft delivery failures - 5.4.4 (unable to route: no mail hosts for domain)
Hi all, On the 5th of October we noticed a spike in delivery failures at domains hotmail.com, outlook.com, msn.com, and live.com. The failures were caused by DNS lookup errors reported by the MTA as "5.4.4 (unable to route: no mail hosts for domain)”. This is problematic as the error is reported as hard bounce causing many addresses to be suppressed. Delivery to other Microsoft domains was fine. The issue also occurred between 20-24 September and now again on the 5th of October. Since then, delivery has been normal. We noticed that Microsoft is using DNS entries for MX hosts with a very short TTL (30 sec) and rotating IP addresses. We think the short TTL may be related to the issue. Aggressive caching of the MX-es seemed to help. Did anyone else experience this issue, or knows about recent DNS changes at Microsoft? Thanks, Maarten ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Gmail marking email from me as spam
Dnia 8.10.2019 o godz. 13:42:32 Matt Palmer via mailop pisze: > > The other commonality is that AWS EC2 is at least as much of a pit of spam > and abuse as OVH is, and I'm not surprised that you don't get treated better > by GMail when you start sending them mail from a rando EC2 address. As I recall and reconsider some facts, I start to be more and more convinced that in my case it's an issue of a domain reputation, not IP reputation. First: some time ago I had an issue with comments on various websites protected by Akismet. My comments just didn't get through, no matter what I wrote. Looks like it was enough to have my e-mail address in the "author" field for the comment to be filtered. It had nothing to do with my server IP as I was submitting the comments from either my home PC or work PC, sometimes from a laptop on a mobile connection - all these are completely different IP ranges and have nothing to do with OVH. I tried to complain to website owners, with little effect, so I contacted Akismet support directly. They sent me a link to a test comment form on their website, which I had to fill. After this, they told me that there was indeed a configuration error on their side, they fixed it and I should no more have problems with my comments. In fact, it worked and I don't have such issues anymore. Second: earlier this year, web filters at the company where I work started blocking HTTP access to my domain - not the IP address, but particularly the domain. I was able to connect just fine when I typed the IP address into the web browser, but if only "rafa.eu.org" appeared in the URL, the access was instantly blocked. Other domains that are hosted on my server, under the same IP address (there are two of them) were working fine. I asked the admins at my company to fix it, it took some time (such things are very slow in big corporations), but they finally did. Third: yesterday, as I was checking my domain with Talos Intelligence website, I noticed that it is categorized as "Parked Domain". That could explain a lot of things - treating e-mail coming from an apparently parked domain as suspicious is quite understandable. I absolutely don't know where did that incorrect classification come from - my website was at this address since I registered the domain, it was never "parked". I submitted a ticket on their website to fix that classification. That's why I suppose that it's just the presence of "rafa.eu.org" anywhere in the e-mail headers that triggers Google's spam filter - @Brandon, could you please look into it? I will send you the sample messages soon. Looks like there is some incorrect information about my domain circulating on the Internet and hitting various services and providers at various times. I would be very happy to trace the origin of that information and have it fixed at the source, but I don't know how to do this... :( -- Regards, Jaroslaw Rafa r...@rafa.eu.org -- "In a million years, when kids go to school, they're gonna know: once there was a Hushpuppy, and she lived with her daddy in the Bathtub." ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
Re: [mailop] Gmail marking email from me as spam
On Mon 07/Oct/2019 23:38:23 +0200 Brandon Long via mailop wrote: > Also, it's hard to optimize for the servers that send us one message a day. If it sends a message a day, it cannot be spam (by the B in UBE). > I've argued before that we should have better handling for the smallest > servers (whitelist the first 5 messages/day for low volume IPs, for > example), Yes, please :-) There don't seem to be big advantages for Google in killing small servers. It might more convenient to support them, for the sake of the mail ecosystem. > but the total volume compared to the effort against the major spam > campaigns, it's hard to get that high enough on the priority list. We did > make some changes for that for smtp time blocking, but it doesn't move any > of our numbers because the number of messages affected is tiny... and when > you're talking about IPv6, even small numbers like that can result in large > enough holes for spam campaigns. IPv4 allows to store a single DB record per IP address, even on a PC disk. And then, blocks like OVH's, formally assigned to different hosts, are not uniformly available to the same 0wner. Best Ale -- ___ mailop mailing list mailop@mailop.org https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
