Re: Direct download link detection
> You"re free to lose any incoming mail you like, including mine :-) > Though apparently you do get my messages, so I am confused about what > your filter actually does. Your mid is good enough... Message-Id: < 20170727190653.qmp6gnvfmdveh...@matica.foolinux.mooo.com > > unbound-host -rvD foolinux.mooo.com foolinux.mooo.com has address 136.25.152.91 (insecure) foolinux.mooo.com has no IPv6 address (insecure) foolinux.mooo.com has no mail handler record (insecure) > Original Message ---- > Subject: Re: Direct download link detection > Local Time: July 27, 2017 9:06 PM > UTC Time: July 27, 2017 7:06 PM > From: i...@very.loosely.org > To: users@spamassassin.apache.org > On 2017-07-27 13:08, Rupert Gallagher wrote: >> The rfc prescribes (MUST) the use of your public domain in the domain >> part of your mid. > If you mean RFC 5322, this is not true. Section 3.6.4: > The message identifier (msg-id) itself MUST be a globally unique > identifier for a message. The generator of the message identifier > MUST guarantee that the msg-id is unique. There are several > algorithms that can be used to accomplish this. Since the msg-id has > a similar syntax to addr-spec (identical except that quoted strings, > comments, and folding white space are not allowed), a good method is > to put the domain name (or a domain literal IP address) of the host > on which the message identifier was created on the right-hand side of > the "@" (since domain names and IP addresses are normally unique), > and put a combination of the current absolute date and time along > with some other currently unique (perhaps sequential) identifier > available on the system (for example, a process id number) on the > left-hand side. Though other algorithms will work, it is RECOMMENDED > that the right-hand side contain some domain identifier (either of > the host itself or otherwise) such that the generator of the message > identifier can guarantee the uniqueness of the left-hand side within > the scope of that domain. > Or do you mean some other RFC, which one? >> So the dns tests are just the first in the queue. The dimain must also >> match early in the Reveived list. > Huh? Even corrected for the obvious typos, this doesn"t make sense. > We"re talking about the Message-ID here. >> If you fail with it, then you have problems with every rfc-compliant >> smtp server world-wide. This filter is especially useful against >> scripts, spamming programs, and web-based mailers. > You"re free to lose any incoming mail you like, including mine :-) > Though apparently you do get my messages, so I am confused about what > your filter actually does. > -- > Please don"t Cc: me privately on mailing lists and Usenet, > if you also post the followup to the list or newsgroup. > Do obvious transformation on domain to reply privately _only_ on Usenet.
Re: Direct download link detection
msg-id = "<" addr-spec ">"; addr-spec = local-part "@" domain; domain = sub-domain *("." sub-domain); sub-domain = domain-ref / domain-literal; <> [RFC 822, pg. 30, section 6.2.3] Re: typing errors due to my fingers, on iPhone 4s's tiny buttons Sent from ProtonMail webmail. > ---- Original Message > Subject: Re: Direct download link detection > Local Time: July 27, 2017 9:06 PM > UTC Time: July 27, 2017 7:06 PM > From: i...@very.loosely.org > To: users@spamassassin.apache.org > On 2017-07-27 13:08, Rupert Gallagher wrote: >> The rfc prescribes (MUST) the use of your public domain in the domain >> part of your mid. > If you mean RFC 5322, this is not true. Section 3.6.4: > The message identifier (msg-id) itself MUST be a globally unique > identifier for a message. The generator of the message identifier > MUST guarantee that the msg-id is unique. There are several > algorithms that can be used to accomplish this. Since the msg-id has > a similar syntax to addr-spec (identical except that quoted strings, > comments, and folding white space are not allowed), a good method is > to put the domain name (or a domain literal IP address) of the host > on which the message identifier was created on the right-hand side of > the "@" (since domain names and IP addresses are normally unique), > and put a combination of the current absolute date and time along > with some other currently unique (perhaps sequential) identifier > available on the system (for example, a process id number) on the > left-hand side. Though other algorithms will work, it is RECOMMENDED > that the right-hand side contain some domain identifier (either of > the host itself or otherwise) such that the generator of the message > identifier can guarantee the uniqueness of the left-hand side within > the scope of that domain. > Or do you mean some other RFC, which one? >> So the dns tests are just the first in the queue. The dimain must also >> match early in the Reveived list. > Huh? Even corrected for the obvious typos, this doesn"t make sense. > We"re talking about the Message-ID here. >> If you fail with it, then you have problems with every rfc-compliant >> smtp server world-wide. This filter is especially useful against >> scripts, spamming programs, and web-based mailers. > You"re free to lose any incoming mail you like, including mine :-) > Though apparently you do get my messages, so I am confused about what > your filter actually does. > -- > Please don"t Cc: me privately on mailing lists and Usenet, > if you also post the followup to the list or newsgroup. > Do obvious transformation on domain to reply privately _only_ on Usenet.
Re: Direct download link detection
On 2017-07-27 13:08, Rupert Gallagher wrote: > The rfc prescribes (MUST) the use of your public domain in the domain > part of your mid. If you mean RFC 5322, this is not true. Section 3.6.4: The message identifier (msg-id) itself MUST be a globally unique identifier for a message. The generator of the message identifier MUST guarantee that the msg-id is unique. There are several algorithms that can be used to accomplish this. Since the msg-id has a similar syntax to addr-spec (identical except that quoted strings, comments, and folding white space are not allowed), a good method is to put the domain name (or a domain literal IP address) of the host on which the message identifier was created on the right-hand side of the "@" (since domain names and IP addresses are normally unique), and put a combination of the current absolute date and time along with some other currently unique (perhaps sequential) identifier available on the system (for example, a process id number) on the left-hand side. Though other algorithms will work, it is RECOMMENDED that the right-hand side contain some domain identifier (either of the host itself or otherwise) such that the generator of the message identifier can guarantee the uniqueness of the left-hand side within the scope of that domain. Or do you mean some other RFC, which one? > So the dns tests are just the first in the queue. The dimain must also > match early in the Reveived list. Huh? Even corrected for the obvious typos, this doesn't make sense. We're talking about the Message-ID here. > If you fail with it, then you have problems with every rfc-compliant > smtp server world-wide. This filter is especially useful against > scripts, spamming programs, and web-based mailers. You're free to lose any incoming mail you like, including mine :-) Though apparently you do get my messages, so I am confused about what your filter actually does. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. Do obvious transformation on domain to reply privately _only_ on Usenet.
Re: Direct download link detection
The rfc prescribes (MUST) the use of your public domain in the domain part of your mid. So the dns tests are just the first in the queue. The dimain must also match early in the Reveived list. If you fail with it, then you have problems with every rfc-compliant smtp server world-wide. This filter is especially useful against scripts, spamming programs, and web-based mailers. Sent from ProtonMail Mobile On Wed, Jul 26, 2017 at 6:07 PM, Ian Zimmerman wrote: > On 2017-07-26 02:48, Rupert Gallagher wrote: > When a mail arrives without > mid, either the sender did not use a real > SMTP server or tried to hide it. > We have a custom SA rule for it. We > also reject upfront any mid with a > syntax error, or whose domain does > not have a rdns (eg. > @localhost.localdomain or @test.com). I suspect you'll miss this message, > then. My Message-IDs intentionally identify the originating host, which makes > me more confident that they're unique. The originating host is behind two > layers of NAT and DHCP, and naturally doesn't have rDNS. I don't know how to > ensure uniqueness if I use the relaying SMTP server's domain, or the domain > of the perimeter of the NATted network, which can have rDNS (and does, via a > dyn-like update service), but which I do not own or control. -- Please don't > Cc: me privately on mailing lists and Usenet, if you also post the followup > to the list or newsgroup. Do obvious transformation on domain to reply > privately _only_ on Usenet.
Re: Direct download link detection
> Are you able to turn it off? I tried. No such option. :-( Sent from ProtonMail Mobile On Wed, Jul 26, 2017 at 6:05 PM, Matus UHLAR - fantomas wrote: > On 26.07.17 02:48, Rupert Gallagher wrote: >+1 to remove that clause from the > RFC. I don't see any reason... btw you'd need to change it to MUST NOT for > all to stop (which is unlikelly to happen). >When a mail arrives without mid, > either the sender did not use a real SMTP > server or tried to hide it. We > have a custom SA rule for it. We also > reject upfront any mid with a syntax > error, or whose domain does not have > a rdns (eg. @localhost.localdomain or > @test.com). nobody forbids you from doing this. SA has rules for both missing > and added Message-ID: header. you are welcome to reject based on anything... > >Sent from ProtonMail Mobile seems that protonmail mobile unwraps the lines, > which makes quoted text very hard to read. Are you able to turn it off? >On > Tue, Jul 25, 2017 at 5:20 PM, John Hardin wrote: > >> On Tue, 25 Jul 2017, > Rupert Gallagher wrote: > Before bothering with body spam, make sure the > header is clear. The > specific email should have been rejected upfront, > because the foreign > sender's message-id pretends to originate from the > recipient's smtp > server. That's potentially valid. If a MTA receives a > message that has no message ID it is valid to add one from the MTA's domain. > -- John Hardin KA7OHZ http://www.impsec.org/~jhardin/ jhar...@impsec.org > FALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 > F507 136C AF76 D822 E6E6 B873 2E79 > --- I > would buy a Mac today if I was not working at Microsoft. -- James Allchin, > Microsoft VP of Platforms > --- 10 > days until the 282nd anniversary of John Peter Zenger's acquittal -- Matus > UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish > NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu > chcem NEDOSTAVAT akukolvek reklamnu postu. Windows 2000: 640 MB ought to be > enough for anybody @impsec.org>
Re: Direct download link detection - new variant
Am 2017-07-26 17:22, schrieb Dianne Skoll: On Wed, 26 Jul 2017 17:15:43 +0200 Michael Storz wrote: [...] /boundary="-{4}=_NextPart_000_[0-9A-F]{4}_[0-9A-F]{8}\.[0-9A-F]{8}"/ You may get FPs. See for example https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk105578 I am guessing that boundary is generated by a library that's also used for legitimate purposes. The boundary is a standard Microsoft Outlook boundary. You can't score on the boundary alone. But if you score on the meta rule a FP is unlikely. Just try it. Regards, Michael
Re: Direct download link detection
On 2017-07-26 02:48, Rupert Gallagher wrote: > When a mail arrives without mid, either the sender did not use a real > SMTP server or tried to hide it. We have a custom SA rule for it. We > also reject upfront any mid with a syntax error, or whose domain does > not have a rdns (eg. @localhost.localdomain or @test.com). I suspect you'll miss this message, then. My Message-IDs intentionally identify the originating host, which makes me more confident that they're unique. The originating host is behind two layers of NAT and DHCP, and naturally doesn't have rDNS. I don't know how to ensure uniqueness if I use the relaying SMTP server's domain, or the domain of the perimeter of the NATted network, which can have rDNS (and does, via a dyn-like update service), but which I do not own or control. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. Do obvious transformation on domain to reply privately _only_ on Usenet.
Re: Direct download link detection
On 26.07.17 02:48, Rupert Gallagher wrote: +1 to remove that clause from the RFC. I don't see any reason... btw you'd need to change it to MUST NOT for all to stop (which is unlikelly to happen). When a mail arrives without mid, either the sender did not use a real SMTP server or tried to hide it. We have a custom SA rule for it. We also reject upfront any mid with a syntax error, or whose domain does not have a rdns (eg. @localhost.localdomain or @test.com). nobody forbids you from doing this. SA has rules for both missing and added Message-ID: header. you are welcome to reject based on anything... Sent from ProtonMail Mobile seems that protonmail mobile unwraps the lines, which makes quoted text very hard to read. Are you able to turn it off? On Tue, Jul 25, 2017 at 5:20 PM, John Hardin wrote: On Tue, 25 Jul 2017, Rupert Gallagher wrote: > Before bothering with body spam, make sure the header is clear. The > specific email should have been rejected upfront, because the foreign > sender's message-id pretends to originate from the recipient's smtp > server. That's potentially valid. If a MTA receives a message that has no message ID it is valid to add one from the MTA's domain. -- John Hardin KA7OHZ http://www.impsec.org/~jhardin/ jhar...@impsec.org FALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- I would buy a Mac today if I was not working at Microsoft. -- James Allchin, Microsoft VP of Platforms --- 10 days until the 282nd anniversary of John Peter Zenger's acquittal -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows 2000: 640 MB ought to be enough for anybody
Re: Direct download link detection - new variant
On Wed, 26 Jul 2017 08:28:52 -0700 (PDT) John Hardin wrote: > ...all of which is, sadly, whack-a-mole. However, there are few to no alternatives to whack-a-mole for this spam run. The messages are pretty bland. We've been diligently adding the URLs to our phishing list and we seem to have caught most of them; there are only a couple of hundred or so URLs. Regards, Dianne.
Re: Direct download link detection - new variant
On Wed, 26 Jul 2017, Michael Storz wrote: Am 2017-07-26 15:08, schrieb Dianne Skoll: On Tue, 25 Jul 2017 08:36:22 -0400 Dianne Skoll wrote: > All of the URLs match this pattern: > /\/[A-Z]{4}\d{6}\/$/ We see a new variant with the subject "Your Virgin Media bill is ready" and URLs that match: uri__RP_D_00108_03 /\/\d{12}\/[A-Z]{6}\/?$/ Nearly all of these spammails can be blocked with header __LRZ_BND_MSContent-Type =~ /boundary="-{4}=_NextPart_000_[0-9A-F]{4}_[0-9A-F]{8}\.[0-9A-F]{8}"/ header __LRZ_MSGID_SPAM_99 MESSAGEID =~ /<\d{8,13}\.2017\d{6,11}\@/ metaLRZ_HEADER_SPAM_99 (__LRZ_MSGID_SPAM_99 && __LRZ_BND_MS) The version before had a different boundary header__LRZ_BND_HU32 Content-Type =~ /boundary="[0-9A-F]{32}"/ ...all of which is, sadly, whack-a-mole. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- ...much of our country's counterterrorism security spending is not designed to protect us from the terrorists, but instead to protect our public officials from criticism when another attack occurs. -- Bruce Schneier --- 9 days until the 282nd anniversary of John Peter Zenger's acquittal
Re: Direct download link detection - new variant
On Wed, 26 Jul 2017 17:15:43 +0200 Michael Storz wrote: [...] > /boundary="-{4}=_NextPart_000_[0-9A-F]{4}_[0-9A-F]{8}\.[0-9A-F]{8}"/ You may get FPs. See for example https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk105578 I am guessing that boundary is generated by a library that's also used for legitimate purposes. Regards, Dianne.
Re: Direct download link detection - new variant
Am 2017-07-26 15:08, schrieb Dianne Skoll: On Tue, 25 Jul 2017 08:36:22 -0400 Dianne Skoll wrote: All of the URLs match this pattern: /\/[A-Z]{4}\d{6}\/$/ We see a new variant with the subject "Your Virgin Media bill is ready" and URLs that match: uri__RP_D_00108_03 /\/\d{12}\/[A-Z]{6}\/?$/ Regards, Dianne. Nearly all of these spammails can be blocked with header __LRZ_BND_MSContent-Type =~ /boundary="-{4}=_NextPart_000_[0-9A-F]{4}_[0-9A-F]{8}\.[0-9A-F]{8}"/ header __LRZ_MSGID_SPAM_99 MESSAGEID =~ /<\d{8,13}\.2017\d{6,11}\@/ metaLRZ_HEADER_SPAM_99 (__LRZ_MSGID_SPAM_99 && __LRZ_BND_MS) The version before had a different boundary header __LRZ_BND_HU32 Content-Type =~ /boundary="[0-9A-F]{32}"/ Regards, Michael
Re: Direct download link detection - new variant
On Tue, 25 Jul 2017 08:36:22 -0400 Dianne Skoll wrote: > All of the URLs match this pattern: > /\/[A-Z]{4}\d{6}\/$/ We see a new variant with the subject "Your Virgin Media bill is ready" and URLs that match: uri__RP_D_00108_03 /\/\d{12}\/[A-Z]{6}\/?$/ Regards, Dianne.
Re: Direct download link detection
+1 to remove that clause from the RFC. When a mail arrives without mid, either the sender did not use a real SMTP server or tried to hide it. We have a custom SA rule for it. We also reject upfront any mid with a syntax error, or whose domain does not have a rdns (eg. @localhost.localdomain or @test.com). Sent from ProtonMail Mobile On Tue, Jul 25, 2017 at 5:20 PM, John Hardin wrote: > On Tue, 25 Jul 2017, Rupert Gallagher wrote: > Before bothering with body > spam, make sure the header is clear. The > specific email should have been > rejected upfront, because the foreign > sender's message-id pretends to > originate from the recipient's smtp > server. That's potentially valid. If a > MTA receives a message that has no message ID it is valid to add one from the > MTA's domain. -- John Hardin KA7OHZ http://www.impsec.org/~jhardin/ > jhar...@impsec.org FALaholic #11174 pgpk -a jhar...@impsec.org key: > 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- I > would buy a Mac today if I was not working at Microsoft. -- James Allchin, > Microsoft VP of Platforms > --- 10 > days until the 282nd anniversary of John Peter Zenger's acquittal
Re: Direct download link detection
On Tue, 25 Jul 2017 10:28:41 -0500 (CDT) David B Funk wrote: > If the original message actually had that message-ID form when it > arrived at the OP's mail MX server, then your assessment makes sense. > > However it's entirely possible that message-ID was added by the OP's > mail server because the incoming message had no message-ID to begin > with. There's insufficient information in that pastbin example to > tell. The fact that it's located between the To and Subject headers suggests that it did predate delivery, but it may just be overzealous munging.
Re: Direct download link detection
On Tue, 25 Jul 2017, Rupert Gallagher wrote: Before bothering with body spam, make sure the header is clear. The specific email should have been rejected upfront, because the foreign sender's message-id pretends to originate from the recipient's smtp server. That's potentially valid. If a MTA receives a message that has no message ID it is valid to add one from the MTA's domain. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- I would buy a Mac today if I was not working at Microsoft. -- James Allchin, Microsoft VP of Platforms --- 10 days until the 282nd anniversary of John Peter Zenger's acquittal
Re: Direct download link detection
On Tue, 25 Jul 2017 13:15:33 +0100 RW wrote: > https://pastebin.com/p7EnFNf7 We've seen lots of those and collected a few dozen unique URLs for our URL blacklists. I added a swath of them to the APER project in this commit: https://sourceforge.net/p/aper/code/11830/ All of the URLs match this pattern: /\/[A-Z]{4}\d{6}\/$/ (The leading/trailing Perl slash delimiters are included, but not part of the pattern.) so I think a URL rule can catch these. Doing a HEAD on the URL gives Content-Type: application/msword. I think it's most likely safe to do HEAD requests, but definitely not GET as others have mentioned. Regards, Dianne.
Re: Direct download link detection
On Mon, 24 Jul 2017 18:00:33 -0400 Alex wrote: > This one's probably already on some blacklists, but I'm still > blocking others: > > https://pastebin.com/p7EnFNf7 It seems to be common for this kind virus spam to pass itself off as an invoice. You might try creating a rule that checks for this kind of language and meta it with KAM_LAZY_DOMAIN_SECURITY.
Re: Direct download link detection
On Mon, 24 Jul 2017 23:00:33 +0100, Alex wrote: Link to malicious file removed... Generally not a good idea to post direct links like that. What would be involved in following these links in SA to determine if they immediately download a file (other than a web page)? Testing links in mail is fraught with issues. Aside from privacy issues, which have already been mentioned, triggering multiple requests to verify remote contents could introduce significant server load. I just looked at one legitimate newsletter and counted 172 remote links. You can probably drop about a third as duplicates due to both plain text and html alternatives but still, that's potentially a lot of requests for both my server and theirs. URIBL lookups are light because they're using a single domain, but at ~120 link checks per message? Yikes. You'd want good caching of the results because otherwise if 10 users receive the same newsletter you're making 1,200 queries, if it's 100... Senders likely wouldn't be very happy with all that automated checking either. There are ways to mitigate the impact, but they will also reduce the effectiveness of any testing. Would that even be a reliable indicator? Not for me. I see a lot of legit traffic that has direct links to images, pdfs, zips, tarballs, even Word files. I don't see it having any real value unless you're then passing the downloaded files to a virus scanner. Also, there are enough web pages which you likely don't want users accessing anyway https://www.sophos.com/en-us/security-news-trends/security-trends/malicious-javascript.aspx
Re: Direct download link detection
On Mon, Jul 24, 2017, at 15:00, Alex wrote: > Hi, > > We're currently experiencing a new spam campaign that involves some > text pertaining to invoicing then a link that immediately downloads a > Word macro file. > > http://sdeflores.com/PHJC579907/ > > What would be involved in following these links in SA to determine if > they immediately download a file (other than a web page)? Would that > even be a reliable indicator? You want to be very careful with your implementation, many "Verify this account" or Subscribe/Unsubscribe links act on a single GET rather than following standards and using a POST, so this type of activity can trigger an action. It is possible that a HEAD would be safer and still provide the needed information, but i'm not clear if this will trigger any actions, but noting how terrible many implementations are, it wouldn't shock me if there are a few home-brewed beasts that take action on a HEAD request.
Re: Direct download link detection
On 07/24/2017 05:00 PM, Alex wrote: Hi, We're currently experiencing a new spam campaign that involves some text pertaining to invoicing then a link that immediately downloads a Word macro file. http://sdeflores.com/PHJC579907/ What would be involved in following these links in SA to determine if they immediately download a file (other than a web page)? Would that even be a reliable indicator? This isn't the first time I've seen such an approach. This one's probably already on some blacklists, but I'm still blocking others: https://pastebin.com/p7EnFNf7 Thanks, Alex Bump up your RCVD_IN_SENDERSCORE_60_69 to 2.2 like I have in my environment and this would have been blocked. Running it through my SA hit BAYES_50 with 0.8 points which also gave it a little higher overall score. Perhaps better spam training would be helpful. Consider setting up masscheck processing which could also help with ham/spam classification for Bayes training making it a higher priority like it did for me. I would also check my mail logs and put a REJECT in the MTA configs for ahf-group.de if all else fails. This is not common but sometimes I find problem domains that send from servers that never make it onto RBLs or DBLs. -- Dave
Re: Direct download link detection
Alex skrev den 2017-07-25 00:00: https://pastebin.com/p7EnFNf7 its more malware then spam https://virustotal.com/da/file/b5f30f3f12d8337750943f35a076e3c9690bd18505f7eb31101c98c72f454629/analysis/1500933955/
Direct download link detection
Hi, We're currently experiencing a new spam campaign that involves some text pertaining to invoicing then a link that immediately downloads a Word macro file. http://sdeflores.com/PHJC579907/ What would be involved in following these links in SA to determine if they immediately download a file (other than a web page)? Would that even be a reliable indicator? This isn't the first time I've seen such an approach. This one's probably already on some blacklists, but I'm still blocking others: https://pastebin.com/p7EnFNf7 Thanks, Alex