Re: SpamSender with 2 @-signs in the address
Am 03.12.2018 um 17:56 schrieb Andreas Galatis : since several weeks I keep getting mails with sender-addresses like „Harald Wieruch - Top Ten GmbH h.wieruch@top10ten.comxandra.hennem...@metco-gmbh.de <mailto:h.wieruch@top10ten.comxandra.hennem...@metco-gmbh.de>“ The first part „Harald Wieruch – Top Ten GmbH h.wier...@top10ten.com <mailto:h.wier...@top10ten.com>“ stays the same, everything behind this address changes. On 12.12.18 14:01, Matthias Leisi wrote: Could it be stolen credentials from a client machine? To access a shared mailbox on an Exchange server, the login name needs be specified as „user\shared“ - and if both use SMTP-formatted addresses, this would look like „u...@example.com\sharedmail...@example.com“. I don't think so. Just today I've seen header likce From: "name surname " -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Spam = (S)tupid (P)eople's (A)dvertising (M)ethod
Re: openssl 1.1.1 , FreeBSd 11.2 and spamassassin-3.4.2_2
On 30.11.18 14:57, The Doctor wrote: Nov 30 14:53:15.964 [74107] dbg: channel: selected mirror http://sa-update.spamassassin.org Nov 30 14:53:15.964 [74107] dbg: http: url: http://sa-update.spamassassin.org/1847701.tar.gz Nov 30 14:53:15.964 [74107] dbg: http: downloading to: /var/db/spamassassin/3.004002/updates_spamassassin_org/1847701.tar.gz, update Nov 30 14:53:15.964 [74107] dbg: util: executable for curl was found at /usr/local/bin/curl Nov 30 14:53:15.965 [74107] dbg: http: /usr/local/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1847701.tar.gz -z 1847701.tar.gz -- http://sa-update.spamassassin.org/1847701.tar.gz Nov 30 14:53:18.418 [74107] dbg: http: process [74232], exit status: exit 0 Nov 30 14:53:18.420 [74107] dbg: http: url: http://sa-update.spamassassin.org/1847701.tar.gz.sha512 Nov 30 14:53:18.420 [74107] dbg: http: downloading to: /var/db/spamassassin/3.004002/updates_spamassassin_org/1847701.tar.gz.sha512, update Nov 30 14:53:18.421 [74107] dbg: util: executable for curl was found at /usr/local/bin/curl Nov 30 14:53:18.421 [74107] dbg: http: /usr/local/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1847701.tar.gz.sha512 -z 1847701.tar.gz.sha512 -- http://sa-update.spamassassin.org/1847701.tar.gz.sha512 Nov 30 14:53:20.259 [74107] dbg: http: process [74286], exit status: exit 0 Nov 30 14:53:20.260 [74107] dbg: http: url: http://sa-update.spamassassin.org/1847701.tar.gz.sha256 Nov 30 14:53:20.260 [74107] dbg: http: downloading to: /var/db/spamassassin/3.004002/updates_spamassassin_org/1847701.tar.gz.sha256, update Nov 30 14:53:20.260 [74107] dbg: util: executable for curl was found at /usr/local/bin/curl Nov 30 14:53:20.260 [74107] dbg: http: /usr/local/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1847701.tar.gz.sha256 -z 1847701.tar.gz.sha256 -- http://sa-update.spamassassin.org/1847701.tar.gz.sha256 Nov 30 14:53:22.161 [74107] dbg: http: process [74329], exit status: exit 0 Nov 30 14:53:22.162 [74107] dbg: http: url: http://sa-update.spamassassin.org/1847701.tar.gz.asc Nov 30 14:53:22.162 [74107] dbg: http: downloading to: /var/db/spamassassin/3.004002/updates_spamassassin_org/1847701.tar.gz.asc, update Nov 30 14:53:22.163 [74107] dbg: util: executable for curl was found at /usr/local/bin/curl Nov 30 14:53:22.163 [74107] dbg: http: /usr/local/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1847701.tar.gz.asc -z 1847701.tar.gz.asc -- http://sa-update.spamassassin.org/1847701.tar.gz.asc Nov 30 14:53:23.603 [74107] dbg: http: process [74380], exit status: exit 0 Nov 30 14:53:23.607 [74107] dbg: sha512: verification wanted: ae6c6249e8a63d4512331ec91e42bf0ba6ead2f8ba323200ebbfe4ed44bf9902635c7ecc7a3b392bdaddc96f070f8fd0293475dace317923854a32ba5238d93d Nov 30 14:53:23.607 [74107] dbg: sha512: verification result: 88fd9fa22e55c00365b8d0548a7ce8fc8c5ac08c339ca383663b5b735337b2ef2a52a83021b6608f186b4163556a8b8d9ecef14c775717294607925577a0dd9f channel: SHA512 verification failed, channel failed Nov 30 14:53:23.608 [74107] dbg: generic: cleaning up temporary directory/files Nov 30 14:53:23.608 [74107] dbg: generic: cleaning directory /tmp/.spamassassin74107u75Bvytmp Nov 30 14:53:23.608 [74107] dbg: diag: updates complete, exiting with code 4 sa-update channel: SHA512 verification failed, channel failed I have notices this problem repeatedly, usually findint out that the downloaded file was one byte long. Can you check the size of /tmp/1847701.tar.gz when that happens? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Microsoft dick is soft to do no harm
Re: spoofing mail
El vie., 30 nov. 2018 a las 3:06, Matus UHLAR - fantomas () escribió: And, yes, there could be rule that catches message-id added by internal server. Note that: - Message-ID is not required (has SHOULD in RFC) - many mailservers add message-id if it doesn't exist. >> https://pastebin.com/ktMUDLps not available anymore :-( On 30.11.18 10:55, Rick Gutierrez wrote: Hi , here it is https://pastebin.com/3TtsjXSX last trace , after my gateway analyzes it https://pastebin.com/76rNVnnp - is "mydomain.com" your real domain? - funny that Message-Id is signed in DKIM and DKIM is valid. hmmm more to think about later. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Linux is like a teepee: no Windows, no Gates and an apache inside...
Re: spoofing mail
On 29.11.18 09:30, Rupert Gallagher wrote: Message-ID and To have the same domain, but From does not. You should have never received that mail. On 30.11.18 21:09, Rupert Gallagher wrote: Although the RFC allows muas not to include the mid, the same RFC does not mandate mtas to accept them. Since 100% of such emails on our records are spam, then we reject them upfront. I understand that spammers and scummers hate our policy, but hey, who cares, right? Our inbox, our rules. you have mistaken "You should have never received that mail." with "We would have never received that mail." I am of course aware of such policies, but they differ site to a site, admin to an admin and company to a company. The fact that you refuse some kind of e-mail does not mean that others should be doing the same. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Linux IS user friendly, it's just selective who its friends are...
Re: spoofing mail
On 29.11.18 09:30, Rupert Gallagher wrote: Message-ID and To have the same domain, but From does not. You should have never received that mail. this happens when message-id is added by mailserver of the recipient. Should hit MSGID_FROM_MTA_HEADER. And, yes, there could be rule that catches message-id added by internal server. Note that: - Message-ID is not required (has SHOULD in RFC) - many mailservers add message-id if it doesn't exist. On Wed, Nov 28, 2018 at 19:15, Rick Gutierrez wrote: El mié., 28 nov. 2018 a las 6:03, Christian Grunfeld () escribió: Hi, this is a logcould you paste the email headers? cheers I do not know if it is useful, the amavisd + spamassassin I have it in front of the mail server. https://pastebin.com/ktMUDLps not available anymore :-( -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The 3 biggets disasters: Hiroshima 45, Tschernobyl 86, Windows 95
Re: X-Relay-Countries not working
On 27.11.18 12:51, Brent Clark wrote: I have the following spam email, and I picked up that the plugin 'Mail::SpamAssassin::Plugin::RelayCountry', is not picking up Korea. https://pastebin.com/i45KsgVk header RELAYCOUNTRY_BAD X-Relay-Countries =~ /^(CN|RU|SU|IN|BR|UA|KR)/ describe RELAYCOUNTRY_BAD Relayed through foreign countries scoreRELAYCOUNTRY_BAD 1.0 add_header all Relay-Country _RELAYCOUNTRY_ In my testing, I added ZA, and it picked up for IP 196.35.198.137. Also, does anyone know why the 27.102.212.207 is in square brackets. Geoip pics up: $ geoiplookup 27.102.212.207 GeoIP Country Edition: KR, Korea, Republic of Would anyone please share a rule, I can use to catch the above spam. tried runinning "spamassassin -D" over the e-mail? just to see if it picks the rule, if it finds the database etc -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Spam = (S)tupid (P)eople's (A)dvertising (M)ethod
Re: multiplying in rules
> But as I said it's the decimal fractions that cause it to fail and the > above rule doesn't need to contain decimal fractions. On Tue, 20 Nov 2018 13:36:52 -0500 micah anderson wrote: How can I do it without the fractions? On 20.11.18 21:05, RW wrote: meta LOCAL_EXCEEDED_PHISH 4*__MAILBOX + 4*__LOCAL_EXCEEDED + 4*__LOCAL_STORAGE + 4*__LOCAL_LIMIT > 10 or meta LOCAL_EXCEEDED_PHISH __MAILBOX + __LOCAL_EXCEEDED + __LOCAL_STORAGE + __LOCAL_LIMIT >= 3 note that the latter does not mean the same when either of those rules has "tflags multiple" set (unless with maxhits=1) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. He who laughs last thinks slowest.
DKIMWL_WL_MED spams
Hello, I have recently noticed spams spreading via amasonses.com and outlook.com. hitting DKIMWL_WL_MED that pushed score below threshold. especially amazonses.com mail seemed to be amazon cloud servers. Has anyone noticed this too? I have disabled DKIMWL_WL_MED for now. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Remember half the people you know are below average.
what is FromNameSpoof supposed to catch?
wasn't FromNameSpoof supposed to catch this kind of mails? From: "RB Techgum " when testing this with rules proposed in FromNameSpoof docs, none hit. because of https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7624 I have applied following patch https://svn.apache.org/viewvc/spamassassin/branches/3.4/lib/Mail/SpamAssassin/Plugin/FromNameSpoof.pm?r1=1842029&r2=1842028&pathrev=1842029&view=patch - hope it's not the culprit. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. On the other hand, you have different fingers.
FromNameSpoof usage examples and experience
Hello, did anyone set up rules to use the FromNameSpoof plugin? Do you have any experiences about it? Thanks. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "They say when you play that M$ CD backward you can hear satanic messages." "That's nothing. If you play it forward it will install Windows."
Re: Forgery with SPF/DKIM/DMARC
On 16.11.18 08:44, Robert Fitzpatrick wrote: We're having an issue with spam coming from the same company even though SPF and DKIM is setup with DMARC to reject. Take this forwarded email for instances does the mail pass or fail SPF and DKIM? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows found: (R)emove, (E)rase, (D)elete
Re: unexpected FN, how to improve/tune to catch
On 15.11.18 09:42, Ian Zimmerman wrote: > # This one disables Bayes. ... > tiny detail. use_learner 0 On Fri, 16 Nov 2018 09:52:05 +0100 Matus UHLAR - fantomas wrote: 1. this description is invalid. use_bayes disables bayes. On 16.11.18 14:13, RW wrote: use_learner 0, in theory, disables all machine learning plug-ins. I would prefer a more thorough explanation, can you please provide it? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. LSD will make your ECS screen display 16.7 million colors
Re: unexpected FN, how to improve/tune to catch
On 15.11.18 09:42, Ian Zimmerman wrote: This little pearl got through upstream filter on a mailing list. such spam is very hard to detect, because mailing lists tend to clear negative-scoring rules and add some positive-scoring. such spam should be filtered at mailing list level before this happens. My scores for it were: RCVD_IN_DNSWL_MED=-2.3,SPF_HELO_PASS=-0.0,MAILING_LIST_MULTI=-1.0,TOTAL=-3.3 these are standard rules, and since the mail came from a mailing list, it's expected to score negatively. what can help you - BAYES - network rules - URI blacklists Do you have those enabled? Here is my user_prefs file: # This one disables Bayes. If you want to use Bayes remove or comment # out this line. You'll need to manage your Bayes database with a # cronjob or something. I can help but I won't do the last tiny detail. use_learner 0 1. this description is invalid. use_bayes disables bayes. 2. bayes is the best to help you to detect spam. Don't complain when you have disabled it. Where are all the other scores? I would have expected at least something for bit.ly and for the misspelled closing line, which is a dead spam give-away to a human ... did you enable/install razor, pyzor, dcc, spf and dkim libraries? I have run spamassassin -D on it and everything seems to work as designed i.e. the tests including URIBL run fine, they just don't catch anything. It's disappointing. apparently it does not contain any URI. Maybe the KAM rules would have got this one? no. They can help, but hardly help you to push -3.3 scoring mail received via mailing list over spam threshold. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Chernobyl was an Windows 95 beta test site.
Re: normalize_charset effects
On Wed, 14 Nov 2018 09:43:25 +0100 Matus UHLAR - fantomas wrote: what are direct effects of normalize_charset? On 14.11.18 14:37, RW wrote: It causes mime text parts that aren't UTF-8 to be translated into UTF-8. does this apply only for rules or even for things like bayes? I mean, when a iso-8859-* word is already tokenized in bayes, will it be missed? (i hope I did get bayes properly) Will enabling normalize_charset cause some immediate benefits or disadvantages for us? It mainly means that rules can assume UTF-8, and you don't have to allow for other character sets. And spam can't avoid tests by using an uncommon character set. These day so much email is in UTF-8 (or the ASCII subset) that it wont make all that much difference. now a question raised if non-UTF8 spam will be caught more likely when normalizing... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe.
normalize_charset effects
Hello, what are direct effects of normalize_charset? Does it affect e.g. bayes? I found some slowness report https://bz.apache.org/SpamAssassin/show_bug.cgi?id=5691 https://rt.perl.org//Public/Bug/Display.html?id=66852 ...that should be fixed in 5.20 and discussion around: http://spamassassin.1065346.n5.nabble.com/Current-best-practices-around-normalize-charset-td105840.html https://mail-archives.apache.org/mod_mbox/spamassassin-dev/200907.mbox/<6c399e450907080218r4def8c85u60823cae6f632...@mail.gmail.com> however are quite old, and recently normalized_charset option appeared in local.cf with 3.4.2 upgrade. Will enabling normalize_charset cause some immediate benefits or disadvantages for us? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: ALL_TRUSTED always shown in X-Spam-Status header
>On Sat, Nov 10, 2018 at 08:04:42PM -0500, listsb wrote:
>>i've just noticed that every mail received seems to be hitting the
ALL_TRUSTED test [ALL_TRUSTED=-1], regardless of where the message has come from. i
have the following:
>>
>>>grep -riF 'internal_networks' /etc/spamassassin/*
>>/etc/spamassassin/99_local-config.cf:internal_networks
198.19.20.50/32
>>/etc/spamassassin/99_local-config.cf:internal_networks
198.19.20.212/32
>>
>>here is a set of sample headers, slightly sanitized:
>>
>>http://dpaste.com/33J7SF5
>>
>>how can i troubleshoot why this is happening?
On 11.11.18 19:23, Henrik K wrote:
>Are you perhaps using amavisd-new 2.11.x ? It has originating bug that
>makes it always hit ALL_TRUSTED.
>
>https://gitlab.com/amavis/amavis/issues/6
On Sun, Nov 11, 2018 at 06:43:27PM +0100, Matus UHLAR - fantomas wrote:
is it the right issue? This one mentions DKIM not signing.
Can it be the patch that causes everything hitting ALL_TRUSTED?
You have also commented you need to investigate the patch, have you already?
On 11.11.18 20:00, Henrik K wrote:
Yes
https://lists.amavis.org/pipermail/amavis-users/2018-November/005539.html
https://lists.amavis.org/pipermail/amavis-users/2018-November/005540.html
It's trivial to see from logs. Incoming external mail is always marked
AcceptedInternal / LOCAL.
current problem is not mentioned there, only here in this list (which is not
even amavis list).
Passed CLEAN {AcceptedInternal,Quarantined}, LOCAL
Amavisd-new passes originating flag to SpamAssassin internally with some
suppl_attr magic.. that's why it's even harder to diagnose, if you don't
know that it happens in the background..
I believe this only applies when originating flag is set.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Posli tento mail 100 svojim znamim - nech vidia aky si idiot
Send this email to 100 your friends - let them see what an idiot you are
Re: ALL_TRUSTED always shown in X-Spam-Status header
On Sat, Nov 10, 2018 at 08:04:42PM -0500, listsb wrote: i've just noticed that every mail received seems to be hitting the ALL_TRUSTED test [ALL_TRUSTED=-1], regardless of where the message has come from. i have the following: grep -riF 'internal_networks' /etc/spamassassin/* /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.50/32 /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.212/32 here is a set of sample headers, slightly sanitized: http://dpaste.com/33J7SF5 how can i troubleshoot why this is happening? On Nov 11, 2018, at 12.23, Henrik K wrote: Are you perhaps using amavisd-new 2.11.x ? It has originating bug that makes it always hit ALL_TRUSTED. https://gitlab.com/amavis/amavis/issues/6 On 11.11.18 13:08, listsb wrote: i'm currently using 2.9.0. in such case, according to previous message, it's important to check amavis settings. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. One OS to rule them all, One OS to find them, One OS to bring them all and into darkness bind them
Re: ALL_TRUSTED always shown in X-Spam-Status header
On Sat, Nov 10, 2018 at 08:04:42PM -0500, listsb wrote: i've just noticed that every mail received seems to be hitting the ALL_TRUSTED test [ALL_TRUSTED=-1], regardless of where the message has come from. i have the following: >grep -riF 'internal_networks' /etc/spamassassin/* /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.50/32 /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.212/32 here is a set of sample headers, slightly sanitized: http://dpaste.com/33J7SF5 how can i troubleshoot why this is happening? On 11.11.18 19:23, Henrik K wrote: Are you perhaps using amavisd-new 2.11.x ? It has originating bug that makes it always hit ALL_TRUSTED. https://gitlab.com/amavis/amavis/issues/6 is it the right issue? This one mentions DKIM not signing. Can it be the patch that causes everything hitting ALL_TRUSTED? You have also commented you need to investigate the patch, have you already? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I don't have lysdexia. The Dog wouldn't allow that.
Re: ALL_TRUSTED always shown in X-Spam-Status header
On 10.11.18 20:04, listsb wrote: i've just noticed that every mail received seems to be hitting the ALL_TRUSTED test [ALL_TRUSTED=-1], regardless of where the message has come from. i have the following: grep -riF 'internal_networks' /etc/spamassassin/* /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.50/32 /etc/spamassassin/99_local-config.cf:internal_networks 198.19.20.212/32 here is a set of sample headers, slightly sanitized: http://dpaste.com/33J7SF5 how can i troubleshoot why this is happening? show us an example of such mail. With complete headers. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. BSE = Mad Cow Desease ... BSA = Mad Software Producents Desease
Re: Bayes underperforming, HTML entities?
On Nov 8, 2018, at 2:30 AM, Matus UHLAR - fantomas wrote: Do you use autolearn? There are a few rules to detect ham (score negatively), many of them based on default whitelists and DNS whitelists, where many mails come from grey area companies, not necessarily spam, but training their mail as ham can lower the detection rate of real spams. On 08.11.18 12:06, Amir Caspi wrote: autolearn is technically enabled, but every single message in ham (inbox) has autolearn=no, and the same is true for my spam store. So, none of my tokens were autolearned, and all (should have) resulted only from my manual training. how many spams and hams did you train then? I found this number of tokens low, and have increased it. bayes_expiry_max_db_size262144 Are you recommending increasing TO this number, or FROM this number? It looks like my spam tokens are approaching this number, so I am assuming you think I should go higher? Any recommended number? I have increased to this number, on some servers even to double of that number. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. There's a long-standing bug relating to the x86 architecture that allows you to install Windows. -- Matthew D. Fuller
Re: Bayes underperforming, HTML entities?
On 07.11.18 12:33, Amir Caspi wrote: In the past couple of weeks I've gotten a number of clearly-spam messages that slipped past SA, and the only reason was because they were getting low Bayes scores (BAYES_50 or even down to BAYES_00 or BAYES_05). I do my Bayes training manually on both ham and spam so there should not be any mis-categorizations... and things worked fine until a few weeks ago, so I don't know what's going on now. I've had similar experience after running SA in some pleaces. Do you use autolearn? There are a few rules to detect ham (score negatively), many of them based on default whitelists and DNS whitelists, where many mails come from grey area companies, not necessarily spam, but training their mail as ham can lower the detection rate of real spams. Here's the magic dump: -bash-3.2$ sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 253112 0 non-token data: nspam 0.000 0 106767 0 non-token data: nham 0.000 0 150434 0 non-token data: ntokens I found this number of tokens low, and have increased it. bayes_expiry_max_db_size262144 could help in the long run. 0.000 0 1536087614 0 non-token data: oldest atime 0.000 0 1541617125 0 non-token data: newest atime 0.000 0 1541614751 0 non-token data: last journal sync atime 0.000 0 1541614749 0 non-token data: last expiry atime 0.000 05529600 0 non-token data: last expire atime delta 0.000 0 1173 0 non-token data: last expire reduction count -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have.
Re: config files in spamasassin is unintended tlds :/
On 4 Nov 2018, at 11:45, Grant Taylor wrote: Why does it matter if there's a naming collision between DNS domain names and file names? Bill Cole skrev den 2018-11-04 19:25: Discussion of config files for SpamAssassin and Postfix has intermittently been matched by URI DNSBLs. Some years ago I discovered just how widespread dumb bounce models were when I talked about the master config file for Postfix on the Postfix Users list, the same week that someone was spamvertising URLs under master (dot) cf. On 04.11.18 19:48, Benny Pedersen wrote: Nov 3 03:22:50 localhost named[2301]: connection refused resolving '72_scores.cf/NS/IN': 2a04:1b00:6::1#53 [...] Oct 31 08:30:38 localhost named[2301]: connection refused resolving '20_imageinfo.cf/NS/IN': 2a04:1b00:6::1#53 so ns.cf blocks my named now, i cant resolve any cf domains with it time to change imho I recommend chasing who is treating those as URLs. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Atheism is a non-prophet organization.
Re: Version 3.4.2, Debian Stretch
On 26.10.18 10:04, Jan Münnich wrote: The Debian package is not well maintained anymore unfortunately. who told you that? 3.4.2 is in unstable sinde Oct 01, testing since Oct 03. But it's very easy to compile SpamAssassin yourself on Debian Stretch: this leads to problems, you must prepared to do builds for yourself always. Simply do NOT do this, not on debian. If you really want built, download source package from sid and try building on stretch/jessie. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. LSD will make your ECS screen display 16.7 million colors
Re: Cannot install SpamAssassin on Ubuntu 18.04.1 (gpg not found?)
On Thu, 25 Oct 2018 16:07:02 +0200 Matus UHLAR - fantomas wrote: >On Thu, 25 Oct 2018 08:37:45 -0400 Alexander Lieflander wrote: >> As a side-note, it seems like the error message returned by dpkg >> (and thus SpamAssassin, I guess) is incorrect. Where it mentions >> “sa-compile”, it should really be mentioning “sa-update”, as the >> man page for sa-update contains the “--nogpg” option, and the man >> page for sa-compile does not. where did it say sa-compile? On 25.10.18 15:16, RW wrote: It failed when sa-compile was being installed I see now - it was in the attachments. The logs are quite clear for me: Setting up spamassassin (3.4.1-8build1) ... error: gpg required but not found! It is not recommended, but you can use "sa-update" with the --no-gpg to skip the verification. dpkg: error processing package spamassassin (--configure): installed spamassassin package post-installation script subprocess returned error exit status 2 ... spamassassin could not be set up, because of the GPG problem. dpkg: dependency problems prevent configuration of sa-compile: sa-compile depends on spamassassin; however: Package spamassassin is not configured yet. dpkg: error processing package sa-compile (--configure): dependency problems - leaving unconfigured ... sa-compile could not be installed because spamassassin wasn't configured. On 25.10.18 14:37, RW wrote: >This is a consequence of Ubuntu (or Debian) splitting off sa-compile >into a separate package. The error occurred while checking >sa-compile's dependency, the spamassassin package. this should not happen at all. when sa-compile is installed, spamassassin (and sa-update) should be installed and configured. I would guess that there was no problem when spamassassin was installed and sa-compile was installed later. well, the problem was there already when spamassassin was installed, Luckily the problem is solved now. It was outside of debian/ubuntu scope - nobody should expect systems working with external changes. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: Cannot install SpamAssassin on Ubuntu 18.04.1 (gpg not found?)
On Thu, 25 Oct 2018 08:37:45 -0400 Alexander Lieflander wrote: As a side-note, it seems like the error message returned by dpkg (and thus SpamAssassin, I guess) is incorrect. Where it mentions “sa-compile”, it should really be mentioning “sa-update”, as the man page for sa-update contains the “--nogpg” option, and the man page for sa-compile does not. where did it say sa-compile? the only mistake I see is that sa-update mentions "--no-gpg" option but really has "--nogpg" option. perl module Mail/SpamAssassin/Util/DependencyInfo.pm mentions that too, btw. nothing with sa-compile. On 25.10.18 14:37, RW wrote: This is a consequence of Ubuntu (or Debian) splitting off sa-compile into a separate package. The error occurred while checking sa-compile's dependency, the spamassassin package. this should not happen at all. when sa-compile is installed, spamassassin (and sa-update) should be installed and configured. well, I checked on debian 8 and debian 9, not on ubuntu 18.04 -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "To Boot or not to Boot, that's the question." [WD1270 Caviar]
Re: Extreme scores from FRNAME rules.
On 25/10/2018 11:43, Matus UHLAR - fantomas wrote: On 25/10/2018 10:33, Matus UHLAR - fantomas wrote: bug number would help more... On 25.10.18 10:58, Reio Remma wrote: The bug contains no additional info. :) I was simply asked to post to the list. and this is exactly why it would be better to post the link to the bug, or at least the bug number, instead of just link to the attachment... On 25.10.18 11:46, Reio Remma wrote: No worries. Here it is: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7644 Good. I don't see FRNAME_IN_MSG_NO_SUBJ in rules now (apparently due to John Hardin's change) , but according to original description, they seem to match: * 2.5 FRNAME_IN_MSG_XPRIO From name in message + X-Priority A+B = 2.5 * 2.5 XPRIO_SHORT_SUBJ Has X-Priority header + short subject B+C = 2.5 * 2.5 FRNAME_IN_MSG_NO_SUBJ From name in message + short or no subject A+C = 2.5 so, in fact neither of them overlaps, but they all three in common seem to match three different conditions, where final score was 3*2.5 currently we have FRNAME_IN_MSG_XPRIO_NO_SUB which matches A+B+C but does not match short subject now. This could fix your problem, can you rescan the mail? current scores: score FRNAME_IN_MSG_NO_SUBJ 0.001 2.499 0.001 2.499 score FRNAME_IN_MSG_XPRIO 0.001 2.499 0.001 2.499 score FRNAME_IN_MSG_XPRIO_NO_SUB2.499 0.001 2.499 0.001 score XPRIO_SHORT_SUBJ 2.499 2.131 2.499 2.131 note that FRNAME_IN_MSG_NO_SUBJ and FRNAME_IN_MSG_XPRIO are not defined. I did first think of FRNAME_IN_MSG_XPRIO_NO_SUB balancing those three rules - it could score negatively, so when mail would match all three meta-rules, the final score wouldn't be triple of their scores. however, I understand that such thing is too much for manual testing. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. We are but packets in the Internet of life (userfriendly.org)
Re: Extreme scores from FRNAME rules.
On 22.10.18 21:34, Reio Remma wrote: I have this perfectly legit mail that has a +7.5 score from these three rules. * 2.5 FRNAME_IN_MSG_XPRIO From name in message + X-Priority * 2.5 XPRIO_SHORT_SUBJ Has X-Priority header + short subject * 2.5 FRNAME_IN_MSG_NO_SUBJ From name in message + short or no subject If it wasn't for the -1.9 from Bayes and -2.6 from TxRep, it would have been thrown away. Should these XPRIO/FRNAME rules stack like this? The e-mail in question is available here: https://bz.apache.org/SpamAssassin/attachment.cgi?id=5607 On 25/10/2018 10:33, Matus UHLAR - fantomas wrote: bug number would help more... On 25.10.18 10:58, Reio Remma wrote: The bug contains no additional info. :) I was simply asked to post to the list. and this is exactly why it would be better to post the link to the bug, or at least the bug number, instead of just link to the attachment... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Linux - It's now safe to turn on your computer. Linux - Teraz mozete pocitac bez obav zapnut.
Re: Extreme scores from FRNAME rules.
On 22.10.18 21:34, Reio Remma wrote: I have this perfectly legit mail that has a +7.5 score from these three rules. * 2.5 FRNAME_IN_MSG_XPRIO From name in message + X-Priority * 2.5 XPRIO_SHORT_SUBJ Has X-Priority header + short subject * 2.5 FRNAME_IN_MSG_NO_SUBJ From name in message + short or no subject If it wasn't for the -1.9 from Bayes and -2.6 from TxRep, it would have been thrown away. Should these XPRIO/FRNAME rules stack like this? The e-mail in question is available here: https://bz.apache.org/SpamAssassin/attachment.cgi?id=5607 bug number would help more... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Posli tento mail 100 svojim znamim - nech vidia aky si idiot Send this email to 100 your friends - let them see what an idiot you are
Re: Error 74 with spamc
"Bill Cole" writes: When I run it again I see in the logging: Oct 22 16:47:15 munus.decebal.nl spamd[17102]: spamd: connection from localhost [::1]:58764 to port 783, fd 5 Oct 22 16:47:15 munus.decebal.nl spamd[17102]: spamd: setuid to imaps succeeded Oct 22 16:47:15 munus.decebal.nl spamd[17102]: spamd: service unavailable: TELL commands are not enabled, set the --allow-tell switch. Oct 22 16:47:15 munus.decebal.nl spamd[17101]: prefork: child states: II It is a bit strange. I had the same problem 1½ year ago. I solved it by adding --allow-tell switch in the service file. Now it contained: ExecStart=/usr/sbin/spamd -d --pidfile=/var/run/spamd.pid $OPTIONS I do not see the OPTIONS defined. I substituted --allow-tell for $OPTIONS and restarted the service. Now it works again. But why the service file has been changed … That would be an issue for whoever packages SA for your system. There is no systemd service file distributed in the SA release. On 23.10.18 00:48, Cecil Westerhof wrote: It is a Debian Stretch system. I will ask on a Debian newsgroup. I don't see init file on debian my stretch SA installation. but if it was there, admins are advised to copy them to /etc and make modifications there, instead of modifying files that get overwritten at upgrade (debian checks for config file changes, but init files aren't apparently config files). -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Spam is for losers who can't get business any other way.
Re: Is fuzzyocr i.e. Image scanning
On 16.10.18 18:42, RW wrote: Bayes might work, but I wouldn't like to see it added to body text because corrupted text could look like obfuscation. On Wed, 17 Oct 2018, Matus UHLAR - fantomas wrote: it should be pushed back to body text just for filters like bayes. The same could/should be done for attachhed .doc, .pdf files etc. On 17.10.18 07:56, John Hardin wrote: ...which would be much more reliable than OCR. If it was a resource-allocation decision for pulling text from doc/pdf vs. updating OCR, I'd push for the former. this could be easily configured by installing modules or loading them. btw, both PDF and word documents can contain images too ... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 99 percent of lawyers give the rest a bad name.
Re: Is fuzzyocr i.e. Image scanning
>On Tue, 16 Oct 2018 11:49:54 +0700 Olivier wrote: >> One of my holdback with FuzzyOCR is that you have to provide an >> independant word list, while we have a very good tool to analyze >> text contents: SpamAssassin itself. So I would much prefer >> FuzzyOCR to feed the OCR'ed text back to SA for further analysis >> (the way pdfAssassin is working). On 16.10.18 13:34, RW wrote: >That works as long as the OCR remains very accurate. What happened >before was that the deployment of OCR lead spammers to make their >text much less readable. On Tue, 16 Oct 2018 15:48:34 +0200 Matus UHLAR - fantomas wrote: I think that original reason was that available OCR programs were not reliable enough. I have tested gocr, ocrad and tesseract some >10 years ago, with not very satisfying results, gocr being best at that time. Since then, google took tesseract and made it much better. I believe tht currently it would bve viable to push ocr output to spamassassin for processing with bayes and other rules. On 16.10.18 18:42, RW wrote: Bayes might work, but I wouldn't like to see it added to body text because corrupted text could look like obfuscation. it should be pushed back to body text just for filters like bayes. The same could/should be done for attachhed .doc, .pdf files etc. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 42.7 percent of all statistics are made up on the spot.
Re: Is fuzzyocr i.e. Image scanning
On Tue, 16 Oct 2018 11:49:54 +0700 Olivier wrote: One of my holdback with FuzzyOCR is that you have to provide an independant word list, while we have a very good tool to analyze text contents: SpamAssassin itself. So I would much prefer FuzzyOCR to feed the OCR'ed text back to SA for further analysis (the way pdfAssassin is working). On 16.10.18 13:34, RW wrote: That works as long as the OCR remains very accurate. What happened before was that the deployment of OCR lead spammers to make their text much less readable. I think that original reason was that available OCR programs were not reliable enough. I have tested gocr, ocrad and tesseract some >10 years ago, with not very satisfying results, gocr being best at that time. Since then, google took tesseract and made it much better. I believe tht currently it would bve viable to push ocr output to spamassassin for processing with bayes and other rules. As for your question about the place for image scanning, if your MTA has the resources to do so, why not? Because it's better if it's combined with other information. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. A day without sunshine is like, night.
Re: Bayes
On 15.10.18 21:04, Antony Stone wrote: I thought http://xkcd.org/2059 was appropriate to highlight on this list :) Any volunteers to implement this in SA? ;-) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Save the whales. Collect the whole set.
Re: RBL
On 10/11/2018 01:35 AM, Matus UHLAR - fantomas wrote: I for example run spamass-milter with -r 10 (rejects score over 10) at one machine, and amavisd-milter with "spam_kill_level_maps=> 10", along with postscreen. This way mail gets refused when listed in DNSBLs, while not when DNSWL (but still when DNSBL score is higher than DNSWL) and also when SA detects it's score is over 10. On 11.10.18 09:03, Grant Taylor wrote: But that's doing the RBL checks in SpamAssassin, not directly in the MTA. postscreen does the hecks as part of the MTA. both DNS and manual whitelists are applicable. ...clients from internal networks run SA as content_filter (post-queue) so they don't complain sending mail (SA scanning at MTA level) taked too long. That's why I tended to have different email hygiene configurations on the MSA and MTA(s). Ideally the client submits to the MSA with minimal checks, after all we know who the message originated from based on authentication. The MSA will then smart host the message through the MTA, which does more hygiene checking. MSAs should run on ports 465 and 587, which are easy to configure differently. different configuration of port 25 (many clients use because of backward compatibility) can be achieved by listening on different interface, e.g. by redirecting internet traffic to different IP or port (on gateway or in the local firewall) I originally migrated to this configuration when I had clients on dial up connections run into timeouts whens s l o w l y sending attachments. So they can take as long as they need to (or not) to send to the MSA, which can then quickly send to the MTA with filtering. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: RBL
On 10/10/2018 01:56 PM, Tom Hendrikx wrote: However, in general it's better to use DNSBLs at the MTA level, which uses a lot less resources than implementing them in Spamassassin. So try and set them up in postfix first. On 10.10.18 14:09, Grant Taylor wrote: I conceptually agree. However, I prefer to do some RBL testing in SpamAssassin because I can easily check multiple RBLs and tag messages as spam, or reject, based on spam score. Conversely, most MTA's implement RBLs as a binary pass / fail situation. Thus SpamAssassin gives more flexibility and provides a configurable gray area that MTA's can't do themselves. note that spamassassin can run at MTA level, refusing mail when it's found to be sure spam and tagging when it's not. I for example run spamass-milter with -r 10 (rejects score over 10) at one machine, and amavisd-milter with "spam_kill_level_maps=> 10", along with postscreen. This way mail gets refused when listed in DNSBLs, while not when DNSWL (but still when DNSBL score is higher than DNSWL) and also when SA detects it's score is over 10. ...clients from internal networks run SA as content_filter (post-queue) so they don't complain sending mail (SA scanning at MTA level) taked too long. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. M$ Win's are shit, do not use it !
Re: repeated sa-update problems
On 20.09.18 16:05, Matus UHLAR - fantomas wrote:
I looked at update times and they are different each day - debian script
sleeps random number of seconds (up to one hour) in order to lower the
impact at mirror servers.
I have removed the "--fail" option from curl and will look at error message
if there's any.
I'll keep you updated and will fill bugreport if I'm able to find out
anything useful.
I was able to repeat this problem now:
# /usr/bin/curl --verbose -L -O --remote-time -g --max-redirs 2
--connect-timeout 30 --max-time 300 -o 1843052.tar.gz --
http://sa-update.spamassassin.org/1843052.tar.gz
* Hostname was NOT found in DNS cache
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0*
Trying 64.142.56.146...
* Connected to sa-update.spamassassin.org (64.142.56.146) port 80 (#0)
GET /1843052.tar.gz HTTP/1.1
User-Agent: curl/7.38.0
Host: sa-update.spamassassin.org
Accept: */*
0 00 00 0 0 0 --:--:-- 0:00:11 --:--:-- 0<
HTTP/1.1 200 OK
< Date: Mon, 08 Oct 2018 14:16:19 GMT
* Server Apache/2.4.6 (CentOS) is not blacklisted
< Server: Apache/2.4.6 (CentOS)
< Last-Modified: Mon, 08 Oct 2018 03:19:20 GMT
< ETag: "4600c-577af16429e00"
< Accept-Ranges: bytes
< Content-Length: 286732
< Content-Type: application/x-gzip
<
{ [data not shown]
0 280k0 10 0 0 0 --:--:-- 0:00:13 --:--:-- 0*
transfer closed with 286731 bytes remaining to read
* Closing connection 0
curl: (18) transfer closed with 286731 bytes remaining to read
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 1 Oct 8 16:16 1843052.tar.gz
look at today's debug log says:
Oct 8 07:12:59.899 [20257] dbg: channel: selected mirror
http://sa-update.spamassassin.org
Oct 8 07:12:59.899 [20257] dbg: http: url:
http://sa-update.spamassassin.org/1843052.tar.gz
Oct 8 07:12:59.899 [20257] dbg: http: downloading to:
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1843052.tar.gz, new
Oct 8 07:12:59.899 [20257] dbg: util: executable for curl was found at
/usr/bin/curl
Oct 8 07:12:59.899 [20257] dbg: http: /usr/bin/curl -s -L -O --remote-time -g
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843052.tar.gz --
http://sa-update.spamassassin.org/1843052.tar.gz
Oct 8 07:13:15.385 [20257] dbg: http: process [20258], exit status: 4608
Oct 8 07:13:15.385 [20257] dbg: channel: selected mirror
http://sa-update.ena.com
Oct 8 07:13:15.385 [20257] dbg: http: url:
http://sa-update.ena.com/1843052.tar.gz
Oct 8 07:13:15.385 [20257] dbg: http: downloading to:
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1843052.tar.gz, update
Oct 8 07:13:15.385 [20257] dbg: util: executable for curl was found at
/usr/bin/curl
Oct 8 07:13:15.385 [20257] dbg: http: /usr/bin/curl -s -L -O --remote-time -g
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843052.tar.gz -z
1843052.tar.gz -- http://sa-update.ena.com/1843052.tar.gz
Oct 8 07:13:15.889 [20257] dbg: http: process [20272], exit status: 0
This looks that invalid file was downloaded from sa-update.spamassassin.org,
and while next curl invocation succeeded with exit code 0, the file was not
overridden:
# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz --
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 243 Oct 8 16:21 1843052.tar.gz
# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz --
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 243 Oct 8 16:21 1843052.tar.gz
# rm 1843052.tar.gz
# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz --
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 286732 Oct 8 05:19 1843052.tar.gz
(the file size changed to 243 because of my tests).
further look at logs says that all failed downloads were from
sa-update.spamassassin.org:
Sep 28 07:43:07.888 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz --
http://sa-update.spamassassin.org/1842077.tar.gz
Sep 28 07:43:21.973 [7018] dbg: http: process [7019], exit status: 4608
Oct 5 06:35:10.552 [29702] dbg: http: /usr/bin/curl -s -L -O --remote-time -g
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842787.tar.gz --
http://sa-update.spamassassin.org/1842787.tar.gz
Oct 5 06:35:29.199 [29702] dbg: http: process [29705], exit status: 4608
Oct 7 07:17:37.644 [30424] dbg: http: /usr/bin/curl -s -L -O --remote-time -g
--max-redirs 2
Re: FPs on FORGED_MUA_MOZILLA (for my own hand-typed messages from my latest-version Thunderbird)
On 10/2/2018 9:59 AM, Matus UHLAR - fantomas wrote:
can you post the headers?
or at least the Message-Id?
On 02.10.18 11:07, Rob McEwen wrote:
Here is the message as THEIR system saw it (with my client's info
masked) - but it looks like their Kerio (or the customer's email
client?) might be not be storing everything as it was originally sent?
it's possible. It _could_ cause the problem.
...but this is what my client sent me, fwiw:
Received: from mail.powerviewmail.com
<http://mail.powerviewmail.com>([204.9.77.40])
by with ESMTPS
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits))
for ;
Mon, 1 Oct 2018 15:17:10 +0200
DKIM-Signature: a=rsa-sha256; t=1538399816; x=1539004616;
s=ivm_invaluement; d=invaluement.com <http://invaluement.com>;
c=relaxed/relaxed; v=1;
bh=C6QzEUsPRf8EoiIEIhSF1hnXxy9JIlmjGFO/079v4QQ=; h=From:Subject:Date:Message-ID:To:MIME-Version:Content-Type:In-Reply-To:References;
b=V5Sv2lZUWL4P29pcEVY6r/8uFRcuNL1hR794r6M1TJZcvw+i4vTgrvWf+CKSN/F1f2FS/0CdF4UCux+dS/vFjj3X9fdmwv9jpizZqwvJseyCYEmT2HItdeqo0NfNIoQwziEPDMgYS3f35iWlcb7wqrPjfx5EslHr+oC0eoeGBaA=
Received: from [204.9.77.40] ([204.9.77.40])
by mail.powerviewmail.com
<http://mail.powerviewmail.com>(IceWarp 12.0.2.1 x64) with ASMTP id
201810010916565985
for ; Mon, 01 Oct 2018 09:16:
No message-id here, but also no X-Spam headers.
Here is an excerpt from the headers, copied from the message in my
Thunderbird "sent" folder:
unwrapped:
Message-ID: <39397904-9830-5010-a3d2-a62af8326...@invaluement.com>
this does seem to match:
MESSAGEID =~
/^<(?:[a-f\d]{8}-(?:[a-f\d]{4}-){3}[a-f\d]{12}|[A-F\d]{8}\.[A-F1-9][A-F\d]{0,7})\@\S+>$/m
8h-4h-4h-4h-12h@
hmmm we need to look at
(__LYRIS_EZLM_REMAILER || __GATED_THROUGH_RCVD_REMOVER ||
__WACKY_SENDMAIL_VERSION || __IPLANET_MESSAGING_SERVER ||
__HOTMAIL_BAYDAV_MSGID || __SYMPATICO_MSGID)
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod
Re: FPs on FORGED_MUA_MOZILLA (for my own hand-typed messages from my latest-version Thunderbird)
On 2 Oct 2018, at 9:36, Rob McEwen wrote: SIDE NOTE: I don't think there was any domain my message that was blacklisted on URIBL - so I can't explain the "URIBL_BLOCKED", but that only scored 0.001, so that was innocuous. I suspect that that rule is malfunctioning on their end, and then they changed the score to .001 - so just please ignore that for the purpose of this discussion. On 02.10.18 11:48, Bill Cole wrote: No, "URIBL_BLOCKED" means that the URIBL DNS returned a value that is supposed to be a message to a mail admin that they are using URIBL wrong A mail filtering system that gets URIBL_BLOCKED hits is broken. A mail filtering system that gets them chronically is mismanaged. Nonsense. There is no such implication here. While URIBL_BLOCKED may and most of the time apparently does mean that system uses DNS server shared with too many clients, any system that receives and checks too much mail may get URIBL_BLOCKED just because they have crossed the limit, withous using it wrong or being broken. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Support bacteria - they're the only culture some people have.
Re: FPs on FORGED_MUA_MOZILLA (for my own hand-typed messages from my latest-version Thunderbird)
On 02.10.18 09:36, Rob McEwen wrote:
A client of mine wasn't getting my own hand-typed messages.
Unfortunately, they had their SA set to block on a score of 3 (which
is aggressive), and this particular rule hit plus a tiny bit of other
things put it above 3. But what is weird - is that it was hitting on
hand typed-messages from me - that I sent directly from my
latest-version of Thunderbird. So this was NOT "forged" at all! (Also,
I suspect that the bayes hit was due to previous such messages from me
getting blocked and feeding his bayes?)
Any suggestions? Could my client be using a very old version of SA -
where this is fixed already? (they are using SA from Kerio).
Here are the headers:
X-Kerio-Anti-Spam: Build: [Engines: 2.15.8.1169, Stamp: 3], Multi:
[Enabled, t: (0.12,0.017258)], BW: [Enabled, t: (0.13)], RTDA:
[Enabled, t: (0.052863), Hit: No, Details: v2.7.15; Id:
15.1i65djr.1conscun2.ocr1k], total: 0(700)
X-Spam-Status: Yes, hits=3.8 required=3.0
tests=KERIO_ANTI_SPAM: -0.000, AWL: -0.000, BAYES_50: 1.567,
FORGED_MUA_MOZILLA: 2.309, HTML_MESSAGE: 0.001, URIBL_BLOCKED: 0.001,
TOTAL_SCORE: 3.878,autolearn=no
Suggestions?
can you post the headers?
or at least the Message-Id?
metaFORGED_MUA_MOZILLA (__MOZILLA_MUA && !__UNUSABLE_MSGID &&
!__MOZILLA_MSGID)
header __MOZILLA_MUA User-Agent =~ /^mozilla\b/i
header __MOZILLA_MSGID MESSAGEID =~
/^<(?:[a-f\d]{8}-(?:[a-f\d]{4}-){3}[a-f\d]{12}|[A-F\d]{8}\.[A-F1-9][A-F\d]{0,7})\@\S+>$/m
meta__UNUSABLE_MSGID(__LYRIS_EZLM_REMAILER ||
__GATED_THROUGH_RCVD_REMOVER || __WACKY_SENDMAIL_VERSION ||
__IPLANET_MESSAGING_SERVER || __HOTMAIL_BAYDAV_MSGID || __SYMPATICO_MSGID)
header __HOTMAIL_BAYDAV_MSGID MESSAGEID =~
/^<[A-Z]{3}\d+-(?:DAV|SMTP)\d+[A-Z0-9]{25}\@phx\.gbl>$/m
header __IPLANET_MESSAGING_SERVER Received =~ /iPlanet Messaging Server/
header __LYRIS_EZLM_REMAILER List-Unsubscribe =~
/$/
header __SYMPATICO_MSGID MESSAGEID =~
/^$/m
header __WACKY_SENDMAIL_VERSIONReceived =~ /\/CWT\/DCE\)/
SIDE NOTE: I don't think there was any domain my message that was
blacklisted on URIBL - so I can't explain the "URIBL_BLOCKED", but
that only scored 0.001, so that was innocuous. I suspect that that
rule is malfunctioning on their end, and then they changed the score
to .001 - so just please ignore that for the purpose of this
discussion.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The only substitute for good manners is fast reflexes.
Re: repeated sa-update problems
On 9/20/2018 8:59 AM, Dave Jones wrote: I will have to check later if someone else can't check today. I am at a customer location where I don't have good VPN connection out and will be traveling this evening. I can check tomorrow if it can wait. On 20.09.18 09:05, Kevin A. McGrail wrote: It can wait. Matus also had the issue hitting my mirror and I know I don't use a CDN. On 20.09.18 16:05, Matus UHLAR - fantomas wrote: I looked at update times and they are different each day - debian script sleeps random number of seconds (up to one hour) in order to lower the impact at mirror servers. I have removed the "--fail" option from curl and will look at error message if there's any. I'll keep you updated and will fill bugreport if I'm able to find out anything useful. the problem repeated today Sep 28 07:43:07.888 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz -- http://sa-update.spamassassin.org/1842077.tar.gz Sep 28 07:43:21.973 [7018] dbg: http: process [7019], exit status: 4608 Sep 28 07:43:21.973 [7018] dbg: channel: selected mirror http://sa-update.space-pro.be Sep 28 07:43:21.974 [7018] dbg: http: url: http://sa-update.space-pro.be/1842077.tar.gz Sep 28 07:43:21.974 [7018] dbg: http: downloading to: /var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz, update Sep 28 07:43:21.974 [7018] dbg: util: executable for curl was found at /usr/bin/curl Sep 28 07:43:21.974 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz -z 1842077.tar.gz -- http://sa-update.space-pro.be/1842077.tar.gz Sep 28 07:43:22.304 [7018] dbg: http: process [7041], exit status: 0 Sep 28 07:43:22.305 [7018] dbg: http: url: http://sa-update.space-pro.be/1842077.tar.gz.sha1 Sep 28 07:43:22.305 [7018] dbg: http: downloading to: /var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz.sha1, new Sep 28 07:43:22.305 [7018] dbg: util: executable for curl was found at /usr/bin/curl Sep 28 07:43:22.305 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz.sha1 -- http://sa-update.space-pro.be/1842077.tar.gz.sha1 Sep 28 07:43:22.376 [7018] dbg: http: process [7043], exit status: 0 Sep 28 07:43:22.376 [7018] dbg: http: url: http://sa-update.space-pro.be/1842077.tar.gz.asc Sep 28 07:43:22.377 [7018] dbg: http: downloading to: /var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz.asc, new Sep 28 07:43:22.377 [7018] dbg: util: executable for curl was found at /usr/bin/curl Sep 28 07:43:22.377 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz.asc -- http://sa-update.space-pro.be/1842077.tar.gz.asc Sep 28 07:43:22.446 [7018] dbg: http: process [7045], exit status: 0 Sep 28 07:43:22.446 [7018] dbg: sha1: verification wanted: cb1b907b4f590fe24d0744cf60939685d51b3443 Sep 28 07:43:22.446 [7018] dbg: sha1: verification result: 953efe8f531a5a87f6d2d5a65b78b05e55599abc channel: SHA1 verification failed, channel failed # ls -lctr --full-time 1842077.* -rw-r--r-- 1 debian-spamd debian-spamd 1 2018-09-28 07:43:21.967880543 +0200 1842077.tar.gz -rw-r--r-- 1 debian-spamd debian-spamd 113 2018-09-28 07:43:22.371884772 +0200 1842077.tar.gz.sha1 -rw-r--r-- 1 debian-spamd debian-spamd 819 2018-09-28 07:43:22.443885519 +0200 1842077.tar.gz.asc # hd 1842077.tar.gz 1f|.| 0001 Sep 28 07:43:23 fgt date=2018-09-28,time=07:43:23,devname=xx,devid=xy,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=192.168.1.1,srcport=52411,srcintf="internal",dstip=176.28.55.20,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=87366444,proto=6,action=close,policyid=62,policytype=policy,dstcountry="France",srccountry="Reserved",trandisp=snat,transip=195.80.174.159,transport=52411,service="HTTP",duration=1,sentbyte=470,rcvdbyte=327,sentpkt=6,rcvdpkt=4,appcat="unscanned",wanin=111,wanout=150,lanin=150,lanout=111 Sep 28 07:43:23 fgt date=2018-09-28,time=07:43:23,devname=xx,devid=xy,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=192.168.1.1,srcport=52412,srcintf="internal",dstip=176.28.55.20,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=87366446,proto=6,action=close,policyid=62,policytype=policy,dstcountry="France",srccountry="Reserved",trandisp=snat,transip=195.80.174.159,transport=52412,service="HTTP",duration=1,sentbyte=425,rcvdbyte=550,sentpkt=6,rcvdpkt=4,appcat="unscanned",wanin=334,wanout=105,lanin=105,lanout=334 Sep 28 07:43:23 fgt date=2018-09-28,time=07:43:23,devname
Re: Hints needed for spf rule
On 9/22/2018 9:55 AM, RW wrote: /^v=spf1 .+(\?|\+)all$/ I believe [?+] would do the same easy to read, parse and maybe even to process (I have no idea how perl RE optimizer works) .+ should be .* or it wont match 'v=spf1 +all' I would remove the '$' as it doesn't appear do anything useful and could prevent matches on weird spf records. It may be worth splitting them into two rules for '?' and '+', there's no dns overhead and they seem like significantly different cases. hypotetically - masschecks should prove that. On 22.09.18 09:57, Kevin A. McGrail wrote: # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY - ifplugin Mail::SpamAssassin::Plugin::AskDNS askdns JMQ_SPF_NEUTRAL _SENDERDOMAIN_ TXT /^v=spf1 .*?\?all/ describe JMQ_SPF_NEUTRAL SPF set to ?all score JMQ_SPF_NEUTRAL 0.5 askdns JMQ_SPF_ALL _SENDERDOMAIN_ TXT /^v=spf1 .*?\+all/ describe JMQ_SPF_ALL SPF set to +all! score JMQ_SPF_ALL 0.5 endif remove those ?'s: /^v=spf1 .*\?all/ and /^v=spf1 .*\+all/ -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows found: (R)emove, (E)rase, (D)elete
Re: repeated sa-update problems
On 9/20/2018 8:59 AM, Dave Jones wrote: I will have to check later if someone else can't check today. I am at a customer location where I don't have good VPN connection out and will be traveling this evening. I can check tomorrow if it can wait. On 20.09.18 09:05, Kevin A. McGrail wrote: It can wait. Matus also had the issue hitting my mirror and I know I don't use a CDN. I looked at update times and they are different each day - debian script sleeps random number of seconds (up to one hour) in order to lower the impact at mirror servers. I have removed the "--fail" option from curl and will look at error message if there's any. I'll keep you updated and will fill bugreport if I'm able to find out anything useful. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
Re: repeated sa-update problems
On Wed, 5 Sep 2018 10:08:24 +0200 Matus UHLAR - fantomas wrote: I (imho too often) get problems when running sa-update (Debian 8, SA 3.4.0) found at /usr/bin/curl Sep 5 07:38:31.810 [16137] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1840016.tar.gz -z 1840016.tar.gz -- http://sa-update.secnap.net/1840016.tar.gz Sep 5 07:38:33.211 [16137] dbg: http: process [16166], exit status: 0 ... Sep 5 07:38:33.799 [16137] dbg: sha1: verification wanted: ea88487c6e9cd48fb3e546606eac2effe4a3a91c Sep 5 07:38:33.799 [16137] dbg: sha1: verification result: 953efe8f531a5a87f6d2d5a65b78b05e55599abc channel: SHA1 verification failed, channel failed ... or should I dig into that deeper to find out what happens? On 06.09.18 18:27, RW wrote: I'd start by editing sa-update and removing the --fail argument to curl. On 9/20/2018 7:18 AM, Matus UHLAR - fantomas wrote: according to curl documentation, it would only cause to output error message instead of the file itself. Is this what you advise me to achieve? ...according to my reading of sa-update, the sa-update selects mirror and only fetches from it. possibility of fetching from multiple mirrors would help here. On 20.09.18 07:58, Kevin A. McGrail wrote: Dave, is secnap one of the mirrors using a CDN? unfortunately, secnap is not the only mirror with which the problem occurs. Sep 20 07:25:34.589 [30532] dbg: http: url: http://www.sa-update.pccc.com/1841300.tar.gz.asc Sep 20 07:25:34.589 [30532] dbg: http: downloading to: /var/lib/spamassassin/3.004000/updates_spamassassin_org/1841300.tar.gz.asc, new Sep 20 07:25:34.589 [30532] dbg: util: executable for curl was found at /usr/bin/curl Sep 20 07:25:34.589 [30532] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1841300.tar.gz.asc -- http://www.sa-update.pccc.com/1841300.tar.gz.asc Sep 20 07:25:34.823 [30532] dbg: http: process [30569], exit status: 0 Sep 20 07:25:34.824 [30532] dbg: sha1: verification wanted: 37342f104bce02b8ede7a769b3c23ddc0b02dc3d Sep 20 07:25:34.824 [30532] dbg: sha1: verification result: 7a65e9db86acc2901a90be825d1efc0990f8c020 channel: SHA1 verification failed, channel failed it also is not the only one roblem that occurs, the "channel: could not find working mirror, channel failed" happens quite often too. I have tried to remove the "--fail" option so maybe the output will help me a bit. Otoh, my questions still apply: - is this possible problem with mirrors? - when do mirrors update? - do mirrors updates propagate atomically? perhaps the sa-update could be changed to try with more mirrors? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "To Boot or not to Boot, that's the question." [WD1270 Caviar]
Re: repeated sa-update problems
On Wed, 5 Sep 2018 10:08:24 +0200 Matus UHLAR - fantomas wrote: I (imho too often) get problems when running sa-update (Debian 8, SA 3.4.0) found at /usr/bin/curl Sep 5 07:38:31.810 [16137] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1840016.tar.gz -z 1840016.tar.gz -- http://sa-update.secnap.net/1840016.tar.gz Sep 5 07:38:33.211 [16137] dbg: http: process [16166], exit status: 0 ... Sep 5 07:38:33.799 [16137] dbg: sha1: verification wanted: ea88487c6e9cd48fb3e546606eac2effe4a3a91c Sep 5 07:38:33.799 [16137] dbg: sha1: verification result: 953efe8f531a5a87f6d2d5a65b78b05e55599abc channel: SHA1 verification failed, channel failed ... or should I dig into that deeper to find out what happens? On 06.09.18 18:27, RW wrote: I'd start by editing sa-update and removing the --fail argument to curl. according to curl documentation, it would only cause to output error message instead of the file itself. Is this what you advise me to achieve? ...according to my reading of sa-update, the sa-update selects mirror and only fetches from it. possibility of fetching from multiple mirrors would help here. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Microsoft dick is soft to do no harm
Re: Hints needed for spf rule
On Tue, Sep 18, 2018 at 12:16 PM Giovanni Bechis wrote: I noticed that Google servers started blocking emails with "suspicious spf records" like for example: "v=spf1 include:musvc.com include:turbo-smtp.com mx a +all". Any idea on how to write a rule to catch something like that ? On 18.09.18 13:01, Kevin A. McGrail wrote: It's in KAM.cf, I believe: # SPF THAT DOESN'T REALLY CARE IF EMAIL IS A FORGERY ifplugin Mail::SpamAssassin::Plugin::AskDNS askdns JMQ_SPF_NEUTRAL_ALL _SENDERDOMAIN_ TXT /^v=spf1 .+\?all$/ describe JMQ_SPF_NEUTRAL_ALL SPF set to ?all! scoreJMQ_SPF_NEUTRAL_ALL 0.5 endif do you not check for "+all" by a reason? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Posli tento mail 100 svojim znamim - nech vidia aky si idiot Send this email to 100 your friends - let them see what an idiot you are
Re: Fwd: Spam Tagging Issue - V3.4.1 with Postfix 3.1.0
On 09.09.18 17:51, thatvolvonut wrote: Good catch - I guess I didn't look over that file close enough. I removed the flag and ran a `systemctl restart spamass-milter` and my subjects are getting tagged now! However, SpamAssassin is still failing to add the custom 'Score' header I've specified in my config. h, it does add X-Spam-Score: here... If I'm understanding it correctly, spamd is launched and run from the spamassassin service in systemd and loads the config from, among other places, /etc/spamassassin/local.cf. corect. In addition, it parses user_prefs of user that calle spamc, or user specified by "-u" spamc option. When the spamass-milter is triggered by Postfix, it connects to that running daemon to have it analyze the message. spamass-milter also provider spamd with recipient user when provided the "-u defaultuser" option. (defaultuser is used when there's multiple recipients) you have this option in detault file for spamass-milter. you can add "-x" parameter to spamass-milter so the users are parsed through aliases and virtusertable. Is that correct, or do I have something off? If it is, I don't understand why the daemonized instance of spamassassin would only selectively read the local.cf file (ignoring one of my add_header lines), or how to tell if that is in fact the file it's reading in the first place. dont't you have option to remove headers in your user_prefs? ubuntu@mail:~$ cat /etc/spamassassin/local.cf bayes_path /var/lib/spamassassin/.spamassassin/bayes bayes_file_mode 0777 ok_languages en de ok_locales en de_ch razor_config /var/lib/spamassassin/.razor/razor-agent.conf pyzor_options --homedir /var/lib/spamassassin/.pyzor add_header all Score _SCORE_/_REQD_ add_header all Report _REPORT_ required_score 8.0 blacklist_to f.restaura...@thelittlefish.net report_safe 0 rewrite_header subject [SPAM - _SCORE_/_REQD_] lock_method flock ifplugin Mail::SpamAssassin::Plugin::Shortcircuit endif (comments and newlines removed, formatting adjusted for easier reading) ubuntu@mail:~$ cat /etc/default/spamassassin SAHOME="/var/lib/spamassassin" SAGLOBALCFGPATH="/etc/spamassassin" OPTIONS="-x --max-children 5 --helper-home-dir /var/lib/spamassassin -u spamd -g spamd --siteconfigpath /etc/spamassassin --socketpath=/var/spool/postfix/spamassassin/spamd.sock --socketowner=spamd --socketgroup=spamd --socketmode=0660" PIDFILE="/var/run/spamd.pid" CRON=1 (comments and newlines removed) ubuntu@mail:~$ cat /etc/default/spamass-milter OPTIONS="-u spamass-milter -i 127.0.0.1 -m -I -- --socket=/var/spool/postfix/spamassassin/spamd.sock" -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I feel like I'm diagonally parked in a parallel universe.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On Fri, 31 Aug 2018, John Hardin wrote: None of the masscheck corpora that hit __HDR_ORDER_FTSDMC also hit ALL_TRUSTED (or at least the portion is so small it falls off the bottom of the report) so I don't feel too worried about adding either !ALL_TRUSTED or __ANY_EXTERNAL (or potentially both) as exclusions. I'm adding __ANY_EXTERNAL now... Comments solicited. On Fri, 31 Aug 2018 16:16:43 -0700 (PDT) John Hardin wrote: Here's one: should __ANY_EXTERNAL be added to any other rules that primarily look for abused MSFT-isms? For example, MIMEOLE_DIRECT_TO_MX, DOS_OE_TO_MX, DOS_OUTLOOK_TO_MX, XPRIO_SHORT_SUBJ, ...? On Sat, 1 Sep 2018, RW wrote: All but the last one is a direct-to-mx rule, which requires one external relay, so adding __ANY_EXTERNAL to those is pointless. On 01.09.18 18:57, John Hardin wrote: Ugh, you're right. I didn't reread the rule details before posting that suggestion - sorry, I've been a little distracted by plumbing issues this week. :) __ANY_EXTERNAL on HDR_ORDER_FTSDMCXX_DIRECT is also pointless because it uses __DOS_SINGLE_EXT_RELAY, which is "exactly one external IP present." Same for HDR_ORDER_FTSDMCXX_NORDNS with __RDNS_NONE. Taking __ANY_EXTERNAL back off of those. Same excuse. :) !ALL_TRUSTED will be masscheck-neutral and will help in the situation you describe, so I'll add it; the only failure mode I can see there is if you add an external ESP to your trusted networks and they discard internal and submission details so that they look like a MUA, and then one of their clients sends spam that would otherwise hit the rule. Is an ESP doing that considered "forging headers" sufficiently to *not* earn trust? Or does simply *discarding* headers not cross that line? I believe that discarding Received: headers DOES cross the line. As long as the point of trusted_hosts is to skip checking for them in blacklists BUT checking hosts behind instead, clearing Received: headers causes sabotaging the trust here. removing/not adding those headers in fact means forging them I don't think __ANY_EXTERNAL is a good idea, it should be sufficient that the headers are all trusted Trusted and Internal are different things. I think it's a bad idea to conflate them or treat them as equivalent and interchangeable. I think __ANY_EXTERNAL is still weakly needed. There's a rule for exactly one external IP (__DOS_SINGLE_EXT_RELAY) and there's a rule for multiple external IPs (__DOS_RELAYED_EXT) but there's nothing for "are there *any* external relays?" __DOS_SINGLE_EXT_RELAY || __DOS_RELAYED_EXT would be equivalent but I feel it should be more direct than that for clarity, unless we have performance concerns with another RE vs. a meta, which is unlikely. I believe that for the rules in account, __DOS_SINGLE_EXT_RELAY is just fine - in my case (I am the OP) we are handling mail that came directly from trusted but external hosts. __ANY_EXTERNAL requires that people read this thread and make a questionable change to their networks to take advantage. I don't think so. The documentation seems to be quite clear in what should and what should not be put into trusted_networks and internal_networks. Actually listing in internal_networks IPs considered "internal to the organization" is questionable? If there's some issue with listing public dialup (presumably dynamic) IPs used by members of the organization in internal_networks, not internal_networks, only in trusted_networks. Just as documented in https://wiki.apache.org/spamassassin/DynablockIssues then maybe we need another way to specify "these IPs are considered internal for submission purposes even though they don't authenticate". This could help much, but it also means much work with SA changes. For now, I believe that using (ALL_TRUSTED && __DOS_SINGLE_EXT_RELAY) is just what I need to prevent all rules from firing: HDR_ORDER_FTSDMCXX_001C HDR_ORDER_FTSDMCXX_BAT HDR_ORDER_FTSDMCXX_DIRECT HDR_ORDER_FTSDMCXX_NORDNS ... as long as some mentioned above: DOS_HIGH_BAT_TO_MX DOS_OE_TO_MX DOS_OE_TO_MX_IMAGE DOS_OUTLOOK_TO_MX MIMEOLE_DIRECT_TO_MX Further thinking about trusted_networks and relaying mail, I think that the difference between remote server and local trusted client, when both are listed in trusted_networks, is that mail from remote server contains more than one Received: header, while local trusted client only one. In this case, trusted server removing or not providing any Received: lines would be understood as client, avoiding hitting rules above. OTOH, trusted clients providing fake headers are something that could cause troubles by hitting whitelist -firsttrusted rules. This is the only problem I can see coming from trusting local clients - but since they must be trusted to avoid local blacklist, I see no way to avoid this than cha
repeated sa-update problems
Hello, I (imho too often) get problems when running sa-update (Debian 8, SA 3.4.0) /etc/cron.daily/spamassassin: sa-update failed for unknown reasons in debug mode I found out: Sep 5 07:38:31.810 [16137] dbg: channel: selected mirror http://sa-update.secnap.net Sep 5 07:38:31.810 [16137] dbg: http: url: http://sa-update.secnap.net/1840016.tar.gz Sep 5 07:38:31.810 [16137] dbg: http: downloading to: /var/lib/spamassassin/3.004000/updates_spamassassin_org/1840016.tar.gz, update Sep 5 07:38:31.810 [16137] dbg: util: executable for curl was found at /usr/bin/curl Sep 5 07:38:31.810 [16137] dbg: http: /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1840016.tar.gz -z 1840016.tar.gz -- http://sa-update.secnap.net/1840016.tar.gz Sep 5 07:38:33.211 [16137] dbg: http: process [16166], exit status: 0 ... Sep 5 07:38:33.799 [16137] dbg: sha1: verification wanted: ea88487c6e9cd48fb3e546606eac2effe4a3a91c Sep 5 07:38:33.799 [16137] dbg: sha1: verification result: 953efe8f531a5a87f6d2d5a65b78b05e55599abc channel: SHA1 verification failed, channel failed the resulting file: # ls -lctr --full-time 1840016.tar.gz -rw-r--r-- 1 debian-spamd debian-spamd 1 2018-09-05 07:38:33.205411724 +0200 1840016.tar.gz running the curl manually downloads correct file: # ls -lct --full-time /tmp/1840016.tar.gz -rw-r--r-- 1 root root 283416 2018-09-05 09:10:02.572034800 +0200 /tmp/1840016.tar.gz with correct checksum: # sha1sum 1840016.tar.gz /tmp/1840016.tar.gz 953efe8f531a5a87f6d2d5a65b78b05e55599abc 1840016.tar.gz ea88487c6e9cd48fb3e546606eac2effe4a3a91c /tmp/1840016.tar.gz we are behind fortigate firewall, which shows this relevant line: Sep 5 07:38:34 fgt date=2018-09-05,time=07:38:34,devname=FGT,devid=XXX,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=x.x.x.x,srcport=60665,srcintf="internal",dstip=204.89.241.6,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=48968219,proto=6,action=close,policyid=62,policytype=policy,dstcountry="United States",srccountry="Reserved",trandisp=snat,transip=y.y.y.y,transport=60665,service="HTTP",duration=1,sentbyte=423,rcvdbyte=604,sentpkt=6,rcvdpkt=4,appcat="unscanned",wanin=388,wanout=103,lanin=103,lanout=388 Sep 5 09:10:03 fgt date=2018-09-05,time=09:10:03,devname=FGT,devid=XXX,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=x.x.x.x,srcport=36269,srcintf="internal",dstip=204.89.241.6,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=49153868,proto=6,action=close,policyid=62,policytype=policy,dstcountry="United States",srccountry="Reserved",trandisp=snat,transip=y.y.y.y,transport=36269,service="HTTP",duration=2,sentbyte=6554,rcvdbyte=294053,sentpkt=124,rcvdpkt=199,appcat="unscanned",wanin=283697,wanout=98,lanin=98,lanout=283697 This kind of error happens with different mirrors. Now my questions: - is this possible problem with mirrors? - when do mirrors update? - do mirrors updates propagate atomically? or should I dig into that deeper to find out what happens? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Fucking windows! Bring Bill Gates! (Southpark the movie)
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On Fri, 31 Aug 2018, Matus UHLAR - fantomas wrote: Note that I list internal clients as trusted, not as internal. Maybe this is the problem. Long time ago I learned to configure dynamic IP addresses (dialups) as trusted, but not as internal. On 31.08.18 12:07, John Hardin wrote: Hrm. Not sure which way to go in that case. Dialup IPs (unless statically assigned to a specific user account) are not really a reliable indicator of internal or trusted... Any of that ISP's clients could get that IP and suddenly be able to get preferential treatment by your mail system. In this case, clients are internal, not dialup, but I still think they should not be listed in internal_networks (as I don't trust them not to spoof anything). Trusting to not spoof headers is what the trusted_networks list is for. I agree and this is something I repeatedfly tought of for long time. But so far we had nothing else to avoid catching non-authenticated clients than listing them in *_networks (and I still found trusted_networks more than internal_networks). HDR_ORDER_FTSDMCXX* is the one I'm trying to solve. Well, that's basically a just check for MSFT MUAs, and spam tools that slavishly mimic the headers such clients produce... unfortunately, they catch MUAs as long as those spam tools. We need something to avoid real MUAs until we have better spam tool detection. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. My mind is like a steel trap - rusty and illegal in 37 states.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On Fri, 31 Aug 2018, John Hardin wrote: None of the masscheck corpora that hit __HDR_ORDER_FTSDMC also hit ALL_TRUSTED (or at least the portion is so small it falls off the bottom of the report) so I don't feel too worried about adding either !ALL_TRUSTED or __ANY_EXTERNAL (or potentially both) as exclusions. I'm adding __ANY_EXTERNAL now... Comments solicited. On 31.08.18 16:16, John Hardin wrote: Here's one: should __ANY_EXTERNAL be added to any other rules that primarily look for abused MSFT-isms? For example, MIMEOLE_DIRECT_TO_MX, DOS_OE_TO_MX, DOS_OUTLOOK_TO_MX, XPRIO_SHORT_SUBJ, ...? Now that you pulled this out... Yes, it would also help on some servers I maintain (where HDR_ORDER_FTSDMCXX* caused troubles). The question I still have is, if this is not in contrast with proposed usage of __ANY_EXTERNAL or !ALL_TRUSTED -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. If Barbie is so popular, why do you have to buy her friends?
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On 31 Aug 2018, at 4:53, Matus UHLAR - fantomas wrote: Long time ago I learned to configure dynamic IP addresses (dialups) as trusted, but not as internal. On 31.08.18 09:37, Bill Cole wrote: They probably should be neither. In this case, clients are internal, not dialup, but I still think they should not be listed in internal_networks (as I don't trust them not to spoof anything). If you do not trust them not to spoof anything, they absolutely must not be in trusted_networks. in fact I have to trust them not to spoof at least the from/envelope addresses. historical reasons, at least until something bad happend. btw note that ALL_TRUSTED means that message was originated by trusted host, not relayed by it - any untrusted host will clear this rule. I have tried to remove them off the trusted_networks. The only change was that ALL_TRUSTED is gone, and without it in meta, HDR_ORDER_FTSDMCX* hit. There are also many rules that search untrusted relays for things like generic helo and DNS name. In thic case, setting UP dns could mess things up even more. As I see it, having those local machines in trusted_networks helps me even more and it also makes me think if this isn't one of reasons trusted_networks exist ... It seems to me that you have a technical & management arrangement unsuited to the SpamAssassin trusted_networks/internal_networks/msa_networks logical model. This is quite possible, but even you have noted that you don't know everything about parsing Received headers. My recommendation would NOT be to modify stock rules that are constructed with that logical model as a base assumption, but rather to create your own mitigating rules to handle the fact that you seem to want to always accept mail from certain internal clients which are nameless, untrustworthy, and sources of mail with features that in the world at large mostly correlate to spam. However, I encounter these problems on multiple hosts with the same conditions, and it's quite possible that different people have similar issues, so I am searching for solution that helps me (ans poddibly others) while does not break anything. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Depression is merely anger without enthusiasm.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On Thu, 30 Aug 2018, Matus UHLAR - fantomas wrote: That further causes hitting HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS in cases where client uses the mail client on local network, without SMTP authentication, and without DNS (which may be quite common in some organizations). On 30.08.18 16:57, John Hardin wrote: Are you experiencing this yourself, so that you can do some testing? Yes. If you do have a repro env, can you check whether that internal network is listed as such in the SA config? Would you be willing to do this and report whether it hits on those messages? header ANY_EXTERNAL_RELAY ALL-EXTERNAL =~ /\S/ score ANY_EXTERNAL_RELAY 0.001 I have tested: ANY_EXTERNAL_RELAY appears when the client's IP is in trusted_networks, it does not when it's in internal_networks. Note that I list internal clients as trusted, not as internal. Maybe this is the problem. Long time ago I learned to configure dynamic IP addresses (dialups) as trusted, but not as internal. In this case, clients are internal, not dialup, but I still think they should not be listed in internal_networks (as I don't trust them not to spoof anything). Filtering on "has an external relay" might be preferable to filtering on !ALL_TRUSTED since "trust" doesn't say anything about rDNS or it being a MUA. note that this problem has been reported on spamassassin-users a month ago: http://spamassassin.1065346.n5.nabble.com/Problem-with-new-rules-td152105.html I'd say the problems aren't. That's because the ESP was relaying mail and not reporting *any* details of the internal handoff, so it looked to the recipient like the MSA was a mail client. rDNS wasn't an issue there. rDNS is not the issue on my side too (it is an issue though). HDR_ORDER_FTSDMCXX* is the one I'm trying to solve. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Atheism is a non-prophet organization.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On Thu, 30 Aug 2018 12:16:33 -0400 Bill Cole wrote: I think the fix for all is for everyone to get their internal_networks and trusted_networks configurations correct. On 30.08.18 20:35, RW wrote: Whatever is happening in this particular case (whatever that is), any rule that works on last-external should distinguish between trusted and untrusted. Tests that use __DOS_SINGLE_EXT_RELAY should be looking for a single untrusted and external relay. '&& ! ALL_TRUSTED' is one way of doing this for __DOS_SINGLE_EXT_RELAY, but unfortunately ALL_TRUSTED is a bit fragile because there's a check for unparsable relays in the perl. __DOS_SINGLE_EXT_RELAY would work in my case (client sending direclty to mailserver). But when considering multiple trusted server (client, trusted and internal MTA, my MTA), it would hit again. I will have to think of this again... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On 08/30/2018 10:16 AM, Bill Cole wrote: It's hard to understand this circumstance based on the generic description. It appears that you have a configuration where a relay is in trusted_networks (i.e. you believe what it asserts in Received headers) but it is NOT in internal_networks so it is in the synthetic X-Spam-Relays-External pseudo-header, it is the only element in X-Spam-Relays-External so the message matches__DOS_SINGLE_EXT_RELAY, and it has no rDNS so the message matches __RDNS_NONE. So: why is that nameless machine that you cannot make a named machine NOT in internal_networks? multiple client PCs in the local network. and as client PCs, I don't want to put them into internal_networks. (And if I remember correctly, I should not). On 30 Aug 2018, at 12:40, Grant Taylor wrote: I don't know if this is the OP's case or not, but the following example comes to mind. SA (running on your receiving MTA) receives a message from an MTA (which is itself an MSA) of an external Business-to-Business partner (thus a trusted MTA that is not internal to the recipient's organization) which itself received the message from a client on an RFC 1918 network without reverse DNS. On 30.08.18 15:08, Bill Cole wrote: If that MSA is requiring authentication (as it should) and recording that in the Received header (as it should) then as I understand it, the handoff of the message will not be considered for __RDNS_NONE. Authentication not implemented yet, and telling the network admins they must to implement it now that I have installed spamassassin, is not acceptable. Tuning DNS is of course possible but it requires some time. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I wonder how much deeper the ocean would be without sponges.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On 30.08.18 09:49, Kevin A. McGrail wrote: I feel that you are fighting a bigger battle than one rule in SA. two rules actually ;-) (with two more possible). Without RDNS, you are running afoul of the postmaster rules of virtually every major email player. You will have massive deliverability issues.. Those IP addresses are in internal network with private IP ranges. When connecting to world, their IPs are NAtted to public. even if I fixed the DNS (and I can't since the network is not in my control), HDR_ORDER_FTSDMCXX_DIRECT would still apply. I believe faking DNS is not what you advise to me, although it would "fix" the problem temporarily (but could create another problem should the DNS be created later). That is why I believe that adding ALL_TRUSTED would solve the problem without unnecessary issues for others. Yes, I can do that locally - but by redefining rule I could miss it getting fixes or improved later. And since different people have already reportted this problem in the past, I would like to make the fix possible for all, if viable. On 30.08.18 09:24, Kevin A. McGrail wrote: Here is my response on the ticket: Outlook express ended production in June 2006. I'm not sure how much weight we can give to an email sent with it. On Thu, Aug 30, 2018 at 9:46 AM, Matus UHLAR - fantomas wrote: note that the issue is exactly the same with Windows Live Mail, which, while unsupported, was available until Jan 2017 (and still seems to be used in some organizations). The issue is at HDR_ORDER_FTSDMCXX_NORDNS with __RDNS_NONE. RDNS is an expected technology to setup a working mail server on the internet. as written below, it's not so easy in organizations where mail server is maintained by diferent people than internal network. (and mailserver is in DMZ, while internal DNS servers in internal networ). Fix that and you have nearly 5 point swing on your email as well as likely more negative scoring rules will fire. of course, there is more to fix and of course some of those fixes are better than others. However, I try to follow order: 1. what I can fix on mailserver 2. what other admins can fix in the network 3. what users can fix on their workstations. This is why I came with the ALL_TRUSTED workaround. Your focus on ALL_TRUSTED implies to me this is 100% internal mail. Is that correct? internal and/or outgoing. Do you (or anyone other) find problems when using ALL_TRUSTED? On Thu, Aug 30, 2018 at 9:14 AM, Matus UHLAR - fantomas wrote: the __HDR_ORDER_FTSDMC rule catches mail sent from windows live mail (and outlook express, which, while obsolete, seems to be still used often) That further causes hitting HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS in cases where client uses the mail client on local network, without SMTP authentication, and without DNS (which may be quite common in some organizations). as a workaround, I recommend to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS rules. an example: X-Spam-Status: Yes, score=9.154 required=5.6 tests=[ALL_TRUSTED=-1, DOS_OE_TO_MX=3.086, FSL_HELO_NON_FQDN_1=0.001, HDR_ORDER_FTSDMCXX_DIRECT=1.999, HDR_ORDER_FTSDMCXX_NORDNS=3.5, HTML_MESSAGE=0.001, MIMEOLE_DIRECT_TO_MX=0.293, RDNS_NONE=1.274] autolearn=no autolearn_force=no X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 I have filled out bug 7607, it got rejected immediately: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7607 while I agree that fixing RDNS will help, internal networks DNS is not always easy, especially when maintained by different people and when internal DNS is in LAn, not in DMZ. note that this problem has been reported on spamassassin-users a month ago: http://spamassassin.1065346.n5.nabble.com/Problem-with-new- rules-td152105.html So, to avoid discussions on bugzilla, I prefer asking here: Is it really a problem to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS ? (maybe even HDR_ORDER_FTSDMCXX_001C and HDR_ORDER_FTSDMCXX_BAT, if their score will be more than zero) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I feel like I'm diagonally parked in a parallel universe. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. You have the right to remain silent. Anything you say will be misquoted, then used against you.
Re: __HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
On 30.08.18 09:24, Kevin A. McGrail wrote: Thanks Matus. This is a good place to debate, thank you. Here is my response on the ticket: Outlook express ended production in June 2006. I'm not sure how much weight we can give to an email sent with it. note that the issue is exactly the same with Windows Live Mail, which, while unsupported, was available until Jan 2017 (and still seems to be used in some organizations). The issue is at HDR_ORDER_FTSDMCXX_NORDNS with __RDNS_NONE. RDNS is an expected technology to setup a working mail server on the internet. as written below, it's not so easy in organizations where mail server is maintained by diferent people than internal network. (and mailserver is in DMZ, while internal DNS servers in internal networ). Fix that and you have nearly 5 point swing on your email as well as likely more negative scoring rules will fire. of course, there is more to fix and of course some of those fixes are better than others. However, I try to follow order: 1. what I can fix on mailserver 2. what other admins can fix in the network 3. what users can fix on their workstations. This is why I came with the ALL_TRUSTED workaround. Your focus on ALL_TRUSTED implies to me this is 100% internal mail. Is that correct? internal and/or outgoing. Do you (or anyone other) find problems when using ALL_TRUSTED? On Thu, Aug 30, 2018 at 9:14 AM, Matus UHLAR - fantomas wrote: the __HDR_ORDER_FTSDMC rule catches mail sent from windows live mail (and outlook express, which, while obsolete, seems to be still used often) That further causes hitting HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS in cases where client uses the mail client on local network, without SMTP authentication, and without DNS (which may be quite common in some organizations). as a workaround, I recommend to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS rules. an example: X-Spam-Status: Yes, score=9.154 required=5.6 tests=[ALL_TRUSTED=-1, DOS_OE_TO_MX=3.086, FSL_HELO_NON_FQDN_1=0.001, HDR_ORDER_FTSDMCXX_DIRECT=1.999, HDR_ORDER_FTSDMCXX_NORDNS=3.5, HTML_MESSAGE=0.001, MIMEOLE_DIRECT_TO_MX=0.293, RDNS_NONE=1.274] autolearn=no autolearn_force=no X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 I have filled out bug 7607, it got rejected immediately: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7607 while I agree that fixing RDNS will help, internal networks DNS is not always easy, especially when maintained by different people and when internal DNS is in LAn, not in DMZ. note that this problem has been reported on spamassassin-users a month ago: http://spamassassin.1065346.n5.nabble.com/Problem-with-new- rules-td152105.html So, to avoid discussions on bugzilla, I prefer asking here: Is it really a problem to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS ? (maybe even HDR_ORDER_FTSDMCXX_001C and HDR_ORDER_FTSDMCXX_BAT, if their score will be more than zero) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I feel like I'm diagonally parked in a parallel universe.
__HDR_ORDER_FTSDMCXXXX hitting windows live mail (and outlook express)
Hello, the __HDR_ORDER_FTSDMC rule catches mail sent from windows live mail (and outlook express, which, while obsolete, seems to be still used often) That further causes hitting HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS in cases where client uses the mail client on local network, without SMTP authentication, and without DNS (which may be quite common in some organizations). as a workaround, I recommend to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS rules. an example: X-Spam-Status: Yes, score=9.154 required=5.6 tests=[ALL_TRUSTED=-1, DOS_OE_TO_MX=3.086, FSL_HELO_NON_FQDN_1=0.001, HDR_ORDER_FTSDMCXX_DIRECT=1.999, HDR_ORDER_FTSDMCXX_NORDNS=3.5, HTML_MESSAGE=0.001, MIMEOLE_DIRECT_TO_MX=0.293, RDNS_NONE=1.274] autolearn=no autolearn_force=no X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 I have filled out bug 7607, it got rejected immediately: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7607 while I agree that fixing RDNS will help, internal networks DNS is not always easy, especially when maintained by different people and when internal DNS is in LAn, not in DMZ. note that this problem has been reported on spamassassin-users a month ago: http://spamassassin.1065346.n5.nabble.com/Problem-with-new-rules-td152105.html So, to avoid discussions on bugzilla, I prefer asking here: Is it really a problem to add && !ALL_TRUSTED to HDR_ORDER_FTSDMCXX_DIRECT and HDR_ORDER_FTSDMCXX_NORDNS ? (maybe even HDR_ORDER_FTSDMCXX_001C and HDR_ORDER_FTSDMCXX_BAT, if their score will be more than zero) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 99 percent of lawyers give the rest a bad name.
Re: Bayes overtraining
>On 08/08/2018 15:04, Matus UHLAR - fantomas wrote: >>...of last 40 mail in my spambox, 14 matches MAILING_LIST_MULTI >>...of last 100 mail in spambox, 27 matches MAILING_LIST_MULTI On 09.08.18 08:54, Daniele Duca wrote: >I practically zeroed MAILING_LIST_MULTI the day it came in the >ruleset. On 09.08.18 23:52, RW wrote: MAILING_LIST_MULTI has the default "nice" score of -1.0 rather than an explicit score. I'm wondering if this is deliberate. I would guess so. ... and so I had to enlarge (-1 => -0.1) the score on another host. seems more and more mailing lists are being abused (or deliberately used) to spread spam. >>but not possible to put: >> >>tflags BAYES_99 learn dothefuckingautolearn >Personally I'll never trust BAYES_* with autolearn_force. I saw some >FPs sometimes and I fear that autolearning would quickly lead to >poisoning I would advise against using auto-training where it's possible to train manually. It's not just a matter of mistraining, autolearning may also bias the database in favour of types of spam that are easily caught, thereby diluting the frequencies of tokens needed to catch the difficult spam. the same applies about ham, however with autolearn_force yes, it could apparently lead to poisoning. However, if "learn" only did its job (whatever it is) and only "noautolearn" would ignore the score, it would be just enough. Currently, as docs say, "learn" in fact implicates "noautolearn". As does userconf. So, both "learn" and "userconf" explicitly implicate "noautolearn"? I wonder why we have them at all. And what is I just don't understand why. Simply use both flags and that's it. If you really must do this just create a new rule without tflags and then score it something like this: 3.0 3.0 0.001 0.001 i.e so it's scored in the non-Bayes score sets. You can just modify the scores and tflags of an original rule, but that's less flexible. I have just listed all rules with negative scores, and surprise, I haven't found any realiable rule with negative score. (MAILING_LIST_MULTI added manually as it doesn't have score set explicitly) It seems that I will need to whitelist and use the hack you have proposed above. - unreliable rules ALL_TRUSTED -1.000 ENCRYPTED_MESSAGE -1.000 -1.000 -1.000 -1.000 ENV_AND_HDR_SPF_MATCH -0.5 DKIM_VALID -0.1 DKIM_VALID_AU -0.1 DKIM_VALID_EF -0.1 HASHCASH_20 -0.5 HASHCASH_21 -0.7 HASHCASH_22 -1.0 HASHCASH_23 -2.0 HASHCASH_24 -3.0 HASHCASH_25 -4.0 HASHCASH_HIGH -5.0 MAILING_LIST_MULTI -1.000 - not used for autolearning BAYES_00 0 0 -1.5 -1.9 BAYES_05 0 0 -0.3 -0.5 - not available everywhere DCC_REPUT_00_12 0 -0.8 0 -0.4 DCC_REPUT_13_19 0 -0.1 0 -0.1 - DNS whitelists RCVD_IN_DNSWL_HI 0 -5 0 -5 RCVD_IN_DNSWL_LOW 0 -0.7 0 -0.7 RCVD_IN_DNSWL_MED 0 -2.3 0 -2.3 RCVD_IN_IADB_DK 0 -0.223 0 -0.095 # n=0 n=1 n=2 RCVD_IN_IADB_DOPTIN 0 -4 0 -4 RCVD_IN_IADB_LISTED 0 -0.380 0 -0.001 # n=0 n=2 RCVD_IN_IADB_MI_CPR_MAT 0 -0.332 0 -0.000 # n=0 n=1 n=2 RCVD_IN_IADB_ML_DOPTIN 0 -6 0 -6 RCVD_IN_IADB_OPTIN 0 -2.057 0 -1.470 # n=0 n=1 n=2 RCVD_IN_IADB_OPTIN_GT50 0 -1.208 0 -0.007 # n=0 n=2 RCVD_IN_IADB_RDNS 0 -0.167 0 -0.235 # n=0 n=1 n=2 RCVD_IN_IADB_VOUCHED 0 -2.2 0 -2.2 RCVD_IN_RP_CERTIFIED 0.0 -3.0 0.0 -3.0 RCVD_IN_RP_SAFE 0.0 -2.0 0.0 -2.0 DKIMDOMAIN_IN_DWL 0 -3.5 0 -3.5 - local whitelists: HEADER_HOST_IN_WHITELIST -100.0 SUBJECT_IN_WHITELIST -100 URI_HOST_IN_WHITELIST -100.0 USER_IN_ALL_SPAM_TO -100.000 USER_IN_DEF_DKIM_WL -7.500 USER_IN_DEF_SPF_WL -7.500 USER_IN_DEF_WHITELIST -15.000 USER_IN_DKIM_WHITELIST -100.000 USER_IN_MORE_SPAM_TO -20.000 USER_IN_SPF_WHITELIST -100.000 USER_IN_WHITELIST -100.000 USER_IN_WHITELIST_TO -6.000 -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. LSD will make your ECS screen display 16.7 million colors
Re: Update to Ubuntu 18.04.1 seems to have partially broken SA
On 17.08.18 11:34, Chris wrote: Aug 17 09:01:43 localhost spamd[1837]: rules: failed to run CLAMAV test, skipping: Aug 17 09:01:43 localhost spamd[1837]: (Can't locate object method "check_clamav" via package "Mail: [...]:SpamAssassin::PerMsgStatus" at (eval 1894) line 19. Aug 17 09:01:43 localhost spamd[1837]: ) Aug 17 09:01:43 localhost spamd[1837]: rules: failed to run __F_DM1 test, skipping: Aug 17 09:01:43 localhost spamd[1837]: (Can't locate object method "from_domains_mismatch" via package "Mail: [...]:SpamAssassin::PerMsgStatus" at (eval 1899) line 19. Any suggestions on a fix? Installed info below: apt-cache policy spamassassin unless ubuntu's spamassassin includes vlamav module, this is not a problem of spamassassin but the clamav plugin -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe.
Re: Bayes overtraining
On 08/08/2018 15:04, Matus UHLAR - fantomas wrote: ...of last 40 mail in my spambox, 14 matches MAILING_LIST_MULTI ...of last 100 mail in spambox, 27 matches MAILING_LIST_MULTI On 09.08.18 08:54, Daniele Duca wrote: I practically zeroed MAILING_LIST_MULTI the day it came in the ruleset. I mean, since there's tflag "noautolearn" designed for this, the flag "learn" should not be ignored. It's easy to put: tflags BAYES_99 learn noautolearn but not possible to put: tflags BAYES_99 learn dothefuckingautolearn Wouldn't tflags BAYES_99 autolearn_force do what you want? Or did I misunderstood completely what you meant? Personally I'll never trust BAYES_* with autolearn_force. I saw some FPs sometimes and I fear that autolearning would quickly lead to poisoning with autolearn_force yes, it could apparently lead to poisoning. However, if "learn" only did its job (whatever it is) and only "noautolearn" would ignore the score, it would be just enough. Currently, as docs say, "learn" in fact implicates "noautolearn". I just don't understand why. Simply use both flags and that's it. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Spam is for losers who can't get business any other way.
Re: Bayes overtraining
>On Wed, 25 Jul 2018 19:49:04 +0200 >Daniele Duca wrote: >> In my current SA setup I use bayes_auto_learn along with some >> custom poison pills (autolearn_force on some rules) , and I'm >> currently wondering if over training SA's bayes could lead to the >> same "prejudice" problem as CRM114. >> >> I'm thinking that maybe it would be better to use >> "bayes_auto_learn_on_error 1" On 26.07.18 15:48, RW wrote: >On a busy server using auto-learning it's probably a good idea to set >this just to increase the token retention, and reduce writes into the >database. On Thu, 26 Jul 2018 17:36:19 +0200 Matus UHLAR - fantomas wrote: well, I have a bit different experience. On 26.07.18 21:25, RW wrote: I didn't say auto-training itself, is a good idea. I mean, if I set bayes_auto_learn_on_error 1, the scores that confirm BAYES decision would never be trained, even if the decision was correct. That could result in BAYES scores geting to the wrong direction. I believe, that after I train BAYES enough, autolearn should be able to do the rest of work and collect further tokens especially when BAYES_00 or BAYES_99 is in effect. re-training a few mismatched mails once a time should be better than pushing back to the _00 and _99 because only mails pointing to opposite direction are trained. There are spams hitting negative scoring rules e.g. MAILING_LIST_MULTI, RCVD_IN_RP_*, RCVD_IN_IADB_* and they are constantly trained as ham. You should be able to work around that by adding noautolearn to the tflags. Well, since I tend to trust those rules less and less Especially because in the meantime I personally get many spams via mailing lists I have never subscribed and never seen subscription confirmation. ...of last 40 mail in my spambox, 14 matches MAILING_LIST_MULTI ...of last 100 mail in spambox, 27 matches MAILING_LIST_MULTI I would like to prevent re-training when bayes disagrees with score soming from other rules. I don't know what you mean by 'prevent re-training', but auto-learning is not supposed to happen if Bayes generates 1 point or more in the opposite direction. either this is new to me, or I have already forgot, but I have different feeling about this. Will try to remember and watch. (I often watch what kind of mail was tagged autolearn=ham) I quite wonder why "learn" tflag causes score being ignored. Only the "noautolearn" flag should be used for this so at least BAYES_99 and BAYES_00 could be takein into account when learning. It's to prevent mistraining from running away in a vicious circle. I mean, since there's tflag "noautolearn" designed for this, the flag "learn" should not be ignored. It's easy to put: tflags BAYES_99 learn noautolearn but not possible to put: tflags BAYES_99 learn dothefuckingautolearn -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. The early bird may get the worm, but the second mouse gets the cheese.
Re: Issues with Yahoo/AOL emails and RCVD_NUMERIC_HELO
On Sun, 29 Jul 2018 12:28:08 +0200
Antony Stone wrote:
On Sunday 29 July 2018 at 12:17:07, Sebastian Arcus wrote yet another
email that's guaranteed to fail DMARC with a reject when posted
through a mailing list, and consequently I didn't receive:
> Or maybe I am misunderstanding completely what is going on? I've
> uploaded a set of headers here: https://pastebin.com/KDV1f0wW
Given that the example you've posted is from a machine with a public
IP 82.132.242.82, but thinks it has a private IP 10.7.54.227, I'm not
entirely surprised there is no rDNS set up for the private address.
On 29.07.18 18:33, RW wrote:
This is the header:
Received: from 82.132.242.82 (EHLO [10.7.54.227]) ([82.132.242.82])
by smtp409.mail.ir2.yahoo.com (Oath Hermes SMTP Server) with
ESMTPA ID 84be422cfd662692400891131b957bd8 for
; Mon, 23 Jul 2018
13:59:41 + (UTC)
I'm not completely certain what this received header format is
supposed to represent, but SA parses the first field, 82.132.242.82, as
the EHLO/HELO.
Is this is true, this should be the problem. The ehlo here is clearly
[10.7.54.227] and 82.132.242.82 is the (missing) rdns, seen when you compare
to another header you have posted. That would indicate bug in header parsing
code.
Received: from ip70-189-131-151.lv.lv.cox.net (EHLO [192.168.0.105])
([70.189.131.151]) ...
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: Why is RCVD_IN_BL_SPAMCOP_NET not '-lastexternal'?
On 28.07.18 17:06, RW wrote: >I don't see anything on the site to suggest that it avoids listing >dynamic IP addresses. And here: > > https://www.spamcop.net/fom-serve/cache/357.html > >commenting on listing history it says: > > > "One also has to remember that IP addresses change hands. Many ISPs > assign IP addresses to customers dynamically, so addresses are > changing all the time." On Sat, 28 Jul 2018 18:12:42 +0200 Matus UHLAR - fantomas wrote: and the point is? A-ha. ou put it in subject: Re: Why is RCVD_IN_BL_SPAMCOP_NET not '-lastexternal'? well, the -lastexternal is for dynamic IPS, and spamcop lists spam sources, not (just) dynamic addresses. On 28.07.18 18:13, RW wrote: Most -lastexternal lists are mixed dynamic/static. Deep checks should be, and mostly are, list for exploitable servers or IP addresses under the control of spammers (or very spam friendly ISPs). RCVD_IN_BL_SPAMCOP_NET seems to be an anomaly. spamcop does list IPs that send spam. It does not care whether static or dynamic, mailserver or open proxy. That means, since spamcop lists exploited servers and IP addresses used by spammers, using it in deep header tests is correct. If you want to be 100% sure, you can split RCVD_IN_BL_SPAMCOP_NET into two rules, one for -lastexternal and one for deep header tests. But I don't think it's worth trying. spamcop delists IP 24 hours after last spam from it is received. ISPs providing dynamic IP addresses should better block port 25 to outside and thus only allow authenticated submission. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Linux IS user friendly, it's just selective who its friends are...
Re: Issues with Yahoo/AOL emails and RCVD_NUMERIC_HELO
On Sunday 29 July 2018 at 12:17:07, Sebastian Arcus wrote:
I've been having a number of emails recently from Yahoo and AOL senders
hitting the RCVD_NUMERIC_HELO rule. I'm trying to understand what is
going on:
1. First off, the rule hits on the EHLO line - which means the it is an
authenticated SMTP submission.
On 29/07/18 11:28, Antony Stone wrote:
Er, what?
No, EHLO simply means "Hello, I'm capable of doing ESMTP".
On 29.07.18 12:29, Sebastian Arcus wrote:
Looking again at it - the 82.132.242.82 is registered as O2/Telefonica
wireless broadband. I wonder if this is a 3G/4G connection - which in
UK always has a private IP address - at the mobile phone level. Maybe
that's why the confusion - the MUA on the mobile phone thinks it is
10.7.54.227 (which it is), but the Yahoo server can only see the
public IP 80.132.242.82, which belongs to the O2 gateway. Could that
explain that particular header?
it does.
Received: from 82.132.242.82 (EHLO [10.7.54.227]) ([82.132.242.82])
by smtp409.mail.ir2.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
84be422cfd662692400891131b957bd8
for ;
Mon, 23 Jul 2018 13:59:41 + (UTC)
Looking at /usr/share/perl5/Mail/SpamAssassin/Plugin/RelayEval.pm
I guess it should not match:
my $rcvd = $pms->{relays_untrusted_str};
if ($rcvd) {
my $IP_ADDRESS = IPV4_ADDRESS;
my $IP_PRIVATE = IP_PRIVATE;
local $1;
if ($rcvd =~ /\bhelo=($IP_ADDRESS)(?=[\000-\040,;\[()<>]|\z)/i # Bug 5878
&& $1 !~ /$IP_PRIVATE/) {
return 1;
}
but maybe I read wrong. Which SA version do you have?
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
M$ Win's are shit, do not use it !
Re: Why is RCVD_IN_BL_SPAMCOP_NET not '-lastexternal'?
On 28.07.18 17:06, RW wrote: I don't see anything on the site to suggest that it avoids listing dynamic IP addresses. And here: https://www.spamcop.net/fom-serve/cache/357.html commenting on listing history it says: "One also has to remember that IP addresses change hands. Many ISPs assign IP addresses to customers dynamically, so addresses are changing all the time." and the point is? A-ha. ou put it in subject: Re: Why is RCVD_IN_BL_SPAMCOP_NET not '-lastexternal'? well, the -lastexternal is for dynamic IPS, and spamcop lists spam sources, not (just) dynamic addresses. Therefore it's useful to do deep header scanning for spamcop listings. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. We are but packets in the Internet of life (userfriendly.org)
Re: Bayes overtraining
On Wed, 25 Jul 2018 19:49:04 +0200 Daniele Duca wrote: In my current SA setup I use bayes_auto_learn along with some custom poison pills (autolearn_force on some rules) , and I'm currently wondering if over training SA's bayes could lead to the same "prejudice" problem as CRM114. I'm thinking that maybe it would be better to use "bayes_auto_learn_on_error 1" On 26.07.18 15:48, RW wrote: On a busy server using auto-learning it's probably a good idea to set this just to increase the token retention, and reduce writes into the database. well, I have a bit different experience. There are spams hitting negative scoring rules e.g. MAILING_LIST_MULTI, RCVD_IN_RP_*, RCVD_IN_IADB_* and they are constantly trained as ham. I would like to prevent re-training when bayes disagrees with score soming from other rules. I quite wonder why "learn" tflag causes score being ignored. Only the "noautolearn" flag should be used for this so at least BAYES_99 and BAYES_00 could be takein into account when learning. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. - Have you got anything without Spam in it? - Well, there's Spam egg sausage and Spam, that's not got much Spam in it.
Re: Help with own RBL
On Tuesday, July 24, 2018, 12:07:52 AM GMT+2, David B Funk wrote: >What kind of 'calculations with that IP' ? On 24.07.18 06:40, Pedro David Marco wrote: Thanks Dave... calculations are complex and done with a an external script that reads some files parsing them... "calculations are complex" is not an answer to "what calculations". Maybe you could do those calculations offline and push their results to DNS. Maybe you could create rules or SA plugin instead. Do any kind of complex calculations for a DNS request is useless, especially when you use it locally. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. My mind is like a steel trap - rusty and illegal in 37 states.
Re: Score from command line is different from the one in the webmail
On 15.07.18 07:41, daniel_1...@protonmail.com wrote: X-Spam-Status: No, score=2.621 tagged_above=-999 required=5 tests=[HTML_IMAGE_ONLY_08=1.781, HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, HTML_SHORT_LINK_IMG_1=0.139, MAILING_LIST_MULTI=-1, SPF_PASS=-0.001, URIBL_BLACK=1.7] autolearn=no autolearn_force=no At the end of X-Spam-Status, you can see URIBL_BLACK=1.7 But when I scan the mail from the command line I have a different score of only 0.9 and no URIBL_BLACK match : it's quite common that some blacklist appear/disappear when checking the same mail after some time. Why do I have different scores and how do I get same score on both configurations ? you can't get the same score when the URI is not in blacklist anymore. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool.
Re: Question regarding auto-learning
On 03.07.18 12:17, J Doe wrote: From reading the documentation, it appears that to train the Bayesian filter I require a minimum of 1,000 pieces of ham and 1,000 pieces of spam. no. You need at least 200 hams and spams for bayes to start firing but you can tune it bu setting bayes_min_ham_num and bayes_min_spam_num. note that too few mails trained can result in false positives/negatives. I am currently collecting spam on one of my servers via a spam trap address and slowly reaching that number. I was wondering, though, if I can use auto learning (bayes_auto_learn 1), before training the database ? autolearning does training instead of you. manual training is still faster and more precise. When autolearn fires on messages at the moment, it is correctly detecting ham and spam based on the default ham and spam thresholds: bayes_auto_learn_threshold_nonspam 0.1 bayes_auto_learn_threshold_spam 12.0 Can this be used before training the database or is it more often used to supplement (on an ongoing basis), a database that has already be trained ? those don't contradict each other. you can use manual and automatic learning both. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Chernobyl was an Windows 95 beta test site.
Re: Remove SA tagging when learning as ham
>> > On Mon, 18 Jun 2018 06:13:06 -0600 @lbutlr wrote: >> >> I have a script that runs when a mail is moved out of the Junk >> >> folder to pass the mail through sa-learn --ham, >You can work around the plugin's deficiencies by using autotraining >or doing some additional training, but then the plugin is of limited >relevance. On Tue, 19 Jun 2018 10:41:51 +0200 Matus UHLAR - fantomas wrote: Of course, both autotraining AND the fixing errors are required to work properly. On 19.06.18 22:27, RW wrote: Then you have worst of both worlds. I'm not saying the plugin is completely useless for Bayes, but 'not completely useless' is not much of a recommendation. I'd say the best, or nearly the best: - autolearning works - user can correct mistakes. one downside is that users will corerct only in case of score mismatch, not bayes mismatch (so, even BAYES_999 won't be reported when not causing FP). do you know of better way than manual reviewing all BAYES scores for all mail? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. My mind is like a steel trap - rusty and illegal in 37 states.
Re: Remove SA tagging when learning as ham
> On Mon, 18 Jun 2018 06:13:06 -0600 @lbutlr wrote: >> I have a script that runs when a mail is moved out of the Junk >> folder to pass the mail through sa-learn --ham, I think this is what the dovecot's Antispam plugin does: https://wiki2.dovecot.org/Plugins/Antispam and maybe ImapSieve: https://wiki2.dovecot.org/HowTo/AntispamWithSieve On 18 Jun 2018, at 08:47, RW wrote: > Whether this is the Dovecot plugin or something local it's a poor > way of training Bayes. You're training on SA errors not Bayes > errors. Most imperfect Bayes results don't translate into > misclassifications. still better than nothing. And it helps us solve the main problem - misclassifications. On Mon, 18 Jun 2018 10:13:04 -0600 @lbutlr wrote: I’m not sure what you’re trying too say here/ Certainly SA does misclassify mail as spam at times, ... Training the messages as ham is useful. On 18.06.18 22:58, RW wrote: The problem is that, unless there is something badly wrong, a typical single user account wont generate enough FPs and FNs for a properly trained database. I found that Bayes's identification of ham improved until I'd trained about 1500 ham, but I wouldn't expect to get anything like 1500 SpamAssassin FPs in a lifetime. It's not even proper train-on-error because it's training on SpamAssassin misclassifications and not correcting Bayes's own errors. It allows Bayes to go uncorrected until it results in an FP or FN. Of course, training BAYES_999 as spam and BAYES_00 as ham won't help change their score, but still can push possible BAYES_20 to BAYES_00 and BAYES_99 to BAYES_999. You can work around the plugin's deficiencies by using autotraining or doing some additional training, but then the plugin is of limited relevance. Of course, both autotraining AND the fixing errors are required to work properly. Unfortunately I have seen spam repeatedly trained as ham, because of some negative scoring rules and too high autolearn threshold. Same can happen in opposite way. having way to fix those manually helps users. IMO the plugin is best left to statistical filters like DSPAM. isn't dspam dead? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I intend to live forever - so far so good.
Re: Question regarding trusted_networks
Matus UHLAR - fantomas skrev den 2018-06-16 16:37: not external networks. only external mail servers you trust not to forge e-mail headers. They may send spam but are not the spam sources. On 16.06.18 19:06, Benny Pedersen wrote: not correct spamassassin need to know all wan ips your own servers use, it does not need to protect forgin senders ips or even trustness of forgin see the docs: A trusted host could conceivably relay spam, but will not originate it, and will not forge header data. DNS blacklist checks will never query for hosts on these networks. adding client IPs means you trust them not to forge mail headers, which is a bad thing for clients. Infected clients WILL send mail with forged headers. spf is a better forgin protector SPF is checked on internal_networks boundary. for SPF check to work properly, you MUST configure your internal_networks properly - SPF is checked where message enters your network, primary or backup servers. For SPF check to be done on proper IP, all your servers in your mail routing should be in internal_networks and nothing more. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 99 percent of lawyers give the rest a bad name.
Re: Question regarding trusted_networks
On 16.06.18 10:12, David Jones wrote: That is basically the same thing worded a little differently. If you have an internal mail relay and your SA server has a private IP on it, then that will be an RFC 1918 IP or range in your internal_networks. Matus UHLAR - fantomas skrev den 2018-06-16 18:07: the differences it that RFC1918 networks should NOT be listed in internal_networks - only mail servers should be listed, no clients. On 16.06.18 18:29, Benny Pedersen wrote: there is no point in have non routeble ips tested by rbls you misunderstand what internal_networks does. it's trusted_networs that prevent IPs from being tested in RBLs, not internal_networks. thats why 127.0.0.1 is forced (RFC 1700) this is a completely different case. but adding RFC1918 does not hurt at all the whole point of internal_networks is to know, where your network ends. Adding client networks into internal_networks means that internal networks boundary is enhanced past that host. That further means, not only the IPs are not checked for *lists (RFC1918 are never checked), but the Received: headers are further checked. you should not trust Received: headers that come from your clients (they may be forged). pleasee understand what it does See quote from the wiki: https://wiki.apache.org/spamassassin/TrustPath set 'internal_networks' to include the hosts that act as MX for your domains, or that may deliver mail internally in your organisation. set 'trusted_networks' to include the same hosts and networks as 'internal_networks', with the addition of some hosts that are external to your organisation which you trust to not be under the control of spammers. For example, very high-volume mail relays at other ISPs, or mailing list servers. Note that it doesn't matter if the server relays spam to you from other hosts; that still means you trust the server not to originate spam, which is what 'trusted_networks' specifies. and further: Why should trusted_networks and internal_networks ever be different? A mail relay that you want to trust in trusted_networks may itself trust its own internal dynamic IP networks. You may trust them not to be a spam source but putting them into your internal_networks list would create a false positive because then those dynamic IPs would be searched for in the DUL lists. This is an example where the two lists need to be different. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. On the other hand, you have different fingers.
Re: MISSING_SUBJECT
On Tue, 12 Jun 2018, micah anderson wrote: > I had a message marked with: > > 2.3 EMPTY_MESSAGE Message appears to have no textual parts and no > Subject: > > It did not have a subject, but it did have content (although only > encrypted) On Wed, 13 Jun 2018 16:36:02 -0700 (PDT) John Hardin wrote: It may not be considering an encrypted message part to be a text body part. What was the MIME type of that part? On 16.06.18 21:12, RW wrote: The rule is: meta EMPTY_MESSAGE !__MIME_ATTACHMENT && !__NONEMPTY_BODY where body __NONEMPTY_BODY /\S/ i.e. it's looking for an attachment or body text. It needs to be something like: !__MIME_ATTACHMENT && !__NONEMPTY_BODY && !ENCRYPTED_MESSAGE ENCRYPTED_MESSAGE already exists. meta ENCRYPTED_MESSAGE __CT_ENCRYPTED header __CT_ENCRYPTEDContent-Type =~ /^multipart\/(?:x-)?(?:pgp-)?encrypted|application\/(?:x-)?pkcs7-mime/ __CT_ENCRYPTED is for now better solution, mostly because of someone could disable ENCRYPTED_MESSAGE in case of FPs. score ENCRYPTED_MESSAGE -1.000 -1.000 -1.000 -1.000 Note that this doesn't remove the redundancy of EMPTY_MESSAGE and MISSING_SUBJECT which is the real problem here. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Fighting for peace is like fucking for virginity...
Re: Question regarding trusted_networks
On 06/15/2018 05:44 PM, J Doe wrote: Jun 15 18:39:23.422 [8422] dbg: config: trusted_networks are not configured; it is recommended that you configure trusted_networks manually My question is: — Should I manually set trusted_networks to have the IP address of the host it is running on and ignore the warning from --lint or … — Should I not set trusted_networks and ignore the warning from --debug ? On 16.06.18 06:33, David Jones wrote: internal_networks should be any RFC 1918 networks that your mail server sees plus any public networks that are in your control. On 06/16/2018 09:37 AM, Matus UHLAR - fantomas wrote: no. only servers that deliver mail to you, as your MX servers or other mailservers directly within your organization should be in internal_networks. On 16.06.18 10:12, David Jones wrote: That is basically the same thing worded a little differently. If you have an internal mail relay and your SA server has a private IP on it, then that will be an RFC 1918 IP or range in your internal_networks. the differences it that RFC1918 networks should NOT be listed in internal_networks - only mail servers should be listed, no clients. Mail with all Received headers of IPs within the internal_networks will hit the ALL_TRUSTED rule. ALL_TRUSTED uses trusted_networks, not internal_networks. listing internal and external clients in trusted_networks is fine, but they don't belong to internal_networks. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Saving Private Ryan... Private Ryan exists. Overwrite? (Y/N)
Re: Question regarding trusted_networks
On 06/15/2018 05:44 PM, J Doe wrote: Jun 15 18:39:23.422 [8422] dbg: config: trusted_networks are not configured; it is recommended that you configure trusted_networks manually My question is: — Should I manually set trusted_networks to have the IP address of the host it is running on and ignore the warning from --lint or … — Should I not set trusted_networks and ignore the warning from --debug ? On 16.06.18 06:33, David Jones wrote: internal_networks should be any RFC 1918 networks that your mail server sees plus any public networks that are in your control. no. only servers that deliver mail to you, as your MX servers or other mailservers directly within your organization should be in internal_networks. trusted_networks should be internal_networks plus any external networks that you trust to not send spam -- in other words they are known to have their own outbound mail filtering. This will tell SA to go back one more Received: header to test for "last_external" checks and RBL checks. not external networks. only external mail servers you trust not to forge e-mail headers. They may send spam but are not the spam sources. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. It's now safe to throw off your computer.
Re: MISSING_SUBJECT
On 15.06.18 09:04, Matus UHLAR - fantomas wrote: On Tue, 12 Jun 2018, micah anderson wrote: I had a message marked with: 2.3 EMPTY_MESSAGE Message appears to have no textual parts and no Subject: It did not have a subject, but it did have content (although only encrypted) John Hardin writes: It may not be considering an encrypted message part to be a text body part. What was the MIME type of that part? On 14.06.18 12:17, micah anderson wrote: pgp/mime and wat is an attachment or just the e-mail came with mime type pgp/mime;2~? OK, again: was it an attachment or just the e-mail came with mime-type PGP/mime ? please show us headers of that message (pastebin for example) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "One World. One Web. One Program." - Microsoft promotional advertisement "Ein Volk, ein Reich, ein Fuhrer!" - Adolf Hitler
Re: A question about DCC and learning.
On 15.06.18 16:24, Reio Remma wrote: I'm curious, if I turn on DCC learning, does it learn with both the learn and report options to sa-learn or only report? sa-learn only trains bayes. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. (R)etry, (A)bort, (C)ancer
Re: MISSING_SUBJECT
On Tue, 12 Jun 2018, micah anderson wrote: I had a message marked with: 2.3 EMPTY_MESSAGE Message appears to have no textual parts and no Subject: It did not have a subject, but it did have content (although only encrypted) John Hardin writes: It may not be considering an encrypted message part to be a text body part. What was the MIME type of that part? On 14.06.18 12:17, micah anderson wrote: pgp/mime and wat is an attachment or just the e-mail came with mime type pgp/mime;2~? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. LSD will make your ECS screen display 16.7 million colors
Re: MISSING_SUBJECT
On 12.06.18 19:37, micah anderson wrote: 2.3 EMPTY_MESSAGE Message appears to have no textual parts and no Subject: It did not have a subject, but it did have content (although only encrypted) it also hit: * 1.8 MISSING_SUBJECT Missing Subject: header which makes sense, because the mail did not have one, but have you looked in your Spam folder lately? All spam has a subject, pretty much always an informal survey of my trash heap showed 4 messages out of 400 did not have a Subject, and two of them were repeats. Matus UHLAR - fantomas writes: and what is your point? On 13.06.18 07:55, micah anderson wrote: The point is EMPTY_MESSAGE scores even though it did have content. so, why did you complain about subjects? But I guess the point is that it had no 'text' parts, because the content was only pgp/mime? Most probably yes. spamassassin -D would show us. The MISSING_SUBJECT and EMPTY_MESSAGE are kind of redundant, since they both catch empty mail. meta MISSING_SUBJECT !__HAS_SUBJECT header __HAS_SUBJECTexists:Subject meta EMPTY_MESSAGE !__MIME_ATTACHMENT && !__NONEMPTY_BODY body __NONEMPTY_BODY/\S/ note that body rules check subject too. I can guess that the mail did NOT include an attachment since it was purely PGP-encrypted mail. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "The box said 'Requires Windows 95 or better', so I bought a Macintosh".
Re: MISSING_SUBJECT
On 12.06.18 19:37, micah anderson wrote: 2.3 EMPTY_MESSAGE Message appears to have no textual parts and no Subject: It did not have a subject, but it did have content (although only encrypted) it also hit: * 1.8 MISSING_SUBJECT Missing Subject: header which makes sense, because the mail did not have one, but have you looked in your Spam folder lately? All spam has a subject, pretty much always an informal survey of my trash heap showed 4 messages out of 400 did not have a Subject, and two of them were repeats. and what is your point? MISSING_SUBJECT is here because when message has no Subject:, it is highly probably spam. it's useless to count how many of spams hit the rule. there are many rules who hit only small percentage of spam, but all of them hit most of spam. what is important is: - how much of mails hitting MISSING_SUBJECT is spam - how much of mails hitting MISSING_SUBJECT is ham. if the percentage is very different in there two cases, the rule gets high positive (or negative) score. Some scores are tuned for safety reasons. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: SPF_HELO_FAIL triggers on domain with valid SPF record and HELO settings
On 11.06.18 08:56, Sebastian Arcus wrote: I am running SA 4.0.0-r1823176 on Perl 5.26.2. On a number of domains I administer, outbound mail triggers the SPF_HELO_FAIL rule - but the regular SPF check passes. I am struggling to see why this is happening, as the HELO name is set to the same value as the name of the server/dns name, it has rDNS - and it clearly passes during the regular SPF check - but not the SPF_HELO check. I have re-checked the domain settings at mxtoolbox.com - and there doesn't seem to be any problem. Any ideas please? do users use SMTP authentication? Is that visible in headers? # spamassassin -D 2>&1 < /test.eml | grep -i spf we need to see the Received: header. Jun 11 08:46:30.758 [5534] dbg: spf: checking HELO (helo=mail.sinclair-accounting.co.uk, ip=80.229.84.190) Jun 11 08:46:30.776 [5534] dbg: spf: query for /80.229.84.190/mail.sinclair-accounting.co.uk: result: fail, comment: Please see http://www.openspf.org/Why?s=helo;id=mail.sinclair-accounting.co.uk;ip=80.229.84.190;r=obelisk.open-t.lan, text: Mechanism '-all' matched -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Linux - It's now safe to turn on your computer. Linux - Teraz mozete pocitac bez obav zapnut.
Re: More outlook phish
On 06/10/2018 12:02 PM, Matus UHLAR - fantomas wrote: I believe M$ requires users to be authenticated within the domain before they are allowed to send using your domain. On 10.06.18 16:55, Grant Taylor wrote: Is that authenticating to the MS SMTP server with any recognized account? Or specifically associated with the purported sending domain? it should be the latter of course. Just the same as microsoft exchange does. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. - Have you got anything without Spam in it? - Well, there's Spam egg sausage and Spam, that's not got much Spam in it.
Re: More outlook phish
On 09.06.18 14:28, Alex wrote: On a somewhat related note, I just noticed one of our customers have listed spf.protection.outlook.com in their SPF record: bestwesternnwcc.com.600 IN TXT "v=spf1 include:spf.protection.outlook.com -all" Doesn't this amount to thousands of IP addresses that could conceivably be used to spoof any other domain that's "hosted" using one of those IPs? I believe M$ requires users to be authenticated within the domain before they are allowed to send using your domain. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Posli tento mail 100 svojim znamim - nech vidia aky si idiot Send this email to 100 your friends - let them see what an idiot you are
Re: Problem with sa-update via proxy
On 05.06.18 08:24, Peter Hutchison wrote: I have recently upgraded my mail mta servers from Ubuntu 14.04 to Ubuntu 16.04 but the daily spamassassin cron job is failing to update the database in /usr/lib/spamassassin/3.9004001/update_spamassassin_org folder. I have made sure that the cron job has the proxy env variables set and also updated /etc/wgetrc But it still regularly fails with this error. I have even configured .curlrc file in root profile with proxy settings. /etc/cron.daily/spamassassin: channel: could not find working mirror, channel failed sa-update failed for unknown reasons I can manually update it ok, but not via a cron job. What else do I need to configure to ensure it works every time? manually? the /usr/lib/spamassassin/3.9004001/update_spamassassin_org should belong to user spamassassin, and the /etc/cron.daily/spamassassin should switch to this user as well. do you run manually /etc/cron.daily/spamassassin or sa-update? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "Two words: Windows survives." - Craig Mundie, Microsoft senior strategist "So does syphillis. Good thing we have penicillin." - Matthew Alton
Re: List From and Reply-To
On Thu, May 31, 2018 at 17:39, Antony Stone wrote: PS: I notice you choose to take the opposite approach with your own Reply-To header, deliberately making it more difficult for people to reply to the list :) On 31.05.18 17:00, Rupert Gallagher wrote: I just use the official ios client, where such regulations are not possible. what has Reply-To: in common with regulations? This is an example of default client settings that may put you in trouble, and the usefulness of server-side enforced policy. I see different problem with proposed approach: Removing or changing Reply-To (or other DKIM-signed header) requires removing DKIM signature. That may require changing From: address (if DKIM policy indicates sender signing all mail), which means that your mail is taken, modified and re-sent, "signed" as someone else. If we take your mail as your artwork, this could get us in trouble :-) Servers can automatically do things to keep both owners and clients on the safe side of the law. We shall not make the mistake of ignoring the GDPR: many sites are going down as we speak. I agree that GDPR apparently needs some polishing (or lawyer recommendation) but I don't like doing it this way -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Eagles may soar, but weasels don't get sucked into jet engines.
Re: Garbage string emails
On 31.05.18 13:14, Palvelin Postmaster wrote: What’s the purpose of emails like this? Should I teach them to bayes or possibly avoid teaching them? bayes should not do any harm but I'm afraid it won't help much also I have fed few of those mails to BAYES, now it seems to catch all of them, but no other rules seem to catch it. Begin forwarded message: From: "ywkazjv" Subject: asyiwtw ykfyydh eryuhlk To: <20130527055448.ga19...@pi.ip.fi> Reply-To: "ilvyzyn" ofzyhsh apvevqn uqqotcd odfeqlz yltumfk one line spam - hard to catch, however - domain seems to be common (for the mail I have seen) - one line, 5 words, 7 characters each seems to be common too. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Despite the cost of living, have you noticed how popular it remains?
Re: [Offtopic] List From and Reply-To
On 30.05.18 15:49, Palvelin Postmaster wrote: Why does this list apparently use the original From header of the poster’s message and doesn't set a Reply-To header at all? because it's the standard behaviour. Hitting reply sends the response to poster directly get a mail client that supports mailing lists. Mozilla should do. This mailing lists sets headers required for list handling: List-Post: <mailto:users@spamassassin.apache.org> note that 1. there are cases when you want to reply personally 2. Reply-To: is supposed to be set by sending user, not someone in between. and DMARC failures occur when posting to list. where you did get this feeling? Those would happen if the list changed the original (or any DKIM-digned) header, or set envelope sender to the original poster. Neither does happen. At least not unless someone configures outgoing MTA to DKIM-sign headers that may change on the way (e.g. Received:) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 42.7 percent of all statistics are made up on the spot.
Re: rewrite_header Subject and Bayes
On 30.05.18 15:12, Palvelin Postmaster wrote: I prepend my spam emails’ subject fields with a specific string to indicate spam, like many do, I presume. Will that string get noticed by bayes and if so, should I do something to prevent it? On 30 May 2018, at 15:21, Matus UHLAR - fantomas wrote: most probably, yes. However, not by your bayes, unless you check for spamminess, tag and check again... On 30.05.18 15:41, Palvelin Postmaster wrote: Yes, forgot to mention I store tagged spam messages and run sa-learn on them to teach spam/ham. it's better to keep spam sign in X-Spam-* headers, which are ignored by spamassassin. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. On the other hand, you have different fingers.
Re: rewrite_header Subject and Bayes
On 30.05.18 15:12, Palvelin Postmaster wrote: I prepend my spam emails’ subject fields with a specific string to indicate spam, like many do, I presume. Will that string get noticed by bayes and if so, should I do something to prevent it? most probably, yes. However, not by your bayes, unless you check for spamminess, tag and check again... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Save the whales. Collect the whole set.
spamcannibal DNSBL issue
Hello, it seems that spamcannibal blacklist is dead, or at least its DNS has expired: Domain Name: SPAMCANNIBAL.ORG Updated Date: 2018-05-30T03:16:26Z Name Server: NS1.RENEWYOURNAME.NET Name Server: NS2.RENEWYOURNAME.NET and, of course: 114.95.168.62.bl.spamcannibal.org. 86385 IN A 91.195.240.117 not mentioning where does its web page redirect... -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Windows 2000: 640 MB ought to be enough for anybody
Re: training bayes database
Am 09.05.2018 um 16:28 schrieb Matthew Broadhead: i guess my dns is set to use my isp's dns server. do i need to set up dns relay on my machine so it comes from my ip? there is no way we send more than 500k emails from our domain so i should qualify for the free lookup? On 09/05/18 20:43, David Jones wrote: Yes. Setup BIND, unbound, or pdns_recursor on your SA server that is not forwarding to another DNS server then set your /etc/resolv.conf or SA dns_server to 127.0.0.1. This will make your DNS queries isolated from your IP to stay under their daily limit. Keep in mind that if your SA box is behind NAT that is not dedicated to your server then other DNS queries could get combined with your shared public IP. This is not likely since others are not going to query RBL/URIBL servers but it's possible. If your SA server is directly on the Internet as an edge mail gateway then this won't be a problem. On 10.05.18 15:02, Reio Remma wrote: On a slightly related note. We're running a PFSense firewall with DNS Forwarder (dnsmasq) in front of our mail server. From what I've gleaned from the net is that it caches as well. Should I still install a local (BIND) on the mail server? The requirement is not for caching server - it's for recursing server dnsmasq is forwarding server, get rid of if when possible. It's even documented: https://wiki.apache.org/spamassassin/CachingNameserver -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. If Barbie is so popular, why do you have to buy her friends?
Re: training bayes database
Am 09.05.2018 um 16:28 schrieb Matthew Broadhead:
i guess my dns is set to use my isp's dns server. do i need to
set up dns relay on my machine so it comes from my ip?
there is no way we send more than 500k emails from our domain so
i should qualify for the free lookup?
On 09/05/18 20:43, David Jones wrote:
Yes. Setup BIND, unbound, or pdns_recursor on your SA server that
is not forwarding to another DNS server then set your
/etc/resolv.conf or SA dns_server to 127.0.0.1. This will make
your DNS queries isolated from your IP to stay under their daily
limit.
Keep in mind that if your SA box is behind NAT that is not
dedicated to your server then other DNS queries could get combined
with your shared public IP. This is not likely since others are
not going to query RBL/URIBL servers but it's possible. If your SA
server is directly on the Internet as an edge mail gateway then
this won't be a problem.
On 10.05.18 12:15, Matthew Broadhead wrote:
i already had bind handling my dns. i just had to add to /etc/named.conf
allow-query-cache {localhost; any;};
NO!
this way everyone is allowed to use your server as recursive DNS.
only allow "localhost;" it defined all ipv4 and ipv6 address on your system.
It's also better to define allow-recursion instead.
While it means something different, they both have same defaults, but
allow-recursion has more clear meaning.
recursion yes;
not needed by default.
and to /etc/resolv.conf
nameserver 127.0.0.1
i cannot believe that is not the default. i always assumed my dns
was working correctly.
It's not default to have DNS server on your system. And it's not default to
have localhost in resolv.conf - it may be authoritative-only.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Eagles may soar, but weasels don't get sucked into jet engines.
Re: rejection w/o sender (or recipient) knowing == dropping
On 29.04.18 20:02, L A Walsh wrote: Stop thinking that silently rejecting an email isn't the same as dropping. I have never said anything about silent rejection. SMTP message "550 5.7.1 Spam Refused" is NOT a silent rejection - it is a VERBOSE rejection by (e.g. my) mail server, and sending (e.g. your) mailserver is supposed to construct DSN to the sender (e.g. you). If your mailserver does not construct the bounce or drops it, it is NOT my fault because my mail server has verbosely refused to take the message. It's even much more likely that this message will get to the sender than if my mail server accepted such mail and sent a bounce, because many bounces are spammy and many spam filters are likely to drop bounces, especially those received from remote servers. If this is what you are angry at, you are whining at wrong side, rejecting mail is correct, not sending or droping bounce is what's wrong and it happens on senders side. Matus UHLAR - fantomas wrote: STOP calling rejection a dropping. Rejecting is NOT dropping. They are two different things. If you try to hand me an envelope, and I will refuse to take it, It is NOT the same as if I took it and dropped to trash. --- That's because I received a rejection. And it's the rejection equivalent to "550 5.7.1 Spam Refused", instead of bounce, which can be compared to another envelope with part of original mail stuck inside. Your rant is completely useless. --- Apparently you don't know what "rejecting" is, vs. silently dropping it into the trash. The latter is dropping. see above. What you are blaming us for, is the proper way to reject e-mail. If some dropping happens, it happens at different stage which outside of receiving server's scope. I will repeat that, if you send mail through your MSP, and the mail gets rejected by remote mail server, it's your MSP's job to properly notify you about the mail being rejected. If it does not, it's your MSP's fault. If your MSP delivered the mail to remote mailserver and the remote server would create a bounce, then it would be your MSP's job to deliver bounce to you. However, your MSP likely drops many of bounces sent to your addresses as result of mail forgeries, which you certainly don't want to receive, and the mentioned bounce may be dropped. If it is dropped, it's your MSP's "fault". I have even encountered complaints about mail sent via remote MSP's (not the one that would receive mail) and then not getting the bounce (because receiving MSP expectedly considered the bounce to be spammy). Simply said, if you want to receive a DSN, you must cooperate with your MSP and avoid sending mail through other MSPs to avoid useless bounces. Rejections are not the problem here. It's just the opposite: accepting mail and then sending a bounce is what causes the loss. And in both cases the loss happens on se3nding MSPs side. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Due to unexpected conditions Windows 2000 will be released in first quarter of year 1901
Re: dropping other's email(s) as a "best practice" for hosted email?
Alan Hodgson wrote: Rejecting the message during receipt causes the sending server to generate a bounce. If it's at all functional. On 27.04.18 09:32, L A Walsh wrote: If a given user wants emails to be dropped at the border -- that would be fine. *I* would not mind configuring a filter that dropped some incoming emails, but if it is going to make the incoming mail server too slow to handle per-user options, it might not be doable. once more: STOP calling rejection a dropping. Rejecting is NOT dropping. They are two different things. If you try to hand me an envelope, and I will refuse to take it, It is NOT the same as if I took it and dropped to trash. The envelope stays in your hands and you are responsible for it. If you drop it later, it's your problem, not mine. You are blaming us for how internet communication works for years. Your rant is completely useless. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 2B|!2B, that's a question!
Re: dropping other's email(s) as a "best practice" for hosted email? (was: "anyone recognize these headers? ...")
On 26.04.18 13:41, L A Walsh wrote: To my way of thinking, dropping someone else's email, telling the sender the email is being rejected for having spam-like characteristics and telling the recipient nothing seems like it might have legal liability for the for the user potentially missing vital email. Refusing to take a mail is not dropping. Noone is required by any means to accept anything because there may be many reasons a mail can't be accepted. For example, mail server that it out of disk space cannot accept a mail thus the only possibility is to refuse accepting it. Dropping mail is the case where mailserver accepts mail and does not deliver it, nor send a bounce. It also would seem to violate what used to be a basic expectation of internet email -- that it is either delivered to the recipient's inbox OR you'll receive a non-delivery notification (a "bounce"). I have no idea where did you get this expectation - your assumption is false. Nearly (if not completely) all mailservers tend to refuse accept mail even from the client, if: - the mail is over allowed size - the sending address is invalid, undeliverable or forged - the mail contains virus, phish, malware or otherwise dangerous content especially in the case the sending address is invalid or undeliverable, it is impossible impossible to send bounce to the sending address. When the address is forged, those bounces would go to a innocent victim. There are many reasons why mailserver (even your submission server) could refuse message. If your submission server accepts a message, it of course SHOULD send a bounce when the recipient's server refuses it (exemptions named above) Note that in this case the recipients server refuses to handle a message, and instead of bouncing, sending the bounce is up to your submission server. I hope some of those who think it was a good practice to delete a user's email (because they think it is malware) might rethink that practice. I hope you now understand what is the difference between deleting and refusing a mail and won't blame us for way how mail system works (and always worked), just because you have misunderstood (or assumed) it. I didn't realize email was no longer considered unreliable afaik e-mail was NEVER considered reliable, mostly because of reasons mentioned above. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. "The box said 'Requires Windows 95 or better', so I bought a Macintosh".
Re: Anti Phish Rules
On 26.04.18 18:00, Nick Edwards wrote: We've been using a separate product to do this, but it struck me, maybe spamassassin can do this easier (or without having to call yet another binary to run as can over mails) Rules that look at URLs in a html message href and src tags, check the "A" tag to see if there is a URL there, and if they do not match, consider it a phis so apply said phis score to the message. Has anyone done this? module even? On 26/04/2018 18:12, Matus UHLAR - fantomas wrote: the main problem: may non-spam senders do that, see: https://wiki.apache.org/spamassassin/AntiPhishFakeUrlRule and further the discussion in linked bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=4255 On 27.04.18 06:51, Noel Butler wrote: I suspect Nick is still using and referring to mailscanner (which is/was written in perl), it has/had this ability, I (like a good few of the names around here) used it back in the day as well, until it became clear it was abandonware, and did not like certain newer versions of perl causing exits after each scan, mind you, I did dump it for amavisd back around 2008/9/10, that said I liked that function, and rarely noticed any FP's, my memorys hazy, but IIRC, it disarmed the links, rather than take any scoring action... I might be wrong though, like I said, its been along time. I believe that the same arguments (need for hugt whitelist) could apply for mailscanner too. I have noticed discussion about this request/issue many times in this mailing list, still the same conclusions, so I wanted to point out to problems rather than telling the OP "go search list archives". Note that I don't like this kind of mismatches too and I would invite having such plugin in SA I would maybe even avoid initial whitelist to force organizations stop using such mismatched URLs (should be safe with not too high scores). -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. REALITY.SYS corrupted. Press any key to reboot Universe.
Re: Anti Phish Rules
On 26.04.18 18:00, Nick Edwards wrote: We've been using a separate product to do this, but it struck me, maybe spamassassin can do this easier (or without having to call yet another binary to run as can over mails) Rules that look at URLs in a html message href and src tags, check the "A" tag to see if there is a URL there, and if they do not match, consider it a phis so apply said phis score to the message. Has anyone done this? module even? the main problem: may non-spam senders do that, see: https://wiki.apache.org/spamassassin/AntiPhishFakeUrlRule and further the discussion in linked bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=4255 -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
Re: Why emails relayedfrom trusted/internal networks trigger rules?
On 26.04.18 10:04, Palvelin Postmaster wrote: I relay mail from another server to my main mail server. I have set its IP 52.28.104.67 in my spamassassin conf in the internal_networks and trusted_networks. I assumed that would prevent spamassassin from scanning the messages but no. Why does this happen? because you feed the messages to spamassassin. SpamAssassin has no option for ignoring messages, if you really want to avoid spamassassin, then avoid spamassassin and don't feed messages into it. SA options trusted_networks and internal_networks are used for better spam detection - there is still possibility of spam soming from trusted server, SA only believes that the trusted server does not fake real sender (in the Received: header). see https://wiki.apache.org/spamassassin/TrustedRelays X-Spam-Status: Yes, score=6.1 required=5.0 tests=AWL,DKIM_ADSP_NXDOMAIN, HELO_DYNAMIC_IPADDR,NO_DNS_FOR_FROM,RDNS_DYNAMIC,T_RP_MATCHES_RCVD autolearn=disabled version=3.4.1 Received: by palvelin.fi (CommuniGate Pro PIPE 6.2.3) Received: from [52.28.104.67] (HELO ip-172-31-20-213.eu-central-1.compute.internal) by palvelin.fi (CommuniGate Pro SMTP 6.2.3) with ESMTPS id 10108357 for i...@.com; Mon, 23 Apr 2018 06:35:44 +0300 Received: from ip-172-31-26-125.eu-central-1.compute.internal (ip-172-31-26-125.eu-central-1.compute.internal [172.31.26.125]) by ip-172-31-20-213.eu-central-1.compute.internal (Postfix) with ESMTP id ECF2CC0C32 for ; Mon, 23 Apr 2018 06:35:43 +0300 (EEST) Received: by ip-172-31-26-125.eu-central-1.compute.internal (Postfix) id DCA21C000C; Mon, 23 Apr 2018 06:35:43 +0300 (EEST) Received: by ip-172-31-26-125.eu-central-1.compute.internal (Postfix, from userid 0) id D6BF2C010F; Mon, 23 Apr 2018 06:35:43 +0300 (EEST) -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. 10 GOTO 10 : REM (C) Bill Gates 1998, All Rights Reserved!
Re: Spamassassin and spamc do not use same rules
On Apr 25, 2018, at 8:57 AM, Paul R. Ganci wrote: Sorry I should have mentioned that. I was aware of that issue. As you can see spamd is running as root in this case and the spamassassin tests were also done as root. do you hace any per-user rules in ~root/.spamassassin/ ? spamassassin will honor those, spamd will not. On 25.04.18 09:06, Amir Caspi wrote: spamd running as root doesn't run as root; it downgrades itself to "nobody." actually, no. it changes to users as needed. This way multiple users can use spamd with per-user config files. Check to make sure that the TxRep plugin is world-readable. Also check to make sure you don't have any per-user preferences in /root that might be enabling TxRep when SA is manually invoked. not preferences - they are honored by both spamd and spamassassin. otoh userrules are honored by spamd only if "allow_user_rules" is enabled, which may be the error I don't advise per-user rules, I would better advise configure rules globally but enable/disable them only for some users, which can be done in user_prefs. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Microsoft dick is soft to do no harm
