Re: dcc on empty email
Perhaps just check your old notes? :-P https://lists.apache.org/thread/6fspd1my9xjdjbz16zp7dk66vn44xccz On Wed, Apr 10, 2024 at 10:42:35PM -0400, Alex wrote: > Hi, > > I'm noticing DCC is triggering on emails with an empty body. I'd like to > create a hash that matches messages with an empty body and other simple > messages. > > What am I doing wrong? I've tried it with a zero-length file as well as > one > with just a few characters. It looks like I don't understand what the > format of the file should be. > > [root@beast dcc]# /usr/bin/dccproc -QCw whiteclnt < dcc-empty > missing message body; fatal error > > > I have a better understanding of how this works now. It apparently still > requires a valid email to be used, just with an actual empty body if I wanted > to whitelist that portion of it. > > /usr/bin/dccproc -QCw whiteclnt < email-with-empty-body > X-DCC-www.nova53.net-Metrics: [1]beast.example.com 1204; Body=0 Fuz1=0 Fuz2=0 > reported: 0 checksum server > env_From: d41d8cd9 8f00b204 e9800998 ecf8427e > From: 55a5141a 442cf35f 22622946 72511b73 > Message-ID: cebc1e5c 40f54129 90709930 ee918829 > Received: 9afca068 dc711459 c84cadb6 627877e9 > Body: d1b04397 6af3d941 68459a63 a155b202 0 > Fuz1: 3d69b970 c60c2b73 95050fee 7971cae8 0 > Fuz2: d35d922e 637a421c 0da33c04 8498ab36 0 > > although I'm not sure now where these values should be used/stored for dcc? > > I also happened across this link that contains a list of checksums for nearly > empty messages, although I don't know how current or useful it is. > [2]https://www.iecc.com/dcc-testmsg-whitelist.txt > > > > > References: > > [1] http://beast.example.com/ > [2] https://www.iecc.com/dcc-testmsg-whitelist.txt
Re: Spamassassin 4 and ClamAVMultipleScores.
That page had it all wrong from the beginning. Adjusting priority only for subrules but not the metas (yes metas don't use priorities in 4.0 but relative priorities are still adjusted for backwards compatibility, __CLAMAV* end up as priority 0 like everything else). The logical way to handle this is to simply run CLAMAV earlier, ditch all the other priority settings. Fixed the wiki. On Thu, Nov 02, 2023 at 09:05:49PM +, Andrew Hearn wrote: > Hello, > > We're using clam, some extra signatures, and the plugin/config as described on > https://cwiki.apache.org/confluence/display/SPAMASSASSIN/ClamAVMultipleScores > to give different signature families different scores. > > Since moving to v4, I don't think it's working... > > The only rule that is matched now, is the generic CLAMAV_VIRUS rule. > The rules for the various other signatures are no longer matched. > Could this be due to the change in priorities for meta rules, and now these > meta rules are running before they get to see the results from clam? > > I can send my config examples and debug output if that's helpful. > > Thanks!
Re: Welcome/unwelcome list not working correctly.
On Sat, Jul 22, 2023 at 10:13:42AM +0200, Benny Pedersen wrote: > Henrik K via users skrev den 2023-07-22 06:50: > > > > | gvk | unwhitelist_from| > > > grant.kel...@sonic.com | 7421538 | > > > | gvk | whitelist_from | *@sonic.com | 7526210 | > > user_prefs in sql/ldap can not do unwhitelist, it > missing priority field in sql/ldap for this to work, > only thing that is possible as now is to remove the > whitelist not add unwhitelist, would need feature > request for priority field No need for feature request, already exists: user_scores_sql_custom_query Create own SQL that sorts as you want, for example MySQL ORDER BY FIELD(...). Then again why even write unneeded stuff in DB. Fix it in the backend or create triggers etc to cleanup redundant stuff.
Re: Welcome/unwelcome list not working correctly.
On Fri, Jul 21, 2023 at 05:06:07PM -0700, Grant Keller wrote: > > > > select username, value from userpref > > where username = _USERNAME_ or username = '@GLOBAL' > > order by username asc; > I don't think the query result order masters here, from what I could > gather in the spamassassin source, the welcome list is built in 2 > steps: > 1. Create the list using the whitelist_from values. > 2. Remove from that list everything in unwhitelist_from Clearly you didn't read the code well enough. There is no such buffering done. The order matters as configuration commands are evaluated line by line. It is no different from reading a physical file. Command order 1: unwhitelist_from grant.kel...@sonic.com - there is no existing grant.kel...@sonic.com in the list, so nothing can be removed, this command is ready never looked at again whitelist_from grant.kel...@sonic.com - address is added and kept in the list, which is why you see USER_IN_WELCOMELIST Command order 2: whitelist_from grant.kel...@sonic.com - address added to the list unwhitelist_from grant.kel...@sonic.com - address is removed from the list You are seeing the first case happening here. > > As you see the only sort is done with username and otherwise MySQL makes > > _no_ guarantees about returned order. If whitelist_from is returned last, > > it works exactly the same as a physical .cf file would have it last. Why do > > you have an obviously redundant whitelist_from in the table messing up > > things? > It isn't redundant, it is just the simplest case I could use for > troubleshooting. The original case I was having issues with was more > like this: > | gvk | unwhitelist_from| grant.kel...@sonic.com | 7421538 | > | gvk | whitelist_from | *@sonic.com | 7526210 | > Which had the same problem as the values for both being identical, I was > just trying to narrow down the issue. You would have been better off looking at the documentation than code. As it clearly states: "The specified email address has to match exactly (although case-insensitively) the address previously used in a welcomelist_from line, which implies that a wildcard only matches literally the same wildcard (not 'any' address)." Above example will always have whitelist_from *@sonic.com active, thus resulting in USER_IN_WELCOMELIST. To unwhitelist it you need unwhitelist_from *@sonic.com.
Re: Welcome/unwelcome list not working correctly.
On Thu, Jul 20, 2023 at 04:14:05PM -0700, Grant Keller wrote: > Could be me doing something wrong, but I can't get Welcome/unwelcome > lists to work the way I expect them to. We are running Spamassassin > 4.0.0 on Centos 7.9 and have the user configs stored via mysql. Custom > scores, Welcomelist, Blocklist, all seem to be working correctly. But if > I have the following config entries: > > | gvk | unwhitelist_from| grant.kel...@sonic.com | 7421538 | > | gvk | whitelist_from | grant.kel...@sonic.com | 7526210 | > > Still, a message from that address to the gvk user results in the > following rules being hit: > > tests=ALL_TRUSTED,SCC_BODY_SINGLE_WORD,SONIC_BX_A2,SONIC_FRIEND,SPF_HELO_NONE, > T_SCC_BODY_TEXT_LINE,USER_IN_WELCOMELIST > > > I'm out of ideas to try on my side. Is there a way to have spamassasin > or spamc print the config, or perhaps debugging I can enable to try to > track down this problem? This is the default query: select username, value from userpref where username = _USERNAME_ or username = '@GLOBAL' order by username asc; As you see the only sort is done with username and otherwise MySQL makes _no_ guarantees about returned order. If whitelist_from is returned last, it works exactly the same as a physical .cf file would have it last. Why do you have an obviously redundant whitelist_from in the table messing up things?
Re: Help with rule
On Tue, Jun 06, 2023 at 12:12:10AM -0400, Bill Cole wrote: > > Escape the @ with a \ > SA uses Perl, so you need to escape %, @, and $ in regular expressions. Perl regular expressions does not mean it's parsed as Perl code, no need to quote such things on any remotely modern SA version.
Re: authres do not parse sender-id
On Sat, Jun 03, 2023 at 08:03:38PM +0200, Benny Pedersen wrote: > Jun 3 19:51:15.822 [17570] dbg: authres: skipping header, unknown property > for spf/smtp: mfrom Fixed this: https://svn.apache.org/viewvc?view=revision&revision=1910234
Re: 4.0.0 noisier than earlier releases?
On Mon, May 15, 2023 at 08:52:23PM -0500, Robert Nicholson wrote: > I remember writing in the past about what I saw in the debugger when running > SA 3.4.6 > > It seems that 4.0.0 seems even noisier. > > Again this is my programmatically calling SpamAssassin in a perlscript. > > I’ve checked and I didn’t find any other version of NetAddr::IP in the @INC > > Subroutine NetAddr::IP::STORABLE_freeze redefined at > /usr/local/lib64/perl5/NetAddr/IP.pm line 365. Funny that you had the same problem almost 10 years ago? It fixed itself until now? https://lists.archive.carbon60.com/spamassassin/users/184448 Cleaned up the code a bit, likely makes zero difference, but try trunk if you want.. https://svn.apache.org/viewvc?view=revision&revision=1909909 Maybe try something like this to check if something is actually loading IP.pm twice. Dunno what else to check, as it seems hard to reproduce. $ strace -f perl yourscript.pl 2>&1 | egrep 'open.*NetAddr/IP.pm' openat(AT_FDCWD, "/usr/local/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/x86_64-linux/NetAddr/IP.pm", O_RDONLY) = 9
Re: mystery score definition
On Fri, May 12, 2023 at 08:31:19AM -0400, Greg Troxel wrote: > > It might be more common, but it's very surprising to me, because the > manual page documents that () works Let's face it, lot of the stuff in SA including documentation is probably over decade old. And documentation is always the last thing someone cares to fix. :-) I remember atleast one similar bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7721
Re: mystery score definition
On Fri, May 12, 2023 at 07:12:35AM -0400, Greg Troxel wrote: > Henrik K writes: > > > From what I've seen, it's very uncommon to use this format. Why rely on > > some vague previously defined score, which can change at any time? Just set > > a static score you like and fits your system. > > It's not vague; it's the score which is defined by the distributed > rules. > > My intent is to say that I want 1 point more than what the rules say, > and I mean that to float with rule changes. It _is_ vague. It's either an educated static score the developer gave, or a corpus generated score, both which might not reflect your personal mailflow at all. > Perhaps you are arguing that all uses of () are confused and thus we > should lean to removing that facility. I just think it's much more common to create meta that checks if the rule you are interested in hit, and add to scoring that way. Yes I realize by that logic things are vague as well, *shrug*. But if you use a non-common method, it's possible that there are bugs and strangness as we now found out.
Re: mystery score definition
On Thu, May 11, 2023 at 11:21:20AM -0400, Greg Troxel wrote: > > But is it good practice for the main distributed rules to rely on this > default? It feels like a lint/pedantic error to define a rule that is > not T_ or __ and does not have an assigned score. But maybe this is > common and normal. It's common and normal. > That says scores in () are relative to the "already set score". So > technically this is not a failure to follow docs, in that no score is > set. But it seems unhelpful to users not to be able to see > > FOO_RULE1 > > in a report and to decide they like that rule and do > > score FOO_RULE (1) >From what I've seen, it's very uncommon to use this format. Why rely on some vague previously defined score, which can change at any time? Just set a static score you like and fits your system. > So maybe that (n) expression should be ok with the implicit 1. Parser processes config files and lines in order, it's not possible to know in advance if the static rule score referred to would actually be defined at a later stage. It would require lots of logic changes.
Re: parameters: use_pyzor and use_razor2
On Sun, Apr 30, 2023 at 01:19:53PM +0300, Henrik K wrote: > > Looking at the code, not sure if the is_admin attributes are correct in all > the plugins or not, maybe someone can check if they actually work properly > per-user and fix for 4.0.1. Committed fixes for a bunch of modules, now use_dcc/pyzor/razor2 is allowed for users amongst other things. Download new trunk or wait for 4.0.1.
Re: parameters: use_pyzor and use_razor2
On Sun, Apr 30, 2023 at 10:36:07AM +0200, i...@servermx.com wrote: > thanks John Hardin and Henrik K, > The parameters "use_pyzor" and "use_razor2"think are correct - > https://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html > I think > > I would like to keep "use_pyzor" and "use_razor2" parameters working per > user-based. I told you, it does not work this way in 4.0. If you want to disable a rule, then use "score RAZOR2_CHECK 0" etc to disable specific rules per user. Looking at the code, not sure if the is_admin attributes are correct in all the plugins or not, maybe someone can check if they actually work properly per-user and fix for 4.0.1.
Re: parameters: use_pyzor and use_razor2
On Sat, Apr 29, 2023 at 12:42:43PM +0200, i...@servermx.com wrote: > Hello, > > we have installed Spamassassin (debian 11.6) vesion 4.0 from source. > With backend MariaDB 10.5.18-MariaDB-0+deb11u1 - Debian 11. > > Spamassassin is raising these messages > > info: config: not parsing, administrator setting: use_pyzor\t0 > info: config: failed to parse line in (sql config) (line 9): use_pyzor\t0 > info: config: not parsing, administrator setting: use_razor2\t0 > info: config: failed to parse line in (sql config) (line 10): use_razor2\t0 > > Nevertheless, in the execution phase Spamassasin performs RAZOR2 check, > here below an excerpt of out log file. > > at Apr 29 11:11:35 2023 [1768541] info: spamd: result: Y 899 - > DMARC_QUAR,FSL_BULK_SIG,GTUBE,NO_RELAYS,PP_MIME_FAKE_ASCII_TEXT,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,THIS_AD,TXREP,USER_IN_WELCOMELIST,USER_IN_WHITELIST > scantime=10.7,size=29004,use > > We do not get these messages when we use the old version 3.4.6. > > Are these parameters still valid or are they deprecated? or what else? In 4.0 they are administrator (site-wide) setting only, 3.4 incorrectly did not have that restriction. To disable rules per-user, set the rule scores to 0 in sql. If you want to disable the plugins completely, then do not load them at all or use the settings in local.cf (can't remember right now if SQL supports site-wide/admin settings).
Re: Fine-tuning SA URI extraction
On Thu, Apr 27, 2023 at 01:45:58AM +0200, Matija Nalis wrote:
>
> - complex but emulating browser behaviour better:
> Add full handling of relative URIs. i.e. have push_uri() detect all
> relative URIs and convert them to absolute URIs before adding them
> to the list of URIs.
If you would have looked at sub push_uri(), that's what it already does:
sub push_uri {
my $target = target_uri($self->{base_href} || "", $uri);
Sure, some things like this could probably be handled more intelligently
when base is missing..
Re: installing spamassassin plugins on debian
On Fri, Mar 17, 2023 at 12:24:39PM -0400, Michael Grant via users wrote: > On Fri, Mar 17, 2023 at 04:03:03PM +0100, Benny Pedersen wrote: > > Michael Grant via users skrev den 2023-03-17 09:52: > > > > > What do people do to keep things up to date easily? > > > > i just use gentoo, or freebsd, not a precompiled problems (hehe) > > > > but what plugin do you need with spamassassin 4 now ? > > > > are you willing to apt maintain a custom plugin in debian ?, i see no > > problem if you do this :) > > I want to try the ExtractText plugin. > > What if I just install this from CPAN? It installs in > /usr/share/perl5/Mail/SpamAssassin/Plugin/ which looks correct. > > It was also recommended to me maybe use cpan2deb and install that, but > then I'm maintaining my own private debian package which I really did > not want to do. What's wrong with just installing from CPAN in this case? https://metacpan.org/pod/Mail::SpamAssassin::Plugin::ExtractText I guess you didn't notice that you are actually installing SpamAssassin 4.0.0, since that's what you are looking at from CPAN? It's part of the official SA package starting from 4.0.0, not a standalone plugin. With luck the plugin will work even on 3.4 without troubles, but uhh yeah..
Re: installing spamassassin plugins on debian
On Fri, Mar 17, 2023 at 05:34:37AM -0400, Michael Grant via users wrote: > On Fri, Mar 17, 2023 at 11:26:21AM +0200, Henrik K wrote: > > On Fri, Mar 17, 2023 at 04:52:41AM -0400, Michael Grant via users wrote: > > > Is there a recommended way of installing a spamassassin plugin on > > > debian (or ubuntu) such that the plugin gets updated via say apt? I'm > > > guessing no because I don't see many spamassassin plugins when I do an > > > "apt search". > > > > > > Up to now, I have been manually putting things in /etc/spamassassin/ > > > but I feel like there has to be a better way to manage these. > > > > > > What do people do to keep things up to date easily? > > > > There is no automated handling of third party plugins. It's up the > > maintainers to provide or not provide any support. Which usually just means > > monitoring some github repo etc. > > What about CPAN? Do people use that? It seems like there's quite a > few modules in CPAN already. I will admit that if I see a debian > package, I go for that, I rarely if ever install stuff from CPAN but I > could be convinced to use it more if this created some order out of > the chaos. Again, it's up to the plugin developer to publish it in CPAN or not, some can be found there. But it really isn't any different or more safe than wgetting some Plugin.pm file manually from Github. It's not recommended to automate either way, since you could just be downloading some incompatible or worst case, a malicious file. Always audit and test updates manually.
Re: installing spamassassin plugins on debian
On Fri, Mar 17, 2023 at 04:52:41AM -0400, Michael Grant via users wrote: > Is there a recommended way of installing a spamassassin plugin on > debian (or ubuntu) such that the plugin gets updated via say apt? I'm > guessing no because I don't see many spamassassin plugins when I do an > "apt search". > > Up to now, I have been manually putting things in /etc/spamassassin/ > but I feel like there has to be a better way to manage these. > > What do people do to keep things up to date easily? There is no automated handling of third party plugins. It's up the maintainers to provide or not provide any support. Which usually just means monitoring some github repo etc.
Re: spamassassin milter auto ip address update
On Mon, Mar 06, 2023 at 11:04:09PM +0100, Benny Pedersen wrote: > Henrik K skrev den 2023-03-06 17:12: > > > It's clamav-milter doing dumb things. Looking at the source code it > > just > > resolves the address at startup and keeps using the IP indefinitely. > > That's > > not how a software is supposed to work. File a bug report to ClamAV, > > and > > good luck since the milter code seems practically unchanged since 2009. > > :-) > > clamav milter is part of clamav, despite spamass-milter is 3dr party, hmm :) > > if spamassassin did the milter itself it would be lots of more stable imho, > no ? What's so special about milter anyway that SA project would need to make it's own? There already exists 100 times more flexible things for that, like amavisd-milter/amavisd-new and mimedefang. ClamAV can be called from either and even from SA itself, there's very few use cases for clamav-milter as a standalone component.
Re: spamassassin milter auto ip address update
On Mon, Mar 06, 2023 at 11:59:03AM +, Marc wrote: > > > I recently had an issue where mail was temporarily rejected because > clamav-milter/spamass-milter could not connect to clamd/spamd. Clamd/Spamd > are a tasks that can automatically change hosts and thus their ips. A simple > restart of the milter fixes this (resolves the new ip). > > However, it would be nice if something could be added to the milter code > that, if it can't contact spamd, it tries to re-resolve the ip address > automatically. > > ps. as you can deduct from the text I am not a 100% sure which milter caused > this actually. It's clamav-milter doing dumb things. Looking at the source code it just resolves the address at startup and keeps using the IP indefinitely. That's not how a software is supposed to work. File a bug report to ClamAV, and good luck since the milter code seems practically unchanged since 2009. :-)
Re: AuthRes plugin (replay RBL queries one hour later)
On Wed, Mar 01, 2023 at 04:46:27PM +0100, Matus UHLAR - fantomas wrote: > > 1. "header.a=rsa-sha256" and "header.s=hege2" options in > Authentication-Results: for dkim where "a" contains algorithm and "s" the > used selector. > > 2. unknown "arc" Authentication-Results: header > > removing mentioned fields in the first header caused one less error message > and A_DKIM_VERIFIED hit. > > removing second header removed error messages completely Fixed these in trunk..
Re: AuthRes plugin (replay RBL queries one hour later)
On Wed, Mar 01, 2023 at 10:50:02AM +0100, Matus UHLAR - fantomas wrote: > > On Wed, Mar 01, 2023 at 09:56:56AM +0100, Matus UHLAR - fantomas wrote: > > > I have SA 4.0 installed and Mail::SpamAssassin::Plugin::AuthRes available. > > > > > > However, I don't see AuthRes plugin mention in .pre files nor in SA rules. > > On 01.03.23 11:28, Henrik K wrote: > > Because it's experimental and unfinished. > > this is the info I was searching for :-) Apparently any info was removed from UPGRADE too https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6918 > However, so far spamassassin --lint produces: > > Mar 1 10:40:36.659 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:40:36.661 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:40:36.661 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:40:36.662 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:40:36.663 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:40:36.666 [19493] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > > I guess it's missing the headers, when I pasted this your mail with headers: > > Authentication-Results: fantomas.fantomas.sk; dmarc=none (p=none dis=none) > header.from=hege.li > Authentication-Results: fantomas.fantomas.sk; > dkim=pass (2048-bit key; unprotected) header.d=hege.li > header.i=@hege.li header.a=rsa-sha256 header.s=hege2 header.b=B6Wp55NL; > dkim-atps=neutral > Authentication-Results: fantomas.fantomas.sk; spf=pass (sender SPF > authorized) smtp.mailfrom=spamassassin.apache.org > (client-ip=3.227.148.255; helo=mxout1-ec2-va.apache.org; > > envelope-from=users-return-126602-uhlar=fantomas.sk@spamassassin.apache. > org; receiver=) > Authentication-Results: fantomas.fantomas.sk; arc=none > smtp.remote-ip=3.227.148.255 > > I only got two lines of errors: > > Mar 1 10:47:17.688 [19813] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. > Mar 1 10:47:17.689 [19813] warn: Use of uninitialized value $result in > string eq at /usr/share/perl5/Mail/SpamAssassin/Plugin/AuthRes.pm line 302. Bah, I think it was tested as atleast working without errors. I'll have a look..
Re: AuthRes plugin (replay RBL queries one hour later)
On Wed, Mar 01, 2023 at 09:56:56AM +0100, Matus UHLAR - fantomas wrote: > > I have SA 4.0 installed and Mail::SpamAssassin::Plugin::AuthRes available. > > However, I don't see AuthRes plugin mention in .pre files nor in SA rules. Because it's experimental and unfinished. > I will try to load it to see if it works. You also need rules for it to do anything. No plugin uses it's parsing at this time. Try the example rules and report back if it works.. https://spamassassin.apache.org/full/4.0.x/doc/Mail_SpamAssassin_Plugin_AuthRes.html
Re: Strange findings debugging bayes results
On Thu, Feb 16, 2023 at 10:18:50AM +0100, hg user wrote: > I was investigating a bunch of bitcoin spam: different titles, > different senders (all from gmail), different text, different pdf > attachment. > > Unfortunately in those days my bayes db was polluted and they all got > a BAYES_50, 0.8. > > I tested the messages now with a recreated bayes db and got some > BAYES_999. So I dug to understand if I already saw the spam... > > But the debug result was unpleasant: > dbg: bayes: tokenized header: 92 tokens > dbg: bayes: token 'HX-Received:Jan' => 0.998028449502134 > dbg: bayes: token 'HX-Google-DKIM-Signature:20210112' => 0.997244532803181 > dbg: bayes: token 'H*r:sk:' => > 0.997244532803181 > dbg: bayes: token 'H*r:a05' => 0.995425742574258 > dbg: bayes: token 'HAuthentication-Results:sk:.' => > 0.986543689320388 > dbg: bayes: token 'HX-Google-DKIM-Signature:reply-to' => 0.916110175863517 > dbg: bayes: token 'H*r:2002' => 0.877842810325844 > dbg: bayes: token 'HAuthentication-Results:2048-bit' => 0.858520043212023 > dbg: bayes: token 'HAuthentication-Results:pass' => 0.855193895034317 > dbg: bayes: score = 0.97915091326 > > > Every score is based on headers, very generic headers. and some > related to my setup. > > Not a single token from the message body The Bayes implementation has been practically unmaintained for a long time, so YMMV. You can try something like this, most headers are parsed badly and generate biasing random garbage (unscientific observation): bayes_ignore_header ARC-Authentication-Results bayes_ignore_header ARC-Message-Signature bayes_ignore_header ARC-Seal bayes_ignore_header Authentication-Results bayes_ignore_header Autocrypt bayes_ignore_header IronPort-SDR bayes_ignore_header suggested_attachment_session_id bayes_ignore_header X-Brightmail-Tracker bayes_ignore_header X-Exchange-Antispam-Report-CFA-Test bayes_ignore_header X-Forefront-Antispam-Report bayes_ignore_header X-Forefront-Antispam-Report-Untrusted bayes_ignore_header X-Gm-Message-State bayes_ignore_header X-Google-DKIM-Signature bayes_ignore_header x-microsoft-antispam bayes_ignore_header X-Microsoft-Antispam-Message-Info bayes_ignore_header X-Microsoft-Antispam-Message-Info-Original bayes_ignore_header X-Microsoft-Antispam-Untrusted bayes_ignore_header X-Microsoft-Exchange-Diagnostics bayes_ignore_header x-ms-exchange-antispam-messagedata bayes_ignore_header x-ms-exchange-antispam-messagedata-0 bayes_ignore_header x-ms-exchange-crosstenant-id bayes_ignore_header x-ms-exchange-crosstenant-network-message-id bayes_ignore_header x-ms-exchange-crosstenant-rms-persistedconsumerorg bayes_ignore_header X-MS-Exchange-CrossTenant-userprincipalname bayes_ignore_header x-ms-exchange-slblob-mailprops bayes_ignore_header x-ms-office365-filtering-correlation-id bayes_ignore_header X-MSFBL bayes_ignore_header X-Provags-ID bayes_ignore_header X-SG-EID bayes_ignore_header X-SG-ID bayes_ignore_header X-UI-Out-Filterresults bayes_ignore_header X-ClientProxiedBy bayes_ignore_header X-MS-Exchange-CrossTenant-FromEntityHeader bayes_ignore_header X-OriginatorOrg bayes_ignore_header X-MS-Exchange-CrossTenant-OriginalArrivalTime bayes_ignore_header X-MS-TrafficTypeDiagnostic bayes_ignore_header X-MS-Exchange-CrossTenant-AuthAs bayes_ignore_header X-MS-Exchange-Transport-CrossTenantHeadersStamped bayes_ignore_header X-MS-Exchange-CrossTenant-AuthSource
Re: DMARC plugin issue
On Fri, Jan 13, 2023 at 02:42:51PM -0300, entregabili...@donweb.com wrote: > Hello, I would like to ask about the following error that we have: > > plugin: eval failed: Can't locate object method "rule_ready" via package > "Mail::SpamAssassin::PerMsgStatus" at > /usr/local/share/perl5/Mail/SpamAssassin/ > Plugin/DMARC.pm line 135. Are you sure it's the DMARC.pm that can came with SA 4.0.0 release, not some old different version?
Re: Rule Help - not sure what is wrong with my syntax
There's no need for any rules:
whitelist_to us...@example.com
whitelist_to *@domain.com
And adjust USER_IN_WHITELIST_TO for score.
(welcomelist_to / USER_IN_WELCOMELIST_TO in 4.0)
On Wed, Jan 11, 2023 at 04:56:21PM -0800, Loren Wilton wrote:
> ?
> Why not do a simple rule rather than inventing some Perl code?
>
> header TO_SPECIFIC_EMAIL To:addr ~= '(?:\b[1]us...@example.com|\b[2]
> us...@example.com|\b[3]us...@example.com)'
> describe TO_SPECIFIC_EMAIL Mail to a specific email address
> score TO_SPECIFIC_EMAIL -2
>
> header TO_SPECIFIC_DOMAIN To:addr '(?:'\@example1\.com | \@example2\.com | \
> @example3\.com)'
> describe TO_SPECIFIC_DOMAIN Mail to specific email domain
> score TO_SPECIFIC_DOMAIN -2
>
> or possibly
>
> header TO_SPECIFIC_DOMAIN To:addr '\@(?:example1\.com | example2\.com |
> example3\.com)$'
>
>
> Loren
>
> - Original Message -
> From: [4]Joey J
> To: [5]users@spamassassin.apache.org
> Sent: Wednesday, January 11, 2023 3:39 PM
> Subject: Rule Help - not sure what is wrong with my syntax
>
> Hello All,
>
> I created this rule to check for email addresses matching a list to get
> added some negative value.
> I also tried it with just domains so it would be more efficient, but I
> can't seem to get them to run.
> Any suggestions?
>
> header TO_SPECIFIC_EMAIL eval:check_to_specific_email()
> describe TO_SPECIFIC_EMAIL Mail to a specific email address
>
> score TO_SPECIFIC_EMAIL -2
>
> sub check_to_specific_email {
> my ($self) = @_;
> my $to = lc($self->get('To:addr'));
> my $list_of_address = qr/[6]us...@example.com|[7]us...@example.com|[8]
> us...@example.com/;
> if ($to =~ $list_of_address) {
> return 1;
> }
> return 0;
> }
>
>
>
>
> This version was to simply check for the domain matches, but can't seem to
> get it to work
>
>
> header TO_SPECIFIC_DOMAIN eval:check_to_specific_domain()
> describe TO_SPECIFIC_DOMAIN Mail to specific email domain
>
> score TO_SPECIFIC_DOMAIN -2
>
> sub check_to_specific_domain {
> my ($self) = @_;
> my $to = lc($self->get('To:addr'));
> if ($to =~ /\@example1\.com$|\@example2\.com$|\@example3\.com$/) {
> return 1;
> }
> return 0;
> }
>
>
>
>
>
>
> --
> Thanks!
> Joey
>
>
>
> References:
>
> [1] mailto:bus...@example.com
> [2] mailto:bus...@example.com
> [3] mailto:bus...@example.com
> [4] mailto:jacklistm...@gmail.com
> [5] mailto:users@spamassassin.apache.org
> [6] mailto:us...@example.com
> [7] mailto:us...@example.com
> [8] http://us...@example.com/
Re: spamhaus abuse free usage rules
On Thu, Jan 12, 2023 at 04:01:02AM +0100, Benny Pedersen wrote: > > my changes does nothing to datafeed users, but it > makes big diffrenses to free usage Makes zero difference how the rules are called, SA never sends duplicate physical queries, they are cached and reused.
Re: perldoc Mail::SpamAssassin::GeoDB and MaxMind wooes
On Mon, Jan 09, 2023 at 07:16:00PM +0100, Benny Pedersen wrote: > > > If you need _ASNCIDR_ tag you have to switch to dns queries. > > is this still not possible with GeoIPLite2-ASN ? Broken record? There is no GeoIP API to query IP and get CIDR. The database is mapping of IP to AS string, nothing more. How is the CIDR useful anyway? Useless info which you can query manually later if needed.
Re: excluding specific RBL checks
On Sun, Jan 08, 2023 at 04:23:11PM -0500, Charles Sprickman wrote: > What did you end up with? > > I have a bunch of zero rules for these yet still keep getting the > "administrative notice" from sbl/zen. > > The fact that those guys don't just send out a "yes, this is on by default in > spamassassin, here is copy pasta to turn us off" email bugs me. > > I've grown to this huge list and still get the warnings. > > # remove spamhaus tests, they want us to pay > # need to include the first base rule or DNS still triggers but is ignored > score __RCVD_IN_ZEN 0 > score RCVD_IN_SBL 0 > score RCVD_IN_XBL 0 > score RCVD_IN_PBL 0 > score URIBL_SBL 0 > score URIBL_CSS 0 > score URIBL_SBL_A 0 > score URIBL_CSS_A 0 > score URIBL_DBL_SPAM 0 > score URIBL_DBL_PHISH 0 > score URIBL_DBL_MALWARE 0 > score URIBL_DBL_BOTNETCC 0 > score URIBL_DBL_ABUSE_SPAM 0 > score URIBL_DBL_ABUSE_REDIR 0 > score URIBL_DBL_ABUSE_PHISH 0 > score URIBL_DBL_ABUSE_MALW 0 > score URIBL_DBL_ABUSE_BOTCC 0 > > Until I can get around to updating I'm considering just nuking the actual > tests from the ruleset. Much easier and reliable way: dns_query_restriction deny spamhaus.org
Re: 4.0.0 dnsbl_subtests.t test failures
On Wed, Dec 28, 2022 at 09:30:30AM -0500, Michael Orlitzky wrote: > On Wed, 2022-12-28 at 16:20 +0200, Henrik K wrote: > > > > Common sense would ask that how is SPF harmful for the user? One would > > think it would be actually desirable like any other network lookups, that > > user might have accidentally left disabled? But sure, if this is the Gentoo > > way, so be it. I had enough of 90's linux flashbacks trying it for the > > first and last time today. :-) > > > > Well, SPF wasn't nearly as reliable in 2005 as it is now, and it pulls > in an extra dependency. > > Probably the best answer is that by having this ability, Gentoo > attracts the sort of user who likes to disable such things to save disk > space, shave off a few CPU cycles, or improve security. And then > there's a feedback loop wherein most of our users want to retain the > ability to control what gets installed/enabled. Doesn't look too good for Gentoo packaging though, if since 2009 v310.pre and newer have been full of all sorts of plugins loaded. It's like nobody actually cared since most of the stuff is useful. :-)
Re: DQS rules for SA 4.0.0+
And it is even mentioned in the UPGRADE notes: - The HashBL plugin in 342.pre is now enabled by default. (sad typo in the filename) On Wed, Dec 28, 2022 at 04:21:45PM +0200, Henrik K wrote: > > This was discussed and approved in some of the 4.0.0 bugs. There should be > no need to revisit it. It still wouldn't make sense to have loadplugin > HashBL in two *.pre files. > > On Wed, Dec 28, 2022 at 09:18:51AM -0500, Kevin A. McGrail wrote: > > Wow, as it's enabled in v342.pre, that would imply it was enabled in 3.4.2. > > We should not have changed a past pre file for the 4.0.0 release IMO but > > added it to the 4.0.0.pre file. Such is life. Should we fix it for 4.0.1? > > > > On 12/28/2022 9:07 AM, Henrik K wrote: > > > Just keep in mind that HashBL is only enabled for fresh 4.0.0 installs, it > > > wasn't default previously. > > > > -- > > Kevin A. McGrail > > kmcgr...@apache.org > > > > Member, Apache Software Foundation > > Chair Emeritus Apache SpamAssassin Project > > https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: DQS rules for SA 4.0.0+
This was discussed and approved in some of the 4.0.0 bugs. There should be no need to revisit it. It still wouldn't make sense to have loadplugin HashBL in two *.pre files. On Wed, Dec 28, 2022 at 09:18:51AM -0500, Kevin A. McGrail wrote: > Wow, as it's enabled in v342.pre, that would imply it was enabled in 3.4.2. > We should not have changed a past pre file for the 4.0.0 release IMO but > added it to the 4.0.0.pre file. Such is life. Should we fix it for 4.0.1? > > On 12/28/2022 9:07 AM, Henrik K wrote: > > Just keep in mind that HashBL is only enabled for fresh 4.0.0 installs, it > > wasn't default previously. > > -- > Kevin A. McGrail > kmcgr...@apache.org > > Member, Apache Software Foundation > Chair Emeritus Apache SpamAssassin Project > https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: 4.0.0 dnsbl_subtests.t test failures
On Wed, Dec 28, 2022 at 09:10:13AM -0500, Michael Orlitzky wrote: > > Without disabling the plugin, how would that work? If the user happens > to install Mail::SPF as a dependency of something else and if the > plugin is *not* disabled, spamassassin will (surprise!) start using SPF > against the user's wishes. Common sense would ask that how is SPF harmful for the user? One would think it would be actually desirable like any other network lookups, that user might have accidentally left disabled? But sure, if this is the Gentoo way, so be it. I had enough of 90's linux flashbacks trying it for the first and last time today. :-)
Re: DQS rules for SA 4.0.0+
On Wed, Dec 28, 2022 at 04:06:01PM +0200, Henrik K wrote: > On Wed, Dec 28, 2022 at 01:58:55PM +, Riccardo Alfieri wrote: > > On 28/12/22 14:44, Henrik K wrote: > > > > > It is enabled by default for new installs in v342.pre (old users must > > > enable > > > it manually). But like with any other loadable plugin, one MUST check use > > > "ifplugin" to check that it's loaded. > > Ok, thanks for the clarification. > > > > Would you then suggest to add also a: > > > > ifplugin Mail::SpamAssassin::Plugin::URIDNSBL > > > > to the .cf files where check_rbl , urirhssub etc are used? > > It would be standard to use it yes. > > Of course it's a bit of a double-edged sword, since with ifplugin the rules > might silently be ignored. Especially for Gentoo users. ;-) Maybe would be even good idea to use something like this: ifplugin Mail::SpamAssassin::Plugin::HashBL else error: Please activate HashBL plugin in v342.pre endif Which would result in: $ spamassassin --lint Dec 28 16:12:54.518 [764158] warn: config: failed to parse line in /var/foo/sh.cf (line 4): error: Please activate HashBL plugin in v342.pre Dec 28 16:12:55.415 [764158] warn: lint: 1 issues detected, please rerun with debug enabled for more information Of course this wouldn't be good for sa-updated rules. Would need some way to generate a warning without failing lint.
Re: DQS rules for SA 4.0.0+
On Wed, Dec 28, 2022 at 09:04:09AM -0500, Kevin A. McGrail wrote: > > However, both URIDNSBL and HashBL are enabled by default from checking the > source code. Just keep in mind that HashBL is only enabled for fresh 4.0.0 installs, it wasn't default previously.
Re: DQS rules for SA 4.0.0+
On Wed, Dec 28, 2022 at 01:58:55PM +, Riccardo Alfieri wrote: > On 28/12/22 14:44, Henrik K wrote: > > > It is enabled by default for new installs in v342.pre (old users must enable > > it manually). But like with any other loadable plugin, one MUST check use > > "ifplugin" to check that it's loaded. > Ok, thanks for the clarification. > > Would you then suggest to add also a: > > ifplugin Mail::SpamAssassin::Plugin::URIDNSBL > > to the .cf files where check_rbl , urirhssub etc are used? It would be standard to use it yes. Of course it's a bit of a double-edged sword, since with ifplugin the rules might silently be ignored. Especially for Gentoo users. ;-)
Re: DQS rules for SA 4.0.0+
On Wed, Dec 28, 2022 at 01:35:22PM +, Riccardo Alfieri wrote: > On 28/12/22 14:20, Kevin A. McGrail wrote: > > > Do you have hashbl plugin enabled? > > > > > Ah, I thought it was enabled by default in SA 4.0. It is enabled by default for new installs in v342.pre (old users must enable it manually). But like with any other loadable plugin, one MUST check use "ifplugin" to check that it's loaded.
Re: 4.0.0 dnsbl_subtests.t test failures
On Wed, Dec 28, 2022 at 02:29:03PM +0100, Benny Pedersen wrote: > > i will like to see default all plugins disabled, and a install howto enabled > needed plugin as needed, there is not anypoint on enabled all, and all it > gets is dns refused . > > or some *_BLCOKED like apache infra cant solve Disabling default plugins solves nothing, just creates a worse experience for user. Educating and guiding users to use DNS properly does not require this.
Re: 4.0.0 dnsbl_subtests.t test failures
On Wed, Dec 28, 2022 at 08:20:04AM -0500, Philippe Chaintreuil via users wrote: > > So there's desire that if a user doesn't want Mail::SPF installed, and > SpamAssassin doesn't REQUIRE it (which it doesn't), it shouldn't be force > installed. But for SpamAssassin to work as installed, that plugin can't be > enabled by default. Even if Mail::SPF is not installed, it doesn't prevents you from loading the SPF plugin, it just automatically disables itself. This should be the behavior for all the other default plugins too. > I went back through Gentoo's history and it's been that way since 2005. Common theme with SA. :-D > There used to be a post-install warning that you'd need to choose which > plugins you wanted enabled, but it got stripped out at some point. There is > a section in the wiki that indicates you should go through the .pre files to > enable/disable plugins. > https://wiki.gentoo.org/wiki/SpamAssassin#Configuration If default plugins aren't default as provided by SA for the rest of the world, this should by explicitly implied in the wiki. Someone could simply come from other standard installs and assume URIDNSBL etc are loaded, as it would make no sense for it to be disabled by default. The default plugin set is intended for scanning to work well out of the box, assuming of course that user installed all required Perl modules.
Re: 4.0.0 dnsbl_subtests.t test failures
I believe Philippe is the package maintainer, so it's up to him I guess. :-) On Wed, Dec 28, 2022 at 06:35:07AM -0500, Kevin A. McGrail wrote: > +1 and over and above by Henrik to install the distro for testing. > > Our project cannot be responsible for the decisions of the distribution > package > maintainers. This is definitely one that is not the right decision. > > Do we have a contact at Gentoo? > > Regards, KAM > > On Wed, Dec 28, 2022, 04:38 Henrik K <[1]h...@hege.li> wrote: > > On Mon, Dec 26, 2022 at 01:57:20PM -0500, Philippe Chaintreuil via users > wrote: > > On 12/25/2022 4:38 PM, Sidney Markowitz wrote: > > > I can get exactly that set of error messages by commenting out the > > > loadplugin for URIDNSBL in rules/init.pre or deleting the file > > > rules/init.pre completely, and running make test with the default > > > setting of run_net_tests=n in t/config.dist. If I change it to > > > run_net_tests=y then the test t/uribl.t also fails where it tries to > use > > > check_uridnsbl > > > > Gentoo disables all plugins in init.pre so users have to choose which > > plugins to use and do any required configuration after install. > > As mentioned on bug: > [2]https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8095 > > It's completely baffling why and how Gentoo does this. It only disables > loadplugins in init.pre, and not other *.pre files. This is already a bug > in itself by not being consistent. > > I even installed Gentoo to check what the fuss is about. There is no > mention from installer or wiki that users need to go enable plugins in > /etc/mail/spamassassin/*.pre files, which are almost universally used by > default on every other distribution. > > The workaround for Gentoo installer is NOT TO MODIFY SUCH SOURCE FILES > DIRECTLY. If you want to confuse users with non-standard defaults, pretty > sure you can figure out how to modify the files without touching the > originals, which the test system makes use of. :-) > > > > References: > > [1] mailto:h...@hege.li > [2] https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8095
Re: 4.0.0 dnsbl_subtests.t test failures
On Mon, Dec 26, 2022 at 01:57:20PM -0500, Philippe Chaintreuil via users wrote: > On 12/25/2022 4:38 PM, Sidney Markowitz wrote: > > I can get exactly that set of error messages by commenting out the > > loadplugin for URIDNSBL in rules/init.pre or deleting the file > > rules/init.pre completely, and running make test with the default > > setting of run_net_tests=n in t/config.dist. If I change it to > > run_net_tests=y then the test t/uribl.t also fails where it tries to use > > check_uridnsbl > > Gentoo disables all plugins in init.pre so users have to choose which > plugins to use and do any required configuration after install. As mentioned on bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8095 It's completely baffling why and how Gentoo does this. It only disables loadplugins in init.pre, and not other *.pre files. This is already a bug in itself by not being consistent. I even installed Gentoo to check what the fuss is about. There is no mention from installer or wiki that users need to go enable plugins in /etc/mail/spamassassin/*.pre files, which are almost universally used by default on every other distribution. The workaround for Gentoo installer is NOT TO MODIFY SUCH SOURCE FILES DIRECTLY. If you want to confuse users with non-standard defaults, pretty sure you can figure out how to modify the files without touching the originals, which the test system makes use of. :-)
Re: 4.0.0 dnsbl_subtests.t test failures
On Mon, Dec 26, 2022 at 11:54:12AM +0100, Giovanni Bechis wrote: > > dnsbl_subtests.t tests runs even with run_net_tests=n (fixed few minutes > ago in trunk) The fix is not needed. dnsbl_subtests.t starts a _local_ nameserver and never sends queries to public internet. The intention of run_net_tests=n is to prevent test scripts from failing if you don't have a internet connection. This test does not require a working connection.
Re: Whitelist or add negative values for score
On Wed, Dec 21, 2022 at 08:43:18AM +0100, Matus UHLAR - fantomas wrote: > > DKIM_INVALID 0.1 DKIM or DK signature exists, but is not valid > > > > DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not > > necessarily valid > > > > HTML_FONT_LOW_CONTRAST 0.001 HTML font color similar or identical to > > background > > > > HTML_MESSAGE0.001 HTML included in message > > > > KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict > > Alignment > > this rule indicates that mail would NOT pass welcomelist_auth > > If this is the mail you want then yes, you need welcomelist_from_rcvd, but > that's sender's faule. It will pass welcomelist_auth, since there is SPF_PASS, which you missed: SPF_PASS -0.001 SPF: sender matches SPF record
Re: urinsrhsbl not trimmed
On Tue, Dec 06, 2022 at 02:35:44PM +0100, Wolfgang Breyha wrote:
> Hi!
>
> I'm using SA 4 trunk and tried the urinsrhsbl. Not sure if this is a bug or
> abuse on my side.
>
> I already have two RBLs... one
> uribl.example.at
> and one
> uriblfull.example.at
>
> The first does not use "tflags notrim", the second does.
>
> Now I added a lookup for uribl.example.at like
> urinsrhssub __URIBL_ZID_16 uribl.example.at.A 16
> body__URIBL_ZID_16 eval:check_uridnsbl('__URIBL_ZID_16')
> tflags __URIBL_ZID_16 net domains_only
> But if I check the debug output I see checks for the FQDN of all three NS
> RRs instead of one lookup for the trimmed NS domain.
>
> According to the man page of URIBLDNS this should only happen if
> urifullnsrhssub is used?
>
> Why are there two variants urinsrhssub/urifullnsrhssub? And what happens if
> "tflags notrim" is used with them?
>
> Is it even valid to use urinsrhssub on a RBL where urirhssub is used as
> well? I felt quite comfortable to do so since it was the easiest way to add
> support for NS listings with an existing uribl.
I would just create a bug, preferably with tested examples of all variants.
:-) Most than likely there has been some oversights with legacy code and
notrim addition, *nsrhs* isn't even used in stock rules.
Re: Mial hits MISSING rules despite presence of headers
Feel free to reopen the bug if you want, I really have no time or desire to work on these right now. I didn't analyze if skipping do_meta_tests for shortcircuiting has any negative consequences, but if someone wants to prove it doesn't, go for it and I'll vote on it. It not enough to just post a patch that is a "possible fix". On Sun, Dec 04, 2022 at 09:42:59AM -0500, Kevin A. McGrail wrote: > I think that will have to go to discussion since if the rules don't short > circuit the way they used to, other rules outside of the ones we control are > going to act oddly. The one that was reported was with validity for example. > > What happens if I have a local rule that's high scoring and meta that would > have been short circuited prior? In 3.4 I would have expected to stop when I > hit the validity rule, now I continue running and hit another rule that's very > high scoring and end up with a mis classification. > > From what I understand that is the real world scenario of what it's occurring. > > At a minimum we would have to announce this change for people to look at their > short circuit rules. > > What are your thoughts? > > On Sun, Dec 4, 2022, 09:36 Henrik K <[1]h...@hege.li> wrote: > > > Of course it does and processing doesn't need to stop into a brickwall > when > it activates. It simply finishes metas which is not that expensive and > might provide some additional useful hits. No sense postponing 4.0.0 to > try > to tweak this further. > > On Sun, Dec 04, 2022 at 09:28:02AM -0500, Kevin A. McGrail wrote: > > I have not checked but does the short circuiting actually work? The goal > of it > > is to lower the resource usage of the tool. If it continues to run and > generate > > longer than we have a problem still. > > > > On Sun, Dec 4, 2022, 08:50 Henrik K <[1][2]h...@hege.li> wrote: > > > > > > Fixed simply with some rule changes as described in the bug. > > > > > > On Tue, Nov 29, 2022 at 05:28:00PM -0500, Kevin A. McGrail wrote: > > > [2][3]https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 is > now open on > > this > > > issue. > > > -- > > > Kevin A. McGrail > > > Member, Apache Software Foundation > > > Chair Emeritus Apache SpamAssassin Project > > > [3][4]https://www.linkedin.com/in/kmcgrail - 703.798.0171 > > > > > > > > > On Tue, Nov 29, 2022 at 1:11 PM <[4][5]giova...@paclan.it> wrote: > > > > > > On 11/28/22 17:47, Bill Cole wrote: > > > > On 2022-11-28 at 11:03:29 UTC-0500 (Mon, 28 Nov 2022 > 11:03:29 > > -0500) > > > > Alex <[5][6]mysqlstud...@gmail.com> > > > > is rumored to have said: > > > > > > > >> On Mon, Nov 28, 2022 at 10:42 AM Kevin A. McGrail < > > > [6][7]kmcgr...@apache.org> > > > >> wrote: > > > > [...] > > > >>> Also, would be helpful to know if this is different than > 3.4.6's > > > behavior. > > > >>> > > > >> > > > >> Oh yes, I meant to mention that it is different behavior > for > > 3.4.6. Same > > > >> score for the rule, but it appears to actually > shortcircuits > the > > > processing > > > >> of additional rules. At the least, it doesn't add those > MISSING_* > > rules. > > > > > > > > This is almost certainly a side-effect of recent reworking > of > the > > > housekeeping around which rules have been run. > > > > > > > > As a temporary work-around, I think it would be wise to give > any > > rule > > > that gets SHORTCIRCUITed an overwhelming score in whichever > direction > > it > > > operates. > > > > > > > > > > > Confirmed, r1904981 is the commit that is causing this > behavior. > > > Giovanni > > > > > > > > > References: > > > > [1] mailto:[8]h...@hege.li > > [2] [9]https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 > > [3] [10]https://www.linkedin.com/in/kmcgrail > > [4] mailto:[11]giova...@paclan.it > > [5] mailto:[12]mysqlstud...@gmail.com > > [6] mailto:[13]kmcgr...@apache.org > > > References: > > [1] mailto:h...@hege.li > [2] mailto:h...@hege.li > [3] https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 > [4] https://www.linkedin.com/in/kmcgrail > [5] mailto:giova...@paclan.it > [6] mailto:mysqlstud...@gmail.com > [7] mailto:kmcgr...@apache.org > [8] mailto:h...@hege.li > [9] https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 > [10] https://www.linkedin.com/in/kmcgrail > [11] mailto:giova...@paclan.it > [12] mailto:mysqlstud...@gmail.com > [13] mailto:kmcgr...@apache.org
Re: Mial hits MISSING rules despite presence of headers
Of course it does and processing doesn't need to stop into a brickwall when it activates. It simply finishes metas which is not that expensive and might provide some additional useful hits. No sense postponing 4.0.0 to try to tweak this further. On Sun, Dec 04, 2022 at 09:28:02AM -0500, Kevin A. McGrail wrote: > I have not checked but does the short circuiting actually work? The goal of it > is to lower the resource usage of the tool. If it continues to run and > generate > longer than we have a problem still. > > On Sun, Dec 4, 2022, 08:50 Henrik K <[1]h...@hege.li> wrote: > > > Fixed simply with some rule changes as described in the bug. > > > On Tue, Nov 29, 2022 at 05:28:00PM -0500, Kevin A. McGrail wrote: > > [2]https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 is now open > on > this > > issue. > > -- > > Kevin A. McGrail > > Member, Apache Software Foundation > > Chair Emeritus Apache SpamAssassin Project > > [3]https://www.linkedin.com/in/kmcgrail - 703.798.0171 > > > > > > On Tue, Nov 29, 2022 at 1:11 PM <[4]giova...@paclan.it> wrote: > > > > On 11/28/22 17:47, Bill Cole wrote: > > > On 2022-11-28 at 11:03:29 UTC-0500 (Mon, 28 Nov 2022 11:03:29 > -0500) > > > Alex <[5]mysqlstud...@gmail.com> > > > is rumored to have said: > > > > > >> On Mon, Nov 28, 2022 at 10:42 AM Kevin A. McGrail < > > [6]kmcgr...@apache.org> > > >> wrote: > > > [...] > > >>> Also, would be helpful to know if this is different than 3.4.6's > > behavior. > > >>> > > >> > > >> Oh yes, I meant to mention that it is different behavior for > 3.4.6. Same > > >> score for the rule, but it appears to actually shortcircuits the > > processing > > >> of additional rules. At the least, it doesn't add those MISSING_* > rules. > > > > > > This is almost certainly a side-effect of recent reworking of the > > housekeeping around which rules have been run. > > > > > > As a temporary work-around, I think it would be wise to give any > rule > > that gets SHORTCIRCUITed an overwhelming score in whichever > direction > it > > operates. > > > > > > > > Confirmed, r1904981 is the commit that is causing this behavior. > > Giovanni > > > > > References: > > [1] mailto:h...@hege.li > [2] https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 > [3] https://www.linkedin.com/in/kmcgrail > [4] mailto:giova...@paclan.it > [5] mailto:mysqlstud...@gmail.com > [6] mailto:kmcgr...@apache.org
Re: Mial hits MISSING rules despite presence of headers
Fixed simply with some rule changes as described in the bug. On Tue, Nov 29, 2022 at 05:28:00PM -0500, Kevin A. McGrail wrote: > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8078 is now open on this > issue. > -- > Kevin A. McGrail > Member, Apache Software Foundation > Chair Emeritus Apache SpamAssassin Project > https://www.linkedin.com/in/kmcgrail - 703.798.0171 > > > On Tue, Nov 29, 2022 at 1:11 PM wrote: > > On 11/28/22 17:47, Bill Cole wrote: > > On 2022-11-28 at 11:03:29 UTC-0500 (Mon, 28 Nov 2022 11:03:29 -0500) > > Alex > > is rumored to have said: > > > >> On Mon, Nov 28, 2022 at 10:42 AM Kevin A. McGrail < > kmcgr...@apache.org> > >> wrote: > > [...] > >>> Also, would be helpful to know if this is different than 3.4.6's > behavior. > >>> > >> > >> Oh yes, I meant to mention that it is different behavior for 3.4.6. > Same > >> score for the rule, but it appears to actually shortcircuits the > processing > >> of additional rules. At the least, it doesn't add those MISSING_* > rules. > > > > This is almost certainly a side-effect of recent reworking of the > housekeeping around which rules have been run. > > > > As a temporary work-around, I think it would be wise to give any rule > that gets SHORTCIRCUITed an overwhelming score in whichever direction it > operates. > > > > > Confirmed, r1904981 is the commit that is causing this behavior. > Giovanni >
Re: spamassassin sometimes suddenly ends scanning
On Tue, Nov 29, 2022 at 09:40:14AM -0500, Greg Troxel wrote: > > I am pretty sure the hardware is OK, but I can't really run memtest86 as > it is a VPS. Memtester is userland, should be on most distributions: apt install memtester You can give it parameter of how much memory you have free, and it'll allocate and test that.
Re: spamassassin sometimes suddenly ends scanning
On Tue, Nov 29, 2022 at 07:34:51AM -0500, Greg Troxel wrote: > > Wolfgang Breyha writes: > > > It doesn't finish any other rules and doesn't display final results at all. > > > > And then I start it simply again and everything is fine. > > > > Has anybody else seen this odd behavior? > > I see occasional coredumps (as in perl.core). It is often enough to be > annoying (beyond worrisome that it happens at all), but not reproducible > and no apparent pattern. Try memtester/memtest86, atleast if it's not a proper server with ECC memory.. And if you have core dumps, running gdb would be helpful: $ gdb /usr/bin/perl /path/to/core (gdb) backtrace
Re: ToCc Header operations
On Sat, Nov 26, 2022 at 08:31:41AM -0500, Jared Hall wrote: > SA: 3.4.6 > > The Header ToCc test doesn't seem to accept :name and :addr modifiers. > > Is that how this function operates? It should accept it just fine. But in 3.4 :addr or :name can only return results of first email. So it makes no sense to use with ToCc, as it's basically always the same as To.
Re: pyzor and failure to parse response
On Sun, Nov 20, 2022 at 11:58:31AM -0500, Alex wrote: > Hi, > I'm using the latest SA from trunk and trying to get pyzor working. It runs > correctly to check a message from the command-line, but SA apparently fails to > properly parse the output? > > Nov 20 11:55:21.970 [2531521] dbg: pyzor: opening pipe: /usr/bin/pyzor > --homedir /var/spool/amavisd/.pyzor --debug --log-file /var/spool/amavisd > /.pyzor/pyzor.log check
Re: SA4rc3: no URL makes uridnsbl rules "unrun"
On Fri, Oct 14, 2022 at 11:55:35AM +0200, Wolfgang Breyha wrote: > Hi! > > If a scanned E-Mail does not contain any URL (URIHOSTS and URIDOMAINS empty) > SA4(rc3) does not mark rules using check_uridnsbl as "run" IMO. > > This makes meta rules depending on them "unrunable" as well. > > Dbg Output from an example: > > Oct 14 11:51:01.140 [3032346] dbg: check: tagrun - tag URIHOSTS is now > > ready, value: ARY:[] > > Oct 14 11:51:01.140 [3032346] dbg: check: tagrun - tag URIDOMAINS is now > > ready, value: ARY:[] > ... > > Oct 14 11:51:01.215 [3032346] dbg: rules-all: running eval rule URIBL_BLACK > > (check_uridnsbl) > ... > > Oct 14 11:51:47.392 [3032346] dbg: rules-all: unrun dependencies prevented > > meta SPF_PASS_SPAM from running: URIBL_BLACK > > Greetings, Wolfgang Thanks, good catch, I opened a bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8062 These have been a bit backlogged on my end, but should be able to look this weekend or next week..
Re: KAM channel disabling lookups?
On Tue, Oct 11, 2022 at 09:29:18AM +0300, Henrik K wrote: > > KAM channel (https://mcgrail.com/template/kam.cf_channel) users might want > to check their rules.. > > KAM_deadweight2_sub.cf contains this: > > meta __RCVD_IN_SORBS 0 > meta __RCVD_IN_ZEN 0 > meta __RCVD_IN_MSPIKE_B 0 > meta __RCVD_IN_MSPIKE_L 0 > meta __RCVD_IN_DNSWL 0 > > Seems it's been disabling many active and useful DNSBL/WL lookups for a long > time? Ah yeah, now I remember this bug: https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7991 Apparently this isn't a "problem" in 3.4, as the channel can't even override anything from official rules.. so only affects recent 4.0.0/trunk users.
Re: KAM channel disabling lookups?
On Tue, Oct 11, 2022 at 12:33:19PM -0400, Kevin A. McGrail wrote: > > On 10/11/2022 2:29 AM, Henrik K wrote: > > Seems it's been disabling many active and useful DNSBL/WL lookups for a long > > time? > > Correct. We found they had overlap or didn't add to the accuracy of > categorization so disabling rules is a key part of reducing weight of rule > scanning and improving efficiency. This is inherent in the KAM ruleset and > has been there for several years. I know, it makes sense for a few things, but I'm surprised you are disabling many popular DNSBLs. How much efficiency do you expect from dropping a few DNS queries? How often do you re-measure the accuracy from all the dropped rules? Why not just drop RCVD_IN_ZEN etc from official ruleset then..
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 02:54:08PM +0200, Damian wrote:
> > > # __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
> > > meta SA4 __SA4
> > > score SA4 1
> > > describe SA4 dummy
> > yields
> > > SA dbg: rules-all: unrun dependencies prevented meta SA4 from
> > > running: __SA4
>
> The above is slightly misleading, even in SA3 one had to predeclare a
> default via
>
> > meta __SA4 0
> > meta SA4 __SA4
> > score SA4 1
> > describe SA4 dummy
>
> Such __SA4 default solves the "unrun dependencies prevented" issue, but
> still produces hits inconsistently.
Uggh.. I even use amavisd, but didn't notice that rule_hits stuff. This is
why we need and appreciate rc-testers. :-)
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8061
> Sometimes
>
> > SA dbg: check: tests=SA4
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
> sometimes
> > SA dbg: check: tests=
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
>
> As far as I can tell, the inconsistency comes from the unsorted key
> iteration in do_meta_tests() and finish_meta_tests().
The unsorted part doesn't matter, actually it's better this way so we able
to notice the problems. When meta handling for this case is fixed, it will
always work regardless..
Re: RCVD_IN_DNSWL_* "unrun" (was: SA 4.0.0-rc3 meta rule results "undefined")
On Tue, Oct 11, 2022 at 02:36:59PM +0200, Wolfgang Breyha wrote: > On 11/10/2022 13:29, Henrik K wrote: > > You can also need to use -D rules,rules-all to see any "unrun" rules. > > I tried that using "all,rules,rules-all" and I think I found an other > problem with RCVD_IN_DNSWL* rules. > > If I run with a testmail I see: > > # spamassassin -D all,rules,rules-all &1|grep RCVD_IN_DNSWL > > Oct 11 14:32:33.399 [1549685] dbg: rules: meta KHOP_HELO_FCRDNS inherits > > tflag net, depends on __RCVD_IN_DNSWL > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_NONE, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.0$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_LOW, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.1$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_MED, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.2$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_HI, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.3$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_BLOCKED, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.255$) > > Oct 11 14:32:33.627 [1549685] dbg: rules-all: ran meta rule > > __RCVD_IN_DNSWL, no hit > > Oct 11 14:32:34.462 [1549685] dbg: rules-all: unrun dependencies prevented > > meta KAM_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, > > RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W > > Oct 11 14:32:34.464 [1549685] dbg: rules-all: unrun dependencies prevented > > meta KAM_QUITE_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, > > RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta XPRIO from running: RCVD_IN_DNSWL_NONE > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE1 from running: RCVD_IN_DNSWL_MED > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE2 from running: RCVD_IN_DNSWL_LOW > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE4 from running: RCVD_IN_DNSWL_HI > > This looks like __RCVD_IN_DNSWL gets evaluated, but not hit. But > RCVD_IN_DNSWL_* are "unrun", what looks very odd. This is because __RCVD_IN_DNSWL is not supposed to be a meta. KAM channel overrides it to "disable" the rule. I just posted on the list about that..
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > > And I've still cases were a simple ">" is not evaluated as well... > > I have > meta __META1 (__SUBMETAX + . + __SUBMETAXN) > with > dbg: rules: ran meta rule __META1 ==> got hit (5) > > I have > meta __META2 (__SUBMETAY + . + __SUBMETAYN) > with > dbg: rules: ran meta rule __META2 ==> got hit (8) > > I have > meta __META_NO (__SUBMETAZ || . || _SUBMETAZN ) > which is not hit. > > And finally > meta RESULT ((! __META_NO) && __META1 > 1 && __META2 > 6) > score RESULT 3 > > And RESULT is not hit/evaluated. Would need exact sample of ruleset, this is too vague to work on. What are all the __SUBMETAs? You can also need to use -D rules,rules-all to see any "unrun" rules. > There is something really odd going on with meta rules on SA4. It's a massive change as described in Bug 7735, more than likely that some bugs can remain. But IMO it's still worth it for the most robust logic especially working with async network rules and stuff.
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > On 11/10/2022 12:23, Henrik K wrote: > > Should be fixed in rc4. > > > > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 > > Well, this indeed fixes this test case. But my initial problem which I > though is described in this test case is not fixed yet:( > > So __SA4TA1 gets not evaluated neither in this ... > > header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ > > meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) > > meta __SA4TA2 (__SA4TA1 * 3 ) + 1 > > > > IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL > > is not hit it is undefined. Remains unclear if you applied the patch from https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ?
Re: _DKIMDOMAIN_ vs. _AUTHORDOMAIN_ (was: Re: dnswl dwl rule)
On Tue, Oct 11, 2022 at 11:52:17AM +0200, Matus UHLAR - fantomas wrote: > > On Sat, Oct 01, 2022 at 04:42:09PM +0200, Matus UHLAR - fantomas wrote: > > > perhaps these all should replace _DKIMDOMAIN_ by _AUTHORDOMAIN_ and > > > AND-ed > > > with DKIM_VALID_AU. > > > > > > can these checks be made the way DNS queries are done only when > > > DKIM_VALID_AU matches? > > > > > > perhaps playing with priority > > On 07.10.22 16:41, Henrik K wrote: > > It's not possible to use priority with askdns. The rule is launched then > > the all dependent tags are set, nothing more, nothing less. > > I see bug 7735 now and am curious if the cvhange only affects order of rule > calling or calling them at all. It has no relevance on rule order or calling. It just affects when meta rule result will be evaluated. > So, if I make meta rule dependend on other rules: > > meta DOMAIN_IN_DNSWL_DWL (DKIM_VALID_AU && __DOMAIN_IN_DNSWL_DWL) > askdns__DOMAIN_IN_DNSWL_DWL _AUTHORDOMAIN_.dwl.dnswl.org TXT > describe __DOMAIN_IN_DNSWL_DWL author domain is listed in dwl.dnswl.org > > will __DOMAIN_IN_DNSWL_DWL always be called? __DOMAIN_IN_DNSWL_DWL is a standalone askdns rule. It does not know about anything metas or stuff that depends on it, so yes it's always called. Network lookups are generally always launched at the start of the scan (priority -100 to be exact), and results are checked later on when answer arrives. If you are hoping to prevent unnecessary DNS query, it's not possible.
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:21:23PM +0300, Henrik K wrote: > On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > > On 11/10/2022 06:59, Henrik K wrote: > > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > > > And another quite simple ruleset... > > > > meta __SA4TA3_1 6 > > > > meta __SA4TA3_2 2 > > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > > > Works fine here. > > > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > > > Tried again with > > meta __SA4TA3_1 6 > > meta __SA4TA3_2 2 > > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > score SA4TA3 0.1 > > > > I see > > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > > > But no line für SA4TA3 and no report or final score. > > Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. > Will investigate. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > On 11/10/2022 06:59, Henrik K wrote: > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > And another quite simple ruleset... > > > meta __SA4TA3_1 6 > > > meta __SA4TA3_2 2 > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > Works fine here. > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > Tried again with > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > score SA4TA3 0.1 > > I see > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > But no line für SA4TA3 and no report or final score. Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. Will investigate.
KAM channel disabling lookups?
KAM channel (https://mcgrail.com/template/kam.cf_channel) users might want to check their rules.. KAM_deadweight2_sub.cf contains this: meta __RCVD_IN_SORBS 0 meta __RCVD_IN_ZEN 0 meta __RCVD_IN_MSPIKE_B 0 meta __RCVD_IN_MSPIKE_L 0 meta __RCVD_IN_DNSWL 0 Seems it's been disabling many active and useful DNSBL/WL lookups for a long time?
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > In case of URIBL_SBL I don't know why it gets undefined, since other RBLs > work in the same place (eg. URIBL_BLACK) as expected and the result is 1. Thanks for the tip, found the problem. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > And another quite simple ruleset... > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > doesn't set __SA4TA3. This was working an SA3.4 as well. Works fine here. > Is this wanted behavior or a bug? Since UPGRADE does not contain any > information about basic changes for meta rules I assume it is a > bug/regression. But if this is wanted how should this be done on SA4? >From UPGRADE: - Meta rules no longer use priority values, they are evaluated dynamically when the rules they depend on are finished. (Bug 7735)
Re: _DKIMDOMAIN_ vs. _AUTHORDOMAIN_ (was: Re: dnswl dwl rule)
On Fri, Oct 07, 2022 at 04:41:57PM +0300, Henrik K wrote: > It's not possible to use priority with askdns. The rule is launched then > the all dependent tags are set, nothing more, nothing less. ... obvious typo but just to clarify, _when_ all tags are set..
Re: _DKIMDOMAIN_ vs. _AUTHORDOMAIN_ (was: Re: dnswl dwl rule)
On Sat, Oct 01, 2022 at 04:42:09PM +0200, Matus UHLAR - fantomas wrote: > > perhaps these all should replace _DKIMDOMAIN_ by _AUTHORDOMAIN_ and AND-ed > with DKIM_VALID_AU. > > can these checks be made the way DNS queries are done only when > DKIM_VALID_AU matches? > > perhaps playing with priority It's not possible to use priority with askdns. The rule is launched then the all dependent tags are set, nothing more, nothing less. So there would have to be a _DKIMAUTHORDOMAIN_ or such, which would be set from From: address when valid DKIM author sig is found. This would obviously require changing DKIM.pm plugin code to set it. Other than that, I have no idea if something like that would be useful, I leave that up for others to ponder.
Re: _DKIMDOMAIN_ vs. _AUTHORDOMAIN_
On Fri, Oct 07, 2022 at 03:01:17PM +0200, Matus UHLAR - fantomas wrote: > > the _DKIMDOMAIN_ can contain multiple domains if mail is signed using > multiple valid keys. Not a problem, as AskDNS doc says: "Tags which produce multiple values will result in multiple queries launched, each with an expanded template using one of the tag values. An example is a DKIMDOMAIN tag which yields a list of signing domains, one for each valid signature in a signed message." _DKIMDOMAIN_ contains verified domains. _AUTHORDOMAIN_ is simply the From: address without any verification. It has nothing to do with DKIM. So it would make no sense to use this.
Re: ExtractText options in user_prefs, spamd problem
On Fri, Oct 07, 2022 at 10:58:07AM +0200, Matus UHLAR - fantomas wrote:
> Hello,
>
> I have configured my personal user_prefs to process options for extracttext.
>
> so far, spamd complains:
>
> Oct 7 09:29:05 fantomas spamd[26887]: spamd: setuid to uhlar succeeded
> Oct 7 09:29:05 fantomas spamd[26887]: config: failed to parse line in
> /home/uhlar/.spamassassin/user_prefs (line 47):
> extracttext_external\tpdftotext\t/usr/bin/pdftotext -nopgbrk -layout -enc
> UTF-8 {} -
> Oct 7 09:29:05 fantomas spamd[26887]: config: failed to parse line in
> /home/uhlar/.spamassassin/user_prefs (line 48):
> extracttext_use\t\tpdftotext\t.pdf application/pdf
>
> is there any problem with extracttext options when spamd is used?
>
> e.g. are they considered rules? (user rules are disabled by default iirc)
They are not allowed in user_prefs
extracttext_use
extracttext_external
Dunno if the code would support that or if it would be safe to allow it.
Re: FQDN and uridnsbl
On Tue, Oct 04, 2022 at 03:47:02PM +, DEMBLANS Mathieu wrote: > Not sure about this solution. > The problem is for all sites listed in surbl.org, not specifically > square.site and its subdomains. I gave you a workaround for single domains for 3.4. I also told you it's already fully solved, but you have to wait for debian 4.0.0 or install manually: > Upcoming 4.0 already supports tflags notrim, which will query the full host > from surbl and other lists that support it. Those are the choices.
Re: FQDN and uridnsbl
On Tue, Oct 04, 2022 at 03:13:29PM +, DEMBLANS Mathieu wrote: > Hello, > > SpamAssassin version 3.4.6 With postfix 3.4.14 on debian 10.12 > > SpamAssassin version 3.4.2 With postfix 3.4.2 on debian 10.3 > > As it is written in the Mail_SpamAssassin_Plugin_URIDNSBL doc and confirmed by > some tests, when a check is done with uridnsbl, only the domain is requested > not the complete FQDN (rhsbl_zone). > > For example if I want to test abc.domain.com it will only request domain.com . > > My problem is that for phishing url search on surbl.org it doesn?t find it. > > On a real test for btinternet-100730.square.site, which is in the surbl.org PH > list, spamassassin do a dns request for square.site.multi.surbl.org. that > can?t > be find. > > If I test manually btinternet-100730.square.site.surbl.org. the response is > good (127.0.0.8). > > So it probably never find anything in this kind of list. > > Is there any thing to do to make it work correctly ? For SA 3.4 you need to use util_rb_2tld cf for all such domains: util_rb_2tld square.site Upcoming 4.0 already supports tflags notrim, which will query the full host from surbl and other lists that support it.
Re: alternatives for deprecated Perl API support in SA's RelayCountry plugin + MaxMind GeoIP2 *.mmdb data?
On Fri, Sep 09, 2022 at 08:15:04AM -0400, PGNet Dev wrote: > > What alternative, non-deprecated support, if any, exists, or is planned, > for SA RelayCountry plugin usage with MaxMind GeoIP2 *.mmdb data? As the database format should not ever change, there is no reason to assume the current code would break in the future. Also, upcoming SpamAssassin 4.0 uses a much smaller subset of modules, only MaxMind::DB::Reader / MaxMind::DB::Reader::XS directly. https://metacpan.org/pod/MaxMind::DB::Reader::XS "This module is deprecated and will only receive fixes for major bugs and security vulnerabilities. New features and functionality will not be added." So it still received updates if necessary.
Re: metholdless URLs bypass DecodeShortURLs link shortner checking
On Mon, Aug 29, 2022 at 02:43:01AM -0500, Dave Funk wrote: > > Where would I find the most recent version of DecodeShortURLs plugin? It is now maintained by SA project and included in 4.0 release. Best to wait for that or try the current trunk/4.0.0-rc1.
Re: DKIM fails on v4
On Sun, Jun 26, 2022 at 12:57:32PM -0400, Alex wrote: > > > Amavisd-new works fine here. Maybe $enable_dkim_verification or something > is different. > > > It's good to know you're using amavisd. It's very dependent upon the SA > version > you're using, though. > > It appears both DKIM and DMARC worked until the May 29th version from svn > (1901385). > > At some point after that, and even until yesterday's version, DKIM stopped > working. DMARC still passes with SPF, but there are no longer any occurrences > of DKIM. I think Giovannis changes don't work when amavisd is passing $suppl_attrib: https://svn.apache.org/viewvc?view=revision&revision=1901719 Sub _check_signature() isn't called at all in that case and things like tags are not set. I'll leave it for Giovanni to fix..
Re: DKIM fails on v4
Amavisd-new works fine here. Maybe $enable_dkim_verification or something is different. On Sun, Jun 26, 2022 at 03:32:14PM +0200, giova...@paclan.it wrote: > All the people that reported DKIM failures to me in SA 4.0 in the last month > are using amavisd-new, could it be related to how amavisd-new is calling SA ? > Giovanni > > On 6/26/22 07:55, Henrik K wrote: > > > > Have you checked what debugging says? > > > > $sa_debug = 'info,dkim,DMARC'; > > > > > > On Sat, Jun 25, 2022 at 03:45:48PM -0400, Alex wrote: > >> Hi, > >> > >> It's definitely a problem with the current spamassassin from github v4. I > >> went > >> back to an old version I built on May 29th and it immediately starts > >> reporting > >> DKIM normally again. > >> > >> I just built the latest version and it's still exhibiting the same problem. > >> Based on my logs, it started happening on or around June 14th. > >> > >> DMARC is not working with my version from May 29th. I wonder if I could > >> drop in > >> the DMARC.pm that was updated at the end of June into the May 29th version > >> and > >> see if now they both work? > >> > >> btw, I previously mentioned github, but meant [1]svn.apache.org. > >> svn checkout [2]http://svn.apache.org/repos/asf/spamassassin/trunk > >> Mail-SpamAssassin-4.0.0 > >> > >> > >> > >> > >> > >> > >> > >> On Sat, Jun 25, 2022 at 3:07 PM Alex <[3]mysqlstud...@gmail.com> wrote: > >> > >> Hi, > >> I've been having problems with DMARC failing over the past few weeks > >> using > >> the latest SA, even on sites I know have passed. It appears to have > >> coincided with an update to DMARC.pm related to timing. I just now > >> happened > >> to notice that maybe the problem is with DKIM, or there's a separate > >> DKIM > >> problem or something I simply don't understand. Installing v3.4.6 over > >> the > >> latest v4 fixes the problem instantly. > >> > >> It appears DKIM is loading in amavis: > >> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer > >> 1.20200907 > >> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier > >> 1.20200907 > >> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code loaded > >> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: > >> ASN, > >> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC, > >> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, > >> HTMLEval, > >> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval, > >> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2, > >> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid, > >> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval, > >> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2 > >> > >> Yet it never fires. The only references to DKIM in emails are > >> from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug? > >> > >> You might also recall from my previous reports that DKIM succeeds on an > >> email where it otherwise failed when running it through SA directly. > >> > >> $ spamassassin --version > >> SpamAssassin version 4.0.0-r1901426 > >> running on Perl version 5.34.1 > >> > >> This is on fedora35. Installing the stock 3.4.6 immediately starts > >> triggering DKIM hits. > >> > >> Is there a backport of RaciallyCharged, Esp and ExtractText (although I > >> don't really use that anymore) that's available for v3.4.6, so my > >> welcomelist entries work in the meantime? > >> > >> > >> > >> > >> References: > >> > >> [1] http://svn.apache.org/ > >> [2] http://svn.apache.org/repos/asf/spamassassin/trunk > >> [3] mailto:mysqlstud...@gmail.com >
Re: DKIM fails on v4
Have you checked what debugging says? $sa_debug = 'info,dkim,DMARC'; On Sat, Jun 25, 2022 at 03:45:48PM -0400, Alex wrote: > Hi, > > It's definitely a problem with the current spamassassin from github v4. I went > back to an old version I built on May 29th and it immediately starts reporting > DKIM normally again. > > I just built the latest version and it's still exhibiting the same problem. > Based on my logs, it started happening on or around June 14th. > > DMARC is not working with my version from May 29th. I wonder if I could drop > in > the DMARC.pm that was updated at the end of June into the May 29th version and > see if now they both work? > > btw, I previously mentioned github, but meant [1]svn.apache.org. > svn checkout [2]http://svn.apache.org/repos/asf/spamassassin/trunk > Mail-SpamAssassin-4.0.0 > > > > > > > > On Sat, Jun 25, 2022 at 3:07 PM Alex <[3]mysqlstud...@gmail.com> wrote: > > Hi, > I've been having problems with DMARC failing over the past few weeks using > the latest SA, even on sites I know have passed. It appears to have > coincided with an update to DMARC.pm related to timing. I just now > happened > to notice that maybe the problem is with DKIM, or there's a separate DKIM > problem or something I simply don't understand. Installing v3.4.6 over the > latest v4 fixes the problem instantly. > > It appears DKIM is loading in amavis: > Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer > 1.20200907 > Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier > 1.20200907 > Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code loaded > Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN, > AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC, > DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval, > HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval, > MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2, > RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid, > Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval, > URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2 > > Yet it never fires. The only references to DKIM in emails are > from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug? > > You might also recall from my previous reports that DKIM succeeds on an > email where it otherwise failed when running it through SA directly. > > $ spamassassin --version > SpamAssassin version 4.0.0-r1901426 > running on Perl version 5.34.1 > > This is on fedora35. Installing the stock 3.4.6 immediately starts > triggering DKIM hits. > > Is there a backport of RaciallyCharged, Esp and ExtractText (although I > don't really use that anymore) that's available for v3.4.6, so my > welcomelist entries work in the meantime? > > > > > References: > > [1] http://svn.apache.org/ > [2] http://svn.apache.org/repos/asf/spamassassin/trunk > [3] mailto:mysqlstud...@gmail.com
Re: DMARC fails for valid record?
On Thu, May 26, 2022 at 03:48:57PM +0200, Matus UHLAR - fantomas wrote: > > > >I also haven't any references to DMARC whatsoever from any SA rules since > > > >it was uninstalled. > > > > > > >I otherwise have no way of telling if there should have been any hits, > > > >but > > > >I'd imagine there should have been at least one in 24-hours. > > > > > > > >It appears to have disabled DMARC functionality entirely. > > > > > > KAM.cf has some DMARC rules even without Mail::SpamAssassin::Plugin::DMARC > > > available, but I'm not sure if loading that plugin doesn't disable them. > > > > > > I have disabled loading it so let's see. > > On 26.05.22 09:34, Alex wrote: > > Any further thoughts on this? It appears removing the DMARC perl library > > has disabled any DMARC support altogether. > > disabling Mail::SpamAssassin::Plugin::DMARC should > make KAM.cf revert to it's simpler DMARC > functioality > > note that it requires: > Mail::SpamAssassin::Plugin::AskDNS > Mail::SpamAssassin::Plugin::DKIM > Mail::SpamAssassin::Plugin::SPF > > no matter if you have Mail::SpamAssassin::Plugin::DMARC loaded or not. Latest trunk has fix for DMARC waiting for SPF and DKIM results. Might be relevant to this thread.
Re: Rule to detect non-standard headers that aren't X- prefixed
On Mon, May 23, 2022 at 10:48:51PM -0600, Philip Prindeville wrote: > > > > On May 11, 2022, at 1:53 AM, Henrik K wrote: > > > > On Wed, May 11, 2022 at 10:49:32AM +0300, Henrik K wrote: > >> On Wed, May 11, 2022 at 10:44:05AM +0300, Henrik K wrote: > >>> On Tue, May 10, 2022 at 06:19:38PM -0600, Philip Prindeville wrote: > >>>> See my original message. > >>>> > >>>> I can't think of a single way to match each header, and then test for > >>>> any of them not matching the pattern... > >>> > >>> Simply use regex negative lookahead. > >>> > >>> ALL =~ /^(?!Foo|Bar):/m > >>> > >>> It will hit any line _not_ starting with Foo: or Bar: > >> > >> Oops I think it was buggy.. more like: > >> > >> ALL =~ /^(?!(?:Foo|Bar):)/m > > > > And for debug logging to log the missing header (to easily inspect what was > > matched) you need some additional string matching, lookahead itself doesn't > > save any string > > > > ALL =~ /^(?!(?:Foo|Bar):)[^:]+/m > > > > > Ended up using .*$ instead of [^:]* but that worked too. > > Is it possible to count how many times we didn't see matching headers and > then count those, setting some threshold, like 3 or more unknown headers? tflags multiple should work header UNKNOWN_HDR ALL ... tflags UNKNOWN_HDR multiple maxhits=3 meta UNKNOWN_HDR_TOOMANY UNKNOWN_HDR >= 3
Re: RCVD_IN_DNSWL
On Wed, May 18, 2022 at 08:16:15AM +0300, Henrik K wrote: > On Fri, May 13, 2022 at 05:42:04PM -0400, Jeff Koch wrote: > > > > Hi: > > > > We're getting numerous false positives on 'RCVD_IN_DNSWL_HI RBL'. When I > > check > > these IP's (193.106.175.39, for example) at [1]https://www.dnswl.org they > > are > > NOT listed. > > > > * -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at [2]https:// > > www.dnswl.org/, high > > * trust > > * [193.106.175.39 listed in list.dnswl.org] > > The test_log() function in 3.4, which produces the "listed in" line you see > is quite broken. I suspect you are simply seeing results for wrong test, > 193.106.175.39 is listed in SORBS. > > To see what is queried, check the debug log for some message: > spamassassin -t -D async < message 2>&1 | grep dnswl.org > > Don't forget that it's reversed: > > 4.3.2.1.list.dnswl.org = 1.2.3.4 Never mind, I think the log text itself should always be correct, the IP and listname should not get mixed up.. Doesn't hurt to always check the debug output, use "-D all" to find everything.
Re: RCVD_IN_DNSWL
On Fri, May 13, 2022 at 05:42:04PM -0400, Jeff Koch wrote: > > Hi: > > We're getting numerous false positives on 'RCVD_IN_DNSWL_HI RBL'. When I check > these IP's (193.106.175.39, for example) at [1]https://www.dnswl.org they are > NOT listed. > > * -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at [2]https:// > www.dnswl.org/, high > * trust > * [193.106.175.39 listed in list.dnswl.org] The test_log() function in 3.4, which produces the "listed in" line you see is quite broken. I suspect you are simply seeing results for wrong test, 193.106.175.39 is listed in SORBS. To see what is queried, check the debug log for some message: spamassassin -t -D async < message 2>&1 | grep dnswl.org Don't forget that it's reversed: 4.3.2.1.list.dnswl.org = 1.2.3.4
Re: Rule to detect non-standard headers that aren't X- prefixed
On Fri, May 13, 2022 at 12:22:48PM -0600, Philip Prindeville wrote: > > How do you look at what a rule is matching? I've never figured that out... Debug output: spamassassin -t -D rules < message.eml 2>&1 | grep 'got hit'
Re: Rule to detect non-standard headers that aren't X- prefixed
On Wed, May 11, 2022 at 10:49:32AM +0300, Henrik K wrote: > On Wed, May 11, 2022 at 10:44:05AM +0300, Henrik K wrote: > > On Tue, May 10, 2022 at 06:19:38PM -0600, Philip Prindeville wrote: > > > See my original message. > > > > > > I can't think of a single way to match each header, and then test for any > > > of them not matching the pattern... > > > > Simply use regex negative lookahead. > > > > ALL =~ /^(?!Foo|Bar):/m > > > > It will hit any line _not_ starting with Foo: or Bar: > > Oops I think it was buggy.. more like: > > ALL =~ /^(?!(?:Foo|Bar):)/m And for debug logging to log the missing header (to easily inspect what was matched) you need some additional string matching, lookahead itself doesn't save any string ALL =~ /^(?!(?:Foo|Bar):)[^:]+/m
Re: Rule to detect non-standard headers that aren't X- prefixed
On Wed, May 11, 2022 at 10:44:05AM +0300, Henrik K wrote: > On Tue, May 10, 2022 at 06:19:38PM -0600, Philip Prindeville wrote: > > See my original message. > > > > I can't think of a single way to match each header, and then test for any > > of them not matching the pattern... > > Simply use regex negative lookahead. > > ALL =~ /^(?!Foo|Bar):/m > > It will hit any line _not_ starting with Foo: or Bar: Oops I think it was buggy.. more like: ALL =~ /^(?!(?:Foo|Bar):)/m Unless you want to write colon to all alternations ALL =~ /^(?!Foo:|Bar:)/m
Re: Rule to detect non-standard headers that aren't X- prefixed
On Tue, May 10, 2022 at 06:19:38PM -0600, Philip Prindeville wrote: > See my original message. > > I can't think of a single way to match each header, and then test for any of > them not matching the pattern... Simply use regex negative lookahead. ALL =~ /^(?!Foo|Bar):/m It will hit any line _not_ starting with Foo: or Bar:
Re: Spamassassin with Galera as SQL-Backend?
On Fri, May 06, 2022 at 12:31:47PM +0200, giova...@paclan.it wrote: > On 5/6/22 11:08, Niels Kobschätzki wrote: > > Hi, > > > > I have a setup where the spamassassin-servers have actually no access to > > the data of the mail-servers. Now I was looking into having per user > > bayes-databases and saw that I can do that with a SQL-database. I have > > already a small galera-cluster and I wonder if spamassassin will work with > > it because of the limitations galera has. > > The limitations are: > > > > * only innodb > > * unsupported explicit locking > > * a primary key on all tables is necessary > > * no XA transactions > > * no reliance on auto-increment > > > > Does anyone have experience with such a setup? > > > Few things to consider: > bayes_expire has no primary key. >From what I see, there's no reason why it shouldn't be. CREATE TABLE bayes_expire ( id int(11) NOT NULL default '0', runtime int(11) NOT NULL default '0', KEY bayes_expire_idx1 (id) ) ENGINE=InnoDB; BayesStore/MySQL.pm has kind of a dumb insert which might insert things multiple times my $sql = "INSERT INTO bayes_expire (id,runtime) VALUES (?,?)"; It should just be converted to UPSERT. Of course this won't help until 4.0.0 is released.. > bayes_vars MySQL table has the id defined as "id int(11) NOT NULL > AUTO_INCREMENT". Google implies Galera supports auto_increment just fine, it just does something funny like incrementing them in 3 multiples or something.
Re: Spamassassin with Galera as SQL-Backend?
On Fri, May 06, 2022 at 11:08:21AM +0200, Niels Kobschätzki wrote: > Hi, > > I have a setup where the spamassassin-servers have actually no access to the > data of the mail-servers. Now I was looking into having per user > bayes-databases and saw that I can do that with a SQL-database. I have already > a small galera-cluster and I wonder if spamassassin will work with it because > of the limitations galera has. > The limitations are: > > * only innodb > * unsupported explicit locking > * a primary key on all tables is necessary > * no XA transactions > * no reliance on auto-increment > > Does anyone have experience with such a setup? I see no reason why it wouldn't work, none of the limitations should apply to SpamAssassin.
Re: Untrustworthy TLDs and KAM
On Sun, May 01, 2022 at 09:45:38PM -0400, Alex wrote: > Hi, > > Four points for a .online TLD with KAM rules > > * 2.0 PDS_OTHER_BAD_TLD Untrustworthy TLDs > * [URI: www.lci-mtc.online (online)] > > * 2.0 KAM_SOMETLD_ARE_BAD_TLD .bar, .buzz, .cam, .casa, .cfd, .club, > * .date, .guru, .live, .online, .press, .pw, .quest, .rest, .sbs, > * .shop, .stream, .top, .trade, .work, .xyz TLD abuse > > $ spamassassin --version > SpamAssassin version 4.0.0-r1898781 > running on Perl version 5.34.1 General FYI, sa-update from trunk/4.0.0 also has these options: --score-multiplier x.x Adjust all scores from update channel, multiply with given value (integer or float). --score-limit x.x Adjust all scores from update channel, limit to given value (integer or float). Limiting is done after possible multiply operation. Developed specially for KAM scoring. ;-)
Re: compat_welcomelist_blocklist errors in latest update
On Fri, Apr 29, 2022 at 10:12:24AM -0600, asparks wrote: > > Is there anything I can do to fix these on SA 3.3.1? Thanks in advance. > -Alan Already committed a fix to replace has() with can() in rules, will be live in a few days..
Re: update-rules script: Error with latest LWP:::UserAgent on FreeBSD
It's harmless and fixed in next libwww: https://github.com/libwww-perl/libwww-perl/issues/410 On Wed, Apr 27, 2022 at 02:04:38PM -0500, Larry Rosenman wrote: > I'm getting the following error when my update_rules script runs: > > "my" variable $uri masks earlier declaration in same scope at > /usr/local/lib/perl5/site_perl/LWP/UserAgent.pm line 783. > > > I think(?) this comes from this package: > ??? pkg info p5-libwww > p5-libwww-6.63 > Name : p5-libwww > Version: 6.63 > Installed on : Tue Apr 26 15:03:58 2022 CDT > Origin : www/p5-libwww > Architecture : FreeBSD:13:* > Prefix : /usr/local > Categories : devel perl5 www > Licenses : ART10, GPLv1+ > Maintainer : sunp...@freebsd.org > WWW: https://metacpan.org/release/libwww-perl > Comment: Perl5 library for WWW access > Annotations: > build_timestamp: 2022-04-26T16:51:34+ > built_by : poudriere-git-3.3.99.20211130 > port_checkout_unclean: no > port_git_hash : 192ed4c74fe5 > ports_top_checkout_unclean: no > ports_top_git_hash: 0f1527691c04 > repo_type : binary > repository : poudriere > Flat size : 419KiB > Description: > Libwww-perl is a collection of Perl modules which provides a simple and > consistent programming interface (API) to the World-Wide Web. The main > focus of the library is to provide classes and functions that allow you > to write WWW clients, thus libwww-perl said to be a WWW client library. > The library also contain modules that are of more general use. > > The main architecture of the library is object oriented. The user > agent, requests sent and responses received from the WWW server are all > represented by objects. This makes a simple and powerful interface to > these services. The interface should be easy to extend and customize > for your needs. > > WWW: https://metacpan.org/release/libwww-perl > > ler in thebighonker in ~ via ??? v1.8.0 via v5.32.1 via v3.0.4 > ??? > > > /usr/local/etc/mail/spamassassin/update-rules.sh > ??? cat /usr/local/etc/mail/spamassassin/update-rules.sh > #!/bin/sh > PATH=$PATH:/usr/local/bin > export PATH > /usr/local/bin/sa-update > EXIT=$? > case $EXIT in > 0) > /usr/local/bin/sa-compile >kill -1 `cat /var/run/spamd/spamd.pid`;; > *) ;; > esac > > ler in thebighonker in ~ via ??? v1.8.0 via v5.32.1 via v3.0.4 > ??? > > ??? pkg info spamassassin > zsh: correct 'spamassassin' to '.spamassassin' [nyae]? n > spamassassin-3.4.5 > Name : spamassassin > Version: 3.4.5 > Installed on : Sun Apr 3 17:05:29 2022 CDT > Origin : mail/spamassassin > Architecture : FreeBSD:13:amd64 > Prefix : /usr/local > Categories : perl5 mail > Licenses : APACHE20 > Maintainer : zeis...@freebsd.org > WWW: http://spamassassin.apache.org/ > Comment: Highly efficient mail filter for identifying spam > Options: > AS_ROOT: on > DCC: off > DKIM : on > DOCS : on > GNUPG : off > GNUPG2 : on > GNUPG_NONE : off > MYSQL : off > PGSQL : on > PYZOR : off > RAZOR : on > RELAY_COUNTRY : on > RLIMIT : off > SPF_QUERY : on > SSL: on > Shared Libs required: > libperl.so.5.32 > Annotations: > FreeBSD_version: 1301501 > build_timestamp: 2022-04-02T22:38:31+ > built_by : poudriere-git-3.3.99.20211130 > cpe: cpe:2.3:a:apache:spamassassin:3.4.5:freebsd13:x64 > port_checkout_unclean: no > port_git_hash : 819f25b36d45 > ports_top_checkout_unclean: no > ports_top_git_hash: d0d63dec4011 > repo_type : binary > repository : poudriere > Flat size : 3.28MiB > Description: > SpamAssassin is a mail filter which attempts to identify spam using text > analysis and several internet-based realtime blacklists. > > Using its rule base, it uses a wide range of heuristic tests on mail > headers and body text to identify "spam", also known as unsolicited > commercial email. > > Once identified, the mail can then be optionally tagged as > spam for later > filtering using the user's own mail user-agent application. > > Additional drop-in rule sets are available at > http://wiki.apache.org/spamassassin/CustomRulesets > > WWW: http://spamassassin.apache.org/ > > ler in thebighonker in ~ via ??? v1.8.0 via v5.32.1 via v3.0.4 > > Ideas? > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: l...@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Re: your mail
On Tue, Apr 26, 2022 at 05:11:47PM +0300, Henrik K wrote:
> On Tue, Apr 26, 2022 at 03:59:36PM +0200, Matus UHLAR - fantomas wrote:
> > On 26.04.22 16:11, Henrik K wrote:
> > > Maybe a bit safer version that doesn't log huge strings and run wild
> > >
> > > full FOO /^(?=.*?\nContent-Type:
> > > message\/rfc822.{0,1024}?\nReceived:(?:[^\n]{1,100}\n\s{1,100}){0,3}[^\n]{0,100}\b1\.2\.3\.4\b)/s
> >
> > Doesn't this requires mime headers in specific order that may not be
> > fullfilled?
>
> Well if you want to match rfc822 contents, it's Received: headers can only
> be after a rfc822 declaration.
>
> Other than that it's up to you to figure out, since there's no samples. Of
> course this doesn't replace a full parser, but as long as the stuff you
> receive doesn't vary much, there's no reason for it not to work.
.. as long as the whole rfc822 contents isn't base64 encoded. Probably not that
common.
Re: your mail
On Tue, Apr 26, 2022 at 03:59:36PM +0200, Matus UHLAR - fantomas wrote:
> On 26.04.22 16:11, Henrik K wrote:
> > Maybe a bit safer version that doesn't log huge strings and run wild
> >
> > full FOO /^(?=.*?\nContent-Type:
> > message\/rfc822.{0,1024}?\nReceived:(?:[^\n]{1,100}\n\s{1,100}){0,3}[^\n]{0,100}\b1\.2\.3\.4\b)/s
>
> Doesn't this requires mime headers in specific order that may not be
> fullfilled?
Well if you want to match rfc822 contents, it's Received: headers can only
be after a rfc822 declaration.
Other than that it's up to you to figure out, since there's no samples. Of
course this doesn't replace a full parser, but as long as the stuff you
receive doesn't vary much, there's no reason for it not to work.
Re: your mail
On Tue, Apr 26, 2022 at 04:04:13PM +0300, Henrik K wrote:
> On Tue, Apr 26, 2022 at 02:35:25PM +0200, Matus UHLAR - fantomas wrote:
> > Hello,
> >
> > is it possible to match message headers in rfc822 atttachments?
> >
> > from what I know, "header" rules only apply to mail headers and mimeheader
> > only apply to mime headers.
> >
> > body and rawbody afaik only search in bodies of messages or included
> > messages.
> >
> > I have asked some time ago but no success:
> >
> > https://marc.info/?l=spamassassin-users&m=132282473328809&w=2
> >
> > is this possible now or do we need out-of SA solution for this?
>
> full FOO /\nContent-Type:
> message\/rfc822.*?\nReceived:(?:[^\n]+\n\s+){0,3}[^\n]*\b1.2.3.4\b/s
Maybe a bit safer version that doesn't log huge strings and run wild
full FOO /^(?=.*?\nContent-Type:
message\/rfc822.{0,1024}?\nReceived:(?:[^\n]{1,100}\n\s{1,100}){0,3}[^\n]{0,100}\b1\.2\.3\.4\b)/s
Re: your mail
On Tue, Apr 26, 2022 at 02:35:25PM +0200, Matus UHLAR - fantomas wrote:
> Hello,
>
> is it possible to match message headers in rfc822 atttachments?
>
> from what I know, "header" rules only apply to mail headers and mimeheader
> only apply to mime headers.
>
> body and rawbody afaik only search in bodies of messages or included
> messages.
>
> I have asked some time ago but no success:
>
> https://marc.info/?l=spamassassin-users&m=132282473328809&w=2
>
> is this possible now or do we need out-of SA solution for this?
full FOO /\nContent-Type:
message\/rfc822.*?\nReceived:(?:[^\n]+\n\s+){0,3}[^\n]*\b1.2.3.4\b/s
Re: How to deal with bounce messages
On Mon, Apr 25, 2022 at 03:48:01PM +0300, Henrik K wrote: > On Mon, Apr 25, 2022 at 02:21:49PM +0200, Giovanni Bechis wrote: > > KAM.cf has already all the needed glue, if you update to trunk and enable > > DMARC plugin, DMARC rules will use new plugin code. > > Giovannin > > KAM.cf is not from SA project, not everyone uses it.. the DMARC stuff > should be implemented in stock rules for 4.0.0. Committed to stock rules: https://svn.apache.org/repos/asf/spamassassin/trunk/rules/25_dmarc.cf Please someone double check that it looks usable, my covid brain can only do so much right now. Automatic updates are still on pause for a few days due to welcomelist changes, so it's not going live yet. Trunk is getting very near to 4.0.0 pre-releases, so if people could actually install and test it, would be great..
Re: How to deal with bounce messages
On Mon, Apr 25, 2022 at 02:21:49PM +0200, Giovanni Bechis wrote: > On Mon, Apr 25, 2022 at 12:50:49PM +0300, Henrik K wrote: > > On Mon, Apr 25, 2022 at 11:48:52AM +0200, Matus UHLAR - fantomas wrote: > > > > > >> >https://pastebin.com/s032ndrA > > > > > >> > > > > > > >> >It's not only hitting DMARC_REJ_NO_DKIM and DMARC_FAIL_REJECT, but > > > > > >> > > > > > >> where did you get these from? > > > > > > > > > > On 22.04.22 10:02, Alex wrote: > > > > > >I just realized these are from my local rules, put together from a > > > > > >conversation many years ago, apparently from before SA had built-in > > > > > >DMARC support. > > > > > > > > > > > >https://www.mail-archive.com/users@spamassassin.apache.org/msg95643.html > > > > > > > > now I really wonder why these aren't part of stock SA rules. > > > > > > On 24.04.22 14:39, Alex wrote: > > > > Does this mean you are investigating further? > > > > > > not me, as I'm not involved in SA deployment more than by being active > > > here. > > > perhaps you could fill a wishlist report... > > > > > > > Are these rules from the link above useful? > > > > > > looks like they are. KAM.cf contains similar rules, but having them in > > > stock > > > SA would be nice. > > > > Soon released 4.0.0 already has a dedicated DMARC plugin, such rules should > > become obsolete. Testers would be appreciated.. > > > KAM.cf has already all the needed glue, if you update to trunk and enable > DMARC plugin, DMARC rules will use new plugin code. > Giovannin KAM.cf is not from SA project, not everyone uses it.. the DMARC stuff should be implemented in stock rules for 4.0.0.
Re: How to deal with bounce messages
On Mon, Apr 25, 2022 at 11:48:52AM +0200, Matus UHLAR - fantomas wrote: > > > >> >https://pastebin.com/s032ndrA > > > >> > > > > >> >It's not only hitting DMARC_REJ_NO_DKIM and DMARC_FAIL_REJECT, but > > > >> > > > >> where did you get these from? > > > > > > On 22.04.22 10:02, Alex wrote: > > > >I just realized these are from my local rules, put together from a > > > >conversation many years ago, apparently from before SA had built-in > > > >DMARC support. > > > > > > > >https://www.mail-archive.com/users@spamassassin.apache.org/msg95643.html > > > > now I really wonder why these aren't part of stock SA rules. > > On 24.04.22 14:39, Alex wrote: > > Does this mean you are investigating further? > > not me, as I'm not involved in SA deployment more than by being active here. > perhaps you could fill a wishlist report... > > > Are these rules from the link above useful? > > looks like they are. KAM.cf contains similar rules, but having them in stock > SA would be nice. Soon released 4.0.0 already has a dedicated DMARC plugin, such rules should become obsolete. Testers would be appreciated..
Re: Linting of local.cf
On Sat, Apr 16, 2022 at 12:21:55PM +0100, Martin Gregorie wrote: > > - I don't test SA-issued rules updates because they've been verified > before being issued and I've never found errors in them. There is no need to manually test sa-updated rules, since it automatically lints them. And hopefully the cron output ends up to the admin..
Re: using spamassassin to classify spam
On Fri, Mar 25, 2022 at 06:01:43AM -0400, Michael Grant wrote: > > Unless there's an existing function in some plugin to do this, I'll > have to write my own. Little surprising that there isn't, this seems > like an obvious check! There is already very basic HEADER() template support added in trunk/4.0.0, this would generally work: askdns UNSUB_NXDOMAIN _HEADER(List-Unsubscribe:host)_ MX [NXDOMAIN] It just tries to find something resembling a hostname (having valid TLD) in the header, preferring to match @(.*) first. So it doesn't differentiate between http, mailto etc.
Re: Getting right GPG key for KAM
On Mon, Mar 21, 2022 at 04:16:19AM -0600, @lbutlr wrote: > On 2022 Mar 21, at 03:54, Henrik K wrote: > > On Mon, Mar 21, 2022 at 03:48:51AM -0600, @lbutlr wrote: > >> When running sa-update on an old system (not updated in at least a year) I > >> am getting: > >> > >> # sa-update --gpgkey 24C063D8 --channel kam.sa-channels.mcgrail.com > >> gpg: process '/usr/local/bin/gpg' finished: exit 2 > >> error: GPG validation failed! > > > > Sounds like the gpg command failed, not relating to keys. sa-update -D > > could give clues. > > Thanks, I did run sa-update (non KAM) and it worked. Running it again with -D > on KAM gives: > > Mar 21 04:13:56.804 [89542] dbg: gpg: calling gpg > Mar 21 04:13:56.811 [89542] dbg: gpg: [GNUPG:] NEWSIG > Mar 21 04:13:56.811 [89542] dbg: gpg: gpg: Signature made Fri Mar 18 10:25:02 > 2022 MDT > Mar 21 04:13:56.812 [89542] dbg: gpg: gpg: using RSA key > 21D97142272C9066FCAA792B4A156DA524C063D8 > Mar 21 04:13:56.812 [89542] dbg: gpg: [GNUPG:] ERRSIG 4A156DA524C063D8 1 8 00 > 1647620702 9 21D97142272C9066FCAA792B4A156DA524C063D8 > Mar 21 04:13:56.812 [89542] dbg: gpg: [GNUPG:] NO_PUBKEY 4A156DA524C063D8 > Mar 21 04:13:56.812 [89542] dbg: gpg: gpg: Can't check signature: No public > key > gpg: process '/usr/local/bin/gpg' finished: exit 2 > > That doesn't look like a configuration issue n my side? Right, it does seem you haven't imported the key.. https://mcgrail.com/template/kam.cf_channel
Re: Getting right GPG key for KAM
On Mon, Mar 21, 2022 at 03:48:51AM -0600, @lbutlr wrote: > When running sa-update on an old system (not updated in at least a year) I am > getting: > > # sa-update --gpgkey 24C063D8 --channel kam.sa-channels.mcgrail.com > gpg: process '/usr/local/bin/gpg' finished: exit 2 > error: GPG validation failed! Sounds like the gpg command failed, not relating to keys. sa-update -D could give clues.
Re: DCC/pyzor questions
On Mon, Mar 14, 2022 at 08:15:49PM -0400, Alex wrote: > > How do I generate that signature? I've been unable to find any > instructions on how to do it. https://www.dcc-servers.net/dcc/dcc-tree/dcc.html dccproc -CQ < message Add to /var/dcc/whiteclnt "Hex ctype cksum starts with the string Hex followed a checksum type, and a string of four hexadecimal numbers obtained from a DCC log file or the dccproc(8) command using -CQ. The check- sum type is body, Fuz1, or Fuz2 or one of the preceding checksum types such as env_From." > Same with pyzor? pyzor local_whitelist < message (which updates .pyzor/whitelist) > Do you train marketing emails for bayes? You teach Bayes either ham or spam. It makes no difference if it's "marketing" or not. Just feed it. > Separately, is ExtractText broken? I have legitimate invoices that are > hitting multiple money rules. Is this the expected behavior? Any > advice on how to deal with it? Invoices contain money. ExtractText feeds the content to body rules. What are you expecting to happen? Don't use it if it doesn't fit your profile. Personally I don't think the concept of the plugin is good - body rules are written with the expectation of hitting stuff from email body, not some random attachments (which might even decode to garbage). But it's put out there for you to decide.
