Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 02:54:08PM +0200, Damian wrote:
> > > # __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
> > > meta SA4 __SA4
> > > score SA4 1
> > > describe SA4 dummy
> > yields
> > > SA dbg: rules-all: unrun dependencies prevented meta SA4 from
> > > running: __SA4
>
> The above is slightly misleading, even in SA3 one had to predeclare a
> default via
>
> > meta __SA4 0
> > meta SA4 __SA4
> > score SA4 1
> > describe SA4 dummy
>
> Such __SA4 default solves the "unrun dependencies prevented" issue, but
> still produces hits inconsistently.
Uggh.. I even use amavisd, but didn't notice that rule_hits stuff. This is
why we need and appreciate rc-testers. :-)
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8061
> Sometimes
>
> > SA dbg: check: tests=SA4
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
> sometimes
> > SA dbg: check: tests=
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
>
> As far as I can tell, the inconsistency comes from the unsorted key
> iteration in do_meta_tests() and finish_meta_tests().
The unsorted part doesn't matter, actually it's better this way so we able
to notice the problems. When meta handling for this case is fixed, it will
always work regardless..
Re: RCVD_IN_DNSWL_* "unrun" (was: SA 4.0.0-rc3 meta rule results "undefined")
On 11/10/2022 15:08, Henrik K wrote: This is because __RCVD_IN_DNSWL is not supposed to be a meta. KAM channel overrides it to "disable" the rule. I just posted on the list about that.. Oh, right! Thanks for the pointer. Didn't catch up to this point yet. Greetings, Wolfgang
Re: RCVD_IN_DNSWL_* "unrun" (was: SA 4.0.0-rc3 meta rule results "undefined")
On Tue, Oct 11, 2022 at 02:36:59PM +0200, Wolfgang Breyha wrote: > On 11/10/2022 13:29, Henrik K wrote: > > You can also need to use -D rules,rules-all to see any "unrun" rules. > > I tried that using "all,rules,rules-all" and I think I found an other > problem with RCVD_IN_DNSWL* rules. > > If I run with a testmail I see: > > # spamassassin -D all,rules,rules-all &1|grep RCVD_IN_DNSWL > > Oct 11 14:32:33.399 [1549685] dbg: rules: meta KHOP_HELO_FCRDNS inherits > > tflag net, depends on __RCVD_IN_DNSWL > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_NONE, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.0$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_LOW, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.1$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_MED, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.2$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_HI, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.3$) > > Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for > > rule RCVD_IN_DNSWL_BLOCKED, set dnswl-firsttrusted, subtest > > (?^aa:^127\\.0\\.\\d+\\.255$) > > Oct 11 14:32:33.627 [1549685] dbg: rules-all: ran meta rule > > __RCVD_IN_DNSWL, no hit > > Oct 11 14:32:34.462 [1549685] dbg: rules-all: unrun dependencies prevented > > meta KAM_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, > > RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W > > Oct 11 14:32:34.464 [1549685] dbg: rules-all: unrun dependencies prevented > > meta KAM_QUITE_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, > > RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta XPRIO from running: RCVD_IN_DNSWL_NONE > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE1 from running: RCVD_IN_DNSWL_MED > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE2 from running: RCVD_IN_DNSWL_LOW > > Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented > > meta WLCOMPENSATE4 from running: RCVD_IN_DNSWL_HI > > This looks like __RCVD_IN_DNSWL gets evaluated, but not hit. But > RCVD_IN_DNSWL_* are "unrun", what looks very odd. This is because __RCVD_IN_DNSWL is not supposed to be a meta. KAM channel overrides it to "disable" the rule. I just posted on the list about that..
Re: SA 4.0.0-rc3 meta rule results "undefined"
# __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
meta SA4 __SA4
score SA4 1
describe SA4 dummy
yields
SA dbg: rules-all: unrun dependencies prevented meta SA4 from
running: __SA4
The above is slightly misleading, even in SA3 one had to predeclare a
default via
meta __SA4 0
meta SA4 __SA4
score SA4 1
describe SA4 dummy
Such __SA4 default solves the "unrun dependencies prevented" issue, but
still produces hits inconsistently.
Sometimes
SA dbg: check: tests=SA4
SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
Total Hits: 1)
sometimes
SA dbg: check: tests=
SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
Total Hits: 1)
As far as I can tell, the inconsistency comes from the unsorted key
iteration in do_meta_tests() and finish_meta_tests().
RCVD_IN_DNSWL_* "unrun" (was: SA 4.0.0-rc3 meta rule results "undefined")
On 11/10/2022 13:29, Henrik K wrote: You can also need to use -D rules,rules-all to see any "unrun" rules. I tried that using "all,rules,rules-all" and I think I found an other problem with RCVD_IN_DNSWL* rules. If I run with a testmail I see: # spamassassin -D all,rules,rules-all &1|grep RCVD_IN_DNSWL Oct 11 14:32:33.399 [1549685] dbg: rules: meta KHOP_HELO_FCRDNS inherits tflag net, depends on __RCVD_IN_DNSWL Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for rule RCVD_IN_DNSWL_NONE, set dnswl-firsttrusted, subtest (?^aa:^127\\.0\\.\\d+\\.0$) Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for rule RCVD_IN_DNSWL_LOW, set dnswl-firsttrusted, subtest (?^aa:^127\\.0\\.\\d+\\.1$) Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for rule RCVD_IN_DNSWL_MED, set dnswl-firsttrusted, subtest (?^aa:^127\\.0\\.\\d+\\.2$) Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for rule RCVD_IN_DNSWL_HI, set dnswl-firsttrusted, subtest (?^aa:^127\\.0\\.\\d+\\.3$) Oct 11 14:32:33.595 [1549685] dbg: dnseval: initialize check_rbl_sub for rule RCVD_IN_DNSWL_BLOCKED, set dnswl-firsttrusted, subtest (?^aa:^127\\.0\\.\\d+\\.255$) Oct 11 14:32:33.627 [1549685] dbg: rules-all: ran meta rule __RCVD_IN_DNSWL, no hit Oct 11 14:32:34.462 [1549685] dbg: rules-all: unrun dependencies prevented meta KAM_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W Oct 11 14:32:34.464 [1549685] dbg: rules-all: unrun dependencies prevented meta KAM_QUITE_BAD_DNSWL from running: KAM_MESSAGE_EMAILBL_PCCC, RCVD_IN_DNSWL_HI, RCVD_IN_HOSTKARMA_W Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented meta XPRIO from running: RCVD_IN_DNSWL_NONE Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented meta WLCOMPENSATE1 from running: RCVD_IN_DNSWL_MED Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented meta WLCOMPENSATE2 from running: RCVD_IN_DNSWL_LOW Oct 11 14:32:34.466 [1549685] dbg: rules-all: unrun dependencies prevented meta WLCOMPENSATE4 from running: RCVD_IN_DNSWL_HI This looks like __RCVD_IN_DNSWL gets evaluated, but not hit. But RCVD_IN_DNSWL_* are "unrun", what looks very odd. Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
Should be fixed in rc4.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060
There remains a bug in the context of $suppl_attrib.
# __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
meta SA4 __SA4
score SA4 1
describe SA4 dummy
yields
SA dbg: rules-all: unrun dependencies prevented meta SA4 from running:
__SA4
SA dbg: check: is spam? score=0 required=5
SA dbg: check: tests=
SA dbg: check: subtests=
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 13:29, Henrik K wrote: Would need exact sample of ruleset, this is too vague to work on. What are all the __SUBMETAs? Would it be ok to send you my ruleset with test.eml offlist? I do not want to send them here publicly or to bugzilla, because they contain a lot of local branding. You can also need to use -D rules,rules-all to see any "unrun" rules. There is something really odd going on with meta rules on SA4. It's a massive change as described in Bug 7735, more than likely that some bugs can remain. But IMO it's still worth it for the most robust logic especially working with async network rules and stuff. This was not meant as critic. Nothing to complain about the basic idea to make things better;-) To be honest I was a bit surprised that my (not that sophisticated) local ruleset breaks that bad on a rc3. But even that is no criticism on your work! I followed your call to test a release candidate of SA4 ... and here we are. Let's get rid of those issues. Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > > And I've still cases were a simple ">" is not evaluated as well... > > I have > meta __META1 (__SUBMETAX + . + __SUBMETAXN) > with > dbg: rules: ran meta rule __META1 ==> got hit (5) > > I have > meta __META2 (__SUBMETAY + . + __SUBMETAYN) > with > dbg: rules: ran meta rule __META2 ==> got hit (8) > > I have > meta __META_NO (__SUBMETAZ || . || _SUBMETAZN ) > which is not hit. > > And finally > meta RESULT ((! __META_NO) && __META1 > 1 && __META2 > 6) > score RESULT 3 > > And RESULT is not hit/evaluated. Would need exact sample of ruleset, this is too vague to work on. What are all the __SUBMETAs? You can also need to use -D rules,rules-all to see any "unrun" rules. > There is something really odd going on with meta rules on SA4. It's a massive change as described in Bug 7735, more than likely that some bugs can remain. But IMO it's still worth it for the most robust logic especially working with async network rules and stuff.
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 13:22, Henrik K wrote: Remains unclear if you applied the patch from https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ? I applied both on rc3 via .spec: + echo 'Patch #1 (SA4rc3_uridnsbl.patch):' Patch #1 (SA4rc3_uridnsbl.patch): + /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .uridnsbl --fuzz=0 patching file lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm + echo 'Patch #2 (SA4rc3_metafix.patch):' Patch #2 (SA4rc3_metafix.patch): + /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .metafix --fuzz=0 patching file lib/Mail/SpamAssassin/Conf/Parser.pm patching file lib/Mail/SpamAssassin/Conf.pm patching file lib/Mail/SpamAssassin/Plugin/Check.pm patching file t/basic_meta2.t Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > On 11/10/2022 12:23, Henrik K wrote: > > Should be fixed in rc4. > > > > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 > > Well, this indeed fixes this test case. But my initial problem which I > though is described in this test case is not fixed yet:( > > So __SA4TA1 gets not evaluated neither in this ... > > header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ > > meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) > > meta __SA4TA2 (__SA4TA1 * 3 ) + 1 > > > > IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL > > is not hit it is undefined. Remains unclear if you applied the patch from https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ?
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 12:23, Henrik K wrote: Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 Well, this indeed fixes this test case. But my initial problem which I though is described in this test case is not fixed yet:( So __SA4TA1 gets not evaluated neither in this ... header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) meta __SA4TA2 (__SA4TA1 * 3 ) + 1 IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL is not hit it is undefined. nor in this ... I also found that modifying __SA4TA1 to meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && __UNDEF ) fails in the same way and __SA4TA2 is always undefined. showcase. The latter is noted in the dbg output with dbg: rules: meta test __SA4TA1 has undefined dependency '__UNDEF' but IMO this should not stop the meta rule from working, because it would basically destroy the possibility to make local dependencies on mainline rules. You would have to check for "undefined dependecy" on every sa-update then, what makes automatic updates impossible. And I've still cases were a simple ">" is not evaluated as well... I have meta __META1 (__SUBMETAX + . + __SUBMETAXN) with dbg: rules: ran meta rule __META1 ==> got hit (5) I have meta __META2 (__SUBMETAY + . + __SUBMETAYN) with dbg: rules: ran meta rule __META2 ==> got hit (8) I have meta __META_NO (__SUBMETAZ || . || _SUBMETAZN ) which is not hit. And finally meta RESULT ((! __META_NO) && __META1 > 1 && __META2 > 6) score RESULT 3 And RESULT is not hit/evaluated. There is something really odd going on with meta rules on SA4. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:21:23PM +0300, Henrik K wrote: > On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > > On 11/10/2022 06:59, Henrik K wrote: > > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > > > And another quite simple ruleset... > > > > meta __SA4TA3_1 6 > > > > meta __SA4TA3_2 2 > > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > > > Works fine here. > > > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > > > Tried again with > > meta __SA4TA3_1 6 > > meta __SA4TA3_2 2 > > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > score SA4TA3 0.1 > > > > I see > > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > > > But no line für SA4TA3 and no report or final score. > > Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. > Will investigate. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > On 11/10/2022 06:59, Henrik K wrote: > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > And another quite simple ruleset... > > > meta __SA4TA3_1 6 > > > meta __SA4TA3_2 2 > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > Works fine here. > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > Tried again with > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > score SA4TA3 0.1 > > I see > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > But no line für SA4TA3 and no report or final score. Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. Will investigate.
Re: SA 4.0.0-rc3 meta rule results "undefined"
But no line für SA4TA3 and no report or final score. Same on Debian bookworm: root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3 (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 describe SA4TA3 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 08:57:26.417 [5489] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS Oct 11 08:57:26.417 [5489] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 08:57:26.420 [5489] dbg: markup: mime_encode_header: Yes, score=7.9 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, X-Spam-Status: Yes, score=7.9 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, root@91d4e83fb538:/# Interestingly, these rules produce insonsistent results: root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3 (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 describe SA4TA3 dummy meta SA4TA4 1 score SA4TA4 0.1 describe SA4TA4 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 09:01:29.899 [5503] dbg: rules: ran meta rule SA4TA4 ==> got hit (1) Oct 11 09:01:29.900 [5503] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 09:01:29.901 [5503] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 09:01:29.904 [5503] dbg: learn: auto-learn: not considered as header or body points, no meta deps (SA4TA4) Oct 11 09:01:29.905 [5503] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 Oct 11 09:01:29.905 [5503] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 09:01:29.907 [5503] dbg: markup: mime_encode_header: Yes, score=8.0 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, Oct 11 09:01:29.907 [5503] dbg: markup: [...] \tNO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no X-Spam-Status: Yes, score=8.0 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no 0.1 SA4TA4 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 09:01:33.798 [5505] dbg: rules: ran meta rule SA4TA4 ==> got hit (1) Oct 11 09:01:33.799 [5505] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 09:01:33.800 [5505] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 09:01:33.801 [5505] dbg: rules: ran meta rule SA4TA3 ==> got hit (1) Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as header or body points, no meta deps (SA4TA4) Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as header or body points, no header/body deps (SA4TA3) Oct 11 09:01:33.804 [5505] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA3,SA4TA4 Oct 11 09:01:33.804 [5505] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 09:01:33.806 [5505] dbg: markup: mime_encode_header: Yes,
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 06:59, Henrik K wrote: On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: And another quite simple ruleset... meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) doesn't set __SA4TA3. This was working an SA3.4 as well. Works fine here. Really? So why doesn't it work here on RHEL7 and RHEL8? Tried again with meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 I see dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) But no line für SA4TA3 and no report or final score. Is this wanted behavior or a bug? Since UPGRADE does not contain any information about basic changes for meta rules I assume it is a bug/regression. But if this is wanted how should this be done on SA4? From UPGRADE: - Meta rules no longer use priority values, they are evaluated dynamically when the rules they depend on are finished. (Bug 7735) I don't see the context with the above problem. Maybe that the dynamic evaluation doesn't work in my case and the final rule is never evaluated. But I still don't know how to possibly fix this on my side. Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > In case of URIBL_SBL I don't know why it gets undefined, since other RBLs > work in the same place (eg. URIBL_BLACK) as expected and the result is 1. Thanks for the tip, found the problem. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > And another quite simple ruleset... > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > doesn't set __SA4TA3. This was working an SA3.4 as well. Works fine here. > Is this wanted behavior or a bug? Since UPGRADE does not contain any > information about basic changes for meta rules I assume it is a > bug/regression. But if this is wanted how should this be done on SA4? >From UPGRADE: - Meta rules no longer use priority values, they are evaluated dynamically when the rules they depend on are finished. (Bug 7735)
SA 4.0.0-rc3 meta rule results "undefined"
Hi! I switched from 3.4.6 to 4.0.0-rc3 last week and since then I experience unexpected/missing results from meta rules which definitely worked on 3.4.6. I noticed it especially on one large meta rule which "summarizes" the results using "+", "&&", "||" and ">". Since SA4 did not complain about any meta rule at all I cutted the rule into peaces and found one specific part which made the result of the whole rule "undefined" at any time while the other parts had correct sums. I was able to reduce this to a test ruleset which fails in the same way... header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) meta __SA4TA2 (__SA4TA1 * 3 ) + 1 IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL is not hit it is undefined. I also found that modifying __SA4TA1 to meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && __UNDEF ) fails in the same way and __SA4TA2 is always undefined. In case of URIBL_SBL I don't know why it gets undefined, since other RBLs work in the same place (eg. URIBL_BLACK) as expected and the result is 1. And another quite simple ruleset... meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) doesn't set __SA4TA3. This was working an SA3.4 as well. Is this wanted behavior or a bug? Since UPGRADE does not contain any information about basic changes for meta rules I assume it is a bug/regression. But if this is wanted how should this be done on SA4? All tests were done on RHEL8.6 by feeding a real EML file and network tests active. So inherited "net" flags should not be an issue. But I also see no reason why a meta rule that does not exclusively depend on "net" sub rules should inherit this flag? Currently it is propagated if a single sub rule has the flag. Not a big issue on my side since I do not plan to use SA without net rules. Greetings, Wolfgang Breyha -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria
