Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 02:54:08PM +0200, Damian wrote: > > > # __SA4 injected inside amavis via $suppl_attrib->{rule_hits} > > > meta SA4 __SA4 > > > score SA4 1 > > > describe SA4 dummy > > yields > > > SA dbg: rules-all: unrun dependencies prevented meta SA4 from > > > running: __SA4 > > The above is slightly misleading, even in SA3 one had to predeclare a > default via > > > meta __SA4 0 > > meta SA4 __SA4 > > score SA4 1 > > describe SA4 dummy > > Such __SA4 default solves the "unrun dependencies prevented" issue, but > still produces hits inconsistently. Uggh.. I even use amavisd, but didn't notice that rule_hits stuff. This is why we need and appreciate rc-testers. :-) https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8061 > Sometimes > > > SA dbg: check: tests=SA4 > > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated > > Total Hits: 1) > sometimes > > SA dbg: check: tests= > > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated > > Total Hits: 1) > > As far as I can tell, the inconsistency comes from the unsorted key > iteration in do_meta_tests() and finish_meta_tests(). The unsorted part doesn't matter, actually it's better this way so we able to notice the problems. When meta handling for this case is fixed, it will always work regardless..
Re: SA 4.0.0-rc3 meta rule results "undefined"
# __SA4 injected inside amavis via $suppl_attrib->{rule_hits} meta SA4 __SA4 score SA4 1 describe SA4 dummy yields SA dbg: rules-all: unrun dependencies prevented meta SA4 from running: __SA4 The above is slightly misleading, even in SA3 one had to predeclare a default via meta __SA4 0 meta SA4 __SA4 score SA4 1 describe SA4 dummy Such __SA4 default solves the "unrun dependencies prevented" issue, but still produces hits inconsistently. Sometimes SA dbg: check: tests=SA4 SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated Total Hits: 1) sometimes SA dbg: check: tests= SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated Total Hits: 1) As far as I can tell, the inconsistency comes from the unsorted key iteration in do_meta_tests() and finish_meta_tests().
Re: SA 4.0.0-rc3 meta rule results "undefined"
Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 There remains a bug in the context of $suppl_attrib. # __SA4 injected inside amavis via $suppl_attrib->{rule_hits} meta SA4 __SA4 score SA4 1 describe SA4 dummy yields SA dbg: rules-all: unrun dependencies prevented meta SA4 from running: __SA4 SA dbg: check: is spam? score=0 required=5 SA dbg: check: tests= SA dbg: check: subtests=
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 13:29, Henrik K wrote: Would need exact sample of ruleset, this is too vague to work on. What are all the __SUBMETAs? Would it be ok to send you my ruleset with test.eml offlist? I do not want to send them here publicly or to bugzilla, because they contain a lot of local branding. You can also need to use -D rules,rules-all to see any "unrun" rules. There is something really odd going on with meta rules on SA4. It's a massive change as described in Bug 7735, more than likely that some bugs can remain. But IMO it's still worth it for the most robust logic especially working with async network rules and stuff. This was not meant as critic. Nothing to complain about the basic idea to make things better;-) To be honest I was a bit surprised that my (not that sophisticated) local ruleset breaks that bad on a rc3. But even that is no criticism on your work! I followed your call to test a release candidate of SA4 ... and here we are. Let's get rid of those issues. Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > > And I've still cases were a simple ">" is not evaluated as well... > > I have > meta __META1 (__SUBMETAX + . + __SUBMETAXN) > with > dbg: rules: ran meta rule __META1 ==> got hit (5) > > I have > meta __META2 (__SUBMETAY + . + __SUBMETAYN) > with > dbg: rules: ran meta rule __META2 ==> got hit (8) > > I have > meta __META_NO (__SUBMETAZ || . || _SUBMETAZN ) > which is not hit. > > And finally > meta RESULT ((! __META_NO) && __META1 > 1 && __META2 > 6) > score RESULT 3 > > And RESULT is not hit/evaluated. Would need exact sample of ruleset, this is too vague to work on. What are all the __SUBMETAs? You can also need to use -D rules,rules-all to see any "unrun" rules. > There is something really odd going on with meta rules on SA4. It's a massive change as described in Bug 7735, more than likely that some bugs can remain. But IMO it's still worth it for the most robust logic especially working with async network rules and stuff.
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 13:22, Henrik K wrote: Remains unclear if you applied the patch from https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ? I applied both on rc3 via .spec: + echo 'Patch #1 (SA4rc3_uridnsbl.patch):' Patch #1 (SA4rc3_uridnsbl.patch): + /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .uridnsbl --fuzz=0 patching file lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm + echo 'Patch #2 (SA4rc3_metafix.patch):' Patch #2 (SA4rc3_metafix.patch): + /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .metafix --fuzz=0 patching file lib/Mail/SpamAssassin/Conf/Parser.pm patching file lib/Mail/SpamAssassin/Conf.pm patching file lib/Mail/SpamAssassin/Plugin/Check.pm patching file t/basic_meta2.t Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote: > On 11/10/2022 12:23, Henrik K wrote: > > Should be fixed in rc4. > > > > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 > > Well, this indeed fixes this test case. But my initial problem which I > though is described in this test case is not fixed yet:( > > So __SA4TA1 gets not evaluated neither in this ... > > header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ > > meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) > > meta __SA4TA2 (__SA4TA1 * 3 ) + 1 > > > > IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL > > is not hit it is undefined. Remains unclear if you applied the patch from https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ?
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 12:23, Henrik K wrote: Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060 Well, this indeed fixes this test case. But my initial problem which I though is described in this test case is not fixed yet:( So __SA4TA1 gets not evaluated neither in this ... header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/ meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL ) meta __SA4TA2 (__SA4TA1 * 3 ) + 1 IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL is not hit it is undefined. nor in this ... I also found that modifying __SA4TA1 to meta __SA4TA1 (((! __SA4T_NOMATCH) || __SPOOFED_URL) && __UNDEF ) fails in the same way and __SA4TA2 is always undefined. showcase. The latter is noted in the dbg output with dbg: rules: meta test __SA4TA1 has undefined dependency '__UNDEF' but IMO this should not stop the meta rule from working, because it would basically destroy the possibility to make local dependencies on mainline rules. You would have to check for "undefined dependecy" on every sa-update then, what makes automatic updates impossible. And I've still cases were a simple ">" is not evaluated as well... I have meta __META1 (__SUBMETAX + . + __SUBMETAXN) with dbg: rules: ran meta rule __META1 ==> got hit (5) I have meta __META2 (__SUBMETAY + . + __SUBMETAYN) with dbg: rules: ran meta rule __META2 ==> got hit (8) I have meta __META_NO (__SUBMETAZ || . || _SUBMETAZN ) which is not hit. And finally meta RESULT ((! __META_NO) && __META1 > 1 && __META2 > 6) score RESULT 3 And RESULT is not hit/evaluated. There is something really odd going on with meta rules on SA4. Greetings, Wolfgang -- Wolfgang Breyha | https://www.blafasel.at/ Vienna University Computer Center | Austria
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:21:23PM +0300, Henrik K wrote: > On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > > On 11/10/2022 06:59, Henrik K wrote: > > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > > > And another quite simple ruleset... > > > > meta __SA4TA3_1 6 > > > > meta __SA4TA3_2 2 > > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > > > Works fine here. > > > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > > > Tried again with > > meta __SA4TA3_1 6 > > meta __SA4TA3_2 2 > > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > score SA4TA3 0.1 > > > > I see > > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > > > But no line für SA4TA3 and no report or final score. > > Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. > Will investigate. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote: > On 11/10/2022 06:59, Henrik K wrote: > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > > > > > And another quite simple ruleset... > > > meta __SA4TA3_1 6 > > > meta __SA4TA3_2 2 > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > > > doesn't set __SA4TA3. This was working an SA3.4 as well. > > > > Works fine here. > > Really? So why doesn't it work here on RHEL7 and RHEL8? > > Tried again with > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > score SA4TA3 0.1 > > I see > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) > > But no line für SA4TA3 and no report or final score. Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. Will investigate.
Re: SA 4.0.0-rc3 meta rule results "undefined"
But no line für SA4TA3 and no report or final score. Same on Debian bookworm: root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3 (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 describe SA4TA3 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 08:57:26.417 [5489] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS Oct 11 08:57:26.417 [5489] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 08:57:26.420 [5489] dbg: markup: mime_encode_header: Yes, score=7.9 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, X-Spam-Status: Yes, score=7.9 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, root@91d4e83fb538:/# Interestingly, these rules produce insonsistent results: root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3 (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 describe SA4TA3 dummy meta SA4TA4 1 score SA4TA4 0.1 describe SA4TA4 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 09:01:29.899 [5503] dbg: rules: ran meta rule SA4TA4 ==> got hit (1) Oct 11 09:01:29.900 [5503] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 09:01:29.901 [5503] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 09:01:29.904 [5503] dbg: learn: auto-learn: not considered as header or body points, no meta deps (SA4TA4) Oct 11 09:01:29.905 [5503] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 Oct 11 09:01:29.905 [5503] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 09:01:29.907 [5503] dbg: markup: mime_encode_header: Yes, score=8.0 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, Oct 11 09:01:29.907 [5503] dbg: markup: [...] \tNO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no X-Spam-Status: Yes, score=8.0 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE, NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no 0.1 SA4TA4 dummy root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 'tests=|SA4TA' Oct 11 09:01:33.798 [5505] dbg: rules: ran meta rule SA4TA4 ==> got hit (1) Oct 11 09:01:33.799 [5505] dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) Oct 11 09:01:33.800 [5505] dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) Oct 11 09:01:33.801 [5505] dbg: rules: ran meta rule SA4TA3 ==> got hit (1) Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as header or body points, no meta deps (SA4TA4) Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as header or body points, no header/body deps (SA4TA3) Oct 11 09:01:33.804 [5505] dbg: check: tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA3,SA4TA4 Oct 11 09:01:33.804 [5505] dbg: check: subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID (Total Subtest Hits: 26 / Deduplicated Total Hits: 26) Oct 11 09:01:33.806 [5505] dbg: markup: mime_encode_header: Yes, score
Re: SA 4.0.0-rc3 meta rule results "undefined"
On 11/10/2022 06:59, Henrik K wrote: On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: And another quite simple ruleset... meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) doesn't set __SA4TA3. This was working an SA3.4 as well. Works fine here. Really? So why doesn't it work here on RHEL7 and RHEL8? Tried again with meta __SA4TA3_1 6 meta __SA4TA3_2 2 meta SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) score SA4TA3 0.1 I see dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2) dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6) But no line für SA4TA3 and no report or final score. Is this wanted behavior or a bug? Since UPGRADE does not contain any information about basic changes for meta rules I assume it is a bug/regression. But if this is wanted how should this be done on SA4? From UPGRADE: - Meta rules no longer use priority values, they are evaluated dynamically when the rules they depend on are finished. (Bug 7735) I don't see the context with the above problem. Maybe that the dynamic evaluation doesn't work in my case and the final rule is never evaluated. But I still don't know how to possibly fix this on my side. Greetings, Wolfgang
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > In case of URIBL_SBL I don't know why it gets undefined, since other RBLs > work in the same place (eg. URIBL_BLACK) as expected and the result is 1. Thanks for the tip, found the problem. Should be fixed in rc4. https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059
Re: SA 4.0.0-rc3 meta rule results "undefined"
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote: > > And another quite simple ruleset... > meta __SA4TA3_1 6 > meta __SA4TA3_2 2 > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1) > doesn't set __SA4TA3. This was working an SA3.4 as well. Works fine here. > Is this wanted behavior or a bug? Since UPGRADE does not contain any > information about basic changes for meta rules I assume it is a > bug/regression. But if this is wanted how should this be done on SA4? >From UPGRADE: - Meta rules no longer use priority values, they are evaluated dynamically when the rules they depend on are finished. (Bug 7735)