Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Henrik K 
On Tue, Oct 11, 2022 at 02:54:08PM +0200, Damian wrote:
> > > # __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
> > > meta  SA4 __SA4
> > > score SA4 1
> > > describe SA4 dummy
> > yields
> > > SA dbg: rules-all: unrun dependencies prevented meta SA4 from
> > > running: __SA4
> 
> The above is slightly misleading, even in SA3 one had to predeclare a
> default via
> 
> > meta __SA4 0
> > meta  SA4 __SA4
> > score SA4 1
> > describe SA4 dummy
> 
> Such __SA4 default solves the "unrun dependencies prevented" issue, but
> still produces hits inconsistently.

Uggh..  I even use amavisd, but didn't notice that rule_hits stuff.  This is
why we need and appreciate rc-testers. :-)

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8061


> Sometimes
> 
> > SA dbg: check: tests=SA4
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
> sometimes
> > SA dbg: check: tests=
> > SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated
> > Total Hits: 1)
> 
> As far as I can tell, the inconsistency comes from the unsorted key
> iteration in do_meta_tests() and finish_meta_tests().

The unsorted part doesn't matter, actually it's better this way so we able
to notice the problems.  When meta handling for this case is fixed, it will
always work regardless..

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Damian 

# __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
meta  SA4 __SA4
score SA4 1
describe SA4 dummy

yields
SA dbg: rules-all: unrun dependencies prevented meta SA4 from 
running: __SA4


The above is slightly misleading, even in SA3 one had to predeclare a 
default via



meta __SA4 0
meta  SA4 __SA4
score SA4 1
describe SA4 dummy


Such __SA4 default solves the "unrun dependencies prevented" issue, but 
still produces hits inconsistently.


Sometimes


SA dbg: check: tests=SA4
SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated 
Total Hits: 1)

sometimes

SA dbg: check: tests=
SA dbg: check: subtests=__SA4 (Total Subtest Hits: 1 / Deduplicated 
Total Hits: 1)


As far as I can tell, the inconsistency comes from the unsorted key 
iteration in do_meta_tests() and finish_meta_tests().

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Damian 

Should be fixed in rc4.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060


There remains a bug in the context of $suppl_attrib.


# __SA4 injected inside amavis via $suppl_attrib->{rule_hits}
meta  SA4 __SA4
score SA4 1
describe SA4 dummy

yields
SA dbg: rules-all: unrun dependencies prevented meta SA4 from running: 
__SA4

SA dbg: check: is spam? score=0 required=5
SA dbg: check: tests=
SA dbg: check: subtests=

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Wolfgang Breyha 

On 11/10/2022 13:29, Henrik K wrote:

Would need exact sample of ruleset, this is too vague to work on.  What are
all the __SUBMETAs?


Would it be ok to send you my ruleset with test.eml offlist? I do not want 
to send them here publicly or to bugzilla, because they contain a lot of 
local branding.



You can also need to use -D rules,rules-all to see any "unrun" rules.


There is something really odd going on with meta rules on SA4.


It's a massive change as described in Bug 7735, more than likely that some
bugs can remain.  But IMO it's still worth it for the most robust logic
especially working with async network rules and stuff.


This was not meant as critic. Nothing to complain about the basic idea to 
make things better;-) To be honest I was a bit surprised that my (not that 
sophisticated) local ruleset breaks that bad on a rc3. But even that is no 
criticism on your work! I followed your call to test a release candidate of 
SA4 ... and here we are. Let's get rid of those issues.


Greetings, Wolfgang

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Henrik K 
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote:
>
> And I've still cases were a simple ">" is not evaluated as well...
> 
> I have
> meta __META1  (__SUBMETAX + . + __SUBMETAXN)
> with
> dbg: rules: ran meta rule __META1 ==> got hit (5)
> 
> I have
> meta __META2  (__SUBMETAY + . + __SUBMETAYN)
> with
> dbg: rules: ran meta rule __META2 ==> got hit (8)
> 
> I have
> meta __META_NO   (__SUBMETAZ || . || _SUBMETAZN )
> which is not hit.
> 
> And finally
> meta  RESULT  ((! __META_NO) && __META1 > 1 && __META2 > 6)
> score RESULT  3
> 
> And RESULT is not hit/evaluated.

Would need exact sample of ruleset, this is too vague to work on.  What are
all the __SUBMETAs?

You can also need to use -D rules,rules-all to see any "unrun" rules.

> There is something really odd going on with meta rules on SA4.

It's a massive change as described in Bug 7735, more than likely that some
bugs can remain.  But IMO it's still worth it for the most robust logic
especially working with async network rules and stuff.

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Wolfgang Breyha 

On 11/10/2022 13:22, Henrik K wrote:

Remains unclear if you applied the patch from
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ?


I applied both on rc3 via .spec:

+ echo 'Patch #1 (SA4rc3_uridnsbl.patch):'
Patch #1 (SA4rc3_uridnsbl.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .uridnsbl --fuzz=0
patching file lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
+ echo 'Patch #2 (SA4rc3_metafix.patch):'
Patch #2 (SA4rc3_metafix.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .metafix --fuzz=0
patching file lib/Mail/SpamAssassin/Conf/Parser.pm
patching file lib/Mail/SpamAssassin/Conf.pm
patching file lib/Mail/SpamAssassin/Plugin/Check.pm
patching file t/basic_meta2.t


Greetings, Wolfgang

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Henrik K 
On Tue, Oct 11, 2022 at 01:09:03PM +0200, Wolfgang Breyha wrote:
> On 11/10/2022 12:23, Henrik K wrote:
> > Should be fixed in rc4.
> > 
> > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060
> 
> Well, this indeed fixes this test case. But my initial problem which I
> though is described in this test case is not fixed yet:(
> 
> So __SA4TA1 gets not evaluated neither in this ...
> > header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/
> > meta   __SA4TA1   (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL )
> > meta   __SA4TA2   (__SA4TA1 * 3 ) + 1
> > 
> > IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL 
> > is not hit it is undefined.

Remains unclear if you applied the patch from
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059 ?

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Wolfgang Breyha 

On 11/10/2022 12:23, Henrik K wrote:

Should be fixed in rc4.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060


Well, this indeed fixes this test case. But my initial problem which I 
though is described in this test case is not fixed yet:(


So __SA4TA1 gets not evaluated neither in this ...

header __SA4T_NOMATCH Subject =~ /dsfasjdhfkjshfjsdklfhaskf/
meta   __SA4TA1   (((! __SA4T_NOMATCH) || __SPOOFED_URL) && URIBL_SBL )
meta   __SA4TA2   (__SA4TA1 * 3 ) + 1

IMO __SA4TA2 should always be 1 or more (4). But in this case if URIBL_SBL is 
not hit it is undefined.


nor in this ...

I also found that modifying __SA4TA1 to
meta   __SA4TA1   (((! __SA4T_NOMATCH) || __SPOOFED_URL) && __UNDEF )
fails in the same way and __SA4TA2 is always undefined. 

showcase. The latter is noted in the dbg output with
dbg: rules: meta test __SA4TA1 has undefined dependency '__UNDEF'
but IMO this should not stop the meta rule from working, because it would 
basically destroy the possibility to make local dependencies on mainline 
rules. You would have to check for "undefined dependecy" on every sa-update 
then, what makes automatic updates impossible.



And I've still cases were a simple ">" is not evaluated as well...

I have
meta __META1  (__SUBMETAX + . + __SUBMETAXN)
with
dbg: rules: ran meta rule __META1 ==> got hit (5)

I have
meta __META2  (__SUBMETAY + . + __SUBMETAYN)
with
dbg: rules: ran meta rule __META2 ==> got hit (8)

I have
meta __META_NO   (__SUBMETAZ || . || _SUBMETAZN )
which is not hit.

And finally
meta  RESULT  ((! __META_NO) && __META1 > 1 && __META2 > 6)
score RESULT  3

And RESULT is not hit/evaluated.

There is something really odd going on with meta rules on SA4.

Greetings, Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Henrik K 
On Tue, Oct 11, 2022 at 12:21:23PM +0300, Henrik K wrote:
> On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote:
> > On 11/10/2022 06:59, Henrik K wrote:
> > > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote:
> > > > 
> > > > And another quite simple ruleset...
> > > > meta __SA4TA3_1  6
> > > > meta __SA4TA3_2  2
> > > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
> > > > doesn't set __SA4TA3. This was working an SA3.4 as well.
> > > 
> > > Works fine here.
> > 
> > Really? So why doesn't it work here on RHEL7 and RHEL8?
> > 
> > Tried again with
> > meta __SA4TA3_1  6
> > meta __SA4TA3_2  2
> > meta  SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
> > score SA4TA3 0.1
> > 
> > I see
> > dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2)
> > dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6)
> > 
> > But no line für SA4TA3 and no report or final score.
> 
> Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. 
> Will investigate.

Should be fixed in rc4.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8060

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Henrik K 
On Tue, Oct 11, 2022 at 10:48:26AM +0200, Wolfgang Breyha wrote:
> On 11/10/2022 06:59, Henrik K wrote:
> > On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote:
> > > 
> > > And another quite simple ruleset...
> > > meta __SA4TA3_1  6
> > > meta __SA4TA3_2  2
> > > meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
> > > doesn't set __SA4TA3. This was working an SA3.4 as well.
> > 
> > Works fine here.
> 
> Really? So why doesn't it work here on RHEL7 and RHEL8?
> 
> Tried again with
> meta __SA4TA3_1  6
> meta __SA4TA3_2  2
> meta  SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
> score SA4TA3 0.1
> 
> I see
> dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2)
> dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6)
> 
> But no line für SA4TA3 and no report or final score.

Ok I can now reproduce it, only sometimes it hits SA4TA3, sometimes not. 
Will investigate.

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Damian 
But no line für SA4TA3 and no report or final score. 


Same on Debian bookworm:


root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf
meta __SA4TA3_1  6
meta __SA4TA3_2  2
meta  SA4TA3    (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
score SA4TA3 0.1
describe SA4TA3 dummy
root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 
'tests=|SA4TA'
Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_2 
==> got hit (2)
Oct 11 08:57:26.413 [5489] dbg: rules: ran meta rule __SA4TA3_1 
==> got hit (6)
Oct 11 08:57:26.417 [5489] dbg: check: 
tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS
Oct 11 08:57:26.417 [5489] dbg: check: 
subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID 
(Total Subtest Hits: 26 / Deduplicated Total Hits: 26)
Oct 11 08:57:26.420 [5489] dbg: markup: mime_encode_header: Yes, 
score=7.9 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE,
X-Spam-Status: Yes, score=7.9 required=5.0 
tests=EMPTY_MESSAGE,MISSING_DATE,

root@91d4e83fb538:/#


Interestingly, these rules produce insonsistent results:


root@91d4e83fb538:/# cat /etc/spamassassin/70_meta_undefined.cf
meta __SA4TA3_1  6
meta __SA4TA3_2  2
meta  SA4TA3    (__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
score SA4TA3 0.1
describe SA4TA3 dummy
meta SA4TA4 1
score SA4TA4 0.1
describe SA4TA4 dummy
root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 
'tests=|SA4TA'
Oct 11 09:01:29.899 [5503] dbg: rules: ran meta rule SA4TA4 ==> 
got hit (1)
Oct 11 09:01:29.900 [5503] dbg: rules: ran meta rule __SA4TA3_2 
==> got hit (2)
Oct 11 09:01:29.901 [5503] dbg: rules: ran meta rule __SA4TA3_1 
==> got hit (6)
Oct 11 09:01:29.904 [5503] dbg: learn: auto-learn: not considered as 
header or body points, no meta deps (SA4TA4)
Oct 11 09:01:29.905 [5503] dbg: check: 
tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4
Oct 11 09:01:29.905 [5503] dbg: check: 
subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID 
(Total Subtest Hits: 26 / Deduplicated Total Hits: 26)
Oct 11 09:01:29.907 [5503] dbg: markup: mime_encode_header: Yes, 
score=8.0 required=5.0 tests=EMPTY_MESSAGE,MISSING_DATE,
Oct 11 09:01:29.907 [5503] dbg: markup: [...] 
\tNO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no
X-Spam-Status: Yes, score=8.0 required=5.0 
tests=EMPTY_MESSAGE,MISSING_DATE,

    NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA4 autolearn=no
 0.1 SA4TA4 dummy
root@91d4e83fb538:/# spamassassin -Dall - &1 | grep -E 
'tests=|SA4TA'
Oct 11 09:01:33.798 [5505] dbg: rules: ran meta rule SA4TA4 ==> 
got hit (1)
Oct 11 09:01:33.799 [5505] dbg: rules: ran meta rule __SA4TA3_2 
==> got hit (2)
Oct 11 09:01:33.800 [5505] dbg: rules: ran meta rule __SA4TA3_1 
==> got hit (6)
Oct 11 09:01:33.801 [5505] dbg: rules: ran meta rule SA4TA3 ==> 
got hit (1)
Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as 
header or body points, no meta deps (SA4TA4)
Oct 11 09:01:33.804 [5505] dbg: learn: auto-learn: not considered as 
header or body points, no header/body deps (SA4TA3)
Oct 11 09:01:33.804 [5505] dbg: check: 
tests=EMPTY_MESSAGE,MISSING_DATE,MISSING_FROM,MISSING_HEADERS,MISSING_MID,MISSING_SUBJECT,NO_HEADERS_MESSAGE,NO_RECEIVED,NO_RELAYS,SA4TA3,SA4TA4
Oct 11 09:01:33.804 [5505] dbg: check: 
subtests=__DKIM_DEPENDABLE,__EMPTY_BODY,__ENV_AND_HDR_FROM_MATCH,__GATED_THROUGH_RCVD_REMOVER,__GB_TO_ADDR,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_128,__KAM_BODY_LENGTH_LT_256,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_128,__LCL__KAM_BODY_LENGTH_LT_512,__MISSING_REF,__MISSING_REPLY,__NOT_SPOOFED,__PDS_MSG_1024,__PDS_MSG_512,__SA4TA3_1,__SA4TA3_2,__SUBJECT_EMPTY,__SUBJ_SHORT,__TO_NO_ARROWS_R,__T_PDS_MSG_512,__UNUSABLE_MSGID 
(Total Subtest Hits: 26 / Deduplicated Total Hits: 26)
Oct 11 09:01:33.806 [5505] dbg: markup: mime_encode_header: Yes,

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-11 Thread Wolfgang Breyha 

On 11/10/2022 06:59, Henrik K wrote:

On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote:


And another quite simple ruleset...
meta __SA4TA3_1  6
meta __SA4TA3_2  2
meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
doesn't set __SA4TA3. This was working an SA3.4 as well.


Works fine here.


Really? So why doesn't it work here on RHEL7 and RHEL8?

Tried again with
meta __SA4TA3_1  6
meta __SA4TA3_2  2
meta  SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
score SA4TA3 0.1

I see
dbg: rules: ran meta rule __SA4TA3_2 ==> got hit (2)
dbg: rules: ran meta rule __SA4TA3_1 ==> got hit (6)

But no line für SA4TA3 and no report or final score.


Is this wanted behavior or a bug? Since UPGRADE does not contain any
information about basic changes for meta rules I assume it is a
bug/regression. But if this is wanted how should this be done on SA4?


 From UPGRADE:

- Meta rules no longer use priority values, they are evaluated
   dynamically when the rules they depend on are finished. (Bug 7735)


I don't see the context with the above problem. Maybe that the dynamic 
evaluation doesn't work in my case and the final rule is never evaluated. 
But I still don't know how to possibly fix this on my side.


Greetings,
Wolfgang

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-10 Thread Henrik K 
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote:
> 
> In case of URIBL_SBL I don't know why it gets undefined, since other RBLs
> work in the same place (eg. URIBL_BLACK) as expected and the result is 1.

Thanks for the tip, found the problem. Should be fixed in rc4.

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=8059

Re: SA 4.0.0-rc3 meta rule results "undefined"

2022-10-10 Thread Henrik K 
On Tue, Oct 11, 2022 at 12:50:38AM +0200, Wolfgang Breyha wrote:
>
> And another quite simple ruleset...
> meta __SA4TA3_1  6
> meta __SA4TA3_2  2
> meta __SA4TA3(__SA4TA3_1 > 2) && (__SA4TA3_2 > 1)
> doesn't set __SA4TA3. This was working an SA3.4 as well.

Works fine here.

> Is this wanted behavior or a bug? Since UPGRADE does not contain any
> information about basic changes for meta rules I assume it is a
> bug/regression. But if this is wanted how should this be done on SA4?

>From UPGRADE:

- Meta rules no longer use priority values, they are evaluated
  dynamically when the rules they depend on are finished. (Bug 7735)