Re: Spamassassin always says DKIM_INVALID

[Henrik K]

On Sat, Jan 18, 2020 at 02:54:27PM +0100, Alex Woick wrote:
> Henrik K schrieb am 18.01.2020 um 08:15:
> >On Sat, Jan 18, 2020 at 06:56:53AM +0200, Henrik K wrote:
> >>On Tue, Jan 14, 2020 at 02:38:06PM +0100, Alex Woick wrote:
> >>>Link to complete message:
> >>>[2]https://pastebin.com/raw/1DLtnuRX
> >>>
> >>>Spamassassin is running as spamc/spamd, and is embedded in Postfix with
> >>>spamass-milter. System is running on CentOS 7.
> >>>...
> >>>Any idea how to find out why Spamassassin isn't able to successfully verify
> >>>dkim sigs, while at the same time Opendkim says it's valid? I just 
> >>>activated
> >>>the dkim plugin of Spamassassin but didn't configure anything dkim-related,
> >>>since there is nothing specific to do.
> >>Naturally first step to debug this, would be enabling debugging.
> >>
> >>Does SA fail if you run it from command line?
> >>
> >>spamassassin -t -D dkim < message
> >>
> >>If not, then add the same "-D dkim" to spamd, probably
> >>/etc/sysconfig/spamassassin if using CentOS package?
> >Actually I already found the cause, spamass-milter is removing CR from
> >wrapped headers, and some 3.4.3 changes made things break.  I'm not sure
> >about the fix, I need some more eyes on the bug below please.  :-)
> >
> >https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7785
> >
> Thanks for picking this up as bug. As far as I know, dkim signature
> generation takes a header as it is, regardless of the header being
> rfc-compliant or not. So if a line wrapping is LF only, and even if this is
> not rfc-5322-compliant, it must be fed this way into the dkim signature
> generation and verification.
> An issue is probably, that you may need 2 versions of wrapped headers. One
> with the original, possbly non-rfc compliant data for processing with
> modules like dkim, and one with "fixed" data to allow normalized and easier
> rule processing.

Wrote a patch for spamass-milter as per the bug..  took much more time than
I wanted, ugh.  Someone else can harass maintainers/distros to update it
now.  :-)

Re: Spamassassin always says DKIM_INVALID

[Alex Woick]

Henrik K schrieb am 18.01.2020 um 08:15:

On Sat, Jan 18, 2020 at 06:56:53AM +0200, Henrik K wrote:

On Tue, Jan 14, 2020 at 02:38:06PM +0100, Alex Woick wrote:

Link to complete message:
[2]https://pastebin.com/raw/1DLtnuRX

Spamassassin is running as spamc/spamd, and is embedded in Postfix with
spamass-milter. System is running on CentOS 7.
...
Any idea how to find out why Spamassassin isn't able to successfully verify
dkim sigs, while at the same time Opendkim says it's valid? I just activated
the dkim plugin of Spamassassin but didn't configure anything dkim-related,
since there is nothing specific to do.

Naturally first step to debug this, would be enabling debugging.

Does SA fail if you run it from command line?

spamassassin -t -D dkim < message

If not, then add the same "-D dkim" to spamd, probably
/etc/sysconfig/spamassassin if using CentOS package?

Actually I already found the cause, spamass-milter is removing CR from
wrapped headers, and some 3.4.3 changes made things break.  I'm not sure
about the fix, I need some more eyes on the bug below please.  :-)

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7785

Thanks for picking this up as bug. As far as I know, dkim signature 
generation takes a header as it is, regardless of the header being 
rfc-compliant or not. So if a line wrapping is LF only, and even if this 
is not rfc-5322-compliant, it must be fed this way into the dkim 
signature generation and verification.
An issue is probably, that you may need 2 versions of wrapped headers. 
One with the original, possbly non-rfc compliant data for processing 
with modules like dkim, and one with "fixed" data to allow normalized 
and easier rule processing.


Alex

Re: Spamassassin always says DKIM_INVALID

[Henrik K]

On Sat, Jan 18, 2020 at 06:56:53AM +0200, Henrik K wrote:
> On Tue, Jan 14, 2020 at 02:38:06PM +0100, Alex Woick wrote:
> > 
> > Link to complete message:
> > [2]https://pastebin.com/raw/1DLtnuRX
> >
> > Spamassassin is running as spamc/spamd, and is embedded in Postfix with
> > spamass-milter. System is running on CentOS 7.
> >...
> > Any idea how to find out why Spamassassin isn't able to successfully verify
> > dkim sigs, while at the same time Opendkim says it's valid? I just activated
> > the dkim plugin of Spamassassin but didn't configure anything dkim-related,
> > since there is nothing specific to do.
> 
> Naturally first step to debug this, would be enabling debugging.
> 
> Does SA fail if you run it from command line?
> 
> spamassassin -t -D dkim < message
> 
> If not, then add the same "-D dkim" to spamd, probably
> /etc/sysconfig/spamassassin if using CentOS package?

Actually I already found the cause, spamass-milter is removing CR from
wrapped headers, and some 3.4.3 changes made things break.  I'm not sure
about the fix, I need some more eyes on the bug below please.  :-)

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7785

Re: Spamassassin always says DKIM_INVALID

[Henrik K]

On Tue, Jan 14, 2020 at 02:38:06PM +0100, Alex Woick wrote:
> 
> Link to complete message:
> [2]https://pastebin.com/raw/1DLtnuRX
>
> Spamassassin is running as spamc/spamd, and is embedded in Postfix with
> spamass-milter. System is running on CentOS 7.
>...
> Any idea how to find out why Spamassassin isn't able to successfully verify
> dkim sigs, while at the same time Opendkim says it's valid? I just activated
> the dkim plugin of Spamassassin but didn't configure anything dkim-related,
> since there is nothing specific to do.

Naturally first step to debug this, would be enabling debugging.

Does SA fail if you run it from command line?

spamassassin -t -D dkim < message

If not, then add the same "-D dkim" to spamd, probably
/etc/sysconfig/spamassassin if using CentOS package?

Re: Spamassassin always says DKIM_INVALID

[Martin Gregorie]

On Sat, 2020-01-18 at 01:29 +0200, Jari Fredriksson wrote:
> On 14.1.2020 15.38, Alex Woick wrote:
> > Spamassassin (3.4.3, the same with previous) declares all or almost 
> > all the incoming DKIM-signed messages as DKIM_INVALID, and I'm not 
> > understanding why.
> > I'm running opendkim on the mail server as milter with Postfix, and 
> > the opendkim headers say the same dkim signatures are all valid.
> > 
> > Example headers of some mail from this list.
> > Opendkim says ok:
> > Authentication-Results: mail.wombaz.de;
> > dkim=pass (2048-bit key) 
> > header.d=linkcheck.co.ukheader.i=@linkcheck.co.uk  header.b="PXrrNHd
> > B"
> > 
> > But Spamassassin says it's invalid:
> > X-Spam-Status: No, score=-15.5 required=5.0
> > tests=BAYES_00,DKIM_ADSP_ALL,
> > DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,
> > MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS,TXREP
> > ,
> > USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no
> > version=3.4.3
> 
> I had the same problem on my mail server, while the server only 1 Gb
> and 
> was an old box. I swapped it to a 4 Gb box and installed OS and SA
> and 
> all as new install.
> 
> Magically the problem went away.
> 
> One more thing: I got DKIM_VALID & DKIM_VALID_AU allright for a day 
> after reboot, but it the started to be DKIM_INVALID. I set the mail 
> server to reboot once a day and it worked. But current system works
> fine 
> without any artificial reboots.
> 
That looks like a combination of:

(a) buffer truncation when memory limits are reached. IOW SA can't get
the full buffer size it asked for, and so truncates the message it was
putting into the buffer rather than aborting on a buffer overflow. Doing
this would certainly screw a checksum.

(b) there's some sort of memory leak, i.e. when releasing a dynamically
requested piece of memory doesn't return all of it, which could slowly
shrink the process's available heap space OR there's code that is
failing to return previously claimed heap space. Something like that
would explain your second issue, which you got round rebooting the
system before SA ran out of heap memory.

Both are things you might expect to see in badly written C programs
and/or C programs whose testing skimped on edge case testing,
particularly when the code uses calls of malloc(), free() and friends to
manage dynamic heap memory use. Equally these are things that I would
not expect to see in Java code because the JVM has a decent garbage
collector and anyway, errors of that are treated as fatal and so would
cause program termination with a diagnostic stack dump. 

However, I'm not familiar enough with Perl to know how it behaves in
these circumstances. 

Still, I hope the above helps with ideas about what to look for. On a
UNIX/Linux box 'top' should show the program size expanding over time of
stack space isn't being released correctly. I've forgotten how you'd
trouble-shoot a Windows system - haven't touched it for over 15 years. 

Martin

Re: Spamassassin always says DKIM_INVALID

[Jari Fredriksson]

On 14.1.2020 15.38, Alex Woick wrote:
Spamassassin (3.4.3, the same with previous) declares all or almost 
all the incoming DKIM-signed messages as DKIM_INVALID, and I'm not 
understanding why.
I'm running opendkim on the mail server as milter with Postfix, and 
the opendkim headers say the same dkim signatures are all valid.


Example headers of some mail from this list.
Opendkim says ok:
Authentication-Results: mail.wombaz.de;
dkim=pass (2048-bit key) header.d=linkcheck.co.ukheader.i=@linkcheck.co.uk  
header.b="PXrrNHdB"

But Spamassassin says it's invalid:
X-Spam-Status: No, score=-15.5 required=5.0 tests=BAYES_00,DKIM_ADSP_ALL,
DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,
MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS,TXREP,
USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.3


I had the same problem on my mail server, while the server only 1 Gb and 
was an old box. I swapped it to a 4 Gb box and installed OS and SA and 
all as new install.


Magically the problem went away.

One more thing: I got DKIM_VALID & DKIM_VALID_AU allright for a day 
after reboot, but it the started to be DKIM_INVALID. I set the mail 
server to reboot once a day and it worked. But current system works fine 
without any artificial reboots.


Go figure.

Re: Spamassassin always says DKIM_INVALID

[David Jones]

Do you have anything modifying the Subject or altering the message body (like a 
signature/disclaimer or external email warning) after opendkim and before the 
spamass-milter?

From: Alex Woick 
Date: Tuesday, January 14, 2020 at 7:38 AM
To: "users@spamassassin.apache.org" 
Subject: Spamassassin always says DKIM_INVALID

Spamassassin (3.4.3, the same with previous) declares all or almost all the 
incoming DKIM-signed messages as DKIM_INVALID, and I'm not understanding why.
I'm running opendkim on the mail server as milter with Postfix, and the 
opendkim headers say the same dkim signatures are all valid.

Example headers of some mail from this list.
Opendkim says ok:

Authentication-Results: mail.wombaz.de;

dkim=pass (2048-bit key) header.d=linkcheck.co.uk 
header.i=@linkcheck.co.uk header.b="PXrrNHdB"


But Spamassassin says it's invalid:

X-Spam-Status: No, score=-15.5 required=5.0 tests=BAYES_00,DKIM_ADSP_ALL,

DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,

MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS,TXREP,

USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.3


Link to complete message:
https://pastebin.com/raw/1DLtnuRX

Spamassassin is running as spamc/spamd, and is embedded in Postfix with 
spamass-milter. System is running on CentOS 7.

Postfix milter config is this:

smtpd_milters =

  unix:/var/run/opendkim-postfix/sock,

  unix:/var/run/opendmarc-postfix/sock,

  unix:/var/run/clamav-milter/clamav-milter.socket,

  unix:/run/spamass-milter/postfix/sock


Any idea how to find out why Spamassassin isn't able to successfully verify 
dkim sigs, while at the same time Opendkim says it's valid? I just activated 
the dkim plugin of Spamassassin but didn't configure anything dkim-related, 
since there is nothing specific to do.

Alex