Re: sa-update

[Matt Anton via users]

On 17 Mar 2023, at 16:03, Bill Cole wrote:

Correct. We've had a problem with RuleQA, in that we have not had 
enough spam in the masscheck submissions to run the rescoring 
properly. I'm not sure whose submissions have dried up...


The existing list of active rules and their scores is fine for now and 
won't likely be problematic in the near to mid term. Obviously we hope 
to resolve the underlying lack of data and to resume proper QA ASAP.


Thanks Bill for letting us know what’s going on!

cheers,

--
matt [at] lv223.org
GPG key ID: 7D91A8CA

Re: sa-update

[Bill Cole]

On 2023-03-17 at 05:11:30 UTC-0400 (Fri, 17 Mar 2023 10:11:30 +0100)
SA list 
is rumored to have said:


Hello,

I didn't get an update since 5 March (1908044).


Correct. We've had a problem with RuleQA, in that we have not had enough 
spam in the masscheck submissions to run the rescoring properly. I'm not 
sure whose submissions have dried up...


The existing list of active rules and their scores is fine for now and 
won't likely be problematic in the near to mid term. Obviously we hope 
to resolve the underlying lack of data and to resume proper QA ASAP.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

sa-update

[SA list]

Hello,

I didn't get an update since 5 March (1908044).

FreeBSD 12.4-RELEASE-p2
spamassassin 4.0.0_2

Mar 17 08:19:41.458 [41854] dbg: channel: metadata version = 1908044, 
from file /var/db/spamassassin/4.00/updates_spamassassin_org.cf
Mar 17 08:19:41.471 [41854] dbg: dns: 0.0.4.updates.spamassassin.org => 
1908044, parsed as 1908044
Mar 17 08:19:41.471 [41854] dbg: channel: current version is 1908044, 
new version is 1908044, skipping channel

sa-update note

[Jared Hall]

Just FYI,

Update available for channel updates.spamassassin.org: 1899935 -> 1900065
http: (curl) GET http://sa-update.verein-clean.net/1900065.tar.gz, success
http: (curl) GET 
http://sa-update.verein-clean.net/1900065.tar.gz.sha512, FAILED, status: 
exit 22
http: (curl) GET 
http://sa-update.verein-clean.net/1900065.tar.gz.sha256, FAILED, status: 
exit 22


-- Jared Hall

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[RW]

On Sun, 25 Apr 2021 13:34:16 -0400
Steve Dondley wrote:


> I’m experimenting with writing my own rules. My machines are using SA
> 3.4.4 so I want to use the 3.4.4 rules.

There is only one set of rules, "if" statements handle any differences.

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Steve Dondley]

On 2021-04-25 01:47 PM, Henrik K wrote:

On Sun, Apr 25, 2021 at 01:28:31PM -0400, Steve Dondley wrote:


> mass-check -c parameter expects to find every config file in that single
> directory.  Now it's missing spamassassin updates and specifically
> 20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
> temporarily, but I'd rather make a completely separate directory that
> should
> include only the relevant *.pre and *.cf files you need for the scan.

OK, thanks. So I created a directory: /root/spam_rules

I copied over every .cf and .pre file from /etc/spamassassin into that 
dir
as well as every .cf and .pre file inside 
/var/lib/spamassassin/3.004004


Don't blindly copy all .cf files from /etc/spamassassin, there's no 
point

using AWL or bayes etc from that config.


OK. I'm setting up a test machine to duplicate a live machine. Not sure 
if that makes a difference or not.




I ran mass-check with "-c=~/root/spam_rules" and now get a ton of 
these

errors:


config: configuration file "/root/spam_rules/20_advance_fee.cf" 
requires
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe
you need to use the -C switch, or remove the old config files? 
Skipping this

file at
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line

414.


svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk
spamassassin-trunk


I have the trunk downloaded via svn, but I have no idea how to find the 
revision for 3.4.4 and roll back to it.


I ended up just downloading the 3.4.4 version from metacpan. After 
downloading and using this version, the errors have gone away.

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Steve Dondley]

spamassassin -V reports: "SpamAssassin version 3.4.4"

I imagine I have to checkout an older 3.4.4 point version from SVN and
use the mass-check command from that. It's been ages since I've used
SVN.

How can I get to the older version via SVN?


I solved this by downloading version 3.4.4 of SA from metacpan and then 
dropping the masses/ dir with the mass-check tool from SVN into the 
3.4.4 version.

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Henrik K]

On Sun, Apr 25, 2021 at 01:28:31PM -0400, Steve Dondley wrote:
> 
> > mass-check -c parameter expects to find every config file in that single
> > directory.  Now it's missing spamassassin updates and specifically
> > 20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
> > temporarily, but I'd rather make a completely separate directory that
> > should
> > include only the relevant *.pre and *.cf files you need for the scan.
> 
> OK, thanks. So I created a directory: /root/spam_rules
> 
> I copied over every .cf and .pre file from /etc/spamassassin into that dir
> as well as every .cf and .pre file inside /var/lib/spamassassin/3.004004

Don't blindly copy all .cf files from /etc/spamassassin, there's no point
using AWL or bayes etc from that config.

> I ran mass-check with "-c=~/root/spam_rules" and now get a ton of these
> errors:
> 
> 
> config: configuration file "/root/spam_rules/20_advance_fee.cf" requires
> version 3.004004 of SpamAssassin, but this is code version 3.004006. Maybe
> you need to use the -C switch, or remove the old config files? Skipping this
> file at
> /root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm line
> 414.

svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk 
spamassassin-trunk

Use the rules-directory found in spamassassin-trunk/rules, instead of
sa-update directory.

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Axb]

On 4/25/21 7:34 PM, Steve Dondley wrote:




On Apr 25, 2021, at 1:31 PM, Axb  wrote:

What are you trying to do?
run masscheck for your rules or for the SA project?


I’m experimenting with writing my own rules. My machines are using SA 3.4.4 so 
I want to use the 3.4.4 rules.



this may give you some pointers

https://svn.apache.org/repos/asf/spamassassin/trunk/masses/contrib/automasscheck-minimal/

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Steve Dondley]

> On Apr 25, 2021, at 1:31 PM, Axb  wrote:
> 
> What are you trying to do?
> run masscheck for your rules or for the SA project?

I’m experimenting with writing my own rules. My machines are using SA 3.4.4 so 
I want to use the 3.4.4 rules.

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Axb]

What are you trying to do?
run masscheck for your rules or for the SA project?

On 4/25/21 7:28 PM, Steve Dondley wrote:



mass-check -c parameter expects to find every config file in that single
directory.  Now it's missing spamassassin updates and specifically
20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
temporarily, but I'd rather make a completely separate directory that 
should

include only the relevant *.pre and *.cf files you need for the scan.


OK, thanks. So I created a directory: /root/spam_rules

I copied over every .cf and .pre file from /etc/spamassassin into that 
dir as well as every .cf and .pre file inside 
/var/lib/spamassassin/3.004004


I ran mass-check with "-c=~/root/spam_rules" and now get a ton of these 
errors:



config: configuration file "/root/spam_rules/20_advance_fee.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_body_tests.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_compensate.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_dnsbl_tests.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.



spamassassin -V reports: "SpamAssassin version 3.4.4"

I imagine I have to checkout an older 3.4.4 point version from SVN and 
use the mass-check command from that. It's been ages since I've used SVN.


How can I get to the older version via SVN?

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Steve Dondley]

mass-check -c parameter expects to find every config file in that 
single

directory.  Now it's missing spamassassin updates and specifically
20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
temporarily, but I'd rather make a completely separate directory that 
should

include only the relevant *.pre and *.cf files you need for the scan.


OK, thanks. So I created a directory: /root/spam_rules

I copied over every .cf and .pre file from /etc/spamassassin into that 
dir as well as every .cf and .pre file inside 
/var/lib/spamassassin/3.004004


I ran mass-check with "-c=~/root/spam_rules" and now get a ton of these 
errors:



config: configuration file "/root/spam_rules/20_advance_fee.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_body_tests.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_compensate.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.
config: configuration file "/root/spam_rules/20_dnsbl_tests.cf" requires 
version 3.004004 of SpamAssassin, but this is code version 3.004006. 
Maybe you need to use the -C switch, or remove the old config files? 
Skipping this file at 
/root/spamassassin-3.4/masses/../lib/Mail/SpamAssassin/Conf/Parser.pm 
line 414.



spamassassin -V reports: "SpamAssassin version 3.4.4"

I imagine I have to checkout an older 3.4.4 point version from SVN and 
use the mass-check command from that. It's been ages since I've used 
SVN.


How can I get to the older version via SVN?

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Henrik K]

On Sun, Apr 25, 2021 at 07:38:44PM +0300, Henrik K wrote:
> On Sun, Apr 25, 2021 at 12:17:51PM -0400, Steve Dondley wrote:
> > I'm running this command:
> > 
> > ./mass-check -n --rules='^LOCAL_AWK_INTRO' -o ham:dir:/spam/Maildir/.INBOX*
> > -c=/etc/spamassassin/ | grep '.  1'
> > 
> > 
> > Everything appears to work as expected but I'm getting this warning/error
> > when I do:
> > 
> > "config: registryboundaries: no tlds defined, need to run sa-update"
> > 
> > Running sa-update doesn't fix the problem and a search didn't uncover
> > anything useful.
> 
> mass-check -c parameter expects to find every config file in that single
> directory.  Now it's missing spamassassin updates and specifically
> 20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
> temporarily, but I'd rather make a completely separate directory that should
> include only the relevant *.pre and *.cf files you need for the scan.

Oh yeah and actually you should never use -c=/etc/spamassassin, as the
mass-check can then potentially then trash your AWL/bayes etc since it's
using your main config.. 

Sorry, it's more of a developer tool and I know the wiki guides are terribly
outdated.  :-)

Re: Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Henrik K]

On Sun, Apr 25, 2021 at 12:17:51PM -0400, Steve Dondley wrote:
> I'm running this command:
> 
> ./mass-check -n --rules='^LOCAL_AWK_INTRO' -o ham:dir:/spam/Maildir/.INBOX*
> -c=/etc/spamassassin/ | grep '.  1'
> 
> 
> Everything appears to work as expected but I'm getting this warning/error
> when I do:
> 
> "config: registryboundaries: no tlds defined, need to run sa-update"
> 
> Running sa-update doesn't fix the problem and a search didn't uncover
> anything useful.

mass-check -c parameter expects to find every config file in that single
directory.  Now it's missing spamassassin updates and specifically
20_aux_tlds.cf from there.  You could copy it to /etc/spamassassin
temporarily, but I'd rather make a completely separate directory that should
include only the relevant *.pre and *.cf files you need for the scan.

Getting "config: registryboundaries: no tlds defined, need to run sa-update" message when running mass-check

[Steve Dondley]

I'm running this command:

./mass-check -n --rules='^LOCAL_AWK_INTRO' -o 
ham:dir:/spam/Maildir/.INBOX*  -c=/etc/spamassassin/ | grep '.  1'



Everything appears to work as expected but I'm getting this 
warning/error when I do:


"config: registryboundaries: no tlds defined, need to run sa-update"

Running sa-update doesn't fix the problem and a search didn't uncover 
anything useful.

Re: sa-update error 3 no mirrors.sought.rules.yerp.org

[Kenneth Porter]

--On Sunday, March 14, 2021 1:23 PM +1000 Simon Wilson 
 wrote:



You've not stated your OS but on a RHEL/CentOS 7 box the correct way to
remove is to go to /etc/mail/spamassassin/channel.d and delete
sought.conf.


RHEL bugzilla for the issue:

Re: sa-update error 3 no mirrors.sought.rules.yerp.org

[Simon Wilson]

- Message from a...@onet.eu -
   Date: Fri, 12 Mar 2021 13:16:25 +0100
   From: a...@onet.eu
Subject: sa-update error 3 no mirrors.sought.rules.yerp.org
 To: users@spamassassin.apache.org



Hi,
I'm getting this from cron since two days:
channel: no 'mirrors.sought.rules.yerp.org' record found, channel failed
11-Mar-2021 05:46:59: SpamAssassin: Unknown error code 3 from sa-update
ls /var/lib/spamassassin/3.004000/
sought_rules_yerp_org
sought_rules_yerp_org.cf
updates_spamassassin_org
updates_spamassassin_org.cf
rm -rf /var/lib/spamassassin/3.004002/
and then
sa-update -D
gives no errors,
but when I try to run cron job it takes a long time and does not finish:
/usr/share/spamassassin/sa-update.cron 2>&1 | tee -a /var/log/sa-update.log



The cron script contains a "sleep" function to randomise the time at  
which clients hit the servers for updates. You are running into the  
sleep when you run the script manually...



 
Greetings,
atat [at] onet.eu



- End message from a...@onet.eu -

See https://cwiki.apache.org/confluence/display/SPAMASSASSIN/SoughtRules

"Update: this is no longer active, and should not be used."

You've not stated your OS but on a RHEL/CentOS 7 box the correct way  
to remove is to go to /etc/mail/spamassassin/channel.d and delete  
sought.conf.




--
Simon Wilson
M: 0400 12 11 16

Re: sa-update error 3 no mirrors.sought.rules.yerp.org

[Axb]

Sought rules have been deprecated at least 5 years ago.
you can remove that part of the config.

h2h
Axb

On 3/12/21 1:16 PM, a...@onet.eu wrote:

Hi,
I'm getting this from cron since two days:
channel: no 'mirrors.sought.rules.yerp.org' record found, channel failed
11-Mar-2021 05:46:59: SpamAssassin: Unknown error code 3 from sa-update
ls /var/lib/spamassassin/3.004000/
sought_rules_yerp_org
sought_rules_yerp_org.cf
updates_spamassassin_org
updates_spamassassin_org.cf
rm -rf /var/lib/spamassassin/3.004002/
and then
sa-update -D
gives no errors,
but when I try to run cron job it takes a long time and does not finish:
/usr/share/spamassassin/sa-update.cron 2>&1 | tee -a /var/log/sa-update.log
  
Greetings,

atat [at] onet.eu

sa-update error 3 no mirrors.sought.rules.yerp.org

[atat]

Hi,
I'm getting this from cron since two days:
channel: no 'mirrors.sought.rules.yerp.org' record found, channel failed
11-Mar-2021 05:46:59: SpamAssassin: Unknown error code 3 from sa-update
ls /var/lib/spamassassin/3.004000/
sought_rules_yerp_org
sought_rules_yerp_org.cf
updates_spamassassin_org
updates_spamassassin_org.cf
rm -rf /var/lib/spamassassin/3.004002/
and then
sa-update -D
gives no errors,
but when I try to run cron job it takes a long time and does not finish:
/usr/share/spamassassin/sa-update.cron 2>&1 | tee -a /var/log/sa-update.log
 
Greetings,
atat [at] onet.eu

Re: Fedora sa-update and systemd randomized timer

[RW]

On Thu, 31 Dec 2020 16:54:15 -0800
Kenneth Porter wrote:

> With the discussion of the KAM channel and Fedora's sa-update script
> that uses directory-based channel configuration, I went snooping into
> their script and systemd units. It looks like sa-update.cron has a
> 2-hour random delay before it looks for updates. I'm thinking it
> would be nice to move that delay into the systemd timer unit using
> the RandomizedDelaySec feature. This would allow the script to be run
> from the command line for immediate testing without waiting for the
> delay. The delay would be done by systemd, instead.

I don't know whether this is completely specific to  Fedora and
systemd, but if you want it to be more generic you can do following in
the script:

if ! [ -t 0 ]; then
   do_random_delay
fi

Re: Fedora sa-update and systemd randomized timer

[Kevin A. McGrail]

Sounds good here. All saupdate does is a very lightweight DNS check.

On Thu, Dec 31, 2020, 19:54 Kenneth Porter  wrote:

> With the discussion of the KAM channel and Fedora's sa-update script that
> uses directory-based channel configuration, I went snooping into their
> script and systemd units. It looks like sa-update.cron has a 2-hour random
> delay before it looks for updates. I'm thinking it would be nice to move
> that delay into the systemd timer unit using the RandomizedDelaySec
> feature. This would allow the script to be run from the command line for
> immediate testing without waiting for the delay. The delay would be done
> by
> systemd, instead. The delay in the script should be configurable and able
> to be disabled in /etc/sysconfig/spamassassin, anyway. (For example, if it
> was to be used with the pre-systemd RHEL 6.) I'm throwing this here in
> case
> anyone sees a reason not to do this, before opening a bugzilla.
>
>

Fedora sa-update and systemd randomized timer

[Kenneth Porter]

With the discussion of the KAM channel and Fedora's sa-update script that 
uses directory-based channel configuration, I went snooping into their 
script and systemd units. It looks like sa-update.cron has a 2-hour random 
delay before it looks for updates. I'm thinking it would be nice to move 
that delay into the systemd timer unit using the RandomizedDelaySec 
feature. This would allow the script to be run from the command line for 
immediate testing without waiting for the delay. The delay would be done by 
systemd, instead. The delay in the script should be configurable and able 
to be disabled in /etc/sysconfig/spamassassin, anyway. (For example, if it 
was to be used with the pre-systemd RHEL 6.) I'm throwing this here in case 
anyone sees a reason not to do this, before opening a bugzilla.

Re: sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Benny Pedersen]

Kevin A. McGrail skrev den 2020-07-18 23:32:

Great, thanks for the update.  With this change we also know how to
best prevent similar issues.


more nutella :=)

Re: sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Kevin A. McGrail]

Great, thanks for the update.  With this change we also know how to best
prevent similar issues.

Regards,
KAM
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Sat, Jul 18, 2020 at 4:59 PM Marcus Schopen  wrote:

> Am Donnerstag, den 16.07.2020, 17:43 -0400 schrieb Kevin A. McGrail:
> > Agreed.  1879851 is after I submitted the fix but doesn;t include the
> > fix.  I will have to check again after the next rule publishing.
> > Sorry for the false alarm.
>
> Feedback: sa-update without errors now. Thanks a lot!
>
> Cheers
> m.
>
>
>

Re: sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Marcus Schopen]

Am Donnerstag, den 16.07.2020, 17:43 -0400 schrieb Kevin A. McGrail:
> Agreed.  1879851 is after I submitted the fix but doesn;t include the
> fix.  I will have to check again after the next rule publishing. 
> Sorry for the false alarm.

Feedback: sa-update without errors now. Thanks a lot!

Cheers
m.

Re: sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Kevin A. McGrail]

Agreed.  1879851 is after I submitted the fix but doesn;t include the fix.
I will have to check again after the next rule publishing.  Sorry for the
false alarm.
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Thu, Jul 16, 2020 at 5:30 PM Marcus Schopen  wrote:

> Am Mittwoch, den 15.07.2020, 08:11 -0400 schrieb Kevin A. McGrail:
> > Just an update that this is fixed:
> >
> > https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7840
> >
> > Should be a rule update published in the next 48 hours with the fix.
> > Feedback welcome because we are making fixes for older versions so
> > it's harder to gauge if it's fixed without user feedback.
>
> Which version should fix it? Same error with version 1879851.
>
> Ciao!
>
>
>

Re: sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Marcus Schopen]

Am Mittwoch, den 15.07.2020, 08:11 -0400 schrieb Kevin A. McGrail:
> Just an update that this is fixed:
> 
> https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7840
> 
> Should be a rule update published in the next 48 hours with the fix. 
> Feedback welcome because we are making fixes for older versions so
> it's harder to gauge if it's fixed without user feedback.

Which version should fix it? Same error with version 1879851.

Ciao!

sa-update for versions 3.4.2 and older fail lint because of description on non-existent rule was Re: update fail

[Kevin A. McGrail]

Just an update that this is fixed:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7840

Should be a rule update published in the next 48 hours with the fix. 
Feedback welcome because we are making fixes for older versions so it's
harder to gauge if it's fixed without user feedback.

Regards,

KAM

On 7/15/2020 7:58 AM, Kevin A. McGrail wrote:
> On 7/15/2020 7:13 AM, Marcus Schopen wrote:
>> -
>> config: warning: description exists for non-existent rule
>> USER_IN_WELCOMELIST_TO
>>
>> Jul 15 13:10:33.621 [2458] dbg: diag: updates complete, exiting with
>> code 4
>> Update failed, exiting with code 4
>> sa-update failed for unknown reasons
>> -
>>
>> I will send you the complete output.
>>
> Ahh, that is a new problem and only affects old versions 3.4.2 and
> before.  Working on it.
>
>
>
> -- 
> Kevin A. McGrail
> kmcgr...@apache.org
>
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: sa-update failing

[Kevin A. McGrail]

Stephan,

The type for the update record is a TXT not an A record, so dig -t txt
3.3.3.updates.spamassassin.org.

I'm not sure if an update has failed for the past 2 days though so this
is just a comment on how to check manually.


;; ANSWER SECTION:
3.3.3.updates.spamassassin.org. 3600 IN TXT "1879218"

Regards,

KAM

On 6/26/2020 4:13 AM, Stephan Fourie wrote:
> Hi everyone,
>
> Our SpamAssassin rules have not gotten any recent updates (looks like
> past 2 days). When investigating, sa-update tries to connect to:
> 2.4.3.updates.spamassassin.org
>
> When doing a DNS lookup on this hostname it appears to be a CNAME
> which points to:  3.3.3.updates.spamassassin.org. When doing a lookup
> for 3.3.3.updates.spamassassin.org it doesn't resolve, which I'm
> leaning towards being the issue. Please see below lookup results:
>
> -
> ;; QUESTION SECTION:
> ;3.3.3.updates.spamassassin.org.    IN    A
>
> ;; AUTHORITY SECTION:
> spamassassin.org.    1799    IN    SOA    ns2.pccc.com.
> pmc.spamassassin.apache.org. 2020062505 7200 3600 604800 3600
>
> ;; Query time: 265 msec
> ;; SERVER: 8.8.8.8#53(8.8.8.8)
> ;; WHEN: Fri Jun 26 10:10:28 SAST 2020
> ;; MSG SIZE  rcvd: 131
> -
>
> Anyone else seeing the same issue?
>
> Thanks!
> Stephan

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: sa-update failing

[Henrik K]

It queries TXT records

$ dig TXT 2.4.3.updates.spamassassin.org

;; ANSWER SECTION:
2.4.3.updates.spamassassin.org. 3424 IN CNAME   3.3.3.updates.spamassassin.org.
3.3.3.updates.spamassassin.org. 79 IN   TXT "1879105"

It is normal that updates might be stale for a few days sometimes.  Use
"sa-update -D" to debug and post relevant lines if you have an actual
problem.


On Fri, Jun 26, 2020 at 10:13:00AM +0200, Stephan Fourie wrote:
> Hi everyone,
> 
> Our SpamAssassin rules have not gotten any recent updates (looks like past 2
> days). When investigating, sa-update tries to connect to:
> 2.4.3.updates.spamassassin.org
> 
> When doing a DNS lookup on this hostname it appears to be a CNAME which points
> to:  3.3.3.updates.spamassassin.org. When doing a lookup for
> 3.3.3.updates.spamassassin.org it doesn't resolve, which I'm leaning towards
> being the issue. Please see below lookup results:
> 
> -
> ;; QUESTION SECTION:
> ;3.3.3.updates.spamassassin.org.    IN    A
> 
> ;; AUTHORITY SECTION:
> spamassassin.org.    1799    IN    SOA    ns2.pccc.com.
> pmc.spamassassin.apache.org. 2020062505 7200 3600 604800 3600
> 
> ;; Query time: 265 msec
> ;; SERVER: 8.8.8.8#53(8.8.8.8)
> ;; WHEN: Fri Jun 26 10:10:28 SAST 2020
> ;; MSG SIZE  rcvd: 131
> -
> 
> Anyone else seeing the same issue?
> 
> Thanks!
> Stephan

sa-update failing

[Stephan Fourie]

Hi everyone,

Our SpamAssassin rules have not gotten any recent updates (looks like 
past 2 days). When investigating, sa-update tries to connect to: 
2.4.3.updates.spamassassin.org


When doing a DNS lookup on this hostname it appears to be a CNAME which 
points to:  3.3.3.updates.spamassassin.org. When doing a lookup for 
3.3.3.updates.spamassassin.org it doesn't resolve, which I'm leaning 
towards being the issue. Please see below lookup results:


-
;; QUESTION SECTION:
;3.3.3.updates.spamassassin.org.    IN    A

;; AUTHORITY SECTION:
spamassassin.org.    1799    IN    SOA    ns2.pccc.com. 
pmc.spamassassin.apache.org. 2020062505 7200 3600 604800 3600


;; Query time: 265 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jun 26 10:10:28 SAST 2020
;; MSG SIZE  rcvd: 131
-

Anyone else seeing the same issue?

Thanks!
Stephan

Re: SA-Update cronjob output rejected by ISP for containing spam

[Chris Pollock]

On Sat, 2019-06-22 at 16:10 -0700, John Hardin wrote:
> On Sat, 22 Jun 2019, Chris Pollock wrote:
> 
> > On Sat, 2019-06-22 at 10:29 -0700, John Hardin wrote:
> > > On Sat, 22 Jun 2019, Chris Pollock wrote:
> > > 
> > > > I'm not sure how to exactly word the problem so the subject is
> > > > the
> > > > best
> > > > I can do for now. Whenever a crojob is run a message is sent
> > > > out
> > > > via
> > > > postfix to me with the contents of that cronjob. This morning
> > > > when
> > > > the
> > > > SA-Update cronjob was run I didn't receive the output back
> > > > (this
> > > > has
> > > > been going on since 7 June but that's another story). I looked
> > > > at
> > > > my
> > > > syslog and saw this:
> > > > 
> > > > https://pastebin.com/hHR0Rvii
> > > > 
> > > > Since I can't see the debug output of SA-Update I have no idea
> > > > what
> > > > CenturyLinks spam filter hit on. I looked back through a weeks
> > > > worth of
> > > > syslogs and this is the only time that the message was rejected
> > > > for
> > > > containing spam. Any ideas what was in the latest rule updates
> > > > to
> > > > cause
> > > > this?
> > > 
> > > Not without seeing the message itself. Is there any way for you
> > > to
> > > pastebin a copy of the message that was sent?
> > 
> > Sorry John, it's been removed from the queue
> > > 
> > > Can you twiddle the aliasing so that the message is (temporarily,
> > > at
> > > least) delivered to a local mailbox in addition to the regular
> > > recipients?
> > 
> > I've been trying to figure that out. What I have done is switch
> > postfix
> > over to using my GMail account however I've run into a tiny
> > roadblock.
> 
> How about delivery to a local mailbox?
> 
Amazingly I've got it working. What fixed it was adding [] around
smtp.gmail.com in my sasl_passwd file. I just let sa-update run and the
postfix output is:

Jun 22 22:12:01 localhost CRON[13838]: (root) CMD (/usr/bin/sa-
update  -D --channelfile /etc/mail/spamassassin/sare-sa-update-
channels.txt --gpgkey 6C6191E3 && /etc/init.d/spamassassin restart # --
gpgkey E8B493D6 )
Jun 22 22:12:01 localhost postfix/pickup[11566]: C76F41000BA1: uid=0
from=
Jun 22 22:12:01 localhost postfix/cleanup[13842]: C76F41000BA1:
message-id=<20190623031201.C76F41000BA1@cpollock.localdomain>
Jun 22 22:12:01 localhost postfix/qmgr[11567]: C76F41000BA1: from=<
chris.pollock1...@gmail.com>, size=5707, nrcpt=1 (queue active)
Jun 22 22:12:01 localhost postfix/local[13844]: C76F41000BA1: to=<
root@cpollock.localdomain>, orig_to=, relay=local, delay=0.13,
delays=0.07/0.01/0/0.05, dsn=2.0.0, status=sent (delivered to command:
/usr/bin/procmail -Y -a $DOMAIN)
Jun 22 22:12:01 localhost postfix/qmgr[11567]: C76F41000BA1: removed

So, it looks like to me in this case it's sending local, but I'm
probably wrong. However, the message hasn't made it to my cron folder
yet. 

-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11972; -97.90167 (Elev. 1092 ft)
22:16:17 up 2 days, 4:26, 1 user, load average: 1.37, 1.26, 1.33
Description:Ubuntu 18.04.2 LTS, kernel 4.18.0-22-generic



signature.asc
Description: This is a digitally signed message part

Re: SA-Update cronjob output rejected by ISP for containing spam

[Chris Pollock]

On Sat, 2019-06-22 at 16:10 -0700, John Hardin wrote:
> On Sat, 22 Jun 2019, Chris Pollock wrote:
> 
> > On Sat, 2019-06-22 at 10:29 -0700, John Hardin wrote:
> > > On Sat, 22 Jun 2019, Chris Pollock wrote:
> > > 
> > > > I'm not sure how to exactly word the problem so the subject is
> > > > the
> > > > best
> > > > I can do for now. Whenever a crojob is run a message is sent
> > > > out
> > > > via
> > > > postfix to me with the contents of that cronjob. This morning
> > > > when
> > > > the
> > > > SA-Update cronjob was run I didn't receive the output back
> > > > (this
> > > > has
> > > > been going on since 7 June but that's another story). I looked
> > > > at
> > > > my
> > > > syslog and saw this:
> > > > 
> > > > https://pastebin.com/hHR0Rvii
> > > > 
> > > > Since I can't see the debug output of SA-Update I have no idea
> > > > what
> > > > CenturyLinks spam filter hit on. I looked back through a weeks
> > > > worth of
> > > > syslogs and this is the only time that the message was rejected
> > > > for
> > > > containing spam. Any ideas what was in the latest rule updates
> > > > to
> > > > cause
> > > > this?
> > > 
> > > Not without seeing the message itself. Is there any way for you
> > > to
> > > pastebin a copy of the message that was sent?
> > 
> > Sorry John, it's been removed from the queue
> > > 
> > > Can you twiddle the aliasing so that the message is (temporarily,
> > > at
> > > least) delivered to a local mailbox in addition to the regular
> > > recipients?
> > 
> > I've been trying to figure that out. What I have done is switch
> > postfix
> > over to using my GMail account however I've run into a tiny
> > roadblock.
> 
> How about delivery to a local mailbox?
> 
I'll have to work on doing that tomorrow John, burned out from messing
with this all day. It should be a lot easier than trying to figure out
the GMail problem.

> 
-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11972; -97.90167 (Elev. 1092 ft)
20:22:30 up 2 days, 2:32, 1 user, load average: 1.12, 1.04, 1.01
Description:Ubuntu 18.04.2 LTS, kernel 4.18.0-22-generic



signature.asc
Description: This is a digitally signed message part

Re: SA-Update cronjob output rejected by ISP for containing spam

[John Hardin]

On Sat, 22 Jun 2019, Chris Pollock wrote:


On Sat, 2019-06-22 at 10:29 -0700, John Hardin wrote:

On Sat, 22 Jun 2019, Chris Pollock wrote:


I'm not sure how to exactly word the problem so the subject is the
best
I can do for now. Whenever a crojob is run a message is sent out
via
postfix to me with the contents of that cronjob. This morning when
the
SA-Update cronjob was run I didn't receive the output back (this
has
been going on since 7 June but that's another story). I looked at
my
syslog and saw this:

https://pastebin.com/hHR0Rvii

Since I can't see the debug output of SA-Update I have no idea what
CenturyLinks spam filter hit on. I looked back through a weeks
worth of
syslogs and this is the only time that the message was rejected for
containing spam. Any ideas what was in the latest rule updates to
cause
this?


Not without seeing the message itself. Is there any way for you to
pastebin a copy of the message that was sent?


Sorry John, it's been removed from the queue


Can you twiddle the aliasing so that the message is (temporarily, at
least) delivered to a local mailbox in addition to the regular
recipients?


I've been trying to figure that out. What I have done is switch postfix
over to using my GMail account however I've run into a tiny roadblock.


How about delivery to a local mailbox?


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  A good high-school education is still essential,
  and college is where you go to get one.-- MiddleAgedKen
---
 814 days since the first commercial re-flight of an orbital booster (SpaceX)

Re: SA-Update cronjob output rejected by ISP for containing spam

[Chris Pollock]

On Sat, 2019-06-22 at 10:29 -0700, John Hardin wrote:
> On Sat, 22 Jun 2019, Chris Pollock wrote:
> 
> > I'm not sure how to exactly word the problem so the subject is the
> > best
> > I can do for now. Whenever a crojob is run a message is sent out
> > via
> > postfix to me with the contents of that cronjob. This morning when
> > the
> > SA-Update cronjob was run I didn't receive the output back (this
> > has
> > been going on since 7 June but that's another story). I looked at
> > my
> > syslog and saw this:
> > 
> > https://pastebin.com/hHR0Rvii
> > 
> > Since I can't see the debug output of SA-Update I have no idea what
> > CenturyLinks spam filter hit on. I looked back through a weeks
> > worth of
> > syslogs and this is the only time that the message was rejected for
> > containing spam. Any ideas what was in the latest rule updates to
> > cause
> > this?
> 
> Not without seeing the message itself. Is there any way for you to 
> pastebin a copy of the message that was sent?

Sorry John, it's been removed from the queue
> 
> Can you twiddle the aliasing so that the message is (temporarily, at 
> least) delivered to a local mailbox in addition to the regular
> recipients?

I've been trying to figure that out. What I have done is switch postfix
over to using my GMail account however I've run into a tiny roadblock.
I keep getting 

localhost postfix/smtp[14383]: 893FD1000B19:
to=, relay=smtp.gmail.com[209.85.235.109]:587,
delay=0.48, delays=0.1/0.04/0.31/0.03, dsn=5.5.1, status=bounced (host
smtp.gmail.com[209.85.235.109] said: 530-5.5.1 Authentication Required.
Learn more at 530 5.5.1  
https://support.google.com/mail/?p=WantAuthError k99sm2494546otk.12 -
gsmtp (in reply to MAIL FROM command))

And I can't for the life of me figure out why. I've gone over my
postifx main.cf and other files for the past 4hrs and still can't find
a problem with any of them.

-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11972; -97.90167 (Elev. 1092 ft)
17:54:24 up 2 days, 4 min, 1 user, load average: 1.69, 1.46, 1.32
Description:Ubuntu 18.04.2 LTS, kernel 4.18.0-22-generic



signature.asc
Description: This is a digitally signed message part

Re: SA-Update cronjob output rejected by ISP for containing spam

[John Hardin]

On Sat, 22 Jun 2019, Chris Pollock wrote:


I'm not sure how to exactly word the problem so the subject is the best
I can do for now. Whenever a crojob is run a message is sent out via
postfix to me with the contents of that cronjob. This morning when the
SA-Update cronjob was run I didn't receive the output back (this has
been going on since 7 June but that's another story). I looked at my
syslog and saw this:

https://pastebin.com/hHR0Rvii

Since I can't see the debug output of SA-Update I have no idea what
CenturyLinks spam filter hit on. I looked back through a weeks worth of
syslogs and this is the only time that the message was rejected for
containing spam. Any ideas what was in the latest rule updates to cause
this?


Not without seeing the message itself. Is there any way for you to 
pastebin a copy of the message that was sent?


Can you twiddle the aliasing so that the message is (temporarily, at 
least) delivered to a local mailbox in addition to the regular recipients?


It's not *too* surprising that cron output would trip over spam filters, 
as the output from shell processes can hit rules intended to detect 
obfuscatury formatting or gibberish, and doesn't generally look like 
english text.


As the message is being bounced by an ISP server, it's unlikely you will 
be able to get trust defined. This is a hazard for using ISP mailboxes for 
purposes like this.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The question of whether people should be allowed to harm themselves
  is simple. They *must*.   -- Charles Murray
---
 814 days since the first commercial re-flight of an orbital booster (SpaceX)

Re: SA-Update cronjob output rejected by ISP for containing spam

[Benny Pedersen]

Chris Pollock skrev den 2019-06-22 19:03:


https://pastebin.com/hHR0Rvii


accepted and bounced ?

if yes fix that

SA-Update cronjob output rejected by ISP for containing spam

[Chris Pollock]

I'm not sure how to exactly word the problem so the subject is the best
I can do for now. Whenever a crojob is run a message is sent out via
postfix to me with the contents of that cronjob. This morning when the
SA-Update cronjob was run I didn't receive the output back (this has
been going on since 7 June but that's another story). I looked at my
syslog and saw this:

https://pastebin.com/hHR0Rvii

Since I can't see the debug output of SA-Update I have no idea what
CenturyLinks spam filter hit on. I looked back through a weeks worth of
syslogs and this is the only time that the message was rejected for
containing spam. Any ideas what was in the latest rule updates to cause
this?


-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11972; -97.90167 (Elev. 1092 ft)
11:56:52 up 1 day, 18:07, 1 user, load average: 1.21, 0.70, 0.65
Description:Ubuntu 18.04.2 LTS, kernel 4.18.0-22-generic


signature.asc
Description: This is a digitally signed message part

Re: How add ITA channel to sa-update?

[Giovanni Bechis]

On 3/26/19 9:05 AM, Alessio Cecchi wrote:
> Hello,
> 
> I'm interesting into add the italian channel to spamassassin from 
> https://spamassassin.snb.it/, but what is the right way?
> 
> I download ITA.conf in /etc/spamassassin/channel.d/ and run sa-update but I 
> don't see any new files in /var/lib/spamassassin/3.004002/.
> 
you can use
sa-update --channel spamassassin.snb.it to update from the specific channel,
otherwise "/usr/share/spamassassin/sa-update.cron" (scheduled daily by default) 
will do that for you.
 Giovanni

> Thanks (to Giovanni for the channel :-) )
> 
> -- 
> Alessio Cecchi
> Postmaster @ http://www.qboxmail.it
> https://www.linkedin.com/in/alessice
> 

How add ITA channel to sa-update?

[Alessio Cecchi]

Hello,

I'm interesting into add the italian channel to spamassassin from 
https://spamassassin.snb.it/, but what is the right way?


I download ITA.conf in /etc/spamassassin/channel.d/ and run sa-update 
but I don't see any new files in /var/lib/spamassassin/3.004002/.


Thanks (to Giovanni for the channel :-) )

--
Alessio Cecchi
Postmaster @ http://www.qboxmail.it
https://www.linkedin.com/in/alessice

Re: Problem with spamassassin update at /usr/bin/sa-update line 1603

[Kevin A. McGrail]

On 2/22/2019 5:36 AM, mbaldov wrote:
> So I ask you if it's possible to intervene on the mirror's list with
> some option so that to exclude the bad mirrors.

Escalating that issue to our sysadmins list.  Thanks for reporting the
problem.

-- 
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Problem with spamassassin update at /usr/bin/sa-update line 1603

[RW]

On Fri, 22 Feb 2019 11:36:38 +0100
mbaldov wrote:

> Hello,
> I have a mailserver with postfix installed (v.2.11.0) on Ubuntu 14.04
> LTS with Amavis, ClamAV and Spamassasins (v. 3.004002)
> Since the last upgrade that I have done next week, the upgrade of new
> signatures fails

The SA rules are not signatures as such. The core rules tend to be
fairly generic, so missing an update here and there isn't that much of a
problem - it's not like missing an antivirus update. 

If you have a problem with reliability there nothing to stop you
running sa-update several times a night. If the rules are up-to-date
sa-update is just a single DNS lookup. 

> Here below there are two manual attempts (wgets):
> One with a bad mirror (1) and one with a good one (2):
> 
> 1)
> wget http://sa-update.spamassassin.org/1853564.tar.gz
...
> HTTP request sent, awaiting response... *503 Service Unavailable*
> 2019-02-15 12:47:51 *ERROR 503*: Service Unavailable

This worked for me when I just tried it, and I don't have any recent
sa-update failures.

> 
> So I ask you if it's possible to intervene on the mirror's list with
> some option so that to exclude the bad mirrors.

In the short term you can edit unwanted mirrors out of the MIRRORED.BY
file. It will get replaced after a week (if you don't touch it).
It's worth trying just to see what happens.

Problem with spamassassin update at /usr/bin/sa-update line 1603

[mbaldov]

Hello,
I have a mailserver with postfix installed (v.2.11.0) on Ubuntu 14.04 LTS
with Amavis, ClamAV and Spamassasins (v. 3.004002)
Since the last upgrade that I have done next week, the upgrade of new
signatures fails with this messages:
"/etc/cron.daily/spamassassin:Cannot open file
/var/lib/spamassassin/3.004002/updates_spamassassin_org/1854020.tar.gz: No
such file or directory at /usr/bin/sa-update line 1603."

The problem isn't the file that clearly  doesn't exist but some mirrors
that it seems have problems when they are called.
I have seen many post where they spoke about permission but I have checked
and all it's correct.

Debugging the perl file called spamassasins, you get to a point where it
appears the mirror's list and their weight;
the first attempts is done on sa-update.spamassassin.org but the response
is 503 as highlighted below:

eb 15 12:45:10.992 [10243] dbg: channel: protocol family available:
inet,inet6
Feb 15 12:45:10.992 [10243] dbg: channel: file
/var/lib/spamassassin/3.004002/updates_spamassassin_org/MIRRORED.BY is too
old, refreshing mirrors file
Feb 15 12:45:10.992 [10243] dbg: channel: DNS lookup on
mirrors.updates.spamassassin.org
Feb 15 12:45:11.156 [10243] dbg: http: url:
http://spamassassin.apache.org/updates/MIRRORED.BY
Feb 15 12:45:11.156 [10243] dbg: http: downloading to:
/var/lib/spamassassin/3.004002/updates_spamassassin_org/MIRRORED.BY, replace
Feb 15 12:45:11.156 [10243] dbg: util: executable for curl was found at
/usr/bin/curl
Feb 15 12:45:11.156 [10243] dbg: http: /usr/bin/curl -s -L -O --remote-time
-g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o MIRRORED.BY
-- http://spamassassin.apache.org/updates/MIRRORED.BY
Feb 15 12:45:11.332 [10243] dbg: http: process [10244], exit status: exit 0
Feb 15 12:45:11.333 [10243] dbg: channel: MIRRORED.BY file for channel
updates.spamassassin.org retrieved
Feb 15 12:45:11.333 [10243] dbg: channel: parsing MIRRORED.BY file for
channel updates.spamassassin.org
Feb 15 12:45:11.333 [10243] dbg: channel: found mirror
http://sa-update.dnswl.org/ weight=3
Feb 15 12:45:11.333 [10243] dbg: channel: found mirror
http://www.sa-update.pccc.com/ weight=5
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.secnap.net/ weight=5
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.space-pro.be/ weight=1
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.ena.com/ weight=5
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.razx.cloud/ weight=5
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.fossies.org/ weight=1
Feb 15 12:45:11.334 [10243] dbg: channel: found mirror
http://sa-update.bitwell.fi/ weight=5
Feb 15 12:45:11.335 [10243] dbg: channel: found mirror
http://sa-update.spamassassin.org/ weight=10
Feb 15 12:45:11.338 [10243] dbg: channel: selected mirror
http://sa-update.spamassassin.org
Feb 15 12:45:11.338 [10243] dbg: http: url:
*http://sa-update.spamassassin.org/1853564.tar.gz
<http://sa-update.spamassassin.org/1853564.tar.gz>*
Feb 15 12:45:11.338 [10243] dbg: http: downloading to:
/var/lib/spamassassin/3.004002/updates_spamassassin_org/1853564.tar.gz, new
Feb 15 12:45:11.338 [10243] dbg: util: executable for curl was found at
/usr/bin/curl
Feb 15 12:45:11.338 [10243] dbg: http: /usr/bin/curl -s -L -O --remote-time
-g --max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o
1853564.tar.gz -- http://sa-update.spamassassin.org/1853564.tar.gz
Feb 15 12:45:11.543 [10243] dbg: http: process [10246], exit status: exit 22
Cannot open file
/var/lib/spamassassin/3.004002/updates_spamassassin_org/1853564.tar.gz: No
such file or directory at /usr/bin/sa-update line 1603.
##

Here below there are two manual attempts (wgets):
One with a bad mirror (1) and one with a good one (2):

1)
wget http://sa-update.spamassassin.org/1853564.tar.gz
--2019-02-15 12:47:51--  http://sa-update.spamassassin.org/1853564.tar.gz
Resolving sa-update.spamassassin.org (sa-update.spamassassin.org)...
64.142.56.146
Connecting to sa-update.spamassassin.org
(sa-update.spamassassin.org)|64.142.56.146|:80...
connected.
HTTP request sent, awaiting response... *503 Service Unavailable*
2019-02-15 12:47:51 *ERROR 503*: Service Unavailable

2)
wget http://sa-update.dnswl.org/1853564.tar.gz
--2019-02-15 12:48:08--  http://sa-update.dnswl.org/1853564.tar.gz
Resolving sa-update.dnswl.org (sa-update.dnswl.org)... 78.47.167.123,
2a01:4f8:d15:2fc0:::20
Connecting to sa-update.dnswl.org (sa-update.dnswl.org)|78.47.167.123|:80...
connected.
HTTP request sent, awaiting response... *200 OK*
Length: 295185 (288K) [application/x-gzip]
Saving to: ‘1853564.tar.gz’
100%[==>]

So I ask you if it's possible to intervene on the mirror's list with some
option so that to exclude the bad mirrors.

Thanks in advance for who will answer me.

Regards.

Re: sa-update when were last updates made?

[LegendGamesMaster]

thanks for your info, sincerely,  most helpful.
I will look into my setup over the next week or so, as its clearly a bit
weird!
a few typos in my message, which I corrected - my old eyes are in need of an
update too



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: sa-update when were last updates made?

[Bill Cole]

On 24 Jan 2019, at 8:44, LegendGamesMaster wrote:


Reindl - thanks.


Note that anything you get from H. Reindl in reply to messages on this 
mailing list is not actually posted to the mailing list.



just checked and yes, i'm updated.
however...

i'm confused as to what rules are being used in preference...

The files in /var/lib/spamassassin/3.0014000/ are clearly todays 
update,


I hope that's a typo, because it can't be correct.

The name of that directory should be 3.004002 if you're running the 
latest release of SA (v3.4.2.)  It could be 3.004001 if you're one 
release back.



except for a directory sought_rules_yerp_org which is 18/10/2018


You should 'rm -rf sought_rules_yerp_org' because those rules are 
obsolete and unsafe. The sa-update channel for them will not work with 
v3.4.2 because the abandoned channel does not include the required hash 
file or a valid signature file.


I note that I have a load of files in /usr/share/spamassassin that are 
all

dated 18-10-2018


Which is probably when you installed SA, implying that this SHOULD be 
v3.4.2.


However, there normally won't be a "load of files" there, just 3. SA has 
not put *.cf or *.pre files in that directory since v3.1.x. This could 
be an anomaly in how your version of SA was built or it could be a 
leftover from an ancient version that was never cleaned up and somehow 
got touched recently. If those files are part of the SA distribution, 
you can find which version they are for by looking for a 
'require_version' line in them, generally near the top of the file.


my updated conf.cf with "report all" lives in /usr/share/spamassassin 
and is

seemingly working correctly.


That's a minor oddity.

Normally, the local config file lives in /etc/mail/spamassassin/ and is 
named local.cf.
You should not add, remove, or change files in /usr/share/spamassassin, 
because your package manager (on CentOS that would be yum & rpm) sees 
that directory as belonging to the SA package and may do bad things when 
updating.


Generally speaking, anything under /usr except for /usr/local/ should be 
left alone if you use a distro-standard package manager and want to 
avoid a battle with it that you will ultimately lose.


so.. are the files in /usr/…. running in preference to the files in 
/var/…

and if so, how do I ensure the latest ones are used?


It's complicated...

SA uses *MULTIPLE* directories for config and rules to allow for layers 
of overrides. See 
https://svn.apache.org/repos/asf/spamassassin/branches/3.4/README for 
details.


You PROBABLY should merge any settings in your "conf.cf" into whatever 
/etc/mail/spamassassin/local.cf is already there and remove 
/usr/share/spamassassin/*.{cf,pre}.

Re: sa-update when were last updates made?

[Matus UHLAR - fantomas]

On 24.01.19 06:44, LegendGamesMaster wrote:

The files in /var/lib/spamassassin/3.0014000/ are clearly todays update,
except for a directory sought_rules_yerp_org which is 18/10/2018


afaik this project is unfortunately dead


I note that I have a load of files in /usr/share/spamassassin that are all
dated 18-10-2018


these are not to be updated, they were installed with SA and are only used
when you don't fetch newer in /var/...


so.. are the files in /usr/…. running in preference to the files in /var/…
and if so, how do I ensure the latest ones are used?


answer above.
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"One World. One Web. One Program." - Microsoft promotional advertisement
"Ein Volk, ein Reich, ein Fuhrer!" - Adolf Hitler

Re: sa-update when were last updates made?

[LegendGamesMaster]

Reindl - thanks.

just checked and yes, i'm updated.
however...

i'm confused as to what rules are being used in preference...

The files in /var/lib/spamassassin/3.0014000/ are clearly todays update,
except for a directory sought_rules_yerp_org which is 18/10/2018

I note that I have a load of files in /usr/share/spamassassin that are all
dated 18-10-2018 

my updated conf.cf with "report all" lives in /usr/share/spamassassin and is
seemingly working correctly.

so.. are the files in /usr/…. running in preference to the files in /var/…
and if so, how do I ensure the latest ones are used?

thanks for your patience.
Andy  




--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

sa-update when were last updates made?

[LegendGamesMaster]

Hi Folks

newbie here.

centos / plesk, postfix, latest SA, run sa-update regularly

My files are showing October 2018 as last update date - this seems a bit old
to me.
can anyone confirm the dates of the last updates?

If im not getting them, I can delve deeper

thanks in advance.
Andy 



--
Sent from: http://spamassassin.1065346.n5.nabble.com/SpamAssassin-Users-f3.html

Re: sa-update --allow-plugins [was: sa-update not properly parsing urls in MIRRORED.BY files?]

[Henrik K]

On Sat, Jan 12, 2019 at 02:10:37PM -0500, listsb wrote:
> 
> that said, i don't quite follow the second statement, if i'm honest.  i
> suppose that some people may run sa-update or spamassassin as root, but i
> don't, and would be filing bugs against any packagers or distributors that
> were delivering it this way.

Things like virtual users or privileged ports require starting as root. 
Even if it is configured to switch users, it can run some linting/compiling
stuff from plugins as root.

> that said, i would think that if there were to be any channel that should
> be trusted to deliver safe plugins [regardless of if the code involved
> were to run as either a privileged or non-privileged user], it would be
> the official channel, wouldn't it?

Sure, but I think it's just a legacy extra feature that has never been used,
and in my opinion there's no reason to use it ever.  Most people don't have
the option enabled anyway, so it would be pointless to distribute anything,
that's what version updates are for..

sa-update --allow-plugins [was: sa-update not properly parsing urls in MIRRORED.BY files?]

[listsb]

> On Jan 11, 2019, at 10.55, Henrik K  wrote:
> 
> On Wed, Jan 09, 2019 at 11:59:36PM -0500, listsb wrote:
>> 
>>> sa-update -vvv --allowplugins ...
> 
> Just a general note, I would never ever use --allowplugins unless it's your
> personal channel.  There is no reason why official channels should ever
> distribute plugins as it would be basically remote code run as root.

thanks for mentioning this.  i'd wondered about that - the documentation 
["Allow downloaded updates to activate plugins."] doesn't quite express what 
exactly --allowplugins does/means, imho.  i would like to better understand 
this.

that said, i don't quite follow the second statement, if i'm honest.  i suppose 
that some people may run sa-update or spamassassin as root, but i don't, and 
would be filing bugs against any packagers or distributors that were delivering 
it this way.  that said, i would think that if there were to be any channel 
that should be trusted to deliver safe plugins [regardless of if the code 
involved were to run as either a privileged or non-privileged user], it would 
be the official channel, wouldn't it?

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[listsb]

On Jan 11, 2019, at 00.24, Bill Cole  
wrote:
> 
> On 10 Jan 2019, at 23:15, listsb wrote:
> 
>> On Jan 10, 2019, at 06.05, Kevin A. McGrail  wrote:
>>> 
>>> I believe this is a known issue fixed in svn.  We need to get 3.4.3 out the 
>>> door for this.  Are you able to test with the 3.4 branch from svn?
>> 
>> thanks.  i've done a crude test just grabbing sa-update from svn, with some 
>> progress:
>> 
>>> sa-update -v --allowplugins --channelfile 
>>> /etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile 
>>> /etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir 
>>> /var/lib/spamassassin/sa-update-keys
>> Update available for channel sought.rules.yerp.org: -1 -> 3402014020421
>> http: (curl) GET http://yerp.org/rules/MIRRORED.BY, success
>> http: (curl) GET 
>> http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz, 
>> success
>> http: (curl) GET 
>> http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha512,
>>  FAILED, status: exit 22
>> http: (curl) GET 
>> http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha256,
>>  FAILED, status: exit 22
>> http: (curl) GET 
>> http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.asc, 
>> success
>> channel 'sought.rules.yerp.org': could not find working mirror, channel 
>> failed
>> Update failed, exiting with code 4
>> 
>> it parses the url properly now, but still fails.
> 
> This breakage is a FEATURE, not a bug.
> 
>> i guess it doesn't like only having the asc file?
> 
> Correct. That channel provides no usable hash file and so cannot work with 
> sa-update. If you would like a version of sa-update that does not require 
> hash files, hack it up at will: that's what open source is for.

thanks, it was not knowing about the change from sha1 to sha2 that was the red 
herring for me.  since an sha1 hash is still published, that wasn't failing 
prior to upgrading.  on a related but different note, it's interesting that 
with an expired gpg key, it wasn't failing before upgrading.  i don't run 
sa-update with --nogpg.

in any case, at least the upgrade process exposed a channel in the config that 
had long since been forgotten about.

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Bill Cole]

On 11 Jan 2019, at 10:22, Kris Deugau wrote:


Bill Cole wrote:

On 10 Jan 2019, at 23:15, listsb wrote:
Update available for channel sought.rules.yerp.org: -1 -> 
3402014020421


And finally: that rule channel has not been updated in almost 4 years 
and almost surely will never be updated again.


I'm pretty sure it's been longer than that even.


Correct. Almost 5, according to the internal & signature timestamps. My 
mistake was a symptom of it being early January...


Last time I checked closely it was empty;  absolutely no __ rules and 
the scored metas were "meta SOUGHT_1 (0)".


$ grep score 20*
20_sought.cf:score JM_SOUGHT_1  0
20_sought_fraud.cf:score JM_SOUGHT_FRAUD_1  0
20_sought_fraud.cf:score JM_SOUGHT_FRAUD_2  3.0
20_sought_fraud.cf:score JM_SOUGHT_FRAUD_3  3.0

Even if it downloads and validates, it's not actually doing anything, 
and hasn't been for years.


Testing 282 simple but long body rules against every message is not 
free.


The danger in the SOUGHT rules still being a part of SA 'lore' is that 
they are a bit of abandoned attack surface. It's still possible to 
download the tarball and forcibly install it or to use an obsolete or 
modified sa-update to do so. If Justin lost control of the channel or 
(less likely) turned malicious, the channel could be revived and turned 
against a relatively inattentive subset of people using SA.


Breaking unmaintained zombie rules channel was a fortuitous side-effect 
of sa-update switching from SHA1 to SHA256 and SHA512.

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[RW]

On Fri, 11 Jan 2019 10:22:13 -0500
Kris Deugau wrote:

> Bill Cole wrote:
> > On 10 Jan 2019, at 23:15, listsb wrote:  
> >> Update available for channel sought.rules.yerp.org: -1 ->
> >> 3402014020421  
> 
> > And finally: that rule channel has not been updated in almost 4
> > years and almost surely will never be updated again.  
> 
> I'm pretty sure it's been longer than that even.  

I download it yesterday and it's 5 years in a few weeks


> Last time I checked 
> closely it was empty;  absolutely no __ rules and the scored metas
> were "meta SOUGHT_1 (0)".

There's nothing left in 20_sought.cf, but two of the three SOUGHT_FRAUD
meta rules are still in 20_sought_fraud.cf.
 
Sought rules were never intended to have any long-term value, they
aren't general spam signs, they were autogenerated rules based on
fairly long phrases found in recent spam.

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Henrik K]

On Wed, Jan 09, 2019 at 11:59:36PM -0500, listsb wrote:
> 
> >sa-update -vvv --allowplugins ...

Just a general note, I would never ever use --allowplugins unless it's your
personal channel.  There is no reason why official channels should ever
distribute plugins as it would be basically remote code run as root.

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Kris Deugau]

Bill Cole wrote:

On 10 Jan 2019, at 23:15, listsb wrote:

Update available for channel sought.rules.yerp.org: -1 -> 3402014020421


And finally: that rule channel has not been updated in almost 4 years 
and almost surely will never be updated again.


I'm pretty sure it's been longer than that even.  Last time I checked 
closely it was empty;  absolutely no __ rules and the scored metas were 
"meta SOUGHT_1 (0)".


Even if it downloads and validates, it's not actually doing anything, 
and hasn't been for years.


-kgd

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Bill Cole]

On 10 Jan 2019, at 23:15, listsb wrote:

On Jan 10, 2019, at 06.05, Kevin A. McGrail  
wrote:


I believe this is a known issue fixed in svn.  We need to get 3.4.3 
out the door for this.  Are you able to test with the 3.4 branch from 
svn?


thanks.  i've done a crude test just grabbing sa-update from svn, with 
some progress:


sa-update -v --allowplugins --channelfile 
/etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile 
/etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir 
/var/lib/spamassassin/sa-update-keys
Update available for channel sought.rules.yerp.org: -1 -> 
3402014020421

http: (curl) GET http://yerp.org/rules/MIRRORED.BY, success
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz, 
success
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha512, 
FAILED, status: exit 22
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha256, 
FAILED, status: exit 22
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.asc, 
success
channel 'sought.rules.yerp.org': could not find working mirror, 
channel failed

Update failed, exiting with code 4

it parses the url properly now, but still fails.


This breakage is a FEATURE, not a bug.


i guess it doesn't like only having the asc file?


Correct. That channel provides no usable hash file and so cannot work 
with sa-update. If you would like a version of sa-update that does not 
require hash files, hack it up at will: that's what open source is for.


Also, the signature is bad:

$ gpg --verify -v 3402014020421.tar.gz.asc
gpg: armor header: Version: GnuPG v1.4.10 (GNU/Linux)
gpg: assuming signed data in '3402014020421.tar.gz'
gpg: Signature made Tue Feb  4 16:48:02 2014 EST
gpg:using DSA key DC85341F6C6191E3
gpg: Note: signature key DC85341F6C6191E3 expired Wed Aug  9 19:29:42 
2017 EDT
gpg: Note: signature key DC85341F6C6191E3 expired Wed Aug  9 19:29:42 
2017 EDT
gpg: Note: signature key DC85341F6C6191E3 expired Wed Aug  9 19:29:42 
2017 EDT

gpg: using pgp trust model
gpg: BAD signature from "Justin Mason Signing Key (Code Signing Only) 
" [expired]

gpg: binary signature, digest algorithm SHA1, key algorithm dsa1024


And finally: that rule channel has not been updated in almost 4 years 
and almost surely will never be updated again. Trying to use sa-update 
with it is pointless and dangerous and so it SHOULD break.  If the 
theory and praxis behind the final round of generation and scoring of 
the SOUGHT rules was valid in 2014, they would be essentially worthless 
against the mythical average mailstream of 2019. They may or may not be 
useful for any particular mailstream today but in any case they are 
unmaintained and unsupported. No one should use them without local 
testing and ongoing local oversight of their performance against one's 
local mailstream.

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Benny Pedersen]

listsb skrev den 2019-01-11 05:15:

sa-update -v --allowplugins --channelfile 
/etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile 
/etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir 
/var/lib/spamassassin/sa-update-keys

Update available for channel sought.rules.yerp.org: -1 -> 3402014020421


is this very old channel waked to life now ? :=)

imho it have being non maintained in many years

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[listsb]

On Jan 10, 2019, at 06.05, Kevin A. McGrail  wrote:
> 
> I believe this is a known issue fixed in svn.  We need to get 3.4.3 out the 
> door for this.  Are you able to test with the 3.4 branch from svn?

thanks.  i've done a crude test just grabbing sa-update from svn, with some 
progress:

>sa-update -v --allowplugins --channelfile 
>/etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile 
>/etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir 
>/var/lib/spamassassin/sa-update-keys
Update available for channel sought.rules.yerp.org: -1 -> 3402014020421
http: (curl) GET http://yerp.org/rules/MIRRORED.BY, success
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz, success
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha512, 
FAILED, status: exit 22
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.sha256, 
FAILED, status: exit 22
http: (curl) GET 
http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz.asc, 
success
channel 'sought.rules.yerp.org': could not find working mirror, channel failed
Update failed, exiting with code 4

it parses the url properly now, but still fails.  i guess it doesn't like only 
having the asc file?  is my test too crude to be viable?

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Marcin Mirosław]

W dniu 2019-01-10 o 12:05, Kevin A. McGrail pisze:
> I believe this is a known issue fixed in svn.  We need to get 3.4.3 out
> the door for this.  Are you able to test with the 3.4 branch from svn?

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7623

Re: sa-update not properly parsing urls in MIRRORED.BY files?

[Kevin A. McGrail]

I believe this is a known issue fixed in svn.  We need to get 3.4.3 out the
door for this.  Are you able to test with the 3.4 branch from svn?

On Wed, Jan 9, 2019, 23:59 listsb  hi-
>
> the subject expresses my uneducated hypothesis as to what might be causing
> a problem i seem to have encountered after upgrading to 3.4.2.
>
> i have an additional channel defined [sought.rules.yerp.org], and updates
> of this channel seem to have broken upon updating to 3.4.2:
>
> >sa-update -vvv --allowplugins --channelfile
> /etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile
> /etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir
> /var/lib/spamassassin/sa-update-keys
> DNS TXT query: 2.4.3.sought.rules.yerp.org -> 3402014020421
> Update available for channel sought.rules.yerp.org: -1 -> 3402014020421
> DNS A query rules.yerp.org.s3.amazonaws.com/rules/stage failed: NXDOMAIN
> DNS  query rules.yerp.org.s3.amazonaws.com/rules/stage failed:
> NXDOMAIN
> channel: could not find working mirror, channel failed
> Update failed, exiting with code 4
>
> we can see it find the txt record for mirrors:
>
> >dig mirrors.sought.rules.yerp.org txt +short
> "http://yerp.org/rules/MIRRORED.BY;
>
> and successfully retrieves and reads the MIRRORED.BY file, which contains:
>
> >curl 'http://yerp.org/rules/MIRRORED.BY'
> http://rules.yerp.org.s3.amazonaws.com/rules/stage/
>
> but then it seems to behave unexpectedly, and appears to not properly
> parse the hostname from within the url, instead attempting to lookup the
> entire url as though it were a hostname ["
> rules.yerp.org.s3.amazonaws.com/rules/stage"], which of course is invalid
> and doesn't exist.
>
> query logs from the recursive nameserver confirm this:
>
> 09-Jan-2019 23:49:04.421 queries: info: client 198.19.20.50#57187 (
> rules.yerp.org.s3.amazonaws.com/rules/stage): view internal: query:
> rules.yerp.org.s3.amazonaws.com/rules/stage IN A + (198.19.20.50)
> 09-Jan-2019 23:49:04.422 queries: info: client 198.19.20.50#39320 (
> rules.yerp.org.s3.amazonaws.com/rules/stage): view internal: query:
> rules.yerp.org.s3.amazonaws.com/rules/stage IN  + (198.19.20.50)
>
> if we follow the url correctly, we can see there is a functional mirror:
>
> >curl -LO '
> http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz'
>   % Total% Received % Xferd  Average Speed   TimeTime Time
> Current
>  Dload  Upload   Total   SpentLeft
> Speed
> 100 10462  100 104620 0  1712k  0 --:--:-- --:--:-- --:--:--
> 2043k
>
> >l
> total 12K
> -rw-r--r-- 1 root root 11K Jan  9 23:51 3402014020421.tar.gz
>
> so this channel would be working, were the url parsed properly.
>
> is my hypothesis wrong?  is this to be expected?  if not, how can i figure
> out why this is happening?
>
> thanks!

sa-update not properly parsing urls in MIRRORED.BY files?

[listsb]

hi-

the subject expresses my uneducated hypothesis as to what might be causing a 
problem i seem to have encountered after upgrading to 3.4.2.

i have an additional channel defined [sought.rules.yerp.org], and updates of 
this channel seem to have broken upon updating to 3.4.2:

>sa-update -vvv --allowplugins --channelfile 
>/etc/spamassassin/sa-update-conf.d/channels.txt --gpgkeyfile 
>/etc/spamassassin/sa-update-conf.d/sa-update-keys.txt --gpghomedir 
>/var/lib/spamassassin/sa-update-keys
DNS TXT query: 2.4.3.sought.rules.yerp.org -> 3402014020421
Update available for channel sought.rules.yerp.org: -1 -> 3402014020421
DNS A query rules.yerp.org.s3.amazonaws.com/rules/stage failed: NXDOMAIN
DNS  query rules.yerp.org.s3.amazonaws.com/rules/stage failed: NXDOMAIN
channel: could not find working mirror, channel failed
Update failed, exiting with code 4

we can see it find the txt record for mirrors:

>dig mirrors.sought.rules.yerp.org txt +short
"http://yerp.org/rules/MIRRORED.BY;

and successfully retrieves and reads the MIRRORED.BY file, which contains:

>curl 'http://yerp.org/rules/MIRRORED.BY'
http://rules.yerp.org.s3.amazonaws.com/rules/stage/

but then it seems to behave unexpectedly, and appears to not properly parse the 
hostname from within the url, instead attempting to lookup the entire url as 
though it were a hostname ["rules.yerp.org.s3.amazonaws.com/rules/stage"], 
which of course is invalid and doesn't exist.

query logs from the recursive nameserver confirm this:

09-Jan-2019 23:49:04.421 queries: info: client 198.19.20.50#57187 
(rules.yerp.org.s3.amazonaws.com/rules/stage): view internal: query: 
rules.yerp.org.s3.amazonaws.com/rules/stage IN A + (198.19.20.50)
09-Jan-2019 23:49:04.422 queries: info: client 198.19.20.50#39320 
(rules.yerp.org.s3.amazonaws.com/rules/stage): view internal: query: 
rules.yerp.org.s3.amazonaws.com/rules/stage IN  + (198.19.20.50)

if we follow the url correctly, we can see there is a functional mirror:

>curl -LO 
>'http://rules.yerp.org.s3.amazonaws.com/rules/stage/3402014020421.tar.gz'
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100 10462  100 104620 0  1712k  0 --:--:-- --:--:-- --:--:-- 2043k

>l
total 12K
-rw-r--r-- 1 root root 11K Jan  9 23:51 3402014020421.tar.gz

so this channel would be working, were the url parsed properly.

is my hypothesis wrong?  is this to be expected?  if not, how can i figure out 
why this is happening?

thanks!

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[@lbutlr]

On 21 Dec 2018, at 15:52, Bill Cole  
wrote:
> cd `mktemp -d -t HappyMichael???`

I'd prefer you did 

cd `mktemp -d -t saupdate`

-- 
I gotta straighten my face This mellow-thighed chick just put my spine
out of place

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Michael Orlitzky]

On 12/21/18 5:52 PM, Bill Cole wrote:


Fine:

#!/bin/sh
cd `mktemp -d -t HappyMichael???`



Yes, Merry Christmas =P

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Bill Cole]

On 21 Dec 2018, at 15:57, Michael Orlitzky wrote:

> On 12/20/18 7:00 PM, Bill Cole wrote:
>>
>>  mkdir /tmp/saupdate-1849156
>
> Never use a fixed path under /tmp =)


Fine:

#!/bin/sh
cd `mktemp -d -t HappyMichael???`
curl -O http://sa-update.spamassassin.org/1849156.tar.gz
curl -O http://sa-update.spamassassin.org/1849156.tar.gz.asc
curl -O http://sa-update.spamassassin.org/1849156.tar.gz.sha245
curl -O http://sa-update.spamassassin.org/1849156.tar.gz.sha512
sa-update -D --install 1849156.tar.gz

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Michael Orlitzky]

On 12/20/18 7:00 PM, Bill Cole wrote:


 mkdir /tmp/saupdate-1849156


Never use a fixed path under /tmp =)

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Bill Cole]

On 20 Dec 2018, at 17:54, Bill Cole wrote:

If you cannot wait 5 more hours and have an updated SVN checkout of 
the 'trunk' code, you can run:


make clean ; echo |perl Makefile.PL ; make build_rules

That will leave a proper set of rules files in the rules/ directory. 
If you copy rules/72_active.cf to your local site-wide rules directory 
(probably  /var/lib/spamassassin/3.004002/updates_spamassassin_org/) 
you will fix the worst effects of last night's broken update.



It has been pointed out to me that a simpler and less error-prone fix 
would be to revert to the prior day's rule collection:


   mkdir /tmp/saupdate-1849156
   cd $_
   curl -O http://sa-update.spamassassin.org/1849156.tar.gz
   curl -O http://sa-update.spamassassin.org/1849156.tar.gz.asc
   curl -O http://sa-update.spamassassin.org/1849156.tar.gz.sha245
   curl -O http://sa-update.spamassassin.org/1849156.tar.gz.sha512
   sa-update -D --install 1849156.tar.gz

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Bill Cole]

On 20 Dec 2018, at 17:56, Kevin A. McGrail wrote:


We've had a few occurrences of essentially the same problem (a bad
rules package due to an ignored lint failure in a nightly update) 
over

the past few years. In addition to correcting the problematic rule I
have also fixed the script which intentionally (!) masked the lint
failure and allowed the broken rules package to be built and 
distributed.




The file shouldn't get installed though because sa-update checks the
lint, doesn't it?


It depends on why the lint failed in the update process and on the local 
config. In the immediate case, sa-update installed the bad package.


The root cause of this particular failure was a 'replace_tag' rule that 
was outside an 'ifplugin Mail::SpamAssassin::Plugin::ReplaceTags' block. 
Because 'make build_rules' runs with minimal plugins loaded, the rule 
failed to parse and the design error in the mkrules script papered over 
the problem with an empty 72_active.cf. The rules package was assembled 
correctly with that empty file. When tested by sa-update after download, 
the rules pass lint because the file where the 'bad' rule would have 
gone was empty.

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Kevin A. McGrail]

On 12/20/2018 5:54 PM, Bill Cole wrote:
> On 20 Dec 2018, at 13:41, Bill Cole wrote:
>
>> This should now be fixed for the next rules update.
>
> And, On 20 Dec 2018, at 17:04, (ignoring an explicit Reply-To header
> in a direct message to me!) Frank Giesecke wrote:
>
>> How can I force the rules update?
>
> You cannot. The "rules update" I referred to is the one that runs
> every night on an Apache infrastructure host, to update the default
> rules channel. The update completes around 03:30 UTC.
>
>> I still get the error on my Debian system.
>
> If you cannot wait 5 more hours and have an updated SVN checkout of
> the 'trunk' code, you can run:
>
>     make clean ; echo |perl Makefile.PL ; make build_rules
>
> That will leave a proper set of rules files in the rules/ directory.
> If you copy rules/72_active.cf to your local site-wide rules directory
> (probably  /var/lib/spamassassin/3.004002/updates_spamassassin_org/)
> you will fix the worst effects of last night's broken update.
>
> We've had a few occurrences of essentially the same problem (a bad
> rules package due to an ignored lint failure in a nightly update) over
> the past few years. In addition to correcting the problematic rule I
> have also fixed the script which intentionally (!) masked the lint
> failure and allowed the broken rules package to be built and distributed.
>
The file shouldn't get installed though because sa-update checks the
lint, doesn't it?

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Bill Cole]

On 20 Dec 2018, at 13:41, Bill Cole wrote:


This should now be fixed for the next rules update.


And, On 20 Dec 2018, at 17:04, (ignoring an explicit Reply-To header in 
a direct message to me!) Frank Giesecke wrote:



How can I force the rules update?


You cannot. The "rules update" I referred to is the one that runs every 
night on an Apache infrastructure host, to update the default rules 
channel. The update completes around 03:30 UTC.



I still get the error on my Debian system.


If you cannot wait 5 more hours and have an updated SVN checkout of the 
'trunk' code, you can run:


make clean ; echo |perl Makefile.PL ; make build_rules

That will leave a proper set of rules files in the rules/ directory. If 
you copy rules/72_active.cf to your local site-wide rules directory 
(probably  /var/lib/spamassassin/3.004002/updates_spamassassin_org/) you 
will fix the worst effects of last night's broken update.


We've had a few occurrences of essentially the same problem (a bad rules 
package due to an ignored lint failure in a nightly update) over the 
past few years. In addition to correcting the problematic rule I have 
also fixed the script which intentionally (!) masked the lint failure 
and allowed the broken rules package to be built and distributed.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole

Re: sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Bill Cole]

On 20 Dec 2018, at 11:55, Marcus Schopen wrote:

> Am Donnerstag, den 20.12.2018, 12:35 +0100 schrieb Marcus Schopen:
>> Hi,
>>
>> I get a warning, when updating the channel:
>>
>> --
>> config: warning: description exists for non-existent rule EXCUSE_24
>>
>> channel: lint check of update failed, channel failed
>> sa-update failed for unknown reasons
>> --
>
> seems not to be a problem of the EXCUSE_24 rule, but a general problem
> with sa-update, as other users do have the same problem since today.


This should now be fixed for the next rules update.

sa-update is broken on updates.spamassassin.org channel [was: Re: config: warning: description exists for non-existent rule EXCUSE_24]

[Marcus Schopen]

Am Donnerstag, den 20.12.2018, 12:35 +0100 schrieb Marcus Schopen:
> Hi,
> 
> I get a warning, when updating the channel:
> 
> --
> config: warning: description exists for non-existent rule EXCUSE_24
> 
> channel: lint check of update failed, channel failed
> sa-update failed for unknown reasons
> --

seems not to be a problem of the EXCUSE_24 rule, but a general problem
with sa-update, as other users do have the same problem since today.

Re: [SA 3.4.2] sa-update doesn't see custom channel

[Marcin Mirosław]

W dniu 19.12.2018 o 16:16, Kris Deugau pisze:
> RW wrote:
>> It looks like sa-update has lost support for paths in mirror URLs. The
>> SA mirrors don't currently have paths, but the commented-out dostech
>> entry suggests that they have been supported in the past.
> 
> I came across this myself since my local channels also use
> subdirectories.  It's fixed for the pending 3.4.3 (I think) and in trunk
> as per https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7623.

Hi,
thank you for link to bug. I'll test fix.
Marcin

Re: [SA 3.4.2] sa-update doesn't see custom channel

[Kris Deugau]

RW wrote:

It looks like sa-update has lost support for paths in mirror URLs. The
SA mirrors don't currently have paths, but the commented-out dostech
entry suggests that they have been supported in the past.


I came across this myself since my local channels also use 
subdirectories.  It's fixed for the pending 3.4.3 (I think) and in trunk 
as per https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7623.


-kgd

Re: [SA 3.4.2] sa-update doesn't see custom channel

[RW]

On Wed, 19 Dec 2018 11:34:32 +0100
Marcin Mirosław wrote:

> W dniu 03.12.2018 o 15:42, Marcin Mirosław pisze:
> > Hi!
> > I have problem with sa-update and my own channel. sa-update queries
> > for A record of strange domain:
> > 
> > # /usr/bin/sa-update --channel sa.mejor.pl --no-gpg -vv
> > DNS TXT query: 2.4.3.sa.mejor.pl -> 3209
> > Update available for channel sa.mejor.pl: -1 -> 3209
> > DNS A query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> > DNS  query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> > channel: could not find working mirror, channel failed
> > Update failed, exiting with code 4
> > 
> > and this is what logged local resolver:
> > 2018-12-03T15:35:42.613624+01:00 jowisz unbound: [8540:0] info:
> > 127.0.0.1 update.sa.mejor.pl?sa-updates. A IN

> any ideas what can be wrong?

It looks like sa-update has lost support for paths in mirror URLs. The
SA mirrors don't currently have paths, but the commented-out dostech
entry suggests that they have been supported in the past.

If I edit the sa.mejor.pl mirror file and strip 'sa-updates/' from the
end, sa-update gets past the DNS error:


$ sa-update --channel sa.mejor.pl --no-gpg -vv  --updatedir /tmp
DNS TXT query: 2.4.3.sa.mejor.pl -> 3399
Update available for channel sa.mejor.pl: -1 -> 3399
DNS A query: update.sa.mejor.pl -> 193.33.111.90
fetching http://update.sa.mejor.pl/3399.tar.gz
http: (curl) GET http://update.sa.mejor.pl/3399.tar.gz, FAILED, status:
exit 22 Cannot open file /tmp/sa_mejor_pl/3399.tar.gz: No such file or
directory at /usr/local/bin/sa-update line 1599.

Re: [SA 3.4.2] sa-update doesn't see custom channel

[Marcin Mirosław]

W dniu 03.12.2018 o 15:42, Marcin Mirosław pisze:
> Hi!
> I have problem with sa-update and my own channel. sa-update queries for
> A record of strange domain:
> 
> # /usr/bin/sa-update --channel sa.mejor.pl --no-gpg -vv
> DNS TXT query: 2.4.3.sa.mejor.pl -> 3209
> Update available for channel sa.mejor.pl: -1 -> 3209
> DNS A query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> DNS  query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> channel: could not find working mirror, channel failed
> Update failed, exiting with code 4
> 
> and this is what logged local resolver:
> 2018-12-03T15:35:42.613624+01:00 jowisz unbound: [8540:0] info:
> 127.0.0.1 update.sa.mejor.pl?sa-updates. A IN
> 2018-12-03T15:35:42.617145+01:00 jowisz unbound: [8540:0] info:
> 127.0.0.1 update.sa.mejor.pl?sa-updates.  IN
> 
> Why sa-update queries for update.sa.mejor.pl?sa-updates (or
> update.sa.mejor.pl/sa-updates) domain?
> 
> I just run sa-update in debug mode, I paste relevant parts:
> [...]
> Dec  3 15:40:10.955 [24739] dbg: channel: attempting channel sa.mejor.pl
> Dec  3 15:40:10.955 [24739] dbg: channel: using existing directory
> /var/lib/spamassassin/3.004002/sa_mejor_pl
> Dec  3 15:40:10.955 [24739] dbg: channel: channel cf file
> /var/lib/spamassassin/3.004002/sa_mejor_pl.cf
> Dec  3 15:40:10.955 [24739] dbg: channel: channel pre file
> /var/lib/spamassassin/3.004002/sa_mejor_pl.pre
> DNS TXT query: 2.4.3.sa.mejor.pl -> 3209
> Dec  3 15:40:10.966 [24739] dbg: dns: 2.4.3.sa.mejor.pl => 3209, parsed
> as 3209
> Update available for channel sa.mejor.pl: -1 -> 3209
> Dec  3 15:40:10.967 [24739] dbg: channel: preparing temp directory for
> new channel
> Dec  3 15:40:10.967 [24739] dbg: channel: created tmp directory
> /tmp/.spamassassin24739FTCF1ttmp
> Dec  3 15:40:10.967 [24739] dbg: generic: lint checking site pre files
> once before attempting channel updates
> [...]
> Dec  3 15:40:11.189 [24739] dbg: channel: protocol family available:
> inet,inet6
> Dec  3 15:40:11.189 [24739] dbg: channel: reading MIRRORED.BY file
> /var/lib/spamassassin/3.004002/sa_mejor_pl/MIRRORED.BY
> Dec  3 15:40:11.189 [24739] dbg: channel: parsing MIRRORED.BY file for
> channel sa.mejor.pl
> Dec  3 15:40:11.189 [24739] dbg: channel: found mirror
> http://update.sa.mejor.pl/sa-updates/
> Dec  3 15:40:11.193 [24739] dbg: dns: query failed:
> update.sa.mejor.pl/sa-updates => NXDOMAIN
> DNS A query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> Dec  3 15:40:11.194 [24739] dbg: dns: query failed:
> update.sa.mejor.pl/sa-updates => NXDOMAIN
> DNS  query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
> Dec  3 15:40:11.195 [24739] dbg: generic: reject mirror
> http://update.sa.mejor.pl/sa-updates: no common address family (IPv4 IPv6)
> channel: could not find working mirror, channel failed
> 
> # cat /var/lib/spamassassin/3.004002/sa_mejor_pl/MIRRORED.BY
> http://update.sa.mejor.pl/sa-updates/
> 
> Something changed how channel should be configured beetwen 3.4.1 and 3.4.2?
> 


Hi,
any ideas what can be wrong?
Marcin

Re: repeated sa-update problems

[Matus UHLAR - fantomas]

Hello,

revoking old thread.

I think I have found some of problems:

1.

looking at old debug outputs, seems that everytime we've had problem,
sa-update first tried to fetch from http://sa-update.spamassassin.org

further fetching from other mirrors did not help, due to issues below.
I feel that I incorrectly blamed the other mirrors for this, sorry.

I don't have output from Sep 05 and Sep 20 stored anymore, but I can guess
it was the same problem.

I asked out fortinet team to look at that issue.
I am currently unable to fetch the update from sa-update.spamassassin.org
- are there any download limits implemented on that server?


2. 


when those problems happened, curl returned "18", in 3.4.0 shown as 4608
(18*256) which means:

  18 Partial file. Only a part of the file was transferred.

- just as it did today:

Dec 17 07:06:59.051 [11809] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 --fail -o 1849014.tar.gz -- 
http://sa-update.spamassassin.org/1849014.tar.gz
Dec 17 07:07:17.618 [11809] dbg: http: process [11812], exit status: exit 18

so, only partial content, but IS returned.

Since the while() loop checks for content returned, and since http_get
function returns $out_fname even if curl did not return 0, the sa-update
does NOT move to next mirror and the checksum comparison fails.

- the 3.4.0 sa-update did detect this problem and http_get only returned
 content when curl exited with status 0


3. 


when curl returns 18 and leaves the target file on filesystem, the filename
does not have original file's timestamp.

when file exists, the "-z filename" is appended to the curl command line,
which causes curl to fail, since the stored timestamp is newer.

the "-z" documentation said it fetches file modified later or
before the given time, which we don't apparently want but:

the "-z" seems to cause If-Modified-Since: header to be appended into the
request, which means it only fetches files newer than which we already have.

According to this, I believe that the "-z" option for CURL should be
dropped.



On 20.09.18 16:05, Matus UHLAR - fantomas wrote:

I looked at update times and they are different each day - debian script
sleeps random number of seconds (up to one hour) in order to lower the
impact at mirror servers.

I have removed the "--fail" option from curl and will look at error message
if there's any.

I'll keep you updated and will fill bugreport if I'm able to find out
anything useful.


On 08.10.18 16:43, Matus UHLAR - fantomas wrote:

I was able to repeat this problem now:

# /usr/bin/curl --verbose -L -O --remote-time -g --max-redirs 2 
--connect-timeout 30 --max-time 300 -o 1843052.tar.gz -- 
http://sa-update.spamassassin.org/1843052.tar.gz
* Hostname was NOT found in DNS cache
% Total% Received % Xferd  Average Speed   TimeTime Time  Current
   Dload  Upload   Total   SpentLeft  Speed
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0*   
Trying 64.142.56.146...
* Connected to sa-update.spamassassin.org (64.142.56.146) port 80 (#0)

GET /1843052.tar.gz HTTP/1.1
User-Agent: curl/7.38.0
Host: sa-update.spamassassin.org
Accept: */*


0 00 00 0  0  0 --:--:--  0:00:11 --:--:-- 0< 
HTTP/1.1 200 OK
< Date: Mon, 08 Oct 2018 14:16:19 GMT
* Server Apache/2.4.6 (CentOS) is not blacklisted
< Server: Apache/2.4.6 (CentOS)
< Last-Modified: Mon, 08 Oct 2018 03:19:20 GMT
< ETag: "4600c-577af16429e00"
< Accept-Ranges: bytes
< Content-Length: 286732
< Content-Type: application/x-gzip
<
{ [data not shown]
0  280k0 10 0  0  0 --:--:--  0:00:13 --:--:-- 0* 
transfer closed with 286731 bytes remaining to read
* Closing connection 0
curl: (18) transfer closed with 286731 bytes remaining to read


# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root  1 Oct  8 16:16 1843052.tar.gz

look at today's debug log says:

Oct  8 07:12:59.899 [20257] dbg: channel: selected mirror 
http://sa-update.spamassassin.org
Oct  8 07:12:59.899 [20257] dbg: http: url: 
http://sa-update.spamassassin.org/1843052.tar.gz
Oct  8 07:12:59.899 [20257] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1843052.tar.gz, new
Oct  8 07:12:59.899 [20257] dbg: util: executable for curl was found at 
/usr/bin/curl
Oct  8 07:12:59.899 [20257] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843052.tar.gz -- 
http://sa-update.spamassassin.org/1843052.tar.gz
Oct  8 07:13:15.385 [20257] dbg: http: process [20258], exit status: 4608
Oct  8 07:13:15.385 [20257] dbg: channel: selected mirror 
http://sa-update.ena.com
Oct  8 07:13:15.385 [20257] dbg: http: url: 
http://sa-update.ena.com/1843052.tar.gz
Oct  8 07:13:15.385 [20257] dbg: http: downloading to: 
/var/lib/spamassass

Re: sa-update error - config: invalid expression for rule T_MIXED_ES (fwd)

[Bill Cole]

On 9 Dec 2018, at 12:50, Kevin Walton wrote:


Hi

I am running:

SpamAssassin version 3.3.2
sa-update version svn917659


This is an unsupported obsolete version of SpamAssassin. If you are 
maintaining your own installation, you should update. If you are using a 
distribution maintained by someone else, you should urge them to upgrade 
their SA package or to backport a fix for this bug.


and I have been getting the following error for the last couple of 
days now when cron runs sa-update:


"
config:  invalid expression for rule T_MIXED_ES: "( __LOWER_E > 20 ) 
&& ( __E_LIKE_LETTER >
( (__LOWER_E * 14 ) / 10) ) && ( ( __E_LIKE_LETTER / __LOWER_E ) < 10 
)": division by zero

possible


Obviously, that's incorrect. As a Perl expression, the fact that '( 
__LOWER_E > 20 )' is first and the use of the short-circuiting logical 
and operator '&&' assures that the only division by a variable '( 
__E_LIKE_LETTER / __LOWER_E )' will never be evaluated if it could 
result in a division by zero.


This would be a bug in either SA v3.3.2 or the version of Perl you are 
using or in the combination of the two.



config: warning:  description exists for non-existent rule T_MIXED_ES

channel: lint check of update failed, channel failed
"


These are secondary to the initial incorrect judgment of T_MIXED_ES 
being invalid.



Running with -D, the error seems to be with the FreeMail plugin:


Not really...


"
Dec  9 16:12:19.254 [2525] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::FreeMail from @INC
Dec  9 16:12:19.303 [2525] dbg: plugin: 
Mail::SpamAssassin::Plugin::FreeMail=HASH(0xc0d8f14) implements 
'parse_config', priority 0


This is noted because the plugin has a callback defined for 
'parse_config' which is called by Mail::SpamAssassin::Parser right 
before it proceeds to judging rules.


config:  invalid expression for rule T_MIXED_ES: "( __LOWER_E > 20 ) 
&& ( __E_LIKE_LETTER > ( (__LOWER_E * 14 ) / 10) ) && (

  ( __E_LIKE_LETTER / __LOWER_E ) < 10 )": division by zero possible

config: warning:  description exists for non-existent rule T_MIXED_ES
"

A google doesnt seem to bring up any help.  Any pointers much 
appretiated.


Update to non-obsolete software.

--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole

sa-update error - config: invalid expression for rule T_MIXED_ES (fwd)

[Kevin Walton]

Hi

I am running:

SpamAssassin version 3.3.2
sa-update version svn917659

and I have been getting the following error for the last couple of days now 
when cron runs sa-update:


"
config:  invalid expression for rule T_MIXED_ES: "( __LOWER_E > 20 ) && ( 
__E_LIKE_LETTER >
( (__LOWER_E * 14 ) / 10) ) && ( ( __E_LIKE_LETTER / __LOWER_E ) < 10 )": 
division by zero

possible

config: warning:  description exists for non-existent rule T_MIXED_ES

channel: lint check of update failed, channel failed
"

Running with -D, the error seems to be with the FreeMail plugin:

"
Dec  9 16:12:19.254 [2525] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::FreeMail from @INC
Dec  9 16:12:19.303 [2525] dbg: plugin: 
Mail::SpamAssassin::Plugin::FreeMail=HASH(0xc0d8f14) implements 'parse_config', 
priority 0
config:  invalid expression for rule T_MIXED_ES: "( __LOWER_E > 20 ) && ( 
__E_LIKE_LETTER > ( (__LOWER_E * 14 ) / 10) ) && (

  ( __E_LIKE_LETTER / __LOWER_E ) < 10 )": division by zero possible

config: warning:  description exists for non-existent rule T_MIXED_ES
"

A google doesnt seem to bring up any help.  Any pointers much appretiated.

Thanks very much
Kevin


--
Kevin Walton

[SA 3.4.2] sa-update doesn't see custom channel

[Marcin Mirosław]

Hi!
I have problem with sa-update and my own channel. sa-update queries for
A record of strange domain:

# /usr/bin/sa-update --channel sa.mejor.pl --no-gpg -vv
DNS TXT query: 2.4.3.sa.mejor.pl -> 3209
Update available for channel sa.mejor.pl: -1 -> 3209
DNS A query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
DNS  query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
channel: could not find working mirror, channel failed
Update failed, exiting with code 4

and this is what logged local resolver:
2018-12-03T15:35:42.613624+01:00 jowisz unbound: [8540:0] info:
127.0.0.1 update.sa.mejor.pl?sa-updates. A IN
2018-12-03T15:35:42.617145+01:00 jowisz unbound: [8540:0] info:
127.0.0.1 update.sa.mejor.pl?sa-updates.  IN

Why sa-update queries for update.sa.mejor.pl?sa-updates (or
update.sa.mejor.pl/sa-updates) domain?

I just run sa-update in debug mode, I paste relevant parts:
[...]
Dec  3 15:40:10.955 [24739] dbg: channel: attempting channel sa.mejor.pl
Dec  3 15:40:10.955 [24739] dbg: channel: using existing directory
/var/lib/spamassassin/3.004002/sa_mejor_pl
Dec  3 15:40:10.955 [24739] dbg: channel: channel cf file
/var/lib/spamassassin/3.004002/sa_mejor_pl.cf
Dec  3 15:40:10.955 [24739] dbg: channel: channel pre file
/var/lib/spamassassin/3.004002/sa_mejor_pl.pre
DNS TXT query: 2.4.3.sa.mejor.pl -> 3209
Dec  3 15:40:10.966 [24739] dbg: dns: 2.4.3.sa.mejor.pl => 3209, parsed
as 3209
Update available for channel sa.mejor.pl: -1 -> 3209
Dec  3 15:40:10.967 [24739] dbg: channel: preparing temp directory for
new channel
Dec  3 15:40:10.967 [24739] dbg: channel: created tmp directory
/tmp/.spamassassin24739FTCF1ttmp
Dec  3 15:40:10.967 [24739] dbg: generic: lint checking site pre files
once before attempting channel updates
[...]
Dec  3 15:40:11.189 [24739] dbg: channel: protocol family available:
inet,inet6
Dec  3 15:40:11.189 [24739] dbg: channel: reading MIRRORED.BY file
/var/lib/spamassassin/3.004002/sa_mejor_pl/MIRRORED.BY
Dec  3 15:40:11.189 [24739] dbg: channel: parsing MIRRORED.BY file for
channel sa.mejor.pl
Dec  3 15:40:11.189 [24739] dbg: channel: found mirror
http://update.sa.mejor.pl/sa-updates/
Dec  3 15:40:11.193 [24739] dbg: dns: query failed:
update.sa.mejor.pl/sa-updates => NXDOMAIN
DNS A query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
Dec  3 15:40:11.194 [24739] dbg: dns: query failed:
update.sa.mejor.pl/sa-updates => NXDOMAIN
DNS  query update.sa.mejor.pl/sa-updates failed: NXDOMAIN
Dec  3 15:40:11.195 [24739] dbg: generic: reject mirror
http://update.sa.mejor.pl/sa-updates: no common address family (IPv4 IPv6)
channel: could not find working mirror, channel failed

# cat /var/lib/spamassassin/3.004002/sa_mejor_pl/MIRRORED.BY
http://update.sa.mejor.pl/sa-updates/

Something changed how channel should be configured beetwen 3.4.1 and 3.4.2?

Marcin

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Friday, 2 November 2018 3:45:08 ACDT RW wrote:
> On Wed, 31 Oct 2018 22:59:55 +1030
> 
> Rodney Baker wrote:
> > On Wednesday, 31 October 2018 7:29:51 ACDT RW wrote:
> > > curl --verbose -L -O --remote-time -g --max-redirs 2
> > > --connect-timeout 30 --max-time 300
> > > http://spamassassin.apache.org/updates/MIRRORED.BY
> > 
> > Here's the output from that command:
> > 
> > 
> > < HTTP/1.1 200 OK
> 
> ...
> 
> > { [data not shown]
> 
> So curl is working.

So, I got the error reported again. I tried running the curl command suggested 
above, and it appeared to complete successfully. I then ran sa-update, and got 
the error message. 


root@mailpi ~ # curl --verbose -L -O --remote-time -g --max-redirs 2 --
connect-timeout 30 --max-time 300 http://spamassassin.apache.org/updates/
MIRRORED.BY
* Hostname was NOT found in DNS cache
  % Total% Received % Xferd  Average Speed   TimeTime Time  
Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0*   Trying 95.216.24.32...
  0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 
0* Connected to spamassassin.apache.org (95.216.24.32) port 80 (#0)
> GET /updates/MIRRORED.BY HTTP/1.1
> User-Agent: curl/7.38.0
> Host: spamassassin.apache.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 06 Nov 2018 11:36:32 GMT
* Server Apache/2.4.18 (Ubuntu) is not blacklisted
< Server: Apache/2.4.18 (Ubuntu)
< Last-Modified: Sat, 27 Oct 2018 16:35:00 GMT
< ETag: "576-579386aca20a2"
< Accept-Ranges: bytes
< Content-Length: 1398
<
{ [data not shown]
100  1398  100  13980 0615  0  0:00:02  0:00:02 --:--:--   615
* Connection #0 to host spamassassin.apache.org left intact

root@mailpi ~ # sa-update
error: unable to refresh mirrors file for channel updates.spamassassin.org, 
using old file
root@mailpi ~ #
---

This does not appear to be a problem with curl, per se, but rather something 
related to sa-update.

-- 
==
Rodney Baker
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Friday, 2 November 2018 3:45:08 ACDT RW wrote:
> On Wed, 31 Oct 2018 22:59:55 +1030
> 
> Rodney Baker wrote:
> > On Wednesday, 31 October 2018 7:29:51 ACDT RW wrote:
> > > curl --verbose -L -O --remote-time -g --max-redirs 2
> > > --connect-timeout 30 --max-time 300
> > > http://spamassassin.apache.org/updates/MIRRORED.BY
> > 
> > Here's the output from that command:
> > 
> > 
> > < HTTP/1.1 200 OK
> 
> ...
> 
> > { [data not shown]
> 
> So curl is working.

Apparently so, at least when run manually. I haven't seen the error repeat for 
close to a week now, so whatever the problem was may have been a transient 
issue that has now been resolved. I'll keep an eye out and report back if it 
happens again.

Regards,
Rodney.

-- 
==
Rodney Baker
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[RW]

On Wed, 31 Oct 2018 22:59:55 +1030
Rodney Baker wrote:

> On Wednesday, 31 October 2018 7:29:51 ACDT RW wrote:

> > curl --verbose -L -O --remote-time -g --max-redirs 2
> > --connect-timeout 30 --max-time 300
> > http://spamassassin.apache.org/updates/MIRRORED.BY  

> 
> Here's the output from that command:
> 
 
> < HTTP/1.1 200 OK 
...
> { [data not shown] 


So curl is working.

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Wednesday, 31 October 2018 7:29:51 ACDT RW wrote:
> On Mon, 29 Oct 2018 09:07:09 -0400
> 
> Kevin A. McGrail wrote:
> > On 10/29/2018 8:03 AM, Rodney Baker wrote:
> > > re: renaming curl and using wget
> > > 
> > > Thanks. Did that - it worked with wget instead of curl, successfully
> > > downloading the mirrors file and all updates. The last failure with
> > > curl was last night.
> 
> Can you try
> 
> curl --verbose -L -O --remote-time -g --max-redirs 2 --connect-timeout 30
> --max-time 300 http://spamassassin.apache.org/updates/MIRRORED.BY
> > Then you are likely caught up in the buggy curl we've seen on other
> > distros.  Thanks for helping confirm it.
> 
> I don't think anything has been confirmed. The problem with curl
> failing on an http uri redirecting to https is no longer relevant, and
> presumably curl has successfully fetched the mirror file since the
> current version of SpamAssassin was installed or there wouldn't be a
> cached copy.

Here's the output from that command:

root@mailpi ~ # curl --verbose -L -O --remote-time -g --max-redirs 2 --
connect-timeout 30 --max-time 300 http://spamassassin.apache.org/updates/
MIRRORED.BY 
* Hostname was NOT found in DNS cache 
 % Total% Received % Xferd  Average Speed   TimeTime Time  Current 
Dload  Upload   Total   SpentLeft  Speed 
 0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0*  
 
Trying 40.79.78.1... 
 0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0* 
Connected to spamassassin.apache.org (40.79.78.1) port 80 (#0) 
> GET /updates/MIRRORED.BY HTTP/1.1 
> User-Agent: curl/7.38.0 
> Host: spamassassin.apache.org 
> Accept: */* 
> 
< HTTP/1.1 200 OK 
< Date: Wed, 31 Oct 2018 12:26:54 GMT 
* Server Apache/2.4.18 (Ubuntu) is not blacklisted 
< Server: Apache/2.4.18 (Ubuntu) 
< Last-Modified: Sat, 27 Oct 2018 16:35:00 GMT 
< ETag: "576-579386aca20a2" 
< Accept-Ranges: bytes 
< Content-Length: 1398 
< 
{ [data not shown] 
100  1398  100  13980 0707  0  0:00:01  0:00:01 --:--:--   707 
* Connection #0 to host spamassassin.apache.org left intact


-- 
==
Rodney Baker VK5ZTV
rodney.ba...@iinet.net.au
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Wednesday, 31 October 2018 7:29:51 ACDT RW wrote:
> On Mon, 29 Oct 2018 09:07:09 -0400
> 
> Kevin A. McGrail wrote:
> > On 10/29/2018 8:03 AM, Rodney Baker wrote:
> > > re: renaming curl and using wget
> > > 
> > > Thanks. Did that - it worked with wget instead of curl, successfully
> > > downloading the mirrors file and all updates. The last failure with
> > > curl was last night.
> 
> Can you try
> 
> curl --verbose -L -O --remote-time -g --max-redirs 2 --connect-timeout 30
> --max-time 300 http://spamassassin.apache.org/updates/MIRRORED.BY
> > Then you are likely caught up in the buggy curl we've seen on other
> > distros.  Thanks for helping confirm it.
> 
> I don't think anything has been confirmed. The problem with curl
> failing on an http uri redirecting to https is no longer relevant, and
> presumably curl has successfully fetched the mirror file since the
> current version of SpamAssassin was installed or there wouldn't be a
> cached copy.

Here's the output from that command:

root@mailpi ~ # curl --verbose -L -O --remote-time -g --max-redirs 2 --
connect-timeout 30 --max-time 300 http://spamassassin.apache.org/updates/
MIRRORED.BY 
* Hostname was NOT found in DNS cache 
 % Total% Received % Xferd  Average Speed   TimeTime Time  Current 
Dload  Upload   Total   SpentLeft  Speed 
 0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0*  
 
Trying 40.79.78.1... 
 0 00 00 0  0  0 --:--:--  0:00:01 --:--:-- 0* 
Connected to spamassassin.apache.org (40.79.78.1) port 80 (#0) 
> GET /updates/MIRRORED.BY HTTP/1.1 
> User-Agent: curl/7.38.0 
> Host: spamassassin.apache.org 
> Accept: */* 
> 
< HTTP/1.1 200 OK 
< Date: Wed, 31 Oct 2018 12:26:54 GMT 
* Server Apache/2.4.18 (Ubuntu) is not blacklisted 
< Server: Apache/2.4.18 (Ubuntu) 
< Last-Modified: Sat, 27 Oct 2018 16:35:00 GMT 
< ETag: "576-579386aca20a2" 
< Accept-Ranges: bytes 
< Content-Length: 1398 
< 
{ [data not shown] 
100  1398  100  13980 0707  0  0:00:01  0:00:01 --:--:--   707 
* Connection #0 to host spamassassin.apache.org left intact


-- 
==
Rodney Baker VK5ZTV
rodney.ba...@iinet.net.au
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[RW]

On Mon, 29 Oct 2018 09:07:09 -0400
Kevin A. McGrail wrote:

> On 10/29/2018 8:03 AM, Rodney Baker wrote:
> > re: renaming curl and using wget  
> 
> > Thanks. Did that - it worked with wget instead of curl, successfully
> > downloading the mirrors file and all updates. The last failure with
> > curl was last night.

Can you try 

curl --verbose -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 
--max-time 300 http://spamassassin.apache.org/updates/MIRRORED.BY

> Then you are likely caught up in the buggy curl we've seen on other
> distros.  Thanks for helping confirm it.

I don't think anything has been confirmed. The problem with curl
failing on an http uri redirecting to https is no longer relevant, and
presumably curl has successfully fetched the mirror file since the
current version of SpamAssassin was installed or there wouldn't be a
cached copy. 

Re: Error running sa-update - cannot refresh mirrors file

[Kevin A. McGrail]

On 10/29/2018 8:03 AM, Rodney Baker wrote:
> re: renaming curl and using wget

> Thanks. Did that - it worked with wget instead of curl, successfully
> downloading the mirrors file and all updates. The last failure with
> curl was last night.
>
Then you are likely caught up in the buggy curl we've seen on other
distros.  Thanks for helping confirm it.  Upgrade curl, upgrade your OS
or stay with wget.

Regards,

KAM

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On 29/10/2018 1:47, RW wrote:

On Sun, 28 Oct 2018 21:34:33 +1030
Rodney Baker wrote:


On Saturday, 27 October 2018 0:14:32 ACDT Kevin A. McGrail wrote:

It does concern me that it can't pull the file.  I had added an
https redirect which broke a lot of places using a broken version
of curl.


According to bug 7626 you removed that. I'm not seeing a redirect.


Can you update curl?  Is wget available instead and try removing
curl?

Curl is the latest version available for the Rasbian version running
on my box. I'm not thrilled about updating to a newer Raspbian
distribution version given that it is so stable as-is. I could try
wget instead - does SA require configuration to tell it to use wget
instead of curl?

The easiest thing to do is to temporarily rename curl.

Thanks. Did that - it worked with wget instead of curl, successfully 
downloading the mirrors file and all updates. The last failure with curl 
was last night.


--
==
Rodney Baker
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[RW]

On Sun, 28 Oct 2018 21:34:33 +1030
Rodney Baker wrote:

> On Saturday, 27 October 2018 0:14:32 ACDT Kevin A. McGrail wrote:

> > 
> > It does concern me that it can't pull the file.  I had added an
> > https redirect which broke a lot of places using a broken version
> > of curl.


According to bug 7626 you removed that. I'm not seeing a redirect.

> > Can you update curl?  Is wget available instead and try removing
> > curl?
>

> Curl is the latest version available for the Rasbian version running
> on my box. I'm not thrilled about updating to a newer Raspbian
> distribution version given that it is so stable as-is. I could try
> wget instead - does SA require configuration to tell it to use wget
> instead of curl? 

The easiest thing to do is to temporarily rename curl. 

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Saturday, 27 October 2018 0:14:32 ACDT Kevin A. McGrail wrote:
> On 10/26/2018 9:30 AM, Rodney Baker wrote:
> > On Friday, 26 October 2018 23:46:18 ACDT RW wrote:
> >> On Fri, 26 Oct 2018 22:40:54 +1030
> >> 
> >> Rodney Baker wrote:
> >>> Should I be concerned about the error updating the mirrors file?
> >> 
> >> No. sa-update tries to update it after a week so you pick-up new
> >> servers and spread the load, but the old one will probably still be
> >> usable for years.
> >> 
> >> The only time it matters is after a SpamAssassin version change when
> >> a new versioned directory is created to hold the rules. Even then you
> >> could just copy over the old MIRRORED.BY file if the server is down.
> > 
> > Ok, thanks - that's good to know.
> 
> It does concern me that it can't pull the file.  I had added an https
> redirect which broke a lot of places using a broken version of curl.
> 
> Can you update curl?  Is wget available instead and try removing curl?
> 
> Regards,
> KAM

Hi Kevin. Your reply came to me direct rather than via the mailing list - I 
suspect my mailer didn't set "Reply-To" correctly when emailing the list. I'll 
need to fix that. 

Curl is the latest version available for the Rasbian version running on my 
box. I'm not thrilled about updating to a newer Raspbian distribution version 
given that it is so stable as-is. I could try wget instead - does SA require 
configuration to tell it to use wget instead of curl? If so, where? 

Thanks
Rodney.

-- 
==
Rodney Baker
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

On Friday, 26 October 2018 23:46:18 ACDT RW wrote:
> On Fri, 26 Oct 2018 22:40:54 +1030
> 
> Rodney Baker wrote:
> > Should I be concerned about the error updating the mirrors file?
> 
> No. sa-update tries to update it after a week so you pick-up new
> servers and spread the load, but the old one will probably still be
> usable for years.
> 
> The only time it matters is after a SpamAssassin version change when
> a new versioned directory is created to hold the rules. Even then you
> could just copy over the old MIRRORED.BY file if the server is down.

Ok, thanks - that's good to know.

Regards,
Rodney.

-- 
==
Rodney Baker
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: Error running sa-update - cannot refresh mirrors file

[RW]

On Fri, 26 Oct 2018 22:40:54 +1030
Rodney Baker wrote:


> Should I be concerned about the error updating the mirrors file?

No. sa-update tries to update it after a week so you pick-up new
servers and spread the load, but the old one will probably still be
usable for years. 

The only time it matters is after a SpamAssassin version change when
a new versioned directory is created to hold the rules. Even then you
could just copy over the old MIRRORED.BY file if the server is down. 

Error running sa-update - cannot refresh mirrors file

[Rodney Baker]

Hi all,

I'm getting the following error when running sa-update on my Raspberry Pi 
(running spamc/spamd with compiled rulesets); 

root@mailpi ~ # sa-update --verbose
Update available for channel updates.spamassassin.org: 1844624 -> 1844740
http: (curl) GET http://spamassassin.apache.org/updates/MIRRORED.BY, FAILED, 
status: 1792
error: unable to refresh mirrors file for channel updates.spamassassin.org, 
using old file
http: (curl) GET http://sa-update.bitwell.fi/1844740.tar.gz, FAILED, status: 
1792
http: (curl) GET http://sa-update.razx.cloud/1844740.tar.gz, success
http: (curl) GET http://sa-update.razx.cloud/1844740.tar.gz.sha1, success
http: (curl) GET http://sa-update.razx.cloud/1844740.tar.gz.asc, success
Update was available, and was downloaded and installed successfully

Should I be concerned about the error updating the mirrors file?

Thanks in advance,
Rodney.

-- 
==
Rodney Baker VK5ZTV
rod...@jeremiah31-10.net
CCNA #CSCO12880208
==

Re: sa-compile after sa-update

[Kenneth Porter]

An RH bug was opened and closed on this in 2014:

https://bugzilla.redhat.com/show_bug.cgi?id=1151565

I attached a patch to the bug for the latest sa-update.cron script from 
the 3.4.2 RPM to invoke sa-compile if the plugin is enabled and re2c is 
installed.

sa-compile after sa-update

[Kenneth Porter]

I'm experimenting with the Rule2XSBody plugin and I've figured out that I 
have to run sa-compile after sa-update to create the compiled versions of 
local rules. I don't see anything in either sa-update or the Red 
Hat-supplied sa-update.cronscript invoked from cron (or a systemd timer) 
that invokes sa-compile. Where the logical place to hook this? (It should 
only be invoked if the plugin is loaded in /etc/mail/spamassassin 
somewhere.) Should I open an enhancement request against Red Hat's package? 
What are other distros doing with sa-compile? 

Re: repeated sa-update problems

[Matus UHLAR - fantomas]

On 20.09.18 16:05, Matus UHLAR - fantomas wrote:

I looked at update times and they are different each day - debian script
sleeps random number of seconds (up to one hour) in order to lower the
impact at mirror servers.

I have removed the "--fail" option from curl and will look at error message
if there's any.

I'll keep you updated and will fill bugreport if I'm able to find out
anything useful.


I was able to repeat this problem now:

# /usr/bin/curl --verbose -L -O --remote-time -g --max-redirs 2 
--connect-timeout 30 --max-time 300 -o 1843052.tar.gz -- 
http://sa-update.spamassassin.org/1843052.tar.gz
* Hostname was NOT found in DNS cache
 % Total% Received % Xferd  Average Speed   TimeTime Time  Current
Dload  Upload   Total   SpentLeft  Speed
 0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0*  
 Trying 64.142.56.146...
* Connected to sa-update.spamassassin.org (64.142.56.146) port 80 (#0)

GET /1843052.tar.gz HTTP/1.1
User-Agent: curl/7.38.0
Host: sa-update.spamassassin.org
Accept: */*


 0 00 00 0  0  0 --:--:--  0:00:11 --:--:-- 0< 
HTTP/1.1 200 OK
< Date: Mon, 08 Oct 2018 14:16:19 GMT
* Server Apache/2.4.6 (CentOS) is not blacklisted
< Server: Apache/2.4.6 (CentOS)
< Last-Modified: Mon, 08 Oct 2018 03:19:20 GMT
< ETag: "4600c-577af16429e00"
< Accept-Ranges: bytes
< Content-Length: 286732
< Content-Type: application/x-gzip
<
{ [data not shown]
 0  280k0 10 0  0  0 --:--:--  0:00:13 --:--:-- 0* 
transfer closed with 286731 bytes remaining to read
* Closing connection 0
curl: (18) transfer closed with 286731 bytes remaining to read


# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root  1 Oct  8 16:16 1843052.tar.gz

look at today's debug log says:

Oct  8 07:12:59.899 [20257] dbg: channel: selected mirror 
http://sa-update.spamassassin.org
Oct  8 07:12:59.899 [20257] dbg: http: url: 
http://sa-update.spamassassin.org/1843052.tar.gz
Oct  8 07:12:59.899 [20257] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1843052.tar.gz, new
Oct  8 07:12:59.899 [20257] dbg: util: executable for curl was found at 
/usr/bin/curl
Oct  8 07:12:59.899 [20257] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843052.tar.gz -- 
http://sa-update.spamassassin.org/1843052.tar.gz
Oct  8 07:13:15.385 [20257] dbg: http: process [20258], exit status: 4608
Oct  8 07:13:15.385 [20257] dbg: channel: selected mirror 
http://sa-update.ena.com
Oct  8 07:13:15.385 [20257] dbg: http: url: 
http://sa-update.ena.com/1843052.tar.gz
Oct  8 07:13:15.385 [20257] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1843052.tar.gz, update
Oct  8 07:13:15.385 [20257] dbg: util: executable for curl was found at 
/usr/bin/curl
Oct  8 07:13:15.385 [20257] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843052.tar.gz -z 
1843052.tar.gz -- http://sa-update.ena.com/1843052.tar.gz
Oct  8 07:13:15.889 [20257] dbg: http: process [20272], exit status: 0

This looks that invalid file was downloaded from sa-update.spamassassin.org,
and while next curl invocation succeeded with exit code 0, the file was not
overridden:

# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz -- 
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 243 Oct  8 16:21 1843052.tar.gz
# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz -- 
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 243 Oct  8 16:21 1843052.tar.gz
# rm 1843052.tar.gz
# /usr/bin/curl -s -L -O --remote-time -g --max-redirs 2 --connect-timeout 30 
--max-time 300 -o 1843052.tar.gz -z 1843052.tar.gz -- 
http://sa-update.ena.com/1843052.tar.gz
# ls -l 1843052.tar.gz
-rw-r--r-- 1 root root 286732 Oct  8 05:19 1843052.tar.gz

(the file size changed to 243 because of my tests).

further look at logs says that all failed downloads were from
sa-update.spamassassin.org:

Sep 28 07:43:07.888 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz -- 
http://sa-update.spamassassin.org/1842077.tar.gz
Sep 28 07:43:21.973 [7018] dbg: http: process [7019], exit status: 4608

Oct  5 06:35:10.552 [29702] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842787.tar.gz -- 
http://sa-update.spamassassin.org/1842787.tar.gz
Oct  5 06:35:29.199 [29702] dbg: http: process [29705], exit status: 4608

Oct  7 07:17:37.644 [30424] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1843008.tar.gz -- 

Re: sa-update and signature verification

[Kevin A. McGrail]

Hi Daniele, You are correct.  3.4.2 does not support rule channels that
only use SHA1.

Please contact the other rule channels and tell them to add sha256.  We
have moved away from SHA1.  It should be trivial on their end to
generate a sha256sum.

Regards,
KAM

On 10/2/2018 10:00 AM, Daniele Duca wrote:
> Hello,
>
> since updating to 3.4.2 I can't download rules from unofficial
> channels. The problem is that in version 3.4.1 sa-update checks the
> hash of the downloaded file using file.sha1 , while version 3.4.2 uses
> file.sha256 or file.sha512. See the relevant differences in the
> following sa-update --help:
>
>
> 3.4.1:
> sa-update --help
> ...
> --install filename  Install updates directly from this file.
> Signature verification will use "file.asc" and "file.sha1"
> ...
>
> 3.4.2
> sa-update --help
> ...
> --install filename  Install updates directly from this file.
> Signature verification will use "file.asc", "file.sha256", and
> "file.sha512".
> ...
>
>
> Using the --nogpg option doesn't help, sa-update still hardfails if it
> doesn't find one of the .sha(256|512) files.
>
> Reading the code in sa-update I found that even if --nogpg is
> specified, the signature file is still tried to be downloaded even if
> it's not used afterwards, and that is what basically causes the update
> procedure to fail.
> For the moment I brutally hacked sa-update to don't care about
> signatures when using unofficial channels, but I'd like to understand
> if I'm missing something obvious that doesn't require code mangling to
> use "old" update channels.
>
> Thanks
>
> Daniele Duca
>

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

sa-update and signature verification

[Daniele Duca]

Hello,

since updating to 3.4.2 I can't download rules from unofficial channels. 
The problem is that in version 3.4.1 sa-update checks the hash of the 
downloaded file using file.sha1 , while version 3.4.2 uses file.sha256 
or file.sha512. See the relevant differences in the following sa-update 
--help:



3.4.1:
sa-update --help
...
--install filename  Install updates directly from this file. 
Signature verification will use "file.asc" and "file.sha1"

...

3.4.2
sa-update --help
...
--install filename  Install updates directly from this file. 
Signature verification will use "file.asc", "file.sha256", and 
"file.sha512".

...


Using the --nogpg option doesn't help, sa-update still hardfails if it 
doesn't find one of the .sha(256|512) files.


Reading the code in sa-update I found that even if --nogpg is specified, 
the signature file is still tried to be downloaded even if it's not used 
afterwards, and that is what basically causes the update procedure to fail.
For the moment I brutally hacked sa-update to don't care about 
signatures when using unofficial channels, but I'd like to understand if 
I'm missing something obvious that doesn't require code mangling to use 
"old" update channels.


Thanks

Daniele Duca

Re: repeated sa-update problems

[Matus UHLAR - fantomas]

On 9/20/2018 8:59 AM, Dave Jones wrote:

I will have to check later if someone else can't check today.  I am at
a customer location where I don't have good VPN connection out and
will be traveling this evening.  I can check tomorrow if it can wait.


On 20.09.18 09:05, Kevin A. McGrail wrote:

It can wait.  Matus also had the issue hitting my mirror and I know I
don't use a CDN.


On 20.09.18 16:05, Matus UHLAR - fantomas wrote:

I looked at update times and they are different each day - debian script
sleeps random number of seconds (up to one hour) in order to lower the
impact at mirror servers.

I have removed the "--fail" option from curl and will look at error message
if there's any.

I'll keep you updated and will fill bugreport if I'm able to find out
anything useful.


the problem repeated today

Sep 28 07:43:07.888 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz -- 
http://sa-update.spamassassin.org/1842077.tar.gz
Sep 28 07:43:21.973 [7018] dbg: http: process [7019], exit status: 4608
Sep 28 07:43:21.973 [7018] dbg: channel: selected mirror 
http://sa-update.space-pro.be
Sep 28 07:43:21.974 [7018] dbg: http: url: 
http://sa-update.space-pro.be/1842077.tar.gz
Sep 28 07:43:21.974 [7018] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz, update
Sep 28 07:43:21.974 [7018] dbg: util: executable for curl was found at 
/usr/bin/curl
Sep 28 07:43:21.974 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz -z 
1842077.tar.gz -- http://sa-update.space-pro.be/1842077.tar.gz
Sep 28 07:43:22.304 [7018] dbg: http: process [7041], exit status: 0
Sep 28 07:43:22.305 [7018] dbg: http: url: 
http://sa-update.space-pro.be/1842077.tar.gz.sha1
Sep 28 07:43:22.305 [7018] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz.sha1, new
Sep 28 07:43:22.305 [7018] dbg: util: executable for curl was found at 
/usr/bin/curl
Sep 28 07:43:22.305 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz.sha1 -- 
http://sa-update.space-pro.be/1842077.tar.gz.sha1
Sep 28 07:43:22.376 [7018] dbg: http: process [7043], exit status: 0
Sep 28 07:43:22.376 [7018] dbg: http: url: 
http://sa-update.space-pro.be/1842077.tar.gz.asc
Sep 28 07:43:22.377 [7018] dbg: http: downloading to: 
/var/lib/spamassassin/3.004000/updates_spamassassin_org/1842077.tar.gz.asc, new
Sep 28 07:43:22.377 [7018] dbg: util: executable for curl was found at 
/usr/bin/curl
Sep 28 07:43:22.377 [7018] dbg: http: /usr/bin/curl -s -L -O --remote-time -g 
--max-redirs 2 --connect-timeout 30 --max-time 300 -o 1842077.tar.gz.asc -- 
http://sa-update.space-pro.be/1842077.tar.gz.asc
Sep 28 07:43:22.446 [7018] dbg: http: process [7045], exit status: 0
Sep 28 07:43:22.446 [7018] dbg: sha1: verification wanted: 
cb1b907b4f590fe24d0744cf60939685d51b3443
Sep 28 07:43:22.446 [7018] dbg: sha1: verification result: 
953efe8f531a5a87f6d2d5a65b78b05e55599abc
channel: SHA1 verification failed, channel failed

# ls -lctr --full-time 1842077.*
-rw-r--r-- 1 debian-spamd debian-spamd   1 2018-09-28 07:43:21.967880543 +0200 
1842077.tar.gz
-rw-r--r-- 1 debian-spamd debian-spamd 113 2018-09-28 07:43:22.371884772 +0200 
1842077.tar.gz.sha1
-rw-r--r-- 1 debian-spamd debian-spamd 819 2018-09-28 07:43:22.443885519 +0200 
1842077.tar.gz.asc

# hd 1842077.tar.gz
  1f|.|
0001

Sep 28 07:43:23 fgt 
date=2018-09-28,time=07:43:23,devname=xx,devid=xy,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=192.168.1.1,srcport=52411,srcintf="internal",dstip=176.28.55.20,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=87366444,proto=6,action=close,policyid=62,policytype=policy,dstcountry="France",srccountry="Reserved",trandisp=snat,transip=195.80.174.159,transport=52411,service="HTTP",duration=1,sentbyte=470,rcvdbyte=327,sentpkt=6,rcvdpkt=4,appcat="unscanned",wanin=111,wanout=150,lanin=150,lanout=111
Sep 28 07:43:23 fgt 
date=2018-09-28,time=07:43:23,devname=xx,devid=xy,logid=13,type=traffic,subtype=forward,level=notice,vd=root,srcip=192.168.1.1,srcport=52412,srcintf="internal",dstip=176.28.55.20,dstport=80,dstintf="wan1",poluuid=9a0df156-900e-51e8-d4d5-7b4de8e07615,sessionid=87366446,proto=6,action=close,policyid=62,policytype=policy,dstcountry="France",srccountry="Reserved",trandisp=snat,transip=195.80.174.159,transport=52412,service="HTTP",duration=1,sentbyte=425,rcvdbyte=550,sentpkt=6,rcvdpkt=4,appcat="unscanned",wanin=334,wanout=105,lanin=105,lanout=334
Sep 28 07:43:23 fgt 

Re: repeated sa-update problems

[RW]

On Thu, 20 Sep 2018 08:58:07 -0400
Kevin A. McGrail wrote:

> On 9/20/2018 8:35 AM, Matus UHLAR - fantomas wrote:
> > unfortunately, secnap is not the only mirror with which the problem
> > occurs.
> > - is this possible problem with mirrors?
> > - when do mirrors update?
> > - do mirrors updates propagate atomically?  
> Dave, is there an issue where DNS is being updated before the mirrors?

It seems unlikely. In the original example there was enough of a sha1
file to provide a 40 character hash and curl downloaded an update file
that was one byte long. I tried running the same command, but with the
version changed by 1 in the URL, and curl didn't create a file at all.

Also IIRC rsync does an atomic rename when a file is added, so the web
server should see  a complete file or no file.

Re: repeated sa-update problems

[Matus UHLAR - fantomas]

On 9/20/2018 8:59 AM, Dave Jones wrote:

I will have to check later if someone else can't check today.  I am at
a customer location where I don't have good VPN connection out and
will be traveling this evening.  I can check tomorrow if it can wait.


On 20.09.18 09:05, Kevin A. McGrail wrote:

It can wait.  Matus also had the issue hitting my mirror and I know I
don't use a CDN.


I looked at update times and they are different each day - debian script
sleeps random number of seconds (up to one hour) in order to lower the
impact at mirror servers.

I have removed the "--fail" option from curl and will look at error message
if there's any.

I'll keep you updated and will fill bugreport if I'm able to find out
anything useful.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...