Re: Newb on sa-learn - didn't get what I expected as a response...
On Fri, 7 Jul 2023, Reindl Harald wrote: OF COURSE! For me, THE key questions have to do with the learning aspect (and maybe logging): What's the directory that, for example, sa-learn has to write into? ... Again, pointers would be nice - it's not like I was planning to spend my day doing this; I have a customer visit planned that's coming up soon! I just don't have much time! sorry - i can't translate our configs and setup dating 9 years back and nothing in common with anything from the distribution - "sa-learn" needs to write where the bayes-db lives, nothing else No worries, you've been a big help! ... You and Jared Hall! Richard
Re: Newb on sa-learn - didn't get what I expected as a response...
On Fri, 7 Jul 2023, Reindl Harald wrote: /usr is package terriotory and MUST NOT BE owned by anybody than root and read-only for the world just give common sense another few seconds! only the files/folders which are supposed to be written by any deamon should be writeable for the user the daemnon is running with you don't want an exploit happening somewhere in teh filter chain modify your binaries/scripts OF COURSE! For me, THE key questions have to do with the learning aspect (and maybe logging): What's the directory that, for example, sa-learn has to write into? ... Again, pointers would be nice - it's not like I was planning to spend my day doing this; I have a customer visit planned that's coming up soon! I just don't have much time! Richard
spamd runs as root on Fedora Server 38 ?! - was Re: Newb on sa-learn - didn't get what I expected as a response...
Hi All, I changed the subject line to hopefully get some insight from a wider audience regarding this situation that Reindl uncovered: It started here: It appears that it IS running as root?! OR maybe as "sa-milt" ... As root I got this: # ps auxwww | grep spamd root 100805 0.0 0.3 158208 121164 ? Ss 00:37 0:05 /usr/bin/perl -T -w /usr/bin/spamd -c -m5 -H --razor-home-dir=/var/lib/razor/ --razor-log-file=sys-syslog Reindl replied: give common sense a few seconds: do you REALLY want to process mails containing junk and malware with root privileges? And went on to share that his Fedora 37 runs as sa-milt. There IS an sa-milt entry in /etc/passwd, so... I just took a few minutes to confirm that the DEFAULT installation on Fedora Server 38 runs spamd as root - at least, that's sure my take from this: # ps -auxwww | grep spam root 192531 2.3 4.0 158360 146936 ? Ss 08:53 0:01 /usr/bin/perl -T -w /usr/bin/spamd -c -m5 -H --razor-home-dir=/var/lib/razor/ --razor-log-file=sys-syslog root 192535 0.0 3.7 158360 137488 ? S08:53 0:00 spamd child root 192536 0.0 3.7 158360 137616 ? S08:53 0:00 spamd child ...GIVEN that this is the DEFAULT on this distribution - a very popular distribution - I'm ... speachless since, as Reindl points out, processing unknown inbound email is NOT a great place to hav a process running as root! THEREFORE: Can anyone give me the quick path to changing this to running as sa-milt, as his system does? Changing the file ownerships is trivial, and I know from doing some packaging for Fedora systems that there's a spot to give the user (and group) IDs programs are supposed to be run under in sysconfig. A quick look shows there are three for Spam Assassin on my system: /etc/sysconfig/spamassassin /etc/sysconfig/spamass-milter /etc/sysconfig/spamass-milter-postfix Before I make changes and possibly screw things up; any advice? Thanks! Richard
Re: Newb on sa-learn - didn't get what I expected as a response...
It appears that it IS running as root?! OR maybe as "sa-milt" ... As root I got this: # ps auxwww | grep spamd root 100805 0.0 0.3 158208 121164 ? Ss 00:37 0:05 /usr/bin/perl -T -w /usr/bin/spamd -c -m5 -H --razor-home-dir=/var/lib/razor/ --razor-log-file=sys-syslog # grep spam /etc/passwd sa-milt:x:976:975:SpamAssassin Milter:/var/lib/spamass-milter:/sbin/nologin So... run it as sa-milt (my guess), or as root? Note that this is on a Fedora Server v 38 - the OS is a couple of months old so your whole setup is more then questionable give common sense a few seconds: do you REALLY want to process mails containing junk and malware with root privileges? Frankly, you make a good point and I was unawares! Back January we had a system failure - nevermind the details - and had to reinstall the OS from scratch, then updated when the new version came out... And I _swear_ I did _NOT_ change anything regarding SA from the defaults not required to just get it running. (We didn't lose /etc, so I just plunked the existing Postfix config back in place and we were up and running!) My guess is that this is the default on Fedora Server, however, I have another system I can confirm that with - but not today, probably. that below is Fedora 37, originally from 2014 cloned from our golden-master VM dating back to 2008 with Fedora 9 not a single distro-systemd-unit in use - never [root@mail-gw:~]$ ps auxwww | grep spam sa-milt 436 0.0 1.2 69708 65192 ?SNs Jun16 11:09 /usr/bin/perl -T -w /usr/bin/spamd --max-children=1 --max-conn-per-child=1000 --local --socketpath=/run/spamd-debug/spamassassin.sock --socketmode=0666 --siteconfigpath=/etc/mail/spamassassin-debug --syslog=stderr 2>/dev/null ...OK, I get it!... I'm not sure "what went wrong" so we ended up with this, but I'm also not sure what the short path is to fixing this issue. There's already an sa-milt in /etc/passwd, but the files are all owned by root - eg: the files in /usr/share/spamassassin Surely these would need to be changed, one would think, and somewhere the code told to run as sa-milt, which I presume isn't THAT hard to find, though I've never dealt with it before. THANKS for pointing this out! Richard
Re: Newb on sa-learn - didn't get what I expected as a response...
(I was running it as root - which the docs don't mention but I figure is what I'm supposed to do!) why do you suppose that? ...Uh... Because otherwise why the -u flag and comments about running it for virtual users? you NEVER run anything as root which isn't a root task - no matter what you run it with the same user you spamd is running Good to know! ...I'd recommend an update to the doc / web page to point out it should be run as the user ID of whatever spamd is using! It appears that it IS running as root?! OR maybe as "sa-milt" ... As root I got this: # ps auxwww | grep spamd root 100805 0.0 0.3 158208 121164 ? Ss 00:37 0:05 /usr/bin/perl -T -w /usr/bin/spamd -c -m5 -H --razor-home-dir=/var/lib/razor/ --razor-log-file=sys-syslog # grep spam /etc/passwd sa-milt:x:976:975:SpamAssassin Milter:/var/lib/spamass-milter:/sbin/nologin So... run it as sa-milt (my guess), or as root? Note that this is on a Fedora Server v 38 - the OS is a couple of months old. Thanks again, Richard
Re: Newb on sa-learn - didn't get what I expected as a response...
On Fri, 7 Jul 2023, Jared Hall wrote: I believe the default format is Maildir. You mention a single file w/ multiple emails which suggests you might be running MBox format? If so, try the --mbox command line switch. -- Jared Hall GREAT CATCH, Jared; you are correct, mine are in mbox format, I think - somehow I guess I overlooked the -mbox switch! Thanks, Richard
Re: Newb on sa-learn - didn't get what I expected as a response...
Am 07.07.23 um 17:04 schrieb Richard: I've FINALLY built up a "corpus" of ham vs spam and also FINALLY had some time to spend on this and just ran sa-learn on, oh, IDK, some 10k email messages or so, I'd guess. And along the way, I NEVER ONCE got the kind of output response back from sa-learn that I expected. For example, here I run it against a file containing just over 2100 spam: $ sa-learn -u richard --spam spam Learned tokens from 0 message(s) (0 message(s) examined) (I was running it as root - which the docs don't mention but I figure is what I'm supposed to do!) why do you suppose that? ...Uh... Because otherwise why the -u flag and comments about running it for virtual users? you NEVER run anything as root which isn't a root task - no matter what you run it with the same user you spamd is running Good to know! ...I'd recommend an update to the doc / web page to point out it should be run as the user ID of whatever spamd is using! Now, I'd guess I should, as root: sa-learn --clear Since I hadn't run sa-learn before, EVER, that I was aware of! ...And THEN run as I've just learned. And, BTW, this makes me happy I scripted calling sa-learn, so re-doing this will be easy! As an aside, "curating" modern ham from my inboxes is time consuming so a lot of the ham I used is older, from saved folders... I saw the warning about old vs new, and the potential effects of that; as my inboxes typically have around 2k messages in them, and going through and making sure NONE are spam is time consuming, is it worth tossing in a few at a time from recent days, such as a day at a time? ...My guess is that nobody can really say what the Bayesian system is going to pick up on exactly, so YES, it can't hurt?! Thanks, Richard
Re: The rewrite_header Subject [SPAM] directive has stopped working?!
Hmmm... I think I'm close here! Thanks for the tip about procmail, and I was delighted to find that my system not only has procmail already installed but there was even an active - APPARENTLY active! - ~/.procmailrc ... that even already had Spam Assassin setup in it?! Nice! Here's what ha(d/s): ## #:0fw #| /usr/bin/spamassassin :0 * ^X-Spam-Status: *Yes * ^X-DSPAM-Result: *!Innocent $HOME/mail/spam/ ## Well... The only thing I want a tiny bit different there is to send it to Spam instead of spam, because I want to use one of them for confirmed spam, for possible future training, and the other for suspected spam. However, it's not actually doing anything at present... Can someone save me from reading a heck of a lot of the docs to find out how to configure this in WITHOUT creating a problem for using Dovecot, too? ...We DO need Dovecot, it's just not authenticating the imap connections properly and I just don't have time right now to focus on it. Parking the damned spam somehow is a great help. And, this is perhaps BETTER than gettting the subject line rewrite working again because it'll be automagically moved for folks! Win! Thanks much, Richard
Re: The rewrite_header Subject [SPAM] directive has stopped working?!
Hi Fokls, Before I get into the replies, so far, no solutions! More ideas? Now, here are my responses the the replies so far: First, thank you for all your replies! I'm avoiding replying to each by consolidating my response to them into this one mail. Normally I delete "all unnecessary materials," but I'll make an exception this time! On 2023-02-28 at 22:46:54 UTC-0500 (Tue, 28 Feb 2023 19:46:54 -0800 (PST)) Richard Troy is rumored to have said: Hi All, I've been subscribed for ... close to 15 years, I think? Heck, 20 is maybe possible! ... Just reading I have learned a hell of a lot, thanks to this community, but have never posted before. Now's the time, though, because I really need some help and am not sure where to look for it. (I've already done the basic searches - if I've missed something, I apologize.) Very recently our entire /var tree got wiped out due to a bug in a backup script someone was testing, and not only on our primary system but also on our alternate (backup) system too. Ouch! We've had to do a complete rebuild and apply what we can from backups. Date: Wed, 1 Mar 2023 09:03:44 +0100 From: Reindl Harald in other words: you don't have offsite backups on unconnected machines and no backup versioning - congratulations Presuming that was intended to be helpful and not sarcastic, yes, we have all those things and more - even spun down, removed disks and even the occasional set of DVDs for archival... We're almost completely ready for an EMP - which could come from a solar flair, you know! The reality is, however, that we first created this system WAY "back in the day" (1997, I think... it was Red Hat 1.1) and back then it wasn't really practical to backup whole system disk trees and the focus was on user data, which is how our backup system evolved. ... We have, for USER data, 24 hr complete live copy of everything, 48 hrs, 72hrs, a week copy renewed at the start of each week and a monthly copy refreshed on the start of each month... And, these backups are kept on two separate live systems, a primary and an alternate, with the software designed to handle an arbitrary number of additional alternates - we are planning on at least two alternates (for a total of three complete systems) live and ready to go "on a moment's notice", but just haven't gotten there yet since it has seemed to be a low priority. In the modern era - fairly recently - we've thought that it was time to take care of the system disk, with an emphais for a live copy as opposed to rebuilding the OS from disaster as a top priority while we sort through many terabytes of backups and reduce the huge number of duplicates of a lot of the data ... How many copies of the stuff we did in 2000, do we really need? One a month for 23 years?... And so that's been our focus of late and THEN we were going to look at completing the rest of this restructuring of backups. ... More funding would have helped a lot! So, we were caught with our pants down - it's embarrassing, but we'll live. BTW, despte this gaff, if anyone wants to know more about how we're doing things, which is pretty sophisticated, some of which is noted above, just send me an email. We have pretty good backups, mostly, but on /var? Well, you learn how good your backups are when you have a disaster just like this! And, it turns out, we didn't have a recent local.cf (or, for that matter a lot more). (We now backup /var and EVERYTHING in /! ... Good advice, now that disk space is dirt cheap!) Date: Wed, 01 Mar 2023 01:01:05 -0500 From: Bill Cole What was local.cf doing on /var? The standard location is in /etc/mail/spamassassin/. Sorry for any confusion; In short, we lost more than /var, it was just what came to mind as I typed because the loss of it was the reason the OS had to be rebuilt. What happened was that in order to help offload the "system disk", an SSD, from write loads (we don't trust them for anything but reads), things like var got moved off the disk and the bug in the backup script (never used for this purpose before!) was that it had the wrong case for a dash el argument - that is it was either -l when it should have been -L, or visa versa - and so everything below links got wiped out. Since /var is a high-update tree, moved! ... And, as we like to keep packages together and SA refreshes nightly via cron job, _all_ its components were moved, too... LIKELY this is a more complicated strategy than it should have been, but the OS wasn't designed based on this kind of concern and write loads are scattered. In our view, at present it's harder to offload heavy write loads completely than it should be and there ought to be a re-think of disk usage when it comes to directory tree design for the modern 'nix systems. As it is, doing this is rather hit-and-miss as there are few whole trees which contain primarily write loads. bla
The rewrite_header Subject [SPAM] directive has stopped working?!
Hi All, I've been subscribed for ... close to 15 years, I think? Heck, 20 is maybe possible! ... Just reading I have learned a hell of a lot, thanks to this community, but have never posted before. Now's the time, though, because I really need some help and am not sure where to look for it. (I've already done the basic searches - if I've missed something, I apologize.) Very recently our entire /var tree got wiped out due to a bug in a backup script someone was testing, and not only on our primary system but also on our alternate (backup) system too. Ouch! We've had to do a complete rebuild and apply what we can from backups. We have pretty good backups, mostly, but on /var? Well, you learn how good your backups are when you have a disaster just like this! And, it turns out, we didn't have a recent local.cf (or, for that matter a lot more). (We now backup /var and EVERYTHING in /! ... Good advice, now that disk space is dirt cheap!) The reason for posting is ultimately that the above denoted directive was working fine as I was trying to rebuild things - namely: rewrite_header Subject [SPAM] But at some point as I made some edits, SA continues to work, and honors everything else in the file so far as I have noted so far - such as required hits, which is directly above it - but the subject is no longer "rewritten" to include the above noted label. People have come to depend on it (because we don't move it to an alternative "folder" for them) so... Presuming this is NOT the place, where do I find help? Or, if someone just recognizes this, please do reply! All input welcome, thanks. I'd never bothered to try before, but since I'm here and you have the background, and I know it's slightly off topic: Is there an easy solution to delivering / moving spam to a specific "folder" not involving Dovecot on a Fedora / Postfix system? I know I could pull it off by directing all pre-mailbox delivery to a script I write myself (via the .forward mechanism if necessary), but I just don't have the time! Private replies welcome! Regards ... and thanks to the list for all the great and useful materials - just wish I could absorb it all! (I'm now trying to relearn years worth of stuff I've forgotten because I don't use it often enough! I only run this one site's systems as an SA!) Richard -- Richard Troy
Why the new changes need to be "depricated" forever
Hi Folks, I post infrequently - and intend to keep it that way - and want to ensure my posts have actual value to the community. First, I'm NOT a member of the d...@spamassassin.apache.org email list and I surely hope someone who is will kindly forward this email to that list. List member Oliver Nicole rightly makes the following observations - here excerpted - about the apparently not just proposed but apparently certain to happen changes to this project which will negatively impact a great many people, with a few in-line comments for context before my conclusion. To wit: From: Olivier To: Kevin A. McGrail Cc: users@spamassassin.apache.org, d...@spamassassin.apache.org Subject: Re: More Responses about Various Questions revolving around WelcomeLIst/BlockList changes [ ... lots deleted, this is just an excerpt ... ] The issue seems to be that you do not understand how real world is working. You assume a closed and controled system, which is far from the truth. Every user can build their own rules, they can have scripts that generate rules for them, things they put up years ago and they completely forget about because it is working fine. Yes, the above is clearly true. Few of us leave sufficient bread-crumbs to find our way back to understanding why we did what we did, etc. Most likely they will not see the message about the obsolescence, and one day, when compatibilty is over, their stuff will stop working and there will be no way to solve that ecvept to painfully go back to an older version of SA or manually go through all the problems of all the angry users. As a system administrator for some 37 years, and as someone who has acted in a support or consulting capacity to others in such role(s) for well over 30 years, I can tell you this observation is quite correct. In fact, this is the dominant circumstance, by far. VERY importantly, nobody wants to be stuck on old versions, as Oliver proposes will happen (and he's right about that), and so this puts system administrators in a VERY difficult position - a position I'd venture the proponents don't really understand. The choice is one painful one versus another painful one. Only someone who's actually been there really gets it. If you offer compatibility with only a warning message, most people will ignore (or simply not see) that message and do nothing. And when the compatibility is over, they will be facing a wall, just the same as if there were no compatibility period. You are just pushing the mayhem back by one year. I'd argue that most won't see it coming at all, though there is, of course, no way to prove that. But it's simple human nature; when we are overloaded, as nearly 100% of us perpetually are, we ignore a LOT of warnings we should have, with our better selves, seen coming, from our health issues like cancer to our children's issues to computer log files, it's just what happens; we're simply so busy in our daily lives just trying to get by that we miss signs we could have seen. The VAST majority of us are in economic instability, especially with the effects of this Covid-19 pandemic; to expect us to be paying close attention to warnings in logs is objectively silly. (Perhaps the proponents of this change are simply too comfortable in their economics and too isolated from actual users to see these truths.) ...I believe the above makes the case for why backwards-compatibility needs to be maintained into perpetuity, but Oliver actually suggests a neat way to solve this AND the political problem that openly saying that would create. He writes: In fact, I would even suggest that SA 4.0 come with the compatibility turned off, so the users are forced to notice the change, with a kind and visible message explaining how they can turn the compatibility on and that they should upgrade. Yes, this is, in fact, a BRILLIANT idea because the concept of a "backwards compatibility" flag in the configuration gives established users the ability to continue forward without undue pain while at the same time permitting the linguistically ignorant social justice warriors a clean victory. "YES, we have vanquished the evil, hurtful words blacklist and whitelist!" AND, "thank the universe the system still works!" Both sides can have their way! AND, of course, the blind-to-what-we-don't-have-to-see populace, such as the potentially offended by Whitelist and Blacklist, won't see this, either. So, what they don't know about backwards compatibility will be completely invisible to them - and even if they see it, they'll think, "OH GOOD, they got rid of that offensive mess!" Of course, if there are things that the development team doesn't want to perpetually support backwards compatibility for, that can easily be worked out, too, such as resolving those first, and also maybe giving a special flag for this such as, perhaps, "BackwardsNamingCompatibility" so it doesn't apply to everything.
Fedora 21 / Postfix config issue
Hello, After hardware failure, I did an OS upgrade, too, and have a new Fedora Core 21 installation with these packages installed: postfix-2.11.3-1.fc21.x86_64 spamass-milter-0.4.0-1.fc21.x86_64 spamass-milter-postfix-0.4.0-1.fc21.noarch spamassassin-3.4.0-13.fc21.x86_64 I'm a long-time postfix user, but new to SpamAssassin. Postfix is up and running fine. However, there's something missing in the directions about how postfix and SpamAssassin get integrated together, and that's my primary concern right now. ...I can get SA running, but nothing I've tried so far in smtpd_milters has worked (I presume that's how I'm supposed to get Postfix to talk to SpamAssassin?) The official fedora-specific directions, I take it, are to be found in /usr/share/doc/spamassassin. I've been through that and haven't found anything on smtpd_milters, though it's certainly piqued my interest in several related topics. ...The README found there cites a document called INSTALL but that document isn't in my distribution. So, I went to what I take to be the official web site documentation and it points me to two pages that (for at least the last two days) don't load (namely two links to http://www.onetforum.com). I did a lot of looking around and found /etc/mail/spamassassin which has a file, local.cf, which includes a URL to a page entitled, How to install and integrate SpamAssassin with Postfix on a CentOS 6 VPS but it was unhelpful - or, rather, I learned a lot but it didn't help solve the issue, again, wrong versions. Looking elsewhere for help, all the (great many) wonderful directions I find on the web appear to be for other distributions OR earlier versions of Fedora which don't work the way this version does. So, I'm coming here for help - hope this is the right place. I was confused at the distinction between spamass-milter and spamassassin (see installed version notes above), and figured these are cooperating packages, all part of SpamAssassin. ...So, I looked into the /usr/share/doc/spamass-milter-postfix directory and found a note that I should set Postfix's smtpd_milters value to: unix:/run/spamass-milter/postfix/sock However, first of all, while the directory exists, that file (socket) does not. I considered making it myself, but there is no mksock on this distribution and I'm a little hesitant to use mknod which I vaguely recall might be able to do this - could well be wrong. The non-existence of the socket has me puzzled. In any event, bravely continuing on, I just tried to start things up anyway, and postfix starts, of course, but gives this error: postfix/smtpd[18151]: warning: connect to Milter service unix:/run/spamass-milter/postfix/sock: No such file or directory Which, of course, I already knew didn't exist - so, I also see that none of the extant, running software decided to create it. I DO NOTE, however, that there's something called spamd ... I have NOT touched it per se; instead, I've started spamassassin.service (with systemctl) and when I check status, it says: spamd[27493]: spamd: server started on IO::Socket::IP [127.0.0.1]:783, IO::Socket::IP [::1]:783 (running version 3.4.0) ...This gives the STRONG suggestion I should be putting that socket number into postfix's smtpd_milters parameter... OK, that's where I'm at; help humbly requested. Regards, Richard
Re: Fedora 21 / Postfix config issue
On Thu, 16 Apr 2015, Marieke Janssen wrote: On 2015-04-16 19:08, Richard Troy wrote: postfix/smtpd[18151]: warning: connect to Milter service unix:/run/spamass-milter/postfix/sock: No such file or directory Postfix probably tries to read /var/spool/postfix/run/spamass-milter/postfix/sock as I do believe the path is relative to the spool directory. OK, I'll check that, thanks. I don't know exactly about ownership and file modes as I don't use spamass-milter myself, but I would suggest a simplefied version of the path you choose, unix:/spamass-milter/postfix.sock OK, again thanks, will try. You have to create the directory yourself, including the right ownership/rights. Um... How do I creat the sock part? mksock doesn't exist on this distribution. Is it a bit like an NFS mount directory? Create an entry and the software takes it over somehow? (both postfix as spamass-milter needs access, how exactly is left to the exercise of the reader:-)) Don't forget to update the milter config accordingly to the full path, /var/spool/postfix/spamass-milter/postfix.sock I know where this is on the Postfix side; to which of the many config directories - and which parameter - shall I be updating, please? Also please note my upate to this query (I replied to my own email with additional data) wherein I had noticed that spamd was reporting its socket, so I tried pluging that into an attempt at smtpd_milter, and got SOME dialogue between the two systems, but with copious errors - and I reported those errors. . . So, there's more to this than meets the eye. It's almost as if one package is looking for a socket and the other is looking for, maybe, a network port? ...See my other mail for more data. I DO NOTE, however, that there's something called spamd ... I have NOT touched it per se; instead, I've started spamassassin.service (with systemctl) and when I check status, it says: spamd is the daemonized version of spamassassin, spamc is the client, the milter uses spamc to connect to spamd. I see - I got PART of that from working with it a little - thanks for the clarification. Good luck, /MJ Thanks MJ! Looks like I need some! Richard
