Re: RCVD_IN_DNSWL_HI false positives
On May 13, 2021, at 12:14 AM, Michael B Allen wrote: > It is not completely trivial setup a caching name server. I literally > have two accounts so it's at least a serious nuisance. It's pretty simple to install unbound and set it up on most systems. > Sending false positives that allows SPAM though is a bad way to enforce > policy. It sounds like they've tried other options but didn't get a response from abusive users so this is the 'last resort' option. -- Daniel J. Luke
Re: txrep duplicated key with postgresql
I uploaded a patch for postgresql on https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7218 a while ago - but I haven't had time to clean it up into something that should be included into a release. It might serve as inspiration for someone else before I end up having time to get to it, though. > On Dec 9, 2019, at 4:00 PM, Martin Gregorie wrote: > > On Mon, 2019-12-09 at 11:41 -0800, John Hardin wrote: >> This sounds more like the "does that tuple already exist?" logic is >> failing, causing it to think it needs to create a new entry, which >> the unique key is (correctly) preventing. >> >> You don't lightly bypass unique keys. They are there for a reason. >> > Fair enough. Since this is the first reference I remember seeing to > using PostgreSQL with TxRef I assumed that Benny's cry for help was due > to a difference in the way it handled duplicate keys compared with the > database that normally supports it. > > Martin > -- Daniel J. Luke
Re: DNS and RBL problems
On Sep 14, 2018, at 3:26 PM, Kevin A. McGrail wrote: > On 9/14/2018 3:22 PM, Alex wrote: >> I wish it were that easy. /etc/resolv.conf is set up to use 127.0.0.1, >> which is bind configured as a my local caching resolver. > Sinister issues like this are hard. I'll try and escalate our plans for > rsync access. Alex - have you looked at bad checksum counters on the host? (netstat -s) - I've seen strange issues before with broken network hardware (or bugs in switch/router code) caused changes to packets as they passed through the 'bad' device. The first hints were those counters increasing at the same time as the mysterious issue happening. -- Daniel J. Luke
Re: Make test fails on macOS High Sierra - help needed
On Sep 8, 2018, at 8:48 PM, Sidney Markowitz wrote: > Macports install of db48 should work for that, but I haven't tried it. I use this (with a perl I built myself outside of macports) and I can confirm it works. -- Daniel J. Luke
Re: getting help with SA sysadmin
On Sep 15, 2017, at 12:24 PM, David Jones <djo...@ena.com> wrote: > You kinda have to work backwards through the scripts to find what is > generating the scores-set0 file and turning it into 72_scores.cf. I am > grep'ing through the work dir on the SA server now but it contains a lot of > files. I need to find the large dirs and exclude them. you may have already done this, but if you modify the scripts to not overwrite (or save a copy) of the intermediate files (which may clue into exactly where the problem is being introduced). ie. runGA lines 57-59, 124-132 (for 50_scores.cf) another 'easy' test I would try would be to set numcpus in runGA to 1 just in case the problem is that somewhere there are multiple writers overwriting parts of the same file -- Daniel J. Luke
Re: getting help with SA sysadmin
_IN_MSPIKE_BL 0.001 0.010 0.001 0.010 -score RCVD_IN_MSPIKE_H2 0.001 -2.800 0.001 -2.800 -score RCVD_IN_MSPIKE_H3 0.001 -0.010 0.001 -0.010 -score RCVD_IN_MSPIKE_H4 0.001 -0.010 0.001 -0.010 -score RCVD_IN_MSPIKE_H5 0.001 -1.000 0.001 -1.000 -score RCVD_IN_MSPIKE_L2 0.001 0.001 0.001 0.001 -score RCVD_IN_MSPIKE_L3 0.001 0.001 0.001 0.001 -score RCVD_IN_MSPIKE_L4 0.001 0.001 0.001 0.001 -score RCVD_IN_MSPIKE_L5 0.001 0.001 0.001 0.001 -score RCVD_IN_MSPIKE_WL 0.001 -0.010 0.001 -0.010 -score RCVD_IN_MSPIKE_ZBI0.001 0.001 0.001 0.001 -score RP_MATCHES_RCVD -1.050 -0.001 -1.050 -0.001 -score SHARE_50_50 2.121 1.818 2.121 1.818 -score SPOOFED_FREEM_REPTO 2.498 1.368 2.498 1.368 -score SPOOFED_FREEM_REPTO_CHN 1.000 1.000 1.000 1.000 -score STATIC_XPRIO_OLE 1.997 0.001 1.997 0.001 -score STOCK_LOW_CONTRAST2.030 2.347 2.030 2.347 -score STOCK_TIP 1.000 1.000 1.000 1.000 -score STYLE_GIBBERISH 2.800 3.093 2.800 3.093 -score SURBL_BLOCKED 0.001 0.001 0.001 0.001 -score SYSADMIN 1.000 1.000 1.000 1.000 -score THIS_AD 0.596 2.200 0.596 2.200 -score TO_EQ_FM_DIRECT_MX2.497 0.622 2.497 0.622 -score TO_EQ_FM_DOM_SPF_FAIL 0.001 0.001 0.001 0.001 -score TO_EQ_FM_SPF_FAIL 0.001 0.001 0.001 0.001 -score TO_IN_SUBJ0.099 0.099 0.099 0.099 -score TO_NO_BRKTS_FROM_MSSP 0.001 0.001 0.001 0.001 -score TO_NO_BRKTS_HTML_IMG 0.001 2.000 0.001 2.000 -score TO_NO_BRKTS_HTML_ONLY 1.997 0.001 1.997 0.001 -score TO_NO_BRKTS_MSFT 2.497 0.001 2.497 0.001 -score TO_NO_BRKTS_NORDNS_HTML 0.398 0.001 0.398 0.001 -score TO_NO_BRKTS_PCNT 2.497 0.001 2.497 0.001 -score TVD_SPACE_ENCODED 2.497 0.001 2.497 0.001 -score TVD_SPACE_ENC_FM_MIME 1.997 0.001 1.997 0.001 -score TVD_SPACE_RATIO_MINFP 2.497 0.001 2.497 0.001 -score TW_GIBBERISH_MANY 1.000 1.000 1.000 1.000 -score UC_GIBBERISH_OBFU 1.000 1.000 1.000 1.000 -score URI_DATA 1.000 1.000 1.000 1.000 -score URI_GOOGLE_PROXY 0.710 1.378 0.710 1.378 -score URI_ONLY_MSGID_MALF 0.001 1.191 0.001 1.191 -score URI_OPTOUT_3LD1.000 1.000 1.000 1.000 -score URI_PHISH 3.995 3.999 3.995 3.999 -score URI_TRY_3LD 0.195 0.001 0.195 0.001 -score URI_TRY_USME 0.001 0.001 0.001 0.001 -score URI_WPADMIN 3.396 3.014 3.396 3.014 -score URI_WP_DIRINDEX 1.000 1.000 1.000 1.000 -score URI_WP_HACKED 2.996 3.000 2.996 3.000 -score URI_WP_HACKED_2 1.187 1.764 1.187 1.764 -score XPRIO 2.248 2.249 2.248 2.249 -score XPRIO_SHORT_SUBJ 1.000 1.000 1.000 1.000 +score ADVANCE_FEE_3_NEW_FRM_MNY 0.001 2.296 0.001 2.296 +score ADVANCE_FEE_4_NEW_FRM_MNY 2.799 2.141 2.799 2.141 +score ADVANCE_FEE_4_NEW_MONEY3.200 2.508 3.200 2.508 +score ADVANCE_FEE_5_NEW_FRM_MNY 3.199 3.099 3.199 3.099 +score ADVANCE_FEE_5_NEW_MONEY2.976 0.558 2.976 0.558 +score AXB_X_FF_SEZ_S 3.600 3.399 3.600 3.399 +score BODY_SINGLE_URI0.001 1.607 0.001 1.607 +score BODY_SINGLE_WORD 2.602 0.001 2.602 0.001 +score COMPENSATION 0.001 0.000 0.001 0.000 +score DEAR_BENEFICIARY 0.483 1.470 0.483 1.470 +score DEAR_EMAIL 3.499 2.715 3.499 2.715 +score DX_TEXT_01 2.699 2.599 2.699 2.599 +score FROM_MISSP_DYNIP 1.536 2.399 1.536 2.399 +score FROM_MISSP_EH_MATCH1.685 1.263 1.685 1.263 +score HK_NAME_MR_MRS 4.085 2.994 4.085 2.994 +score HK_SCAM_N3 2.799 2.699 2.799 2.699 +score HTML_FONT_TINY 2.194 2.648 2.194 2.648 +score KHOP_DYNAMIC 3.030 1.997 3.030 1.997 +score LIST_PARTIAL_SHORT_MSG 2.499 2.276 2.499 2.276 +score MILLION_USD3.157 2.189 3.157 2.189 > You kinda have to work backwards through the scripts to find what is > generating the scores-set0 file and turning it into 72_scores.cf. I am > grep'ing through the work dir on the SA server now but it contains a lot of > files. I need to find the large dirs and exclude them. -- Daniel J. Luke
Re: getting help with SA sysadmin
On Sep 15, 2017, at 9:46 AM, David Jones <djo...@ena.com> wrote: > 3. I have narrowed down the problem to the general area of a perl Makefile > which builds a custom garescorer.c file which does some statistical analysis > to determine the best score for rules in the 72_scores.cf. These > 72_scores.cf are excluded from 50_scores.cf (static scores) and are currently > incomplete making these rules default to 1.0. Most of theses missing rules > should be much higher than 1.0 causing SA to allow spam through on most > installations that don't have an optimized MTA in front of SA. > > https://wiki.apache.org/spamassassin/InfraNotes2017#mkupdates > > ~/svn/trunk/build/mkupdates/mkupdate-with-scores > > masses -> perl Makefile.PL && make (complete build of SA and test) > - perl hit-frequencies > - garescorer - compiles and runs it, requires build/pga ><--- THE PROBLEM IS NEAR HERE where are hit-frequencies and garescorer? > I think the problem is somewhere behind line 127 and 128 which does many > things/steps: line 127 and 128 of mkupdate-withscores? (that just looks like it's building a version of SpamAssassin from trunk svn? maybe you are referring to something else?) -- Daniel J. Luke
Re: DNS again
On Jun 3, 2016, at 12:51 PM, Daniel J. Luke <dl...@geeklair.net> wrote: >> if the first hop in dns is 127.0.0.1 it works > > that's not how +trace works oh, nevermind - you are right. It will query for the root servers from your configured resolvers. -- Daniel J. Luke
Re: DNS again
On Jun 3, 2016, at 12:30 PM, Benny Pedersen <m...@junc.eu> wrote: > dig +trace ipv4.google.com > > if the first hop in dns is 127.0.0.1 it works that's not how +trace works from the manpage: When tracing is enabled, dig makes iterative queries to resolve the name being looked up. It will follow referrals from the root servers, showing the answer from each server that was used to resolve the lookup. If @server is also specified, it affects only the initial query for the root zone name servers. > make sure /etc/resolv.conf only have one single line with nameserver > 127.0.0.1 nothing more nothing less good advise. > drop unbound if it cant make it right, replace it with bind9 either works fine if configured correctly (and not so well if configured incorrectly). -- Daniel J. Luke
Re: understanding HELO_DYNAMIC_IPADDR
On May 13, 2016, at 4:24 PM, David Jones <djo...@ena.com> wrote: > This is a very simple concept and yet most mail admins don't know it or > follow it. indeed. I haven't measured in a while, but the equivalent of postfix's 'reject_unknown_client_hostname' was the single most-effective anti-spam measure I ever took (I had to stop using it to outright reject mail because of too many false positives, though). -- Daniel J. Luke
Re: understanding HELO_DYNAMIC_IPADDR
On May 13, 2016, at 2:26 PM, Kim Roar Foldøy Hauge <k...@samfunnet.no> wrote: > This is NOT a practical solution. You can't expect administrators to know > about this problem, some styles of hostnames not playing well with SA. Note that this isn't just a 'spamassassin' issue. You will likely experience delivery problems to many hosts as long as your dns or rdns 'looks like' a dynamic system. While you are at it, make sure your forward and reverse dns match. -- Daniel J. Luke
Re: spamd running much slower than spamassassin?
On Mar 29, 2016, at 10:41 AM, Daniel J. Luke <dl...@geeklair.net> wrote: > On Mar 28, 2016, at 8:57 PM, Bill Cole > <sausers-20150...@billmail.scconsult.com> wrote: >> On 28 Mar 2016, at 14:42, Daniel J. Luke wrote: >>> On Mar 24, 2016, at 12:10 PM, Daniel J. Luke <dl...@geeklair.net> wrote: >>>> /usr/bin/time spamassassin < spam.msg >>>> 7.92 real 1.85 user 0.13 sys >>>> >>>> /usr/bin/time spamc -U /var/run/spamd.sock < spam.msg >>>> 126.44 real 0.00 user 0.00 sys >>> >>> well, it looks like it's DNS related, somehow. >> >> The 2 minute pause had me thinking that, but nothing jumped out as a >> specific explanation and nothing yet does... >> >>> I'm still confused as to why 'spamassassin' doesn't have a problem but >>> 'spamd' does. I'm running SA 3.4.1 with perl5.22.1. I've tried both >>> downgrading Net::DNS to 0.83 and upgrading it to 1.05_2 >>> >>> Any thoughts would be appreciated. >> >> You haven't mentioned your platform, that I've seen, but it may be relevant, >> e.g. historically FreeBSD jails can't do real loopback (not sure on 10.2...) >> EL6/7 derivatives have SELinux on by default, etc... >> >> So: more clues please? > > Sorry, this is a Mac OS X 10.11.4 system, perl5.22.1 is self-built > (perlbrew). I'm not sure exactly when this started, I noticed it after I > upgraded to 10.11.4 from 10.11.3, but it may have been happening before. What > else would be helpful to know? OK, I figured this out (using fs_usage -f network ), I traced this down to spamd waiting on mDNSResponder. Turning up mDNSResponder logging gave me the answer that it was 'unhappy' with ::1 as a resolver address for some reason. Setting this up so only '127.0.0.1' is used instead makes spamd work like normal again. I /think/ this is regression in 10.11.4, but as I said before, I'm not entirely sure (I only noticed things were slow after the upgrade, they could have been slow for a little while before). -- Daniel J. Luke
Re: spamd running much slower than spamassassin?
On Mar 28, 2016, at 8:57 PM, Bill Cole <sausers-20150...@billmail.scconsult.com> wrote: > On 28 Mar 2016, at 14:42, Daniel J. Luke wrote: >> On Mar 24, 2016, at 12:10 PM, Daniel J. Luke <dl...@geeklair.net> wrote: >>> /usr/bin/time spamassassin < spam.msg >>> 7.92 real 1.85 user 0.13 sys >>> >>> /usr/bin/time spamc -U /var/run/spamd.sock < spam.msg >>>126.44 real 0.00 user 0.00 sys >> >> well, it looks like it's DNS related, somehow. > > The 2 minute pause had me thinking that, but nothing jumped out as a specific > explanation and nothing yet does... > >> I'm still confused as to why 'spamassassin' doesn't have a problem but >> 'spamd' does. I'm running SA 3.4.1 with perl5.22.1. I've tried both >> downgrading Net::DNS to 0.83 and upgrading it to 1.05_2 >> >> Any thoughts would be appreciated. > > You haven't mentioned your platform, that I've seen, but it may be relevant, > e.g. historically FreeBSD jails can't do real loopback (not sure on 10.2...) > EL6/7 derivatives have SELinux on by default, etc... > > So: more clues please? Sorry, this is a Mac OS X 10.11.4 system, perl5.22.1 is self-built (perlbrew). I'm not sure exactly when this started, I noticed it after I upgraded to 10.11.4 from 10.11.3, but it may have been happening before. What else would be helpful to know? -- Daniel J. Luke
Re: spamd running much slower than spamassassin?
On Mar 24, 2016, at 12:10 PM, Daniel J. Luke <dl...@geeklair.net> wrote: > /usr/bin/time spamassassin < spam.msg >7.92 real 1.85 user 0.13 sys > > /usr/bin/time spamc -U /var/run/spamd.sock < spam.msg > 126.44 real 0.00 user 0.00 sys well, it looks like it's DNS related, somehow. The first (and longest) pause is between these two log messages (spamd -D): Mar 28 13:53:47 lintilla spamd[1532]: dns: LocalAddr: [0.0.0.0]:48704, name server: [127.0.0.1]:53, module IO::Socket::IP Mar 28 13:54:48 lintilla spamd[1532]: dns: resolver socket rx buffer size is 196724 bytes, local port 48704 The second pause is during the adsp lookup. On the box, I'm running unbound (listening on 127.0.0.1 and ::1). It's the only nameserver available in /etc/resolv.conf and I've tried both with and without setting dns_server in my local.cf I'm still confused as to why 'spamassassin' doesn't have a problem but 'spamd' does. I'm running SA 3.4.1 with perl5.22.1. I've tried both downgrading Net::DNS to 0.83 and upgrading it to 1.05_2 Any thoughts would be appreciated. -- Daniel J. Luke
spamd running much slower than spamassassin?
I recently noticed that spamd was running /much/ slower than usual. With a test message: /usr/bin/time spamassassin < spam.msg 7.92 real 1.85 user 0.13 sys /usr/bin/time spamc -U /var/run/spamd.sock < spam.msg 126.44 real 0.00 user 0.00 sys spamassassin -D timing < spam.msg Mar 24 12:02:54.767 [79829] dbg: timing: total 7030 ms - init: 1003 (14.3%), b_tie_ro: 10 (0.1%), parse: 2.3 (0.0%), extract_message_metadata: 59 (0.8%), get_uri_detail_list: 7 (0.1%), tests_pri_-1000: 39 (0.5%), compile_gen: 111 (1.6%), compile_eval: 25 (0.4%), tests_pri_-950: 2.2 (0.0%), tests_pri_-900: 2.8 (0.0%), tests_pri_-400: 80 (1.1%), check_bayes: 76 (1.1%), b_tokenize: 13 (0.2%), b_tok_get_all: 12 (0.2%), b_comp_prob: 2.3 (0.0%), b_tok_touch_all: 47 (0.7%), b_finish: 1.00 (0.0%), tests_pri_0: 5770 (82.1%), check_spf: 49 (0.7%), poll_dns_idle: 0.03 (0.0%), dkim_load_modules: 20 (0.3%), check_dkim_signature: 1.30 (0.0%), check_dkim_adsp: 3.5 (0.1%), check_dcc: 150 (2.1%), check_razor2: 5012 (71.3%), check_pyzor: 271 (3.9%), tests_pri_500: 50 (0.7%), tests_pri_1000: 14 (0.2%), total_txrep: 11 (0.2%), check_txrep_msg_id: 2.2 (0.0%), update_txrep_msg_id: 0.02 (0.0%) spamd running with -D timing - spamc -U /var/run/spamd.sock < spam.msg Mar 24 12:06:17 lintilla spamd[79885]: timing: total 130906 ms - read_scoreonly_config: 444 (0.3%), signal_user_changed: 12 (0.0%), b_tie_ro: 9 (0.0%), parse: 3.0 (0.0%), extract_message_metadata: 60281 (46.0%), get_uri_detail_list: 5.0 (0.0%), tests_pri_-1000: 38 (0.0%), compile_gen: 28 (0.0%), compile_eval: 27 (0.0%), tests_pri_-950: 1.32 (0.0%), tests_pri_-900: 1.73 (0.0%), tests_pri_-400: 90 (0.1%), check_bayes: 79 (0.1%), b_tokenize: 13 (0.0%), b_tok_get_all: 12 (0.0%), b_comp_prob: 2.6 (0.0%), b_tok_touch_all: 49 (0.0%), b_finish: 1.04 (0.0%), tests_pri_0: 69951 (53.4%), check_dkim_signature: 1.71 (0.0%), check_dkim_adsp: 60184 (46.0%), check_spf: 240 (0.2%), poll_dns_idle: 215 (0.2%), check_dcc: 4325 (3.3%), check_razor2: 5004 (3.8%), check_pyzor: 0.25 (0.0%), tests_pri_500: 6 (0.0%), tests_pri_1000: 12 (0.0%), total_txrep: 10 (0.0%), check_txrep_msg_id: 2.3 (0.0%), update_txrep_msg_id: 0.02 (0.0%), rewrite_mail: 1.12 (0.0%), copy_config: 41 (0.0%) So, I guess extract_message_metadata and check_dkim_adsp are causing the problem when running under spamd - but I'm not sure why that would be / where I should look next to troubleshoot and fix. -- Daniel J. Luke
