Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? you can disable SPF or clear all scores The question was *as supplied by sa-update* Q: Any other ideas on how to learn what rules are actually being used? huh? Please read the rest of this thread. Q: Any suggestions as to why SPF rules would not fire on a Gmail message where Gmail uses SPF, my SPF plugin and rule initiation seem to be in place, and a Return-Path header with the envelope from address exists? (please see my previous messages on this thread) I haven't found the headers in apache archive, maybe I didn't search carefully enough, I recommend gmane.org but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again..
Re: SPF and DKIM tests by default?
On 2/15/2012 7:08 PM, email builder wrote: OK, but: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? Not that I can think of. Q: Any other ideas on how to learn what rules are actually being used? What I would likely do is save the gmail message to an mbox format file. Then I would run spamassassin -D -t /tmp/mboxfile 21 | grep -i SPF and see what I find. Well, that was actually the other more general question that you kindly already offered your help for - how to determine all rules currently in use at execution time. Short of other opinions, we'll wait to see how the bugzilla item I created progresses. But your advice here is in fact quite useful and may do a fine job at pointing to the issue. Keep in mind, all rules are as given by sa-update. I copied in all the output below but here are what I see as key points by line number: Line 8: Someone earlier pointed out that SA uses this Received-SPF header, but then I think it was you that pointed out that this shouldn't be necessary, and I added that it would seem odd to me if SA didn't also look for the quasi-standard Return-Path header which for some mailers such as Postfix will include the envelope from address. The lack of this header doesn't seem to stop SPF execution though. When I copy my Return-Path header into a Received-SPF header, line 8 becomes two lines: Feb 16 14:19:59.263 [12846] dbg: spf: found a Received-SPF header added by an internal host: Received-SPF: emailbuilde...@gmail.com Feb 16 14:19:59.263 [12846] dbg: spf: could not parse result from existing Received-SPF header I tried this with a couple different formats of email and/or domain name. Not sure what's going on here. The rest of the lines are the same in both cases. Line 12: Any comments why the SPF lookup returns nothing? How can I do this same lookup by hand? Could this be a DNS problem? Line 13: Weren't the last few lines DNS checks already? Line 14: I don't know why this happens. It is true that Postfix relays mail to amavis, then it goes back to Postfix then it is handed off to maildrop for delivery - SA is called from maildrop. So there is some local relaying here, but why does this stop SA from checking the hop from the outside to my Postfix? Is this where having a non-default trusted_networks setting would help? Thanks again for the great help and patience. 1) Feb 16 14:13:17.361 [12806] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC 2) Feb 16 14:13:17.774 [12806] dbg: config: fixed relative path: /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf 3) Feb 16 14:13:17.774 [12806] dbg: config: using /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf for included file 4) Feb 16 14:13:17.774 [12806] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org/25_spf.cf 5) Feb 16 14:13:17.894 [12806] dbg: config: fixed relative path: /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf 6) Feb 16 14:13:17.895 [12806] dbg: config: using /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf for included file 7) Feb 16 14:13:17.895 [12806] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org/60_whitelist_spf.cf 8) Feb 16 14:13:19.595 [12806] dbg: spf: checking to see if the message has a Received-SPF header that we can use 9) Feb 16 14:13:19.646 [12806] dbg: spf: using Mail::SPF for SPF checks 10) Feb 16 14:13:19.646 [12806] dbg: spf: checking HELO (helo=mail-iy0-f181.google.com, ip=209.85.210.181) 11) Feb 16 14:13:19.648 [12806] dbg: dns: providing a callback for id: 13553/mail-iy0-f181.google.com/SPF/IN 12) Feb 16 14:13:19.984 [12806] dbg: spf: query for /209.85.210.181/mail-iy0-f181.google.com: result: none, comment: , text: No applicable sender policy available 13) Feb 16 14:13:19.988 [12806] dbg: spf: already checked for Received-SPF headers, proceeding with DNS based checks 14) Feb 16 14:13:19.988 [12806] dbg: spf: relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping 15) Feb 16 14:13:19.995 [12806] dbg: spf: def_spf_whitelist_from: already checked spf and didn't get pass, skipping whitelist check 16) Feb 16 14:13:19.997 [12806] dbg: spf: whitelist_from_spf: already checked spf and didn't get pass, skipping whitelist check 17) Feb 16 14:13:25.566 [12806] dbg: timing: total 8235 ms - init: 1912 (23.2%), parse: 1.82 (0.0%), extract_message_metadata: 74 (0.9%), poll_dns_idle: 381 (4.6%), get_uri_detail_list: 1.24 (0.0%), tests_pri_-1000: 27 (0.3%), compile_gen: 171 (2.1%), compile_eval: 39 (0.5%), tests_pri_-950: 9 (0.1%), tests_pri_-900: 9 (0.1%), tests_pri_-400: 8 (0.1%), tests_pri_0: 5996 (72.8%), dkim_load_modules: 33 (0.4%), check_dkim_signature: 11 (0.1%), check_spf: 389 (4.7%), check_dcc: 190 (2.3%), check_razor2: 5003 (60.8%), check_pyzor: 0.54 (0.0%), tests_pri_500: 100 (1.2%), tests_pri_1000: 15
Re: SPF and DKIM tests by default?
On 2/16/2012 4:54 PM, email builder wrote: but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again.. Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. Makes sense, especially if my hunch about the relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping part of the debug output I just sent to this list is on track. Is there a way to set trusted_networks on the command line of the spamassassin command just for testing?
Re: SPF and DKIM tests by default?
On 2/16/2012 4:54 PM, email builder wrote: but it's misconfigured trusted_networks and internal_networks what causes SPF to misfire... Thank you sincerely for your help. I can only imagine that SPF wouldn't fire if I accidentally specified Google in one of those settings or had an error in one of them. In this case, those are at their defaults of empty, so I'm hoping there are other suggestions. Thanks again.. Letting trusted_networks empty is not generally a good idea. In particular, if your SA server is using a private IP, it will default to trusting too much. Specify your local networks in trusted_networks and see if that helps your problem. Leaving trusted_networks empty does not mean trust nothing; it means let SA figure out what to trust. Makes sense, especially if my hunch about the relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping part of the debug output I just sent to this list is on track. Is there a way to set trusted_networks on the command line of the spamassassin command just for testing? This didn't work: spamassassin -D --cf='trusted_networks 127.0.0.1' -t example_email_no_spf 21 | grep -i SPF All my local handoffs are to localhost [127.0.0.1] so I wouldn't know what else to use (it's an all-in-one single server simple system)
Re: SPF and DKIM tests by default?
Q: Will some rules not fire if some condition exists based on other rules? A: Correct. There are plenty of rules that build on other rules. We call these meta rules. OK, but: Q: Are there any default rules as supplied by sa-update that would prevent SPF rules from firing? Q: Any other ideas on how to learn what rules are actually being used? Q: Any suggestions as to why SPF rules would not fire on a Gmail message where Gmail uses SPF, my SPF plugin and rule initiation seem to be in place, and a Return-Path header with the envelope from address exists? (please see my previous messages on this thread)
Re: SPF and DKIM tests by default?
On 2/10/2012 9:35 PM, email builder wrote:
Hi Kevin, thank you for your reply! But I think you should send it to the
list :)
Hi Thanks for bringing it back to the list. Sometimes I'm just trying
to bang out answers too quickly!
You should look in /var/lib/spamassassin. Because rules are no longer
paired to releases but released nearly continuously, there is no wiki
list of all the rules.
Gotcha - but is it certain that all rules in
/var/lib/spamassassin/3.003001/updates_spamassassin_org are being used?
No. There are many plugins, configuration options dependencies that could
affect what rules are used.
Oh OK, that's a little surprising, but I understand it can get complex, so
that's fine.
Oh, sorry for the noob question, but how do I know if I have Mail::DKIM
installed?
For example:
perl -e 'if (require Mail::DKIM) { print Mail::DKIM Version is:
$Mail::DKIM::VERSION\n; exit 0;} else {exit 1;}' || echo
'Mail::DKIM Not Present!'
Mail::DKIM Version is: 0.37
Great! Thank you
Re: SPF and DKIM tests by default?
On 2/10/2012 9:20 PM, email builder wrote:
Wonder if I can delete the older one
Sure. Worst case just run sa-update again if you delete the wrong one.
OK, thank you. I'll report back if it causes any problems but I can't imagine
it would.
Hm, well is there a file or somewhere to look and see what rules are
active?
Do you mean something like: With my configuration, what rules might possibly
be
triggered?
yes
That's an interesting question. Perhaps we could use a spamassassin
parameter to run, parse config and dump all possible rules that would run
(with
scores) based on all plugins, etc. that are believed to be configured. If
that
is what you want, please open a bug at
https://issues.apache.org/SpamAssassin/
assuming no one knows a way this can occur now.
OK it's a feature request then huh? I added it:
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6757
I believe for SPF you *should* be doing the detecting at your MTA
(mail server software) and inserting a header for spamassassin to use:
Received-SPF. (Because SPF is supposed to use the envelope
from,
which is not necessarily included in a header.)
I see. That makes sense. Is there a wiki page suggesting solutions for
this? Anyone know of tips for doing this in postfix? Or during amavis
processing?
Interesting thought though while the envelope sender is not in a header per
se,
it is in the From line for mbox format email, I believe. If you are using
procmail for delivery, for example, there shouldn't be an issue.
Actually, you're right - it seems as long as the envelope info is available you
would not need to add a new header, no? That depends if the SA SPF rules know
how to check the envelope or if they only look for a Received-SPF header.
Anyone know the details in that regard?
I use maildrop for delivery out of postfix (and SA runs from maildrop). Postfix
passes what I think is the envelope sender to maildrop by -f ${sender} (I'll
double check but I think that's accurate).
I'm uncertain if the envelope info gets to SA, though, as my maildrop call to
SA is: xfilter /usr/bin/spamc -u $LOGNAME
I'd rather not add a header if not necessary. Second choice is to do it using
amavis, as adding a policy server just for this to be pretty extreme.
Me too. I sent emails to myself from Yahoo and Gmail and got these in my
X-Spam-Status:
Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU
Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID
(that last one is interesting - not sure how the message gets altered to
break the signature, especially if Gmail works fine (running SA from
Maildrop))
Chasing down DKIM errors can be interesting to say the least. I found a bug
in
Sendmail, for example, where it canonicalized the email address in the To:
Header which was case-sensitive on the signing so DKIM validation failed.
Have you looked at the received headers to confirm it is in fact a valid
Yahoo!
email?
I believe SPF tests are also enabled by default, but won't do quite
the
right thing unless you're inserting the Received-SPF header at your
MTA.
Well I guess so because I see no SPF hits and I think at least Gmail uses
SPF. I'd appreciate any tips on getting those headers inserted.
Gmail does publish SPF.
Check your *.pre files and see if you have loadplugin
Mail::SpamAssassin::Plugin::SPF
Also make sure you have Mail::SPF. This command can help determine that:
perl -e 'if (require Mail::SPF) { print Mail::SPF Version is:
$Mail::SPF::VERSION\n; exit 0;} else {exit 1;}' || echo
'Mail::SPF Not Present!'
Mail::SPF Version is: v2.005
Regards,
KAM
Re: SPF and DKIM tests by default?
On 02/10, email builder wrote: I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? I use postfix-policyd-spf-perl. Which appears to currently be officially hosted at: https://launchpad.net/postfix-policyd-spf-perl/ Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that Received-SPF header or can use the envelope sender if it's present?
Re: SPF and DKIM tests by default?
On 02/10, email builder wrote: I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? I use postfix-policyd-spf-perl. Which appears to currently be officially hosted at: https://launchpad.net/postfix-policyd-spf-perl/ Thanks for that, although see my last post - do you know if the SPF tests only know how to look for that Received-SPF header or can use the envelope sender if it's present? If your MTA provides sufficient info for SA to determine the envelope sender that is enough. I agree and I've done some more research and found that Postfix adds the envelope sender as a Return-Path header (its pipe and virtual delivery agent at least do this). So I *do* have a header in my messages with the envelope sender. Either the SPF rules don't know how to look for Return-Path (which would surprise me given that it is quasi-standard and highly used) or I have some other problem. Will some rules not fire if some condition exists based on other rules? I've been using sendmail+milter+sa for years with SPF DKIM rules and never had any kind of special MTA added 'Received-SPF' header. OK, good. One thing that -is- a factor; sa depends upon specific perl modules for that functionality; DNS, SPF, DKIM modules (EG Net::DNS, Mail::DKIM, Mail::SPF ), and 'loadplugin' statements in the correct .pre files. I think I forgot to reply to the hints on checking for the SPF module earlier in this thread, but I do have it installed. And the SPF plugin is loaded from init.pre (is that OK?). Occasionally issues arise with problematic versions of those modules. For example, search this list archive for disussions about problems caused by buggy versions of the DNS module. If you execute the test: % spamassassin --lint -D 21 | grep -i -E 'spf|dkim|dns' [snip ...] If you don't see those 'plugin: loading' lines for SPF DKIM, then there's your problem. Either they're not installed on your system in a way that SA can find them, wrong verions, or not invoked by 'loadplugin' statements. Thanks that was helpful, and I did in fact find the plugin loading for the SPF plugin, so it's there, but I'm not getting hits on messages from Gmail which does use SPF. Hmmm any other suggestions anyone? Thanks for the excellent help so far!
Re: SPF and DKIM tests by default?
Thanks a lot for your reply Run: sa-update -D 21| grep DIR That will output something like: Feb 9 12:08:49.609 [20855] dbg: generic: Perl 5.010001, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin On this system, sa-update downloads rules to /var/lib/spamassassin, so I guess you're looking for the LOCAL_STATE_DIR. OK, makes sense. Mine is the same as yours. That directory will contain a directory related to your SA version, something like 3.003001, which will contain updates_spamassassin_org, which will contain the files defining all the rules. Hmm, in there I find TWO directories: 3.002005 3.003001 Strangely, both have dates of today, but the *contents* of 3.002005 are from Apr 3 2011. So I guess my system uses 3.003001 since it's files are dated currently Wonder if I can delete the older one Although that doesn't necessarily tell you which are enabled by default. Some require configuration changes. Hm, well is there a file or somewhere to look and see what rules are active? I believe for SPF you *should* be doing the detecting at your MTA (mail server software) and inserting a header for spamassassin to use: Received-SPF. (Because SPF is supposed to use the envelope from, which is not necessarily included in a header.) I see. That makes sense. Is there a wiki page suggesting solutions for this? Anyone know of tips for doing this in postfix? Or during amavis processing? From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM I don't have anything in my /etc/spamassassin/local.cf related to DKIM, and I'm getting DKIM rule hits, so I agree that DKIM is enabled by default (although I'm running trunk / v3.4.0 which is unreleased). Me too. I sent emails to myself from Yahoo and Gmail and got these in my X-Spam-Status: Gmail: DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU Yahoo: DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,T_DKIM_INVALID (that last one is interesting - not sure how the message gets altered to break the signature, especially if Gmail works fine (running SA from Maildrop)) I believe SPF tests are also enabled by default, but won't do quite the right thing unless you're inserting the Received-SPF header at your MTA. Well I guess so because I see no SPF hits and I think at least Gmail uses SPF. I'd appreciate any tips on getting those headers inserted. None of the SPF or DKIM rules are particularly highly ranked in spamassassin rule QA, so I wouldn't actually expect significant improvements in accuracy from it: http://ruleqa.spamassassin.org/?daterev=20120204 They both have some substantial flaws. I'm OK with that (have been weary about their limitations and not always 100% sure about using either of them on my domains), and it's actually the reason I'm asking about SA support for them because I would never want to use either of them to outright block mail.Just some influence on SA scoring is good.
Re: SPF and DKIM tests by default?
From: Kevin A. McGrail kmcgr...@pccc.com Hi Kevin, thank you for your reply! But I think you should send it to the list :) Is this the right place to look to know what tests the server should be running? https://spamassassin.apache.org/tests_3_0_x.html You should look in /var/lib/spamassassin. Because rules are no longer paired to releases but released nearly continuously, there is no wiki list of all the rules. Gotcha - but is it certain that all rules in /var/lib/spamassassin/3.003001/updates_spamassassin_org are being used? From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM Yes, 3.1.2 enabled DKIM by default if you have Mail::DKIM installed, I believe. Oh, sorry for the noob question, but how do I know if I have Mail::DKIM installed? Also, where can I look to verify the tests/rules currently in place on the server? (per-user rules are not implemented) In the version dir under /var/lib/spamassassin. I looked in /usr/share/spamassassin and there are a few files with spf and dkim in their names. Does that mean those tests are active? ls *spf* -rw-r--r-- 1 root root 3100 Mar 15 2010 25_spf.cf -rw-r--r-- 1 root root 3584 Mar 15 2010 60_whitelist_spf.cf ls *dkim* -rw-r--r-- 1 root root 4407 Mar 15 2010 25_dkim.cf -rw-r--r-- 1 root root 9288 Mar 15 2010 60_adsp_override_dkim.cf -rw-r--r-- 1 root root 6455 Mar 15 2010 60_whitelist_dkim.cf I believe SPF and DKIM are enabled by default but that doesn't mean you have all the supporting modules installed. Did you configure the installation yourself or did you use a package? I used yum to install a package on centOS
SPF and DKIM tests by default?
Hello, I have a server where I never customized any of the SA rules/tests (SA v.3.3.1). The server does run sa-update every day. Is this the right place to look to know what tests the server should be running? https://spamassassin.apache.org/tests_3_0_x.html From that page, it seems that SPF checks are normal but DKIM is not. Is this right? Contrary to that, this page suggests that DKIM test are enabled by default in version 3.3: https://wiki.apache.org/spamassassin/Plugin/DKIM Also, where can I look to verify the tests/rules currently in place on the server? (per-user rules are not implemented) I looked in /usr/share/spamassassin and there are a few files with spf and dkim in their names. Does that mean those tests are active? ls *spf* -rw-r--r-- 1 root root 3100 Mar 15 2010 25_spf.cf -rw-r--r-- 1 root root 3584 Mar 15 2010 60_whitelist_spf.cf ls *dkim* -rw-r--r-- 1 root root 4407 Mar 15 2010 25_dkim.cf -rw-r--r-- 1 root root 9288 Mar 15 2010 60_adsp_override_dkim.cf -rw-r--r-- 1 root root 6455 Mar 15 2010 60_whitelist_dkim.cf
Re: sa-update / perl error again
After some help on the CentOS list, I may have found the problem: perl-NetAddr-IP-4.044-1.el5.rf === I think that is the problem package I don't know if that version is required by the repoforge packages ... but base contains perl-NetAddr-IP-4.027-5.el5_6 I would see if I could replace perl-NetAddr-IP-4.044-1.el5.rf from repoforge with perl-NetAddr-IP-4.027-5.el5_6 from base. rpm -e --nodeps perl-NetAddr-IP vi /etc/yum.repos.d/rpmforge.repo -- change all enabled = 1 to enabled = 0 temporarily (seems like yum priorities is going to be a good idea) -- yum install perl-NetAddr-IP /etc/init.d/spamassassin condrestart Stopping spamd: [ OK ] Starting spamd: [ OK ] That seems to have done it. Does that make sense? Should I report this to the RepoForge people? FYI: I did in fact report it to them, and it was eventually moved to the RepoForge extras repository, which finally fixes everything. Thanks again for the help
Re: sa-update / perl error again
rpm -e --nodeps perl-IO-Socket-INET6
By the way, is there a way to grep for the errant code? My
feeble attempt didn't turn up much:
as in one of my previous emails:
'locate IO-Socket-INET6'
Sorry I missed that!
This gives only docs:
/usr/share/doc/perl-IO-Socket-INET6-2.51
/usr/share/doc/perl-IO-Socket-INET6-2.51/README
locate INET6
/usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm
/usr/share/doc/perl-IO-Socket-INET6-2.51
/usr/share/doc/perl-IO-Socket-INET6-2.51/README
/usr/share/man/man3/IO::Socket::INET6.3pm.gz
Using the find command from below (for fedora since
this is CentOS) I get
/usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm
This locates that package, but correct me if I'm wrong, I
don't think it finds the OTHER package that is creating
the duplicate subroutine definition for AF_INET6.
and/or:
(here was previous email)
or, you could just delete (manually) IO-Socket-INET6 (make a backup first!)
on freebsd (with perl 5.10.1):
/usr/local/lib/perl5/5.10.1/man/man3/IO::Socket::INET6.3.gz
/usr/local/lib/perl5/site_perl/5.10.1/IO/Socket/INET6.pm
/usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6
/usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6/.packlist
on fedora, try:
find /usr/lib/ -name 'INET6*'
(back them up) you should see them as above. but, yum won't know they are
gone.
might be in /usr/lib/perl5/{version}
and /usr/lib/perl5{version}|vendor}
ask on linux users group how to get yum to rm a dependency without the
package.
on freebsd, it would be something like 'pkg_delete -f
p5-IO-SOCKET-INET6' (the -f to force it to be removed)
and, pkgdb -F (to FIX the package database and remove the dependency link)
Re: sa-update / perl error again
used 'it' for YEARS in production, (commercial product,)
several
platforms, i386, amd64, FreeBSD versions 6.4-7.4.
ONE DAY, ONE BRAND NEW CLIENT was having real problems with their
mailq. email was backing up.
Two days to figure it out, I deleted the INET6 module (on freebsd, its a
lot easier, I suppose than on your linux thing).
Now, all the email flowed perfectly. SA was trying to do ipv6
lookups, the
kernel did NOT have ipv6 compiled in.. NONE OF OUR PRODUCTION SYSTEMS
DO, and there is no logical explanation for it.
SA does NOT need INET6, unless you have two things:
#0, INET6 compiled into your kernel
#1, INET6 dns server as the first server in /etc/resolv.conf
#2, INET6 firewall, routing, mx records, etc.
Strong argument for removing it I guess, I think with:
rpm -e --nodeps perl-IO-Socket-INET6
By the way, is there a way to grep for the errant code? My
feeble attempt didn't turn up much:
cd /usr/lib/perl5
grep -rin 'af_inet6' *
Only gave 40 lines, which I could post if it would help.
This only gave one result:
grep -rin 'sub af_inet6' *
5.8.8/i386-linux-thread-multi/bits/socket.ph:66: eval 'sub AF_INET6 () {
PF_INET6;}' unless defined(AF_INET6);
Is it pointless to try to diagnose in this manner?
After some help on the CentOS list, I may have found the problem:
perl-NetAddr-IP-4.044-1.el5.rf === I think that is the problem package
I don't know if that version is required by the repoforge packages ... but
base contains perl-NetAddr-IP-4.027-5.el5_6
I would see if I could replace perl-NetAddr-IP-4.044-1.el5.rf from repoforge
with perl-NetAddr-IP-4.027-5.el5_6 from base.
rpm -e --nodeps perl-NetAddr-IP
vi /etc/yum.repos.d/rpmforge.repo
-- change all enabled = 1 to enabled = 0 temporarily (seems like
yum priorities is going to be a good idea) --
yum install perl-NetAddr-IP
/etc/init.d/spamassassin condrestart
Stopping spamd: [ OK ]
Starting spamd: [ OK ]
That seems to have done it. Does that make sense? Should
I report this to the RepoForge people?
Re: sa-update / perl error again
while I*DO* appreciate your suggestion, since I am fairly confident to say I doubt that my config is the problem in a DNS resolver/IPv6 function redefinition, I'm not too interested in proving that point by making those changes on a production machine. Again, thanks anyway. I am the ports maintainer for the FreeBSD version of SpamAssassin. used 'it' for YEARS in production, (commercial product,) several platforms, i386, amd64, FreeBSD versions 6.4-7.4. ONE DAY, ONE BRAND NEW CLIENT was having real problems with their mailq. email was backing up. Two days to figure it out, I deleted the INET6 module (on freebsd, its a lot easier, I suppose than on your linux thing). Now, all the email flowed perfectly. SA was trying to do ipv6 lookups, the kernel did NOT have ipv6 compiled in.. NONE OF OUR PRODUCTION SYSTEMS DO, and there is no logical explanation for it. SA does NOT need INET6, unless you have two things: #0, INET6 compiled into your kernel #1, INET6 dns server as the first server in /etc/resolv.conf #2, INET6 firewall, routing, mx records, etc. Strong argument for removing it I guess, I think with: rpm -e --nodeps perl-IO-Socket-INET6 But as others noted, yum supposedly will complain from then on about the missing package. Thanks for the input!
Re: sa-update / perl error again
I am the ports maintainer for the FreeBSD version of SpamAssassin.
used 'it' for YEARS in production, (commercial product,) several
platforms, i386, amd64, FreeBSD versions 6.4-7.4.
ONE DAY, ONE BRAND NEW CLIENT was having real problems with their mailq.
email
was backing up.
Two days to figure it out, I deleted the INET6 module (on freebsd, its a
lot
easier, I suppose than on your linux thing).
Now, all the email flowed perfectly. SA was trying to do ipv6 lookups, the
kernel did NOT have ipv6 compiled in.. NONE OF OUR PRODUCTION SYSTEMS DO,
and
there is no logical explanation for it.
SA does NOT need INET6, unless you have two things:
#0, INET6 compiled into your kernel
#1, INET6 dns server as the first server in /etc/resolv.conf
#2, INET6 firewall, routing, mx records, etc.
Strong argument for removing it I guess, I think with:
rpm -e --nodeps perl-IO-Socket-INET6
By the way, is there a way to grep for the errant code? My
feeble attempt didn't turn up much:
cd /usr/lib/perl5
grep -rin 'af_inet6' *
Only gave 40 lines, which I could post if it would help.
This only gave one result:
grep -rin 'sub af_inet6' *
5.8.8/i386-linux-thread-multi/bits/socket.ph:66: eval 'sub AF_INET6 () {
PF_INET6;}' unless defined(AF_INET6);
Is it pointless to try to diagnose in this manner?
Re: sa-update / perl error again
Sure, but the point is that my spamassassin and per-Net-DNS (where the error is happening?) are up to date from the CentOS repo so shouldn't they work without an error when spamassassin restarts? It isn't the job of the SA project to worry about specific distros and repositories. This is a question for a CentOS/RedHat forum Fair enough Right now, this is most likely: A bug in a Perl module long since fixed or a local configuration error. I've sent you information asking you to test without any cf files to see if the error goes away. I don't see how a configuration problem could cause this, especially when I've been using this configuration on more than one machine without problems for years and it's all basic - change the default required score and rewrite header settings, set up database access for bayes and user scores. So while I *DO* appreciate your suggestion, since I am fairly confident to say I doubt that my config is the problem in a DNS resolver/IPv6 function redefinition, I'm not too interested in proving that point by making those changes on a production machine. Again, thanks anyway. And you'll need to work with people familiar with your Distro to update the likely culprits. Net::DNS is where I would focus. I am going to look for where to file a bug report.
Re: sa-update / perl error again
Wow, really? Then why wouldn't RedHat or CentOS have a fixed updated version in their repo? That seems egregious if what you say is indeed the case. RedHat (and CentOS, since their whole mission is to match RHEL feature-for-feature and bug-for-bug) believes that their Enterprise Linux customers value consistency over currency. They release updates to patch security holes, but their general attitude is that if Red Hat 5.0 shipped with foo_1.1.3 in 2007, then Red Hat 5.7 should also ship with foo_1.1.3 because their customers may have whole workflows built around the way foo_1.1.3 handles a specific command flag and foo_1.2.7 may have changed that. If necessary, they'll backport security patches from later versions of foo back to the current, leading to RPM names like foo_1.1.3-17.el5_7 -- but they won't add feature changes unless absolutely unavoidable. Sure, but the point is that my spamassassin and per-Net-DNS (where the error is happening?) are up to date from the CentOS repo so shouldn't they work without an error when spamassassin restarts? insisting and asking the SA list why Centos does something is not going to get you anywhere. You were told why - and if not send your complaints to RedHAt which is responsible for the sources. Centos only repackages the upstream sources. run the update I suggested and tell us what happened. Please don't misunderstand - I do very much appreciate your help. I'm hesitant to do as suggested and obtain a newer perl-Net-DNS from an external repo because of what seems to be a general opinion that the more you mix external packages the more you risk things like this continuing to happen. So I thought keep as many packages as native CentOS as I can. I'm going to try to figure out where to file a bug I guess, but I have a fear I'll get rebuffed without any help at all.
Re: sa-update / perl error again
Wow, really? Then why wouldn't RedHat or CentOS have a fixed updated version in their repo? That seems egregious if what you say is indeed the case. RedHat (and CentOS, since their whole mission is to match RHEL feature-for-feature and bug-for-bug) believes that their Enterprise Linux customers value consistency over currency. They release updates to patch security holes, but their general attitude is that if Red Hat 5.0 shipped with foo_1.1.3 in 2007, then Red Hat 5.7 should also ship with foo_1.1.3 because their customers may have whole workflows built around the way foo_1.1.3 handles a specific command flag and foo_1.2.7 may have changed that. If necessary, they'll backport security patches from later versions of foo back to the current, leading to RPM names like foo_1.1.3-17.el5_7 -- but they won't add feature changes unless absolutely unavoidable. Sure, but the point is that my spamassassin and per-Net-DNS (where the error is happening?) are up to date from the CentOS repo so shouldn't they work without an error when spamassassin restarts?
Re: sa-update / perl error again
Does spamassassin -D --lint 21 | grep -i Resolver show the same error? Yes And if you temporarily move all your config files and run the same command, does the error go away? Yikes, I'm reluctant to do this on a production machine. I have only made config changes in local.cf (pretty nominal) and in database_config.cf where I have DBs for user scores, auto whitelist and bayes. Why would my (I'd guess fairly innocuous) config settings trigger a DNS resolver error where it looks like a function has been redefined?
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. But you do have rpmforge-extras disabled... How could yum not be seeing the newer package? and the newer version is in rpmforge-extras, as denoted by the .rfx tab. That's really great how they move the package from one repo to the other. I was trying to have as few third party repos enabled as possible. That is why yum was not seeing the updated version. Makes sense. However, I'm a little confused as to why I want the perl-Net-DNS package from RepoForge? Is not the CentOS one OK? My spamassassin package (and perl-IO-Socket-INET6) is from CentOS so...?
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. How could yum not be seeing the newer package? cat /etc/yum.repos.d/rpmforge.repo Well, knowing there was a newer package out there, hopefully no matter where yum looked for it, I took a chance and removed it: rpm -e --nodeps perl-Net-DNS Then when I asked yum about it again, it found the new one from the CentOS repo, so I installed it... it also needed to install perl-Net-IP which I didn't have. So now I have perl-Net-DNS-0.59-3.el5.i386.rpm Running sa-update on the command line doesn't produce errors, so I guess that the cron won't either. Bad news - the error happened again when run from cron. It turns out it's not sa-update specifically doing this, but the restart of spamassassin itself: /etc/init.d/spamassassin condrestart Stopping spamd: [ OK ] Starting spamd: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 66 [ OK ] With my spamassassin, perl-Net-DNS and per-IO-Socket-INET6 packages all being from CentOS repo, I'm unsure why this would happen. What else can I look at? Recap on my versions: perl-IO-Socket-INET6-2.51-2.fc6 perl-Net-DNS-0.59-3.el5 spamassassin-3.3.1-2.el5
Re: sa-update / perl error again
/usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 66 [ OK ] With my spamassassin, perl-Net-DNS and per-IO-Socket-INET6 packages all being from CentOS repo, I'm unsure why this would happen. What else can I look at? Recap on my versions: perl-IO-Socket-INET6-2.51-2.fc6 perl-Net-DNS-0.59-3.el5 spamassassin-3.3.1-2.el5 Does spamassassin -D --lint 21 | grep -i Resolver show the same error? Yes
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. How could yum not be seeing the newer package? cat /etc/yum.repos.d/rpmforge.repo Well, knowing there was a newer package out there, hopefully no matter where yum looked for it, I took a chance and removed it: rpm -e --nodeps perl-Net-DNS Then when I asked yum about it again, it found the new one from the CentOS repo, so I installed it... it also needed to install perl-Net-IP which I didn't have. So now I have perl-Net-DNS-0.59-3.el5.i386.rpm Running sa-update on the command line doesn't produce errors, so I guess that the cron won't either. Bad news - the error happened again when run from cron. It turns out it's not sa-update specifically doing this, but the restart of spamassassin itself: /etc/init.d/spamassassin condrestart Stopping spamd: [ OK ] Starting spamd: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 66 [ OK ] With my spamassassin, perl-Net-DNS and per-IO-Socket-INET6 packages all being from CentOS repo, I'm unsure why this would happen. What else can I look at? Recap on my versions: perl-IO-Socket-INET6-2.51-2.fc6 perl-Net-DNS-0.59-3.el5 spamassassin-3.3.1-2.el5 Net-DNS-0.59 is ancient and buggy get the latest for your CentOS version from http://pkgs.repoforge.org/perl-Net-DNS/ Wow, really? Then why wouldn't RedHat or CentOS have a fixed updated version in their repo? That seems egregious if what you say is indeed the case. Why wouldn't the rest of the world be seeing the same errors I am since I'm running the most up to date version of that and spamassassin both from the CentOS repo??? (and thus someone fix it...) run a rpm -hUv so yum won't fiddle around with it during next CentOS update that should hopefully solve your problem. I'm going to hold out on this a little longer per my questions above, but I'm definitely thinking this is what my next step will be barring any better suggestions, so thank you.
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf I don't use IPv6 that I know of (the errant package is installed whether I like it or not as a dependency of spamassassin). 32 bit. Also, have you opened a bug with centos or redhat? I've asked for help on the CentOS users mailing list. So far not much to go on except maybe it's a conflict with a package from a non-CentOS repo (rpmforge?). Try removing/commenting all your cf files especially anything dealing with ip's. Other than that, I would likely update net dns with cpan at least temporarily to see if it fixes thinks. Regards, KAM email builder emailbuilde...@yahoo.com wrote: Anyone have any other insights? Thanks! Running CentOS5 with SpamAssassin v3.3.1-2.el5 installed via yum I remember getting this error a while ago, and it was fixed (don't remember how, but I think just by upgrading), but now it's happening again: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 65 are you still running perl 5.8.8? (perl -v) ! Yes, still. I say that because that's the newest Centos 5 provides. if you have multiple perl sitelibs, you might have a conflict. cd /usr/lib/perl5. ls. how many vendor_perl and site_perl's do you have? cd /usr/lib/perl5/ ls 5.8.8 site_perl vendor_perl I'm not sure what this tells you, but maybe this is helpful: locate INET6.pm /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm or, you could just delete (manually) IO-Socket-INET6 (make a backup first!) on freebsd (with perl 5.10.1): /usr/local/lib/perl5/5.10.1/man/man3/IO::Socket::INET6.3.gz /usr/local/lib/perl5/site_perl/5.10.1/IO/Socket/INET6.pm /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6 /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6/.packlist on fedora, try: find /usr/lib/ -name 'INET6*' (back them up) Same result as above: /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm As I stated in my inquiry, I'd prefer to do my package management via yum if at all possible. Yum tells me to uninstall this package, spamassassin depends on it so has to be removed also. So it does not seem like a good idea to remove it out from under SA The results I get from Google regarding this are all circa 2008. The only hints I can find seem to suggest to remove perl-IO-Socket-INET6, but trying to do so using yum (I don't want to start using another method of package management) tells me that spamassassin is a dependency and will also be removed - obviously undesirable. Perl is up to date on the machinge. no its not :-) Yes, it is :-) Centos/RHEL 5 seems to maintain 5.8.8 with its own updates or some such thing. Thank you for your help
Re: sa-update / perl error again
I have spamassassin-3.3.2-2.el5 installed from rpmforge on el5 - that package, besides being more up to date than the distro version also does not require perl-IO-Socket-INET6. I suspect your version does not really require perl-IO-Socket-INET6 either. You may be right. It was suggested that there may be a conflict due to mixing of packages between CentOS and other (rpmforge?) repos, so maybe it'd sure be nice if CentOS had more up to date software and there was no need to resort to using other repos (rpmforge doesn't have such up to date packages either in some cases I've noted). Do you actually need perl-IO-Socket-INET6? I assume not as you tried to remove it. I did not try to remove it, only saw others suggesting to remove it on the 'net. What does the following show: rpm -q --requires perl-IO-Socket-INET6 perl(:MODULE_COMPAT_5.8.8) perl(Carp) perl(Errno) perl(Exporter) perl(IO::Socket) perl(Socket) perl(Socket6) perl(strict) rpmlib(CompressedFileNames) = 3.0.4-1 rpmlib(PayloadFilesHavePrefix) = 4.0-1 rpmlib(VersionedDependencies) = 3.0.3-1 I would update SpamAssassin from rpmforge and then remove perl-IO-Socket-INET6. Would I lose my current configuration in the process? it's a bit elaborate. IIRC, I'd have to temporarily disable CentOS repo to get the rpmforge one? Your opinion is that there's no fixing it without dumping my current SA package? For reference, here are the perl packages I have installed on my system running SA/Amavisd-new/Clam (.el5 = distro, .rf = rpmforge, .rfx = rpmforge-extras) # rpm -q spamassassin amavisd-new clamav spamassassin-3.3.2-2.el5.rfx.x86_64 amavisd-new-2.6.6-1.el5.rf.x86_64 clamav-0.97.3-1.el5.rf.x86_64 # rpm -qa perl* | sort perl-5.8.8-32.el5_7.6.x86_64 perl-Archive-Tar-1.39.1-1.el5_5.2.noarch perl-Archive-Zip-1.16-1.2.1.noarch perl-Authen-SASL-2.15-1.el5.rf.noarch perl-BerkeleyDB-0.43-1.el5.rf.x86_64 perl-Bit-Vector-6.4-2.2.2.1.x86_64 perl-Carp-Clan-5.3-1.2.1.noarch perl-Compress-Raw-Bzip2-2.037-1.el5.rf.x86_64 perl-Compress-Raw-Zlib-2.037-1.el5.rf.x86_64 perl-Convert-ASN1-0.20-1.1.noarch perl-Convert-BinHex-1.119-2.2.el5.rf.noarch perl-Convert-TNEF-0.17-3.2.el5.rf.noarch perl-Convert-UUlib-1.34-1.el5.rf.x86_64 perl-Crypt-OpenSSL-Random-0.04-1.el5.rf.x86_64 perl-Crypt-OpenSSL-RSA-0.26-1.el5.rf.x86_64 perl-Date-Calc-5.4-1.2.2.1.x86_64 perl-DBI-1.52-2.el5.x86_64 perl-Digest-HMAC-1.01-15.noarch perl-Digest-SHA1-2.11-1.2.1.x86_64 perl-Digest-SHA-5.50-1.el5.rf.x86_64 perl-Email-Date-Format-1.002-1.el5.rf.noarch perl-Encode-Detect-1.01-1.el5.rf.x86_64 perl-Error-0.17016-1.el5.rf.noarch perl-Geography-Countries-2009041301-1.el5.rf.noarch perl-Git-1.7.6.4-1.el5.rf.x86_64 perl-HTML-Parser-3.59-1.el5.x86_64 perl-HTML-Tagset-3.10-2.1.1.noarch perl-IO-Compress-2.037-1.el5.rfx.noarch perl-IO-Multiplex-1.10-1.el5.rf.noarch perl-IO-Socket-SSL-1.01-1.fc6.noarch perl-IO-stringy-2.110-1.2.el5.rf.noarch perl-IO-Zlib-1.04-4.2.1.noarch perl-IP-Country-2.27-1.el5.rf.noarch perl-libwww-perl-5.805-1.1.1.noarch perl-Mail-DKIM-0.39-1.el5.rf.noarch perl-Mail-SPF-2.006-1.el5.rf.noarch perl-MailTools-2.08-1.el5.rf.noarch perl-MIME-Lite-3.027-1.el5.rf.noarch perl-MIME-tools-5.420-2.el5.rf.noarch perl-NetAddr-IP-4.044-1.el5.rf.x86_64 perl-Net-DNS-0.59-3.el5.x86_64 perl-Net-Ident-1.23-1.el5.rf.noarch perl-Net-IP-1.25-2.fc6.noarch perl-Net-Server-0.99-1.el5.rf.noarch perl-Net-SMTP-SSL-1.01-1.el5.rf.noarch perl-Net-SSLeay-1.30-4.fc6.x86_64 perl-Parse-Syslog-1.10-1.el5.rf.noarch perl-Pod-Escapes-1.04-1.2.el5.rf.noarch perl-Pod-Simple-3.16-1.el5.rf.noarch perl-Razor-Agent-2.84-1.el5.rf.x86_64 perl-SGMLSpm-1.03ii-16.2.1.noarch perl-Socket6-0.19-3.fc6.x86_64 perl-String-CRC32-1.4-2.fc6.x86_64 perl-Test-Pod-1.45-1.el5.rf.noarch perl-TimeDate-1.16-5.el5.noarch perl-Unix-Syslog-1.1-1.el5.rf.x86_64 perl-URI-1.35-3.noarch perl-version-0.91-1.el5.rf.x86_64
Re: sa-update / perl error again
What does the following show: rpm -q --requires perl-IO-Socket-INET6 Oops, I meant: rpm -q --whatrequires perl-IO-Socket-INET6 no package requires perl-IO-Socket-INET6 Woa wait a minute! Then why this??? yum remove perl-IO-Socket-INET6 Loaded plugins: fastestmirror Setting up Remove Process Resolving Dependencies -- Running transaction check --- Package perl-IO-Socket-INET6.noarch 0:2.51-2.fc6 set to be erased -- Processing Dependency: perl(IO::Socket::INET6) for package: spamassassin -- Running transaction check --- Package spamassassin.i386 0:3.3.1-2.el5 set to be erased -- Processing Dependency: perl(Mail::SpamAssassin) for package: amavisd-new -- Running transaction check --- Package amavisd-new.i386 0:2.6.6-1.el5.rf set to be erased -- Finished Dependency Resolution Dependencies Resolved == Package Arch Version Repository Size == Removing: perl-IO-Socket-INET6 noarch 2.51-2.fc6 installed 22 k Removing for dependencies: amavisd-new i386 2.6.6-1.el5.rf installed 2.7 M spamassassin i386 3.3.1-2.el5 installed 3.1 M Transaction Summary == Remove 3 Package(s) Reinstall 0 Package(s) Downgrade 0 Package(s) Is this ok [y/N]:
Re: sa-update / perl error again
What does the following show: rpm -q --requires perl-IO-Socket-INET6 Oops, I meant: rpm -q --whatrequires perl-IO-Socket-INET6 no package requires perl-IO-Socket-INET6 Try rpm -q --whatrequires 'perl(IO::Socket::INET6)' - Perl dependencies in RPM-land are rarely expressed in terms of the package names: Yeah: spamassassin-3.3.1-2.el5 -- Running transaction check --- Package perl-IO-Socket-INET6.noarch 0:2.51-2.fc6 set to be erased -- Processing Dependency: perl(IO::Socket::INET6) for package: spamassassin ^^^ For my own RPM-based systems, I keep my own .spec file around since even the RPMForge package pulls in too many other things that I don't use. It's just an older version of the RPMForge one (the one from SA v3.2.3), lightly updated as necessary to build the current release without build errors.
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. How could yum not be seeing the newer package? cat /etc/yum.repos.d/rpmforge.repo ### Name: RPMforge RPM Repository for RHEL 5 - dag ### URL: http://rpmforge.net/ [rpmforge] name = RHEL $releasever - RPMforge.net - dag baseurl = http://apt.sw.be/redhat/el5/en/$basearch/rpmforge mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge enabled = 1 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1 [rpmforge-extras] name = RHEL $releasever - RPMforge.net - extras baseurl = http://apt.sw.be/redhat/el5/en/$basearch/extras mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge-extras #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge-extras enabled = 0 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1 [rpmforge-testing] name = RHEL $releasever - RPMforge.net - testing baseurl = http://apt.sw.be/redhat/el5/en/$basearch/testing mirrorlist = http://apt.sw.be/redhat/el5/en/mirrors-rpmforge-testing #mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge-testing enabled = 0 protect = 0 gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag gpgcheck = 1
Re: sa-update / perl error again
What is the Net::DNS version, are you pure ipv6 and are you 64-bit? perl-Net-DNS-0.63-1.el5.rf You are in no man's land there - the distro uses perl-Net-DNS-0.59-3.el5 and the latest rpmforge package is perl-Net-DNS-0.66-1.el5.rfx. If you're going to use rpmforge packages, keep them up to date (you'll need to enable the rpmforge-extras repo). Hrm, not sure how that could happen, since I don't have rpmforge disabled. How could yum not be seeing the newer package? cat /etc/yum.repos.d/rpmforge.repo Well, knowing there was a newer package out there, hopefully no matter where yum looked for it, I took a chance and removed it: rpm -e --nodeps perl-Net-DNS Then when I asked yum about it again, it found the new one from the CentOS repo, so I installed it... it also needed to install perl-Net-IP which I didn't have. So now I have perl-Net-DNS-0.59-3.el5.i386.rpm Running sa-update on the command line doesn't produce errors, so I guess that the cron won't either. H... how did it get like that in the first place? (and how'd you know to check perl-Net-DNS?)
Re: sa-update / perl error again
Anyone have any other insights? Thanks! Running CentOS5 with SpamAssassin v3.3.1-2.el5 installed via yum I remember getting this error a while ago, and it was fixed (don't remember how, but I think just by upgrading), but now it's happening again: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 65 are you still running perl 5.8.8? (perl -v) Yes, still. I say that because that's the newest Centos 5 provides. if you have multiple perl sitelibs, you might have a conflict. cd /usr/lib/perl5. ls. how many vendor_perl and site_perl's do you have? cd /usr/lib/perl5/ ls 5.8.8 site_perl vendor_perl I'm not sure what this tells you, but maybe this is helpful: locate INET6.pm /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm or, you could just delete (manually) IO-Socket-INET6 (make a backup first!) on freebsd (with perl 5.10.1): /usr/local/lib/perl5/5.10.1/man/man3/IO::Socket::INET6.3.gz /usr/local/lib/perl5/site_perl/5.10.1/IO/Socket/INET6.pm /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6 /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6/.packlist on fedora, try: find /usr/lib/ -name 'INET6*' (back them up) Same result as above: /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm As I stated in my inquiry, I'd prefer to do my package management via yum if at all possible. Yum tells me to uninstall this package, spamassassin depends on it so has to be removed also. So it does not seem like a good idea to remove it out from under SA The results I get from Google regarding this are all circa 2008. The only hints I can find seem to suggest to remove perl-IO-Socket-INET6, but trying to do so using yum (I don't want to start using another method of package management) tells me that spamassassin is a dependency and will also be removed - obviously undesirable. Perl is up to date on the machinge. no its not :-) Yes, it is :-) Centos/RHEL 5 seems to maintain 5.8.8 with its own updates or some such thing. Thank you for your help
Re: sa-update / perl error again
Running CentOS5 with SpamAssassin v3.3.1-2.el5 installed via yum I remember getting this error a while ago, and it was fixed (don't remember how, but I think just by upgrading), but now it's happening again: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 65 are you still running perl 5.8.8? (perl -v) Yes, still. I say that because that's the newest Centos 5 provides. if you have multiple perl sitelibs, you might have a conflict. cd /usr/lib/perl5. ls. how many vendor_perl and site_perl's do you have? cd /usr/lib/perl5/ ls 5.8.8 site_perl vendor_perl I'm not sure what this tells you, but maybe this is helpful: locate INET6.pm /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm or, you could just delete (manually) IO-Socket-INET6 (make a backup first!) on freebsd (with perl 5.10.1): /usr/local/lib/perl5/5.10.1/man/man3/IO::Socket::INET6.3.gz /usr/local/lib/perl5/site_perl/5.10.1/IO/Socket/INET6.pm /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6 /usr/local/lib/perl5/site_perl/5.10.1/mach/auto/IO/Socket/INET6/.packlist on fedora, try: find /usr/lib/ -name 'INET6*' (back them up) Same result as above: /usr/lib/perl5/vendor_perl/5.8.8/IO/Socket/INET6.pm As I stated in my inquiry, I'd prefer to do my package management via yum if at all possible. Yum tells me to uninstall this package, spamassassin depends on it so has to be removed also. So it does not seem like a good idea to remove it out from under SA The results I get from Google regarding this are all circa 2008. The only hints I can find seem to suggest to remove perl-IO-Socket-INET6, but trying to do so using yum (I don't want to start using another method of package management) tells me that spamassassin is a dependency and will also be removed - obviously undesirable. Perl is up to date on the machinge. no its not :-) Yes, it is :-) Centos/RHEL 5 seems to maintain 5.8.8 with its own updates or some such thing. Thank you for your help
sa-update / perl error again
Hi, Running CentOS5 with SpamAssassin v3.3.1-2.el5 installed via yum I remember getting this error a while ago, and it was fixed (don't remember how, but I think just by upgrading), but now it's happening again: Subroutine Net::DNS::Resolver::Base::AF_INET6 redefined at /usr/lib/perl5/5.8.8/Exporter.pm line 65. at /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi/Net/DNS/Resolver/Base.pm line 65 The results I get from Google regarding this are all circa 2008. The only hints I can find seem to suggest to remove perl-IO-Socket-INET6, but trying to do so using yum (I don't want to start using another method of package management) tells me that spamassassin is a dependency and will also be removed - obviously undesirable. Perl is up to date on the machinge. Am I the only one seeing this? What can I do to fix it?
error writing to filter -- Who are you?
Hi, Been a while since I had to look at our mail system (a tribute to postfix/courier/spamassassin/sasl/related software), but when upgrading all our software recently, I ran into a couple issues, the 2nd one being particularly confounding (and not much turns up via a Google search, thus I thought I would post this in case someone else has the same problem(s)). I thought spamc might be the cause of the 2nd, more confusing one, but it turns out to have been innocent, but in case others come here looking for the error I had, I am posting for their sake. We upgraded to Fedora Core 9. We have a virtual users setup with our users in MySQL and no local users. First, I was unable to get any SMTP connections to work - (sasl) auth was failing. We use the pam-mysql package so that sasl can auth against our database. I eventually tracked this down to some odd symlinks -- I had edited /etd/pam.d/smtp as I always do without realizing that it was a symlink to /etc/alternatives/mta-pam, which itself is a symlink to /etc/alternatives/smtp.sendmail. First mistake. Second mistake was I was in too much of a hurry to stop and ask what's this symlink 'alternatives' crap? I just mucked with the symlinks until it worked. Much later, I started to see that our autoresponder was not working. I was seeing some errors I'd never seen before: Aug 3 10:02:19 mail postfix/pipe[6291]: 7BA7D38F21C: to=[EMAIL PROTECTED], relay=maildrop, delay=0.92, delays=0.06/0.01/0/0.85, dsn=5.3.0, status=bounced (internal software error. Command output: sh: /usr/bin/spamc: No such file or directory maildrop: error writing to filter. Who are you? ) Huh? Well, after a while being stupid, I managed to realize I have to look above that for any other errors in the mail log. I found: Aug 3 10:02:18 mail sendmail[6297]: m192B5Z9926297: SYSERR(UID5021): Who are you? Aug 3 10:02:18 mail sendmail[6297]: m192B5Z9926297: Authentication-Warning: mail.example.com: Unknown UID 5021 set sender to using -f OK, so the problem is that the MTA is not understanding who the message is from, when it gets reinjected from spam checks. The 5021 uid is a virtual uid, not a system one. Why is the MTA looking for a *system* uid? Hmm, looks like it's the real sendmail and not postfix masquerading as postfix. I don't know how to tell the difference, so I Googled around a lot, but didn't find much until I see once again that the sendmail binary is actually a symlink. Ah, here we go again with this alternatives crap: /usr/sbin/sendmail -- /etc/alternatives/mta -- /usr/sbin/sendmail.sendmail First I just changed that last symlink to point to /usr/sbin/sendmail.postfix, but when starting postfix I get: postfix/postfix-script: warning: /usr/lib/sendmail and /usr/sbin/sendmail differ postfix/postfix-script: warning: Replace one by a symbolic link to the other postfix/postfix-script: starting the Postfix mail system More Googling - not many results. OK, OK, I'll try to understand what this all means. Google still didn't give me much, but I see it's some dumb package switcher. man alternatives http://dailypackage.fedorabook.com/index.php?/archives/6-Wednesday-Why-The-Alternatives-System.html http://linux.derkeiler.com/Mailing-Lists/RedHat/2004-08/0256.html That's about all I found, but it's enough. Personally, this seems ridiculous to me, but oh well. I changed all the symlinks for both the mta stuff and the pam stuff back to what they were in the first place (and edited the correct pam.d/smtp.postfix file) and did it the Right Way: alternatives --set mta /usr/sbin/sendmail.postfix Verify with: alternatives --display mta Yeesh. Now it works. HTH
Re: per-user or global bayes (was: HUGE bayes DB (non-sitewide) advice?)
bump --- Michael Monnerie [EMAIL PROTECTED] wrote: My users are quite happy with overall markup of the spam. We occasionally get a HAM marked as SPAM. We have an odd client base though. The question is: when to use global and when per-user bayes? On our server, we have people of different languages, communicating with different countries all over the world, in different areas (advertising, production, IT, etc.). I thought in that case a per-user bayes would be much better, as viagra is something good for the one, but bad for the other. What's the general recommendation for bayes? __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
Re: HUGE bayes DB (non-sitewide) advice?
Just a follow-up to my own brain-lapse:
If you define a custom user scores query like this:
user_scores_sql_custom_querySELECT preference, value FROM
spamassassin_settings WHERE username = _USERNAME_ OR username = '!GLOBAL'
OR
username = CONCAT('@', _DOMAIN_) ORDER BY username ASC
Then you can easily decide to use bayes on a per-domain basis for one or
more
of your domains (and still have per-user bayes for all other domains). A
sample insert row into the settings table, then, would be:
INSERT INTO spamassassin_settings (username, preference, value) VALUES
('@example.com', 'bayes_sql_override_username', 'example.com');
So everyone in the example.com domain shares all bayes information which
is
placed under the username example.com.
is that in the FAQ? because it certainly sounds like a cool tip for
Bayes/SQL users.
I don't think so. One other thing to note about this setup is:
I think I caught the idea of using !GLOBAL from someone's how-to a while back
(IIRC, the manual suggests @GLOBAL), this way the global settings can be
ordered in the query to always override any per-domain settings.
(there should really be a section of the FAQ dedicated to that stuff.)
Would be nice.
__
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com
RE: HUGE bayes DB (non-sitewide) advice?
Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively capture your various user email habits? Default. How can you be running the default value, when the manual says that 15 tokens is only 8MB?? How do you end up with 40MB of data?: bayes_expiry_max_db_size (default: 15) What should be the maximum size of the Bayes tokens database? When expiry occurs, the Bayes system will keep either 75% of the maximum value, or 100,000 tokens, whichever has a larger value. 150,000 tokens is roughly equivalent to a 8Mb database file. __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: HUGE bayes DB (non-sitewide) advice?
I guess the relevant point for this thread is that I don't necessarily think that this is the silver bullet as implied. Even if you use a high-availability clustering technology that can mirror writes and reads, you are STILL dealing with the possibility of a database that is just massive. Processing this size of database will still be disk-bound unless you have an unheard-of amount of memory; I don't think there's any reason to think that clustering the problem will make it go away. So I still wonder if anyone has any musings on my earlier questions? A few spamassassin hacks could help. 1. Have multiple mysql servers, split your users into A-J, K-S, T-Z OR smaller units and distribute them over different servers, with some HA / failover mechanism (possibly drbd). 2. Have 2 level of bayes, one large global and the other smaller per user if thats possible. Of course SA will need to be changed to use both the bayes'. This way you could have 2 large servers for the global bayes db and 2 for the per user bayes dbs. Also see if this SQL failover patch can help you in any way. http://issues.apache.org/SpamAssassin/show_bug.cgi?id=2197 Thanks for the good thoughts. Sounds like the ultimate answer is that not many people are using per-user Bayes, at least at this level, and that any solutions are yet to be realized in practice. I don't think we've got the resources or time to contribute any SA patches, but the food for thought is very much appreciated! Finally to speed up the database have a look at this, the people at wikimedia / livejournal seem to be happy using it. http://www.danga.com/memcached/ That's very cool. I'll *definitely* be keeping this one in mind. __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
RE: HUGE bayes DB (non-sitewide) advice?
Thanks a lot for checking, Gary! --- Gary W. Smith [EMAIL PROTECTED] wrote: You're right, my guy gave me the size of bayes + awl. The real number is 14.5mb. (with an overhead of 3.2mb). Not sure, that's just what phpmyadmin is reporting. I'll check again. I can't remember if the DB is in double byte or not. One of my guys tweaked it for some other little databases on the same box. Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively capture your various user email habits? Default. How can you be running the default value, when the manual says that 15 tokens is only 8MB?? How do you end up with 40MB of data?: bayes_expiry_max_db_size (default: 15) What should be the maximum size of the Bayes tokens database? When expiry occurs, the Bayes system will keep either 75% of the maximum value, or 100,000 tokens, whichever has a larger value. 150,000 tokens is roughly equivalent to a 8Mb database file. __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: OT: DB connections coming from unqualified hostname
When we connect to our bayes/awl/user_scores databases, the connections are being made by clients with unqualified hostnames. If we try to use GRANTs such as 'user'@'%.example.com', connections are refused since only the hostname portion is being used to connect I guess. For example, if a hostname is gaia, a GRANT of 'user'@'gaia' works correctly, but not the above wildcard. Our connections are all over a local area network. Can anyone shed light on how to force connecting clients to be recognized with a fully qualified hostname so we don't have to keep track of GRANTs for every one of our spamd client machines? Thanks! /etc/hosts is your friend.. have a simple mapping of the IP to Hostname (assuming that the IP address doesn't change) and a corresponding entry for the hostname in the mysql (db/user) tables. Um, not sure I follow you. We have this in /etc/hosts 10.10.10.50gaia gaia.example.com So that DNS resolves correctly for our hosts. If I switch it to this, will that do the trick? 10.10.10.50gaia.example.com gaia No, doesn't seem to help. Our MySQL users are only allowed to log in if their hostname is just gaia, whereas we'd like to just use %.example.com instead. The client gives the error: ERROR 1045 (28000): Access denied for user 'user'@'gaia' (using password: YES) Which seems to indicate that the client is connecting with an unqualified hostname. However, the machine's hostname appears to be correct: # hostname gaia.example.com Surely I am missing something simple. :) Also make sure you have this in /etc/nsswitch.conf, to ensure that /etc/hosts gets a higher priority over /etc/resolv.conf hosts: files dns __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: OT: DB connections coming from unqualified hostname
When we connect to our bayes/awl/user_scores databases, the connections are being made by clients with unqualified hostnames. If we try to use GRANTs such as 'user'@'%.example.com', connections are refused since only the hostname portion is being used to connect I guess. For example, if a hostname is gaia, a GRANT of 'user'@'gaia' works correctly, but not the above wildcard. Our connections are all over a local area network. Can anyone shed light on how to force connecting clients to be recognized with a fully qualified hostname so we don't have to keep track of GRANTs for every one of our spamd client machines? Thanks! /etc/hosts is your friend.. have a simple mapping of the IP to Hostname (assuming that the IP address doesn't change) and a corresponding entry for the hostname in the mysql (db/user) tables. Um, not sure I follow you. We have this in /etc/hosts 10.10.10.50gaia gaia.example.com So that DNS resolves correctly for our hosts. If I switch it to this, will that do the trick? 10.10.10.50gaia.example.com gaia No, doesn't seem to help. Our MySQL users are only allowed to log in if their hostname is just gaia, whereas we'd like to just use %.example.com instead. The client gives the error: ERROR 1045 (28000): Access denied for user 'user'@'gaia' (using password: YES) Same happens with clients on the same machine, except the connection is seen from localhost (makes sense, sure, but I would like the connection to be approved as if being made from any other machine in our LAN) Access denied for user 'user'@'localhost' (using password: YES) TIA Which seems to indicate that the client is connecting with an unqualified hostname. However, the machine's hostname appears to be correct: # hostname gaia.example.com Surely I am missing something simple. :) Also make sure you have this in /etc/nsswitch.conf, to ensure that /etc/hosts gets a higher priority over /etc/resolv.conf hosts: files dns __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
Re: OT: DB connections coming from unqualified hostname
--- email builder [EMAIL PROTECTED] wrote: When we connect to our bayes/awl/user_scores databases, the connections are being made by clients with unqualified hostnames. If we try to use GRANTs such as 'user'@'%.example.com', connections are refused since only the hostname portion is being used to connect I guess. For example, if a hostname is gaia, a GRANT of 'user'@'gaia' works correctly, but not the above wildcard. Our connections are all over a local area network. Can anyone shed light on how to force connecting clients to be recognized with a fully qualified hostname so we don't have to keep track of GRANTs for every one of our spamd client machines? Thanks! /etc/hosts is your friend.. have a simple mapping of the IP to Hostname (assuming that the IP address doesn't change) and a corresponding entry for the hostname in the mysql (db/user) tables. Um, not sure I follow you. We have this in /etc/hosts 10.10.10.50gaia gaia.example.com So that DNS resolves correctly for our hosts. If I switch it to this, will that do the trick? 10.10.10.50gaia.example.com gaia No, doesn't seem to help. Our MySQL users are only allowed to log in if their hostname is just gaia, whereas we'd like to just use %.example.com instead. The client gives the error: ERROR 1045 (28000): Access denied for user 'user'@'gaia' (using password: YES) Same happens with clients on the same machine, except the connection is seen from localhost (makes sense, sure, but I would like the connection to be approved as if being made from any other machine in our LAN) Access denied for user 'user'@'localhost' (using password: YES) Also tried putting '10.10.%' in the user table, but no dice. Which seems to indicate that the client is connecting with an unqualified hostname. However, the machine's hostname appears to be correct: # hostname gaia.example.com Surely I am missing something simple. :) Also make sure you have this in /etc/nsswitch.conf, to ensure that /etc/hosts gets a higher priority over /etc/resolv.conf hosts: files dns __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
Re: HUGE bayes DB (non-sitewide) advice?
Well, I know there have to be some admins out there who have a lot of
users
and do not use sitewide bayes.. RIGHT? See original email snippet at
bottom.
snip
* Other ideas:
- increase system memory as much as possible
- per-domain Bayes instead of per-user???
This might be our 2nd best choice (unless there is a good
bayes_expiry_max_db_size solution), but I don't see anything in the manual
about the syntax of bayes_sql_override_username. The manual mentions
grouping, but gives no examples of how I could, for instance, group bayes
data by domain (my usernames are in the form [EMAIL PROTECTED]).
Just a follow-up to my own brain-lapse:
If you define a custom user scores query like this:
user_scores_sql_custom_querySELECT preference, value FROM
spamassassin_settings WHERE username = _USERNAME_ OR username = '!GLOBAL' OR
username = CONCAT('@', _DOMAIN_) ORDER BY username ASC
Then you can easily decide to use bayes on a per-domain basis for one or more
of your domains (and still have per-user bayes for all other domains). A
sample insert row into the settings table, then, would be:
INSERT INTO spamassassin_settings (username, preference, value) VALUES
('@example.com', 'bayes_sql_override_username', 'example.com');
So everyone in the example.com domain shares all bayes information which is
placed under the username example.com.
- cluster Bayes DB???
This apparently is not an option, since clustered MySQL databases are kept
entirely in memory. We don't have any 10GB RAM machines sadly :)
From the MySQL manual:
In-memory storage:
All data stored in each data node is kept in memory on the node's host
computer. For each data node in the cluster, you must have available an
amount of RAM equal to the size of the database times the number of
replicas,
divided by the number of data nodes. Thus, if the database takes up 1
gigabyte of memory, and you wish to set up the cluster with 4 replicas and
8
data nodes, a minimum of 500 MB memory will be required per node. Note that
this is in addition to any requirements for the operating system and any
other applications that might be running on the host.
__
Start your day with Yahoo! - Make it your home page!
http://www.yahoo.com/r/hs
Re: HUGE bayes DB (non-sitewide) advice?
In-memory storage: All data stored in each data node is kept in memory on the node's host computer. For each data node in the cluster, you must have available an amount of RAM equal to the size of the database times the number of replicas, This refers to the first line: In-memory storage. Of course you can't do that with 160GB DBs. You can still cluster - look at DRBD http://www.drbd.org/ I guess the relevant point for this thread is that I don't necessarily think that this is the silver bullet as implied. Even if you use a high-availability clustering technology that can mirror writes and reads, you are STILL dealing with the possibility of a database that is just massive. Processing this size of database will still be disk-bound unless you have an unheard-of amount of memory; I don't think there's any reason to think that clustering the problem will make it go away. So I still wonder if anyone has any musings on my earlier questions? __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
RE: HUGE bayes DB (non-sitewide) advice?
Our production database for a large number of emails (but using site wide) is about 40mb. What is your bayes_expiry_max_db_size set to? Do you feel that it has been enough to effectively capture your various user email habits? __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: HUGE bayes DB (non-sitewide) advice?
Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. I'll start the ball rolling with what few tweaks we've made, although they are not enough; we desperately need more ideas to make this viable. * bayes_auto_expire is turned on; cronning the expiry of 20K+ accounts every night seems outrageous * bayes_expiry_max_db_size is at its default value; if 20K accounts used the maximum allowable space, then, we'd have a 160GB bayes DB. If 8MB is considered sufficient for a whole domain for some people, then perhaps we can reduce this size for per-user bayes...?? * MySQL tuning for InnoDB: pretty much straight from the MySQL manual... - multiple data files (approx 10G each) - innodb_flush_log_at_trx_commit=0 because it's faster and we don't care about Bayes data enough that the risk of losing one second of data is fine - innodb_buffer_pool_size as large as we can handle, but even if this was 3 or more GB, it's only a fraction of a 160GB database - innodb_additional_mem_pool_size=20M because that's what we saw for their big example, although I am wondering in particular about the value of increasing this one - innodb_log_file_size 25% of innodb_buffer_pool_size * Other ideas: - increase system memory as much as possible - per-domain Bayes instead of per-user??? - cluster Bayes DB??? - revert to MyISAM -- will this help THAT much? I'm wondering if anyone out there hosts a large number of users with per-USER bayes (in MySQL)? Our user base is varied enough that we do not feel bayes would be effective if done site-wide. Some people like their spammy newsletters, some are geeks who would deeply resent someone training newsletters to be ham. As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). We went to InnoDB when we upgraded to 3.1 per the upgrade doc's recommendation, so that also means things are a bit slower. Watching mytop, most all the activity we get is from bayes inserts, which is not surprising, and is probably the cause of why we get a lot of iowait, trying to keep writing to an 8G tablespace... We've tuned the InnoDB some, but performance is still not all that good -- is there anyone out there who runs a system like this? * What kinds of MySQL tuning are people using to help cope? * Are there any SA settings to help allieviate performance problems? * If we want to walk away from per-user bayes, is the only option to go site-wide? What other options are there? __ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Re: HUGE bayes DB (non-sitewide) advice?
Well, I know there have to be some admins out there who have a lot of users and do not use sitewide bayes.. RIGHT? See original email snippet at bottom. snip * Other ideas: - increase system memory as much as possible - per-domain Bayes instead of per-user??? This might be our 2nd best choice (unless there is a good bayes_expiry_max_db_size solution), but I don't see anything in the manual about the syntax of bayes_sql_override_username. The manual mentions grouping, but gives no examples of how I could, for instance, group bayes data by domain (my usernames are in the form [EMAIL PROTECTED]). - cluster Bayes DB??? This apparently is not an option, since clustered MySQL databases are kept entirely in memory. We don't have any 10GB RAM machines sadly :) From the MySQL manual: In-memory storage: All data stored in each data node is kept in memory on the node's host computer. For each data node in the cluster, you must have available an amount of RAM equal to the size of the database times the number of replicas, divided by the number of data nodes. Thus, if the database takes up 1 gigabyte of memory, and you wish to set up the cluster with 4 replicas and 8 data nodes, a minimum of 500 MB memory will be required per node. Note that this is in addition to any requirements for the operating system and any other applications that might be running on the host. __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
RE: HUGE bayes DB (non-sitewide) advice?
As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire Using? So you are saying you use non-sitewide bayes but you limit your max DB size to something much smaller than the default? Care to share your settings? No, I use sitewide bayes. We left these at their defaults (not unintentionally). If we have 20K users, the default max of 150,000 tokens at roughly 8MB comes out to 160GB. We have the disk space, but just not sure if we have the tuning it would take to handle a DB of that size. What I am looking for is tuning help or other ideas on how to achieve some reasonable level of bayes personalization without drowning our DB resources. For optimum performance you probably want the bayes database to fit into RAM, along with all of your spamassassin objects and anything else on the server. You might consider buying a dedicated Bayes DB server with 4 GB of RAM, and cutting bayes_expiry_max_db_size in half. That should do it. That should do it today (actually, the database is now 9GB), but not when it has grown to 160GB. I appreciate the tips, but what I am looking for is MySQL tuning advice and thoughts/ideas/other approaches to having at least somewhat personalized Bayes stores for well over 20K users. *SOMEONE* out there has to be doing something like this, no??? If the DB fits into RAM, the SQL engine should be able to make transactional changes in RAM and lazily spool them to the disk without forcing other transactions to wait. __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
HUGE bayes DB (non-sitewide) advice?
Hi all, I'm wondering if anyone out there hosts a large number of users with per-USER bayes (in MySQL)? Our user base is varied enough that we do not feel bayes would be effective if done site-wide. Some people like their spammy newsletters, some are geeks who would deeply resent someone training newsletters to be ham. As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). We went to InnoDB when we upgraded to 3.1 per the upgrade doc's recommendation, so that also means things are a bit slower. Watching mytop, most all the activity we get is from bayes inserts, which is not surprising, and is probably the cause of why we get a lot of iowait, trying to keep writing to an 8G tablespace... Oh, and we let bayes do its token cleanup on the spot (sorry, not remembering the config setting name right now), not at night, since a small lag in delivery is acceptable, but figuring out how to run an absolutely huge cleanup by cron every night in this scenario seems like it'd really kill the DB (and we'd have to run sa-learn once for every single user, right... ugh) We've tuned the InnoDB some, but performance is still not all that good -- is there anyone out there who runs a system like this? * What kinds of MySQL tuning are people using to help cope? * Are there any SA settings to help allieviate performance problems? * If we want to walk away from per-user bayes, is the only option to go site-wide? What other options are there? __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
RE: HUGE bayes DB (non-sitewide) advice?
--- [EMAIL PROTECTED] wrote: email builder wrote: As a result of this, however, we are currently burdened with an 8GB(! yep, you read it right) bayes database (more than 20K users having mail delivered). Consider using bayes_expiry_max_db_size in conjunction with bayes_auto_expire Using? So you are saying you use non-sitewide bayes but you limit your max DB size to something much smaller than the default? Care to share your settings? We left these at their defaults (not unintentionally). If we have 20K users, the default max of 150,000 tokens at roughly 8MB comes out to 160GB. We have the disk space, but just not sure if we have the tuning it would take to handle a DB of that size. What I am looking for is tuning help or other ideas on how to achieve some reasonable level of bayes personalization without drowning our DB resources. Thanks __ Yahoo! FareChase: Search multiple travel sites in one click. http://farechase.yahoo.com
RE: best of RBLs without the FPs
But again, since almost no legitimate email is ever greylisted only almost nothing DESIRABLE EVER gets delayed. So you ONLY greylist what the RBLs tell you is on their list? Maybe I need to go back and re-read your original email, which I skimmed perhaps too lightly... because even back in the day before we used greylisting (we use straight), and only had something like four RBLs rejecting mail outright, we still saw a lot of spam getting through (for SA to score). So I just can't imagine that selective greylisting of whatever is on the RBLs will catch nearly as much as you'd want it to. What are your other mechanisms for tempfailing beside RBL? __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
RE: best of RBLs without the FPs
--- email builder [EMAIL PROTECTED] wrote: But again, since almost no legitimate email is ever greylisted only almost nothing DESIRABLE EVER gets delayed. So you ONLY greylist what the RBLs tell you is on their list? Maybe I need to go back and re-read your original email, which I skimmed perhaps too lightly... because even back in the day before we used greylisting (we use straight), and only had something like four RBLs rejecting mail outright, we still saw a lot of spam getting through (for SA to score). So I just can't imagine that selective greylisting of whatever is on the RBLs will catch nearly as much as you'd want it to. What are your other mechanisms for tempfailing beside RBL? Sorry, your subsequent emails answered this -- SA seems to be the other tool that pushes a message into the greylist zone. With these two (two right? not any more?) tools driving your greylisting, I'm curious how many (suspicious) mails make it to your spam buckets (or even to your inbox)? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
RE: Hotmail on sorbs?!? (and eliminating false positives)
--- Herb Martin [EMAIL PROTECTED] wrote: I am new to postfix and spamassassin, but we are already using greylist, and I liked a lot what you said here. How can I greylist messages by means of RBL checking? How should I setup Postfix to do that? Regards, Carlos. I am not a Postfix expert, and cannot really call myself an Exim expert either but the strategy goes something like this: During (various) SMTP ACL (Access Control Lists) run the checks for things like RBL etc (this is easy in Exim) and mark the results (in either an ACL variable or by adding a header.*) * Header had the disadvantage of requiring the Greylist check to wait until SMTP DATA time where the headers are available when all we really need is SenderIP-FromName-RCPT which are all available by RCPT ACL time. When you have made all of your checks, and before checking SpamAssassin, run the Greylist on any message that was flagged above -- if the greylist returns true this is where we tempfail (Defer in Exim) the message. The above can probably be done in Postfix with one or two restriction classes. http://www.postfix.org/postconf.5.html#smtpd_restriction_classes http://www.postfix.org/RESTRICTION_CLASS_README.html I'd be curious to hear if anyone else is using this kind of strategy. Thanks __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Hotmail on sorbs?!?
On Donnerstag, 22. September 2005 22:24 email builder wrote: How so? I can't believe you don't hear me when I say for the 100th time that services like ours that have a lot of users who expect to communicate with hotmail users cannot use an RBL in the MTA if it lists hotmail. Larry said it already: There are RCVD_IN_SORBS_* rules in 20_dnsbl_tests.cf for each of the various SORBS lists. The ones for RCVD_IN_SORBS_SPAM are commented out. We're also having lots of customers communicating with hotmail.com, didn't get a report of problems for months. Just pick the right rules. If the RCVD_IN_SORBS_SPAM doesn't fit you, don't activate it, it's disabled by default (I guess for a reason...). No. Please understand that there is a difference between using SORBS in the MTA (ala Postfix's smtpd_recipient_restrictions) where a listing equates to an immediate rejection and using SORBS in SA for scoring. You are referring to the latter. I have said many times that the thread was about the former. I don't think anyone disagrees with using SORBS in SA scoring. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Hotmail on sorbs?!?
We removed sorbs. I don't think it's even open for debate at the current point. If places like hotmail mx's end up on the blacklist you *will* have upset customers. Yeah. It would be nice if there were a blacklist out there that took the best of all the others but refused to list things like hotmail for those of us who are in the situation of having users who expect connectivity to Hotmail and their ilk. Yes, it sucks, but this is what it is to have paying customers with friends who use MSN, etc, etc. So, then, where should they draw that line? Let in hotmail, yahoo, aol, verizon, and earthlink yes. i don't think any administrator with paying customers to please would be happy if any of these were blacklisted. I think you're wrong. I think it depends upon the customer base. Of course, that's why I've said over and over that for situations like ours where a large customer base expects to be able to correspond with Hotmail users, this situation is not tenable. If you are lucky enough to have a group of users who are all that savvy and none of them care about hotmail, then you're lucky, and I suspect, you are also rare. ... but who not to whitelist? the small guys. unfortunately, large ISPs like that have power in the number of users they have. in no way do I advocate defending that as a good thing, but the fact that this gives them an immense amount of power to do whatever they want regarding rfcs and whatnot remains a reality. smaller services are the only organizations who are going to actually be potentially moved to action by landing on one of these RBLs. when was the last time SORBS managed to change Hotmail's policies? Has SORBS ever really changed anyones policies? That's certainly not what I use RBL's for. I couldn't give a rats posterior about whether or not some spammer changes careers, or some mail server changes configurations, or some ISP changes their appropriate use policies. It's too bad you have such a self-centered attitude about it. I like the idea that people care enough to want to see the Internet change for the better. If SORBS is making life hard for spammers and those who host them, I want to support them as best I can. Until now, that meant taking the time to explain to our users what the SORBS bounces meant and to have them go bug the offending admins. And, what if half of your user/customer base does NOT want you to white list aol but c'mon, when was the last time someone's user base was emailing their support staff begging for aol to be blacklisted? beside, this is what per-user settings for something like SA are for. If you're in a situation where users can have per-user settings. For example, that doesn't work here. Or, if that's how you're using your RBLs. People DO use rbls as block lists, and people do use SORBS as a block list. It's hard to have per-user settings for that. That is rough. You might look into SQL-based SA per user settings. It's very handy. does want you to whitelist hotmail ... while the other half of your base is exactly the opposite? It isn't a solvable problem, IMO. Everyone will want to draw the line differently, so there wont be an easy solution of that nature. But BLOCKING all mails from somewhere like Hotmail *IS* a decision that someone has made which is not acceptable to we who support large user bases. But it's not their job to cater to YOUR user base decisions. That's _your_ job. So we have to make the opposite decision to only use those RBLs in SA scoring. The baseline here is that you cannot outright ban whole large services -- Actually, yes, I can. And I have, for some periods of time (only, in my case, it was yahoo). Sure, but who here in their right mind thinks that's a good idea? At least those of us with a userbase that actually corresponds with users on yahoo, aol, hotmail... since you apparently live in the world the rest of us fantasize about. And SORBS can. And, really, you can too, you just choose not to. But even if you remove that from the argument, the point is, it's not the RBL's job to cater to your policies. And if they were to try to cater to everyones policies, they would be so conflicting that it would be pointless. Which was my point for the above quoted sections. And, it's not just that I don't think the RBL can do it, I don't think that kind of thing is the job of the RBL. I think that kind of thing is your job (or, in my case, it's my job). What's our job? Banning all of Hotmail? No. Your job is to tailor the tools you use so that they fit your organization. SORBS job is to provide a list of sites that fit a particular behavior. If you want there to be exceptions to that list, then it is YOUR job to make those exceptions, not theirs. Of course. Didn't you read the part of
Re: Hotmail on sorbs?!?
... but who not to whitelist? the small guys. unfortunately, large ISPs like that have power in the number of users they have. in no way do I advocate defending that as a good thing, but the fact that this gives them an immense amount of power to do whatever they want regarding rfcs and whatnot remains a reality. smaller services are the only organizations who are going to actually be potentially moved to action by landing on one of these RBLs. when was the last time SORBS managed to change Hotmail's policies? Has SORBS ever really changed anyones policies? That's certainly not what I use RBL's for. I couldn't give a rats posterior about whether or not some spammer changes careers, or some mail server changes configurations, or some ISP changes their appropriate use policies. It's too bad you have such a self-centered attitude about it. Me!? You're the one who presented the when was the last time SORBS ... question as though it was some universal benchmark of RBL value. Nope. The point that you entirely missed was that RBLs don't have much sway with services with huge numbers of users like Hotmail. Thus many of us cannot use RBLs on the front-line that block services like that. I never once said that SORBS et al. are useless, and did not reject their more lofty goals as irrelevant as you explicitly did. I, for one, enjoy the thought of people working together for goals outside of their own self-interest. You're the one projecting their opinion onto other people here. I was How so? I can't believe you don't hear me when I say for the 100th time that services like ours that have a lot of users who expect to communicate with hotmail users cannot use an RBL in the MTA if it lists hotmail. The only projecting I've done is to assume (reasonably surely) that there are a lot of other admins/servcies in the same boat. merely pointing out that it's not a universally relevant question to determining the value of the RBL and who it chooses to block. An RBL is most certainly of less value *to our company* as a tool to be used in the MTA if it lists hotmail. Of course that is relevant. I'm still not saying that means they should change their approach -- I am, however, musing about how nice it'd be to have another RBL that worked mostly the same sans the hotmail listings. And, it's not just that I don't think the RBL can do it, I don't think that kind of thing is the job of the RBL. I think that kind of thing is your job (or, in my case, it's my job). What's our job? Banning all of Hotmail? No. Your job is to tailor the tools you use so that they fit your organization. SORBS job is to provide a list of sites that fit a particular behavior. If you want there to be exceptions to that list, then it is YOUR job to make those exceptions, not theirs. Of course. Didn't you read the part of my post that started all this? Yes, I did. And I a) pointed out that such a service can't exist on anything approaching a large scale (large enough to be worth running) because there is no universal place to draw the cut-off for who to whitelist and who to not whitelist, Most RBLs consist of some amount of listings based on subjective decisions. There is no reason that someone else can't start a list with this goal in mind. Dunno if it'd be popular or even successful, but I was musing. Allow one to muse, why dontcha. b) that you can perform that service for yourself, by using their blacklist as a starting point and trimming out those addresses that match your whitelist, and then using that as your production list, And I said that I don't think it is desirable to place the onus of RBL maintenance onto all the sysadmins out there. Why are you so pissed off at ME for putting that out there? Who said anything about being pissed off? Your choice of words intone more than just arguing a point, the most obvious being asking if I know what a cron job is. Give me a break. Though, you clearly don't get what I'm saying, so it does make the conversation rather pointless. That's what I mean. Of course I understand your point, and I am explaining somewhat repetitively the parts I don't think you are addressing accurately. that you quoted ... it is performed by a script. I do no such manual thing. I get an email every few hours that tells me what happened, I scan it for references to networks that I am responsible for, and it tells me yes, I removed all of those networks from our copy of the RBL zone. Then I put the zone into production on my own name servers, so that I never see those sites showing up as RBL'ed. My point was that generally pulling apart RBL functionality and placing part of the onus of managing it back on the admin's plate is not going to be something that goes over well, even if you have a nifty script that works with one RBL. Sorry you
Re: Hotmail on sorbs?!?
goals as irrelevant as you explicitly did. I, for one, enjoy the thought of people working together for goals outside of their own self-interest. So we circle back to the start: you apparently have a utopian user community that does not have any need to correspond with hotmail users. For the rest of us, SORBS can't be used in the MTA if it blocks Hotmail. Too bad - really too bad. OTOH, after reading stuff like this, it makes me wish everyone decided to block hotmail, which might finally get their attention: http://chris-linfoot.net/d6plinks/CWLT-6FYBJY http://chris-linfoot.net/d6plinks/CWLT-64QC9K Wondering how long we can fend off our unhappy users before we start to loose business due to the Hotmail block (which still seems to be going strong in our logs) __ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com
Hotmail on sorbs?!?
Complaint from a user led me to find this in our logfile: Sep 21 09:07:07 gaia postfix/smtpd[6392]: NOQUEUE: reject: RCPT from bay101-f11.bay101.hotmail.com[64.4.56.21]: 554 Service unavailable; Client host [64.4.56.21] blocked using dnsbl.sorbs.net; Spam Received See: http://www.sorbs.net/lookup.shtml?64.4.56.21; from=[EMAIL PROTECTED] to=[EMAIL PROTECTED] proto=ESMTP helo=hotmail.com We are a semi-high volume site with plenty of people who expect to receive hotmail mail, so this is REALLY BAD. I cringe at the thought of making a MTA top-level whitelist entry for all the hotmail IPs that I can find, or of removing sorbs from our list of postfix RBLs. I also don't much care for the idea of using Sorbs only to tally points in SA, since we get so much crap, we'd like to reject most of the obvious stuff out of the gate - otherwise I envision our hard drives filling up twice as fast with crap nobody wants anyway. It sucks that microsoft can just do whatever they want since they have sooo many users, but as this is the current state of our reality, I am interested in what people are doing to deal with it as is. Is this causing anyone else problems? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Hotmail on sorbs?!?
Sep 21 09:07:07 gaia postfix/smtpd[6392]: NOQUEUE: reject: RCPT from bay101-f11.bay101.hotmail.com[64.4.56.21]: 554 Service unavailable; Client host [64.4.56.21] blocked using dnsbl.sorbs.net; Spam Received See: http://www.sorbs.net/lookup.shtml?64.4.56.21; from=[EMAIL PROTECTED] to=[EMAIL PROTECTED] proto=ESMTP helo=hotmail.com I also don't much care for the idea of using Sorbs only to tally points in SA, since we get so much crap, we'd like to reject most of the obvious stuff out of the gate - otherwise I envision our hard drives filling up twice as fast with crap nobody wants anyway. I'm afraid you're going to have to do that, in my opinion -- you'll see lots of FPs using SORBS as a front-line block. Try other DNSBLs, but SORBS is just not suitable. OK, well other people also seem to be saying that SORBS is just not acceptable for a front-line RBL. Really too bad. Really. We do already use a ton of other RBLs, but just hate to let any of them go. Can someone remind me if I am correct in my recollection that SORBS comes enabled by default for use in contributing to SA scores? Thanks a lot __ Yahoo! for Good Donate to the Hurricane Katrina relief effort. http://store.yahoo.com/redcross-donate3/
Re: Hotmail on sorbs?!?
Complaint from a user led me to find this in our logfile: Sep 21 09:07:07 gaia postfix/smtpd[6392]: NOQUEUE: reject: RCPT from bay101-f11.bay101.hotmail.com[64.4.56.21]: 554 Service unavailable; Client host [64.4.56.21] blocked using dnsbl.sorbs.net; Spam Received See: http://www.sorbs.net/lookup.shtml?64.4.56.21; from=[EMAIL PROTECTED] to=[EMAIL PROTECTED] proto=ESMTP helo=hotmail.com A customer led me to the same block. Unfortunately they were some very important emails and the customer was more than a little irate. eeck. ;) We removed sorbs. I don't think it's even open for debate at the current point. If places like hotmail mx's end up on the blacklist you *will* have upset customers. Yeah. It would be nice if there were a blacklist out there that took the best of all the others but refused to list things like hotmail for those of us who are in the situation of having users who expect connectivity to Hotmail and their ilk. Yes, it sucks, but this is what it is to have paying customers with friends who use MSN, etc, etc. I also don't much care for the idea of using Sorbs only to tally points in SA, since we get so much crap, we'd like to reject most of the obvious stuff out of the gate - otherwise I envision our hard drives filling up twice as fast with crap nobody wants anyway. Look at other rbl's, consider some or all of: abuse.rfc-ignorant.org dsn.rfc-ignorant.org OK, we use dsn only. I'm curious if their other lists are widely seen to be just as useful? list.dsbl.org sbl-xbl.spamhaus.org opm.blitzed.org We are using all these with the addition of: bl.spamcop.net combined.njabl.org relays.ordb.org cbl.abuseat.org blackhole.securitysage.com (reject_rhsbl_sender in postfix, although I'm not sure how useful this one is nowadays?) blackhole.securitysage.com (reject_rhsbl_client in postfix, although I'm not sure how useful this one is nowadays?) Thanks, Nathanael __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Hotmail on sorbs?!?
We removed sorbs. I don't think it's even open for debate at the current point. If places like hotmail mx's end up on the blacklist you *will* have upset customers. Yeah. It would be nice if there were a blacklist out there that took the best of all the others but refused to list things like hotmail for those of us who are in the situation of having users who expect connectivity to Hotmail and their ilk. Yes, it sucks, but this is what it is to have paying customers with friends who use MSN, etc, etc. So, then, where should they draw that line? Let in hotmail, yahoo, aol, verizon, and earthlink yes. i don't think any administrator with paying customers to please would be happy if any of these were blacklisted. ... but who not to whitelist? the small guys. unfortunately, large ISPs like that have power in the number of users they have. in no way do I advocate defending that as a good thing, but the fact that this gives them an immense amount of power to do whatever they want regarding rfcs and whatnot remains a reality. smaller services are the only organizations who are going to actually be potentially moved to action by landing on one of these RBLs. when was the last time SORBS managed to change Hotmail's policies? using something as strict as a RBL that lists Hotmail can only be useful for scoring but not as an outright block. I really don't think people who regularly correspond or who have to support ppl who correspond with hotmail users would argue with that. Sounds like you aren't one of those ppl. And, what if half of your user/customer base does NOT want you to white list aol but c'mon, when was the last time someone's user base was emailing their support staff begging for aol to be blacklisted? beside, this is what per-user settings for something like SA are for. does want you to whitelist hotmail ... while the other half of your base is exactly the opposite? It isn't a solvable problem, IMO. Everyone will want to draw the line differently, so there wont be an easy solution of that nature. But BLOCKING all mails from somewhere like Hotmail *IS* a decision that someone has made which is not acceptable to we who support large user bases. So we have to make the opposite decision to only use those RBLs in SA scoring. The baseline here is that you cannot outright ban whole large services -- you HAVE to work from there, meaning that then if stuff doesn't score where your users like it, they have to adjust their own SA settings (ours do it on their own through a SquirrelMail interface). And, it's not just that I don't think the RBL can do it, I don't think that kind of thing is the job of the RBL. I think that kind of thing is your job (or, in my case, it's my job). What's our job? Banning all of Hotmail? Our job is to avoid that - it's obviously not workable at least for those in a position like the one I've described. So we have to stop using SORBS at the outset. And I'm pretty sad to do it, because so far it has been one of the best front-line defenses we've had. In general I think they are great, but this hotmail thing is NOT workable in our situation, and probably in many others Or are you saying I should sit around all day and monitor ever-changing lists of potential spammer IPs and manually adjust our MTA white/black lists? That's not exactly realistic, so I'm not sure what you are suggesting (I think I am about to find out...) Here at UCSC, we use spamhaus (both SBL and XBL). In order to make sure my own users/customers don't get blacklisted, I have a cron job that: a) use rsync to get a local copy of the zones. b) grep the files to notify me if any of my own addresses are listed, so that I can follow up on why. c) grep -v the files to remove any of those addresses from the zone. d) takes the end result and puts it into a place where my name servers will pick it up. (I'm also trying to get this for SURBL and RFC-Ignorant, but SURBL is taking some time, and RFC-I is unresponsive to my requests) Don't get me wrong, I am fully supportive of the people taking their time to run those services (where would we be without them), but their general lack of responsiveness seems strange -- no matter which service it is, I always hear people say things about how non-responsive they are. Is it that they can't manage to parse through the number of insulting inquiries they get from the legit ones? Are these people that overworked? Seems like being more responsive, even if to just tell spam-friendly ISPs to take a hike, would give them more credibility. SPEWS seems to be the most common target of this criticism, but I've heard it for SORBS, etc too If I wanted to be sure that hotmail didn't get in there, I would add their to the grep -v expression (or pipe it through another layer of grep -v). If a host gets listed that my users need to hear from, then they can notify me, and I'll
DNS cache size for moderatly busy sites?
Hello, We just migrated to Tinydns from BIND and are looking at our cache size (OK, so I am really talking about dnscache, not tinydns itself). Looking at our cache logs from the last 12 hours (2am Friday night to 2pm Saturday afternoon), I see our cache motion is already 75MB of data. Wow. That's in a relatively low activity time for us. We get an average of somewhere under 100,000 mails a day. I am curious what other people's cache sizes are set to. If the numbers we are seeing hold up (especially during peak), and if we wanted to cache 3 days worth of DNS queries, it seems like we'd need something like a 500MB+ cache size. Is it me, or does that seem rather large? I wonder how efficient dnscache would be at that size anyway... Thanks for any tips! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: ANNOUNCE: SpamAssassin 3.1.0-rc1 release candidate available!
bump. can anyone answer at least the questions about dcc and razor? Exellent. This is the information I needed! Is there any chance of getting an updated release schedule (I checked the wiki, but the schedule info for 3.1.0 seems out of date)? Might also be nice to see some pointers in the docs about how to reenable the DCC and Razor plugins for those of us who will continue to use those tools. Is having use_dcc and use_razor2 in our local.cf set to one (instead of relying on the default which has now changed) what you mean by trivial? Thanks! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
RE: Help with RR DNS for spamd?
Telnet uses the built-in resolver -- most ordinary applications work this way. Hrm. Any tips on how to make it aware of my new spam. subdomain? If a application uses a resolver (it's own or the built in resovler) that points solely to a DNS server (set) WITH that zone, or that can/will FIND that zone then it has to work if the record is created. I dunno, but maybe you were still talking about the Windoze world. I tried telnet from the machine that DNS is running on and it resolved correctly if the first entry in /etc/resolve.conf was 127.0.0.1. That told me that the faulty link was also probably /etc/resolve.conf on the SMTP machine. So I went back to that machine and fiddled some more with /etc/resolve.conf. Turns out for some reason it did not like hostnames therein, but once I put the local network address for the machine running DNS as the first entry, my telnets began to work. SpamAssassin (not sure about spamc which is compiled) uses the resolver in Net::DNS which is configurable to it's own (perhaps different than the computers own defaults) DNS server -- usually as an Environment variable or as a config setting. Once my /etc/resolve.conf was corrected, spamc started correctly hitting both of my spamd servers set up in BIND. That is really cool. Now too bad BIND does not support weighted RR as does tinydns next step is to wave goodbye to BIND. Ugh. Thanks anyway. Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
BIND with forward first as caching DNS?
Hello,
I'm no DNS expert, so am wondering if I am shooting myself in the foot by
having forwarders set up in my BIND config file, especially with forward
first:
options {
directory /var/named;
forward first;
forwarders {
xxx.xxx.x.x
yyy.yyy.y.y
};
};
Where xxx and yyy are the DNS servers for my colo provider where I host the
system in question. Does this defeat the purpose of local caching or am I
OK?
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
RE: BIND with forward first as caching DNS?
--- Herb Martin [EMAIL PROTECTED] wrote: I'm no DNS expert, so am wondering if I am shooting myself in the foot by having forwarders set up in my BIND config file, especially with forward first: Where xxx and yyy are the DNS servers for my colo provider where I host the system in question. Does this defeat the purpose of local caching or am I OK? No. Resolution by forwarders is also cached by the requesting (forwarding) DNS server. It does expose you to any corruption (e.g., cache polution) of your colocator's DNS -- but if you trust them to do as good or better job of running DNS securely (than you can do) then that probably doesn't matter. (You did say you are not an expert.) Thank you! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
Help with RR DNS for spamd?
All, I recognize this is a bit OT, but not sure where to turn...? I am trying to use DNS to load balance a couple spamd servers. I am attempting to return more than one A record for spam.mydomain.com, and I am getting a correct dig, but just a test telnet is returning Unknown host. What am I doing wrong? Few more details: DNS hosted on a separate machine, say dns.mydomain.com. Our SMTP server is where the spamc calls out to spamd, and that machine's /etc/resolve has the dns machine's hostname as the first nameserver entry. From the SMTP machine, I do dig spam.mydomain.com and I get my desired results: ;; ANSWER SECTION: spam.mydomain.com. 259200 IN A 10.10.10.105 spam.mydomain.com. 259200 IN A 10.10.10.106 I was pretty excited when I got that far, but then the next step - to try to manually connect to one of the spamd daemons listening on those 10. addresses - failed miserably: # telnet spam.mydomain.com 2009 telnet: spam.mydomain.com: Name or service not known spam.mydomain.com: Unknown host # (the port number above is what spamd is configured to listen on) Seems like the DNS server is not the problem, but FWIW, the zone file for mydomain.com has these entries: spamA 10.10.10.105 spamA 10.10.10.106 Help?!??! __ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail
RE: BIND with forward first as caching DNS?
Herb, this is just FYI. I am *NOT* sending from a fake Yahoo server - this mail was legit, so seems like your server is being a little over-zealous? --- [EMAIL PROTECTED] wrote: Date: 17 Aug 2005 23:16:08 - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: failure notice Hi. This is the qmail-send program at yahoo.com. I'm afraid I wasn't able to deliver your message to the following addresses. This is a permanent error; I've given up. Sorry it didn't work out. [EMAIL PROTECTED]: 68.178.144.61 does not like recipient. Remote host said: 550 Fake Yahoo mail Giving up on 68.178.144.61. --- Below this line is a copy of the message. Return-Path: [EMAIL PROTECTED] Received: (qmail 69465 invoked by uid 60001); 17 Aug 2005 23:16:02 - DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kFeUKnrDxm4Y+XJNGAjmiKk5ZWlKlRIwiDc4zVNhgR4CyXMc/1LVYUdp+By5RVeAggd2+s0RB2WJIbrG+yE8PxHHW+1BqYEtK+MMxJUkTh49JFhGn0NEWiKgHcDmqS06AYxSsU3U+itOkbDn+2aLfIkMKzRdoPfAztHWnEMdiIQ= ; Message-ID: [EMAIL PROTECTED] Received: from [64.171.185.165] by web51909.mail.yahoo.com via HTTP; Wed, 17 Aug 2005 16:16:02 PDT Date: Wed, 17 Aug 2005 16:16:02 -0700 (PDT) From: email builder [EMAIL PROTECTED] Subject: RE: BIND with forward first as caching DNS? To: Herb Martin [EMAIL PROTECTED], users@spamassassin.apache.org In-Reply-To: [EMAIL PROTECTED] MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit --- Herb Martin [EMAIL PROTECTED] wrote: I'm no DNS expert, so am wondering if I am shooting myself in the foot by having forwarders set up in my BIND config file, especially with forward first: Where xxx and yyy are the DNS servers for my colo provider where I host the system in question. Does this defeat the purpose of local caching or am I OK? No. Resolution by forwarders is also cached by the requesting (forwarding) DNS server. It does expose you to any corruption (e.g., cache polution) of your colocator's DNS -- but if you trust them to do as good or better job of running DNS securely (than you can do) then that probably doesn't matter. (You did say you are not an expert.) Thank you! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
RE: Help with RR DNS for spamd?
I recognize this is a bit OT, but not sure where to turn...? I am trying to use DNS to load balance a couple spamd servers. I am attempting to return more than one A record for spam.mydomain.com, and I am getting a correct dig, but just a test telnet is returning Unknown host. What am I doing wrong? If it only lasts 5 minutes (or less) then your previous (before creating the records) would typically be cache by a modern Windows CLIENT. While DNS has long been cached at the server, Microsoft started caching at the client (by default) in Win2000. This is all being done in linux, no Windoze involved, thank god. Anyway, the problem has lasted much longer than 5 minutes. :) Few more details: DNS hosted on a separate machine, say dns.mydomain.com. Separate machines? Do you mean different DNS servers or just separate from the SMTP/spamd machine? Just that Bind is running on a different machine than is SMTP and spamc. They are both on the same local network. If your spam.mydomain.com is not on the same machine as the mydomain.com zone then you would (likely) need to delegate (but that doesn't seem to be your issue.) Not sure I follow, but just to clarify a bit, the spamd, SMTP(spamc) and DNS machines are all separate machines but in the same local network. I don't even think it matters how/if/what/why about the connection to the outside Internet since I am just trying to resolve locally. Our SMTP server is where the spamc calls out to spamd, and that machine's /etc/resolve has the dns machine's hostname as the first nameserver entry. From the SMTP machine, I do dig spam.mydomain.com and I get my desired results: You cannot depend on first DNS setting on a client -- most DNS clients may try ANY of the one's listed -- Windows clients for instance certainly work this way. OK, fair enough (although from my experience, dig always seems to take the first one it finds that works). So I took all other entries out of /etc/resolve.conf on the SMTP(spamc) machine, but still getting the same results: dig works great, but telnet bails. Again, I am using telnet from a linux command prompt on the SMTP box -- is there a telnet host cache for linux?? A DNS client typically expects EVERY DNS Server to return the SAME answers (although a resolver/OS could be constructed to keep trying this is not typical.) ;; ANSWER SECTION: spam.mydomain.com. 259200 IN A 10.10.10.105 spam.mydomain.com. 259200 IN A 10.10.10.106 I was pretty excited when I got that far, but then the next step - to try to manually connect to one of the spamd daemons listening on those 10. addresses - failed miserably: # telnet spam.mydomain.com 2009 telnet: spam.mydomain.com: Name or service not known spam.mydomain.com: Unknown host # (the port number above is what spamd is configured to listen on) Likely you have more than one DNS server listed on the telnet client and the client resolver is using the wrong one. Linux telnet has this kind of thing? Where is it?? I thought that is what /etc/resolve.conf is. You should generally point clients to ONE CONSISTENT (set of) DNS servers which return all the correct answers the client will ever need. If the DNS server (set) doesn't know the answer it must forward or recurse to find it. Well, since the ultimate client will be spamc, what does spamc use? Something other than /etc/resolve.conf? Also, will my local IP addresses in my zone file get propagated to other DNS servers? (If what you say about consistency is important, and I put more than one nameserver in /etc/resolve.conf, I will want to make sure they propagate to the other DNS servers). Thanks, Herb. Seems like the DNS server is not the problem, but FWIW, the zone file for mydomain.com has these entries: spamA 10.10.10.105 spamA 10.10.10.106 -- Herb Martin Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
RE: Help with RR DNS for spamd?
You cannot depend on first DNS setting on a client -- most DNS clients may try ANY of the one's listed -- Windows clients for instance certainly work this way. OK, fair enough (although from my experience, dig always seems to take the first one it finds that works). So I took all other entries out of /etc/resolve.conf on the SMTP(spamc) Dig has/is it's own resolver if I recall correctly. Linux telnet has this kind of thing? Where is it?? I thought that is what /etc/resolve.conf is. Telnet uses the built-in resolver -- most ordinary applications work this way. Hrm. Any tips on how to make it aware of my new spam. subdomain? You should generally point clients to ONE CONSISTENT (set of) DNS servers which return all the correct answers the client will ever need. If the DNS server (set) doesn't know the answer it must forward or recurse to find it. Well, since the ultimate client will be spamc, what does spamc use? Something other than /etc/resolve.conf? Most (almost all) regular applications use the built-in resolver but IIRC SpamC has this as a configuartion/ environemnt setting so it (this is true for SpamAssassin and Net::DNS actually) might be using a different setting for DNS than the computer as a whole. OK, so instead of mucking around with telnet, I tried it with spamc, but no dice. Maillog shows: Aug 17 18:35:40 gaia spamc[27064]: gethostbyname(spam.mydomain.com) failed: h_errno=1 Aug 17 18:35:40 gaia spamc[27097]: gethostbyname(spam.mydomain.com) failed: h_errno=1 Aug 17 18:35:41 gaia spamc[27143]: gethostbyname(spam.mydomain.com) failed: h_errno=1 Aug 17 18:35:41 gaia spamc[27144]: gethostbyname(spam.mydomain.com) failed: h_errno=1 ... and so on... Thanks so much! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: ANNOUNCE: SpamAssassin 3.1.0-rc1 release candidate available!
Exellent. This is the information I needed! Is there any chance of getting an updated release schedule (I checked the wiki, but the schedule info for 3.1.0 seems out of date)? Might also be nice to see some pointers in the docs about how to reenable the DCC and Razor plugins for those of us who will continue to use those tools. Is having use_dcc and use_razor2 in our local.cf set to one (instead of relying on the default which has now changed) what you mean by trivial? Thanks! - - added PostgreSQL, MySQL 4.1+, and local SDBM file Bayes storage modules. SQL storage is now recommended for Bayes, instead of DB_File. NDBM_File support has been dropped due to a major bug in that module. What's the difference between the MySQL support that already existed in prior versions? Is there anything those of us who already have our bayes data in MySQL should do differently as of 3.1.0? The previous SQL support (Mail::SpamAssassin::BayesStore::SQL) was very generic, usable by multiple database drivers. With 3.1.0 we broke out the support and now include 2 very specific SQL backends (MySQL 4.1+ and PostgreSQL) in addition to the more generic backend. These specific backends make use of non-standard SQL features to get a speed boost. That said, if you were previously using SQL support with a MySQL database then you should be able to simply switch to using Mail::SpamAssassin::BayesStore::MySQL and get an instant speedup, assuming you already have MySQL 4.1+ installed. We do suggest that you switch your tables to InnoDB type tables (not currently the default) to get better data integrity (with the support of transactions). If you were using PostgreSQL with the previous support, you should switch (we're talking about a 7x - 27x improvement) ASAP, which might involve a complete wipe and rebuild of your database. Although, I would try an sa-learn --backup and sa-learn --restore before I completely gave up on the data. If you are interested in how well the various backends perform, compared to the others, see http://wiki.apache.org/spamassassin/BayesBenchmarkResults It is very hard to compare to previous versions, due to changes in other factors such as rules and message parsing code, but the improvments in 3.1 represent anywhere from a 2x - 27x improvements in previous performance. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Manual bayes expiration in MySQL database
Don't expire things manually. 1. Why not? 2. On a Bayes SQL setup with multiple servers feeding/reading the db, should one server be responsible for expiration or should each opportunistically take care of it? I'll be more specific, don't expire things by doing the SQL commands yourself. It is fine to expire manually by running sa-learn --force-expire. Default auto_expire setting is 1, is it not? Why do these other people cron sa-learn to manually exprire then? What advantage does that have over letting SA do it opportunistically (unless perhaps your server is NEVER not busy?)?? Does sa-learn --force-expire need to be executed within a username context (-u option unless you run it as the right user), or does it not care about users? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
RE: Load balancing spamd
Bump. :) Gary, please share how you do this! Thanks! How do you (make and) balance the calls to the AV servers? How do you (make and) balance the calls to the spamd machines? I am very interested in these details! We just call them in order case on the connection line. On two of the 4 SMTP gateways we use node 1 as the primary and node 2 as the secondary, on the other two, just the opposite. I know this is the poor mans way of doing this but we are lazy and haven't made our way to using something like LVS. Please show how you do this. :) Please! :) For example, are you calling your AV backend with Postfix's content_filter setting? I'm not sure if/how it supports more than one host? Here is a simple one: content_filter = amavis:[123.456.7.8]:10024 How do you point it to more than one place? Then for SA, are you using spamc and spamd with -d and -H options to use DNS-based round robin load balancing? Is the spamc in something like a global maildrop filter? How are you doing these things? I presume you are not using weighted load balancing? We are edging up to 95K a day now on only two machines. You can imagine we are anxious to start using the other boxes we have rarin' to go! Ironically, when we first started this we had everything running on 4 machines and it started choking. So, we went with the two backend ends. It chocked. Then we kicked the -m from 30 to 6. 6 is a small number but it seems to be working fine. We have found for our environment that 6 to 8 works well. I've seen the same thing. We started with a dedicated SA box and set it to 20 children and it just choked. It is not a slow box, either. There were comments on another thread a day ago that dedicated boxes can handle that many children, but our experience is that SA hums along much better at around the default, even on a beefy dedicated box. We recently upgrade all of the hardware to Dell Dimension 4700's with 1.5gb ram each. Budget was $5200. Machines are idle. Sweet. ;) And it was overall cheap Why? Because your DNS costs to query your RBL list in Postfix is very heavy/slowing you down? Are you going to mirror just one chosen RBL out there or a combination of several?? Do you run DCC in your SA environment? If so, you are over their recommended limit for hosting a DCC server (we are nearing it - 100K a day I think). Do you run a DCC server for yourself? Any issues to be aware of? It's on the TODO list. Item 629 I believe... :) There are other pressing items to fix/work on. This is working great but will be readdressed during the next maintenance upgrade (which is about every 90 days). Please elaborate on your RBL plans (and why you decided to do it). Thanks a TON! __ Yahoo! Mail for Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail
RE: Load balancing spamd
How do you (make and) balance the calls to the AV servers? How do you (make and) balance the calls to the spamd machines? I am very interested in these details! We just call them in order case on the connection line. On two of the 4 SMTP gateways we use node 1 as the primary and node 2 as the secondary, on the other two, just the opposite. I know this is the poor mans way of doing this but we are lazy and haven't made our way to using something like LVS. Please show how you do this. :) Please! :) For example, are you calling your AV backend with Postfix's content_filter setting? I'm not sure if/how it supports more than one host? Here is a simple one: content_filter = amavis:[123.456.7.8]:10024 How do you point it to more than one place? Then for SA, are you using spamc and spamd with -d and -H options to use DNS-based round robin load balancing? Is the spamc in something like a global maildrop filter? How are you doing these things? I presume you are not using weighted load balancing? We are edging up to 95K a day now on only two machines. You can imagine we are anxious to start using the other boxes we have rarin' to go! Ironically, when we first started this we had everything running on 4 machines and it started choking. So, we went with the two backend ends. It chocked. Then we kicked the -m from 30 to 6. 6 is a small number but it seems to be working fine. We have found for our environment that 6 to 8 works well. I've seen the same thing. We started with a dedicated SA box and set it to 20 children and it just choked. It is not a slow box, either. There were comments on another thread a day ago that dedicated boxes can handle that many children, but our experience is that SA hums along much better at around the default, even on a beefy dedicated box. We recently upgrade all of the hardware to Dell Dimension 4700's with 1.5gb ram each. Budget was $5200. Machines are idle. Sweet. ;) And it was overall cheap Why? Because your DNS costs to query your RBL list in Postfix is very heavy/slowing you down? Are you going to mirror just one chosen RBL out there or a combination of several?? Do you run DCC in your SA environment? If so, you are over their recommended limit for hosting a DCC server (we are nearing it - 100K a day I think). Do you run a DCC server for yourself? Any issues to be aware of? It's on the TODO list. Item 629 I believe... :) There are other pressing items to fix/work on. This is working great but will be readdressed during the next maintenance upgrade (which is about every 90 days). Please elaborate on your RBL plans (and why you decided to do it). Thanks a TON! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Load balancing spamd
--- Jason Frisvold [EMAIL PROTECTED] wrote: On 8/1/05, email builder [EMAIL PROTECTED] wrote: Even if I had forgotten the -A, I think I would have been seeing connection refused notices, but right now, it just seems to time out. I'm pretty sure this is a LVS question more than a spamc/d question, since I've no problems with the latter -- I am only asking here to see if anyone else does SA weighted load balancing. I kinda went the other way around.. I have multiple mail machines, each with their own instance of spamd. I use a Cisco 7206 VXR to do the load balancing. Works like a charm. Wow, a bit out of our price range here. :) We have also considered just continuing to build out MTA boxes each with an Amavis/Clamd and SA on them to share our increasing load (just use LVS to balance the incoming SMTP traffic and there is little reason to worry about balancing SA or Amavis/Clam), but our first choice is to split the layers -- have a couple separate machines that just do MTA-ish things, and a separate set of boxes that serve as a SA (and Clam-av) farm. The thing that's better about doing it that way is the redundancy that you don't get if you aren't sharing spamd instances across all your MTA machines. Technically, this should be feasible with just plain DNS load balancing, but in our current medium/low budget scenario, we don't have the rackspace to have numerous boxes that are dedicated ONLY to SA/clam, thus our desire is to figure out a way to *WEIGHT* our spamd balancing. I'm surprised there's not a lot of folks out there who have done this before? Thanks again! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
Re: Load balancing spamd
--- Charles Sprickman [EMAIL PROTECTED] wrote: On Tue, 2 Aug 2005, email builder wrote: Technically, this should be feasible with just plain DNS load balancing, but in our current medium/low budget scenario, we don't have the rackspace to have numerous boxes that are dedicated ONLY to SA/clam, thus our desire is to figure out a way to *WEIGHT* our spamd balancing. I've been very happy with DNS load balancing. The frontend mxer runs tinydns on a local zone blah.local.domain.com, and an instance of dnscache with the round-robin patch is pointed to in resolv.conf. While I thought that the load balancing would be a little rough, looking at the stats I sent 17011 messages through #1, 17025 through #2, and 17016 through #3 yesterday. I can also weight this by having multiple records, ie: spamd1 gets three identical entries in tinydns spamd2 gets three identical entries in tinydns spamd3 gets three identical entries in tinydns spamd4 gets one entry O, some good bits! We have always been plenty satisfied with Bind, but maybe this is the straw that broke the camel's back unless anyone knows if Bind will behave the same way if we have multiple entries for one host?? that will leave spamd4 seeing about 1/3 the load of the other boxes. It's not clustering, but when using the -d flag: -d host Connect to spamd server on given host. If host resolves to multi- ple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to. it should hit another box if one goes down. Or some easy scripting could remove the appropriate entries from tinydns if one machine stops responding. Speaking of low budget, we have three SA boxes, each of which has a 2GHz AMD processor, 1GB RAM. The first two cost about $550, the last one about $425. They are pretty crappy boxes with no RAID, etc., but it's cheaper for me to keep one more box than needed in the equation than to build out a few uber spamd boxes. They are in mini-atx cases, so they barely take up more room than an equivalent number of 1U boxes. I spawn 30 spamd children on each. I have been very happy with the performance so far. I'm surprised there's not a lot of folks out there who have done this before? Maybe they're all cheap like me. :) Awesome! Thanks for the advice!!! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
RE: Load balancing spamd
--- Gary W. Smith [EMAIL PROTECTED] wrote: We have 4 front end servers running postfix. These servers call and AV process on two additional AV servers behind the wall. Then these servers these being the AV server calls spamd or it goes back to the MTA first? How do you (make and) balance the calls to the AV servers? How do you (make and) balance the calls to the spamd machines? I am very interested in these details! call spamd on two additional servers behind the wall. Those two servers have a simple MySQL cluster (running Linux-HA and DRBD). In all we have 8 boxes that handle all of our email for our clients. We are generating about 170k emails per day coming into the network. We are edging up to 95K a day now on only two machines. You can imagine we are anxious to start using the other boxes we have rarin' to go! We recently upgrade all of the hardware to Dell Dimension 4700's with 1.5gb ram each. Budget was $5200. Machines are idle. Sweet. ;) Something new we have been looking at as well. We are looking at setting up simple relays that will run RBL on the front end and then just hand them off to our 4 backend servers. But since it works right now we're not going to fix it. Why? Because your DNS costs to query your RBL list in Postfix is very heavy/slowing you down? Are you going to mirror just one chosen RBL out there or a combination of several?? Do you run DCC in your SA environment? If so, you are over their recommended limit for hosting a DCC server (we are nearing it - 100K a day I think). Do you run a DCC server for yourself? Any issues to be aware of? Thanks a TON!! -Original Message- From: email builder [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 02, 2005 5:19 PM To: Jason Frisvold Cc: Gary W. Smith; users@spamassassin.apache.org Subject: Re: Load balancing spamd --- Jason Frisvold [EMAIL PROTECTED] wrote: On 8/1/05, email builder [EMAIL PROTECTED] wrote: Even if I had forgotten the -A, I think I would have been seeing connection refused notices, but right now, it just seems to time out. I'm pretty sure this is a LVS question more than a spamc/d question, since I've no problems with the latter -- I am only asking here to see if anyone else does SA weighted load balancing. I kinda went the other way around.. I have multiple mail machines, each with their own instance of spamd. I use a Cisco 7206 VXR to do the load balancing. Works like a charm. Wow, a bit out of our price range here. :) We have also considered just continuing to build out MTA boxes each with an Amavis/Clamd and SA on them to share our increasing load (just use LVS to balance the incoming SMTP traffic and there is little reason to worry about balancing SA or Amavis/Clam), but our first choice is to split the layers -- have a couple separate machines that just do MTA-ish things, and a separate set of boxes that serve as a SA (and Clam-av) farm. The thing that's better about doing it that way is the redundancy that you don't get if you aren't sharing spamd instances across all your MTA machines. Technically, this should be feasible with just plain DNS load balancing, but in our current medium/low budget scenario, we don't have the rackspace to have numerous boxes that are dedicated ONLY to SA/clam, thus our desire is to figure out a way to *WEIGHT* our spamd balancing. I'm surprised there's not a lot of folks out there who have done this before? Thanks again! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs __ Yahoo! Mail for Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail
RE: Load balancing spamd
Do you happen to have any firewall rules in place on the LVS instance? Have you specified which IP's are allowed to access the instance? As best I can tell, we have no firewall restrictions blocking intranet packets at all. Both of the above are what I ran into on the default RH build (even though I don't run LVS). spamd -s local5 -d -c -m10 -H -A 10.0.8.0/21 I believe without the -A and IP range the machine will only answer to localhost. This is more than likely your problem since I don't see you mentioning even playing with that. Oh, no, I didn't mean to give that impression. I am fully ready to take such connections as far as I know: /usr/bin/spamd -d -q -x --max-children=5 -H /etc/razor -u maildrop -r /var/run/spamd/spamd.pid -i 10.10.10.170 -p 2054 -A 10.10. Even if I had forgotten the -A, I think I would have been seeing connection refused notices, but right now, it just seems to time out. I'm pretty sure this is a LVS question more than a spamc/d question, since I've no problems with the latter -- I am only asking here to see if anyone else does SA weighted load balancing. Thanks! From: email builder [mailto:[EMAIL PROTECTED] Sent: Monday, August 01, 2005 2:43 PM To: users@spamassassin.apache.org Subject: Load balancing spamd Hi, I am looking for advice on how to load balance spamd servers. I (think I) understand that the -d option used with -H for spamc will randomize multiple addresses from a DNS lookup of the given hostname (and still include failover support???). However, I am wanting to do weighted load balancing ala something more substantial like LVS' ldirector. I am very much a newb to LVS in general, but have it installed (ultramonkey.org) and working for HTTP from the outside world to two different Apache boxes. But there seems to be a difference between balancing requests that come from external interfaces and requests that are completely internal. That is, I point my MTA to connect to a spamd port on the ldirector box, make the appropriate settings in ldirector, but the connection doesn't even seem to happen at all. Do I need to run another instance of ldirector on an internal interface somehow? How are other people doing this? TIA! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
Re: DNS failing... why? (works fine on cmd line)
All, Thank you to everyone who replied on this thread. FWIW, the issue was in fact with Net::DNS. I actually had previously had contact with him regarding other problems, but 0.51 was working for me on another system, so I was a little surprised that this was the fix. I upgraded to the newest (0.53) and the problem has gone away. Thanks everyone! email builder [EMAIL PROTECTED] wrote: I have a new spamd instance I am trying to start up on a server that sitsbehind another firewall (linux) machine (which I *think* is irrelevant, butthat's the only different thing from our other setups that work fine) that issomehow missing DNS connections:'''debug: is Net::DNS::Resolver available? yesdebug: Net::DNS version: 0.51debug: trying (3) motorola.com...debug: looking up NS for 'motorola.com'debug: NS lookup of motorola.com failed horribly = Perhaps your resolv.confisn't pointing at a valid server?debug: All NS queries failed = DNS unavailable (set dns_available tooverride)debug: is DNS available? 0'''However, when I telnet to port 53 of one of the IP addresses given in/etc/resolv.conf, it works just fine:'''[EMAIL PROTECTED] cat /etc/resolv.conf nameserver 123.456.7.8nameserver 987.654.1.1[EMAIL PROTECTED] telnet 123.456.7.8 53Trying 123.456.7.8...Connected to 123.456.7.8.xxx.yyy.net (123.456.7.8).Escape character is '^]'.quitConnection closed by foreign host.'''So, is spamd trying to dig the NS of motorola.com? That works on the commandline too:'''[EMAIL PROTECTED] dig ns motorola.com; DiG 9.2.5 ns motorola.com;; global options: printcmd;; Got answer:;; -HEADER- opcode: QUERY, status: NOERROR, id: 24784;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0;; QUESTION SECTION:;motorola.com. IN NS;; ANSWER SECTION:motorola.com. 3594 IN NS motgate.mot.com.motorola.com. 3594 IN NS ftpbox.mot.com.motorola.com. 3594 IN NS dns31.mot.com.motorola.com. 3594 IN NS dns11.mot.com.motorola.com. 3594 IN NS motgate.motorola.de.;; Query time: 3 msec;; SERVER: 123.456.7.8#53(123.456.7.8);; WHEN: Tue Jul 19 13:14:17 2005;; MSG SIZE rcvd: 150'''So does this mean that it's actually an issue with Net::DNS orNet::DNS::Resolver? They are about as up to date as they get I think(Net::DNS .52 is out now, but I don't really think that's going to fixit...?).What should I look at next? What is spamd doing that I am not doing on thecommand line???TIA!Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs Start your day with Yahoo! - make it your home page
DNS failing... why? (works fine on cmd line)
I have a new spamd instance I am trying to start up on a server that sits behind another firewall (linux) machine (which I *think* is irrelevant, but that's the only different thing from our other setups that work fine) that is somehow missing DNS connections: ''' debug: is Net::DNS::Resolver available? yes debug: Net::DNS version: 0.51 debug: trying (3) motorola.com... debug: looking up NS for 'motorola.com' debug: NS lookup of motorola.com failed horribly = Perhaps your resolv.conf isn't pointing at a valid server? debug: All NS queries failed = DNS unavailable (set dns_available to override) debug: is DNS available? 0 ''' However, when I telnet to port 53 of one of the IP addresses given in /etc/resolv.conf, it works just fine: ''' [EMAIL PROTECTED] cat /etc/resolv.conf nameserver 123.456.7.8 nameserver 987.654.1.1 [EMAIL PROTECTED] telnet 123.456.7.8 53 Trying 123.456.7.8... Connected to 123.456.7.8.xxx.yyy.net (123.456.7.8). Escape character is '^]'. quit Connection closed by foreign host. ''' So, is spamd trying to dig the NS of motorola.com? That works on the command line too: ''' [EMAIL PROTECTED] dig ns motorola.com ; DiG 9.2.5 ns motorola.com ;; global options: printcmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 24784 ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;motorola.com. IN NS ;; ANSWER SECTION: motorola.com. 3594IN NS motgate.mot.com. motorola.com. 3594IN NS ftpbox.mot.com. motorola.com. 3594IN NS dns31.mot.com. motorola.com. 3594IN NS dns11.mot.com. motorola.com. 3594IN NS motgate.motorola.de. ;; Query time: 3 msec ;; SERVER: 123.456.7.8#53(123.456.7.8) ;; WHEN: Tue Jul 19 13:14:17 2005 ;; MSG SIZE rcvd: 150 ''' So does this mean that it's actually an issue with Net::DNS or Net::DNS::Resolver? They are about as up to date as they get I think (Net::DNS .52 is out now, but I don't really think that's going to fix it...?). What should I look at next? What is spamd doing that I am not doing on the command line??? TIA! Start your day with Yahoo! - make it your home page http://www.yahoo.com/r/hs
Help debugging spamc/spamd
Hi, We recently changed some of our network topology so that we are temporarily connecting with spamc to spamd over a regular external network connection (we usually keep it inside our LAN, but this is a temporary thing... don't ask). Unfortunately, spamd stops (mostly) responding it seems. I can watch spamc sitting and waiting on the MTA by using ps ax | grep spam but I don't see anything happening on the spamd server except for once every 15 minutes or so, a few messages will process (there are hundreds a minute to process). I'm not sure where it would be choking. I ran spamd in the foreground (-D), painstakingly read all the debug info for a couple messages, and nothing looked bad. When messages DID scan, they took no more than a second or two, so I don't think there are DNS issues, but I don't know where else to look. Things just seem to stop processing suddenly; I don't get it. Anyone have hints? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: Net::DNS problem?
All, I also ran into this problem: 0.51 has already been released that addresses the overlooked debug statement (http://www.net-dns.org/).I still get failures in the 11-escapedchars.t test under Solaris-8/Perl-5.8.6 though. I contacted the author and he said it's fixed in SVN: I fixed this bug about 2 days ago. If you need it quickly you can use the SVN repository. svn co http://www.net-dns/svn/net-dns/trunk I plan do post a developers release this week. 0.51_02 that will contain the fix. Yahoo! Sports Rekindle the Rivalries. Sign up for Fantasy Football http://football.fantasysports.yahoo.com
Re: What is a caching name server?
in several posts I have noticed people refer to a caching nameserver. What exactly is that? Would BIND 9.3.1 qualify? Any advice would be greatly appreciated. yes Bind will become a caching only name server if you don;t have any local zone files to lookup. Basically think of it as a proxy with memory. It will remember previous look ups so it won't ask it's resolvers again (unless the timeout value on the record has been reached). Really? 1) why would Bind NOT cache domain lookups for domains that are not listed in your local zone files? that seems rediculous. is there any way to host your company's domains in a Bind instance that also caches lookups? 2) is there a way to test a Bind server to make sure it is in fact caching its lookups? __ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/
sql-based global use_auto_whitelist error?
Hi,
I have a global setting in my SQL-based userprefs for use_auto_whitelist.
I am noticing that spamd -D shows these messages about it, which confuse me:
debug: retrieving prefs for [EMAIL PROTECTED] from SQL server
debug: config: not parsing, administrator setting: use_auto_whitelist 1
debug: config: SpamAssassin failed to parse line, skipping:
use_auto_whitelist 1
debug: user has changed
What's going on here? Same thing when I set it to zero. Obviously, it
seems to be querying the DB correctly, but barfing on what it finds. Why? I
also have use_bayes set up the same way, which is working just fine.
my version: SA 3.0.2
mysql select * from spamassassin_user_settings;
+---+++
| username | preference | value
|
+---+++
| !GLOBAL | use_bayes | 1
|
| !GLOBAL | use_auto_whitelist | 1
|
+---+++
in local.cf:
user_scores_sql_custom_querySELECT preference, value FROM
spamassassin_user_settings WHERE username = _USERNAME_ OR username =
'!GLOBAL' OR username = CONCAT('@', _DOMAIN_) ORDER BY username ASC
__
Celebrate Yahoo!'s 10th Birthday!
Yahoo! Netrospective: 100 Moments of the Web
http://birthday.yahoo.com/netrospective/
multiple hosts for spamc -d ?
All, Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? TIA! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: multiple hosts for spamc -d ?
Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? TIA! From http://spamassassin.apache.org/full/3.0.x/dist/doc/spamc.html -d host In TCP/IP mode, connect to spamd server on given host (default: localhost). If host resolves to multiple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to You need to have a host that has multiple A records. spamd.domain.com A 123.123.123.123 spamd.domain.com A 123.123.123.124 spamd.domain.com A 123.123.123.125 /usr/bin/spamc -d spamd.domain.com If your DNS server sends the results back in a different order each time then it will not be a fallback but a round robin. You might be able to simply use /etc/host entries. I've never tried it as I use qmail which will not use the host file, so I always rely on DNS. Don't know if spamc will use the host file or not. Huh, I am not familiar with how to use /etc/hosts as a DNS source. Can you clarify? Mainly my question was if/how I could avoid making it a DNS query. I'd like to simply hand spamc the two addresses that I want it to have manually, and I do *NOT* want round-robin, I want failover Your help is much appreciated! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: multiple hosts for spamc -d ?
Some postings a while back led me to believe that I could specify multiple hosts for the -d option of spamc. I understood that it would operate basically on a fallback basis (not load balancing). However, I can't seem to get spamc to use more than one of the -d listings. I've tried: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 /usr/bin/spamc -d 123.45.67.8 127.0.0.1 And switched the order around and fiddled with hostnames vs IP addresses, but no dice. I understand the man page to say that it will use fallback logic if the hostname resolves (via DNS query, right?) to more than one host... so why can't I give it those hosts directly? From http://spamassassin.apache.org/full/3.0.x/dist/doc/spamc.html -d host In TCP/IP mode, connect to spamd server on given host (default: localhost). If host resolves to multiple addresses, then spamc will fail-over to the other addresses, if the first one cannot be connected to You need to have a host that has multiple A records. spamd.domain.com A 123.123.123.123 spamd.domain.com A 123.123.123.124 spamd.domain.com A 123.123.123.125 /usr/bin/spamc -d spamd.domain.com If your DNS server sends the results back in a different order each time then it will not be a fallback but a round robin. You might be able to simply use /etc/host entries. I've never tried it as I use qmail which will not use the host file, so I always rely on DNS. Don't know if spamc will use the host file or not. Huh, I am not familiar with how to use /etc/hosts as a DNS source. Can you clarify? I didn't mean use it as a dns source, but many programs can look first to your /etc/hosts file prior to doing a DNS lookup. I do not know if spamc will do that, I do not believe it does. Mainly my question was if/how I could avoid making it a DNS query. I'd like to simply hand spamc the two addresses that I want it to have manually, and I do *NOT* want round-robin, I want failover I do not think you can do this. You can use a IP address with spamc to save a lookup. I use the following run script under demontools, #!/sbin/sh PATH=/usr/bin:/usr/local/bin exec /usr/local/bin/softlimit -a 12800 \ /usr/local/bin/spamd -i 10.0.240.253 -p 1783 -A 10.0.240.0/24 \ -m 10 --max-conn-per-child=200 -u vpopmail -x -q -s stderr 21 I do not believe you can have mulitple addresses behind the -i switch, at least the docs do not lead me to believe it is possible. Maybe someone else knows better. spamd is not a problem for me. I run spamd on two machines, one being my main SA server, and one being a fallback just in case something goes awry (something recently did). I'd really like to be able to tell spamc that it can go to a 2nd IP address in case the first one fails, possibly by doing as I wrote above: /usr/bin/spamc -d 123.45.67.8 -d 127.0.0.1 but it seems I can't do this unless I go the DNS route (which I don't know how to do, since my main SA server must be routed to using an internal network IP). Thanks! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
sa-learn for per-user sql bayes
Hi, I'm looking for how to tell sa-learn to learn against a per-user Bayes database (in MySQL) instead of learning sitewide. I swear I saw this was going to be in 3.0.2; some kind of change to sa-learn... but a couple hours searching around didn't turn up anything helpful. If I missed something obvious, a tip on where to look would be great! TIA! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: sa-learn for per-user sql bayes
I'm looking for how to tell sa-learn to learn against a per-user Bayes database (in MySQL) instead of learning sitewide. As in manually training the database? sa-learn -u username I don't know... as long as this will do the same thing spamd does with the username to work against that user's sql-bayes stuff. Looking at the POD docs on the SA website does not show this option, but I do see it in the tarball. I thought the docs on the site would be the most up to date guess not. Thanks much! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: sa-learn for per-user sql bayes
I'm looking for how to tell sa-learn to learn against a per-user Bayes database (in MySQL) instead of learning sitewide. As in manually training the database? sa-learn -u username Can anyone tell me how to verify that a message was learnt correctly for the right user? Running with -D helps a little bit: /usr/bin/sa-learn -D --spam [EMAIL PROTECTED] message Had this line in the output: debug: bayes: Using username: [EMAIL PROTECTED] But I would like to either dig in the Bayes database and verify or send myself another message with the same content and watch it get tagged as spam next time (which doesn't seem to work with my first tests). Any pointers on how to verify learning? Thanks! __ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/
Re: spamd still burning CPU in 3.0.1
All, email builder wrote: How much email are you processing ? Well, just the other day we had an average of 48 msgs/min (max 255/min) get run through SA. Can't say today yet because can't run our stats tools until the busy hours are over cuz SA is hogging the CPU. ;) Hi, Your CPU is over loaded. At 48 a minute it should run just ok on a 2.8 Ghz machine, much over that it's going to start having problems. On our 2.4 Ghz (not HT) processor if I process over 35 a minute I start having problems with load. I'd recommend upgrading to a dual server or perhaps putting in a second server with round robin DNS (or if you can do it, a load balancer). SA is that CPU intensive, it really is. Maybe try adding RBL's in front of the MTA to reduce the number of messages you have to scan, that's what we do. Regards, Rick Just to top off this thread, I wanted to let all the wonderful people who offered their system stats/specs know that we added a 2nd machine that is a dedicated SA server where the only other app running is MySQL (for Bayes/AWL) and things are humming along very nicely. One server apparently just couldn't handle the load we had. Thanks again all! __ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail
Re: [OT] Making two machines talk to one another
I am attempting to offload SA to a machine that is not my main MX server. I have two machines, two NIC cards and a crossover cable, but after that I get very lost. I believe there should be a way to make them aware of one another using this direct connection w/out the need for DHCP or a router but I have no idea how to set this up. Can someone offer advice or point me in the direction of where I should be looking, reading, asking? If I understand you right, you want to the checking on another machine? Look for spamd / spamc the spamassassin daemon and client. No, I am looking for help on setting up the actual networking between the machines. Software level setup is the easy part. ;) One nice soul gave me this offlist: -- On RedHat variants, at least, in /etc/sysconfig/network-scripts/ifcfg-eth1 assuming the 2nd NIC card is device eth1 on the 2nd nic for both. use a different IP/subnet than your LAN. machine one - 10.10.0.1/255.255.255.0 gateway 10.10.0.1 machine two - 10.10.0.2/255.255.255.0 gateway 10.10.0.1 should be able to ping back and forth once you plug in the crossover cable. create iptable rules/excepts if needed. replace 10.10.0.x with whatever non-routable RFC compliant subnet you want to use. __ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail
[OT] Making two machines talk to one another
Hello, I am attempting to offload SA to a machine that is not my main MX server. I have two machines, two NIC cards and a crossover cable, but after that I get very lost. I believe there should be a way to make them aware of one another using this direct connection w/out the need for DHCP or a router but I have no idea how to set this up. Can someone offer advice or point me in the direction of where I should be looking, reading, asking? many thanks! __ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail
make test error installing Net::Ident
All, I understand this may be the wrong place to ask a perl question, but I have a question that Google nor any list I can find will answer... trying to install perl module Net::Ident, I get make test errors as follows. I'd really appreciate any tips. I've no idea why this is happening (Fedora Core 1): # perl -MCPAN -e'CPAN::Shell-install(Net::Ident)' CPAN: Storable loaded ok Going to read /root/.cpan/Metadata Database was generated on Sat, 27 Nov 2004 04:00:47 GMT Running install for module Net::Ident Running make for J/JP/JPC/Net-Ident-1.20.tar.gz CPAN: Digest::MD5 loaded ok Checksum for /root/.cpan/sources/authors/id/J/JP/JPC/Net-Ident-1.20.tar.gz ok Scanning cache /root/.cpan/build for sizes Net-Ident-1.20/ Net-Ident-1.20/t/ Net-Ident-1.20/t/0use.t Net-Ident-1.20/t/apache/ Net-Ident-1.20/t/apache/logs/ Net-Ident-1.20/t/apache/logs/.exists Net-Ident-1.20/t/apache/conf/ Net-Ident-1.20/t/apache/conf/httpd.conf.in Net-Ident-1.20/t/apache/conf/apache_config.pl.in Net-Ident-1.20/t/apache/conf/access.conf.in Net-Ident-1.20/t/apache/conf/srm.conf.in Net-Ident-1.20/t/apache/conf/mime.types Net-Ident-1.20/t/apache/html/ Net-Ident-1.20/t/apache/html/testapache.txt Net-Ident-1.20/t/apache/perl/ Net-Ident-1.20/t/apache/perl/testmodperl Net-Ident-1.20/t/apache/perl/testident Net-Ident-1.20/t/apache.t Net-Ident-1.20/t/compat.t Net-Ident-1.20/t/hosts Net-Ident-1.20/t/Ident.t Net-Ident-1.20/Makefile.PL Net-Ident-1.20/README Net-Ident-1.20/INSTALL Net-Ident-1.20/MANIFEST Net-Ident-1.20/Changes Net-Ident-1.20/Ident.pm Removing previously used /root/.cpan/build/Net-Ident-1.20 CPAN.pm: Going to build J/JP/JPC/Net-Ident-1.20.tar.gz Checking if your kit is complete... Looks good Do not worry if any of the following items are not found Checking for previous Net::Ident... Not found. You could force installing the backwards-compatible Net::Ident, for example because you want to use scripts that rely on the backwards compatible calling syntax [not that that's very likely; I'm not aware of any existing scripts that use it -- JohnPC]. To enable this, re-run this Makefile.PL using: perl Makefile.PL --force-compat Checking for Apache.pm... not found Writing Makefile for Net::Ident cp Ident.pm blib/lib/Net/Ident.pm Manifying blib/man3/Net::Ident.3pm /usr/bin/make -- OK Running make test PERL_DL_NONLAZY=1 /usr/bin/perl -MExtUtils::Command::MM -e test_harness(0, 'blib/lib', 'blib/arch') t/*.t t/0use..Net::Ident::_export_hooks() called too early to check prototype at /root/.cpan/build/Net-Ident-1.20/blib/lib/Net/Ident.pm line 29. t/0use..ok t/apacheNet::Ident::_export_hooks() called too early to check prototype at /root/.cpan/build/Net-Ident-1.20/blib/lib/Net/Ident.pm line 29. skipped all skipped: no reason given t/compatNet::Ident::_export_hooks() called too early to check prototype at /root/.cpan/build/Net-Ident-1.20/blib/lib/Net/Ident.pm line 29. skipped all skipped: no reason given t/Ident.Net::Ident::_export_hooks() called too early to check prototype at /root/.cpan/build/Net-Ident-1.20/blib/lib/Net/Ident.pm line 29. t/Ident.FAILED tests 1-3 Failed 3/7 tests, 57.14% okay Failed Test Stat Wstat Total Fail Failed List of Failed --- t/Ident.t 73 42.86% 1-3 2 tests skipped. Failed 1/4 test scripts, 75.00% okay. 3/8 subtests failed, 62.50% okay. make: *** [test_dynamic] Error 255 /usr/bin/make test -- NOT OK Running make install make test had returned bad status, won't install without force __ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail
RE: multiple spamd machines
--- Dan Barker [EMAIL PROTECTED] wrote: You can try it with one machine, multiple IP's/multiple spamd's with the -i. Mmm, interesting idea, but why would anyone do that instead of just using a single instance of spamd and increasing max-children? Oh--! You were just giving me a way to try this idea. I get it. ;) Thanks! Dan snip Best way to see is to try it. Indeed, although I don't have both machines online... just trying to get my ducks in a row before taking the plunge. /snip __ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
Re: Customizing the SA error message?
Anyone? How is it possible to use the report_hostname template setting to pick up on virtual domains? Or is it? Mine always uses the actual domain name of the machine itself; I'd rather use the virtual host name from the domain of the target user. On 7/11/04 7:55 AM, Theo Van Dinter [EMAIL PROTECTED] wrote: On Sun, Nov 07, 2004 at 07:39:00AM +1100, Gavin Cato wrote: Is there a way to edit this apart from editing the source code? Yeah, it's configurable. Check out perldoc Mail::SpamAssassin::Conf, look for report and clear_report_template. :) __ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
Re: multiple spamd machines
OK, distilling this conversation a little bit, can anyone comment on this: You are darn close there... What you want is /usr/bin/spamc -u username -d spa.yourdomain.com -H And spa.yourdomain.com has two ptr records, one to 127.0.0.1 and the other to 123.45.6.789 in Bind talk that would be spa.yourdomain.com. IN A127.0.0.1 spa.yourdomain.com. IN A123.45.6.789 and in tinydns +spa.yourdomain.com:127.0.0.1:3600 +spa.yourdomain.com:123.45.6.789:3600 I'm not sure if you really need the -H, I know I don't using dnscache as my local dns server. Why? Does dnscache randomize for you? I haven't tested using a -d 127.0.0.1,123.45.6.789 to see if spamc will fail over to the second host if all the connections are busy on the first host. I could be wrong but I don't think it will fail over to the second host because the first host will just place it in the queue to be processed. I could very well be wrong though. But using the DNS-based approach as you do, it *will* fail over?? Why? It seems like if spamd tries to queue up any request it gets, then it would happen to you, too. Spamd should not have any way of knowing if you used DNS to resolve it or the addresses were listed on the command line, no? Thanks! __ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
RE: multiple spamd machines
--- Bowie Bailey [EMAIL PROTECTED] wrote: From: email builder [mailto:[EMAIL PROTECTED] OK, distilling this conversation a little bit, can anyone comment on this: You are darn close there... What you want is /usr/bin/spamc -u username -d spa.yourdomain.com -H And spa.yourdomain.com has two ptr records, one to 127.0.0.1 and the other to 123.45.6.789 in Bind talk that would be spa.yourdomain.com. IN A127.0.0.1 spa.yourdomain.com. IN A123.45.6.789 and in tinydns +spa.yourdomain.com:127.0.0.1:3600 +spa.yourdomain.com:123.45.6.789:3600 I'm not sure if you really need the -H, I know I don't using dnscache as my local dns server. Why? Does dnscache randomize for you? I haven't tested using a -d 127.0.0.1,123.45.6.789 to see if spamc will fail over to the second host if all the connections are busy on the first host. I could be wrong but I don't think it will fail over to the second host because the first host will just place it in the queue to be processed. I could very well be wrong though. But using the DNS-based approach as you do, it *will* fail over?? Why? It seems like if spamd tries to queue up any request it gets, then it would happen to you, too. Spamd should not have any way of knowing if you used DNS to resolve it or the addresses were listed on the command line, no? DNS will not fail over, but it should give you crude 50/50 load balancing. Sometimes the server will get one IP and sometimes it will get the other. The server should either rotate or randomize the IPs. You will need to test your DNS server to see how it works. This is assuming that you will query the server each time. Local caching can kill this scheme. OK, so that makes sense, and that to me is an argument to use: spamc -d 127.0.0.1 123.4.5.678 -H If I understand, this will do the same thing as the DNS-based scheme, but will always give round robin results (man page says it is randomized I think), and it has the benefit of no worries about DNS cache causing problems. Thoughts? As far as I know, there is no way to weight the traffic toward a particular server without some sort of load balancing or proxy solution. Sounds reasonable. So if the processing capabilities of my two SpamAssassin servers is *vastly* different, it seems like I should just forget the idea of trying to use the underpowered one unless I can figure out a way to load balance... yeah? __ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
Re: multiple spamd machines
--- Rick Macdougall [EMAIL PROTECTED] wrote: email builder wrote: OK, distilling this conversation a little bit, can anyone comment on this: You are darn close there... What you want is /usr/bin/spamc -u username -d spa.yourdomain.com -H And spa.yourdomain.com has two ptr records, one to 127.0.0.1 and the other to 123.45.6.789 in Bind talk that would be spa.yourdomain.com.IN A127.0.0.1 spa.yourdomain.com.IN A123.45.6.789 and in tinydns +spa.yourdomain.com:127.0.0.1:3600 +spa.yourdomain.com:123.45.6.789:3600 I'm not sure if you really need the -H, I know I don't using dnscache as my local dns server. Why? Does dnscache randomize for you? I haven't tested using a -d 127.0.0.1,123.45.6.789 to see if spamc will fail over to the second host if all the connections are busy on the first host. I could be wrong but I don't think it will fail over to the second host because the first host will just place it in the queue to be processed. I could very well be wrong though. But using the DNS-based approach as you do, it *will* fail over?? Why? It seems like if spamd tries to queue up any request it gets, then it would happen to you, too. Spamd should not have any way of knowing if you used DNS to resolve it or the addresses were listed on the command line, no? Thanks! Hi, Yes, if I telnet spa.yourdomain.com, then telnet spa.yourdomain.com I will connect first to one server, then the next. That how ever is not fail over, that's dns round robin. If machine one really failed, then queries to that machine would still fail (I think, haven't tested it) Understood. How ever, if I use the syntax -d 127.0.0.1, x.x.x.x, it will always connect first to 127.0.0.1 and I believe, will only fail over to x.x.x.x if the 127.0.0.1 spamd daemon is down (different from being overloaded) But if I understood the man page, if you add -H, then it will randomize between 127.0.0.1 and x.x.x.x, right? This is roughly the same thing as the DNS-based solution you are using, isn't it? Or am I missing something? Best way to see is to try it. Indeed, although I don't have both machines online... just trying to get my ducks in a row before taking the plunge. Many thanks for your advice! __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
