> I am running Redhat AS 3.0 and SA 2.63 with sendmail and spamass-milter. > I will start by mentioning the following: > > - I am processing about 10,000 msgs / hour > - This is a dual proc 3.2GHz Zeon with 2G of RAM. The system is not > swapping at all (based on vmstat) > - I have already fixed the UTF8 issue > - I am not running any external checks (have run with spamd -L and still > have the same issue) > - I have disabled the Bayes stuff (from the local.cf file) > - I am not running the auto white list stuff (from the local.cf file) > > Now for the fun part... I am hitting loads of 10+ on the server on a > regular basis which makes sendmail back off (I know how to fix the > sendmail issue so please do not comment on that). > > I have tried everything I can think of to take the load off the system > but nothing is working for me. > > When I do a TOP on the system, this is what I see... > > 18:57:50 up 8 days, 3:29, 2 users, load average: 9.37, 9.35, 9.38 > 124 processes: 108 sleeping, 16 running, 0 zombie, 0 stopped > CPU states: cpu user nice system irq softirq iowait idle > total 74.4% 0.0% 22.9% 0.0% 0.0% 0.0% 2.4% > cpu00 76.1% 0.0% 21.9% 0.0% 0.1% 0.0% 1.7% > cpu01 70.2% 0.0% 26.0% 0.0% 0.0% 0.1% 3.4% > cpu02 76.7% 0.0% 21.1% 0.0% 0.1% 0.0% 1.9% > cpu03 74.6% 0.0% 22.4% 0.0% 0.0% 0.1% 2.6% > Mem: 2061652k av, 1496032k used, 565620k free, 0k shrd, 84660k > buff > 905668k actv, 258248k in_d, 8904k in_c > Swap: 2097112k av, 0k used, 2097112k free 439292k > cached > > PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU > COMMAND > 17262 mail 25 0 107M 107M 2400 R 7.9 5.3 0:01 2 spamd > 17298 mail 25 0 108M 108M 2400 R 4.7 5.3 0:00 1 spamd > 17284 mail 25 0 108M 108M 2392 R 4.5 5.3 0:00 3 spamd > 17286 mail 25 0 108M 108M 2392 R 4.3 5.3 0:00 3 spamd > 17301 mail 25 0 108M 108M 2412 R 3.5 5.3 0:00 3 spamd > 17295 mail 25 0 107M 107M 2392 R 3.4 5.3 0:00 3 spamd > 17333 mail 25 0 108M 108M 2392 R 3.4 5.3 0:00 2 spamd > 17319 mail 25 0 108M 108M 2416 R 3.3 5.3 0:00 0 spamd > 17310 mail 25 0 108M 108M 2416 R 3.0 5.3 0:00 1 spamd > 17314 mail 25 0 108M 108M 2416 R 2.9 5.3 0:00 0 spamd > 17330 mail 25 0 107M 107M 2412 R 2.9 5.3 0:00 0 spamd > 7944 mail 23 0 107M 107M 2376 S 1.2 5.3 5:03 2 spamd > > Does anyone have any suggestions on what I can try next? Any help is > greatly appriciated...
Hmm, I don't know what is causing your problem specifically, but I see one thing that is VERY wrong. Spamd should not be using 108M of ram, which is what seems to be happening in your case. Normally Spamd will use about 18M of ram per process. Find out why that is happening (I can only suggest a process of elimination) and you should be well on the way to finding what the heck is going on....a freshly started spamd should only be using about 18M of ram. A couple of others have had a similar problem with excess memory usage before although I don't recall what the reason was - extra rulesets with errors in them perhaps. Maybe they will see your message and respond. Also consider that 10,000 message per hour is a BIG load for a single machine of the specs you listed when running spamassassin, and depending on how you integrate spamassassin, and you may simply be loading the machine too much. You don't say how you're calling spamassassin but since you're using spamd and sendmail, I'll put two and two together and guess that you're calling spamc from procmail ? This is what I'm doing too, and we're starting to notice serious load issues now that the amount of spam is going up... Spec's here are P4 2.4Ghz, 1GB ram, single CPU, and by comparision we're only doing about 50,000 messages per DAY, and I'm seeing load average problems (>10) frequenly. According to some on this list, sendmail + procmail + spamc/spamd simply doesn't scale, and I'd have to grudgingly agree with this, and I'm looking into my options to improve this... The problem seems to be 50/50 procmail/spamc overhead, and sendmail's poor local delivery scheduling and queuing system. (Which tends try and scan a lot of things at once without any proper throttling of maximum local simultanous deliverys) The options are to basically call spamassassin directly from sendmail using something like MimeDefang, Mailscanner or Amavisd, which apparently performs a lot better, but you lose a lot of per-customer control of spam settings, (a problem for us) since it calls SA *before* a message is seperated out to each recipient, Or to use a different mailserver altogether that supports better integration with spamassassin, and I've been taking a look at Exim recently, but havn't got very far yet. So in summary, you do have something screwy going on there with the ram usage, but even if you resolve that, if you are using a sendmail/procmail/spamc combination you can expect to be getting performance problems with the number of messages you're processing, so you need to look seriously at a different integration approach for spamassassin or perhaps dumping sendmail...(both of which I'm exploring myself right now....) Regards, Simon
