Hi Kyle, It's probably going to take a while for Qmail to hammer through its queue, and deal with everything that simscan still has to handle. You probably wont notice a difference for a few hours, or even tomorrow.
regards, Dairenn Lombard Linux Engineer, Systems Administration Department BroadSpire, Inc. ----------------------------- Hosting | Colocation | Design > -----Original Message----- > From: Kyle Quillen [mailto:[EMAIL PROTECTED] > Sent: Friday, May 23, 2008 1:01 PM > To: qmailtoaster-list@qmailtoaster.com > Subject: RE: [qmailtoaster] Load balancing > > > Well all, > > After doing everything in here except the firewall rules and > with the reordering of the RBL's server loads are 1.14, 1.37, > 1.46. They seem to be staying that way consistently only > time will tell though. If things kick up a little I will > drop the Firewall rules in. > > I want to say thanks to everyone for the help. The Dups seem > to have stopped as well. > > This is my server > Intel xeon Dual cpu 3.2ghz 6 gigs of ram with 500 gig scsi drives. > > I have almost 2000 email accounts on 3 domains and this box > also does mail cleaning for a fourth. > > Thank you very much > Kyle > > > > > > On Fri, 2008-05-23 at 10:50 -0700, Dairenn Lombard wrote: > > Hi Kyle, > > > > We have the same issue with our toasters. > > > > The long and short of it is, try to keep your total mail > users under > > about 1,000 (this usually works out to about 100-200 domains). > > Anything over that, and you should deploy another mail server. > > > > When a remote SMTP server connects to your Qmail Toaster > and delivers > > a message, it is timing out awaiting for Qmail to send back an > > acknowledgement it got your message, because, for some > weird reason, > > Qmail waits until after simscan has finished processing > (which itself > > is waiting for clamav and spamassassin to do the actual processing) > > before returning such an acknowledgement. > > > > Several other things can cause duplicates too like POP3 clients > > leaving a copy of mail messages on the server and then > losing track of > > what it's already downloaded (a common scenario when the > client has to > > go through a local anti-virus application on the mail user's > > computer). But also, bad .qmail files in the user's vpopmail > > directory can cause delivery to happen two times. > > > > That's easy to fix. What's not easy is taming spamassassin and > > clamav. We have had to do a lot of work--a lot more than I feel we > > should have had to for a proported out-of-the-box solution--to keep > > spamassassin and clamav from killing your mail server... > > > > First of all, throttle SMTP traffic with iptables to > prevent excessive > > connections (and resultant spamd/clamd instances) in the > first place: > > > > -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp > --dport 25 > > -m recent --set -A RH-Firewall-1-INPUT -m state --state NEW > -m tcp -p > > tcp --dport 25 -m recent --update --seconds 60 --hitcount 12 -j DROP > > > > (works in typical CentOS/RedHat /etc/sysconfig/iptables files) > > > > Then, SERIOUSLY change how spamd starts and runs: > > > > /var/qmail/supervise/spamd/run: > > > > #!/bin/sh > > exec /bin/nice --adjustment=20 /usr/bin/spamd -m 4 --max-children=2 > > --max-conn-per-child=15 -l -L -x -u vpopmail -s stderr 2>&1 > > > > Tweak /etc/mail/spamassassin/local.cf: > > > > ok_locales all > > skip_rbl_checks 0 > > rbl_timeout 5 > > > > required_hits 5 > > report_safe 0 > > rewrite_header Subject ***SPAM*** > > > > use_pyzor 1 > > > > # Use for any MTA servers from which you want to trust will > not spam > > you, such as another server in your # own network. > > # trusted_networks 127.0.0.1/18 > > # If you use Postini, uncomment this line: > > # trusted_networks 64.18.6.10 > > > > > > use_auto_whitelist 0 > > > > use_bayes 1 > > use_bayes_rules 1 > > bayes_auto_learn 1 > > bayes_auto_expire 1 > > bayes_expiry_max_db_size 18750 > > > > # Inserted to ignore all mail from Postini, if you use it, > and pass it > > on unmodified # This should lessen our overall load. > > bayes_ignore_header X-pstn-levels > > > > Make sure to be running THE most current version of > SpamAssassin and > > ClamAV (SpamAssassin in particular is exceedingly buggy, > and they're > > constantly fixing it all the time). This may require you > to install > > or upgrade a variety of perl modules (get comfortable with using > > CPAN). > > > > As for ClamAV? Well, here's what we do in /etc/clamd.conf: > > > > PhishingSignatures no > > PhishingScanURLs no > > ScanHTML no > > MaxScanSize 1M > > MaxFileSize 1M > > > > Changes to spamd or clamd configuration files need a svc -d > and then > > svc -u from the /var/qmail/supervise directory (i.e., svc -d spamd). > > > > /var/qmail/control/concurrencyincoming is set to something > reasonable > > for a 2.4GHz P4 with 1GB of RAM, 100 - concurrencyremote is 300 and > > concurrencylocal is 200 > > > > We set /var/qmail/control/databytes to 20 MB (this is industry > > standard anyway). > > > > Changing these files will require a HUP of qmail-smtpd > (service qmail > > restart does this). > > > > We use the following simcontrol file (so that these files, > if attached > > to incoming e-mail, don't even get delivered, saving clamd > the trouble > > of even having to run): > > > > > :clam=yes,spam=yes,spam_hits=12,attach=.ade:.adp:.app:.asd:.asx:.bas:. > > ba > > > t:.bin:.chm:.cil:.cla:.class:.cmd:.com:.cpl:.crt:.csh:.dll:.do > t:.email:. > > > eml:.exe:.fxp:.hlp:.hta:.inf:.ins:.isp:.js:.jse:.ksh:.lnk:.mda > :.mdb:.mde > > > :.mdt:.mdw:.mdz:.mpe:.msc:.msi:.msp:.mst:.nws:.ocx:.ops:.pcd:. > pif:.pl:.p > > > m:.pot:.prf:.prg:.ps:.reg:.scf:.scr:.sct:.shb:.shm:.shs:.url:. > vb:.vbe:.v > > bs:.vxd:.wmd:.wmf:.wms:.wmz:.wsc:.wsf:.wsh:.wsz:.xsl:.xlt:.xlw > > > > Be sure to run qmailctl cdb to rehash the simcontrol.cdb file. > > > > Finally, we've decided to not allow catch-all aliases. > This has been > > the single biggest helpful thing we have done to resolve high load > > issues on our mail servers. I can't tell you how badly your mail > > server can get beat up by a domain catch-all accepting > loads of spam. > > (Just look at your MRTG or ISOQLOG pages, and you'll see > what I mean.) > > So, you'll want to set catchall to BOUNCE (not delete, > because it goes > > through the ENTIRE process of accepting an e-mail before finally > > realizing it should be deleted). Here's a way to do this to the > > entire mail server: > > > > Make a file called .qmail-default in /usr/local/etc that looks like > > this: > > > > | /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox > > > > Then run this command: > > > > find /home/vpopmail/domains -name ".qmail-default" -type f -exec > > /bin/cp -rp /usr/local/etc/.qmail-default {} \; > > > > I was able to get mail servers running loads in excess of > 7-14 down to > > a nominal average of .55 (on servers running 700-900 > domains or over > > 2,000 mailboxes), and I've been able to get the queues down from an > > average of hundreds of messages to dozens. > > > > Do these things at your own risk. I Rarely check this > mailing list so > > definitely test, experiment and investigate these settings on a > > NON-production server first before trying it out on a mail > server you > > have any paying customers use. If it works out for you, and I > > sincerely hope it does, please pass on the knowledge. > > > > > > regards, > > Dairenn Lombard > > BroadSpire, Inc. > > Linux Engineer, Systems Administration Dept. > > ----------------------------- > > Hosting | Colocation | Design > > > > > > > > > > > -----Original Message----- > > > From: Kyle Quillen [mailto:[EMAIL PROTECTED] > > > Sent: Friday, May 23, 2008 9:33 AM > > > To: qmailtoaster-list@qmailtoaster.com > > > Subject: Re: [qmailtoaster] Load balancing > > > > > > > > > Spamdyke is installed and has been for about a week It has > > > seemed to help but my loads are still staying around 3.5-4.5 > > > > > > The largest problem that I have right now is that users are > > > getting duplicate emails and I can't figure out how to stop it. > > > > > > Since I implemented the greylisting things seem to be calming > > > a little bit but the dups are still coming in. I have gotten > > > multiple copies of emails that were sent yesterday at like > > > 1030 in the am and i did get them. > > > > > > Suggestions? > > > > > > > > > thanks > > > q > > > > > > > > > > > > On Fri, 2008-05-23 at 08:51 -0700, Eric Shubert wrote: > > > > Kyle Quillen wrote: > > > > > All, > > > > > > > > > > Does anyone have any resources that I can pull from that > > > will point > > > > > me in the direction of setting up load balancing with the > > > toaster? > > > > > I want to keep using this mail server but I have to > find a way > > > > > to > > > > > deal with the large amount of mail that I am having to > > > process. It > > > > > is mostly spam so maybe what I am looking for is a spam > > > scanning system. > > > > > > > > > > Thoughts? > > > > > > > > > > > > > Spamdyke. > > > > Do it. > > > > > > > -- > > > Thanks, > > > Kyle Quillen > > > Lightspeed Wireless > > > [EMAIL PROTECTED] > > > 330.473.1231 ext.202 > > > > > --------------------------------------------------------------------- > > QmailToaster hosted by: VR Hosted <http://www.vr.org> > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > -- > Thanks, > Kyle Quillen > Lightspeed Wireless > [EMAIL PROTECTED] > 330.473.1231 ext.202 --------------------------------------------------------------------- QmailToaster hosted by: VR Hosted <http://www.vr.org> --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
