Re: [Mimedefang] tmpfs on Linux
On Fri, Nov 12, 2004 at 05:15:04PM -0500, scohen wrote: > > On my machine: > > > > /dev/ram0 48388646458840 1% > > /var/spool/MIMEDefang > > /dev/ram1 483886132675326211 29% /var/spool/bayes > > > > > With your setup you can lose 23 hours worth of data. The question for you > is is that acceptable or not? Yeah, that is an acceptable worst-case scenario. > Btw, do you make sure your database isn't > being used when you copy it? If not, have you tried to use a backup? I don't do anything to ensure it isn't in use. My startup script copies the latest backup over to the ramdisk after creating it, and so far with three servers and a few reboots over the last couple years I haven't had an issue. The other performance boost I got was to specify noatime for the var filesystem, where sendmail's spool dir resides. Matt -- Matthew S. Cramer <[EMAIL PROTECTED]> Office: 717-396-5032 Infrastructure Security Analyst Fax:717-396-5590 Armstrong World Industries, Inc.Cell: 717-917-7099 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On Fri, 12 Nov 2004, Matthew S. Cramer wrote: > On Fri, Nov 12, 2004 at 12:25:12PM -0500, Jeff Rife wrote: > > On 12 Nov 2004 at 9:03, Aleksandar Milivojevic wrote: > > > > > For later (simpler) global solution, just add these lines to > > > sa-mimedefang.cf: > > > > > > auto_whitelist_path /var/spool/MIMEDefang/awl > > > > > > bayes_path /var/spool/MIMEDefang/bayes > > > > These are really *bad* paths if you put /var/spool/MIMEDefang on any > > sort of ramdisk (like many of us do). > > Why? I found this greatly improved performance. I have a cron job > that copies the bayes db files over to a physical disk once every > day so there is no danger of losing the entire database if the ramdisk > would suddenly go away. > > On my machine: > > /dev/ram0 48388646458840 1% /var/spool/MIMEDefang > /dev/ram1 483886132675326211 29% /var/spool/bayes > > With your setup you can lose 23 hours worth of data. The question for you is is that acceptable or not? Btw, do you make sure your database isn't being used when you copy it? If not, have you tried to use a backup? > Matt Steve Cohen ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On 12 Nov 2004 at 15:46, Matthew S. Cramer wrote: > > These are really *bad* paths if you put /var/spool/MIMEDefang on any > > sort of ramdisk (like many of us do). > > Why? I found this greatly improved performance. I have a cron job > that copies the bayes db files over to a physical disk once every > day so there is no danger of losing the entire database if the ramdisk > would suddenly go away. Many people who read the list archives just follow instructions blindly, and won't know to do this. > On my machine: > > /dev/ram0 48388646458840 1% /var/spool/MIMEDefang > /dev/ram1 483886132675326211 29% /var/spool/bayes You've got a lot more RAM than I have to spare for this (500MB for each ramdisk). Also, you end up using double for the bayes database because the DB code caches a lot of the database in RAM. Using a journal for the bayes database should result in acceptable performance under most circumstances. -- Jeff Rife| Sam: How's life in the fast lane, Normie? SPAM bait: | [EMAIL PROTECTED] | Norm: Beats me, I can't find the on-ramp. [EMAIL PROTECTED] | ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On Fri, Nov 12, 2004 at 12:25:12PM -0500, Jeff Rife wrote: > On 12 Nov 2004 at 9:03, Aleksandar Milivojevic wrote: > > > For later (simpler) global solution, just add these lines to > > sa-mimedefang.cf: > > > > auto_whitelist_path /var/spool/MIMEDefang/awl > > > > bayes_path /var/spool/MIMEDefang/bayes > > These are really *bad* paths if you put /var/spool/MIMEDefang on any > sort of ramdisk (like many of us do). Why? I found this greatly improved performance. I have a cron job that copies the bayes db files over to a physical disk once every day so there is no danger of losing the entire database if the ramdisk would suddenly go away. On my machine: /dev/ram0 48388646458840 1% /var/spool/MIMEDefang /dev/ram1 483886132675326211 29% /var/spool/bayes Matt -- Matthew S. Cramer <[EMAIL PROTECTED]> Office: 717-396-5032 Infrastructure Security Analyst Fax:717-396-5590 Armstrong World Industries, Inc.Cell: 717-917-7099 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
Jeff Rife wrote: On 12 Nov 2004 at 9:03, Aleksandar Milivojevic wrote: For later (simpler) global solution, just add these lines to sa-mimedefang.cf: auto_whitelist_path /var/spool/MIMEDefang/awl bayes_path /var/spool/MIMEDefang/bayes These are really *bad* paths if you put /var/spool/MIMEDefang on any sort of ramdisk (like many of us do). In my defense, those were example paths (mine don't look like that either). I've put them as examples since MIMEDefang directory is owned by defang user, so it is one possiblity (if, as you said, one doesn't use ramdisk for that directory). If somebody does use ramdisk, he'll probably have enough of common sense to change them to some more permanent location. -- Aleksandar Milivojevic <[EMAIL PROTECTED]>Pollard Banknote Limited Systems Administrator 1499 Buffalo Place Tel: (204) 474-2323 ext 276 Winnipeg, MB R3T 1L7 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On 12 Nov 2004 at 9:03, Aleksandar Milivojevic wrote: > For later (simpler) global solution, just add these lines to > sa-mimedefang.cf: > > auto_whitelist_path /var/spool/MIMEDefang/awl > > bayes_path /var/spool/MIMEDefang/bayes These are really *bad* paths if you put /var/spool/MIMEDefang on any sort of ramdisk (like many of us do). -- Jeff Rife| SPAM bait: | http://www.nabs.net/Cartoons/Dilbert/Evaluation.jpg [EMAIL PROTECTED] | [EMAIL PROTECTED] | ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
Greg Miller wrote: Currently not using bayesian or whitelist. This is a dedicated sendmail box. You can use bayesian and/or whitelist on dedicated sendmail box (no local users). There are two solutions. More complicated is to keep them in SQL database. That way you can have them on per-user basis. Simpler is to have them global for all users. For later (simpler) global solution, just add these lines to sa-mimedefang.cf: # Enable AWL use_auto_whitelist 1 auto_whitelist_path /var/spool/MIMEDefang/awl auto_whitelist_file_mode 0640 # Enable Bayes use_bayes1 use_bayes_rules 1 bayes_path /var/spool/MIMEDefang/bayes bayes_file_mode 0640 bayes_auto_learn 1 You'll probably need DB_File Perl module installed. -- Aleksandar Milivojevic <[EMAIL PROTECTED]>Pollard Banknote Limited Systems Administrator 1499 Buffalo Place Tel: (204) 474-2323 ext 276 Winnipeg, MB R3T 1L7 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: Timeout settings (was Re: [Mimedefang] tmpfs on Linux)
Quoting "David F. Skoll" <[EMAIL PROTECTED]> Date: Thu, 11 Nov 2004 17:06:13 > On Thu, 11 Nov 2004, Greg Miller wrote: > > > During my investigations I noticed that many of my sendmail processes > > hang around for quite some time, presumably because the host on the > > other end is slow. I stumbled across a recommendation that the sendmail > > default timeouts be tuned as follows: Anyone else doing this? > > Some of those numbers are way too short. In particular, a confTO_DATAFINAL > of 5 minutes is definitely too low. RFC 2821 says that one SHOULD be > at least 10 minutes, and I would be conservative and make it 30 minutes. I'd leave that one at Sendmail's default one hour. Setting it too low may result in bandwith waste and multiple copies of email delivered. I've saw ClamAV + MIMEDefang taking some 10-15 minutes to complete when scanning emails with huge compressed attachments (on reasonably fast machine). If receiving side has some more milters, or is simply overloaded because it got several large emails to process at the same time, it could easilly take even longer. If somebody is going to DOS you, even timeout set to as short as one minute would be more than enough to allow for DOS attack. And you would need to be the one connecting to attacker's server (that's what this timeout controls). So really there's no point in lowering this. If you already transferred the email, give the other side as much time as it needs to do whatever it needs to do before accepting that email. -- Aleksandar Milivojevic <[EMAIL PROTECTED]>Pollard Banknote Limited Systems Administrator 1499 Buffalo Place Tel: (204) 474-2323 ext 276 Winnipeg, MB R3T 1L7 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Timeout settings (was Re: [Mimedefang] tmpfs on Linux)
On Thu, 11 Nov 2004, Greg Miller wrote: > During my investigations I noticed that many of my sendmail processes > hang around for quite some time, presumably because the host on the > other end is slow. I stumbled across a recommendation that the sendmail > default timeouts be tuned as follows: Anyone else doing this? Some of those numbers are way too short. In particular, a confTO_DATAFINAL of 5 minutes is definitely too low. RFC 2821 says that one SHOULD be at least 10 minutes, and I would be conservative and make it 30 minutes. See http://www.ietf.org/rfc/rfc2821.txt Section 4.5.3.2 for recommended minimum values. Regards, David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
Thanks to everyone who helped with my performance problems. In the end, I doubled the amount of RAM to 2GB. This prevented swapping and allowed by 50 sendmail processed and 15 mimedefang slaves to run with sufficient memory. In the process, I learned a lot about sendmail performance tuning, mostly that I need to learn more. :) During my investigations I noticed that many of my sendmail processes hang around for quite some time, presumably because the host on the other end is slow. I stumbled across a recommendation that the sendmail default timeouts be tuned as follows: Anyone else doing this? define(`confTO_INITIAL', `30s') define(`confTO_CONNECT', `30s') define(`confTO_ICONNECT', `30s') define(`confTO_HELO', `1m') define(`confTO_MAIL', `2m') define(`confTO_RCPT', `2m') define(`confTO_DATAINIT', `2m') define(`confTO_DATABLOCK', `2m') define(`confTO_DATAFINAL', `5m') define(`confTO_RESET', `1m') define(`confTO_QUIT', `1m') define(`confTO_MISC', `2m') define(`confTO_COMMAND', `1m') define(`confTO_IDENT', `0s') define(`confTO_FILEOPEN', `1m') define(`confTO_CONTROL', `1m') define(`confTO_HOSTSTATUS', `5m') -- Greg Miller, RHCE, CCNA, MCSE Senior Network Specialist University of Richmond [EMAIL PROTECTED] (804) 289-8546 On Thu, 2004-11-11 at 09:13 -0500, Kevin A. McGrail wrote: > > How would you suggest I do this? I have tried setting MaxDaemonChildren > > to 20, but those quickly get eaten up and I just end up refusing lots of > > mail. What is the recommended course of action in this case? > > Well, it just sounds like you need more RAM first which I think you agree > on. > > Second, you may need to lower the amount of time your MIMEDefang spends on > messages. Have you considered turning off the SpamAssassin Network-Based > tests? > > Third, you need to look at your mail volume. Do you know how many messages > per day/per hour you are getting? You might just simply need a more > powerful machine or a cluster of machines to share the load. > > Fourth, are you having any issues with dictionary attacks or email > harvesting? Is this machine the mail destination or just a gateway to > another mail server? > > > > True. Maybe we should just stop this email business. It's just a fad, > > right? :) > > I'd laugh if I didn't have a customer once argue this with me. > > > Regards, > KAM > > ___ > Visit http://www.mimedefang.org and http://www.canit.ca > MIMEDefang mailing list > [EMAIL PROTECTED] > http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
> How would you suggest I do this? I have tried setting MaxDaemonChildren > to 20, but those quickly get eaten up and I just end up refusing lots of > mail. What is the recommended course of action in this case? Well, it just sounds like you need more RAM first which I think you agree on. Second, you may need to lower the amount of time your MIMEDefang spends on messages. Have you considered turning off the SpamAssassin Network-Based tests? Third, you need to look at your mail volume. Do you know how many messages per day/per hour you are getting? You might just simply need a more powerful machine or a cluster of machines to share the load. Fourth, are you having any issues with dictionary attacks or email harvesting? Is this machine the mail destination or just a gateway to another mail server? > True. Maybe we should just stop this email business. It's just a fad, > right? :) I'd laugh if I didn't have a customer once argue this with me. Regards, KAM ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On 10 Nov 2004 at 14:17, Greg Miller wrote: > Currently not using bayesian or whitelist. This is a dedicated sendmail > box. Part of what my company does is marketing (opt-in only lists, of course), and some of that marketing is about drugs. Because of that, we get a lot of what might be "spam" from clients we work with, because of the keywords, etc. The auto-whitelist keeps these false positives down to zero. For other sites, this tool might not be helpful, but if you know that you expect to get *some* "spammy" e-mail from people who send you a lot of "good" e-mail, it does a good job. -- Jeff Rife| SPAM bait: | http://www.nabs.net/Cartoons/Dilbert/NoHelpDesk.jpg [EMAIL PROTECTED] | [EMAIL PROTECTED] | ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On Wed, 2004-11-10 at 11:17 -0500, Kevin A. McGrail wrote: > Let's say you are running 40 sendmails. That should take about 120MB of > ram. To run 40 sendmails, you need 10 to 15 mimedefangs running eating up > say 400 to 600MB of RAM as a guess. Add a 128MB tmpfs and you are using a > lot of RAM. You're absolutely right. I will double the amount of RAM first off. > > How large is your bayesian and whitelist files? What else is running on the > box? Currently not using bayesian or whitelist. This is a dedicated sendmail box. > > I think you should look into the 15-20 sendmail realm and you'll be much > better off. > How would you suggest I do this? I have tried setting MaxDaemonChildren to 20, but those quickly get eaten up and I just end up refusing lots of mail. What is the recommended course of action in this case? > If you scale the numbers above to 50 or 100 sendmails, you are just swapping > out of RAM. True. Maybe we should just stop this email business. It's just a fad, right? :) -- Greg Miller, RHCE, CCNA, MCSE Senior Network Specialist University of Richmond [EMAIL PROTECTED] (804) 289-8546 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
Let's say you are running 40 sendmails. That should take about 120MB of ram. To run 40 sendmails, you need 10 to 15 mimedefangs running eating up say 400 to 600MB of RAM as a guess. Add a 128MB tmpfs and you are using a lot of RAM. How large is your bayesian and whitelist files? What else is running on the box? I think you should look into the 15-20 sendmail realm and you'll be much better off. If you scale the numbers above to 50 or 100 sendmails, you are just swapping out of RAM. Regards, KAM I typically have 50-100 sendmail processes that are accepting mail and are in the "cmd read" state. This does seem high to me, especially since mimdefang is only processing a few messages per second. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
On Wed, 2004-11-10 at 10:19 -0500, Brenden Conte wrote: > How large is your tmpfs drive? Whats your usual mail volume? > > By default Linux allows itself to use 1/2 your ram per tmpfs drive. > > -Brenden > tmpfs is 128MB. Most of the time only 1MB or less is in use. Perhaps I should make it smaller? This is typical output from free -m total used free sharedbuffers cached Mem: 1004793210 0 86 194 -/+ buffers/cache:513491 Swap: 1023149874 During problem periods, used memory will go from 513 to 800-900 and used swap will climb to 400-500. I typically have 50-100 sendmail processes that are accepting mail and are in the "cmd read" state. This does seem high to me, especially since mimdefang is only processing a few messages per second. Thanks for your help. -- Greg Miller, RHCE, CCNA, MCSE Senior Network Specialist University of Richmond [EMAIL PROTECTED] (804) 289-8546 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs on Linux
How large is your tmpfs drive? Whats your usual mail volume? By default Linux allows itself to use 1/2 your ram per tmpfs drive. -Brenden On Wed, 2004-11-10 at 09:04, Greg Miller wrote: > Platform is RedHat EL AS 3 > Dual Xeon 3.0 Ghz CPU > 1 GB RAM > Load is between 1-3 messages per second. > > System performs very well, most of the time, with only 2-3 busy slaves. > However, on occasion, I will see all 15 of my slaves busy, lots of disk > I/O to swap, and "Please try again later" messages in the maillog. > > I am using tmpfs for /var/spool/MIMEDEFANG as recommended in the FAQ. I > believe my tmpfs is being sent to swap during these periods and causing > horrible performance. > > Has anyone else seen this behavior? Any solutions out there? > Thanks. -- Brenden Conte System Programmer, C&CT Rensselaer Polytechnic Institute (518)276-2540 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] tmpfs on Linux
It is likely that your tmpfs (ramdisk) is too big for the amount of total physical RAM installed in the system. Secondly, you don't state how many concurrent processes you are running, and so on. The system will swap when it runs low on usable RAM. Whatever RAM you've dedicated to the tmpfs is not otherwise "useable" to the running programs. Thus, paging/swapping occurs. Add more RAM to the system, and take a good practical look at how big your ramdisk truly needs to be. I know the worst-case calculation says to multiply the max message size by the max number of allowed child processes. However, it has been my experience that I in no way need a RAM disk of 2GB, which is equal to 50MB (my max message size) x 40 (max concurrent child processes). My average message size lately has been around 46K. And as rare as it is for me to receive a single 50MB email, I have not yet seen an instance where I've had to process 2 messages of that size, concurrently (though I have seen a mix of 10, 15, 4, and 30MB messages all at once, for example.) My RAMdisk is set to 128MB, and I have only seen it go 100% full once or twice, in my daily reports. In those cases, the messages were tempfailed by MIMEDefang, and succesfully re-transmitted on the next attempt by the sending servers. As an example (from last night's report of yesterday's traffic), this configuration succesfully handles the following message rates, on a 2GB dual-proc system: AVERAGE RATE - MESSAGES PER MINUTE MIDNIGHT-8AM: 36 8AM-5PM: 104 5PM-MIDNIGHT: 50 24 HOUR: 66 TOP 10 BUSIEST MINUTES: 278 Msgs/Min @ 10:27 275 Msgs/Min @ 10:21 263 Msgs/Min @ 10:33 255 Msgs/Min @ 10:32 253 Msgs/Min @ 10:31 251 Msgs/Min @ 10:26 250 Msgs/Min @ 10:23 247 Msgs/Min @ 10:25 246 Msgs/Min @ 10:29 PEAK RAMDISK UTILIZATION: 53% Time of Peak Utilization: 15:30 Ken -Original Message- From: Greg Miller [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 10, 2004 9:05 AM To: [EMAIL PROTECTED] Subject: [Mimedefang] tmpfs on Linux Platform is RedHat EL AS 3 Dual Xeon 3.0 Ghz CPU 1 GB RAM Load is between 1-3 messages per second. System performs very well, most of the time, with only 2-3 busy slaves. However, on occasion, I will see all 15 of my slaves busy, lots of disk I/O to swap, and "Please try again later" messages in the maillog. I am using tmpfs for /var/spool/MIMEDEFANG as recommended in the FAQ. I believe my tmpfs is being sent to swap during these periods and causing horrible performance. Has anyone else seen this behavior? Any solutions out there? Thanks. -- Greg Miller, RHCE, CCNA, MCSE Senior Network Specialist University of Richmond [EMAIL PROTECTED] (804) 289-8546 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] tmpfs on Linux
Platform is RedHat EL AS 3 Dual Xeon 3.0 Ghz CPU 1 GB RAM Load is between 1-3 messages per second. System performs very well, most of the time, with only 2-3 busy slaves. However, on occasion, I will see all 15 of my slaves busy, lots of disk I/O to swap, and "Please try again later" messages in the maillog. I am using tmpfs for /var/spool/MIMEDEFANG as recommended in the FAQ. I believe my tmpfs is being sent to swap during these periods and causing horrible performance. Has anyone else seen this behavior? Any solutions out there? Thanks. -- Greg Miller, RHCE, CCNA, MCSE Senior Network Specialist University of Richmond [EMAIL PROTECTED] (804) 289-8546 ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
