Keith,
Windows DNS service will handle over a million lookups a day without blinking. There should be no reason to switch to a different DNS server. It hardly even registers any CPU load on my boxes. The biggest CPU hog is the virus scanners, and choosing your virus scanners carefully will have a great benefit. F-Prot is the champ followed by ClamAV in daemon mode (the non-daemon is a hog), followed by McAfee at a distant third, though there are many others that are far worse. The AVAFTERJM switch will stop most messages from being virus scanned and hence the magic there, however if you don't delete any messages with JunkMail there is no real advantage. I'm not clear on whether or not the ROUTETO will bypass scanning, but you could create some filters using VBScript to tag messages with attachments associated with viruses and handle them differently.
Personally, I haven't found a huge impact from running Sniffer in persistent mode, but it does have a slightly measurable effect on my server. If you are hurting for disk I/O or memory, this could help immensely. If you are running into an issue with disk I/O, it could back things up significantly. Also, if you have any domains where the addresses aren't validated (nobody aliases or gateway domains), this could easy be attacked in such a way so that it overwhelmed your server. We are presently only validating for about 2/3 of our customer base and this morning the address validation software/service failed an automatic restart and it allowed everything through to IMail/Declude and it pegged our server at 100% until it was turned back on. Normally at that time of day, our server runs at an average of about 25% (and it will get better when the other 1/3 becomes validated).
BODY and ANYWHERE filters in Declude can also be huge hogs if you don't limit them to a reasonable level. I probably have about 1,500 lines of BODY filters and that isn't causing me any real issues but I am also using SKIPIFWEIGHT and other methods of skipping such filters when it isn't beneficial to run them. Managing my Declude filtering better definitely helped me steal back some CPU.
Placing Sniffer in persistent mode definitely shouldn't cause things to slow down unless maybe it was configured improperly. I use the same SERVANY setup that you said that you are using and it has worked flawlessly for me since the day that Pete released that functionality. I am thinking that you might want to scrutinize your setup.
Hope that this helps.
Matt
Keith Johnson wrote:
Pete, Wow, thank you for the explanation. I did let the persistent server run for 30 min after I restarted the services. However, I did stop the services, then started Sniffer service, then restart Imail services. I could have gotten a backlog of retries at that moment that pegged the CPU as you stated. We have batted around running BIND for NT/2000 on the local machine, but my fear was overhead of another major process running. I don't have any good stats on how much CPU/Memory BIND on an Imail Server requires, thus, we have a SUN/BIND box local to the switch. Are you aware of any stats on this?
We don't run the AVAFTERJM switch. This is done in part due to
so many of our customers still look at their spam email from time to
time. We heavily use the ROUTETO and MAILBOX command, thus, if I let a
virus go through to their to mailbox, they could potentially open a
virus spam email and hurt themselves.
We defrag each partition every night using Diskeeper and it works great. I regularly look at the Sniffer directory to ensure no left over .fin files and others that could cause server load. I will retry it again tonight and see what type of results I get and post them here. It could be as you say, I am on the far side :)
Thanks again,
Keith
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Friday, April 01, 2005 2:16 PM To: Keith Johnson Subject: Re[4]: [sniffer] Persistent Sniffer
On Friday, April 1, 2005, 11:44:07 AM, Keith wrote:
KJ> Pete,
KJ> Thanks for the reply.
KJ> Running on an IBM Xseries 225 Dual Xeon 2.4Ghz w/ 1GB RAM - KJ> running IBM's ServerRAID 5i in IBM's RAID 10 config (4 73GB 10K KJ> drives)
KJ> - O/S is Windows 2000 Standard Server SP4
KJ> Running Imail 8.15HF1 with Declude JM/Virus 1.82 - BIND DNS KJ> Server is 1 hop away (on switch backbone). I had to drop back to KJ> the non-persistent mode, thus the .stat file disappeared. I will KJ> run it again tonight and copy the file away and post it here
tonight.
KJ> Thanks again for the time and aid.
I don't see any problems with this setup.
Your description sounds like your server is fairly heavily loaded (35-55% cpu in peer-server mode), though I would expect more from the hardware you've described.
I suspect that you may have run into the far side of the power curve when you went to persistent server mode. In peer-server mode the failure mode for overload conditions is much softer than with the persistent peer server mode.
Up to the failure point in the power curve the persistent server mode will provide a significant savings over peer-server, however once that point is reached the persistent server mode tends to degrade much more quickly and requires a significant drop in load before recovery occurs.
I'm working on some strategies to soften that curve a bit, but in the mean time let's explore these options to get the best performance from your server and reduce it's load. The we can see if the persistent server engine will give you even more headroom:
1. I recommend running AVAFTERJM - are you doing this? Typically 80% or more of email traffic is spam and so there is no good reason to attempt a virus scan on these messages. If you hold messages and occasionally re-insert them into the queue then they will not be scanned, however there are ways to work around this when needed - and it is very likely you would not re-insert a message that contained a virus anyway.
2. Consider running bind as a dns resolver on your mail server and pointing the server to itself via the loopback address (127.0.0.1) for DNS services. This tends to speed up processing significantly which also reduces the number of message processes that are running at any given time. YMMV, but I have seen this work consistently to improve performance.
--- when trying persistent mode (minor adjustments really) ---
A. Set the Persistence value in your snflicid.cfg file to 3600. - no need to check for a new rulebase every 10 minutes usually. These loop events tear down the server momentarily which can perturb an otherwise smooth running system when under heavy loads - thus minimizing the frequency of these events may help.
B. Set LogFormat in your snflicid.cfg file to SingleLine. This provides sufficient data for our purposes (most of the time) and should significantly reduce the size of your log file.
C. Be sure to keep any unnecessary files out of the SNF working directory - in particular you should clean out any orphaned files that might still be lurking from previous crashes.
--- General ---
Be sure your drives are regularly defragmented.
Hope this helps,
_M
PS: I just had another random thought really --- Could it be that the high CPU value was appropriate? If you had built up a queue of messages to be processed then once the persistent server was put in place and the system started processing messages again the CPU would probably be much higher for a period of time until all of the backlog had been eliminated. The persistent server would do its best to "nail up" at least one of the CPUs until this was accomplished, so looking at the CPU load during that period might not be the best way to understand the situation. The CPU load would not drop back down again until the backlog had be eliminated.
In comparison to the persistent server mode, the peer-server mode imposes a greater restriction on message throughput and puts a higher load on IO due to repeatedly loading the rulebase file. This can have the effect of reducing the overall CPU load at the expense of raw throughput under some circumstances. This, in fact, explains why the peer-server mode has a softer performance failure curve than the persistent server mode (in theory). Put another way, sometimes the peer-server mode prevents the CPU from getting out of it's own way to scan the messages - so the CPU load looks lower because it spends more time waiting. In these cases putting the persistent server in place has the effect of removing the obstacles so the CPU works harder and the messages go through faster.
A better way to judge might be to check the overflow queue... the rate at which it is being emptied (or slowness at which it grows) would indicate a better throughput and that is probably the better goal.
-- just a random thought.
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
-- ===================================================== MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =====================================================
This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
