Hello Sniffer Folks, <http://www.sortmonster.com/MessageSniffer/Betas/MessageSniffer2-3.0i8-Distribution.zip>
This version radically adjusts the client timing when a persistent server is detected. This new logic avoids a potentially catastrophic failure when a system is under EXTREMELY heavy loads. The prior versions used a standard peer-server timing model with the persistent server. This was ok while the persistent server captured a single job at a time. The enhanced persistent server captures all available jobs in a batch and then processes all of them before making it's next "loop" through the workspace. As a result, a system under extremely heavy loads might reach a condition where the batch processing or "loop" time approaches the length of the standard "Client Death Clock" - - about 8 seconds. If this condition occurs some of the clients will "defect" and abandon the server to process their own jobs. This causes the overloaded server to process work more slowly so that additional batches also take 8 or more seconds thus causing additional clients to defect until the entire advantage of the persistent server is lost and converted to an additional delay in processing. Under these conditions an overloaded server is usually unable to recover without stopping all message processing. The new timing model allows clients to wait for 30 seconds if they have detected a persistent server. As a result the persistent server is always able to process a batch before any clients will defect. In testing on our server we seldom see a batch take longer than 10 seconds even under the most extreme conditions. At this point the operations on the server become slow enough that additional jobs arrive at a much reduced rate so that a natural limiting occurs. After such a spike the server recovers, even though it is obviously overloaded. (Our testing server is a P2/450 w/ 256KRam, 2x IDE33 HD in software (NT4) mirror raid, processing about 30K messages per day - almost all spam. Nominally Message Sniffer processes a message in under 200ms on this server, however under heavy loads the processing time can become quite large. Also, this system uses the most aggressive rulebase settings. After the upgrade to i8 on this server it has sustained a throughput of 42msg/sec for approximately 6 hours now (still going) without reaching a breakdown condition and while remaining characteristically responsive to other requests (pop3, imap, webmsg... 42ms/sec is about 60K msg/day if sustained. The additional load in this case is due to our gateway server unloading a heavy backlog of messages after some adjustments to it's queue handling characteristics.) It is unlikely that production mail servers will be operated in such extreme conditions but now if they are Message Sniffer is ready to deal with them gracefully. After all, planned or not - these things do happen and critical servers should not fail no matter the conditions (if possible). This version has taken a real beating on our server today. Please let us know how it does on your systems. I am anxious to move forward with the next round of updates to the code and then finally to publish this new version as production ready once enough positive reports have been received. I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. I look forward to any and all comments. Thanks, _M Pete McNeil (Madscientist) President, MicroNeil Research Corporation Chief SortMonster (www.sortmonster.com) PS: The remaining updates for this next version (2-3.1) will be the addition of minor filtering capabilities that will largely be of internal use, such as masking rules, and some message condition tagging and image/file hashing functions. This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
