[sniffer] Re: New proactive false positive prevention initiatives
Hi Steve Since this was asked, MxScan for SmarterMail is currently available for Free in beta mode. Cheers -Matt From: Message Sniffer Community [mailto:snif...@sortmonster.com] On Behalf Of Steve Guluk Sent: Friday, February 05, 2010 6:10 AM To: Message Sniffer Community Subject: [sniffer] Re: New proactive false positive prevention initiatives Hey Pete, Is there a hook to use Sniffer in SmarterMail 6? I just had to move to SmarterMail rather than pay over $3k to upgrade iMail to run on a 64bit windows box. I'm using eWall at this point for Message Sniffer but may retire that with iMail. On Feb 4, 2010, at 1:57 PM, Pete McNeil wrote: Hello Sniffer Folks, I thought I would drop you a note to let you know some things we're doing behind the scenes to improve filtering accuracy and prevent false positives. Unqualified false positive candidates: In partnership with our larger customers we have created a new system to proactively review captured messages that _might_ be unreported false positives (usually they are spam, but some aren't). Through this review process we are able to remove and modify pattern rules that cause occasional low-level false positives that would otherwise not be reported. This system is already allowing us to recode or remove dozens of rules per day to make them more accurate; and to update our rule coding practices and support systems to further improve our accuracy moving forward. Real-time rule / IP conflict analysis: Today we have completed a new false-positive early-warning system. This system monitors conflicts between IP reputations and pattern rule matches across the entire fleet of Message Sniffer installations in real-time. Any time a pattern match is in disagreement with a source IP's reputation that information is analyzed and pumped through a sophisticated collection of filters and data-mining tools. The resulting analysis is displayed in real-time in our spam-weather center so that our staff can respond immediately (24x365) if there is any sign of a "bad rule". Since we launched this new system and operating protocols earlier today we have already had several "events" -- All of them turned out to be valid anti-spam rules capturing content from bot nets that had previously sent *berserkers to improve their IP reputations, or where some of the campaigns in question had leaked sufficiently to produce temporary positive IP reputations on some systems. This information itself is very interesting now that we can see it more clearly and we are already working on ways to identify these cases and reduce the leakage associated with them. As always your comments, ideas, and suggestions are both welcome and encouraged. Best, _M PS: *berserkers - Blackhats sometimes send messages that are random and/or carry no payload. These "berserkers", sometimes sent by accident by broken bots or broken spam scripts, have the effect of improving the IP reputations of the systems that send them because there is no sufficient content to filter against. In addition these messages are often sent at such low rates that most adaptive filtering systems fail to respond to them--- if those systems were to be (conventionally) sensitized to the berserkers they would also significantly increase their false-positive rates. We call these berserkers based on the practice of old Norse warriors who, in an uncontrollable state (chaotic, berserk (in a fit of madness), and with the belief they are immune to weapons), would charge directly into the enemies ranks fearlessly attacking anything and everything (friend or foe). http://en.wikipedia.org/wiki/Berserker # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to Regards, Steve Guluk SGDesign (949) 661-9333
[sniffer] Re: New proactive false positive preventioninitiatives
Steve: MxGuard is availabe for SmarterMail now. Eric --Original Message-- From: Pete McNeil Sender: Message Sniffer Community To: Message Sniffer Community ReplyTo: Message Sniffer Community Subject: [sniffer] Re: New proactive false positive preventioninitiatives Sent: Feb 4, 2010 14:25 Steve Guluk wrote: > Hey Pete, > Is there a hook to use Sniffer in SmarterMail 6? I haven't looked closely at SM6,... there may be something new. However, eWall will still work. Also MXGuard and Declude (Declude just integrated SNF directly). Also it is possible to run SNF as a command line scanner in SM, though most are not happy with that solution. If their SpamAssassin support has improved you _might_ be able to use SNF4SA -- last I heard it was not possible to add plugins, that may have changed. If you have a resolver setup for your mail system (you should) then you might also try our truncate bl to block connections -- let me know if you're interested in trying that. If there are newer better ways to integrate I'd love to know about them. Best, _M # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to Sent from my BlackBerry® using speech recognition so may be brief and may contain errors. Please don't hesitate to ask for confirmation if anything seems incomplete or innacurate. E&OE. # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: New proactive false positive prevention initiatives
Steve Guluk wrote: Hey Pete, Is there a hook to use Sniffer in SmarterMail 6? I haven't looked closely at SM6,... there may be something new. However, eWall will still work. Also MXGuard and Declude (Declude just integrated SNF directly). Also it is possible to run SNF as a command line scanner in SM, though most are not happy with that solution. If their SpamAssassin support has improved you _might_ be able to use SNF4SA -- last I heard it was not possible to add plugins, that may have changed. If you have a resolver setup for your mail system (you should) then you might also try our truncate bl to block connections -- let me know if you're interested in trying that. If there are newer better ways to integrate I'd love to know about them. Best, _M # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
[sniffer] Re: New proactive false positive prevention initiatives
Hey Pete, Is there a hook to use Sniffer in SmarterMail 6? I just had to move to SmarterMail rather than pay over $3k to upgrade iMail to run on a 64bit windows box. I'm using eWall at this point for Message Sniffer but may retire that with iMail. On Feb 4, 2010, at 1:57 PM, Pete McNeil wrote: > Hello Sniffer Folks, > > I thought I would drop you a note to let you know some things we're doing > behind the scenes to improve filtering accuracy and prevent false positives. > > Unqualified false positive candidates: > > In partnership with our larger customers we have created a new system to > proactively review captured messages that _might_ be unreported false > positives (usually they are spam, but some aren't). Through this review > process we are able to remove and modify pattern rules that cause occasional > low-level false positives that would otherwise not be reported. This system > is already allowing us to recode or remove dozens of rules per day to make > them more accurate; and to update our rule coding practices and support > systems to further improve our accuracy moving forward. > > Real-time rule / IP conflict analysis: > > Today we have completed a new false-positive early-warning system. This > system monitors conflicts between IP reputations and pattern rule matches > across the entire fleet of Message Sniffer installations in real-time. Any > time a pattern match is in disagreement with a source IP's reputation that > information is analyzed and pumped through a sophisticated collection of > filters and data-mining tools. The resulting analysis is displayed in > real-time in our spam-weather center so that our staff can respond > immediately (24x365) if there is any sign of a "bad rule". > > Since we launched this new system and operating protocols earlier today we > have already had several "events" -- All of them turned out to be valid > anti-spam rules capturing content from bot nets that had previously sent > *berserkers to improve their IP reputations, or where some of the campaigns > in question had leaked sufficiently to produce temporary positive IP > reputations on some systems. This information itself is very interesting now > that we can see it more clearly and we are already working on ways to > identify these cases and reduce the leakage associated with them. > > As always your comments, ideas, and suggestions are both welcome and > encouraged. > > Best, > > _M > > PS: *berserkers - Blackhats sometimes send messages that are random and/or > carry no payload. These "berserkers", sometimes sent by accident by broken > bots or broken spam scripts, have the effect of improving the IP reputations > of the systems that send them because there is no sufficient content to > filter against. In addition these messages are often sent at such low rates > that most adaptive filtering systems fail to respond to them--- if those > systems were to be (conventionally) sensitized to the berserkers they would > also significantly increase their false-positive rates. > > We call these berserkers based on the practice of old Norse warriors who, in > an uncontrollable state (chaotic, berserk (in a fit of madness), and with the > belief they are immune to weapons), would charge directly into the enemies > ranks fearlessly attacking anything and everything (friend or foe). > > http://en.wikipedia.org/wiki/Berserker > > > > # > This message is sent to you because you are subscribed to > the mailing list . > This list is for discussing Message Sniffer, > Anti-spam, Anti-Malware, and related email topics. > For More information see http://www.armresearch.com > To unsubscribe, E-mail to: > To switch to the DIGEST mode, E-mail to > To switch to the INDEX mode, E-mail to > Send administrative queries to > > Regards, Steve Guluk SGDesign (949) 661-9333
[sniffer] New proactive false positive prevention initiatives
Hello Sniffer Folks, I thought I would drop you a note to let you know some things we're doing behind the scenes to improve filtering accuracy and prevent false positives. Unqualified false positive candidates: In partnership with our larger customers we have created a new system to proactively review captured messages that _might_ be unreported false positives (usually they are spam, but some aren't). Through this review process we are able to remove and modify pattern rules that cause occasional low-level false positives that would otherwise not be reported. This system is already allowing us to recode or remove dozens of rules per day to make them more accurate; and to update our rule coding practices and support systems to further improve our accuracy moving forward. Real-time rule / IP conflict analysis: Today we have completed a new false-positive early-warning system. This system monitors conflicts between IP reputations and pattern rule matches across the entire fleet of Message Sniffer installations in real-time. Any time a pattern match is in disagreement with a source IP's reputation that information is analyzed and pumped through a sophisticated collection of filters and data-mining tools. The resulting analysis is displayed in real-time in our spam-weather center so that our staff can respond immediately (24x365) if there is any sign of a "bad rule". Since we launched this new system and operating protocols earlier today we have already had several "events" -- All of them turned out to be valid anti-spam rules capturing content from bot nets that had previously sent *berserkers to improve their IP reputations, or where some of the campaigns in question had leaked sufficiently to produce temporary positive IP reputations on some systems. This information itself is very interesting now that we can see it more clearly and we are already working on ways to identify these cases and reduce the leakage associated with them. As always your comments, ideas, and suggestions are both welcome and encouraged. Best, _M PS: *berserkers - Blackhats sometimes send messages that are random and/or carry no payload. These "berserkers", sometimes sent by accident by broken bots or broken spam scripts, have the effect of improving the IP reputations of the systems that send them because there is no sufficient content to filter against. In addition these messages are often sent at such low rates that most adaptive filtering systems fail to respond to them--- if those systems were to be (conventionally) sensitized to the berserkers they would also significantly increase their false-positive rates. We call these berserkers based on the practice of old Norse warriors who, in an uncontrollable state (chaotic, berserk (in a fit of madness), and with the belief they are immune to weapons), would charge directly into the enemies ranks fearlessly attacking anything and everything (friend or foe). http://en.wikipedia.org/wiki/Berserker # This message is sent to you because you are subscribed to the mailing list . This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: To switch to the DIGEST mode, E-mail to To switch to the INDEX mode, E-mail to Send administrative queries to
