Re: [sniffer] Increase in FPs

[Pete McNeil]

On Wednesday, September 15, 2004, 2:06:22 AM, Landry wrote:

LW> I have seen a fairly substantial increase on false positives
LW> today.  I have submitted several FPs to the false@ address.  Has
LW> there been a big change in the core rulebase today?  I wouldn't
LW> think that upgrading to the new code this morning would cause
LW> this, would it?

No, the upgrade should not have this effect.
It appears that a number of secondary services we reference have had
problems recently such as SORBS and SURBL. I've been pushing false
processing to mitigate the problems quickly, we are adjusting our
tuning parameters for candidate generation, and will continue to
monitor conditions closely.

_M




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

[sniffer] Increase in FPs

[Landry William]

Title: Increase in FPs





I have seen a fairly substantial increase on false positives today.  I have submitted several FPs to the false@ address.  Has there been a big change in the core rulebase today?  I wouldn't think that upgrading to the new code this morning would cause this, would it?

Bill




---This message and any included attachments are from Siemens Medical Solutions USA, Inc. and are intended only for the addressee(s).  The information contained herein may include trade secrets or privileged or otherwise confidential information.  Unauthorized review, forwarding, printing, copying, distributing, or using such information is strictly prohibited and may be unlawful.  If you received this message in error, or have reason to believe you are not authorized to receive it, please promptly delete this message and notify the sender by e-mail with a copy to [EMAIL PROTECTED] Thank you

Re[6]: [sniffer] Surprising missed spam

[Pete McNeil]

On Tuesday, September 14, 2004, 4:06:47 PM, Jonathan wrote:

JH> How does a user go about modifying the custom sniffer rules?  Must Sort
JH> Monster be contacted or is it possible to do this with some other system
JH> (such as a web based interface)?

The "normal" way right now is to work through us. Rulebase adjustments
can be complicated, so it is usually best if we can coordinate the
effort.

We do have a web based application which can be used by some advanced
users with special training but it is not available generally.

We also have a Java based utility which allows rulebase updates
through XML files. (RESCU = REmote SCripted Updates)

Both the online application and the use of our RescU utility are
considered experimental and generally require additional support
costs.

If you have something specific in mind, please contact us at [EMAIL PROTECTED]
I will work through your plans with you and help to develop a solid
plan that will work for you.

In general though, most of the adjustments anyone needs are handled
well through our false positive process or occasionally by special
request to [EMAIL PROTECTED]

Hope this helps,
_M




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

RE: Re[4]: [sniffer] Surprising missed spam

[Jonathan Hickman]

How does a user go about modifying the custom sniffer rules?  Must Sort
Monster be contacted or is it possible to do this with some other system
(such as a web based interface)?

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil
Sent: Tuesday, September 14, 2004 3:28 PM
To: Landry William
Subject: Re[4]: [sniffer] Surprising missed spam


On Tuesday, September 14, 2004, 1:05:29 PM, Landry wrote:


LW> Pete, I started running the new code this morning, and so far, so 
LW> good. I'll let you know if I see anything strange.

Thanks.
_M




This E-Mail came from the Message Sniffer mailing list. For information
and (un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html


This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

[sniffer] On the edge... Anybody try Message Sniffer on Mac OS X yet?

[Pete McNeil]

Hello Sniffer folks,

  I'm curious if anybody has tried compiling and running Message
  Sniffer on a Mac yet? Since OS-X is bsd? based this should be an
  easy thing to do.

  I know it's rare, but how rare is it that folks will use a Mac for
  an email server? I've had clients do it in the past - usually video
  production houses though.

  Any info welcome.

Thanks,
_M

Pete McNeil (Madscientist)
President, MicroNeil Research Corporation
Chief SortMonster (www.sortmonster.com)



This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re: [sniffer] Surprising missed spam

[Matt]

Actually, we scan for many businesses as well as home users, and have
clients with mail boxes on every continent except Antarctica.  To me
it's really a matter of what classifies spam, and while these phrases
are spammy, they are not accurate enough to use in my rulebase.  Pete
knows what he is doing however, and you will note that most of his
rules are based on 'payload' hits, which are generally links.  Without
a payload, the message is merely a statement, and while that has
happened (Nazi spamdemic), it is not the norm.  These guys do change
their payloads around regularly, but the ones that use these sorts of
phrases in spam are highly likely to also get tagged by other
obfuscation techniques in Sniffer.  Of course there are also many
blacklists that are good at tagging both zombie and static spam sources.

My point was really that I prefer to tag spam based on a positive hit
instead of a suggestive one, and for the most part, Sniffer does this. 
It is especially effective in combination with other spam blocking
techniques.  If for instance you have 3 hits on perfectly unassociated
patterns, and each one is 99% accurate, or rather 1% inaccurate, the
net result is that the combination of hits would produce a false
positive rate 0.0001%.  A good example of this would be a message that
is tagged by Sniffer for a link in the body, tagged by SpamCop for
leaking spam by the IP, and forges the Mail From domain.  Unfortunately
I do see false positives frequently enough when Sniffer hits in
combination with some other less accurate test giving it enough points
to be held on my system, many of which might fall into a gray category
or results from a more generic/suggestive hit in combination with some
technical shortcoming.

Spam bothers me a whole bunch, that's why I'm in the business, but
false positives bother me even more.  I do wish that over time Pete
could further separate his rules into more positive and more suggestive
ones so that things like known URL's would be examples of more positive
ones and things like "horny teenagers" would be an example of a
suggestive one.  Given that, I could weight accordingly.

Matt



Agid, Corby wrote:

  
  
  
  I suppose everyone's
userbases have differenent requirements.  An ISP or private
enterprise might worry about false postives on "horny teenagers" and
"penis enlargement", but for our local government agency, it causes
problems.  
   
  Corby
  
  
  

 From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
Behalf Of Matt
Sent: Monday, September 13, 2004 5:25 PM
To: [EMAIL PROTECTED]
Subject: Re: [sniffer] Surprising missed spam


Corby,

Personally, I'm a fan of leaving the generic stuff out due to the
potential of false positives.  Those of us that are using Sniffer in
addition to other spam blocking mechanisms can afford to lose some
Sniffer hits on such phrases because they will be picked up by other
means almost all of the time.  Including such phrases however would
increase our false positive rate without a measurable benefit in spam
capture rates.  I have even asked Pete to remove some phrase hits from
my own rulebase for exactly this reason.

Matt



Agid, Corby wrote:

  

  Hello, 
  I was surprised recently by some
spam that got through without getting caught by the sniffer.   We've
been getting some plain text messages that have obvious spam words in
the subject line.   For example, a plain text message with "horny
teenagers" came through.  The content was also very spammy, but all
plain text.   I tried sending myself a few messages with standard spam
phrases and none of them tripped any sniffer rules.
  Am I missing something? 
  Corby 


-- 
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=
  


-- 
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=

Re[4]: [sniffer] Surprising missed spam

[Pete McNeil]

On Tuesday, September 14, 2004, 1:05:29 PM, Landry wrote:


LW> Pete, I started running the new code this morning, and so far, so good.
LW> I'll let you know if I see anything strange.

Thanks.
_M




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re: [sniffer] Group 62

[Pete McNeil]

On Tuesday, September 14, 2004, 12:40:43 PM, Jorge wrote:

JA> What is Group 62? Is there anywhere I can get a list of all group types?



62 - Abstract patterns for spam structures.

This group also contains some domain rules that are generated
automatically from our spamtraps.

_M




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

RE: Re[2]: [sniffer] Surprising missed spam

[Landry William]

Pete, I started running the new code this morning, and so far, so good.
I'll let you know if I see anything strange.

Bill

-Original Message-
From: Pete McNeil [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, September 14, 2004 8:56 AM
To: Agid, Corby
Subject: Re[2]: [sniffer] Surprising missed spam


On Tuesday, September 14, 2004, 11:41:48 AM, Corby wrote:

AC>  To which addresss should I send these?

AC> Also, I mis-stated the spam.  They were not plain text, but html, 
AC> but clearly have many "classic" spam attributes.  I will send them 
AC> along, but need to know where.

Please zip them and send them to support@

However, before you do this you might consider upgrading to the latest
interim release. We had another report like yours that was "apparently"
solved by the newest update (V2-3.0i2). It might be worth trying this first
to see if it solves the problem.

Please keep us posted.

Thanks,
_M




This E-Mail came from the Message Sniffer mailing list. For information and
(un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html

---
This message and any included attachments are from Siemens Medical Solutions 
USA, Inc. and are intended only for the addressee(s).  
The information contained herein may include trade secrets or privileged or 
otherwise confidential information.  Unauthorized review, forwarding, printing, 
copying, distributing, or using such information is strictly prohibited and may 
be unlawful.  If you received this message in error, or have reason to believe 
you are not authorized to receive it, please promptly delete this message and 
notify the sender by e-mail with a copy to [EMAIL PROTECTED] 

Thank you

This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re: [sniffer] Group 62

[John Cline]

JA> What is Group 62? Is there anywhere I can get a list of all group types?


http://www.sortmonster.com/MessageSniffer/Help/ResultCodesHelp.html

-- 
John M. Cline, LTC, USAFR  "Never Give Up!"




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

[sniffer] Group 62

[Jorge Asch]

What is Group 62? Is there anywhere I can get a list of all group types?
--
Jorge Asch Revilla
CONEXION DCR
www.conexion.co.cr
800-CONEXION

This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re[2]: [sniffer] Surprising missed spam

[Pete McNeil]

On Tuesday, September 14, 2004, 11:48:43 AM, Corby wrote:

AC> I suppose everyone's userbases have differenent
AC> requirements.  An ISP or private enterprise might worry about
AC> false postives on "horny teenagers" and "penis enlargement", but
AC> for our local government agency, it causes problems.  
AC>  
AC> Corby

This is why each user's rule base can be customized. If you have
requirements for additional black-rules then we can work with you to
create them.

Each rule base can be customized by blocking rules from the core,
adding local white rules, and adding local black rules. (New rule
types are also on the way.)

The end result is a customized version of our core rulebase for each
license ID.

_M



This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re[2]: [sniffer] Surprising missed spam

[Pete McNeil]

On Tuesday, September 14, 2004, 11:41:48 AM, Corby wrote:

AC>  To which addresss should I send these?

AC> Also, I mis-stated the spam.  They were not plain text, but
AC> html, but clearly have many "classic" spam attributes.  I will
AC> send them along, but need to know where.

Please zip them and send them to support@

However, before you do this you might consider upgrading to the latest
interim release. We had another report like yours that was
"apparently" solved by the newest update (V2-3.0i2). It might be worth
trying this first to see if it solves the problem.

Please keep us posted.

Thanks,
_M




This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

Re: [sniffer] Surprising missed spam

[Jim Matuska]

I just forwarded half a dozen myself, they have been coming in for the last 
week or so, much more so than before.

Jim Matuska Jr.
Computer Tech II
CCNA
Nez Perce Tribe
Information Systems
[EMAIL PROTECTED]
- Original Message - 
From: "Agid, Corby" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, September 14, 2004 8:41 AM
Subject: RE: [sniffer] Surprising missed spam

To which addresss should I send these?
Also, I mis-stated the spam.  They were not plain text, but html, but 
clearly have many "classic" spam attributes.  I will send them along, but 
need to know where.


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil
Sent: Monday, September 13, 2004 4:29 PM
To: Agid, Corby
Subject: Re: [sniffer] Surprising missed spam
On Monday, September 13, 2004, 7:22:03 PM, Corby wrote:
AC> Hello,
AC> I was surprised recently by some spam that got through without
AC> getting caught by the sniffer. We've been getting some
plain text
AC> messages that have obvious spam words in the subject line. For
AC> example, a plain text message with "horny teenagers"
AC> came through. The content was also very spammy, but all
plain text.
AC> I tried sending myself a few messages with standard spam
phrases and
AC> none of them tripped any sniffer rules.
AC> Am I missing something?
Can you zip up some examples and send them to me?
I'm researching this issue right now and I need more data.
Thanks,
_M
PS: A number of word / phrase based rules have been dropped
from the core rule base due to false positives - not many,
but this might explain some of what you're seeing - I will
know more when I have some examples. If that's the case I can
always put the rules back in for your local rule base.


This E-Mail came from the Message Sniffer mailing list. For
information and (un)subscription instructions go to
http://www.sortmonster.com/MessageSniffer/Help/Help.html

This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html

This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html

RE: [sniffer] Surprising missed spam

[Agid, Corby]

I suppose everyone's userbases have differenent 
requirements.  An ISP or private enterprise might worry about false 
postives on "horny teenagers" and "penis enlargement", but for our local 
government agency, it causes problems.  
 
Corby


  
  
  From: [EMAIL PROTECTED] 
  [mailto:[EMAIL PROTECTED] On Behalf Of 
  MattSent: Monday, September 13, 2004 5:25 PMTo: 
  [EMAIL PROTECTED]Subject: Re: [sniffer] Surprising missed 
  spam
  Corby,Personally, I'm a fan of leaving the generic stuff 
  out due to the potential of false positives.  Those of us that are using 
  Sniffer in addition to other spam blocking mechanisms can afford to lose some 
  Sniffer hits on such phrases because they will be picked up by other means 
  almost all of the time.  Including such phrases however would increase 
  our false positive rate without a measurable benefit in spam capture 
  rates.  I have even asked Pete to remove some phrase hits from my own 
  rulebase for exactly this reason.MattAgid, Corby 
  wrote:
  

Hello, 
I was surprised recently by some spam that got 
through without getting caught by the sniffer.   We've been 
getting some plain text messages that have obvious spam words in the subject 
line.   For example, a plain text message with "horny teenagers" 
came through.  The content was also very spammy, but all plain 
text.   I tried sending myself a few messages with standard spam 
phrases and none of them tripped any sniffer rules.
Am I missing something? 
Corby -- 
=
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=

RE: [sniffer] Surprising missed spam

[Agid, Corby]

 To which addresss should I send these?

Also, I mis-stated the spam.  They were not plain text, but html, but clearly have 
many "classic" spam attributes.  I will send them along, but need to know where.



> -Original Message-
> From: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil
> Sent: Monday, September 13, 2004 4:29 PM
> To: Agid, Corby
> Subject: Re: [sniffer] Surprising missed spam
> 
> On Monday, September 13, 2004, 7:22:03 PM, Corby wrote:
> 
> AC> Hello,
> 
> AC> I was surprised recently by some spam that got through without 
> AC> getting caught by the sniffer.   We've been getting some 
> plain text 
> AC> messages that have obvious spam words in the subject line.   For 
> AC> example, a plain text message with "horny teenagers"
> AC> came through.  The content was also very spammy, but all 
> plain text.   
> AC> I tried sending myself a few messages with standard spam 
> phrases and 
> AC> none of them tripped any sniffer rules.
> 
> AC> Am I missing something?
> 
> Can you zip up some examples and send them to me?
> I'm researching this issue right now and I need more data.
> 
> Thanks,
> _M
> 
> PS: A number of word / phrase based rules have been dropped 
> from the core rule base due to false positives - not many, 
> but this might explain some of what you're seeing - I will 
> know more when I have some examples. If that's the case I can 
> always put the rules back in for your local rule base.
> 
> 
> 
> 
> 
> 
> This E-Mail came from the Message Sniffer mailing list. For 
> information and (un)subscription instructions go to 
> http://www.sortmonster.com/MessageSniffer/Help/Help.html
> 
> 

This E-Mail came from the Message Sniffer mailing list. For information and 
(un)subscription instructions go to 
http://www.sortmonster.com/MessageSniffer/Help/Help.html