RE: Sorry for the duplicate messages
> > The last two messages I sent were duplicated on this list. I'm not sure > why and I hope this one isn't duplicated. I'm using Exim and I'm only > seeing one entry in my log. If anyone knows what might be causing this > I'd be more than happy to fix the problem. Until then, I apologize for > the dups. > > Marc Don't feel bad, the list software allowed me to post from an email address I did not subscribe Thing is, I had to unsubscribe that email address. What is going on behind the scenese? :-) - rh
Sorry for the duplicate messages
The last two messages I sent were duplicated on this list. I'm not sure why and I hope this one isn't duplicated. I'm using Exim and I'm only seeing one entry in my log. If anyone knows what might be causing this I'd be more than happy to fix the problem. Until then, I apologize for the dups. -- Marc Perkel - Sales/Support [EMAIL PROTECTED] http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3401
Re: Postfix-Procmail-Spamassassin duplicate messages
From: "Jacob Cord" <[EMAIL PROTECTED]>
Hi everyone,
I've had an awful time trying to figure this one out on my own, and have
been scouring the 'net for someone with a similar problem, but can't
find anything.
I'm converting an existing Postfix (2.0.6)/Procmail setup to use
SpamAssassin. I installed SpamAssassin via CPAN.
I followed Greg Webster's howto at
http://www.geekly.com/entries/archives/0155.htm and got things up
and running.
Then I noticed that for messages tagged as spam, I get two messages
delivered for every one received by the system. The headers are
identical, I just get two copies of the spam in my inbox. This happens
system-wide.
Here are my configuration details:
/etc/mail/spamassassin/local.cf:
rewrite_header Subject [SPAM]
report_safe 2
trusted_networks my.ip.addrs.
lock_method flock
required_score 8.0
use_bayes 1
bayes_auto_learn 1
ok_languagesen
ok_locales en
/etc/postfix/master.cf
#services:
smtp inet n - n - - smtpd
-o content_filter=spamfilter:
#interfaces:
spamfilter unix - n n - - pipe
flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter.sh -f
${sender} -- ${recipient}
/etc/postfix/main.cf
mailbox_command = /usr/bin/procmail -Y -a $DOMAIN
/usr/local/bin/spamfilter.sh:
#!/bin/bash
/usr/bin/spamc | /usr/sbin/sendmail -i "$@"
exit $?
Why are you not simply running spamc from procmail?
{o.o}
Postfix-Procmail-Spamassassin duplicate messages
Hi everyone,
I've had an awful time trying to figure this one out on my own, and have
been scouring the 'net for someone with a similar problem, but can't
find anything.
I'm converting an existing Postfix (2.0.6)/Procmail setup to use
SpamAssassin. I installed SpamAssassin via CPAN.
I followed Greg Webster's howto at
http://www.geekly.com/entries/archives/0155.htm and got things up
and running.
Then I noticed that for messages tagged as spam, I get two messages
delivered for every one received by the system. The headers are
identical, I just get two copies of the spam in my inbox. This happens
system-wide.
Here are my configuration details:
/etc/mail/spamassassin/local.cf:
rewrite_header Subject [SPAM]
report_safe 2
trusted_networks my.ip.addrs.
lock_method flock
required_score 8.0
use_bayes 1
bayes_auto_learn 1
ok_languagesen
ok_locales en
/etc/postfix/master.cf
#services:
smtp inet n - n - - smtpd
-o content_filter=spamfilter:
#interfaces:
spamfilter unix - n n - - pipe
flags=Rq user=spamfilter argv=/usr/local/bin/spamfilter.sh -f
${sender} -- ${recipient}
/etc/postfix/main.cf
mailbox_command = /usr/bin/procmail -Y -a $DOMAIN
/usr/local/bin/spamfilter.sh:
#!/bin/bash
/usr/bin/spamc | /usr/sbin/sendmail -i "$@"
exit $?
I can provide the full files if necessary, I removed what I thought was
irrelevant to keep this message short.
Any ideas?
Thanks,
Jacob Cord
Re: detecting duplicate messages
At 02:49 AM 1/23/2005, Ron E. wrote: Sounds interesting, but I thought DCC, like pyzor/razor simply checks message content against known spam databases? How would one use it to detect duplicate messages? Any idea? Technicaly, DCC does not detect the "spamminess" of a message, it's just how common a message is, with no consideration if it's common because it is spam vs common nonspam. Whitelists are used to reduce some obvious nonspam hits, but in general DCC does occasionally fire off on high-volume nonspam mailings, such as vendor announcements from Intel... (Then again, despite it's claims of being spam only, razor seems to FP much more often in the past couple years that I've been using both) Every non-whitelisted message that goes into the DCC client for check is also is reported to the DCC server by default. You actually need to pass a parameter to dccproc in order for it to NOT report the message. The server provides a count of messages with the same hash. All you need to do is run a local DCC server and configure the DCC client to have a pretty low count threshold and you'll be detecting duplicate messages.
Re: detecting duplicate messages
On Sat, 22 Jan 2005, Matt Kettler wrote: > At 04:35 PM 1/22/2005, Ron E. wrote: > > I have spent some time searching for something I assumed existed, > >which is a method of detecting when large quantities of the same message, > >or messages with nearly the same content (body & subject) are passing > >through an MTA within a specified time period. It seems to me this could > >be a useful way to detect not only spam but other types of problems as > >well - error conditions, mail bombs, etc. > > Hmm, DCC works very well for this kind of thing, and you could run it as a > local-only server. > > Sounds interesting, but I thought DCC, like pyzor/razor simply checks message content against known spam databases? How would one use it to detect duplicate messages? Any idea? Thanks
Re: detecting duplicate messages
At 04:35 PM 1/22/2005, Ron E. wrote: I have spent some time searching for something I assumed existed, which is a method of detecting when large quantities of the same message, or messages with nearly the same content (body & subject) are passing through an MTA within a specified time period. It seems to me this could be a useful way to detect not only spam but other types of problems as well - error conditions, mail bombs, etc. Hmm, DCC works very well for this kind of thing, and you could run it as a local-only server.
Re: detecting duplicate messages
You might want to check www.ix.de/nixspam Their system calculates a checksum from, among others, the number of words per line Wolfgang Hamann >> >> >> Dear All, >> >> I have spent some time searching for something I assumed existed, >> which is a method of detecting when large quantities of the same message, >> or messages with nearly the same content (body & subject) are passing >> through an MTA within a specified time period. It seems to me this could >> be a useful way to detect not only spam but other types of problems as >> well - error conditions, mail bombs, etc. Clearly, one could not block >> such messages until a certain number of them had already been delivered, >> but still it could be useful. A whitelist function would obviously be >> used to allow legitimate traffic, but otherwise a threshold could be set >> and when enough messages with the same or mostly the same body content >> are detected, any further could be quarantined or tagged. >> >> Just curious if anyone knows of any method of doing this, either >> with spamassassin or with another tool. >> >> Thanks. >> >> >>
detecting duplicate messages
Dear All, I have spent some time searching for something I assumed existed, which is a method of detecting when large quantities of the same message, or messages with nearly the same content (body & subject) are passing through an MTA within a specified time period. It seems to me this could be a useful way to detect not only spam but other types of problems as well - error conditions, mail bombs, etc. Clearly, one could not block such messages until a certain number of them had already been delivered, but still it could be useful. A whitelist function would obviously be used to allow legitimate traffic, but otherwise a threshold could be set and when enough messages with the same or mostly the same body content are detected, any further could be quarantined or tagged. Just curious if anyone knows of any method of doing this, either with spamassassin or with another tool. Thanks.
Re: duplicate messages
Jeff Ramsey <[EMAIL PROTECTED]> wrote on 11/11/2004 04:59:52 PM: > Hi all, > I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. Seems > to work fine. I am using it with the sql tables and everything. There is > one problem. I am getting a duplicate for about 2/3 of the mail that is > processed by the server. Here are the flags set for each app: > Spamassassin: > SPAMDOPTIONS="-u spamassassin -D -d -m10 -x -q" > > Spamass-milter: > SOCKET=/var/run/spamassassin/spamass.sock > EXTRA_FLAGS="-f -d3 -b junkmail" > > Sendmail: > sendmail.mc compilied with: > INPUT_MAIL_FILTER(`spamassassin', > `S=local:/var/run/spamassassin/spamass.sock, F=, > T=C:15m;S:4m;R:4m;E:10m')dnl > > I thought I had this working, and then I started to notice the > duplicates. Maybe there is something I have changed? Maybe it's just a > configuration issue. I have stopped the SA and milter apps, and > recompiled my cf file to not use the milter to make certain that this is > what is causing the issue. > > Jeff Ramsey > MIS Administrator > Tubafor Mill, Inc. > I ran into a similar problem, but in my case it was ClamAV. I'd start getting a bunch of ERROR: ScanStream: accept timeout. in the clamd log, and then would start getting duplicates. Restarting clamd would fix the problem. This was with 0.75, I've since upgraded to 0.80, and haven't seen the problem recur. Running: FBSD 5.2.1 Sendmail 8.13 Milter-spamc 0.24 SA 3.0.1 ClamAV 0.80 milter-greylist 1.5.3 Andy
Re: duplicate messages
On Thu, 2004-11-11 at 18:22, Dan Nelson wrote: > > If there's just one delivery line, it points the finger at something > happening after sendmail finishes with the message. Do you have any > procmail rules that might be duplicating messages? Maybe something > that would match text within the X-Spam-* headers? No. Should be a default install of Fedora Core 2. Perhaps there is a problem with Dovecot iMap/Pop3 server and SA headers? Jeff
Re: duplicate messages
On Thu, 2004-11-11 at 15:36, Dan Nelson wrote: > If you see two, then it's sendmail or the milter's fault. If you see > one, then it could be a local delivery or procmail rule issue. I am only seeing one line like this for each message. (disregard that last 'empty' reply. I hit instead of . - sorry) So does this mean that it is a local delivery issue? Why did it stop happening when I stopped using SA and SA-milter for an hour? Thanks, Jeff
Re: duplicate messages
On Thu, 2004-11-11 at 15:36, Dan Nelson wrote: > In the last episode (Nov 11), Jeff Ramsey said: > > On Thu, 2004-11-11 at 15:26, Dan Nelson wrote: > > > In the last episode (Nov 11), Jeff Ramsey said: > > > > I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. > > > > Seems to work fine. I am using it with the sql tables and > > > > everything. There is one problem. I am getting a duplicate for > > > > about 2/3 of the mail that is processed by the server. > > > > > > Exact dupes? Same Received: headers and timestamps? The only > > > thing that should be able to cause duplicates is the -B flag, and > > > only on spam. > > > > They are exact. Same timestamp, same headers, and message id. And not > > all of them are being marked as spam. I turned off the SA and > > SA-milter functions for an hour, and I did not have any duplicates. > > It has to be something with SA or SA-milter... > > Do you see two sendmail delivery lines in your maillog for the > messages? Ones like this: > > Nov 11 17:31:53 <2.6> dan sm-mta[27804]: iABNVriE027802: > to=, delay=00:00:00, xdelay=00:00:00, mailer=local, > pri=32492, dsn=2.0.0, stat=Sent > > If you see two, then it's sendmail or the milter's fault. If you see > one, then it could be a local delivery or procmail rule issue.
Re: duplicate messages
I too am having this problem without resolve, but, ive been having complaints even with SA 3.0.0. Im not running milter or sql queries and im just stumped... running: sendmail 8.13 rh 9.0 sa 3.0.1 MailScanner 4.33.3 Michael SpyderNet At 05:35 PM 11/11/2004, you wrote: At 02:59 PM 11.11.2004 -0800, Jeff Ramsey wrote: >Hi all, > I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. Seems >to work fine. I am using it with the sql tables and everything. There is >one problem. I am getting a duplicate for about 2/3 of the mail that is >processed by the server. Here are the flags set for each app: >Spamassassin: >SPAMDOPTIONS="-u spamassassin -D -d -m10 -x -q" > >Spamass-milter: >SOCKET=/var/run/spamassassin/spamass.sock >EXTRA_FLAGS="-f -d3 -b junkmail" > >Sendmail: >sendmail.mc compilied with: >INPUT_MAIL_FILTER(`spamassassin', >`S=local:/var/run/spamassassin/spamass.sock, F=, >T=C:15m;S:4m;R:4m;E:10m')dnl > >I thought I had this working, and then I started to notice the >duplicates. Maybe there is something I have changed? Maybe it's just a >configuration issue. I have stopped the SA and milter apps, and >recompiled my cf file to not use the milter to make certain that this is >what is causing the issue. > >Jeff Ramsey >MIS Administrator >Tubafor Mill, Inc. > Hm Interesting. I'm trying to troubleshoot a similar problem of dups. Some of the members of our majordomo discussion lists have been complaining of dups. Not every message, just random which is why I've had a hard time figuring out what has changed in the system to cause this. However, it definitely after the time of upgrade to SA-3.0x. I'm running: FBSD-4.10p2 Sendmail-8.12.11 spamass-milter-0.2.0_5 Soamassassin-3.0.1_1 Happy trails, Jack L. Stone System Admin Sage-american -- Spydernet has scanned this message for viruses and dangerous content.
Re: duplicate messages
On Thu, 2004-11-11 at 14:59, Jeff Ramsey wrote: > Hi all, > I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. Seems > to work fine. I am using it with the sql tables and everything. There is > one problem. I am getting a duplicate for about 2/3 of the mail that is > processed by the server. Here are the flags set for each app: > Spamassassin: > SPAMDOPTIONS="-u spamassassin -D -d -m10 -x -q" > > Spamass-milter: > SOCKET=/var/run/spamassassin/spamass.sock > EXTRA_FLAGS="-f -d3 -b junkmail" > > Sendmail: > sendmail.mc compilied with: > INPUT_MAIL_FILTER(`spamassassin', > `S=local:/var/run/spamassassin/spamass.sock, F=, > T=C:15m;S:4m;R:4m;E:10m')dnl > > I thought I had this working, and then I started to notice the > duplicates. Maybe there is something I have changed? Maybe it's just a > configuration issue. I have stopped the SA and milter apps, and > recompiled my cf file to not use the milter to make certain that this is > what is causing the issue. Is there a timeout period for message parsing for either sendmail, SA, or SA-milter? I think this may be due to me having my SQL lookups take a bit too long. I'm testing this theory out now... Jeff
Re: duplicate messages
At 02:59 PM 11.11.2004 -0800, Jeff Ramsey wrote: >Hi all, > I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. Seems >to work fine. I am using it with the sql tables and everything. There is >one problem. I am getting a duplicate for about 2/3 of the mail that is >processed by the server. Here are the flags set for each app: >Spamassassin: >SPAMDOPTIONS="-u spamassassin -D -d -m10 -x -q" > >Spamass-milter: >SOCKET=/var/run/spamassassin/spamass.sock >EXTRA_FLAGS="-f -d3 -b junkmail" > >Sendmail: >sendmail.mc compilied with: >INPUT_MAIL_FILTER(`spamassassin', >`S=local:/var/run/spamassassin/spamass.sock, F=, >T=C:15m;S:4m;R:4m;E:10m')dnl > >I thought I had this working, and then I started to notice the >duplicates. Maybe there is something I have changed? Maybe it's just a >configuration issue. I have stopped the SA and milter apps, and >recompiled my cf file to not use the milter to make certain that this is >what is causing the issue. > >Jeff Ramsey >MIS Administrator >Tubafor Mill, Inc. > Hm Interesting. I'm trying to troubleshoot a similar problem of dups. Some of the members of our majordomo discussion lists have been complaining of dups. Not every message, just random which is why I've had a hard time figuring out what has changed in the system to cause this. However, it definitely after the time of upgrade to SA-3.0x. I'm running: FBSD-4.10p2 Sendmail-8.12.11 spamass-milter-0.2.0_5 Soamassassin-3.0.1_1 Happy trails, Jack L. Stone System Admin Sage-american
duplicate messages
Hi all, I am using SA 3.0.1, spamass-milter 0.2.0, and Sendmail 8.12.11. Seems to work fine. I am using it with the sql tables and everything. There is one problem. I am getting a duplicate for about 2/3 of the mail that is processed by the server. Here are the flags set for each app: Spamassassin: SPAMDOPTIONS="-u spamassassin -D -d -m10 -x -q" Spamass-milter: SOCKET=/var/run/spamassassin/spamass.sock EXTRA_FLAGS="-f -d3 -b junkmail" Sendmail: sendmail.mc compilied with: INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamassassin/spamass.sock, F=, T=C:15m;S:4m;R:4m;E:10m')dnl I thought I had this working, and then I started to notice the duplicates. Maybe there is something I have changed? Maybe it's just a configuration issue. I have stopped the SA and milter apps, and recompiled my cf file to not use the milter to make certain that this is what is causing the issue. Jeff Ramsey MIS Administrator Tubafor Mill, Inc.
