Harry,
I use SURBL myself in addition to 85 other filter files, for a total of
265 KB of filters. Probably only 20% of them are BODY filter lines
though, and I don't think I have any ANYWHERE filters in use. I
consider our installation to be heavy, but I have spent a lot of time
making it efficient.
I think what you should do is tier your spam blocking by weight. We
operate a Hold and a Drop range, and when something hits the Drop
weight we stop processing filters on it. Over 80% of the spam never
runs our custom filters and that has saved us an enormous amount of CPU
cycles. You would do this with the SKIPIFWEIGHT setting in the top of
every custom filter file. We Hold starting at a score of 10 (mostly 13
though) and Drop at a score of 25. We manage to get 98% of the spam to
land in our Drop range which we don't review at all. Our false
positive rate in the Drop range is far less than 1 in 10,000, and
typically results from widely blacklisted sources that no one complains
about. I am only aware of about 3 FP's to land in this range over the
last year. More importantly, it allows us to focus on the > 2% that
lands in our Hold range where we typically find about 2 to 3 FP's per
100 messages that land in there, though most of that is what we
consider to be legitimate advertising or newsletters from mixed sources.
I highly recommend that you focus on adding SKIPIFWEIGHT to your
filters and tiering your scoring and actions appropriately. It is
generally safe to toss what scores 3 times your hold weight, though
some filter architectures can enhance false positives and it is
important to limit incidences where the same FP issue can trip multiple
filters.
Matt
Harry Vanderzand wrote:
Message
thank you Matt,
I am running 179i16 so I may have another issue at
hand here
I have 42k myfilter file with every entry set to
anywhere which essentially does a similar thing that surbl is doing. I
mine the web info from them manually everyday.
I do it on my own account as my account attracts
a tremendous amount of spam I guess because it has been around for 10
years. Whatever gets through to it after declude has been going into
my filter file
I have surbl running with its 35k file
I have today eliminated my filter file and will
likely eliminate surbl once I get the full version of sniffer going.
So far I see no more going through as it is likely that surbl has been
better at that process than me.
I am starting to realize that these body filters
are expensive in cpu cycles
I will share what I learn from all this
I appreciate your assistance.
Harry Vanderzand
inTown Internet & Computer Services
11 Belmont Ave. W.
Kitchener, ON
N2M 1L2
519-741-1222
Did you know we offer:
- Province wide dial-up and high speed internet access
- Web accessible email with anti-spam\antivirus protection
- Computer hardware sales and service
- Experienced website developers
Harry,
Sniffer is a great addition to any Declude setup, however your issues
are not due to just simply the size of your processors. We run a dual
1 GHz PIII system with RAID 5 and 5x10K Cheetahs, and we've managed to
exceed 90,000 messages a day with dual virus scanners, and we could
handle a bit more still. My thought is that you are either running a
ton of BODY filters, a very slow virus scanner/scanners, or you are
experiencing some form of I/O limitation. The idle processes also
suggest that maybe there is an issue and an upgrade to a more recent
version of Declude such as 1.79 or an interim release thereafter would
be a good idea and most around here run them.
You should be able to minimally do 10 times your current volume, so
keep looking and keep describing your environment and a solution will
likely come along.
Matt
Harry Vanderzand wrote:
I am getting service timeouts due mostly to all the declude instances of
traffic volume
I handle about 20000 messages a day, most of them during business hours
I find that I accumulate declude processes that have consumed up to a minute
of cpu time only to be idle and just sit there
This also causes accumulated memory to be consumed
I have been rebooting this server about twice a week
I have also been spending time everyday adding to my filter files
The server is a dual Xeon 2.4Ghz, 533 frontside bus with an Intel SATA raid
card running Raid 10
It has about 100 small web site that do not get much traffic
My goal is to reduce management time of the machine and to stabilize it so
the need to reboot it is lessened
I am prepared to put in a dual Xeon 3.4GH, etc but also want to make sure
that I do not overkill
Harry Vanderzand
inTown Internet & Computer Services
11 Belmont Ave. W.
Kitchener, ON
N2M 1L2
519-741-1222
Did you know we offer:
- Province wide dial-up and high speed internet access
- Web accessible email with anti-spam\antivirus protection
- Computer hardware sales and service
- Experienced website developers
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Markus Gufler
Sent: Wednesday, September 08, 2004 11:22 AM
To: [EMAIL PROTECTED]
Subject: RE: [Declude.JunkMail] Test needed along with sniffer
I am testing sniffer right now and wonder if I need to run
all the other tests along side it.
I am trying to reduce my daily workload of analyzing the
"spamtrap" and hope that sniffer and surbl will do this.
Do I even need surbl?
Do you have so much workload on your mailserver that you need
to downsize your spam-filter to one or two tests?
Maybe http://www2.spamchk.com/public.htm will give you some answer.
Markus
---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be
found at http://www.mail-archive.com.
---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
|