Hey Scott,
Just throwing an idea out there, since most spam mail hits multiple
addresses in the same server, often as seperate messages, how about a failed
message cache that could automatically fail a message if it failed in the
last hour or some other configurable time span? It would probably have to
rely on a weghted combo match of several headers since they sometimes use
different senders or servers, and I've even seen them add the recipients
name or email in the subject and/or the body.
This would help in that you are not completely reprocessing the same spam
message hundreds possibly thousands of times on high volume servers.
The only drawback I can think of might be the extra resources to manage the
cache if it got huge.
Another thought, it could also be useful for the AV side.
Chuck Frolick
ArgoNet, Inc.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of R. Scott Perry
Sent: Tuesday, September 25, 2001 9:45 AM
To: [EMAIL PROTECTED]
Subject: Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
>I have tried my best to help with this issue, research, testing, etc.
>During a normal business day, declude is awesome - working very nicely as
>advertised and very reliable. Perhaps, on an email server with less
>traffic, some declude customers wouldn't even know of any reliability
>issues.
From the information we have gathered so far, the 99% CPU issue only
affects Declude JunkMail on v1.24 and higher, on highish volume systems
(our mail server processes about 3,000 E-mails on an average day, and we
haven't seen the problem even once here).
>During a normal time period, we do pass a ton of emails, but during the
>moments when the flood gates open and spam flows in, this causes many
>declude.exe's to run. Some use 99% of the CPU, while others simply use up
>memory. In my opinion, an opinion from a non-programmer, I would think
that
>there should only be one declude.exe running (as a service perhaps).
That's actually an IMail limitation. Most programs do work that way --
there is one process that handles all requests. However, IMail is set up
to start a new process for each E-mail that needs to be delivered. It
seems pretty inefficient (actually, a terrible design now that the
Microsoft heap issue has been identified), but they chose it for a
reason. Declude is stuck with that architecture -- IMail will start one
declude.exe process for each E-mail, no matter what. Note that without
Declude running, IMail will start one smtp32.exe process for each E-mail.
Note that the 99% CPU issue was not reported before v1.24.
>In the event that declude can't handle a "High Surge of Spam", then
>declude should
>pass the junk mail on to imail for normal processing.
I haven't yet seen Declude not be able to handle high volumes of
spam. It's survived spam attacks, where a spammer will try to send through
100,000's of E-mails.
>With Imail limiting smtp32.exe to just 30 instances (configurable by the
>registry), this would not cause a problem with the desktop heap. You could
>even trim down the smtp32.exe count to 10. Then, when the flood gates
open,
>the mail will just be a little slow, but reliable.
This is a separate issue, and again an IMail issue. This is the problem
that causes the "DLL initialization failure" crashes with IMail, and causes
mail delivery to be sooooooo slow when there is overflow (E-mail arriving
when all processes are being used).
We are working on a way to minimize this problem, where Declude will take
over the limiting of the number of processes running at once, and will
create a separate queue for E-mail that hasn't been attempted yet
(normally, IMail just puts the mail in the spool, which should just contain
E-mail that couldn't be delivered on the first attempt). Then, as soon as
there are free processes available, Declude will start things back up again
with multiple processes (rather than waiting 1/2 hour or so for the next
queue run, which only starts up 1 process).
That should help reduce the DLL initialization problem, as well as the
IMail slow mail delivery of overflow. It's not going to be perfect, since
IMail can start several new processes on the same timeslice, which means
that all of those processes will start before they have a chance to stop
themselves. So, for example, if Windows will let you run 35 server-started
processes before running out of their mystery heap, and IMail can start up
to 10 processes on one timeslice, Declude would have to start the overflow
procedure after 25 processes. If it let the 26th in, then IMail could
create 10 more before Declude next had a chance to see how many processes
were in memory, which would pass the 35 process limit, and crash the server.
>I think this would make a more reliable server. Again, I am only TRYING to
>help and give ideas. My intent is not to step on anyones toes.
That's not a problem, ideas are what makes improvements possible.
-Scott
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". You can E-mail
[EMAIL PROTECTED] for assistance. You can visit our web
site at http://www.declude.com .
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". You can E-mail
[EMAIL PROTECTED] for assistance. You can visit our web
site at http://www.declude.com .
