Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
>should we use this even if we don't have the 99% cpu problem? No, you should not. The interim release should only be used by people who are experiencing that problem. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
should we use this even if we don't have the 99% cpu problem? - Original Message - From: "R. Scott Perry" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, September 25, 2001 6:44 PM Subject: Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU > We now have a new interim declude.exe online, at > http://www.declude.com/release/126/declude.exe . This one is based on > 1.25a, except that the DNS engine has been modified to make sure that > infinite loops are not possible. Right now, the most likely reason for the > 99% CPU usage seems to be during DNS queries, since the DNS engine changed > quite a bit between 1.20 and 1.25a. If the 99% CPU usage is indeed caused > by a problem with the DNS engine, this will take care of it. >-Scott > > --- > > This E-mail came from the Declude.JunkMail mailing list. To > unsubscribe, just send an E-mail to [EMAIL PROTECTED], and > type "unsubscribe Declude.JunkMail". You can E-mail > [EMAIL PROTECTED] for assistance. You can visit our web > site at http://www.declude.com . > --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
We now have a new interim declude.exe online, at http://www.declude.com/release/126/declude.exe . This one is based on 1.25a, except that the DNS engine has been modified to make sure that infinite loops are not possible. Right now, the most likely reason for the 99% CPU usage seems to be during DNS queries, since the DNS engine changed quite a bit between 1.20 and 1.25a. If the 99% CPU usage is indeed caused by a problem with the DNS engine, this will take care of it. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
Re: [Declude.JunkMail] H:Most effective spam test tips?
www.ipswitch.com search for spam, you will see how to impliment a filter. Travis - Original Message - From: "Wes Harper" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, September 25, 2001 1:08 PM Subject: [Declude.JunkMail] H:Most effective spam test tips? > I'm a newbie at using both IMail and Declude. I was wondering if I > could get some tips on the most effective spam tests that others are > using. > > Since I work for a telephone cooperative, I'm guessing there ought > to be an easy way to exclude all mail pertaining to "viagra", etc. (I > guess our insurance department might have a legitimate reason to > get something about that ... but I really doubt it). > > Anyway, I figure why re-invent the wheel when there have to be > other folks out there who have already developed effective > strategies. > > Wes Harper > Network Administrator > Pioneer Telephone Cooperative > [EMAIL PROTECTED] > --- > > This E-mail came from the Declude.JunkMail mailing list. To > unsubscribe, just send an E-mail to [EMAIL PROTECTED], and > type "unsubscribe Declude.JunkMail". You can E-mail > [EMAIL PROTECTED] for assistance. You can visit our web > site at http://www.declude.com . > --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
[Declude.JunkMail] H:Most effective spam test tips?
I'm a newbie at using both IMail and Declude. I was wondering if I could get some tips on the most effective spam tests that others are using. Since I work for a telephone cooperative, I'm guessing there ought to be an easy way to exclude all mail pertaining to "viagra", etc. (I guess our insurance department might have a legitimate reason to get something about that ... but I really doubt it). Anyway, I figure why re-invent the wheel when there have to be other folks out there who have already developed effective strategies. Wes Harper Network Administrator Pioneer Telephone Cooperative [EMAIL PROTECTED] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU
If you were to do a cache like that, you might do it using a hash of the content. The hash wouldn't take much memory or processing time. It's an interesting concept (we've got features like this built into server software we have on our drawing boards.) The neat thing about this is that if the network path changes for the message, it will still be recognized... versions of this capability are a great way to prevent DOS attacks like nimda - once a bad request is cached, all others like it are rejected out of hand. For example, a hack attempt would be seen as multiple requests to a web server with a 404 result in a specific time limit... presumably, a user making the attempt would see the first 404 and give up... if (say 5 in 1 minute) too many of these happen in a period then the signature of the request would be cached as a 32 bit hash value and rejected until the "attack" stops (no such requests for a period of time.) A similar mechanism can be used as part of a heuristic to reject spam where the signature of a message to a bad user is hashed, and if that message is sent to too many bad users in a period, then that message is loaded into the rejection scheme - thus thwarting dictionary attacks... Similarly a message that fails some other test could also be loaded and then automatically rejected even if the case for failure of the first test is resolved by the attacker. Build in some automated notification mechanisms and you have yourself a very robust system... say for example trusted servers were able to share rejection schemes... then an attack on one server would automatically be rejected by all participating servers - protecting those that had not yet seen the attack... but I'm letting the cat out of the bag now. Hopefully this is a helpful glimpse. Sorry for rambling. _M | -Original Message- | From: [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED]]On Behalf Of Charles Frolick | Sent: Tuesday, September 25, 2001 11:16 AM | To: [EMAIL PROTECTED] | Subject: RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU | | | Hey Scott, | | Just throwing an idea out there, since most spam mail hits multiple | addresses in the same server, often as seperate messages, how | about a failed | message cache that could automatically fail a message if it failed in the | last hour or some other configurable time span? It would probably have to | rely on a weghted combo match of several headers since they sometimes use | different senders or servers, and I've even seen them add the recipients | name or email in the subject and/or the body. | | This would help in that you are not completely reprocessing the same spam | message hundreds possibly thousands of times on high volume servers. | | The only drawback I can think of might be the extra resources to | manage the | cache if it got huge. | | Another thought, it could also be useful for the AV side. | | Chuck Frolick | ArgoNet, Inc. | -Original Message- | From: [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED]]On Behalf Of R. Scott Perry | Sent: Tuesday, September 25, 2001 9:45 AM | To: [EMAIL PROTECTED] | Subject: Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU | | | | >I have tried my best to help with this issue, research, testing, etc. | >During a normal business day, declude is awesome - working very nicely as | >advertised and very reliable. Perhaps, on an email server with less | >traffic, some declude customers wouldn't even know of any reliability | >issues. | | From the information we have gathered so far, the 99% CPU issue only | affects Declude JunkMail on v1.24 and higher, on highish volume systems | (our mail server processes about 3,000 E-mails on an average day, and we | haven't seen the problem even once here). | | >During a normal time period, we do pass a ton of emails, but during the | >moments when the flood gates open and spam flows in, this causes many | >declude.exe's to run. Some use 99% of the CPU, while others | simply use up | >memory. In my opinion, an opinion from a non-programmer, I would think | that | >there should only be one declude.exe running (as a service perhaps). | | That's actually an IMail limitation. Most programs do work that way -- | there is one process that handles all requests. However, IMail is set up | to start a new process for each E-mail that needs to be delivered. It | seems pretty inefficient (actually, a terrible design now that the | Microsoft heap issue has been identified), but they chose it for a | reason. Declude is stuck with that architecture -- IMail will start one | declude.exe process for each E-mail, no matter what. Note that without | Declude running, IMail will start one smtp32.exe process for each E-mail. | | Note that the 99% CPU issue was not reported before v1.24. | | >In the event that declude can't handle a "High Surge of Spam", then | >declude should | >pass the junk mail on to imail for normal processing. | | I haven't yet seen Declude not be able to
RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU
I stand corrected, but the purpose is the same. Thanks, _M | -Original Message- | From: [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED]]On Behalf Of R. Scott Perry | Sent: Tuesday, September 25, 2001 11:16 AM | To: [EMAIL PROTECTED] | Subject: RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU | | | | >There is a tweak for this in declude. You can set the maximum number of | >decludes that will run at one time. This helps set limits in cases like | >this. | | Actually, the MAXATONCE option only affects the number of virus scanner | processes that are started. It does not affect the number of | declude.exe's | that are started. | -Scott | | --- | | This E-mail came from the Declude.JunkMail mailing list. To | unsubscribe, just send an E-mail to [EMAIL PROTECTED], and | type "unsubscribe Declude.JunkMail". You can E-mail | [EMAIL PROTECTED] for assistance. You can visit our web | site at http://www.declude.com . | --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU
Hey Scott, Just throwing an idea out there, since most spam mail hits multiple addresses in the same server, often as seperate messages, how about a failed message cache that could automatically fail a message if it failed in the last hour or some other configurable time span? It would probably have to rely on a weghted combo match of several headers since they sometimes use different senders or servers, and I've even seen them add the recipients name or email in the subject and/or the body. This would help in that you are not completely reprocessing the same spam message hundreds possibly thousands of times on high volume servers. The only drawback I can think of might be the extra resources to manage the cache if it got huge. Another thought, it could also be useful for the AV side. Chuck Frolick ArgoNet, Inc. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of R. Scott Perry Sent: Tuesday, September 25, 2001 9:45 AM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU >I have tried my best to help with this issue, research, testing, etc. >During a normal business day, declude is awesome - working very nicely as >advertised and very reliable. Perhaps, on an email server with less >traffic, some declude customers wouldn't even know of any reliability >issues. From the information we have gathered so far, the 99% CPU issue only affects Declude JunkMail on v1.24 and higher, on highish volume systems (our mail server processes about 3,000 E-mails on an average day, and we haven't seen the problem even once here). >During a normal time period, we do pass a ton of emails, but during the >moments when the flood gates open and spam flows in, this causes many >declude.exe's to run. Some use 99% of the CPU, while others simply use up >memory. In my opinion, an opinion from a non-programmer, I would think that >there should only be one declude.exe running (as a service perhaps). That's actually an IMail limitation. Most programs do work that way -- there is one process that handles all requests. However, IMail is set up to start a new process for each E-mail that needs to be delivered. It seems pretty inefficient (actually, a terrible design now that the Microsoft heap issue has been identified), but they chose it for a reason. Declude is stuck with that architecture -- IMail will start one declude.exe process for each E-mail, no matter what. Note that without Declude running, IMail will start one smtp32.exe process for each E-mail. Note that the 99% CPU issue was not reported before v1.24. >In the event that declude can't handle a "High Surge of Spam", then >declude should >pass the junk mail on to imail for normal processing. I haven't yet seen Declude not be able to handle high volumes of spam. It's survived spam attacks, where a spammer will try to send through 100,000's of E-mails. >With Imail limiting smtp32.exe to just 30 instances (configurable by the >registry), this would not cause a problem with the desktop heap. You could >even trim down the smtp32.exe count to 10. Then, when the flood gates open, >the mail will just be a little slow, but reliable. This is a separate issue, and again an IMail issue. This is the problem that causes the "DLL initialization failure" crashes with IMail, and causes mail delivery to be sooo slow when there is overflow (E-mail arriving when all processes are being used). We are working on a way to minimize this problem, where Declude will take over the limiting of the number of processes running at once, and will create a separate queue for E-mail that hasn't been attempted yet (normally, IMail just puts the mail in the spool, which should just contain E-mail that couldn't be delivered on the first attempt). Then, as soon as there are free processes available, Declude will start things back up again with multiple processes (rather than waiting 1/2 hour or so for the next queue run, which only starts up 1 process). That should help reduce the DLL initialization problem, as well as the IMail slow mail delivery of overflow. It's not going to be perfect, since IMail can start several new processes on the same timeslice, which means that all of those processes will start before they have a chance to stop themselves. So, for example, if Windows will let you run 35 server-started processes before running out of their mystery heap, and IMail can start up to 10 processes on one timeslice, Declude would have to start the overflow procedure after 25 processes. If it let the 26th in, then IMail could create 10 more before Declude next had a chance to see how many processes were in memory, which would pass the 35 process limit, and crash the server. >I think this would make a more reliable server. Again, I am only TRYING to >help and give ideas. My intent is not to step on anyones toes. That's not a problem, ideas are what makes improvements possible.
RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU
>There is a tweak for this in declude. You can set the maximum number of >decludes that will run at one time. This helps set limits in cases like >this. Actually, the MAXATONCE option only affects the number of virus scanner processes that are started. It does not affect the number of declude.exe's that are started. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
RE: [Declude.JunkMail] Declude crashing and taking up 99% CPU
There is a tweak for this in declude. You can set the maximum number of decludes that will run at one time. This helps set limits in cases like this. _M | -Original Message- | From: [EMAIL PROTECTED] | [mailto:[EMAIL PROTECTED]]On Behalf Of Travis Sullivan | Sent: Tuesday, September 25, 2001 10:25 AM | To: [EMAIL PROTECTED] | Subject: Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU | | | I have tried my best to help with this issue, research, testing, etc. | During a normal business day, declude is awesome - working very nicely as | advertised and very reliable. Perhaps, on an email server with less | traffic, some declude customers wouldn't even know of any reliability | issues. I think I have narrowed down the issue with our declude | operation. | | During a normal time period, we do pass a ton of emails, but during the | moments when the flood gates open and spam flows in, this causes many | declude.exe's to run. Some use 99% of the CPU, while others simply use up | memory. In my opinion, an opinion from a non-programmer, I would | think that | there should only be one declude.exe running (as a service | perhaps). In the | event that declude can't handle a "High Surge of Spam", then | declude should | pass the junk mail on to imail for normal processing. | | With Imail limiting smtp32.exe to just 30 instances (configurable by the | registry), this would not cause a problem with the desktop heap. | You could | even trim down the smtp32.exe count to 10. Then, when the flood | gates open, | the mail will just be a little slow, but reliable. | | I think this would make a more reliable server. Again, I am only | TRYING to | help and give ideas. My intent is not to step on anyones toes. | I will keep | looking at the list to see when they have this problem fixed, but we can't | continue to provide poor uptime on our mail server. | | I do want to add one final note, my hats off to Scott for a great product | design and truely remarkable tech support. I honestly don't know when he | sleeps. Outstanding work ethics Scott!!! | | Travis Sullivan | | | --- | | This E-mail came from the Declude.JunkMail mailing list. To | unsubscribe, just send an E-mail to [EMAIL PROTECTED], and | type "unsubscribe Declude.JunkMail". You can E-mail | [EMAIL PROTECTED] for assistance. You can visit our web | site at http://www.declude.com . | --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
>I have tried my best to help with this issue, research, testing, etc. >During a normal business day, declude is awesome - working very nicely as >advertised and very reliable. Perhaps, on an email server with less >traffic, some declude customers wouldn't even know of any reliability >issues. From the information we have gathered so far, the 99% CPU issue only affects Declude JunkMail on v1.24 and higher, on highish volume systems (our mail server processes about 3,000 E-mails on an average day, and we haven't seen the problem even once here). >During a normal time period, we do pass a ton of emails, but during the >moments when the flood gates open and spam flows in, this causes many >declude.exe's to run. Some use 99% of the CPU, while others simply use up >memory. In my opinion, an opinion from a non-programmer, I would think that >there should only be one declude.exe running (as a service perhaps). That's actually an IMail limitation. Most programs do work that way -- there is one process that handles all requests. However, IMail is set up to start a new process for each E-mail that needs to be delivered. It seems pretty inefficient (actually, a terrible design now that the Microsoft heap issue has been identified), but they chose it for a reason. Declude is stuck with that architecture -- IMail will start one declude.exe process for each E-mail, no matter what. Note that without Declude running, IMail will start one smtp32.exe process for each E-mail. Note that the 99% CPU issue was not reported before v1.24. >In the event that declude can't handle a "High Surge of Spam", then >declude should >pass the junk mail on to imail for normal processing. I haven't yet seen Declude not be able to handle high volumes of spam. It's survived spam attacks, where a spammer will try to send through 100,000's of E-mails. >With Imail limiting smtp32.exe to just 30 instances (configurable by the >registry), this would not cause a problem with the desktop heap. You could >even trim down the smtp32.exe count to 10. Then, when the flood gates open, >the mail will just be a little slow, but reliable. This is a separate issue, and again an IMail issue. This is the problem that causes the "DLL initialization failure" crashes with IMail, and causes mail delivery to be sooo slow when there is overflow (E-mail arriving when all processes are being used). We are working on a way to minimize this problem, where Declude will take over the limiting of the number of processes running at once, and will create a separate queue for E-mail that hasn't been attempted yet (normally, IMail just puts the mail in the spool, which should just contain E-mail that couldn't be delivered on the first attempt). Then, as soon as there are free processes available, Declude will start things back up again with multiple processes (rather than waiting 1/2 hour or so for the next queue run, which only starts up 1 process). That should help reduce the DLL initialization problem, as well as the IMail slow mail delivery of overflow. It's not going to be perfect, since IMail can start several new processes on the same timeslice, which means that all of those processes will start before they have a chance to stop themselves. So, for example, if Windows will let you run 35 server-started processes before running out of their mystery heap, and IMail can start up to 10 processes on one timeslice, Declude would have to start the overflow procedure after 25 processes. If it let the 26th in, then IMail could create 10 more before Declude next had a chance to see how many processes were in memory, which would pass the 35 process limit, and crash the server. >I think this would make a more reliable server. Again, I am only TRYING to >help and give ideas. My intent is not to step on anyones toes. That's not a problem, ideas are what makes improvements possible. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
Re: [Declude.JunkMail] Declude crashing and taking up 99% CPU
I have tried my best to help with this issue, research, testing, etc. During a normal business day, declude is awesome - working very nicely as advertised and very reliable. Perhaps, on an email server with less traffic, some declude customers wouldn't even know of any reliability issues. I think I have narrowed down the issue with our declude operation. During a normal time period, we do pass a ton of emails, but during the moments when the flood gates open and spam flows in, this causes many declude.exe's to run. Some use 99% of the CPU, while others simply use up memory. In my opinion, an opinion from a non-programmer, I would think that there should only be one declude.exe running (as a service perhaps). In the event that declude can't handle a "High Surge of Spam", then declude should pass the junk mail on to imail for normal processing. With Imail limiting smtp32.exe to just 30 instances (configurable by the registry), this would not cause a problem with the desktop heap. You could even trim down the smtp32.exe count to 10. Then, when the flood gates open, the mail will just be a little slow, but reliable. I think this would make a more reliable server. Again, I am only TRYING to help and give ideas. My intent is not to step on anyones toes. I will keep looking at the list to see when they have this problem fixed, but we can't continue to provide poor uptime on our mail server. I do want to add one final note, my hats off to Scott for a great product design and truely remarkable tech support. I honestly don't know when he sleeps. Outstanding work ethics Scott!!! Travis Sullivan --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". You can E-mail [EMAIL PROTECTED] for assistance. You can visit our web site at http://www.declude.com .
