RE: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service

[Colbeck, Andrew]

This may be of some use... These are two of the things checked by theMicrosoft Exchange Best Practices Tool (sorry, there is no KB listed):   1) How to restrict the size of the bounce messages you generate (just like the big boys do with their postfix and sendmail MTAs, but it's possible that Exchange, not IIS uses this, and that Exchange just uses this registry key as a good place to put the value):        MaxDSNSize not set  Send Feedback      The Microsoft® Exchange Server Best Practices Analyzer Tool reads the following registry entry to determine whether the maximum size for delivery status notification (DSN) messages has been set:   HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSvc\Queuing\MaxDSNSize   The Exchange Server Best Practices Analyzer also queries the Active Directory® directory service to determine the value of the serialNumber attribute for all objects with an object class of msExchExchangeServer. If the string value includes "Version 5.5," the computer is running Microsoft Exchange Server 5.5. If the string value includes "Version 6.0," the computer is running Microsoft Exchange 2000 Server. If the string value includes "Version 6.5," the computer is running Microsoft Exchange Server 2003.   If the Exchange Server Best Practices Analyzer finds that the MaxDSNSize registry value does not exist on a computer that is running Exchange 2000 Server or Exchange Server 2003, a warning is displayed.   The MaxDSNSize registry value is an optional value in Exchange Server that limits the size of DSNs. This option configures Exchange to remove attachments if a message cannot be delivered. If you add this registry value, it is recommended that a value of 10240000 bytes (10 megabytes) be used.   Note:  The MaxDSNSize registry value was introduced in Service Pack 2 (SP2) for Exchange 2000 Server, and therefore requires Exchange 2000 Server SP2 or later versions to work. Additionally, in Service Pack 1 for Exchange Server 2003 and later versions, a value of 10 megabytes is used in the absence of the MaxDSNSize registry value. If you want, you can add the MaxDSNSize registry value to override this default behavior.     If you enable this option, you can save server and network resources. However, there are drawbacks to this implementation of Simple Mail Transfer Protocol (SMTP) attachment stripping. If you enable this option to strip the attachments from the non-delivery report (NDR), the details that are necessary to display the notification in the preview pane are also stripped, and the originator of the message cannot use the Send Again option. If the originator of the message tries to use the Send Again option from the NDR, the originator of the message receives the following error message: Unable to resend the message. The nondelivery report does not contain sufficient information about the original message. To resend the message, open it in your Sent Items folder, click the Actions menu, and click "Resend this message".   However, the originator of the message cannot resend the message, even by using the method in the error message.   Important:  This article contains information about editing the registry. Before you edit the registry, make sure you understand how to restore the registry if a problem occurs. For information about how to restore the registry, view the "Restore the Registry" Help topic in Regedit.exe or Regedt32.exe.     To correct this warning Open a registry editor, such as Regedit.exe or Regedt32.exe.   Navigate to: HKLM\System\CurrentControlSet\Services\SMTPSvc   Right-click SMTPSvc and select New | Key. Name the new key Queuing and leave the class blank.   Right-click Queuing and select New | DWORD Value. Name the new value MaxDSNSize.   In the right pane, double-click the MaxDSNSize registry value and enter the size limit (in bytes) that you want in the Value data field. Messages that are larger than this value that generate an NDR do not return attachments or full message properties.   Close the registry editor, and then restart the Simple Mail Transfer Protocol (SMTP) service for the change to take effect.   For more information about the MaxDSNSize registry setting, see the following Microsoft Knowledge Base articles:   308303, "XCON: Option to Strip Attachments for Messages That Generate an NDR," (http://go.microsoft.com/fwlink/?linkid=3052&kbid=308303)   323484, "XCON: Description of the Multipart/Report Internet Message Format in Exchange 2000," (http://go.microsoft.com/fwlink/?linkid=3052&kbid=323484)   2) One possible registry key that would scale up the number of threads used by IIS (hey, it seems like a good entry):   The Microsoft® Exchange Server Best Practices Analyzer Tool reads the following registry entries to determine whether the maximum number of Internet Information Services (IIS) pool threads has been modified from the default value: HKEY_LOCAL_MACHINE\ System\CurrentControlSet\Services\Inetinfo\Parameters\PoolThreadLimit If the Exchange Server Best Practices Analyzer finds PoolThreadLimit to be present and configured with an alternate value, a non-default configuration message is displayed. The PoolThreadLimit registry value specifies the maximum number of input/output (I/O) worker threads that can be created in the Inetinfo.exe process, which in turn controls the maximum number of simultaneous connections that can be made to IIS. Each pool thread watches for a network request and processes it, either by sending back a static file or by passing the request to an ISAPI extension DLL (such as ASP.DLL) or to a Common Gateway Interface (CGI) application. If the ISAPI extension processes a request synchronously and it takes a long time to process requests, it will tie up the worker thread, leaving IIS with fewer worker threads to process other requests. For this reason, well-written ISAPI extensions, such as ASP, implement their own thread pools, place requests in a queue, and process them asynchronously with their own threads, so as not to tie up IIS worker threads. By default, IIS sets PoolThreadLimit to: 2 * number of megabytes of RAM present in the machine If this value is larger than 256, it will be clamped to 256. By default, this registry value does not exist. If this value is present in the registry, it overrides IIS's default calculation. Generally, if you find that the default limit of 256 threads is inadequate, you probably have a poorly written ISAPI extension tying up IIS worker threads. 256 is a lot of threads to have active simultaneously and will incur significant overhead in synchronization and context switching. The PoolThreadLimit registry value is a hard limit that includes all IIS worker threads, including the HTTP, FTP, NNTP, and SMTP services.   Andrew 8) From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Tuesday, January 10, 2006 4:45 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] OT: Issues with Windows 2003 FTP service Sandy, you have such a delicate way with words... I am only passing on what I was told by Peter from VamSoft.  There is a whole thread on this where Peter confirmed the affect on FTP (but not IIS) in VamSoft's o.e.support newsgroup, which I see you found after sending this message. I personally would be surprised to see a fixed (non-tweakable) thread limit, but I can't make a judgment without some form of verification of your suggestion considering the guidance that I was given (I'm not the programmer nor the expert).  This server averages only about 2% CPU utilization, so it would be a shame to have it top out so prematurely.  I did previously search for "CDO_OnArrival thread", but came up with nothing at all that was useful.  Looks like I should have searched for "SMTPSVC threads" instead since I did come across an article mentioning the registry parameters that you mentioned.  I'll wait for Peter to reply to your suggestion. Regarding the MS SMTP tarpitting, you have to trust me on that, or at least test it yourself before suggesting that I am wrong here.  This was a different issue than that other server had.  The issue with that server was that MS SMTP returns 552 errors in the middle of the DATA, and while many servers support that, IMail doesn't.  I have since convinced people at Ipswitch that this needs to be changed even if it isn't technically RFC compliant due to real-world conditions, and the fact that Microsoft's implementation makes more sense and should be easy enough to support.  This hasn't apparently been worked into the software yet because they were in the final stages of testing IMail 2006, and this isn't a quick fix.  I expect to see it happen soon though, hopefully right after they get the major issues worked out with the 2006 release. The MS SMTP tarpitting issue is one that I came across under other circumstances.  When I had this turned on (not the ORF version, but the registry hack), I experienced the condition repeatedly from all sorts of servers where I was the recipient.  After extensive research and testing, I found that it was this registry hack that was causing the majority of the servers to never recognize the 552 error code, and since all this did was delay the response, one has to assume that the delay of the 552 error was responsible.  With this turned off (the default), I am still experiencing some issues with requeued incoming oversized E-mail, but they are much lower in number and related to the sending of 552 errors in the middle of the DATA command.  Microsoft shouldn't be haphazardly delaying all 5xx errors, but instead should just be delaying the ones related to invalid recipients.  This was a bad oversight on their part, and I am 100% positive about the cause.  It may be almost unnoticeable to those that are running this in front of less E-mail addresses and don't actively monitor their bandwidth like I do.  Due to my volume, this issue became so large that it was eating up many times my normal incoming bandwidth on a regular basis. Note that I tried to work around both MS SMTP 552 issues on incoming E-mail by setting MS SMTP to accept messages of unlimited size and then configuring IMail to signal the error, but MS SMTP hears the 552 error that IMail generates and then tries to bounce the entire oversized message through the configured Smart Host, which was the same IMail server that wouldn't accept the message in the first place.  Why you would want to send a 50 MB bounce message is beyond me.  I could kludge something together to bypass the problem, that's not the issue though.  My point is that I have been through this stuff with a fine-toothed comb. Matt Sanford Whiteman wrote: The exact source of the problem was a limit in threads in MS SMTP, which is something like 20 per processor (not sure if hyperthreading counts in this case). . . This isn't an accurate claim. It sounds like you just haven't tried to tune MS SMTP appropriately for load/resources/growth. That's fine, since MS doesn't push you too hard on this, but please do some more lab work before deciding that MS SMTP is broken. Look into the following values: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters\PoolThreadLimit HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\MaxPercentPoolThreads HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\AdditionalPoolThreadsPerProc You also need to realize the differences between I/O completion port architecture, callbacks, and pure multithreading. It's possible to bottleneck anything, but that's an implementation problem with a given event sink, not a problem with the overall framework. . . . this limitation in the MS SMTP architecture makes it inappropriate for any full scale spam blocking solution such as Declude unless you have that application ride behind MS SMTP instead of being plugged into it. . . . Sorry, man, but that's FUD. I appreciate what you found with FTP in your config, but let's not jump over the available data. One other note. I had previously used the Windows registry hack to enable a native tarpitting feature in MS SMTP with even worse affects. The built-in tarpitting in MS SMTP will delay all 5xx error codes by the time that you set, and this included the 552 error used when messages are over the maximum size. The result of this terrible oversight is that a fair number of servers will not recognize the delayed 552 error and will requeue the oversized messages over and over again, and that eats up a lot of bandwidth real, real quick. Matt, you _know_ this isn't cause and effect, since you and I both looked at a specific server _without_ tarpitting enabled that exhibited the problem with oversize messages. There may be an additional wrinkle added by ttarpitting, but it's not the root cause of the problem. --Sandy ------------------------------------ Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/ http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude EVA www.declude.com] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.