I just confirmed that this works properly. If the size limit is
exceeded for the DSN, it strips all attachments out and only sends back
the original headers and the top 20 lines of the original message
body. I set the MaxDSNSize to 100 KB, removed the MS SMTP size
limitation, and let IMail return the 552 error to my MS SMTP gateway.
Beautiful. No more of these retransmissions need to occur now because
of Microsoft's shortcoming.
Matt
Matt wrote:
Andrew,
I will try the MaxDSNSize hack and see if it does anything. I do know
that on vanilla MS SMTP, the limit is not defaulted to 10 MB because my
IMail is set to 25 MB and this was creating a lot of issues back when I
was configured that way. Basically, every 552 bounce was landing in
Badmail due to the sending of the DSN with the full original message
(and the messages in Badmail clearly showed sizes above 10 MB).
Hopefully this is something that can be tweaked without using
Exchange. If this works, it would be a good workaround for the issues
on my MS SMTP gateways where some oversized messages are rebroadcast
over and over again until they expire from their queue.
Regarding the captured text for the PoolThreadLimit, unfortunately this
is somewhat different from the information on Microsoft's site. It
would be good to determine exactly what is going on, though I do have
connection limits set in MS SMTP that are of a fairly low value, or at
least low enough that setting it to 256 shouldn't be an issue. Note
that this key is not present on a vanilla MS SMTP setup.
Thanks,
Matt
Colbeck, Andrew wrote:
This may be of some use... These
are two of the things checked by theMicrosoft Exchange Best Practices
Tool (sorry, there is no KB listed):
1) How to restrict the
size of the bounce messages you generate (just like the big boys do
with their postfix and sendmail MTAs, but it's possible that Exchange,
not IIS uses this, and that Exchange just uses this registry key as a
good place to put the value):
MaxDSNSize not set Send Feedback
The Microsoft® Exchange Server
Best Practices Analyzer Tool reads the following registry entry to
determine whether the maximum size for delivery status notification
(DSN) messages has been set:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSvc\Queuing\MaxDSNSize
The Exchange Server Best
Practices Analyzer also queries the Active Directory® directory service
to determine the value of the serialNumber attribute for all objects
with an object class of msExchExchangeServer. If the string value
includes "Version 5.5," the computer is running Microsoft Exchange
Server 5.5. If the string value includes "Version 6.0," the computer is
running Microsoft Exchange 2000 Server. If the string value includes
"Version 6.5," the computer is running Microsoft Exchange Server 2003.
If the Exchange Server Best
Practices Analyzer finds that the MaxDSNSize registry value does not
exist on a computer that is running Exchange 2000 Server or Exchange
Server 2003, a warning is displayed.
The MaxDSNSize registry value is
an optional value in Exchange Server that limits the size of DSNs. This
option configures Exchange to remove attachments if a message cannot be
delivered. If you add this registry value, it is recommended that a
value of 10240000 bytes (10 megabytes) be used.
Note:
The MaxDSNSize registry value was introduced in Service Pack 2 (SP2)
for Exchange 2000 Server, and therefore requires Exchange 2000 Server
SP2 or later versions to work. Additionally, in Service Pack 1 for
Exchange Server 2003 and later versions, a value of 10 megabytes is
used in the absence of the MaxDSNSize registry value. If you want, you
can add the MaxDSNSize registry value to override this default behavior.
If you enable this option, you can save server and network resources.
However, there are drawbacks to this implementation of Simple Mail
Transfer Protocol (SMTP) attachment stripping. If you enable this
option to strip the attachments from the non-delivery report (NDR), the
details that are necessary to display the notification in the preview
pane are also stripped, and the originator of the message cannot use
the Send Again option. If the originator of the message tries to use
the Send Again option from the NDR, the originator of the message
receives the following error message: Unable to resend the message. The
nondelivery report does not contain sufficient information about the
original message. To resend the message, open it in your Sent Items
folder, click the Actions menu, and click "Resend this message".
However, the originator of the message cannot resend the
message, even by using the method in the error message.
Important:
This article contains information about editing the registry. Before
you edit the registry, make sure you understand how to restore the
registry if a problem occurs. For information about how to restore the
registry, view the "Restore the Registry" Help topic in Regedit.exe or
Regedt32.exe.
To correct this warning
Open a registry editor, such as Regedit.exe or Regedt32.exe.
Navigate to: HKLM\System\CurrentControlSet\Services\SMTPSvc
Right-click SMTPSvc and select New | Key. Name the new key
Queuing and leave the class blank.
Right-click Queuing and select New | DWORD Value. Name the
new value MaxDSNSize.
In the right pane, double-click the MaxDSNSize registry value
and enter the size limit (in bytes) that you want in the Value data
field. Messages that are larger than this value that generate an NDR do
not return attachments or full message properties.
Close the registry editor, and then restart the Simple Mail
Transfer Protocol (SMTP) service for the change to take effect.
For more information about the MaxDSNSize registry setting,
see the following Microsoft Knowledge Base articles:
2) One possible registry key that would scale up the number
of threads used by IIS (hey, it seems like a good entry):
The Microsoft® Exchange Server
Best Practices Analyzer Tool reads the following registry entries to
determine whether the maximum number of Internet Information Services
(IIS) pool threads has been modified from the default value:
HKEY_LOCAL_MACHINE\
System\CurrentControlSet\Services\Inetinfo\Parameters\PoolThreadLimit
If the Exchange
Server
Best Practices Analyzer finds PoolThreadLimit to be present and
configured with an alternate value, a non-default configuration message
is displayed.
The PoolThreadLimit
registry value specifies the maximum number of input/output (I/O)
worker threads that can be created in the Inetinfo.exe process, which
in turn controls the maximum number of simultaneous connections that
can be made to IIS. Each pool thread watches for a network request and
processes it, either by sending back a static file or by passing the
request to an ISAPI extension DLL (such as ASP.DLL) or to a Common
Gateway Interface (CGI) application. If the ISAPI extension processes a
request synchronously and it takes a long time to process requests, it
will tie up the worker thread, leaving IIS with fewer worker threads to
process other requests. For this reason, well-written ISAPI extensions,
such as ASP, implement their own thread pools, place requests in a
queue, and process them asynchronously with their own threads, so as
not to tie up IIS worker threads.
By default, IIS sets
PoolThreadLimit
to:
2 * number of
megabytes of RAM present in the machine
If this value is
larger than 256, it will be clamped to 256. By default, this registry
value does not exist. If this value is present in the registry, it
overrides IIS's default calculation.
Generally, if you
find
that the default limit of 256 threads is inadequate, you probably have
a poorly written ISAPI extension tying up IIS worker threads. 256 is a
lot of threads to have active simultaneously and will incur significant
overhead in synchronization and context switching.
The PoolThreadLimit
registry value is a hard limit that includes all IIS worker threads,
including the HTTP, FTP, NNTP, and SMTP services.
Andrew 8)
Sandy, you have such a delicate way with words...
I am only passing on what I was told by Peter from VamSoft. There is a
whole thread on this where Peter confirmed the affect on FTP (but not
IIS) in VamSoft's o.e.support newsgroup, which I see you found after
sending this message.
I personally would be surprised to see a fixed (non-tweakable) thread
limit, but I can't make a judgment without some form of verification of
your suggestion considering the guidance that I was given (I'm not the
programmer nor the expert). This server averages only about 2% CPU
utilization, so it would be a shame to have it top out so prematurely.
I did previously search for "CDO_OnArrival thread", but came up with
nothing at all that was useful. Looks like I should have searched for
"SMTPSVC threads" instead since I did come across an article mentioning
the registry parameters that you mentioned. I'll wait for Peter to
reply to your suggestion.
Regarding the MS SMTP tarpitting, you have to trust me on that, or at
least test it yourself before suggesting that I am wrong here. This
was a different issue than that other server had. The issue with that
server was that MS SMTP returns 552 errors in the middle of the DATA,
and while many servers support that, IMail doesn't. I have since
convinced people at Ipswitch that this needs to be changed even if it
isn't technically RFC compliant due to real-world conditions, and the
fact that Microsoft's implementation makes more sense and should be
easy enough to support. This hasn't apparently been worked into the
software yet because they were in the final stages of testing IMail
2006, and this isn't a quick fix. I expect to see it happen soon
though, hopefully right after they get the major issues worked out with
the 2006 release.
The MS SMTP tarpitting issue is one that I came across under other
circumstances. When I had this turned on (not the ORF version, but the
registry hack), I experienced the condition repeatedly from all sorts
of servers where I was the recipient. After extensive research and
testing, I found that it was this registry hack that was causing the
majority of the servers to never recognize the 552 error code, and
since all this did was delay the response, one has to assume that the
delay of the 552 error was responsible. With this turned off (the
default), I am still experiencing some issues with requeued incoming
oversized E-mail, but they are much lower in number and related to the
sending of 552 errors in the middle of the DATA command. Microsoft
shouldn't be haphazardly delaying all 5xx errors, but instead should
just be delaying the ones related to invalid recipients. This was a
bad oversight on their part, and I am 100% positive about the cause.
It may be almost unnoticeable to those that are running this in front
of less E-mail addresses and don't actively monitor their bandwidth
like I do. Due to my volume, this issue became so large that it was
eating up many times my normal incoming bandwidth on a regular basis.
Note that I tried to work around both MS SMTP 552 issues on incoming
E-mail by setting MS SMTP to accept messages of unlimited size and then
configuring IMail to signal the error, but MS SMTP hears the 552 error
that IMail generates and then tries to bounce the entire oversized
message through the configured Smart Host, which was the same IMail
server that wouldn't accept the message in the first place. Why you
would want to send a 50 MB bounce message is beyond me. I could kludge
something together to bypass the problem, that's not the issue though.
My point is that I have been through this stuff with a fine-toothed
comb.
Matt
Sanford Whiteman wrote:
The exact source of the problem was a limit in threads in MS SMTP,
which is something like 20 per processor (not sure if hyperthreading
counts in this case). . .
This isn't an accurate claim. It sounds like you just haven't tried to
tune MS SMTP appropriately for load/resources/growth. That's fine,
since MS doesn't push you too hard on this, but please do some more
lab work before deciding that MS SMTP is broken.
Look into the following values:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters\PoolThreadLimit
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\MaxPercentPoolThreads
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\AdditionalPoolThreadsPerProc
You also need to realize the differences between I/O completion port
architecture, callbacks, and pure multithreading. It's possible to
bottleneck anything, but that's an implementation problem with a given
event sink, not a problem with the overall framework.
. . . this limitation in the MS SMTP architecture makes it
inappropriate for any full scale spam blocking solution such as
Declude unless you have that application ride behind MS SMTP instead
of being plugged into it. . . .
Sorry, man, but that's FUD. I appreciate what you found with FTP in
your config, but let's not jump over the available data.
One other note. I had previously used the Windows registry hack to
enable a native tarpitting feature in MS SMTP with even worse
affects. The built-in tarpitting in MS SMTP will delay all 5xx error
codes by the time that you set, and this included the 552 error used
when messages are over the maximum size. The result of this terrible
oversight is that a fair number of servers will not recognize the
delayed 552 error and will requeue the oversized messages over and
over again, and that eats up a lot of bandwidth real, real quick.
Matt, you _know_ this isn't cause and effect, since you and I both
looked at a specific server _without_ tarpitting enabled that
exhibited the problem with oversize messages. There may be an
additional wrinkle added by ttarpitting, but it's not the root cause
of the problem.
--Sandy
------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: [EMAIL PROTECTED]
SpamAssassin plugs into Declude!
http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/
Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases!
http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/
http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/
---
[This E-mail was scanned for viruses by Declude EVA www.declude.com]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
|