I wrote a batch file once on a number of the exchange servers that used VBS and LDAP to generate a list of valid exchange recipients and then FTP them to the server where a CF script parsed it clean. Michael, it sounds like you were most of the way there. Alligate does have the feature you were working towards, which was a recipient file for a given domain, the magic phrase being the "rvInput" folder. I don't use it, but the rough idea is that you periodically drop in a plain text file, say, per domain, in that folder and an Alligate process picks them up. Another one of the problems is that most all of my clients don't want to disable NDRs with whatever solution I come up with, which makes it fairly impossible to avoid backscatter. This lets your gateway accept only email for valid recipients and reject mail during the envelope conversation, thus you are not generating spam backscatter and you are emitting valid NDRs only (when the bad guys spoof a MAILFROM and you accept a message because your gateway can't validate the recipient yet, then later bounce the message as undeliverable, your backscatter spams the spoofed sender). Darin's earlier message describes a way to accomplish the same thing via IMail and aliases; I believe this method was pioneered by Sandy (Sanford Whiteman) back in 2004 and the thread can be picked up here: http://www.mail-archive.com/search?q=Exchange2aliases&l=declude.junkmail %40declude.com The trouble for you is that this is an even more significant implementation for your clients than your scraping of their AD. Andrew.
________________________________ From: [email protected] [mailto:[email protected]] On Behalf Of Michael Cummins Sent: Wednesday, May 12, 2010 11:14 AM To: [email protected] Subject: RE: [Declude.JunkMail] Fine tuning Declude I wrote a batch file once on a number of the exchange servers that used VBS and LDAP to generate a list of valid exchange recipients and then FTP them to the server where a CF script parsed it clean. I didn't quite know what to do with them when they got there though (I was originally going to use them in Alligate, but never got that up and going) and I don't have the full "granular" cooperation of all the Exchange network peeps, only most of them, so it was difficult to implement a one-size-fits-all policy regardless. I'll put my thinking cap on. Another one of the problems is that most all of my clients don't want to disable NDRs with whatever solution I come up with, which makes it fairly impossible to avoid backscatter. It goes in me one way, and out another :p Very Respectfully, Michael Cummins From: [email protected] [mailto:[email protected]] On Behalf Of Darin Cox Sent: Wednesday, May 12, 2010 10:55 AM To: [email protected] Subject: Re: [Declude.JunkMail] Fine tuning Declude Hi Michael, I may be able to help with this. You mention doing gateway filtering for Exchange servers. We also do that, but instead of accepting any address with the domain, we have accounts set up on our server and refuse connections that don't go to one of those accounts. Now your next comment is probably that you don't want the extra management of setting up accounts on both servers. Well we've handled that by using a sync process we developed to extract the list of accounts from the Exchange server, ship that up to the gateway server, and check to see what accounts need to be added or deleted. We've been using this process for a couple of years with perfect success. Since it is a batch process, it is scheduled to run every few minutes, so there could be a few minute delay when new accounts are added, but it has worked flawlessly for a couple of years. There are checks in place to make sure incomplete transfers don't result in accounts being deleted or incorrect accounts getting added to the gateway, and notifications are sent every time accounts are added or deleted. Currently it runs as a script on the destination Exchange or IMail server, and a scheduled process on a SQL database on our mail gateway server. Also, our gateway is an IMail server, but we could easily adapt it to use the account creation command line utilities I assume SmarterMail has. One other comment about the implementation. We maintain a hosts file for forwarding to the destination mail server, and use a subdomain to forward the mail for routing purposes, so the destination mail server is configured to accept mail for the subdomain. That's a simple change in Exchange to add an SMTP alias, and can be added to the default policy in Exchange so it is automatically added when an account is created. Anyway, if you have any interest, let me know. I know we wouldn't be able to survive if we were accepting email for any address in a domain, so I feel your pain. Best, Darin Cox 4C Web A division of 4C Design Technology Corp. (813) 413-4883 Tampa Bay, FL (919) 533-5000 Research Triangle, NC ----- Original Message ----- From: Michael Cummins <mailto:[email protected]> To: [email protected] Sent: Wednesday, May 12, 2010 9:25 AM Subject: [Declude.JunkMail] Fine tuning Declude So this past week has been fairly hellish for me, buried in the thick of Botnet Spam storms. (Quite a number of people seem to be experiencing them, at least as reported over on the [SNIFFER] list) My implementation of Declude seems to be pressed to its limits to handle the volume. 1) Dedicated SmarterMail 6.8 2) Declude, Invaluement RBLs added, running off a SimpleDNSPlus install on another local machine 3) INVURIBL with Invaluement and SpamEatingMonkey added 4) SNIFFER, integrated with Declude This is the root of my volume issues: this box is a dedicated Incoming Gateway for several dozen Exchange servers for SMBs, which means it accepts ALL mail for those domains. It's not like my other mail server that rejects bad addresses right off the bat. When the spam storms hit, it's like a hurricane. My usual Sniffer-measured rate of about 150-200k messages per day kick up as high as 850k. I don't really handle that much mail, but that's the rate when it storms. My regular SmarterMail server that dishes out POP/IMAP handles a more appropriate level of 50k messages per day. 1) If I keep WAITBETWEENTHREADS too low, DecludeProc will race up to the top of THREADS and crash when the storms hit. I currently find that 45 is the bleeding edge of sanity (for my config) with INVURIBL and SNIFFER running, but in a bad storm, even that is too low, and sometimes I have to drop it back to 60 or 65; but then it's just keeping up with things, and it's difficult to reduce the backlog that swelled during the crash. 2) If I keep WAITBETWEENTHREADS too high, like around 100, Declude is stable as a rock, but can't keep up with the mail load when times get tough. 3) When things get bad, I go into GLOBAL.CFG and comment out INVURIBL and/or the many SNIFFER tests. Does anyone have any useful advice for beefing up or streamlining this process? What hardware choices have the biggest impact on Declude? As an aside, I imagine that you could prevent a lot of Declude crashes if WAITBETWEENTRHEADS was a dynamic setting, derived from the mail rate. Yes? No? On a related note, I've been building a Declude Management interface in ColdFusion that makes excellent use of Mark Russinovich's Sysinternals suite of tools, most specifically PsList and PsKill, so I can keep a careful eye on DecludeProc on my two machines, and using the Microsoft FSO to keep an eye on file counts. Sysinternals http://technet.microsoft.com/en-us/sysinternals/bb842062.aspx FSO http://msdn.microsoft.com/en-us/library/z9ty6h50(VS.85).aspx I really recommend those tools. FSO is really responsive when inspecting large file counts, for keeping an eye on /spool/ /proc/ and /review/. You can write a parse the results of PsList to keep an eye on the number of Threads that Declude is spawning, and even detect a crash. Oh, and I have to compliment Linda and David for their relentless and professional service. They are a fantastic and responsive team. BZ! -- Michael Cummins --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [email protected], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [email protected], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [email protected], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [email protected], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
