Re: [Declude.JunkMail] OT: IMAIL - AD
Sandy, You're quite a capable person, and some of this stuff might be trivial for you, or maybe you just like tinkering with such things...but, it's overreaching to assume that this is the same for the vast majority of users. A long time ago when I was in high school and proud member of the geeky A/V club, we often found ourselves without the proper cable to connect two devices...so we improvised. One cable into another, switching genders, over and over again, eventually we got what we needed. We were thinking on our feet; we were being resourceful. However, had the proper cable been available, we would have been greatly overly complicating matters. I guess what I'm saying is if you can do it without LDAP or ActiveDirectory, why not do it without LDAP or ActiveDirectory. There's certainly other ways to go about doing this, especially if you only have one or a small handful of machines that need to access this data. Sourcing directly from text files (not in real time as previously clarified) is likely the most universal and uncomplicated method, however some situations may well benefit by sourcing from LDAP, such as being a dedicated backup to an Exchange server, or a dedicated backup to an IMail server that doesn't gateway...or if you just simply prefer for it to be that way. I don't think LDAP is bad, I just think that supporting a distributed LDAP environment is unnecessary if done solely for the purpose of storing several hundred to several tens of thousands of E-mail addresses. Matt Sanford Whiteman wrote: I'm not dumping on LDAP, I think it can be very useful, however in this case, is it really necessary? Why not just support loading a text file into memory and using that? Because it's poor architecture that I wouldn't trust on my mailserver. It's the lowest common denominator... Yep, that's the problem, all right. :) The only reason not to use text files would be a technical limitation, but I'm not suggesting that it be accessed once per message, so that isn't at issue. Then you clearly don't see the _other_ technical problems involved. Disk I/O is not the primary problem. I would certainly look to VAMsoft for this application if they supported text files... Well, you _can_ use ORF for this! Just use their "everybody but" recipient blacklist, whose addresses are stored in the .INI file and read once at service start (ORF service, not SMTP service). Every time you update the file, net restart ORF. It's _already_ there for you in ORF if this is the way you want to swing it. I believe that if you have a single domain, AD via LDAP is the better way to go. As a longtime LDAP user, I believe your concerns about the complexity of having a built-in LDAP service running with the sole purpose of MX user lookup are unfounded. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
However, had the proper cable been available, we would have been greatly overly complicating matters. Indeed, your proper cable already exists in the form of the everything but recipient list in ORF, as I mentioned in my last message. I think you should use it. I guess what I'm saying is if you can do it without LDAP or ActiveDirectory, why not do it without LDAP or ActiveDirectory. There's a difference between doing it and doing it right, of course. For your environment and traffic, ORF alone might well do it right, so go for it. My issue is with encouraging the _development_ of subpar or non-scaleable solutions. If the application _already exists_, on the other hand, it should be used and tweaked in as many ways as you can (witness our continued use of IMail!). :) I just think that supporting a distributed LDAP environment is unnecessary if done solely for the purpose of storing several hundred to several tens of thousands of E-mail addresses. Several hundred in an unindexed in-memory array would probably work jsut fine. Tens of thousands is a very, very different story. Again, you seem to be missing the point in thinking these two situations don't present different requirements. Solely for the purpose of scaleability is one of the purest and most commendable motivations in application design, since it encompasses both in the wild stability and performance under a simple umbrella. Far from a dirty word, scaleability is what makes so many open-source projects work in the enterprise, despite their many other foibles. If you start a development project with an express disregard for it, count out the most capable programmers. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Sanford Whiteman wrote: Jsut fine. Tens of thousands is a very, very different story. Again, you seem to be missing the point in thinking these two situations don't present different requirements. "Solely for the purpose of scaleability" is one of the purest and most commendable motivations in application design, since it encompasses both "in the wild" stability and performance under a simple umbrella. Far from a dirty word, scaleability is what makes so many open-source projects work in the enterprise, despite their many other foibles. If you start a development project with an express disregard for it, count out the most capable programmers. My friend is one of the most capable programmers that you will find, he's done a great deal of work in the last 5 years within Microsoft's framework, and I don't expect for this to be a challenge for him. I'm still waiting to see if he wants to take this on. In terms of scale, I would expect to see a server handle not much more than 500,000 messages in a full Declude/IMail environment, and with an average of more than 10 pieces of spam per address per day, a solution of this sort would need to effectively resolve against 50,000 or so E-mail addresses. While I'm not at all sure how to properly index this information for rapid use, I do know that you could split the data into user and domain, and first query the domain, and then the user, and that would likely mean for the most part that you would need to do one query (full string match) on about 1,000 domains, and then another query on an average of maybe 50 user addresses. Pete over at Sniffer has figured out how to search the entire source of a message with tens of thousands of rules complete with wildcards, and he does that quite efficiently considering that the application loads the entire rule base every time it is hit with a message. I think a capable programmer would not at all be bothered by the demands. There's absolutely no reason why this couldn't be done. If you have a recommendation for how to best handle the task where data is initially sourced from a text file, please share it and I will pass that on. Thanks, Matt -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
My friend is one of the most capable programmers that you will find, he's done a great deal of work in the last 5 years within Microsoft's framework, and I don't expect for this to be a challenge for him. This is not at all a comment on his skills--many of us program for Win32 as well--but you're talking about an OS platform whose companion mail platform (Exchange) had no way (zero) to reject at the envelope until last year. In terms of scale, I would expect to see a server handle not much more than 500,000 messages in a full Declude/IMail environment, and with an average of more than 10 pieces of spam per address per day, a solution of this sort would need to effectively resolve against 50,000 or so E-mail addresses. # of messages has no intrinsic relationship to # of users. These are different requirements, though they are related insofar as the former predicts the number of simultaneous lookups against the data source that must be completed without quenching socket, memory, or CPU resources. In any case, you're defining this requirement: Must support up to 50,000 addresses. That's fine for you. MXs we support service millions of accounts in constant flux due to adds and changes. Something built to your requirements would not be sufficient for us. As I mentioned, however, _even you_ have no need to build anything: ORF already does what you need. While I'm not at all sure how to properly index this information for rapid use, I do know that you could split the data into user and domain, and first query the domain, and then the user, and that would likely mean for the most part that you would need to do one query (full string match) on about 1,000 domains, and then another query on an average of maybe 50 user addresses. You're goldmanning--I guess that's the opposite of strawman :)--one of a zillion use cases to match your design, so that's not an accurate general depiction of MXs that accept mail for 50,000 accounts. Our largest installations by user count have very small numbers by domain count. Pete over at Sniffer has figured out how to search the entire source of a message with tens of thousands of rules complete with wildcards, and he does that quite efficiently considering that the application loads the entire rule base every time it is hit with a message. A very different task. I won't bother you with any more differentiators. Suffice it to say that tens of thousands of objects is not a realistic target for a scaleable mail application. It may be a realistic target for a particular deployment. I am not questioning that it may work for you, but (see below) there's nothing to build! I think a capable programmer would not at all be bothered by the demands. There's absolutely no reason why this couldn't be done. My ultimate point is that _there is no reason for anything to be written_. If you want 50,000 users and text file input is what you want, use ORF. Geez, it's 99 bucks. Vamsoft has done a very fine job with their product. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Sandy, I would prefer to pay $99 for a product that did what I wanted it to. My issue is that I don't want to rely on AD or LDAP, though I would consider a DNS implementation (with translation of addresses to valid values, like [EMAIL PROTECTED] becomes matt.mail.example.com.my-filter-domain.com). If VAMsoft builds this, please let me know. If I find that there is no interest on the part of my friend in programming this, I may very well think about going the LDAP route for lack of the "proper cable." :) Matt Sanford Whiteman wrote: My friend is one of the most capable programmers that you will find, he's done a great deal of work in the last 5 years within Microsoft's framework, and I don't expect for this to be a challenge for him. This is not at all a comment on his skills--many of us program for Win32 as well--but you're talking about an OS platform whose companion mail platform (Exchange) had no way (zero) to reject at the envelope until last year. In terms of scale, I would expect to see a server handle not much more than 500,000 messages in a full Declude/IMail environment, and with an average of more than 10 pieces of spam per address per day, a solution of this sort would need to effectively resolve against 50,000 or so E-mail addresses. # of messages has no intrinsic relationship to # of users. These are different requirements, though they are related insofar as the former predicts the number of simultaneous lookups against the data source that must be completed without quenching socket, memory, or CPU resources. In any case, you're defining this requirement: "Must support up to 50,000 addresses." That's fine for you. MXs we support service millions of accounts in constant flux due to adds and changes. Something built to your requirements would not be sufficient for us. As I mentioned, however, _even you_ have no need to build anything: ORF already does what you need. While I'm not at all sure how to properly index this information for rapid use, I do know that you could split the data into user and domain, and first query the domain, and then the user, and that would likely mean for the most part that you would need to do one query (full string match) on about 1,000 domains, and then another query on an average of maybe 50 user addresses. You're goldmanning--I guess that's the opposite of strawman :)--one of a zillion use cases to match your design, so that's not an accurate general depiction of MXs that accept mail for 50,000 accounts. Our largest installations by user count have very small numbers by domain count. Pete over at Sniffer has figured out how to search the entire source of a message with tens of thousands of rules complete with wildcards, and he does that quite efficiently considering that the application loads the entire rule base every time it is hit with a message. A very different task. I won't bother you with any more differentiators. Suffice it to say that tens of thousands of objects is not a realistic target for a scaleable mail application. It may be a realistic target for a particular deployment. I am not questioning that it may work for you, but (see below) there's nothing to build! I think a capable programmer would not at all be bothered by the demands. There's absolutely no reason why this couldn't be done. My ultimate point is that _there is no reason for anything to be written_. If you want 50,000 users and text file input is what you want, use ORF. Geez, it's 99 bucks. Vamsoft has done a very fine job with their product. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
If VAMsoft builds this, please let me know. If I find that there is no interest on the part of my friend in programming this, I may very well think about going the LDAP route for lack of the proper cable. Did you fail to read (twice) the part of my posts about the accept only for these users option in ORF, which is loaded from a text file? This has nothing to do with LDAP. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Sanford Whiteman wrote: Did you fail to read (twice) the part of my posts about the accept only for these users option in ORF, which is loaded from a text file? This has nothing to do with LDAP. To be honest, yes, I don't think I saw that in your messages. Take it from a fellow rambler...you could condense things from time to time...and maybe spend less time describing how I'm wrong or how impossible a task might be :) I saw a reference to an everybody but blacklists, but their help file makes no such reference. I suppose that you inquired about this functionality? My read of their help file shows that it might be possible to blacklist everything, and then whitelist the addresses that you want to accept...if they process the whitelist first. Or maybe this is undocumented or at least difficult to find in their documentation. Nevertheless, thanks for the pointer. Matt -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
To be honest, yes, I don't think I saw that in your messages. Take it from a fellow rambler...you could condense things from time to time...and maybe spend less time describing how I'm wrong or how impossible a task might be :) Maybe... I saw a reference to an everybody but blacklists, but their help file makes no such reference. I suppose that you inquired about this functionality? My read of their help file shows that it might be possible to blacklist everything, and then whitelist the addresses that you want to accept...if they process the whitelist first. It's simple and built-in functionality, not a tweak or anything like that all. You simply enable the recipient blacklist button in the everybody but these people mode (one of two modes). There's no need to worry about processing order. All addresses are in plain-text and will reload when the ORF service restarts. It's exactly what your spec suggests. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Sanford Whiteman wrote: It's simple and built-in functionality, not a tweak or anything like that all. You simply enable the recipient blacklist button in the everybody but these people mode (one of two modes). There's no need to worry about processing order. All addresses are in plain-text and will reload when the ORF service restarts. It's exactly what your spec suggests. I don't have a functioning install, just something on an incapable machine which exposes their help files, so I didn't get to see their config screens. This definitely looks like it will work just fine, even if it doesn't scale to 50,000 addresses :) I figure that I'll probably take a look at the IMail to IMGate export script that I've seen mentioned before, or maybe a utility on the Ipswitch site for generating the locally hosted portion of this file, and I'll probably write a database app for managing the gatewayed domains, combining the two into a suitable format for ORF. What I hope to do is establish this audit trail for my customers where they have almost real-time access to add or remove addresses from the service. If they don't want to maintain a list, then I'll just charge them a bit more since that means more utilization. Best of both worlds I figure. This is also the type of thing that I can handle without much help. Thanks, Matt -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
In terms of scale, I would expect to see a server handle not much more than 500,000 messages in a full Declude/IMail environment, and with an average of more than 10 pieces of spam per address per day, a solution of this sort would need to effectively resolve against 50,000 or so E-mail addresses. While I'm not at all sure how to properly index this information for rapid use, I do know that you could split the data into user and domain, and first query the domain, and then the user, and that would likely mean for the most part that you would need to do one query (full string match) on about 1,000 domains, and then another query on an average of maybe 50 user addresses. Pete over at Sniffer has figured out how to search the entire source of a message with tens of thousands of rules complete with wildcards, and he does that quite efficiently considering that the application loads the entire rule base every time it is hit with a message. I think a capable programmer would not at all be bothered by the demands. There's absolutely no reason why this couldn't be done. If you have a recommendation for how to best handle the task where data is initially sourced from a text file, please share it and I will pass that on. Speaking of Sniffer - One thing you might consider is creating a special rulebase (we do contracts like that) that would contain 50K rules to match, well, practically any text you wish. We regularly match 50K heuristics these days in sub 100ms. Perhaps there is a special solution to be worked out here. We have tools to make this kind of thing feasible... Depending upon the rate of change, this might not require any unique software. We have a prototype java based utility for scripting updates to any rulebase in our system. Contact me off list if you'd like to pursue this direction. _M RESCU - REmote SCripted Updater, accepts an XML file representing changes/commands for the rulebase and produces a matching XML file result. Not quite ready for release into the wild, but close.
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
Pete, Everything that Sniffer does is after submission, so it really wouldn't apply. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
Sorry about that - I seem to have stepped into a bit of a tiff. I was skimming and saw a Sniffer reference and jumped in - I shouldn't do that (I should get more sleep). At any rate, the pattern matching engine can run at any point... Sniffer as it is packaged now runs after submission, but the engine can easily be used up-front during the SMTP conversation before or after DATA. That's just not how it's currently packaged. The pattern matching engine came from my robotics research and was later adapted to fast interpreted scripting engines in he early 80s (When cpus and memory were slow and bulky). The concept for robotics was that a complex hierarchical reflex mechanism capable of real-time responses would be continually tuned by slower analysis engines. What is now inside Message Sniffer was once designed to interpret a wide array of sensor data and produce complex, directed real-time responses under the guidance (symbiotically anyway) of a goal seeking machine learning system. It was a kind of autonomic nervous system with a bit of brain-stem attached. If anybody cares to take the technology to the SMTP end of an email application (or even level 3 routing / filtering / switching) it can be done extremely well... We have to start somewhere though... So we filter spam - go figure. Anyway, as has been pointed out, for this application there are tools available that need no repackaging or development. (even if it might be fun) Best, _M At 11:46 PM 2/10/2004, you wrote: Pete, Everything that Sniffer does is after submission, so it really wouldn't apply. It could be adapted to any application where a rapid recognition and response to data patterns is required. For example, picking an email address out of an SMTP envelope, or for that matter implementing the entire protocol (though that might be a silly thing to do). It does spam filtering after submission right now just because it's packaged that way. (I'm not bad, I'm just drawn that way... Jessica Rabit) --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Pete, I try not to get too passionate about things around here, so I welcome your contribution. You are correct though, after a couple of days of discussion, the solution to this need does appear to exist. I have a great appreciation for your skill, and your willingness to share both insight and code (open source). If I was a programmer, I would probably be spending my time playing with your code as opposed to playing with filters in Declude. While none of this is a technical requirement of mine at the moment, there's lots of opportunity I think for someone to make use of the things that have appeared in this thread. In going back to the don't have the right cable analogy, it would be great to have a backup MX that didn't require IMail (or another full mail server), and I think that could be done within MS SMTP without needing to re-create the wheel, and maybe more efficiently. On my wishlist would be the following: 1) Envelope rejection (and all that comes with it). 2) SMTP AUTH (so it can co-exist on the same server as IMail, and handle hosted accounts with redundancy). 3) An external application handler that would allow for things like Declude JunkMail, Virus, and Message Sniffer. 4) A message splitter, so actions can be based on individual addresses instead of individual messages. If you guys could work this out, Declude in combination with Message Sniffer could truly go multi-platform (as far as Windows mail servers go). Who knows, maybe MS SMTP has some serious issues that would make you want to avoid it. BTW, I'm looking forward to the 3.0 features. Bayesian filtering with Sniffer's rule base I believe will significantly strengthen your system, though I would like to see your customer submitted data grow so that the rule strengths can become more accurate. Hopefully this will allow one to tune their system to their own definition of what spam is, right now it's tough in general for us guys that want to accept virtually all E-mail from sources that maintain direct relationships. I've taken to creating my own database for managing this information in 10 different categories, which then outputs credit files for Declude to use. I'm now thinking that your solution may be more efficient, and sometimes more accurate because of greater filtering capability, though it can't handle things like reverse DNS entries and the SMTP envelope sender...I'll have to give it some thought. Right now these lists are short, and Declude easily handles them in custom filter form. Matt Pete McNeil wrote: Sorry about that - I seem to have stepped into a bit of a tiff. I was skimming and saw a Sniffer reference and jumped in - I shouldn't do that (I should get more sleep). At any rate, the pattern matching engine can run at any point... Sniffer as it is packaged now runs after submission, but the engine can easily be used up-front during the SMTP conversation before or after DATA. That's just not how it's currently packaged. The pattern matching engine came from my robotics research and was later adapted to fast interpreted scripting engines in he early 80s (When cpus and memory were slow and bulky). The concept for robotics was that a complex hierarchical reflex mechanism capable of real-time responses would be continually tuned by slower analysis engines. What is now inside Message Sniffer was once designed to interpret a wide array of sensor data and produce complex, directed real-time responses under the guidance (symbiotically anyway) of a goal seeking machine learning system. It was a kind of autonomic nervous system with a bit of brain-stem attached. If anybody cares to take the technology to the SMTP end of an email application (or even level 3 routing / filtering / switching) it can be done extremely well... We have to start somewhere though... So we filter spam - go figure. Anyway, as has been pointed out, for this application there are tools available that need no repackaging or development. (even if it might be fun) Best, _M At 11:46 PM 2/10/2004, you wrote: Pete, Everything that Sniffer does is after submission, so it really wouldn't apply. It could be adapted to any application where a rapid recognition and response to data patterns is required. For example, picking an email address out of an SMTP envelope, or for that matter implementing the entire protocol (though that might be a silly thing to do). It does spam filtering after submission right now just because it's packaged that way. (I'm not bad, I'm just drawn that way... Jessica Rabit) --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
1) Envelope rejection (and all that comes with it). Already extant, as previously discussed. 2) SMTP AUTH (so it can co-exist on the same server as IMail, and handle hosted accounts with redundancy). This is going to be very difficult relative to the other ideas, if you continue to resist AD. With AD as the back end, you can authenticate to SMTP using any valid credentials in any permissioned context. It's already done like this by people who run Exchange and want to instantly offload SMTP AUTH sessions from their mailbox servers. I do not think that adding an additional out-of-context authentication method is going to be worth attempting. 3) An external application handler that would allow for things like Declude JunkMail, Virus, and Message Sniffer. Well...we're basically already doing this with a transport event sink. I didn't want to mention it yet, but I've been using our own external tests under MS SMTP for the past month on one server, for example. 4) A message splitter, so actions can be based on individual addresses instead of individual messages. Easy enough to code within an event sink, though I've never had a desire for this because the overhead could be crippling and it's quite counter to SMTP as a protocol. Giving Declude the ability to (a) natively interpret a single RFC822 file with MS headers, as passed by MS SMTP (right now, you'd have to write out a dummy Q file, which is easy but an admitted extra step) would be nice to have. And being able to disable all daisy-chaining with a GLOBAL.CFG setting, since MS will automatically proceed with message processing once control is returned to the service, would make SMTP32 log errors go away. IMO+E, none of this requires anything crazy to be done by SortMonster or Declude--except for licensing clarifications! :) --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
[Declude.JunkMail] OT: IMAIL - AD
Hi Sandy: It's no-brainer if you use IMail's NT integration on an AD DC. I don't want to reinvent the wheel, so I'm trying to research this by reading the Imail 8 manual. It doesn't reference AD directly (only the NT User setup and that you have to run on a DC). So before I invest time and play around with it, I have three no-brainer questions, which I could not answer myself: - It says that you can't use the Imail Explorer to manage accounts (users, aliases, etc.) - does that imply that my clients wouldn't be able to use WebMail to add/administer their own mailboxes either? - Does the AD only store Users (mailboxes) - or also Alias (e.g., simple alias, group alias, program alias, etc.)? If not, then how do you accomplish using the AD information to verify valid RCTP TO information? A good portion of the email we process is addressed to an alias!? - Does the Imail/NT/AD integration support (multiple) virtual domains (ip-less) - or will it only create users for the AD domain name? Accordingly, how will it know that two mailboxes and/or aliases by the same name, but on two different mail domains, should be kept as two different entities in AD? Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanford Whiteman Sent: Monday, February 09, 2004 03:47 PM To: Andy Schmidt Subject: Re[2]: [Declude.JunkMail] [Declude.Junkmail] MS SMTP LDAP Routing I would seriously consider funding some of the development for an IMAIL/LDAP lookup event sink as it would help my SMTP server to disconnect on dictionary attacks. I already use ORF to reject at the envelope using LDAP lookups and really have no need for any other intermediary. It's no-brainer if you use IMail's NT integration on an AD DC. All you need to do is add the Exchange schema extensions to the AD domain, since ORF is expecting the extended schema--you don't have to install or purchase Exchange itself. You can run the ORF queries against any server in the domain (which doesn't have to be the same as your primary domain), meaning that you can scale out from hitting the mailbox server directly to hitting dedicated AD DCs that only service such MX lookups. Building anything designed to interact with IMail's own ILDAP daemon is a very bad move, as the service is barely functional, compliant, or stable. AD's LDAP services, on the other hand, are mature and resilient. The other options that involve local text files would certainly work, but performance under load could not exceed that of indexed LDAP lookups. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Andy, I think what he meant was that you would import the data from IMail into AD. IMail would still use it's own methods for storing and accessing account information, but ORF would make use of the AD stuff that you exported to it. Personally, I don't use AD on my server because it doesn't seem to offer me anything of value and adds complexity. The server is a stand-alone box, and from a security standpoint, I believe it is best for it to remain that way. I'm asking my buddy to look into this. We certainly wouldn't come out with something that did RBL scanning (DNSBL if Scott's listening :) ), but I'm pretty sure we could get this to make use of a text file dump fairly easily. ORF was written for an Exchange environment and it might be easier to write something more appropriate for ours. If Vamsoft came out with a different tool, then I would be all for giving them money instead. Matt Andy Schmidt wrote: Hi Sandy: It's no-brainer if you use IMail's NT integration on an AD DC. I don't want to reinvent the wheel, so I'm trying to research this by reading the Imail 8 manual. It doesn't reference AD directly (only the NT User setup and that you have to run on a DC). So before I invest time and play around with it, I have three "no-brainer" questions, which I could not answer myself: - It says that you can't use the Imail "Explorer" to manage accounts (users, aliases, etc.) - does that imply that my clients wouldn't be able to use WebMail to add/administer their own mailboxes either? - Does the AD only store "Users" (mailboxes) - or also "Alias" (e.g., simple alias, group alias, program alias, etc.)? If not, then how do you accomplish using the AD information to verify "valid" RCTP TO information? A good portion of the email we process is addressed to an alias!? - Does the Imail/NT/AD integration support (multiple) virtual domains (ip-less) - or will it only create users for the AD domain name? Accordingly, how will it know that two mailboxes and/or aliases by the same name, but on two different mail domains, should be kept as two different entities in AD? Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Sanford Whiteman Sent: Monday, February 09, 2004 03:47 PM To: Andy Schmidt Subject: Re[2]: [Declude.JunkMail] [Declude.Junkmail] MS SMTP LDAP Routing I would seriously consider funding some of the development for an IMAIL/LDAP lookup event sink as it would help my SMTP server to "disconnect" on dictionary attacks. I already use ORF to reject at the envelope using LDAP lookups and really have no need for any other intermediary. It's no-brainer if you use IMail's NT integration on an AD DC. All you need to do is add the Exchange schema extensions to the AD domain, since ORF is expecting the extended schema--you don't have to install or purchase Exchange itself. You can run the ORF queries against any server in the domain (which doesn't have to be the same as your primary domain), meaning that you can scale out from hitting the mailbox server directly to hitting dedicated AD DCs that only service such MX lookups. Building anything designed to interact with IMail's own ILDAP daemon is a very bad move, as the service is barely functional, compliant, or stable. AD's LDAP services, on the other hand, are mature and resilient. The other options that involve local text files would certainly work, but performance under load could not exceed that of indexed LDAP lookups. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
I think what he meant was that you would import the data from IMail into AD. IMail would still use it's own methods for storing and accessing account information, but ORF would make use of the AD stuff that you exported to it. Nope, it's total and automatic synchronization of the userbase, which can then be replicated to as many LDAP slaves as necessary. That's the power. Personally, I don't use AD on my server because it doesn't seem to offer me anything of value and adds complexity. LDAP is its thing of value. :) The server is a stand-alone box, and from a security standpoint, I believe it is best for it to remain that way. You can still run AD on a standalone DC, like I mentioned. Restrict LDAP queries to the MXs, etc. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
- It says that you can't use the Imail Explorer to manage accounts (users, aliases, etc.) - does that imply that my clients wouldn't be able to use WebMail to add/administer their own mailboxes either? You can't add or delete AD users via the IMail interfaces, but you can set IMail properties, such as Reply-To:, disabled, max mailbox size, etc. - Does the AD only store Users (mailboxes) - or also Alias (e.g., simple alias, group alias, program alias, etc.)? If not, then how do you accomplish using the AD information to verify valid RCTP TO information? A good portion of the email we process is addressed to an alias!? You add the Alias information in AD as well (using the particular LDAP attribute designed for this). Users without access to AD couldn't do this. - Does the Imail/NT/AD integration support (multiple) virtual domains (ip-less) - or will it only create users for the AD domain name? Accordingly, how will it know that two mailboxes and/or aliases by the same name, but on two different mail domains, should be kept as two different entities in AD? It's a solution built for single or overlapping userbases (which is not the same as just having a single domain--we use this for sites that have several distinct domains, maildirs, etc., though they are all corporate). It's not for non-overlapping userbases as you'd have insituations where you're offering delegated, segregated administration for multiple client domains. So...never said it would work for everyone, but it's dirt easy. We do use _much_ more complex and time-costly stuff than this when we need to segregate and subdelegate mail domains, but this has come handy quite a bit. --Sandy P.S. Something else to consider when contemplating an event sink for a subdelegated environment is using ODBC lookups against per-domain tables. Anything that relies on scanning unindexed data on the scale of tens of thousands of items is just bad programming. Anything but that. Remember, the mighty ASCII-centric Postfix knows enough to use indexed data at runtime. Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: IMAIL - AD
Andy (and Sandy), I'm not dumping on LDAP, I think it can be very useful, however in this case, is it really necessary? Why not just support loading a text file into memory and using that? It's the lowest common denominator and people without LDAP knowledge or software could make use of it. The only reason not to use text files would be a technical limitation, but I'm not suggesting that it be accessed once per message, so that isn't at issue. I would certainly look to VAMsoft for this application if they supported text files, otherwise it looks easy enough to create. I'm assuming that many around here that would consider such a tool would prefer that all spam processing be done within Declude...maybe not either or you, but most definitely some. I would prefer this myself as long as capacity wasn't an issue. The only stuff that I would block with VAMsoft is efficiently taken care of within Declude without touching a single custom filter (besides spamdomains, fromfile and ipfile types which currently can't be skipped, but no doubt will have that capability soon). Matt Andy Schmidt wrote: Message VAMsoft has indicated on their newsgroup that they consider supporting non-AD LDAP validation. It has been requested several times afterthey introduced the AD synch and VAMsoft has been very responsive to customer requests in the past. (If Sandy's idea was to EXPORT the user and alias and import it into LDAP (or Active Directory), then, yes, that could be workable way. However he was very specific in saying "if you use IMail's NT integration" and that would be the feature where you make all your users NT users (or AD users).) Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax: +1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Monday, February 09, 2004 05:17 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] OT: IMAIL - AD Andy, I think what he meant was that you would import the data from IMail into AD. IMail would still use it's own methods for storing and accessing account information, but ORF would make use of the AD stuff that you exported to it. Personally, I don't use AD on my server because it doesn't seem to offer me anything of value and adds complexity. The server is a stand-alone box, and from a security standpoint, I believe it is best for it to remain that way. I'm asking my buddy to look into this. We certainly wouldn't come out with something that did RBL scanning (DNSBL if Scott's listening :) ), but I'm pretty sure we could get this to make use of a text file dump fairly easily. ORF was written for an Exchange environment and it might be easier to write something more appropriate for ours. If Vamsoft came out with a different tool, then I would be all for giving them money instead. Matt Andy Schmidt wrote: Hi Sandy: It's no-brainer if you use IMail's NT integration on an AD DC. I don't want to reinvent the wheel, so I'm trying to research this by reading the Imail 8 manual. It doesn't reference AD directly (only the NT User setup and that you have to run on a DC). So before I invest time and play around with it, I have three "no-brainer" questions, which I could not answer myself: - It says that you can't use the Imail "Explorer" to manage accounts (users, aliases, etc.) - does that imply that my clients wouldn't be able to use WebMail to add/administer their own mailboxes either? - Does the AD only store "Users" (mailboxes) - or also "Alias" (e.g., simple alias, group alias, program alias, etc.)? If not, then how do you accomplish using the AD information to verify "valid" RCTP TO information? A good portion of the email we process is addressed to an alias!? - Does the Imail/NT/AD integration support (multiple) virtual domains (ip-less) - or will it only create users for the AD domain name? Accordingly, how will it know that two mailboxes and/or aliases by the same name, but on two different mail domains, should be kept as two different entities in AD? Best Regards Andy Schmidt HM Systems Software, Inc. 600 East Crescent Avenue, Suite 203 Upper Saddle River, NJ 07458-1846 Phone: +1 201 934-3414 x20 (Business) Fax:+1 201 934-9206 http://www.HM-Software.com/ -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Sanford Whiteman Sent: Monday, February 09, 2004 03:47 PM To: Andy Schmidt Subject: Re[2]: [Declude.JunkMail] [Declude.Junkmail] MS SMTP LDAP Routing I would seriously consider funding some of the development for an IMAIL/LDAP lookup event sink as it woul
Re[2]: [Declude.JunkMail] OT: IMAIL - AD
I'm not dumping on LDAP, I think it can be very useful, however in this case, is it really necessary? Why not just support loading a text file into memory and using that? Because it's poor architecture that I wouldn't trust on my mailserver. It's the lowest common denominator... Yep, that's the problem, all right. :) The only reason not to use text files would be a technical limitation, but I'm not suggesting that it be accessed once per message, so that isn't at issue. Then you clearly don't see the _other_ technical problems involved. Disk I/O is not the primary problem. I would certainly look to VAMsoft for this application if they supported text files... Well, you _can_ use ORF for this! Just use their everybody but recipient blacklist, whose addresses are stored in the .INI file and read once at service start (ORF service, not SMTP service). Every time you update the file, net restart ORF. It's _already_ there for you in ORF if this is the way you want to swing it. I believe that if you have a single domain, AD via LDAP is the better way to go. As a longtime LDAP user, I believe your concerns about the complexity of having a built-in LDAP service running with the sole purpose of MX user lookup are unfounded. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/download/software/freeutils/SPAMC32/Release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
