This is quite long. My apologies. After only two responses, I found I had a lot to clarify and more to add to the debate. I knew it would be controversial, but I think it's a direction we need to go. We need to at least have this capability. Every MTA in the world doesn't have to implement it, but I believe we need a solution (or multiple solutions to choose from) to implement if we so choose.
DM: Derek Martin KL: Keneth Lussier [My intro about user controlled SMTP rejections snipped] DM> This is difficult, if not impossible, to acheive. At least, if I DM> understand you correctly. The SMTP gateway is a mail TRANSFER agent, DM> and knows nothing about what the client wants or doesn't want, BY DM> DESIGN. The exception is probably MS Exchange, which I don't know DM> much about, and wouldn't ever use/manage unless I were starving to DM> death. I think I'ld flip burgers before managing an Exchange environment :-). Anyhow, sendmail already provides ways of filtering based on To: or From: fields in the access database. All that's needed is to combine the two to provide the functionality I'm talking about. That and, of course, a way for recipients to securely control it without compromising the entire mail system. Impossible? I doubt it. Difficult, yes, but that's why I posted to this group -- to get some tips. Whether or not it was originally by design that the MTA knows nothing about the client (client referring to end user in this case) wants or doesn't want matters not. The mere existence of rfc2505 (particularly section 3.2) indicates that I'm not the only one that wants that to change. KL> There are ways of doing this, but they are not clean, nor are they secure. KL> Most major, and some minor, MTA's allow the use of "Black List" lookups. KL> Your MTA can consult things like the RBL to see if mail is from a black KL> listed domain, sender, etc. One of the problems with this is that your MTA KL> is now depending on someone else's judgement, and someone elses security. If KL> the RBL is ever cracked, it would wreak havoc with the mail systems that KL> use it. Even depending on my own judgement as a sysadmin is not so good. I want to give the choice to individuals. I have heard the RBL is actually becoming less and less popular. They've taken more questionable actions over recent years and it still doesn't give the end user the choice. It doesn't even give organizations the choice. It's all or nothing. Vipul's Razor is a great idea as it deals with specific messages, but it remains to be seen how effective it is. Still, it only deals with messages already delivered through SMTP -- not what I looking to do. I want it rejected before being sucked into my servers. [My criticism of mail filtering tools snipped] DM> This is not a shortcoming in procmail. It's a problem with your DM> corporate policy. If you're going to lay blame, please do so where it DM> belongs... DM> DM> A workaround for this problem is to run fetchmail to get your mail DM> from your IMAP server, and filter it with procmail locally. And DM> IMNSHO, it's a much cleaner way than what you're about to suggest... As Paul L. mentions in a later message, there's no problem with this as a corporate policy. There are many arguments for sealed servers, but I believe that it's beyond the scope of the points I'ld to make. (I'm happy to discuss it, but I just think it's a different, though related, discussion.) I'm not really 'blaming' procmail, anyhow. It does what it does well. Problem is, that it's not designed for filtering using only IMAP. I personally cringe at alternating my access of an INBOX between IMAP and procmail or any other non-IMAP access. Leaving Cyrus out the picture for the moment, let's take a look at UW-IMAP. Upon accessing a folder, it puts a dummy message in the front of the file with some folder information. It looks pretty innocuous and I doubt it would ever be a problem deleting it and letting UW-IMAP recreate it, but I just don't trust letting non-techie users do stuff to IMAP folders unless it's through IMAP only. It's comparable (though not as bad as) to modifying an rcs file by hand. I'm not looking for workarounds, I'm looking for real solutions. Sieve, IMO, is actually a much cleaner solution. It allows me to shut my desktop down (if I so choose) and still have stuff faithfully filtered as I've specified. And I can then access my folders from home through a VPN without having make sure my procmail filters are in sync between home and work (which *I might not want*, anyhow). KL> Filtering should always be done at the client side, IMO.It's the user KL> that chooses the client, and the user that wants the filters. There KL> is no reason to put extra strain on a mail server, especially if it KL> is a high traffic environment, by asking the MTA to think for you as KL> well. I respectfully disagree. I've wanted server-side filtering for a long time. I was pleased to find that sieve has actually made it to the rfc stage and that Cyrus has already implemented it. I'm not trying to suggest that we load up are mail servers with a bunch of unneeded functionality and risk them getting too bloated. But let's not be afraid of taking advantage of the fact that you can easily find a 1GHz box with 1/2 Gig of RAM for a mere 600 bucks these days. [My promotion of sieve snipped] DM> Unless I'm mistaken (which is very possible), Cyrus mail tools require DM> the use of Maildir format mailboxes, which just aren't supported all DM> that well. True, a lot of major mail clients support it, but a lot of DM> popular clients don't. And if you need to read mail with mailx in an DM> emergency, forget it. Cyrus is not a set of client tools, it's an IMAP server. You access folders through IMAP running on a sealed server -- users don't need shell accounts in order to have IMAP accounts. True, you can't use mailx, but the typical non-techie is not going to be a mailx user any time soon. If the mail server is down, they're down. It's my job to make that possibility minimal. What you may be thinking of is the fact that Cyrus stores each message in a separate file and has some additionally db files for index info. I used to be a big proponent of one file per folder -- until my folders (especially my INBOX) grew to tens and even a few hundred megabytes. UW-IMAP falls over badly in situations like this. [My details about SMTP rejections snipped] DM> This seems to me to be making the process of delivering mail to a user DM> entirely too complex. The LAST thing I want, as a system DM> administrator, is dependence on a database program to make delivery of DM> mail work. When you do this, you've probably increased the complexity DM> of mail delivery by more than 100%, given how basically simple DM> sendmail is. That means headaches for me, and I don't like it. DM> DM> You can even configure procmail to send the sender bounce messages, if DM> you really want to. YAY! So I've doubled the complexity of email delivery. IMHSO, that's not that bad, given the potential payoff. And you don't have to use this, but there are many sysadmins that are as fed up with spam as I am. IMO, the payoff is great. I don't want my users to have send bounces, I want the mail they've chosen to reject to never enter my servers. A possible analogy is giving homeowners the right to not answer their doors when unwanted and/or unknown visitors arrive. That's one right that's a given. I don't think it's right for me to be opening the door for these strangers to harass the homeowners (it's not *their* equipment in most cases, but it is *their* time). Really, the door is just too open by default. I'm trying to find a way to close it and let my users decide who to open it for. KL> You can configure procmail to do pretty much everything. If you just KL> really don't like procmail for some odd reason, then use maildrop or KL> seive. They are all relativly the same. The difference with sieve, however, has to do with it's server-side focus and a protocol for transfering scripts. Mulberry from Cyrusoft (http://www.cyrusoft.com -- non-free) already supports a gui for generating and transfering scripts for the average user. It's kind of procmail meets the newbie-non-techie-gui-"what's a command line" user. KL> I 100% agree here. Making your MTA depend on a database backend just KL> seems suicidal. Not to mention the performance hit you would take if KL> you have a high traffic environment. If every single piece of e-mail KL> requires a database query, you will slow the mail server down KL> considerably. Especially if you have tables with thousands of entries, KL> which you would have to have if there needs to be an entry for everyone KL> that you *WILL* accept mail from. Besides the performance hit to the mail KL> server, just imagine the performance hit the sysadmins would take!! I KL> spend hours a day responding to e-mail. Now, imagine having to respong KL> to an e-mail saying that you will accept mail from this person, then KL> waiting for their e-mail, then responding to it... I don't have time KL> for that. I admit making mail delivery depend on a db is risky. In fact, rfc2505 does mention in general terms the performance and complexity issues as well as potential new DoS risks. Failure modes and high load situations definitely need to be studied. I'ld like to start on the path of getting something implemented so it *can* be studied rather than just debated. Debate is good, but as is often said the world of Free Software, show me the code. Although I'm sure that having additional processing like this done at the SMTP gateway will increase load, I have to wonder what the tradeoff would be with the decrease in the amount of spam actually sucked into our servers and clogging up our queues and user INBOXes. It may not be an even trade, but I have a suspicion that it would be enough that the remaining performance issues could be addressed to the point of being tolerable. It may not be as zippy as it once was, but it's quite possible it'll be acceptable in the face of the amount of spam that is averted. DM> Yes, I have some thoughts. What you're describing is a sysadmin's DM> worst nightmare, and I for one want no part of it. It's a great DM> example of overengineering a reletively simple problem. The best DM> solution for dealing with spam always was and always will be to press DM> the delete key. Anything else runs the risk of losing legitimate DM> mail if it's not configured meticulously, and if you ask me what you DM> describe introduces enough aditional complexity to make it very much DM> not worthwhile. The best solution for dealing with spam has never been and will never be the delete key. The first (but certainly not the last) story I have of this is a case about five years ago when someone I worked with was deleting the mucho spam she had received since the previous night. She inadvertently deleted a *very* important legitimate email about checking some code in for that night's build. She missed the email, missed the build, and as a result development lost some time (time == money in business) all due to using the 'delete' key for spam. Given the amount of spam she was getting, she can't really be faulted that much for inadvertently deleting the legitimate message. That was five years ago. There have been similar situations since then. She's not the only one. And the volume of spam has increased about 650% since last year (http://news.zdnet.co.uk/story/0,,t269-s2100516,00.html). The problem is getting so much worse that soon searching for legitimate email will be like trying to find a needle in a haystack...of pins. Filtering users' email based on *my* decisions runs the risk of losing legitimate email. Giving users the choice to reject mail at the SMTP gateway will only reject mail that they've specifically rejected or rejected through inaction. Coded well (which is my endeavor, of course), rejected mail will be bounced because the upstream SMTP server gets a permanent failure error code. If it's legitimate mail, it'll have a correct return address and it will reach the sender. Then the sender will know that he needs to contact the recipient through another means to alert him of the problem. I guess one man's nightmare, is another's dream ;-). I am fanatically against spam and I'm passionate about find a solution. Resigning myself to the delete key is exactly what the those low-lifes at the Direct Marketing Association what me to do. I refuse to do it. I don't think legislation is necessarily the appropriate way to deal with it, so a technological one is what's needed. And we need one that doesn't have people screaming (mostly, legitimately) of constitutional violations. KL> Beyond the complexity issue, which is bad enough, this proposal KL> introduces several new points of failure, as well as several points KL> of risk, that, IMNSHO, just aren't worth it. Besides, it's time KL> consuming, wasteful, risky, and probably ineffectual. If a sysadmin KL> has to spend their entire day replying to e-mails just to say that KL> they will accept the e-mail, you have just invented the first DDoS KL> attack against a human being!! Well, second, actually. The first being KL> stupid questions from the marketing department ;-) It just seems like a KL> lot of work, a lot of risk, and a lot of time spent for very little KL> return. All they have to do is spoof their mail and/or IP address (Oh, KL> like *THAT's* hard). Points of failure, yes. But I can think of one acceptable solution to a failed database -- just ignore the filters and let the spam roll in. Sounds silly at first, but that's basically what we're tolerating today. A failed database doesn't have to be a disaster in this case. Hell, if it's totally hosed and you have to wipe it clean and start from scratch, is there much harm in that? Not really, given what we have today. It means reloading some default accept lists, but it's a one time action we only need to take in the case of a total failure. I hope we (I?) can code it well enough to avoid frequent *total* failures. Time consuming? From the context, I assume you're talking mostly about mail from people in your organization. What's wrong with adopting a default policy that says when a new user account is created, seed it with an accept list that includes everybody in the organization, possibly by domain name. Doesn't work for something like yahoo mail, but any corporate structure shouldn't have a problem with a policy like this. If an employee really want to reject all mail from the CEO, he can do that himself, later. Wasteful? How wasteful is the spam that clogs our servers daily? Especially since it's only going to get worse. Risky? I assume you mean to implement as a sysadmin. Of course it is. Anything that's unproven is risky. But I'ld like to see it proven to the point that it's more risky *not* to implement it with the amount of spam that would flow into our organizations otherwise. Ineffectual? Not if I have anything to say about it. If we can implement it in our organizations (and our upstream MX providers), it'll be the relays that have accepted the mail that get clogged. If your relaying mail for spammers, you need to take a hard look at your policies. Here's a excerpt from a file I threw together while doing some brainstorming: === Rationale: If you are seeing messages pile up in your queue, you need to ask yourself these questions: 1) Am I allowing relaying from a ISP that tolerates spammers? 2) Are any of my customers spammers? (If I'm an ISP.) 3) Do I have an open relay? Either: a) through neglect or oversight b) by design c) because I've been hacked For 1, 2, 3a, and 3c you need to take action to correct the problem. For 3b, perhaps you need to review why the choice was made to keep the relay open. === -- -Paul Iadonisi Senior Systems Administrator Red Hat Certified Engineer / Local Linux Lobbyist Ever see a penguin fly? -- Try Linux. GPL all the way: Sell services, don't lease secrets ***************************************************************** To unsubscribe from this list, send mail to [EMAIL PROTECTED] with the text 'unsubscribe gnhlug' in the message body. *****************************************************************
