RE: Contributing a mailet
Validating IP against the domain was one of many tests I was planning to do. It will probably block a few legal mail servers, but that should provide an incentive to add those servers/domains to DNS and be better net-citizens. The TMDA whitelist-centric strategy looks similar to what I was thinking of. But I find the email-based challenge/response might not be too difficult for spammers to circumvent. In any case they seem to have looked at many of the issues and is a great cross-check to ensure I did not miss something important. Thanks for the link!!! Bruno -Original Message- From: ext Noel J. Bergman [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 03, 2004 4:50 PM To: James Developers List Subject: RE: Contributing a mailet spammers are getting very smart about bypassing normal anti-spam tools. They have gone from being annoying, with dubious legal status for UCE, to being outright criminals. Spammers are behind the new worms that turn consumer systems into mail daemons. They can bypass whitelists by using the address books of the systems they have infected, and distributing the information so that databases of e-mail associations can be constructed. I feel that no other solution other than pure whitelisting will work in the long run. What alternative would you propose to whitelist-only email? Are you familar with www.tmda.net? There is a proposal where an MTA would validate that the IP address attaching to it is a valid server for the MAIL FROM domain. There are proposals to have servers sign messages to verify that the message did, indeed, originate from them. --- Noel - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Why fighting spam with whitelists doesn't work [was Re: Contributing a mailet]
Stefano, Thanks, you make a great point against reject emails. It was not my intent to create a new reject email but rather to reject it at the incoming SMTP message level. But, as Serge mentions, I might not be able to do include the URL to apply for whitelist at the SMTP reject level, and anyway the mailet API does not support such functionality. I was counting on such capabilities to do rejects without the annoying side effects you mentioned. I obviously need to do more research into how to properly reject without causing extra emails. As to a whitelisted sender being infected by a worm and sending spam, I do not see that as a big flaw, especially if you already have an anti-virus filter on your inbound mail filter chain (a normal precaution). I can't imagine a huge number of spams coming that way, and it would be easy to contact the sender and warn him of his infection. Impersonating someone in my whitelist would probably be an issue, but should be manageable if combined with additional inbound checks like validating IP against DNS. -Original Message- From: ext Stefano Mazzocchi [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 03, 2004 10:23 PM To: James Developers List Subject: Why fighting spam with whitelists doesn't work [was Re: Contributing a mailet] On 3 Feb 2004, at 17:16, [EMAIL PROTECTED] wrote: Stefano, I found your questions quite thought-provoking. Good. Would you mind answering a couple of questions? of course not. 1) I feel that no other solution other than pure whitelisting will work in the long run. A whitelist approach estimates that the send is a human being (so that is able to judge and take an action) and that the from: address was not forged. Both are pathetically wrong assumptions these days, especially after SoBig and MyDoom worm outbreaks. I have had my personal email address for many years and there are days when I receive over 1000 spams per day. Join the club. I am currently using several public blacklists and SpamAssassin set at its most aggressive setting, which worked for years until a few months ago, but now spammers are getting very smart about bypassing normal anti-spam tools. I use bayesian filtering (bogofilter because it's very fast). It's good enough for almost all sort of spam, but the random dictionary + image type. But that's easily modelled with a rule engine (but I receive so few of them lately that it's not even worth bothering writing one) Over the last two weeks, I had 4800 spam messages and only 50 false negatives (99% correctness) and no false positive so far (even if it's admittedly hard to tell, my filter is better than I am in rating spam, that's for sure) My bogofilter database contains something like 3 ham messages and 1 spam messages from my own inbox and it's 35Mb big. The database is retrained differentially every 5 minutes so that it adapts to messages I move from my inbox to the spam folder or the various ham folders [i use my 'outbox' as ham folder as well, since I'm likely to like email that looks like the one I send out] What alternative would you propose to whitelist-only email? a computational based approach for senders [see http://research.microsoft.com/research/sv/PennyBlack/] plus digital signatures for receives (so that you can check that the from address was forged or not) [see the one attached to this message] You will still need some sort of statistical analysis to remove that email that manages to come thru, but the volume would be dramatically reduced if they find a proper algorithm for the computation-based approach [which is very interesting problem from a research perspective] 2) I know that creating a new reply email directed to the from or reply-to address can be abused for relaying. no, that's not my concern. My concern is: if I'm *NOT* the one who sent that email, I don't want your stinking are you really you whitelist message because that's unsolicited email and that's exactly what we are trying to avoid in the first place! But wouldn't a reject of the incoming SMTP transaction itself (with an appropriate error message) go back ONLY to the real sender? what real sender? you have no way to tell if the from: address is really the guy who sent the email with some sort of trust facility... and trust is not something that you can take for granted or write an algorithm in a piece of software for. The point is that if somebody isn't willing to go through some necessary hassle the first (and only the first) time he sends email to me, then that person is not someone I want to hear from - EVER. I am assuming that the mailet API is called --before-- the transaction is complete. And of course, there are situations, like when joining a mailing list, where whitelisting would have to be done in advance by the recipient. But please correct me if I am wrong. It's not about being right or wrong, it's about assumptions. You
JavaMail InternetHeaders could be more friendly to server developers
Three months ago in November we exchanged a few ideas about how the JavaMail API might be made more useful for development of mail servers -- as distinct from development of mail clients. Now I have discovered one more point that I would like to add. I might be mistaken again because of my incomplete understanding of the API, but I hope this is right. My comment concerns javax.mail.internet.InternetHeaders. When a SMTP server receives a message it needs to add a Received: header at the top of the headers. But InternetHeaders does not seem to offer any way to add a header specifically at the top of the list. This class seems to offer no way for its users to learn or set the order of the headers. (I am referencing the JavaDocs for JavaMail 1.3). As a consequence the code in James* goes through an awkward workaround: constructing a new InternetHeaders object, adding the new Received: header, and then adding the remaining header lines by looping through an Enumeration of header lines retrieved from the original incoming InternetHeaders object. This workaround relies upon behavior which I have not seen documented, namely that the addHeaderLine() method always adds to the bottom of the list of headers. So, I would suggest that InternetHeaders could be improved, for the use of server developers, by addition of a method addHeaderLineAtTop(), and by promising and documenting the present behavior of addHeaderLine(), that it adds to the bottom. Rich Hammer Hillsborough, N.C. *The code in James to which I refer is in: org.apache.james.smtpserver.SMTPHandler.processMailHeaders(InternetHeaders) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Why fighting spam with whitelists doesn't work [was Re: Contributing a mailet]
On 4 Feb 2004, at 11:23, [EMAIL PROTECTED] wrote: Stefano, Thanks, you make a great point against reject emails. It was not my intent to create a new reject email but rather to reject it at the incoming SMTP message level. But, as Serge mentions, I might not be able to do include the URL to apply for whitelist at the SMTP reject level, and anyway the mailet API does not support such functionality. I was counting on such capabilities to do rejects without the annoying side effects you mentioned. I obviously need to do more research into how to properly reject without causing extra emails. As to a whitelisted sender being infected by a worm and sending spam, I do not see that as a big flaw, [sound of stefano banging his head on the wall] especially if you already have an anti-virus filter on your inbound mail filter chain (a normal precaution). A clever worm spreads much more quickly than any anti-virus update. If you think that worms are not a problem in today's internet, think again. Then again. I can't imagine a huge number of spams coming that way, don't know what huge is for you but 400 a day is enough for me. and it would be easy to contact the sender and warn him of his infection. Are you reading what I write? there is no way for me to know *who* is infected. Ok, enough. I already spent too much time on this. -- Stefano. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
cvs commit: james-server/src/xdocs/stylesheets site.xsl
noel2004/02/04 12:11:56 Modified:src/xdocs FAQ.xml announcement_2_1.xml build_instructions_2_1.xml changelog.xml contribute.xml design_objectives.xml documentation_2_1.xml download.xml index.xml install.xml installation_instructions_2_1.xml weare.xml src/xdocs/stylesheets site.xsl Log: More cleanup related to Avalon, Ant and James moving to TLP status. Revision ChangesPath 1.26 +1 -1 james-server/src/xdocs/FAQ.xml Index: FAQ.xml === RCS file: /home/cvs/james-server/src/xdocs/FAQ.xml,v retrieving revision 1.25 retrieving revision 1.26 diff -u -r1.25 -r1.26 --- FAQ.xml 6 Feb 2003 06:01:04 - 1.25 +++ FAQ.xml 4 Feb 2004 20:11:56 - 1.26 @@ -2,7 +2,7 @@ document properties titleFrequently Asked Questions/title - author email=[EMAIL PROTECTED]James Mail Server Project/author + author email=[EMAIL PROTECTED]James Project Web Team/author /properties body section name=This FAQ 1.4 +3 -3 james-server/src/xdocs/announcement_2_1.xml Index: announcement_2_1.xml === RCS file: /home/cvs/james-server/src/xdocs/announcement_2_1.xml,v retrieving revision 1.3 retrieving revision 1.4 diff -u -r1.3 -r1.4 --- announcement_2_1.xml 13 Feb 2003 21:54:50 - 1.3 +++ announcement_2_1.xml 4 Feb 2004 20:11:56 - 1.4 @@ -31,9 +31,9 @@ pInformation about James can be found at the a href=http://james.apache.org/;James web site/a at http://james.apache.org/. Users interested in subscribing to the James -a href=mailto:[EMAIL PROTECTED]user/a and -a href=mailto:[EMAIL PROTECTED]developer/a mailings lists should send emails -to [EMAIL PROTECTED] and [EMAIL PROTECTED], respectively/p +a href=mailto:[EMAIL PROTECTED]user/a and +a href=mailto:[EMAIL PROTECTED]developer/a mailings lists should send emails +to [EMAIL PROTECTED] and [EMAIL PROTECTED], respectively/p /section /body /document 1.2 +3 -4 james-server/src/xdocs/build_instructions_2_1.xml Index: build_instructions_2_1.xml === RCS file: /home/cvs/james-server/src/xdocs/build_instructions_2_1.xml,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- build_instructions_2_1.xml1 Dec 2002 09:55:00 - 1.1 +++ build_instructions_2_1.xml4 Feb 2004 20:11:56 - 1.2 @@ -14,9 +14,8 @@ section name=Getting the Source Code pThere are two ways to get the James source code./p p1. Download the source distribution - the source is available from the -a href=http://jakarta.apache.org/builds/jakarta-james/latest/;James download directory/a. -Simply choose the version of James you'd like to download, navigate into the src subdirectory -for that version, and pick the source distribution appropriate for your platform. +a href=http://james.apache.org/download.cgi;James release mirrors/a. +Simply choose the version of James you'd like to download, and pick the source distribution appropriate for your platform. /p p2. Get the source code using CVS - this method gives you access to the cutting edge code base. Instructions on how to use CVS to get the James source code (the jakarta-james distribution) @@ -30,7 +29,7 @@ installed before the build can run./p p2. Ant - This is a Java-tailored, XML-configured, extensible build or make system. The James source tree includes Ant v1.5. You can get the latest version of Ant -a href=http://jakarta.apache.org/ant;here/a. Since Ant is currently included in the source +a href=http://ant.apache.org/;here/a. Since Ant is currently included in the source distribution, it is not necessary to download it separately./p /section section name=Building the Distribution 1.27 +1 -1 james-server/src/xdocs/changelog.xml Index: changelog.xml === RCS file: /home/cvs/james-server/src/xdocs/changelog.xml,v retrieving revision 1.26 retrieving revision 1.27 diff -u -r1.26 -r1.27 --- changelog.xml 24 Jun 2003 19:41:16 - 1.26 +++ changelog.xml 4 Feb 2004 20:11:56 - 1.27 @@ -4,7 +4,7 @@ properties titleChangeLog/title - author email=[EMAIL PROTECTED]James Project/author + author email=[EMAIL PROTECTED]James Project Web Team/author /properties body 1.8 +1 -1 james-server/src/xdocs/contribute.xml Index: contribute.xml
cvs commit: james-server/www FAQ.html adding_users_2_1.html announcement_2_1.html architecture_v1_2.html architecture_v2_0.html build_instructions_2_1.html changelog.html code-standards.html configuration_v2_0.html contribute.html custom_mailet_2_1.html custom_matcher_2_1.html design_objectives.html dns_configuration_2_1.html document_archive.html documentation_2_1.html download.html fetchpop_configuration_2_1.html index.html install.html installation_instructions_2_1.html james_and_sendmail.html license.html mailet_api_2_1.html mailing_lists_2_1.html migratingDatasource_v3.0.html nntp_configuration_2_1.html pop3_configuration_2_1.html provided_mailets_2_1.html provided_matchers_2_1.html remotemanager_configuration_2_1.html repositories_2_1.html rfclist.html serverwide_configuration_2_1.html smtp_auth_2_1.html smtp_configuration_2_1.html spoolmanager_2_1.html spoolmanager_configuration_2_1.html summary_2_1.html todo.html usingJDBC_v2.0.html usingLDAP_v1_2.html usingTLS_2_1.html usingTLS_v1_2.html using_database_2_1.html weare.html
noel2004/02/04 12:16:41 Modified:www FAQ.html adding_users_2_1.html announcement_2_1.html architecture_v1_2.html architecture_v2_0.html build_instructions_2_1.html changelog.html code-standards.html configuration_v2_0.html contribute.html custom_mailet_2_1.html custom_matcher_2_1.html design_objectives.html dns_configuration_2_1.html document_archive.html documentation_2_1.html download.html fetchpop_configuration_2_1.html index.html install.html installation_instructions_2_1.html james_and_sendmail.html license.html mailet_api_2_1.html mailing_lists_2_1.html migratingDatasource_v3.0.html nntp_configuration_2_1.html pop3_configuration_2_1.html provided_mailets_2_1.html provided_matchers_2_1.html remotemanager_configuration_2_1.html repositories_2_1.html rfclist.html serverwide_configuration_2_1.html smtp_auth_2_1.html smtp_configuration_2_1.html spoolmanager_2_1.html spoolmanager_configuration_2_1.html summary_2_1.html todo.html usingJDBC_v2.0.html usingLDAP_v1_2.html usingTLS_2_1.html usingTLS_v1_2.html using_database_2_1.html weare.html Log: clean up remaining references from when Ant, Avalon and James become TLPs. Revision ChangesPath 1.28 +3 -3 james-server/www/FAQ.html Index: FAQ.html === RCS file: /home/cvs/james-server/www/FAQ.html,v retrieving revision 1.27 retrieving revision 1.28 diff -u -r1.27 -r1.28 --- FAQ.html 4 Feb 2004 17:41:40 - 1.27 +++ FAQ.html 4 Feb 2004 20:16:39 - 1.28 @@ -2,8 +2,8 @@ head META http-equiv=Content-Type content=text/html; charset=iso-8859-1 titleApache James - Frequently Asked Questions - Java Mail and News server, SMTP POP3 NNTP/title -meta value=James Mail Server Project name=author -meta value=[EMAIL PROTECTED] name=email +meta value=James Project Web Team name=author +meta value=[EMAIL PROTECTED] name=email meta content=mail,email,news,SMTP,POP3,NNTP,IMAP,MAILET,JAVA,WINDOWS,LINUX,server,Windows,2k,NT,98,ME,XP,Linux. name=keywords meta content=Free (Open Source) Java mail server (mailserver) from Apache. Providing SMTP POP3 IMAP and News NNTP runs on Microsoft Windows 2k,NT,98,ME,XP, and most flavours of Linux. name=description meta content=ALL name=ROBOTS @@ -924,7 +924,7 @@ td colspan=2 div align=center font size=-1 color=#525D76em -Copyright copy; 1999-2003, Apache Software Foundation +Copyright copy; 1999-2004, Apache Software Foundation /em/font /div /td 1.9 +1 -1 james-server/www/adding_users_2_1.html Index: adding_users_2_1.html === RCS file: /home/cvs/james-server/www/adding_users_2_1.html,v retrieving revision 1.8 retrieving revision 1.9 diff -u -r1.8 -r1.9 --- adding_users_2_1.html 4 Feb 2004 17:41:40 - 1.8 +++ adding_users_2_1.html 4 Feb 2004 20:16:39 - 1.9 @@ -211,7 +211,7 @@ td colspan=2 div align=center font size=-1 color=#525D76em -Copyright copy; 1999-2003, Apache Software Foundation +Copyright copy; 1999-2004, Apache Software Foundation /em/font /div /td 1.10 +4 -4 james-server/www/announcement_2_1.html Index: announcement_2_1.html === RCS file: /home/cvs/james-server/www/announcement_2_1.html,v retrieving revision 1.9 retrieving revision 1.10 diff -u -r1.9 -r1.10 --- announcement_2_1.html 4 Feb 2004 17:41:40 - 1.9 +++ announcement_2_1.html 4 Feb 2004 20:16:39 - 1.10 @@ -178,9 +178,9 @@ pInformation about James can be found at the a href=http://james.apache.org/;James web site/a at http://james.apache.org/. Users interested in subscribing to the James -a href=mailto:[EMAIL PROTECTED]user/a and -a href=mailto:[EMAIL PROTECTED]developer/a mailings lists should send emails -to [EMAIL PROTECTED] and [EMAIL PROTECTED], respectively/p +a href=mailto:[EMAIL PROTECTED]user/a and +a href=mailto:[EMAIL PROTECTED]developer/a mailings lists should send emails +to [EMAIL PROTECTED] and [EMAIL PROTECTED], respectively/p /blockquote /td @@ -199,7
cvs commit: james-server/www FAQ.html adding_users_2_1.html announcement_2_1.html architecture_v1_2.html architecture_v2_0.html build_instructions_2_1.html changelog.html code-standards.html configuration_v2_0.html contribute.html custom_mailet_2_1.html custom_matcher_2_1.html design_objectives.html dns_configuration_2_1.html document_archive.html documentation_2_1.html download.html fetchpop_configuration_2_1.html index.html install.html installation_instructions_2_1.html james_and_sendmail.html license.html mailet_api_2_1.html mailing_lists_2_1.html migratingDatasource_v3.0.html nntp_configuration_2_1.html pop3_configuration_2_1.html provided_mailets_2_1.html provided_matchers_2_1.html remotemanager_configuration_2_1.html repositories_2_1.html rfclist.html serverwide_configuration_2_1.html smtp_auth_2_1.html smtp_configuration_2_1.html spoolmanager_2_1.html spoolmanager_configuration_2_1.html summary_2_1.html todo.html usingJDBC_v2.0.html usingLDAP_v1_2.html usingTLS_2_1.html usingTLS_v1_2.html using_database_2_1.html weare.html
noel2004/02/04 19:18:05 Modified:www FAQ.html adding_users_2_1.html announcement_2_1.html architecture_v1_2.html architecture_v2_0.html build_instructions_2_1.html changelog.html code-standards.html configuration_v2_0.html contribute.html custom_mailet_2_1.html custom_matcher_2_1.html design_objectives.html dns_configuration_2_1.html document_archive.html documentation_2_1.html download.html fetchpop_configuration_2_1.html index.html install.html installation_instructions_2_1.html james_and_sendmail.html license.html mailet_api_2_1.html mailing_lists_2_1.html migratingDatasource_v3.0.html nntp_configuration_2_1.html pop3_configuration_2_1.html provided_mailets_2_1.html provided_matchers_2_1.html remotemanager_configuration_2_1.html repositories_2_1.html rfclist.html serverwide_configuration_2_1.html smtp_auth_2_1.html smtp_configuration_2_1.html spoolmanager_2_1.html spoolmanager_configuration_2_1.html summary_2_1.html todo.html usingJDBC_v2.0.html usingLDAP_v1_2.html usingTLS_2_1.html usingTLS_v1_2.html using_database_2_1.html weare.html Log: updating to reference Jira for bug reporting Revision ChangesPath 1.29 +1 -1 james-server/www/FAQ.html Index: FAQ.html === RCS file: /home/cvs/james-server/www/FAQ.html,v retrieving revision 1.28 retrieving revision 1.29 diff -u -r1.28 -r1.29 --- FAQ.html 4 Feb 2004 20:16:39 - 1.28 +++ FAQ.html 5 Feb 2004 03:18:02 - 1.29 @@ -79,7 +79,7 @@ /p ul li -a href=http://jakarta.apache.org/site/bugs.html;Bug Database /a +a href=http://issues.apache.org/jira/;Bug Database /a /li li a href=http://jakarta.apache.org/site/cvsindex.html;CVS Repositories/a 1.10 +1 -1 james-server/www/adding_users_2_1.html Index: adding_users_2_1.html === RCS file: /home/cvs/james-server/www/adding_users_2_1.html,v retrieving revision 1.9 retrieving revision 1.10 diff -u -r1.9 -r1.10 --- adding_users_2_1.html 4 Feb 2004 20:16:39 - 1.9 +++ adding_users_2_1.html 5 Feb 2004 03:18:03 - 1.10 @@ -77,7 +77,7 @@ /p ul li -a href=http://jakarta.apache.org/site/bugs.html;Bug Database /a +a href=http://issues.apache.org/jira/;Bug Database /a /li li a href=http://jakarta.apache.org/site/cvsindex.html;CVS Repositories/a 1.11 +1 -1 james-server/www/announcement_2_1.html Index: announcement_2_1.html === RCS file: /home/cvs/james-server/www/announcement_2_1.html,v retrieving revision 1.10 retrieving revision 1.11 diff -u -r1.10 -r1.11 --- announcement_2_1.html 4 Feb 2004 20:16:39 - 1.10 +++ announcement_2_1.html 5 Feb 2004 03:18:03 - 1.11 @@ -77,7 +77,7 @@ /p ul li -a href=http://jakarta.apache.org/site/bugs.html;Bug Database /a +a href=http://issues.apache.org/jira/;Bug Database /a /li li a href=http://jakarta.apache.org/site/cvsindex.html;CVS Repositories/a 1.7 +1 -1 james-server/www/architecture_v1_2.html Index: architecture_v1_2.html === RCS file: /home/cvs/james-server/www/architecture_v1_2.html,v retrieving revision 1.6 retrieving revision 1.7 diff -u -r1.6 -r1.7 --- architecture_v1_2.html4 Feb 2004 20:16:39 - 1.6 +++ architecture_v1_2.html5 Feb 2004 03:18:03 - 1.7 @@ -77,7 +77,7 @@ /p ul li -a href=http://jakarta.apache.org/site/bugs.html;Bug Database /a +a href=http://issues.apache.org/jira/;Bug Database /a /li li a href=http://jakarta.apache.org/site/cvsindex.html;CVS Repositories/a 1.20 +1 -1 james-server/www/architecture_v2_0.html Index: architecture_v2_0.html === RCS file: /home/cvs/james-server/www/architecture_v2_0.html,v retrieving revision 1.19 retrieving revision 1.20 diff -u -r1.19 -r1.20 --- architecture_v2_0.html4 Feb 2004 20:16:39 - 1.19 +++ architecture_v2_0.html5 Feb 2004 03:18:03 - 1.20 @@
cvs commit: james-server/src/xdocs/stylesheets project.xml
noel2004/02/04 19:19:58 Modified:src/xdocs/stylesheets project.xml Log: switch to Jira from bugzilla Revision ChangesPath 1.29 +1 -1 james-server/src/xdocs/stylesheets/project.xml Index: project.xml === RCS file: /home/cvs/james-server/src/xdocs/stylesheets/project.xml,v retrieving revision 1.28 retrieving revision 1.29 diff -u -r1.28 -r1.29 --- project.xml 3 Feb 2004 20:17:26 - 1.28 +++ project.xml 5 Feb 2004 03:19:58 - 1.29 @@ -27,7 +27,7 @@ /menu menu name=Project -item name=Bug Database href=http://jakarta.apache.org/site/bugs.html/ +item name=Bug Database href=http://issues.apache.org/jira// item name=CVS Repositories href=http://jakarta.apache.org/site/cvsindex.html/ item name=Who We Arehref=/weare.html/ item name=How Can I Contribute? href=/contribute.html/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
