Maildir + User Directory Repository (read: LDAP+) + Virtual Domains + SMTPACL patch for james server v3.0a1 (cvs20040120)
Author: Alexander Zhukov <proboez@ukrpost.net>

For installation read INSTALL file
This patch adds the following features to James:

    * Maildir support 
        Maildir is a format used in qmail, courier and several other known MTAs 
        Main advantage of Maildir is there's no need for locking. 
        Which is very important if you have a large mail storage and/or it is 
        shared over NFS with other nodes.
        It is also safe for multiple simultaneous deliveries to the same folder.
        This is achieved by Maildir by storing incoming mail in Maildir/tmp/ 
        directory before message is completely delivered.
        Then message is rename()d to Maildir/new/ directory. 
        So to get unread message count you just have to get the number of files 
        stored in Maildir/new/ directory
        Courier IMAPs Maildir++ specification supports sub-folders making it  
        possible to create IMAPMailbox provider for james to properly support 
        imap. 
        Maildir++ is compatible with Maildir. If some delivery program doesnt 
        know about Maildir++ it will still do its job.
        On the other side disadvantage of Maildir is that stores each message in
        its own file which wastes your inodes.
        
        My implementation of Maildir support is pretty basic but it handles 
        local delivery and message removal/retrieval.
        What I mean you can deliver to Maildir mailboxes with SMTP and 
        read/delete your mail with POP3. (read: replace your 
        qmail+your_pop3_server_here with james!)
        This implementation is a wrapper around Javamaildir provider 
        (http://javamaildir.sf.net/) for javamail I wrote. 
        It is pretty mature now and is used on my production servers.
        I support only MAIL storage type, not SPOOL and i dont think Maildir is 
        well suited for spool storage.

        To configure Maildir support 
        1. add the following to your james-config.xml:
        config->mailstore->repositories
        ---- cut here ----
            <repository class="org.apache.james.mailrepository.MaildirMailRepository">
                <protocols>
                    <protocol>maildir</protocol>
                </protocols>
                <types>
                    <type>MAIL</type>
                </types>
            </repository>
        ---- cut here ----
        This will add support for maildir protocol.

        2. Replace your inboxRepository URL to something that starts with 
        "maildir:/"
        config->James
        ---- cut here ----
            <inboxRepository>
                <repository destinationURL="maildir:/home/" type="MAIL"/>
            </inboxRepository>
        ---- cut here ----
        use "maildir:///home" for relative paths
        use "maildir:/home" or "maildir:////home" for abolsute path because url 
        is parsed with javax.mail.URLName class

        3. Enjoy. Send bugs/patches/complaints to me proboez@ukrpost.net

    * User Directory Repository (read: LDAP+)
        Current UserLDAPRepository has several drawbacks 
        first it is LDAP-only but based on JNDI, it is tied to Sun Microsystems
        LDAP provider, you can not customize lookups to your needs and your 
        LDAP directory design decisions this is just to name a few.
        I decided to reimplement it. This is what I did:
        My UserDirectoryRepository while intended for LDAP is neither tied to 
        LDAP itself nor its JNDI provider, you must specify InitialDirContext 
        provider for you directory. Second lookups are customizable so instead 
        of doing hardcoded LDAP lookups (filters in terms of JNDI) you can 
        customize it, for example for my LDAP directory current hardcoded ldap 
        filter '(mail=user@domain.com)' might give two or more results (ok ok 
        i know you change "mail" to anything else but still it is not enough 
        for me). With my provider you can specify your lookup in 
        configuration file so that it will look like 
        '(&(objectclass=inetuser)(uid=${userlocalpart}))' 
        or 
        '(&(myattribute=${userlocalpart})(anotherattr=${userhostpart}))' 
        or if you just need the good old lookup write: 
        '(mail=${useraddress})'
        this lookup template is transformed in runtime to a normal filter and
        looked up in directory. 
        For now I implemented only these three (${userlocalpart} 
        ${userhostpart} ${useraddress}) variables but nothing stops you to 
        implement any other like ${remote_host}.

        Plus I invented DirectoryUser interface that extends JamesUser which 
        gives access to extended attributes in your directory. Reasons for this 
        interface read later (Virtual domains part). And of course I provide 
        basic implementation of this interface.

        Configuration is very similar (well it is copy-and-paste plus minor 
        changes) to UserLDAPRepository:

        1. add the following to your james-config.xml:
        config->users-store:
        ---- cut here ----
            <repository name="LocalUsers" class="org.apache.james.userrepository.UsersDirectoryRepository">
                <destination URL="UNUSED"/>
                <LDAPServer>UNUSED</LDAPServer>
                <LDAPRoot>UNUSED</LDAPRoot>
                <ThisServerRDN>UNUSED</ThisServerRDN>
                <MailAddressAttribute>UNUSED</MailAddressAttribute>
                <IdentityAttribute>UNUSED</IdentityAttribute>

                <AuthenticationType>simple</AuthenticationType>
                <Principal>USERNAME</Principal>
                <Password>PASSWORD</Password>
                <MembersAttribute>UNUSED</MembersAttribute>
                <ManageGroupAttribute>false</ManageGroupAttribute>
                <GroupAttribute>UNUSED</GroupAttribute>
                <ManagePasswordAttribute>true</ManagePasswordAttribute>
                <PasswordAttribute>userPassword</PasswordAttribute>
                <initialDirContextFactory>PROVIDERNAME</initialDirContextFactory>
                <initialDirContextUrl>INITIALDIRCONTEXT</initialDirContextUrl>
                <lookupTemplate>YOUR_TEMPLATE</lookupTemplate>
            </repository>
        ---- cut here ----
        Most of the fields are set to UNUSED this is to keep compatible (ok just
        because I copy-pasted the UserLDAPRepository class :) ) with 
        UserLDAPRepository.
        Fields you have to set are:
        USERNAME (in LDAP it is called binddn)
        PASSWORD - guess what it is 
        PROVIDERNAME - "com.sun.jndi.ldap.LdapCtxFactory" for Suns LDAP provider
        INITIALDIRCONTEXT - this is your InitialDirContext for me it is:
            ldap://ldapserver:389/o=ldapsuffix
        YOUR_TEMPLATE - this is a lookup (filter) template explained before 
            for me it is: (&amp;(objectclass=upuser)(mail=${useraddress}))
            '&amp;' is xml way of saying '&'

        2. Done

    * Virtual domains
        I have been looking for an opportunity to migrate to James for a 
        long time, but one critical feature I really needed was support for 
        virtual domains.
        Let me explain what I mean by saying virtual domains. My server handles 
        mail (is MX for) several domains lets call them domain1.com, domain2.org
        domain3.net. I have user called johndoe in all of these domains but they
        are not the same person. So johndoe@domain1.com should have no access to
        johndoe@domain3.net's mail.
        Before this patch James would deliver johndoe@domain1.com's _AND_ 
        johndoe@domain3.net's mail to folder '/var/mail/johndoe' which is 
        totally unacceptable for me. Lets look inside 
        src/java/org/apache/james/James.java on line 765 

        username = recipient.getUser();
        ...
        user = (JamesUser)localusers.getUserByName(username);

        this code strips off the domain part then JamesUser object is looked up 
        in usersstore based solely on localpart.
        What I did I lookup user based on his full address

        username = supportVirtual ? recipient.toString() : recipient.getUser();
        ...
        user = (JamesUser)localusers.getUserByName(username);

        Another import change to note I added getUserInbox(User user) method and
        wrapped already existing getUserInbox(String username) to call 
        getUserInbox(localusers.getUserByName(userName))
        This change gives ability to flexibly configure users inboxUrl.
        At the moment users name is simply appended to root url of current 
        storage. This is totally ok if you store mail in database but it is 
        not if you store it in filesystem and especially incovenient if you 
        have 100K+ users in this case (i have ~140K users).
        Imagine a directory with 100K subdirectories with users mailboxes, most 
        filesystems will be _very_ slow searching users mailbox.
        Better approach in this situation would be (look how sourceforge does 
        this) to make a tree of these 100K subdirectories, for example:

        now:
        /var/mail/user1@domain1.com/ or even /var/mail/user1
        /var/mail/johndoe@domain2.net/
        ...
        /var/mail/100Kth_user@domain3.org/
        
        more scalable approach:
        /var/mail/domain1.com/us/e/user1/
        /var/mail/domain2.net/jo/h/johndoe/
        ...
        /var/mail/domain3.org/10/0/100Kth_user/
        
        Problem here is where do we find "domain3.org/10/0/100Kth_user"?
        Answer is simple - in directory in "homeDirectory" attribute (for ldap) 
        or configure for your type of directory. This config is still TODO but 
        very simple to implement, so patches are very welcome. :)
        Here comes into play DirectoryUser if User passed as an argument to 
        getUserInbox method implements DirectoryUser interface then we can ask 
        for such information otherwise do as usual(append username to root url).
        I think this change makes James more scalable and suitable for large 
        ISPs.
        Note that POP3 supports this feature automatically because it calls 
        getUserInbox(String) method of James.

    * SMTPACL
        Name is taken from Exim (www.exim.org) mailers smtpacl feature.
        Currently all mail received by james goes to spool then it is processed
        and some mail-routing decisions are made. Before mail is received during
        SMTP dialog only some very basics things (like recipient syntactical 
        validity, smtp auth flag, relaying etc.) are checked. So if message is 
        received from some spammer it is first stored in spool (read: takes tons
        of disk space) then it is processed by Matcher that gets its source IP 
        address and only then it is rejected. Exim allows to reject such mails 
        during SMTP dialog and not only based on IP but any information you can 
        get from SMTP dialog. Another example would be rejecting mails to some 
        local recipient (maybe even a domain if you are an ISP and want to 
        temporarily disable mail delivery for customer that doesnt pay his 
        bills) during SMTP dialog so that mail doesnt even reach spool.
        This feature saves space and network bandwidth if check is made before 
        message is received.
        James has a very thought-out Matcher-mailet api so i decided not to 
        reimplement the wheel and make all available Matchers work for SMTP ACL.
        For now I made a smtpacl to work only after "RCPT TO" command.

        Configuration is very similar to mailet configration of spool manager.
        For example:
        1. config->smtpserver->handler->smtpacl
        ---- cut here ----
            <mailetpackages>
                <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
            </mailetpackages>
            <matcherpackages>
                <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
            </matcherpackages>

            <rcpt>
            <!-- allow relaying for local hosts -->
                <mailet match="HostIsLocal" class="SMTPACLAccept"/>
                <mailet match="RemoteAddrInNetwork=192.168.55.0" class="SMTPACLAccept"/>

                <!-- if recipient domain is local but he/she cannot be looked up in user database then reject -->
                <mailet match="VirtualRecipientIsNotLocal" class="SMTPACLReject">
                    <notice>550 unknown user</notice>
                </mailet>
                <mailet match="SenderInFakeDomain" class="SMTPACLReject">
                    <notice>550 cannot find MX for your domain</notice>
                </mailet>
                <mailet match="RecipientIs=badboy@ukrpost.net" class="SMTPACLReject">
                    <notice>550 wont deliver to badboy. admin is angry.</notice>
                </mailet>
                <!-- todo: if authenticated accept -->

                <mailet match="All" class="SMTPACLAccept"/>
            </rcpt>
        ---- cut here ----
        Mailets can only extend SMTPACLResult abstract class and usually are 
        either SMTPACLReject or SMTPACLAccept. 
        Matchers are processed linearly by SMTPACLProcessor (a simplified 
        LinearProcessor) and if match is found Mailet of specified type is 
        returned to SMTPHandler. SMTPHandler writes notice to user and modifies 
        SMTP state accordingly.
        Some changes to note here are: I introduced SMTPState class which is a 
        wrapper around HashMap state. SMTPState is used to construct a stub Mail
        object of type SMTPACLMail. Then Matchers match() method receives 
        SMTPACLMail. Now Matcher implementation thinks it processes normal Mail 
        from spool but instead it operates on SMTP state, which gives us ability
        to use all already written Matchers for SMTPACLs.
