Got a solution for this; thanks for all your help. The problem was qmail was
unable to deliver mail if the Maildir is stored on AFS.
Andrea mentioned that AFS is very similar to CODA, and the CODA solution is
to use rename() as I was planning on doing. Peter had cautioned that rename
will overwrite existing files whereas hard links will not lose any existing
files. Archived discussion threads indicate that renaming is still
relatively safe because the email file name is composed of the current
date/time, PID and host name and it is unlikely that the PID would recycle
within 1 second.
The extended explanation of the problem is that AFS implements its own ACL,
so that even the Unix root user may have no access to a users ~home
directory; the holds true also that a user who has managed to log in to the
Linux/Unix box may have no access to his own ~home directory. Access to AFS
files are granted by tokens issued by the AFS/Kerberos authentication
server. So during mail delivery, qmail-lspawn will setuid to become the
email receipient, except that in most cases, this user will not have a valid
token. So before we can even come to the problem of link() versus rename(),
qmail is stymied by a lack to access to the Maildir. My solution of choice
is to make qmail-local.c setuid to a mail delivery user (I picked qmaill)
and make the qmail-local binary be setuid and owned by qmaill. The
Maildir/tmp directory will grant allow lookup, insert, and delete privileges
to qmaill, and Maildir/new will grant lookup, and insert to qmaill. The lack
of a delete privilege on Maildir/new means that if a rename() is going to
overwrite an existing file, AFS fails this operation because it implies a
delete of the existing file, which privilege has not been granted to qmaill.
(This scenario was tested by me by making qmail-local.c always generate the
same email file name; on the first delivery it succeeds; on subsequent
deliveries qmail defers the delivery as long as a file of the same name
exists in Maildir/new.)
In conjunction with all of the above, I will need a cron job to periodically
refresh qmaill's AFS/Kerberos token so that qmaill will always have a valid
token in order to make use of the ACL privileges that have been given to
qmaill.
The unified diff of my patch to qmail-local.c appears below for whatever you
want to do with it.
--
...Ru (a low-cost superhero)
On, on! Blue skies. Think snow.
1740484I 1232222 998300172 076662 82968/A17215 045124P E286/184435
975-203608 11859 DS1160
--- qmail-local.c.orig Wed Aug 8 14:34:18 2001
+++ qmail-local.c Thu Aug 9 01:05:42 2001
@@ -1,5 +1,6 @@
#include <sys/types.h>
#include <sys/stat.h>
+#include <errno.h>
#include "readwrite.h"
#include "sig.h"
#include "env.h"
@@ -44,6 +45,77 @@
int flagdoit;
int flag99;
+/* Wed Aug 8 16:02:32 2001, Rudy Zung on vice1.bluezulu.com
+ * The AFS patch short form:
+ * 1) Incorporate the AFS patch into qmail-local.c
+ * 2) Compile per INSTALL
+ * 3) Issue: chmod u+s /var/bin/qmail/qmail-local
+ * 4) Issue: chown qmaill /var/bin/qmail/qmail-local
+ * 5) Issue: fs setacl ~home qmaill l
+ * 6) Issue: fs setacl ~home/Maildir/tmp qmaill lidk
+ * 7) Issue: fs setacl ~/home/Maildir/new qmaill lik
+ * IMPORTANT: ~home/Maildir/new MUST NEVER HAVE d privilege for qmaill,
+ * system:authuser, or system:anyuser. The qmail-local process must
+ * not have the ability to delete files.
+ * 8) Create qmaill as an AFS user; use the AFS kas and pts utilities.
+ * Make sure that the AFS UID for qmaill matches the /etc/passwd UID
+ * for qmaill
+ * 9) su to qmaill and run the AFS utility klog.
+ * (Consider cron job to refresh this token prior to its expiration)
+ * 10) Follow INSTALL for starting up qmail processes.
+ * The commands needed for the fs setacl can be embedded into the
+ * AFS uss template file for subsequent users who will be created via the
+ * uss utility.
+ *
+ * Long form: The AFS patch is designed to solve a couple of problems where
+ * the Maildir is in AFS space: AFS does not support hard links thus
+ * instead of the normal qmail operation of using link(2) to move an email
+ * message from Maildir/tmp to Maildir/new, we have to use rename(2). Under
+ * AFS, a processes Unix UID has no bearing on file access on AFS volumes.
+ * Access to AFS space is managed via AFS/Kerberos tokens, which are valid
+ * for a limited lifetime usually of less than 24 hours. An AFS/Kerberos
+ * token is granted by a separate authentication to the authentication
+ * server, and may not necessarily be integrated with the Unix login(1)
+ * command. Thus it is a very posssible scenario that a user may have his
+ * home directory in AFS, and has managed to log in to the Unix box, but
+ * have no access to his ~home directory because he hasn't requested
+ * a AFS/Kerberos token. In the normal case, qmail-lspawn will setuid to
+ * the email receipient which usually will have full access to ~home and
+ * Maildir directories; however in the AFS case, as has been pointed out, a
+ * valid token for access to the user's home volume may not be available.
+ * The solution to this is to allow qmail-local setuid to become a mail
+ * delivery UID, such as qmaill, and set up AFS ACLs on the Maildir to give
+ * special privileges to qmaill. In this scenario, qmail-local will start
+ * up with the real UID of the Maildir owner, and an effective UID of
+ * the qmail-local setuid owner.
+ *
+ * At the same time, a cron job belonging to qmaill can be written to read
+ * a mode 600 non-AFS file that contains qmaill's Kerberos password to
+ * renew the token prior to its expiration.
+ *
+ * The implementation of this solution is to make qmail-local have the
+ * owner setuid bit flag set, and be owned by qmaill and NOT OWNED by root,
+ * which is the default case. For AFS Maildir deliveries, we will setuid to
+ * our effective UID (should be qmaill), which will have lidk permissions
+ * on Maildir/tmp, and lik on Maildir/new. These specific ACLs allow
+ * qmail-local running as qmaill to stat(2) files in Maildir/tmp to ensure
+ * no-duplicates of file names. During the rename(2) operation to move the
+ * email from Maildir/tmp to Maildir/new, the li privileges on Maildir/new
+ * will allow the file to be created there, but if it comes at the expense
+ * overwriting an existing file, the lack of the d privilege prevents
+ * rename(2) from deleting the existing file and the rename(2) operation
+ * would fail in a similar behaviour as link(2) would. The d privilege on
+ * Maildir/tmp will allow the file there to be unlinked once the email
+ * message has been moved/renamed into Maildir/new. The l and k privileges
+ * are useful for stat(2) purposes and for possible directory locking needs
+ * during mail delivery operations.
+ *
+ * Now, for other purposes that don't involve calling maildir_child(),
+ * we need to setreuid(2) to the real UID that qmail-lspawn gave us;
+ * this is necessary so that if the .qmail file uses |qbiff, qbiff
+ * will know who to contact about new mail. */
+int hassetuid; /* binary has set UID bit set */
+
char *user;
char *homedir;
char *local;
@@ -74,6 +146,23 @@
void tryunlinktmp() { unlink(fntmptph); }
void sigalrm() { tryunlinktmp(); _exit(3); }
+int qmsetreuid(uid_t uid)
+{
+ if (setreuid(uid,uid) == -1)
+ {
+ strerr_die3x(111,"Unable to setreuid: ",error_str(errno),". (4.x.x)");
+ return(-1);
+ }
+
+ if (!getuid() || !geteuid())
+ {
+ strerr_die1x(111,"Not allowed to setreuid to root. (#4.x.x)");
+ return(-1);
+ }
+
+ return(0);
+}
+
void maildir_child(dir)
char *dir;
{
@@ -86,8 +175,46 @@
int fd;
substdio ss;
substdio ssout;
+ uid_t uid;
sig_alarmcatch(sigalrm);
+
+ /* Wed Aug 8 22:49:44 2001, Rudy Zung on vice1.bluezulu.com
+ * needed for AFS patch. For AFS delivery, we have to set our real
+ * UID to be the same as our effective UID (which is the UID of the
+ * owner of this binary) so that the AFS/Kerberos token that is
+ * associated with the mail delivery agent UID (which owns this binary)
+ * can be matched correctly for access to the Maildir which has its ACL
+ * set appropriately for the mail delivery agent (tmp=lidk new=li).
+ * For non-AFS delivery, we need to set our effective UID to be the
+ * same as our real UID.
+ */
+ if (hassetuid)
+ {
+ /* Thu Aug 9 00:34:59 2001, Rudy Zung on vice1.bluezulu.com
+ * By historical decree, AFS volume mount points must be rooted at
+ * /afs/<cellName>/<volumeMountPoint>/...
+ * AFS deliveries setuid to the owner of this binary; non-AFS
+ * deliveries setuid to the email receipient.
+ * This may turn out to be a security hole where a user may piggy
+ * back on the AFS/Kerberos tokens afforded by the effective UID
+ * (this is the UID of the owner of this binary file) and directing
+ * the email to be written to someone else's Maildir. In the Unix
+ * non-AFS case, this would not usually be possibly because the user
+ * should/would not normally have write access to someone else's
+ * Maildir directory (but this same effect can be accomplished by
+ * making .qmail forward the mail to someone else)
+ */
+ s = getcwd((char *) 0, 0);
+ if (!strncmp(s,"/afs/",5) || !strncmp(dir,"/afs/",5)) uid = geteuid();
+ else uid = getuid();
+
+ free(s);
+ s = (char *) 0;
+
+ if (qmsetreuid(uid) == -1) _exit(111);
+ }
+
if (chdir(dir) == -1) { if (error_temp(errno)) _exit(1); _exit(2); }
pid = getpid();
host[0] = 0;
@@ -127,8 +254,22 @@
if (fsync(fd) == -1) goto fail;
if (close(fd) == -1) goto fail; /* NFS dorks */
- if (link(fntmptph,fnnewtph) == -1) goto fail;
- /* if it was error_exist, almost certainly successful; i hate NFS */
+ /* Wed Aug 8 16:23:56 2001, Rudy Zung on vice1.bluezulu.com
+ * needed for AFS patch. if hard link fails because AFS does not support
+ * cross directory hard links, try to do a rename(2) instead if we have
+ * also had to do a setuid
+ */
+ if ((link(fntmptph,fnnewtph) == -1) && hassetuid)
+ {
+ /* Wed Aug 8 16:37:35 2001, Rudy Zung on vice1.bluezulu.com
+ * needed for AFS patch. AFS does not support cross directory hard links
+ * and will raise EXDEV
+ */
+ if (errno != EXDEV) goto fail;
+ else if (rename(fntmptph,fnnewtph) == -1) goto fail;
+ }
+
+ /* if it was error_exist, almost certainly successful; i hate NFS */
tryunlinktmp(); _exit(0);
fail: tryunlinktmp(); _exit(1);
@@ -241,6 +382,15 @@
case -1:
temp_fork();
case 0:
+ /* Wed Aug 8 22:56:01 2001, Rudy Zung on vice1.bluezulu.com
+ * needed for AFS patch
+ * we have to make our effective UID be the same as our real UID.
+ * our real UID is set by qmail-lspawn just prior to spawning us.
+ * certain programs like qbiff rely on the effective UID to determine
+ * who to notify about incoming mail
+ */
+ if (hassetuid && (qmsetreuid(getuid()) == -1)) _exit(111);
+
args[0] = "/bin/sh"; args[1] = "-c"; args[2] = prog; args[3] = 0;
sig_pipedefault();
execv(*args,args);
@@ -458,12 +608,27 @@
datetime_sec starttime;
int flagforwardonly;
char *x;
+ struct stat st;
+ hassetuid = 0;
umask(077);
sig_pipeignore();
if (!env_init()) temp_nomem();
+ /* Wed Aug 8 16:03:51 2001, Rudy Zung on vice1.bluezulu.com
+ * needed for AFS patch to allow maildir_child() to setuid to the owner of
+ * this binary. because main() will perform a chdir before call
+ * maildir_child(), we may have a hard time trying to get the path to
+ * this binary, so we stat ourselves now
+ */
+ if (stat(argv[0], &st) == -1)
+ {
+ strerr_die5x(111,"Unable to stat ",argv[0],": ",error_str(errno),". (#4.x.x)");
+ _exit(111);
+ }
+ else hassetuid = ((st.st_mode & S_ISUID) && (st.st_mode & S_IXUSR));
+
flagdoit = 1;
while ((opt = getopt(argc,argv,"nN")) != opteof)
switch(opt)
@@ -696,3 +861,4 @@
count_print();
_exit(0);
}
+
