Re: qmail on AFS
Got a solution for this; thanks for all your help. The problem was qmail was unable to deliver mail if the Maildir is stored on AFS. Andrea mentioned that AFS is very similar to CODA, and the CODA solution is to use rename() as I was planning on doing. Peter had cautioned that rename will overwrite existing files whereas hard links will not lose any existing files. Archived discussion threads indicate that renaming is still relatively safe because the email file name is composed of the current date/time, PID and host name and it is unlikely that the PID would recycle within 1 second. The extended explanation of the problem is that AFS implements its own ACL, so that even the Unix root user may have no access to a users ~home directory; the holds true also that a user who has managed to log in to the Linux/Unix box may have no access to his own ~home directory. Access to AFS files are granted by tokens issued by the AFS/Kerberos authentication server. So during mail delivery, qmail-lspawn will setuid to become the email receipient, except that in most cases, this user will not have a valid token. So before we can even come to the problem of link() versus rename(), qmail is stymied by a lack to access to the Maildir. My solution of choice is to make qmail-local.c setuid to a mail delivery user (I picked qmaill) and make the qmail-local binary be setuid and owned by qmaill. The Maildir/tmp directory will grant allow lookup, insert, and delete privileges to qmaill, and Maildir/new will grant lookup, and insert to qmaill. The lack of a delete privilege on Maildir/new means that if a rename() is going to overwrite an existing file, AFS fails this operation because it implies a delete of the existing file, which privilege has not been granted to qmaill. (This scenario was tested by me by making qmail-local.c always generate the same email file name; on the first delivery it succeeds; on subsequent deliveries qmail defers the delivery as long as a file of the same name exists in Maildir/new.) In conjunction with all of the above, I will need a cron job to periodically refresh qmaill's AFS/Kerberos token so that qmaill will always have a valid token in order to make use of the ACL privileges that have been given to qmaill. The unified diff of my patch to qmail-local.c appears below for whatever you want to do with it. -- ...Ru (a low-cost superhero) On, on! Blue skies. Think snow. 1740484I 123 998300172 076662 82968/A17215 045124P E286/184435 975-203608 11859 DS1160 --- qmail-local.c.orig Wed Aug 8 14:34:18 2001 +++ qmail-local.c Thu Aug 9 01:05:42 2001 @@ -1,5 +1,6 @@ #include sys/types.h #include sys/stat.h +#include errno.h #include readwrite.h #include sig.h #include env.h @@ -44,6 +45,77 @@ int flagdoit; int flag99; +/* Wed Aug 8 16:02:32 2001, Rudy Zung on vice1.bluezulu.com + * The AFS patch short form: + * 1) Incorporate the AFS patch into qmail-local.c + * 2) Compile per INSTALL + * 3) Issue: chmod u+s /var/bin/qmail/qmail-local + * 4) Issue: chown qmaill /var/bin/qmail/qmail-local + * 5) Issue: fs setacl ~home qmaill l + * 6) Issue: fs setacl ~home/Maildir/tmp qmaill lidk + * 7) Issue: fs setacl ~/home/Maildir/new qmaill lik + * IMPORTANT: ~home/Maildir/new MUST NEVER HAVE d privilege for qmaill, + * system:authuser, or system:anyuser. The qmail-local process must + * not have the ability to delete files. + * 8) Create qmaill as an AFS user; use the AFS kas and pts utilities. + * Make sure that the AFS UID for qmaill matches the /etc/passwd UID + * for qmaill + * 9) su to qmaill and run the AFS utility klog. + * (Consider cron job to refresh this token prior to its expiration) + * 10) Follow INSTALL for starting up qmail processes. + * The commands needed for the fs setacl can be embedded into the + * AFS uss template file for subsequent users who will be created via the + * uss utility. + * + * Long form: The AFS patch is designed to solve a couple of problems where + * the Maildir is in AFS space: AFS does not support hard links thus + * instead of the normal qmail operation of using link(2) to move an email + * message from Maildir/tmp to Maildir/new, we have to use rename(2). Under + * AFS, a processes Unix UID has no bearing on file access on AFS volumes. + * Access to AFS space is managed via AFS/Kerberos tokens, which are valid + * for a limited lifetime usually of less than 24 hours. An AFS/Kerberos + * token is granted by a separate authentication to the authentication + * server, and may not necessarily be integrated with the Unix login(1) + * command. Thus it is a very posssible scenario that a user may have his + * home directory in AFS, and has managed to log in to the Unix box, but + * have no access to his ~home directory because he hasn't requested + * a AFS/Kerberos token. In the normal case, qmail-lspawn will setuid to + * the email receipient which usually will have full access to ~home and + * Maildir
Re: qmail on AFS
-BEGIN PGP SIGNED MESSAGE- Andrea mentioned that AFS is very similar to CODA, and the CODA solution is to use rename() as I was planning on doing. Peter had cautioned that rename will overwrite existing files whereas hard links will not lose any existing files. Archived discussion threads indicate that renaming is still relatively safe because the email file name is composed of the current date/time, PID and host name and it is unlikely that the PID would recycle within 1 second. I think rename() is safe. Even link() is of course better, the event that two nodes of the cluster are writing a mail with same data,pid,hostname (???) is obviously very very rare (impossible?). The extended explanation of the problem is that AFS implements its own ACL, so that even the Unix root user may have no access to a users ~home directory; the holds true also that a user who has managed to log in to the Linux/Unix box may have no access to his own ~home directory. Like Coda. Access to AFS files are granted by tokens issued by the AFS/Kerberos authentication server. Like Coda, but Coda doesn't use kerberos. So during mail delivery, qmail-lspawn will setuid to become the email receipient, except that in most cases, this user will not have a valid token. I don't know well AFS, but with Coda / Vpopmail is trivial. It's sufficient to get a token for vpopmail and root users. When qmail starts, it launch 2 scripts to get root and vpopmail tokens. Then, crontab perform authentication for root / vpopmail every 4 hours (security reasons, tokens are valid for much longer). I really didn't need to patch qmail-local. In conjunction with all of the above, I will need a cron job to periodically refresh qmaill's AFS/Kerberos token so that qmaill will always have a valid token in order to make use of the ACL privileges that have been given to qmaill. Yep. But I don't see the needs for patching qmail-local.c (except for rename()). - --- Cordiali saluti / Best regards Andrea Cerrito ^^ Net.Admin @ Centro MultiMediale di Terni S.p.A. P.zzale Bosco 3A 05100 Terni IT Tel. +39 0744 5441330 Fax. +39 0744 5441372 -BEGIN PGP SIGNATURE- Version: PGPfreeware 6.5.3 for non-commercial use http://www.pgp.com iQEVAwUBO3K16vo9HK4+yTI3AQF0IQgAmgnnlQABR9szQN2KQHrxOLaSo92xxs8u 4vIkfmYT7eZrEIlYoRBazBa+8TfDyUqORNxatydzIBqiHBQcEf7AxBT8BhegNu9n QN8UNhCOEJNRHf/DvluInZm2I6+MRxity2o6psKfkWliFFaP6Lu6G8bw41J0qKul sVUkm5XtBzC5cfkiDzqAHmr+J8yv5CJiOAYuOueQ+yY2KJd0qlvMEmewBYr0rDsK kCuGEqrcxc0khtu8Wt1mdqHGhLN2yoMafhw2CUCggxiqrB5xLuoydNdAM/i/YuA1 76REBt/7LckUAH2Lb0Ej8TD0UC5w1G7MoQ1m1QEwY3ONfgdAsZ/L/Q== =wble -END PGP SIGNATURE-
Re: qmail on AFS
-BEGIN PGP SIGNED MESSAGE- It appears that AFS is like Coda (you have to use rename() instead of link()). Try to search in the archive about Coda, you'll find a lot of discussions, with some patches. Vpopmail is being patched to work with Coda too, in the dev release. Good luck - --- Cordiali saluti / Best regards Andrea Cerrito ^^ Net.Admin @ Centro MultiMediale di Terni S.p.A. P.zzale Bosco 3A 05100 Terni IT Tel. +39 0744 5441330 Fax. +39 0744 5441372 -BEGIN PGP SIGNATURE- Version: PGPfreeware 6.5.3 for non-commercial use http://www.pgp.com iQEVAwUBO2+AWfo9HK4+yTI3AQF3DAf+N9I8iXUkdVWmxMyQa70LSJdVMyDsDyoX 8vfuTExnCULfPRAf7f6LuTqW+bcFyxw+8K8qvE9Q1aeV4yGKIp3LPoSsKY9FU7te xxgWtRnsfJeeQRUEo1Uh82ATuw201fOk2Uc6qwEGupu7picp3xFK6VeaUvhW3+L0 5V46DC0atz+fKVjAHUJq1ZArrt48+As2WqDKAFHjrFVK1jn8BA0VAREt3Ay2bFVe Fp5HHzfeRaAWQx1lLdqFm2YKmavwbWnTvDI5TsHUjTu3RqlNKID4C5kqyvmLDmza PEsqsYdwC+/rbRnLgDBfBdbtYqG1HnX9nt7BahYLr6HevV8ZnIedYQ== =KjBI -END PGP SIGNATURE-
Re: qmail on AFS
On Mon, Aug 06, 2001 at 09:44:36PM -0400, Rudy Zung wrote: [snip] Hmmm. I hadn't thought of that possibility; however, I am hoping that I wouldn't casually lose an email message because the file name is constructed by the combination of the system time and the process ID, which should be sufficiently unique, except possibly when the system clock is reset, and the PIDs recycle. The tmp file name is given to stat() which makes sure that the filename does not yet exist where the tmp directory has AFS ACL system:anyuser lidk (so, this allows anybody to delete the file, but should that happen, then the rename would fail, and qmail should notice that error and defer delivery (I hope)). Yes, if the rename() fails and you haven't broken anything massively, qmail will note that delivery failed and try again later. The next part is that I am now using rename() to move the file from tmp to new. For the new directory, I have AFS ACL system:anyuser ik. The system:anyuser has no permission to delete files, and this is what I am using to hope that the rename() will not be able to overwrite an existing file because that would mean a delete would have to be done, or the file would have to be rewritten, which are not permissible with ik ACLs. I am completely unfamiliar with AFS. If the non-delete clause makes sure rename() doesn't replace any file, that would be cool and should be sufficient as far as I can see. I guess what I can do is to modify qmail-local.c's maildir_child() and make it always generate the same file name as a test case and see how it behaves. Sounds like a plan. Please do :) That's just what I'm surmising; certainly glad for any input so that I don't totally mangle my email and start losing all sorts of email. Nevertheless, thank you for the rename() unsafe warning. You're welcome. Let us know how you progress. Oh, and be sure to, whenever you ask questions in the future, point out that you are using modified maildir delivery. We love to know about that before we break our heads :) Just a hint. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
Re: qmail on AFS
On Thu, Aug 02, 2001 at 10:17:06PM -0400, Rudy Zung wrote: On Thu, Aug 02, 2001 at 01:53:55PM -0700, Greg White wrote: On Thu, Aug 02, 2001 at 09:44:47AM -0400, Rudy Zung wrote: On Wed, Aug 01, 2001 at 03:27:49PM -0700, Greg White wrote: On Wed, Aug 01, 2001 at 05:17:46PM -0400, Rudy Zung wrote: [Don't remember if I've already posted this; sorry if it's a repeat.] Trying to run qmail with Maildir on a Slackware with AFS. The AFS is installed and runs properly. Users' home volumes and Maildir are on AFS. Qmail is generating temporary delivery errors; the mail never gets delivered. Seeing the temporary delivery errors would likely be helpful here. What Do the Logs Say? [...ellide...] Doing some digging, I found some folks discussing qmail and AFS, and discussing AFS/Kerberos tokens, and a workaround for same -- I presume that a token is required? [...ellided...] Best that I can guess, is that the error is generated in local.c; looks like it performs a chdir(), and is doing some unlinks and hard links. Solution found; traced qmail-local.c and determined failure encountered in maildir_child() during a call to link(). Rummaging around AFS FAQ in chapter on differences between AFS and Unix FS, it says that hard links across directories are not supported by AFS because AFS ACLs protect directories and not files, and allowing hard links to cross directories would circumvent the ACL. I changed the link() in qmail-local.c to rename() and my mail delivery now works. Anyone want to put in some input as to whether a rename() would fail in some cases where a link() might not? -- ...Ru (a low-cost superhero) On, on! Blue skies. Think snow. 1740484I 998300172 076662 82968/A17215 045124P E286/184435 975-203608 11859 DS1160
Re: qmail on AFS
On Mon, Aug 06, 2001 at 11:00:09AM -0400, Rudy Zung wrote: [snip] I changed the link() in qmail-local.c to rename() and my mail delivery now works. Anyone want to put in some input as to whether a rename() would fail in some cases where a link() might not? Using rename() instead of link() takes away the reliability of Maildir. If for some reason, two processes generate the same filename for a message, no matter within what timespan, you lose the first of these two. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
RE: qmail on AFS
Hi, I am testing log rotation with isolog on multilog. I can't understand why after sending a ALRM signal to the multilog process, the current log is not immediatly dumped into the isolog log. Thanks
Re: qmail on AFS
On Mon, Aug 06, 2001 at 07:02:25PM +0200, Olivier Dupuis wrote: Hi, I am testing log rotation with isolog on multilog. I can't understand why after sending a ALRM signal to the multilog process, the current log is not immediatly dumped into the isolog log. And what, exactly, does this have to do with 'qmail on AFS'? Please start a new thread with a new subject if you are asking a new, unrelated question. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
Re: qmail on AFS
On Mon, Aug 06, 2001 at 07:04:42PM +0200, Peter van Dijk wrote: On Mon, Aug 06, 2001 at 11:00:09AM -0400, Rudy Zung wrote: [snip] I changed the link() in qmail-local.c to rename() and my mail delivery now works. Anyone want to put in some input as to whether a rename() would fail in some cases where a link() might not? Using rename() instead of link() takes away the reliability of Maildir. If for some reason, two processes generate the same filename for a message, no matter within what timespan, you lose the first of these two. Greetz, Peter Hmmm. I hadn't thought of that possibility; however, I am hoping that I wouldn't casually lose an email message because the file name is constructed by the combination of the system time and the process ID, which should be sufficiently unique, except possibly when the system clock is reset, and the PIDs recycle. The tmp file name is given to stat() which makes sure that the filename does not yet exist where the tmp directory has AFS ACL system:anyuser lidk (so, this allows anybody to delete the file, but should that happen, then the rename would fail, and qmail should notice that error and defer delivery (I hope)). The next part is that I am now using rename() to move the file from tmp to new. For the new directory, I have AFS ACL system:anyuser ik. The system:anyuser has no permission to delete files, and this is what I am using to hope that the rename() will not be able to overwrite an existing file because that would mean a delete would have to be done, or the file would have to be rewritten, which are not permissible with ik ACLs. I guess what I can do is to modify qmail-local.c's maildir_child() and make it always generate the same file name as a test case and see how it behaves. That's just what I'm surmising; certainly glad for any input so that I don't totally mangle my email and start losing all sorts of email. Nevertheless, thank you for the rename() unsafe warning. -- ...Ru (a low-cost superhero) On, on! Blue skies. Think snow. 1740484I 998300172 076662 82968/A17215 045124P E286/184435 975-203608 11859 DS1160
Re: qmail on AFS
On Thu, Aug 02, 2001 at 01:53:55PM -0700, Greg White wrote: On Thu, Aug 02, 2001 at 09:44:47AM -0400, Rudy Zung wrote: On Wed, Aug 01, 2001 at 03:27:49PM -0700, Greg White wrote: On Wed, Aug 01, 2001 at 05:17:46PM -0400, Rudy Zung wrote: [Don't remember if I've already posted this; sorry if it's a repeat.] Trying to run qmail with Maildir on a Slackware with AFS. The AFS is installed and runs properly. Users' home volumes and Maildir are on AFS. Qmail is generating temporary delivery errors; the mail never gets delivered. Seeing the temporary delivery errors would likely be helpful here. What Do the Logs Say? [...ellide...] Doing some digging, I found some folks discussing qmail and AFS, and discussing AFS/Kerberos tokens, and a workaround for same -- I presume that a token is required? A google search on qmail + AFS returned the hits very quickly, and they contained the basics of the workaround, from what it seemed to me. Not being an AFS user, I can't help you much more than that. The only other hits on that error message were quota-overfull, or unwritable homes, and such. Any qmail+AFS users care to step into the conversation at this point? I tried google search as well but seemed to have come across a lot of hits of emails from people where the RFC headers included the with qmail words and the subject matter had something to do with AFS (but not necessarily qmail with AFS.) AFS imposes its own idea on the filesystem; you identity is authenticated with Kerberos, which then issues you a token which is valid for approximately 24 hours; the token basically proves that you are who you claim to be by virtue of the fact that you knew the authentication password. AFS also allows system:anyuser full access to files and directories; system:anyuser is really basically anyone, including those without tokens. I have tried setting the directory to essentially be unprotected and accessible by all. Starting a tokenless session, I am able to touch(1) and redirect echo(1) into files in the Maildir directory; so the problem that my qmail has may not necessarily be a tokens thing (although once I get qmail delivering, I will change it and require a token, and put in a cron job to reauth the token on a periodic basis so that the qmail jobs will always have a valid token.) Best that I can guess, is that the error is generated in local.c; looks like it performs a chdir(), and is doing some unlinks and hard links. Anyway, thanks for looking into this. -- ...Ru (a low-cost superhero) On, on! Blue skies. Think snow. 1740484I 998300172 076662 82968/A17215 045124P E286/184435 975-203608 11859 DS1160
qmail on AFS
[Don't remember if I've already posted this; sorry if it's a repeat.] Trying to run qmail with Maildir on a Slackware with AFS. The AFS is installed and runs properly. Users' home volumes and Maildir are on AFS. Qmail is generating temporary delivery errors; the mail never gets delivered. I've searched the archives but haven't found a solution, just tantalizing hints that other people have done this type of install also. I've even tried fs setacl on the Maildir, the cur and tmp directories to be system:anyuser rlidwka, but still no success. So, what's the magic configuration that allows qmail to work before go off and start writing cron jobs that automate the AFS/kerberos tokens renewal? -- ...Ru (a low-cost superhero) On, on! Blue skies. Think snow. 1740484I 998300172 076662 82968/A17215 045124P E286/184435 975-203608 11859 DS1160
Re: qmail on AFS
On Wed, Aug 01, 2001 at 05:17:46PM -0400, Rudy Zung wrote: [Don't remember if I've already posted this; sorry if it's a repeat.] Trying to run qmail with Maildir on a Slackware with AFS. The AFS is installed and runs properly. Users' home volumes and Maildir are on AFS. Qmail is generating temporary delivery errors; the mail never gets delivered. Seeing the temporary delivery errors would likely be helpful here. What Do the Logs Say? GW
