Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
TGPlatt, WebMaster wrote: I've probably just misunderstood something here. When you helped me with my setup in July, you had me create a mailman directory in the user's web space and we used that directory somehow. I've forgotten the exact reason we did that now but I vaguely recall it had something to do with working around my old Redhat 7.2 and Apache 1.2 server's suEXEC setup to gain access to the mailman programs. The user's web account still has that mailman directory present in it. It hadn't occurred to me yet that it might not need to be there at all now. To help jog your memory, its setup looks like this: myserver:/home/mylist/www# ls -als mailman total 12 4 drwxr-xr-x 3 mylist mylist 4096 Oct 20 07:22 . 4 drwxr-xr-x 12 mylist mylist 4096 Oct 29 17:46 .. 4 drwxrwsr-x 2598 mylist 4096 Oct 28 06:53 mail myserver:/home/mylist/www# ls -als mailman/mail total 48 4 drwxrwsr-x 2598 mylist 4096 Oct 28 06:53 . 4 drwxr-xr-x 3 mylist mylist 4096 Oct 20 07:22 .. 40 -rwxr-xr-x 1 598 mylist 39801 Jun 29 18:41 mailman So, when I asked about ownership and permissions and mentioned the UserAccount/mailman directory, that's the directory I was referring to. Sorry if I confused you. :-( Should I just remove that spare mailman directory from the user's space now? The only thing there is the mailman/mail/mailman wrapper whose sole purpose is to be SETGID mailman so that mail delivery runs in the mailman group. Since this wrapper isn't SETGID or group mailman, it wouldn't work in this installation anyway, so yes, just remove this directory. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
Thanks, Mark. That's all I needed to know! -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Friday, October 31, 2008 12:59 AM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: I've probably just misunderstood something here... blah... blah... blah... ...when I asked about ownership and permissions and mentioned the UserAccount/mailman directory, that's the directory I was referring to. Sorry if I confused you. :-( Should I just remove that spare mailman directory from the user's space now? The only thing there is the mailman/mail/mailman wrapper whose sole purpose is to be SETGID mailman so that mail delivery runs in the mailman group. Since this wrapper isn't SETGID or group mailman, it wouldn't work in this installation anyway, so yes, just remove this directory. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
I'm sorry to turn up in your support forum again, Mark. I've fought hard here to try to avoid that. The truth is it was our struggle with trying to configure my old server for mailman back in June and July that proved to be the straw that broke the camel's back with my old dedicated server hosting service. In early July I asked them about getting an updated server and software but the price they quoted was so darn high I decided to bail out on them and ended up choosing a hosting company that provided much more server for the $ plus the latest version of Debian Etch (rather than the 5 year old version of RedHat I was running) for the same price I'd been paying to my old hosting service. The BIG difference was I had to take full admin responsibility for my server setup and configuration; but I figured what the hell, I'm already doing 80% of that job anyway with very little support coming from my old host. So, when my old server came up for renewal at the end of July, I went month-to-month on my lease with them and bought the new server to take its place. I struggled through server setup and the migration of all my existing sites from the old server in August and September. At the end of September with just 3 sites left to move I grabbed the last 3 sites, made my final backups, and pulled the plug on the old server on the weekend before it would have renewed for another month... jumping out the window (from what turned out to be the 12th floor) with my final backups under my arm. :-) During October, I struggled to get the server set up to support mailman for multiple accounts. That entailed installing and testing it for the current client and setting up the server to support virtual domain hosting under Apache2, postfix and mailman so that we can eventually support mailman for multiple domains on this server. Until you mentioned suEXEC in your message yesterday, I'd nearly forgotten that part of our June-July nightmare. When I checked this morning, I found that when Apache2 was installed on this server it's standard installation process did include suEXEC. However, under Debian's Apache2 setup, it's easy to disable suEXEC and restart Apache. As far as I know, nothing else on the server was relying on the presence of suEXEC and it certainly wasn't my intent to install that Apache feature to begin with. So, suEXEC has now been disabled. Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? Or should I just put take a large dose of cyanide and go take a long nap instead? Thanks! -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 28, 2008 3:21 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: Could this be a group/owner-ship issue? Yes. It took me a while, but I finally connected you with our exchanges from last June/July :) It hadn't occurred to me to look for mailman's error log. Plus I had no idea where it was. I've seen it mentioned but I'd seen nothing in the docs that said where to find it. But with a bit of looking, I found it in /usr/local/mailman/logs/error The error log that was saved when we made our final September 28 backup on the old server was last updated 9/25. The more I look the more this looks like an ownership issue to me. It may be I screwed up somewhere back in July. In our current mailman directory structure a smattering of files throughout the mailman directory tree seem to be owned by root / mailman now; whereas in the old backup everything seems to have been owned by mailman / mailman or www-data (Debian's default Apache user) / mailman. On 9/28, the error log was owned by mailman / mailman. Indeed everything except mischief, subscribed and vet were owned by mailman / mailman back then. Those three files were different and were owned by www-data / mailman. Today in our running copy of mailman, all logs are owned by mailman/mailman except error which is owned by root / mailman and mischief, vet and subscribed which are owned by www-data / mailman. There were undoubtedly problems on the old server because of SUExec issues which I told you at the time was incompatible with Mailman's security model since Mailman's CGI wrappers can't be SETGID under SUExec and that's the whole point of the wrappers in the first place. So ownership and permissions within Mailman have to be such that the SUExec user can read and write. But the qrunner processes also have to be able to read and write and they will run as the mailman user:group. Normally the owner doesn't matter because everything runs as group mailman, but that may not be the case here. The reason I think this is an ownership issue is because when I look at the July - September error log I see lots of errors like this: Jul 09 07:29:22 2008
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
I'm sorry to turn up in your support forum again, Mark. I've fought hard here to try to avoid that. The truth is it was our struggle with trying to configure my old server for mailman back in June and July that proved to be the straw that broke the camel's back with my old dedicated server hosting service. In early July I asked them about getting an updated server and software but the price they quoted was so darn high I decided to bail out on them and ended up choosing a hosting company that provided much more server for the $ plus the latest version of Debian Etch (rather than the 5 year old version of RedHat I was running) for the same price I'd been paying to my old hosting service. The BIG difference was I had to take full admin responsibility for my server setup and configuration; but I figured what the hell, I'm already doing 80% of that job anyway with very little support coming from my old host. So, when my old server came up for renewal at the end of July, I went month-to-month on my lease with them and bought the new server to take its place. I struggled through server setup and the migration of all my existing sites from the old server in August and September. At the end of September with just 3 sites left to move I grabbed the last 3 sites, made my final backups, and pulled the plug on the old server on the weekend before it would have renewed for another month... jumping out the window (from what turned out to be the 12th floor) with my final backups under my arm. :-) During October, I struggled to get the server set up to support mailman for multiple accounts. That entailed installing and testing it for the current client and setting up the server to support virtual domain hosting under Apache2, postfix and mailman so that we can eventually support mailman for multiple domains on this server. Until you mentioned suEXEC in your message yesterday, I'd nearly forgotten that part of our June-July nightmare. When I checked this morning, I found that when Apache2 was installed on this server it's standard installation process did include suEXEC. However, under Debian's Apache2 setup, it's easy to disable suEXEC and restart Apache. As far as I know, nothing else on the server was relying on the presence of suEXEC and it certainly wasn't my intent to install that Apache feature to begin with. So, suEXEC has now been disabled. Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? Shouldn't all (or most of) these files be owned by mailman / mailman? Is that also true in the /archives/private/mylist directory? What about over in the /home/mylist/www directory? Who should own the mailman directory there? I have run bin/check_perms several times but it doesn't complain about any problems. Thanks! Or should I just put take a large dose of cyanide and go take a long nap instead? Thanks! -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 28, 2008 3:21 PM TGPlatt, WebMaster wrote: Could this be a group/owner-ship issue? Yes. It took me a while, but I finally connected you with our exchanges from last June/July :) snip --- There were undoubtedly problems on the old server because of SUExec issues which I told you at the time was incompatible with Mailman's security model since Mailman's CGI wrappers can't be SETGID under SUExec and that's the whole point of the wrappers in the first place. So ownership and permissions within Mailman have to be such that the SUExec user can read and write. But the qrunner processes also have to be able to read and write and they will run as the mailman user:group. Normally the owner doesn't matter because everything runs as group mailman, but that may not be the case here. snip and when I check the files in that directory, they're owned by root as a member of the group mailman. That should be OK because ArchRunner should be running as group mailman and the files should be group writable. snip when I look at the qfiles/shunt directory in the 9/28 backup, the oldest file there seems to be from 09-20-2008. So it looks to me like it was only keeping those shunt files 5 or 6 days before discarding them. If you are running Mailman 2.1.11, there is a cron that runs daily and by default it discards anything in qfiles/bad and qfiles/shunt older than 7 days. From Defaults.py # The length of time after which a qfiles/bad or qfiles/shunt file is # considered to be stale. Set to zero to disable culling of qfiles/bad and # qfiles/shunt entries. BAD_SHUNT_STALE_AFTER = days(7) # The pathname of a directory (searchable and writable by the Mailman cron # user) to which the culled qfiles/bad and qfiles/shunt entries will be # moved. Set to None to simply delete the culled entries. BAD_SHUNT_ARCHIVE_DIRECTORY = None snip So you still have permissions issues. You could start with bin/check_perms which
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
TGPlatt, WebMaster wrote: Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? Shouldn't all (or most of) these files be owned by mailman / mailman? Is that also true in the /archives/private/mylist directory? What about over in the /home/mylist/www directory? Who should own the mailman directory there? I have run bin/check_perms several times but it doesn't complain about any problems. check_perms should complain about ownership and permission problems. Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. In general everything from /usr/local/mailman on down should be group mailman (owner doesn't matter) and directories need to be g+rws and files g+rw. See the post at http://mail.python.org/pipermail/mailman-users/2008-October/063748.html. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
It looks to me like check_perms has a hole in it, Mark. If you take a close look at the error I've been getting repeatedly since early July (see below), you'll see it consistently occurs on the index.html table in the /usr/local/mailman/archives/private/mylist directory. Here are the permissions on that file: 12 -rw-r--r-- 1 mailman mailman 11452 Oct 20 07:01 index.html Please note that the permissions on that file are 644 and not 664. I just ran check_perms on this account. It reported No Problems Found. Yet when Archrunner runs and tries to open that file to replace it, it reports a permissions error for that file. In short, check_perms reports those 644 permissions are just fine while you're telling me they should be 664 and ARCHrunner complains they're NOT fine. This suggests to me check_perms isn't doing its job very well. Am I wrong about this? Here's the error ARCHrunner has been reporting repeatedly since early July... Oct 28 10:09:30 2008 (2589) Uncaught runner exception: [Errno 13] Permission denied: '/usr/local/mailman/archives/private/mylist/index.html' Oct 28 10:09:30 2008 (2589) Traceback (most recent call last): File /usr/local/mailman/Mailman/Queue/Runner.py, line 120, in _oneloop self._onefile(msg, msgdata) File /usr/local/mailman/Mailman/Queue/Runner.py, line 191, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File /usr/local/mailman/Mailman/Queue/ArchRunner.py, line 73, in _dispose mlist.ArchiveMail(msg) File /usr/local/mailman/Mailman/Archiver/Archiver.py, line 217, in ArchiveMa il h.close() File /usr/local/mailman/Mailman/Archiver/pipermail.py, line 324, in close self.write_TOC() File /usr/local/mailman/Mailman/Archiver/HyperArch.py, line 1097, in write_T OC toc = open(os.path.join(self.basedir, 'index.html'), 'w') IOError: [Errno 13] Permission denied: '/usr/local/mailman/archives/private/mylist/index.html' What am I missing here? -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 29, 2008 5:50 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? Shouldn't all (or most of) these files be owned by mailman / mailman? Is that also true in the /archives/private/mylist directory? What about over in the /home/mylist/www directory? Who should own the mailman directory there? I have run bin/check_perms several times but it doesn't complain about any problems. check_perms should complain about ownership and permission problems. Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. In general everything from /usr/local/mailman on down should be group mailman (owner doesn't matter) and directories need to be g+rws and files g+rw. See the post at http://mail.python.org/pipermail/mailman-users/2008-October/063748.html. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
You said: Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. But mailman's home directory (where all its programs, scripts, archives and discussion lists are stored) IS /usr/local/mailman and according to my Mailman/Defaults.py, both var_prefix and prefix are also /user/local/mailman PREFIX = '/usr/local/mailman' VAR_PREFIX = '/usr/local/mailman' How does that change things? -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 29, 2008 5:50 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? Shouldn't all (or most of) these files be owned by mailman / mailman? Is that also true in the /archives/private/mylist directory? What about over in the /home/mylist/www directory? Who should own the mailman directory there? I have run bin/check_perms several times but it doesn't complain about any problems. check_perms should complain about ownership and permission problems. Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. In general everything from /usr/local/mailman on down should be group mailman (owner doesn't matter) and directories need to be g+rws and files g+rw. See the post at http://mail.python.org/pipermail/mailman-users/2008-October/063748.html. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
TGPlatt, WebMaster wrote: Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. But mailman's home directory (where all its programs, scripts, archives and discussion lists are stored) IS /usr/local/mailman and according to my Mailman/Defaults.py, both var_prefix and prefix are also /user/local/mailman PREFIX = '/usr/local/mailman' VAR_PREFIX = '/usr/local/mailman' How does that change things? When you said: Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? it seemed to me you were talking about two separate directories, /usr/local/mailman and UserAccount/mailman. Whether or not these are separate directories or the same directory, the ownership and permissions on /usr/local/mailman are the only ones that matter. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
TGPlatt, WebMaster wrote: It looks to me like check_perms has a hole in it, Mark. If you take a close look at the error I've been getting repeatedly since early July (see below), you'll see it consistently occurs on the index.html table in the /usr/local/mailman/archives/private/mylist directory. Here are the permissions on that file: 12 -rw-r--r-- 1 mailman mailman 11452 Oct 20 07:01 index.html Please note that the permissions on that file are 644 and not 664. I just ran check_perms on this account. It reported No Problems Found. Yet when Archrunner runs and tries to open that file to replace it, it reports a permissions error for that file. In short, check_perms reports those 644 permissions are just fine while you're telling me they should be 664 and ARCHrunner complains they're NOT fine. This suggests to me check_perms isn't doing its job very well. This appears to be a problem with check_perms. You are correct that it isn't doing its job very well. There are a lot of files it doesn't check, and at least some of those should be checked. I'll look into fixing that. Thanks for pointing it out. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
I've probably just misunderstood something here. When you helped me with my setup in July, you had me create a mailman directory in the user's web space and we used that directory somehow. I've forgotten the exact reason we did that now but I vaguely recall it had something to do with working around my old Redhat 7.2 and Apache 1.2 server's suEXEC setup to gain access to the mailman programs. The user's web account still has that mailman directory present in it. It hadn't occurred to me yet that it might not need to be there at all now. To help jog your memory, its setup looks like this: myserver:/home/mylist/www# ls -als mailman total 12 4 drwxr-xr-x 3 mylist mylist 4096 Oct 20 07:22 . 4 drwxr-xr-x 12 mylist mylist 4096 Oct 29 17:46 .. 4 drwxrwsr-x 2598 mylist 4096 Oct 28 06:53 mail myserver:/home/mylist/www# ls -als mailman/mail total 48 4 drwxrwsr-x 2598 mylist 4096 Oct 28 06:53 . 4 drwxr-xr-x 3 mylist mylist 4096 Oct 20 07:22 .. 40 -rwxr-xr-x 1 598 mylist 39801 Jun 29 18:41 mailman So, when I asked about ownership and permissions and mentioned the UserAccount/mailman directory, that's the directory I was referring to. Sorry if I confused you. :-( Should I just remove that spare mailman directory from the user's space now? -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 29, 2008 8:56 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: Assuming mailman's home directory is NOT /usr/local/mailman and assuming that prefix, and var_prefix are /usr/local/mailman, mailman's home directory is irrelevant. But mailman's home directory (where all its programs, scripts, archives and discussion lists are stored) IS /usr/local/mailman and according to my Mailman/Defaults.py, both var_prefix and prefix are also /user/local/mailman PREFIX = '/usr/local/mailman' VAR_PREFIX = '/usr/local/mailman' How does that change things? When you said: Now can we talk about the proper ownership of all the mailman files both IN /usr/local/mailman directory structure and in the UserAccount/mailman directory as well? it seemed to me you were talking about two separate directories, /usr/local/mailman and UserAccount/mailman. Whether or not these are separate directories or the same directory, the ownership and permissions on /usr/local/mailman are the only ones that matter. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
LOL!! Bear in mind my guru friend that what's apparent to someone with your years of experience and wisdom isn't the least bit obvious to a mailman newcomers like me. Naïve as it may be, we tend to assume that when the program says it checked permissions and found no problems we're not likely to encounter permission issues with mailman's own automated processes later. I know that's a silly assumption to make, but we'll tend to make it anyway. ;-) That's why I deliberately avoided seeking your help while I struggled and did extensive testing and research for DAYS trying to figure out exactly how to configure Apache, Postfix and Mailman with virtual alias domains so that I might at least have a prayer someday of being able to support more than one mailman discussion list on my server. As a result of that struggle, I may not know everything there IS to know, but at least understand the basic concepts now. I'm engaged in a similar struggle now in trying to understand mailman ownership and permissions. I'm not sure I understand how ownership and permissions play out in mailman. So, I'm forced to proceed with absolute faith that YOU know what you're doing, sir. For my part, I feel like a blind man who is flying a wide-body jet filled with screaming passengers into Atlanta at rush hour for the first time. And YOU and the mailman tools are my ONLY guides. So, since the check_perms tool doesn't work flawlessly, I'm relying on you to tell me how to do it right, oh Guru! B-) Thanks for being so patient and understanding. -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 29, 2008 9:18 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: RE: [Mailman-Users] What happened to my archive? Why isn't the archive process running? TGPlatt, WebMaster wrote: It looks to me like check_perms has a hole in it, Mark. If you take a close look at the error I've been getting repeatedly since early July (see below), you'll see it consistently occurs on the index.html table in the /usr/local/mailman/archives/private/mylist directory. Here are the permissions on that file: 12 -rw-r--r-- 1 mailman mailman 11452 Oct 20 07:01 index.html Please note that the permissions on that file are 644 and not 664. I just ran check_perms on this account. It reported No Problems Found. Yet when Archrunner runs and tries to open that file to replace it, it reports a permissions error for that file. In short, check_perms reports those 644 permissions are just fine while you're telling me they should be 664 and ARCHrunner complains they're NOT fine. This suggests to me check_perms isn't doing its job very well. This appears to be a problem with check_perms. You are correct that it isn't doing its job very well. There are a lot of files it doesn't check, and at least some of those should be checked. I'll look into fixing that. Thanks for pointing it out. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
First, I was wrong here. I was so horrified at the thought of having lost all those old messages that I got the date wrong. It's actually worse than I remembered. The last archive update appears to have happened in early July... not early August. My last backup from the old server was September 28. It took 3 weeks for me to get mailman working again on the new host in multi-domain (virtual host) mode. So there are 3 months worth of archive updates that appear to be lost. I looked close at our 9/28 tar-ball backup from the old server and there seems to be nothing in the qfiles/archive directory at all. Indeed, there's nothing in qfiles/bad, bounces, command, in, news, out, retry or virgin either. Shunt has a few pck files but they all begin on 10/21 which is when mailman started running again on the new server. For the record, I see no evidence the cron for archive updates is running now on the new server either. In fact from your comments, I'd say the pck files in qfiles/shunt strongly suggest that it's NOT running or at least not running correctly. Here's what I see when I look at the current mailman archive/private/listname.mbox directory 4 drwxrwsr-x 2 ourlistmailman4096 2008-10-28 06:34 . 4 drwxrws--- 8 nobody mailman4096 2008-07-10 06:53 .. 0 lrwxrwxrwx 1 root mailman 41 2008-10-28 06:34 arc - /www/ourlist/newtest/mailman archives/ 5624 -rw-r--r-- 1 root mailman 5743206 2008-07-07 06:22 ourlist.mbox Note the date on the ourlist.mbox file. That's the last time the archive process ran successfully. Something is obviously wrong here; but I'm not sure exactly what. Can you advise me, Mark? Thanks. -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 5:51 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: Re: [Mailman-Users] What happened to my archive? Why isn't the archiveprocess running? TGPlatt, WebMaster wrote: I seem to be missing two full months worth of archives. From what I can tell looking at my backups, it appears that the last time the archive process actually ran on our old server was in early August. Then for some reason it stopped funning and thee have been no archive updates since then. I suspect those lost messages aren't really lost at all but are merely caught in some sort of internal blockage in mailman. What I'm trying to figure out here is how to clear the clogged drain pipe and get the archive working again. There are a few possibilities: 1) ArchiveRunner died on the old server in early August. In this case, the messages would be on the old server in the qfiles/arch/ directory. If that is the case, you could just move the contents of that directory to the corresponding directory on the new server and that should do it. 2) Some error was shunting the archived messages in which case they may be in qfiles/shunt/ and they may or may not be in the individual archives/private/listname.mbox/listname.mbox files. If the messages are in qfiles/shunt/, you could just move them and rin bin/unshunt, but you need to be careful as not all the files in qfiles/shunt/ may be messages you want. You can look at these entries with bin/show_qfiles or bin/dumpdb. If the messages are in the listname.mbox files, the easiest thing is probably to rebuild the archives with bin/arch --wipe. When I first installed the backup from the old server to the new one and then checked the list's configuration parameters, I saw some parameters on the Archiving options page that provided for rebuilding the archive. But now those options seem to have disappeared now too. I don't know what you say, but there's nothing in the web Archiving Options page about rebuilding archives. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
Follow-up to last message. Wasn't sure exactly how to examine the contents of the ourlist.mbox file in /usr/local/mailman/archives/private/ourlist.mbox SO, I opened it with emacs just to see what it contained. The last message in that file is one I posted on July 7th. The prospects of recovering all those missing messages are looking dimmer and dimmer... Sigh! :-( -Original Message- From: Mark Sapiro [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 5:51 PM To: [EMAIL PROTECTED]; mailman-users@python.org Subject: Re: [Mailman-Users] What happened to my archive? Why isn't the archiveprocess running? TGPlatt, WebMaster wrote: I seem to be missing two full months worth of archives. From what I can tell looking at my backups, it appears that the last time the archive process actually ran on our old server was in early August. Then for some reason it stopped funning and thee have been no archive updates since then. I suspect those lost messages aren't really lost at all but are merely caught in some sort of internal blockage in mailman. What I'm trying to figure out here is how to clear the clogged drain pipe and get the archive working again. There are a few possibilities: 1) ArchiveRunner died on the old server in early August. In this case, the messages would be on the old server in the qfiles/arch/ directory. If that is the case, you could just move the contents of that directory to the corresponding directory on the new server and that should do it. 2) Some error was shunting the archived messages in which case they may be in qfiles/shunt/ and they may or may not be in the individual archives/private/listname.mbox/listname.mbox files. If the messages are in qfiles/shunt/, you could just move them and rin bin/unshunt, but you need to be careful as not all the files in qfiles/shunt/ may be messages you want. You can look at these entries with bin/show_qfiles or bin/dumpdb. If the messages are in the listname.mbox files, the easiest thing is probably to rebuild the archives with bin/arch --wipe. When I first installed the backup from the old server to the new one and then checked the list's configuration parameters, I saw some parameters on the Archiving options page that provided for rebuilding the archive. But now those options seem to have disappeared now too. I don't know what you say, but there's nothing in the web Archiving Options page about rebuilding archives. -- Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Users] What happened to my archive? Why isn't the archive process running?
TGPlatt, WebMaster wrote: Could this be a group/owner-ship issue? Yes. It took me a while, but I finally connected you with our exchanges from last June/July :) It hadn't occurred to me to look for mailman's error log. Plus I had no idea where it was. I've seen it mentioned but I'd seen nothing in the docs that said where to find it. But with a bit of looking, I found it in /usr/local/mailman/logs/error The error log that was saved when we made our final September 28 backup on the old server was last updated 9/25. The more I look the more this looks like an ownership issue to me. It may be I screwed up somewhere back in July. In our current mailman directory structure a smattering of files throughout the mailman directory tree seem to be owned by root / mailman now; whereas in the old backup everything seems to have been owned by mailman / mailman or www-data (Debian's default Apache user) / mailman. On 9/28, the error log was owned by mailman / mailman. Indeed everything except mischief, subscribed and vet were owned by mailman / mailman back then. Those three files were different and were owned by www-data / mailman. Today in our running copy of mailman, all logs are owned by mailman/mailman except error which is owned by root / mailman and mischief, vet and subscribed which are owned by www-data / mailman. There were undoubtedly problems on the old server because of SUExec issues which I told you at the time was incompatible with Mailman's security model since Mailman's CGI wrappers can't be SETGID under SUExec and that's the whole point of the wrappers in the first place. So ownership and permissions within Mailman have to be such that the SUExec user can read and write. But the qrunner processes also have to be able to read and write and they will run as the mailman user:group. Normally the owner doesn't matter because everything runs as group mailman, but that may not be the case here. The reason I think this is an ownership issue is because when I look at the July - September error log I see lots of errors like this: Jul 09 07:29:22 2008 (10592) Archive file access failure: /usr/local/mailman/archives/private/ourlist.mbox/ourlist.mbox [Errno 13] Permission denied: '/usr/local/mailman/archives/private/ourlist.mbox/ourlist.mbox' Jul 09 07:29:22 2008 (10592) Uncaught runner exception: [Errno 13] Permission denied: '/usr/local/mailman/archives/private/ourlist .mbox/ourlist.mbox' .and when I check the files in that directory, they're owned by root as a member of the group mailman. That should be OK because ArchRunner should be running as group mailman and the files shoud be group writable. snip Sep 25 21:09:31 2008 (16599) SHUNTING: 1222394969.723012+6122e5f84bb97875a24698a159961936bed8802b Sadly, when I look at the qfiles/shunt directory in the 9/28 backup, the oldest file there seems to be from 09-20-2008. So it looks to me like it was only keeping those shunt files 5 or 6 days before discarding them. If you are running Mailman 2.1.11, there is a cron that runs daily and by default it discards anything in qfiles/bad and qfiles/shunt older than 7 days. From Defaults.py # The length of time after which a qfiles/bad or qfiles/shunt file is # considered to be stale. Set to zero to disable culling of qfiles/bad and # qfiles/shunt entries. BAD_SHUNT_STALE_AFTER = days(7) # The pathname of a directory (searchable and writable by the Mailman cron # user) to which the culled qfiles/bad and qfiles/shunt entries will be # moved. Set to None to simply delete the culled entries. BAD_SHUNT_ARCHIVE_DIRECTORY = None Here are the first and last error in the current error file: Oct 18 10:43:54 2008 mailmanctl(25124): Site list is missing: mailman Oct 18 10:43:54 2008 (25124) Site list is missing: mailman Oct 18 18:38:42 2008 (14433) admin.py access for non-existent list: ourlist Oct 18 18:39:34 2008 (14458) admin.py access for non-existent list: ourlist Oct 18 20:34:08 2008 (19034) admin.py access for non-existent list: ourlist Oct 20 08:58:31 2008 (9822) admin.py access for non-existent list: ourlist Oct 21 22:33:46 2008 (1431) Uncaught runner exception: [Errno 13] Permission den ied: '/usr/local/mailman/archives/private/ourlist/index.html' Oct 21 22:33:46 2008 (1431) Traceback (most recent call last): File /usr/local/mailman/Mailman/Queue/Runner.py, line 120, in _oneloop self._onefile(msg, msgdata) ... Oct 28 10:09:30 2008 (2589) Uncaught runner exception: [Errno 13] Permission den ied: '/usr/local/mailman/archives/private/ourlist/index.html' Oct 28 10:09:30 2008 (2589) Traceback (most recent call last): File /usr/local/mailman/Mailman/Queue/Runner.py, line 120, in _oneloop self._onefile(msg, msgdata) File /usr/local/mailman/Mailman/Queue/Runner.py, line 191, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File /usr/local/mailman/Mailman/Queue/ArchRunner.py, line 73, in _dispose