[Mailman-Users] Re: Member settings etc. not applied
You brought me on the right track In mm_cfp.py it had to be DEFAULT_URL_PATTERN = 'https://%s/mailman/' instead of DEFAULT_URL_PATTERN = 'http://%s/mailman/' Simple as that, but I had overlooked it. bin/withlist -l -r fix_url [liste] -u [domain] did the rest. Thank you! Lothar Am 21.06.2021 um 17:24 schrieb Stephen J. Turnbull: > Lothar Schilling writes: > > > I set up mailman on a new server (Ubuntu 20, Mailman 2.1.29) and moved > > all user data (archives, lists) from the old server to that new > > environment. > > How did you set it up? From Ubuntu packages? > > > "The information you're about to submit ist not secure." > > I don't think this is a Mailman-supported message. I expect it is an > Ubuntu thing. I don't know if that is why your configuration update > is failing for sure, but it seems likely, and you will probably need > to get help from Ubuntu sources. > > The other possibility is that some of your URLs are https and others > are http. Check exactly what the URLs are. > > Steve > > > > -- Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-le...@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
[Mailman-Users] Member settings etc. not applied
Hi everybody, I set up mailman on a new server (Ubuntu 20, Mailman 2.1.29) and moved all user data (archives, lists) from the old server to that new environment. There were no major problems, at least that's what I thought. But now I found out - there are. Whenever I try to change in a list, let's say, member settings or approve administrative requests I get this: "The information you're about to submit ist not secure." And my changes won't be applied. Even when setting up a new list I'm running into this very error. Any ideas appreciated, thank you. Kind regards Lothar Schilling -- Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-le...@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
[Mailman-Users] Re: Trouble migrating
Am 27.04.2020 um 18:42 schrieb Mark Sapiro: On 4/26/20 11:04 PM, Lothar Schilling wrote: ps -fwwA | grep qrunner mailman 7439 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s mailman 7440 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s mailman 7441 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s mailman 7442 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s mailman 7443 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s mailman 7444 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s mailman 7445 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s mailman 7446 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=RetryRunner:0:1 -s mailman 7468 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s mailman 7469 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s mailman 7470 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s mailman 7471 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s mailman 7472 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s mailman 7473 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s mailman 7474 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s mailman 7475 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=RetryRunner:0:1 -s This may be a separate issue, but you have two Mailman instances running. See <https://wiki.list.org/x/4030715> for how this can happen and what to do about it. You're right: There were two mailman instances running simultaneously. I doubt, though, that this was the reason for the migration going wrong. But I'll give try again next weekend and then come back to you - if needed. Thank you. -- Mailman-Users mailing list -- mailman-users@python.org To unsubscribe send an email to mailman-users-le...@python.org https://mail.python.org/mailman3/lists/mailman-users.python.org/ Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/ https://mail.python.org/archives/list/mailman-users@python.org/
Re: [Mailman-Users] Trouble migrating
Am 26.04.2020 um 19:46 schrieb Mark Sapiro: On 4/26/20 9:57 AM, Lothar Schilling wrote: Yes I did check the permissons several times, everything is fine. And no, SELinux was and still is disabled. What user is running Mailman's qrunners? ps -fwwA|grep qrunner And what does ls -laR /var/spool/mailman /var/lib/mailman give? ps -fwwA | grep qrunner mailman 7439 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s mailman 7440 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s mailman 7441 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s mailman 7442 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s mailman 7443 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s mailman 7444 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s mailman 7445 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s mailman 7446 7438 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=RetryRunner:0:1 -s mailman 7468 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s mailman 7469 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s mailman 7470 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s mailman 7471 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s mailman 7472 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s mailman 7473 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s mailman 7474 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s mailman 7475 7467 0 07:51 ? 00:00:00 /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=RetryRunner:0:1 -s ls -laR /var/spool/mailman drwxrws--- mailman mailman 4096 26. Apr 17:38 archive drwxrws--- mailman mailman 4096 26. Apr 17:52 bad drwxrws--- mailman mailman 4096 26. Apr 17:39 bounces drwxrws--- mailman mailman 4096 9. Okt 2015 commands drwxrws--- root mailman 4096 26. Apr 17:52 in drwxrws--- mailman mailman 4096 9. Okt 2015 news drwxrws--- mailman mailman 77824 26. Apr 17:52 out drwxrws--- mailman mailman 4096 9. Okt 2015 retry drwxrws--- mailman mailman 4096 26. Apr 17:38 shunt drwxrws--- root mailman 86016 26. Apr 17:52 virgin ls -laR /var/lib/mailman All files & directories have: -rwxrwsr-x apache mailman /usr/lib/mailman/bin/check_perms -f "Keine Probleme aufgetreten" -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Trouble migrating
Hi Tom, this is not meant for me, is it? My name's not David. Also, my problem is of a different kind. Regards Lothar Schilling Am 26.04.2020 um 18:44 schrieb Tom Coradeschi: David: Your qrunner exceptions sound surprisingly like a problem I had earlier this week with a new install of 2.1.30. In Mailman/Archiver/pipermail.py change line 60 from: while i>0 and (L[i-1][0] in lowercase or to: while i>0 and (L[i-1][0] in lowercase[:26] or Restart mailman and see how things go from there. — Tom Coradeschi tc...@skylands.ibmwr.org On 26 Apr 2020, at 12:05 PM, Lothar Schilling wrote: Hi everybody, today I tried to migrate our whole mailman installation to another server. Old server: Centos 6.10, Mailman 2.1.29, Python 2.6.6 New server: Centos 7.7, Mailman 2.1.30 (from the tgz file), Python 2.7.5 It seemed to be an easy transition. But then I started to realize I've run into problems. For example, I can't release a moderated message. Also, I cannot switch users from moderated to not moderated or hidden to unhidden. The error logs look like this: Apr 26 17:38:16 2020 (11727) Uncaught runner exception: [Errno 1] Operation not permitted Apr 26 17:38:16 2020 (11727) Traceback (most recent call last): Apr 26 17:38:16 2020 (11727) SHUNTING: 1587915493.168592+122b90464edb8dfe2a33eb4e84fd036f0943019c Apr 26 17:38:16 2020 (11727) Failed to unlink/preserve backup file: /var/spool/mailman/virgin/1587915493.168592+bfc8cbc934795ac0c0d663ac31c3ea3df13e9b59.bak Apr 26 17:52:18 2020 (11727) Uncaught runner exception: [Errno 2] No such file or directory: '/var/spool/mailman/virgin/1587916338.266272+f1e11334def7f62140a3442c10bd6b56268d9010.pck' Apr 26 17:52:18 2020 (11727) Traceback (most recent call last): Apr 26 17:52:18 2020 (11727) Skipping and preserving unparseable message: 1587916338.266272+f1e11334def7f62140a3442c10bd6b56268d9010 Apr 26 17:52:19 2020 (11726) Failed to unlink/preserve backup file: /var/spool/mailman/out/1587916338.266272+5c0165dec7b0fdd4e3851688a77b8e31a99b015d.bak Apr 26 17:52:20 2020 (11727) Failed to unlink/preserve backup file: /var/spool/mailman/virgin/1587916338.285911+c4b5275fd082841b395994955b37f5f85569bba9.bak Apr 26 17:53:13 2020 qrunner(11694): Traceback (most recent call last): Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/bin/qrunner", line 278, in Apr 26 17:53:13 2020 qrunner(11694): main() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/bin/qrunner", line 238, in main Apr 26 17:53:13 2020 qrunner(11694): qrunner.run() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 87, in run Apr 26 17:53:13 2020 qrunner(11694): self._cleanup() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 263, in _cleanup Apr 26 17:53:13 2020 qrunner(11694): BounceMixin._cleanup(self) Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 137, in _cleanup Apr 26 17:53:13 2020 qrunner(11694): self._register_bounces() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 126, in _register_bounces Apr 26 17:53:13 2020 qrunner(11694): mlist.Save() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/MailList.py", line 578, in Save Apr 26 17:53:13 2020 qrunner(11694): self.__save(dict) Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/MailList.py", line 555, in __save Apr 26 17:53:13 2020 qrunner(11694): os.link(fname, fname_last) Apr 26 17:53:13 2020 qrunner(11694): OSError : [Errno 1] Operation not permitted Apr 26 17:53:16 2020 qrunner(11722): Traceback (most recent call last): Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/bin/qrunner", line 278, in Apr 26 17:53:16 2020 qrunner(11722): main() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/bin/qrunner", line 238, in main Apr 26 17:53:16 2020 qrunner(11722): qrunner.run() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 87, in run Apr 26 17:53:16 2020 qrunner(11722): self._cleanup() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 263, in _cleanup Apr 26 17:53:16 2020 qrunner(11722): BounceMixin._cleanup(self) Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 137, in _cleanup Apr 26 17:53:16 2020 qrunner(11722): self._register_bounces() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 126, in _register_bounces Apr 26 17:53:16 2020 qrunner(11722): mlist.Save() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/MailList.py", line 578, in Save Apr 26 17:53:16 2020 qrunner(11722): self.__sav
Re: [Mailman-Users] Trouble migrating
Yes I did check the permissons several times, everything is fine. And no, SELinux was and still is disabled. Am 26.04.2020 um 18:37 schrieb Mark Sapiro: On 4/26/20 9:05 AM, Lothar Schilling wrote: Hi everybody, today I tried to migrate our whole mailman installation to another server. Old server: Centos 6.10, Mailman 2.1.29, Python 2.6.6 New server: Centos 7.7, Mailman 2.1.30 (from the tgz file), Python 2.7.5 It seemed to be an easy transition. But then I started to realize I've run into problems. For example, I can't release a moderated message. Also, I cannot switch users from moderated to not moderated or hidden to unhidden. The error logs look like this: Apr 26 17:38:16 2020 (11727) Uncaught runner exception: [Errno 1] Operation not permitted ... It looks like you have permission issues. Have you run Mailman's bin/check_perms? Do you have SELinux enabled? If so, try disabling it. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Trouble migrating
Hi everybody, today I tried to migrate our whole mailman installation to another server. Old server: Centos 6.10, Mailman 2.1.29, Python 2.6.6 New server: Centos 7.7, Mailman 2.1.30 (from the tgz file), Python 2.7.5 It seemed to be an easy transition. But then I started to realize I've run into problems. For example, I can't release a moderated message. Also, I cannot switch users from moderated to not moderated or hidden to unhidden. The error logs look like this: Apr 26 17:38:16 2020 (11727) Uncaught runner exception: [Errno 1] Operation not permitted Apr 26 17:38:16 2020 (11727) Traceback (most recent call last): Apr 26 17:38:16 2020 (11727) SHUNTING: 1587915493.168592+122b90464edb8dfe2a33eb4e84fd036f0943019c Apr 26 17:38:16 2020 (11727) Failed to unlink/preserve backup file: /var/spool/mailman/virgin/1587915493.168592+bfc8cbc934795ac0c0d663ac31c3ea3df13e9b59.bak Apr 26 17:52:18 2020 (11727) Uncaught runner exception: [Errno 2] No such file or directory: '/var/spool/mailman/virgin/1587916338.266272+f1e11334def7f62140a3442c10bd6b56268d9010.pck' Apr 26 17:52:18 2020 (11727) Traceback (most recent call last): Apr 26 17:52:18 2020 (11727) Skipping and preserving unparseable message: 1587916338.266272+f1e11334def7f62140a3442c10bd6b56268d9010 Apr 26 17:52:19 2020 (11726) Failed to unlink/preserve backup file: /var/spool/mailman/out/1587916338.266272+5c0165dec7b0fdd4e3851688a77b8e31a99b015d.bak Apr 26 17:52:20 2020 (11727) Failed to unlink/preserve backup file: /var/spool/mailman/virgin/1587916338.285911+c4b5275fd082841b395994955b37f5f85569bba9.bak Apr 26 17:53:13 2020 qrunner(11694): Traceback (most recent call last): Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/bin/qrunner", line 278, in Apr 26 17:53:13 2020 qrunner(11694): main() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/bin/qrunner", line 238, in main Apr 26 17:53:13 2020 qrunner(11694): qrunner.run() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 87, in run Apr 26 17:53:13 2020 qrunner(11694): self._cleanup() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 263, in _cleanup Apr 26 17:53:13 2020 qrunner(11694): BounceMixin._cleanup(self) Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 137, in _cleanup Apr 26 17:53:13 2020 qrunner(11694): self._register_bounces() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 126, in _register_bounces Apr 26 17:53:13 2020 qrunner(11694): mlist.Save() Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/MailList.py", line 578, in Save Apr 26 17:53:13 2020 qrunner(11694): self.__save(dict) Apr 26 17:53:13 2020 qrunner(11694): File "/usr/lib/mailman/Mailman/MailList.py", line 555, in __save Apr 26 17:53:13 2020 qrunner(11694): os.link(fname, fname_last) Apr 26 17:53:13 2020 qrunner(11694): OSError : [Errno 1] Operation not permitted Apr 26 17:53:16 2020 qrunner(11722): Traceback (most recent call last): Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/bin/qrunner", line 278, in Apr 26 17:53:16 2020 qrunner(11722): main() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/bin/qrunner", line 238, in main Apr 26 17:53:16 2020 qrunner(11722): qrunner.run() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 87, in run Apr 26 17:53:16 2020 qrunner(11722): self._cleanup() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 263, in _cleanup Apr 26 17:53:16 2020 qrunner(11722): BounceMixin._cleanup(self) Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 137, in _cleanup Apr 26 17:53:16 2020 qrunner(11722): self._register_bounces() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/Queue/BounceRunner.py", line 126, in _register_bounces Apr 26 17:53:16 2020 qrunner(11722): mlist.Save() Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/MailList.py", line 578, in Save Apr 26 17:53:16 2020 qrunner(11722): self.__save(dict) Apr 26 17:53:16 2020 qrunner(11722): File "/usr/lib/mailman/Mailman/MailList.py", line 555, in __save Apr 26 17:53:16 2020 qrunner(11722): os.link(fname, fname_last) Apr 26 17:53:16 2020 qrunner(11722): OSError : [Errno 1] Operation not permitted /usr/lib/mailman/bin/check_perms -f didn't give me any errors. I also tried to fall back to Centos' standard mailman package which is 2.1.15 but to no avail. This took me all day and I'm still where I was in the morning. Any help
Re: [Mailman-Users] Fwd: Uncaught runner exception
Am 21.03.2019 um 17:48 schrieb Mark Sapiro: > On 3/21/19 9:20 AM, Lothar Schilling wrote: >> Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) >> [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2 >> (InteractiveConsole) >>>>> from string import lowercase >>>>> lowercase >> 'abcdefghijklmnopqrstuvwxyz' >>>>> import string >>>>> string >> > > I'm out of ideas. I have no clue as to where the extended > string.lowercase is coming from and why only Mailman is seeing it, but > to work around that, I suggest you change my suggested patch to > > === modified file 'Mailman/Archiver/pipermail.py' > --- Mailman/Archiver/pipermail.py 2018-05-03 21:23:47 + > +++ Mailman/Archiver/pipermail.py 2019-03-02 04:51:23 + > @@ -60,9 +60,12 @@ > else: > # Mixed case; assume that small parts of the last name will be > # in lowercase, and check them against the list. > -while i>0 and (L[i-1][0] in lowercase or > - L[i-1].lower() in smallNameParts): > -i = i - 1 > +try: > +while i>0 and (L[i-1][0] in lowercase or > + L[i-1].lower() in smallNameParts): > +i = i - 1 > +except UnicodeDecodeError: > +pass > author = SPACE.join(L[-1:] + L[i:-1]) + ', ' + SPACE.join(L[:i]) > return author > > > This will catch the exception and ignore it without the logging. > > Also, I have no idea why when you run bin/unshunt, the .pck files remain > in qfiles/shunt without errors being reported to the terminal or logged > in mailman's error log. Something is strange about that, but I don't > know what could cause it. > Applied the patch. Thanks again for your kind advice, Mark! -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Fwd: Uncaught runner exception
Am 20.03.2019 um 18:54 schrieb Mark Sapiro: > On 3/20/19 5:31 AM, Lothar Schilling wrote: >> There is just one >> >> /usr/lib64/python2.6/string.py >> >> yielding >> >> lowercase = 'abcdefghijklmnopqrstuvwxyz'. > > You could try one more thing. Run Mailman's > > bin/withlist -i > ... >>>> from string import lowercase >>>> lowercase > (response) >>>> import string >>>> string > (response) > > and see if that returns the extended lowercase and where it says > lowercase comes from. > It doesn't return the extended lowercase: Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37) [GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2 (InteractiveConsole) >>> from string import lowercase >>> lowercase 'abcdefghijklmnopqrstuvwxyz' >>> import string >>> string -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Fwd: Uncaught runner exception
Am 18.03.2019 um 17:06 schrieb Mark Sapiro: > On 3/18/19 2:29 AM, Lothar Schilling wrote: >> Exception in fixAuthor: D▒rthe Hinz - Fl▒chtlingsrat Nds. >> Mar 18 08:33:51 2019 (23745) lowercase value: >> 'abcdefghijklmnopqrstuvwxyz\xb5\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' >> Mar 18 10:06:50 2019 (23745) Exception in fixAuthor: Philipp Millius >> (Fl▒chtlingsrat THR e.V.) >> Mar 18 10:06:50 2019 (23745) lowercase value: >> 'abcdefghijklmnopqrstuvwxyz\xb5\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' > which is iso-8859-1 encoding of > 'abcdefghijklmnopqrstuvwxyzµßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ' and which > is the problem. > > I don't know why in testing string.lowercase does not return the above > string, but in the Python that's running ArchRunner, and presumably the > rest of mailman, string.lowercase has that value. This is what needs to > be fixed. The first step is possibly to do > > locate string.py There is just one /usr/lib64/python2.6/string.py yielding lowercase = 'abcdefghijklmnopqrstuvwxyz'. > And then look at all the ones you find to see which defines lowercase > that way and then figure out why the Python that runs Mailman uses that > one rather that the ones you see in testing. > > Also, > > which -a python points to /usr/bin/python as it is supposed to. > > may help find things. As your fix makes for the mails being archived maybe we should leave it at that. Thank you very much anyway for your patient helping me. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Fwd: Uncaught runner exception
Mit herzlichem Dank und freundlichen Grüßen! Lothar Schilling Förderverein Pro Asyl e.V. Postfach 160624 60069 Frankfurt Tel. 069 242314-40 Am 15.03.2019 um 17:25 schrieb Mark Sapiro: > On 3/15/19 4:27 AM, Lothar Schilling wrote: >> Am 14.03.2019 um 18:55 schrieb Mark Sapiro: >>> On 3/14/19 2:39 AM, Lothar Schilling wrote: > ... >>>> string >>>> > ... >> Sadly not: >> >> import string >> string.lowercase >> 'abcdefghijklmnopqrstuvwxyz' >> string >> > > This is interesting. Here you get > and before (above) you got '/usr/lib64/python2.6/string.pyc'>. > > I wonder why the difference. > > In the message at > <https://mail.python.org/pipermail/mailman-users/2019-March/084243.html> > I suggested this patch > > === modified file 'Mailman/Archiver/pipermail.py' > --- Mailman/Archiver/pipermail.py 2018-05-03 21:23:47 + > +++ Mailman/Archiver/pipermail.py 2019-03-02 04:51:23 + > @@ -60,9 +60,12 @@ > else: > # Mixed case; assume that small parts of the last name will be > # in lowercase, and check them against the list. > -while i>0 and (L[i-1][0] in lowercase or > - L[i-1].lower() in smallNameParts): > -i = i - 1 > +try: > +while i>0 and (L[i-1][0] in lowercase or > + L[i-1].lower() in smallNameParts): > +i = i - 1 > +except: > +syslog('error', 'Exception in fixAuthor: %s', author) > author = SPACE.join(L[-1:] + L[i:-1]) + ', ' + SPACE.join(L[:i]) > return author > > > > I assume you still have that installed. In any case, if you add a second > syslog line after the first > > syslog('error', 'lowercase value: %r', lowercase) > > What do you then get in Mailman's error log > > Exception in fixAuthor: D▒rthe Hinz - Fl▒chtlingsrat Nds. Mar 18 08:33:51 2019 (23745) lowercase value: 'abcdefghijklmnopqrstuvwxyz\xb5\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' Mar 18 10:06:50 2019 (23745) Exception in fixAuthor: Philipp Millius (Fl▒chtlingsrat THR e.V.) Mar 18 10:06:50 2019 (23745) lowercase value: 'abcdefghijklmnopqrstuvwxyz\xb5\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff' -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Fwd: Uncaught runner exception
Am 14.03.2019 um 18:55 schrieb Mark Sapiro: > On 3/14/19 2:39 AM, Lothar Schilling wrote: >> Am 12.03.2019 um 23:47 schrieb Mark Sapiro: >>> What is the path to the Python interpreter that Mailman is using? >> /usr/bin/python >> /usr/bin/python -V: Python 2.6.6 > > What is the python command in the result of > > ps -fwwA|grep ArchRunner /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s > >>> What do you get if in that Python you do >>> >>> import string >>> lowercase > As noted in other replies, this should have been 'string.lowercase'. > > >>> string > ... >> string >> > > Get the python command from the 'ps' result above. run that full path > and then do > >>>> import string >>>> string.lowercase >>>> string > I am expecting (hoping actually because it's the only explanation I can > see) that the result of 'string.lowercase' will be longer than 26 > characters and will contain non-ascii. Then the result of 'string' will > point to the string.py(c) that this comes from. Sadly not: import string string.lowercase 'abcdefghijklmnopqrstuvwxyz' string -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Fwd: Uncaught runner exception
Am 12.03.2019 um 23:47 schrieb Mark Sapiro: > On 3/12/19 2:37 AM, Lothar Schilling wrote: >> An example of a complete error message: >> >> Mar 11 17:02:22 2019 (15400) SHUNTING: >> 1552320069.504329+15fb4f71a5404fd1af3dafbe2780486d5b474cc7 >> Mar 11 17:22:37 2019 (15400) Uncaught runner exception: 'ascii' codec >> can't decode byte 0xb5 in position 26: ordinal not in range(128) >> Mar 11 17:22:37 2019 (15400) Traceback (most recent call last): >> File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop >> self._onefile(msg, msgdata) >> File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile >> keepqueued = self._dispose(mlist, msg, msgdata) >> File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose >> mlist.ArchiveMail(msg) >> File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in >> ArchiveMail >> h.processUnixMailbox(f) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 602, in >> processUnixMailbox >> self.add_article(a) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 646, in >> add_article >> author = fixAuthor(article.decoded['author']) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in >> fixAuthor >> while i>0 and (L[i-1][0] in lowercase or >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xb5 in position 26: >> ordinal not in range(128) > Again, this is the code trying to "Canonicalize a name into Last, First > format". The complete statement is > > while i>0 and (L[i-1][0] in lowercase or >L[i-1].lower() in smallNameParts): > i = i - 1 > > smallNameParts is all lower case ascii, i.e. > > = ['van', 'von', 'der', 'de'] > > and L is the list returned by > > L = author.split() > > and none of the shunted .pck files you provided me have any name pieces > longer than 14 bytes and only 2 of the 5 even have any non-ascii, but > the decode error refers to byte 0xb5 in position 26 (B5 in iso-8859-1 is µ). > > I know you said in your original post > >> I am using Python 2.6.6 on a Centos 6 system. The output of >> /usr/lib64/python >>> import string >>> string.lowercase is >> 'abcdefghijklmnopqrstuvwxyz' (as it is supposed to be). > but see below. > >> The problem is site-wide, not just for one or two lists. Anyway, there >> are no altered templates or messages for a specific list. The only files >> deviating from the default configuration are: >> Defaults.py >> mm_cfg.py >> >> Of course Defaults.py has >> DEFAULT_SERVER_LANGUAGE = 'en'. >> >> So in mm_cfg.py I added >> DEFAULT_SERVER_LANGUAGE = 'de' >> add_language('en', 'English (USA)', 'iso-8859-1', 'ltr') >> >> But I cannot see how this would cause the problem. > Neither can I, but I tried again with those settings just to be sure, > and got no issues. > > All of this leads me to conclude that the issue must be non-ascii > following the 'z' in string.lowercase. > > What is the path to the Python interpreter that Mailman is using? /usr/bin/python /usr/bin/python -V: Python 2.6.6 > What do you get if in that Python you do > > import string > lowercase > string > > For example, I get > >>>> import string >>>> lowercase > 'abcdefghijklmnopqrstuvwxyz' >>>> string > > import string lowercase Traceback (most recent call last): File "", line 1, in NameError: name 'lowercase' is not defined string -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Uncaught runner exception
> On 3/8/19 8:17 AM, Mark Sapiro wrote: >> On 3/8/19 12:05 AM, Lothar Schilling wrote: >>> Am 07.03.2019 um 23:54 schrieb Mark Sapiro: >>>> I would still like to find the underlying issue. I have been unable to >>>> duplicate it locally. I would appreciate it if before you run >>>> bin/unshunt, you could copy a few of the .pck files aside and send them >>>> to me off-list. >>> Will send some .pck files to your private address. Some may hold content >>> which is not meant to be publicly available, they are for your eyes only :-) >> >> Thank you. I will treat them confidentially. > > I have set up a test wherein I symlinked the qfiles path from the > metadata in the shunted .pck files to the qfiles directory in the test > installation, created the 5 lists that the shunted messages were > destined to with preferred language = German and unshunted the 5 messages. > > The test Mailman is the head of the > https://code.launchpad.net/~mailman-coders/mailman/2.1 branch, but this > differs from the 2.1.29 release in minor ways, none of which have to do > with archiving. > > Each of the unshunted messages was properly archived with no exceptions > being thrown and qfiles/shunt/ was empty after unshunting. > > I then removed the 5 lists and started over, this time with > > ARCHIVER_OBSCURES_EMAILADDRS = No > > which is the only setting I see offhand that might affect this, but the > result was the same, no error. > > Also, I note that some of the shunted messages had no non-ascii in From: > > I don't know what's going on in your installation, but I can't duplicate > the issue. It may have something to do with list configuration or your > German message catalog or templates if you have any changes in that area > (i.e. any templates/site/de/* templates/HOST_NAME/de/* or > templates/lists/LIST_NAME/de/* files not properly encoded as > iso-8859-1). It could also be something in the > archives/private/LIST_NAME/database/* files, although this would > probably only affect the period in which you upgraded. > > In order to go any further with this, I'd need to see complete error and > traceback information from Mailman's error log for the recently shunted > messages. > An example of a complete error message: Mar 11 17:02:22 2019 (15400) SHUNTING: 1552320069.504329+15fb4f71a5404fd1af3dafbe2780486d5b474cc7 Mar 11 17:22:37 2019 (15400) Uncaught runner exception: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128) Mar 11 17:22:37 2019 (15400) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose mlist.ArchiveMail(msg) File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in ArchiveMail h.processUnixMailbox(f) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 602, in processUnixMailbox self.add_article(a) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 646, in add_article author = fixAuthor(article.decoded['author']) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or UnicodeDecodeError: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128) The problem is site-wide, not just for one or two lists. Anyway, there are no altered templates or messages for a specific list. The only files deviating from the default configuration are: Defaults.py mm_cfg.py Of course Defaults.py has DEFAULT_SERVER_LANGUAGE = 'en'. So in mm_cfg.py I added DEFAULT_SERVER_LANGUAGE = 'de' add_language('en', 'English (USA)', 'iso-8859-1', 'ltr') But I cannot see how this would cause the problem. -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Re: [Mailman-Users] Uncaught runner exception
Thank you and sorry for my belated answer. Was out of office for a few days, carneval season in Germany... Am 02.03.2019 um 23:20 schrieb Mark Sapiro: > On 3/1/19 9:00 PM, Mark Sapiro wrote: >> To try to diagnose this further, you could try: >> >> === modified file 'Mailman/Archiver/pipermail.py' >> --- Mailman/Archiver/pipermail.py2018-05-03 21:23:47 + >> +++ Mailman/Archiver/pipermail.py2019-03-02 04:51:23 + >> @@ -60,9 +60,12 @@ >> else: >> # Mixed case; assume that small parts of the last name will be >> # in lowercase, and check them against the list. >> -while i>0 and (L[i-1][0] in lowercase or >> - L[i-1].lower() in smallNameParts): >> -i = i - 1 >> +try: >> +while i>0 and (L[i-1][0] in lowercase or >> + L[i-1].lower() in smallNameParts): >> +i = i - 1 >> +except: >> +syslog('error', 'Exception in fixAuthor: %s', author) >> author = SPACE.join(L[-1:] + L[i:-1]) + ', ' + SPACE.join(L[:i]) >> return author >> >> >> and see what gets logged in Mailman's error log and what the archived >> message looks like >> Following your instruction errors get now logged like this (where x is lower case, X upper case alphabet). Mar 06 10:36:56 2019 (16870) Exception in fixAuthor: Kxx Wxxx Mar 06 10:36:56 2019 (16870) Exception in fixAuthor: NXX - Fx▒xxx XX x.X. Mar 06 10:36:57 2019 (16870) Exception in fixAuthor: Fx▒ Tx▒xxx x.X. Mar 06 10:36:57 2019 (16870) Exception in fixAuthor: RXX x.X. The missing characters ▒ are German "Umlaute". The messages have now been archived and look ok, not mutilated anyhow. But of course they are still stuck in qfiles/shunt > Also, in your original post you posted > > >> Feb 28 12:29:02 2019 (3123) Uncaught runner exception: 'ascii' codec >> can't decode byte 0xb5 in position 26: ordinal not in range(128) >> Feb 28 12:29:02 2019 (3123) Traceback (most recent call last): >> File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop >> self._onefile(msg, msgdata) >> File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile >> keepqueued = self._dispose(mlist, msg, msgdata) >> File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose >> mlist.ArchiveMail(msg) >> File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in >> ArchiveMail >> h.processUnixMailbox(f) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 596, in >> processUnixMailbox >> self.add_article(a) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 640, in >> add_article >> author = fixAuthor(article.decoded['author']) >> File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in >> fixAuthor >> while i>0 and (L[i-1][0] in lowercase or [error message stops right >> here] > Following that in the log there should be two more lines, one describing > the UnicodeDecodeError exception and one referring to the message being > shunted and pointing to a qfiles/shunt/*.pck file. You can see the > contents of that file with either Mailman's bin/show_qfiles or > bin/dumpdb commands and the From: header in the message may give more info. > As for the "From: headers", some appear distorted, some don't: From: =?UTF-8?Q?L_M=c3=bc?= From: Kxx W -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org
[Mailman-Users] Uncaught runner exception
Hi everybody, a few weeks ago I upgraded from 2.1.16 (as far as I can remember...) to 2.1.29. Everything seemed to work fine at first. But then I found out that a lot of posts - actually far more than half of them - aren't archived any longer. What logging the errors tells me is this: Feb 28 12:29:02 2019 (3123) Uncaught runner exception: 'ascii' codec can't decode byte 0xb5 in position 26: ordinal not in range(128) Feb 28 12:29:02 2019 (3123) Traceback (most recent call last): File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop self._onefile(msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 77, in _dispose mlist.ArchiveMail(msg) File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 216, in ArchiveMail h.processUnixMailbox(f) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 596, in processUnixMailbox self.add_article(a) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 640, in add_article author = fixAuthor(article.decoded['author']) File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor while i>0 and (L[i-1][0] in lowercase or [error message stops right here] As I read in a previous thread the reason for this may be non-ascii compliant characters in the post, especially the "from:"-line. But why would Python or Mailman now all of a sudden use ASCII instead of UTF-8 in the first place? And if so: How can I change that behaviour? I am using Python 2.6.6 on a Centos 6 system. The output of /usr/lib64/python >>> import string >>> string.lowercase is 'abcdefghijklmnopqrstuvwxyz' (as it is supposed to be). Any help would be appreciated. Thank you Lothar Schilling -- Mailman-Users mailing list Mailman-Users@python.org https://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: http://wiki.list.org/x/QIA9 Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-users/archive%40jab.org