JC, Thanks for the suggestion and the link. This is what I'm seeing with my single list of 140,000 subs
(1) MTA performance is not a bottleneck OutgoingRunner is sending to Postfix at between 15-30 emails /sec. Since the mail queue never grows beyond 1000 (from failed deliveries) I'm assuming that Mailman (or the I/O to /var/log/maillog) is the bottleneck limiting performance. In any case I'm more than satisfied with 900-1800 emails/minute. (2) Excessive Memory usage BounceRunner, OutgoingRunner, CommandRunner, ArchRunner, IncomingRunner and VirginRunner are EACH using 120-135 Megabytes of Memory. That's over 700Mbytes RAM just to load these processes (and periodically trigger swap on my 1.5GB 2GHZ P4). I've been able to track the memory usage to the use of a Python dictionary to hold the mailing list (the extra memory required for the hash space probably results in 500% memory overhead compared to a Python list structure). (3) Excessive Processing Time BounceRunner was processing over 80 bounces per minute peak with a list size of 48,000 subs. That went to 40 bounces per minute peak at 89,000 subs and 25 per minute at 138,000. I believe this is due to the CPU time and disk I/O required to write out 17 megabytes of data for every bounce (at 142,000 subs). CommandRunner is showing a similar drop in performance probably due to the same reasons If there is another approach I should take PLEASE PLEASE let me know. But from what I'm seeing, short of re-writing a lot of code, splitting the list is my only hope of reducing both the memory usage and the processing time. John ----- Original Message ----- From: "JC Dill" <[EMAIL PROTECTED]> To: "Links at Momsview" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Wednesday, February 12, 2003 3:36 PM Subject: Re: [Mailman-Users] Umbrella List Spam Protection > Links at Momsview wrote: > > I currently have a VERY large ( >100k) announce only list (always > > moderated) running mailman 2.1 under Postfix. > > > > For performance reasons I'm considering splitting this list into 26 sublists > > based on the first letter of each subscriber email. > > I would then use an umbrella list to send to all of the sublists at once. > > > > As I understand it, unless I want to separately approve the message for the > > 26 sublists I need to allow at least one email address to post unmoderated > > on ALL of the sublists. This is very scary since spammers currently try to > > post on my current large list almost daily. > > Before you go down this road, be sure you have done everything possible > in terms of selecting and configuring your outgoing mail server to > handle the volume. I suspect that handling the outgoing volume will be > easier and more effective when you direct your attention towards your > mail server, than setting up sublists on the list server. > > FAQ entry 1.15 has more on this topic, and directs you to related topics: > > <http://www.python.org/cgi-bin/faqw-mm.py?req=all#1.15> > > HTH > > jc > > > ------------------------------------------------------ Mailman-Users mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ This message was sent to: archive@jab.org Unsubscribe or change your options at http://mail.python.org/mailman/options/mailman-users/archive%40jab.org