Mike, many thanks for your (as always) very helpful response - I added the 1 liner to mm_cfg.py to increase the threads to 16. Now I am observing (via memory trend graphs) an acceleration of what looks like a memory leak - maybe from python - currently at 2.4
I am compiling the latest 2.5.2 to see if that helps - for now the workaround is to restart mailman occasionally. (and yes the spamassassin checks are the source of the 4-10 second delay - now those happen in parallel x16 - so no spikes in the backlog...) Thanks again On 6/20/08 9:01 AM, "Mark Sapiro" <[EMAIL PROTECTED]> wrote: > Fletcher Cocquyt wrote: > >> Hi, I am observing periods of qfiles/in backlogs in the 400-600 message >> count range that take 1-2hours to clear with the standard Mailman 2.1.9 + >> Spamassassin (the vette log shows these messages process in an avg of ~10 >> seconds each) > > > Is Spamassassin invoked from Mailman or from the MTA before Mailman? If > this plain Mailman, 10 seconds is a hugely long time to process a > single post through IncomingRunner. > > If you have some Spamassassin interface like > <http://sourceforge.net/tracker/index.php?func=detail&aid=640518&group_id=103&; > atid=300103> > that calls spamd from a Mailman handler, you might consider moving > Spamassassin ahead of Mailman and using something like > <http://sourceforge.net/tracker/index.php?func=detail&aid=840426&group_id=103&; > atid=300103> > or just header_filter_rules instead. > > >> Is there an easy way to parallelize what looks like a single serialized >> Mailman queue? >> I see some posts re: multi-slice but nothing definitive > > > See the section of Defaults.py headed with > > ##### > # Qrunner defaults > ##### > > In order to run multiple, parallel IncomingRunner processes, you can > either copy the entire QRUNNERS definition from Defaults.py to > mm_cfg.py > and change > > ('IncomingRunner', 1), # posts from the outside world > > to > > ('IncomingRunner', 4), # posts from the outside world > > > which says run 4 IncomingRunner processes, or you can just add > something like > > QRUNNERS[QRUNNERS.index(('IncomingRunner',1))] = ('IncomingRunner',4) > > to mm_cfg.py. You can use any power of two for the number. > > >> I would also like the option of working this into an overall loadbalancing >> scheme where I have multiple smtp nodes behind an F5 loadbalancer and the >> nodes share an NFS backend... > > > The following search will return some information. > > <http://www.google.com/search?q=site%3Amail.python.org++inurl%3Amailman++%22lo > ad+balancing%22> -- Fletcher Cocquyt Senior Systems Administrator Information Resources and Technology (IRT) Stanford University School of Medicine Email: [EMAIL PROTECTED] Phone: (650) 724-7485 ------------------------------------------------------ Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
