Re: [Mailman-Users] Python process size grows 30x in 8 hours (memory

Wed, 02 Jul 2008 20:56:28 -0700 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Jul 2, 2008, at 11:15 PM, Mark Sapiro wrote:
Yes, I know. I was just about to resend. It is attached here. The MUA I 
used to send the previous message gives any attachment without an
extension Content-Type: application/octet-stream, so the list's content 
filtering removed it.


Ah, np.


|> It does the following compared to the normal script.
|
|> The normal script reads the message from the pipe from the MTA and
|> queues it in the 'in' queue for processing by an IncomingRunner. This |> script receives the message and instead queues it in the 'bad' queue. 
|> It then looks at the size of the 'bad' queue entry (a Python pickle
|> that will be just slightly larger than the message text). If the size 
|> is less than MAXSIZE bytes (a parameter near the beginning of the
|> script, currently set to 1000000, but which you can change as you
|> desire), it moves the queue entry from the 'bad' queue to the 'in'
|> queue for processing.
|
| I'm not sure 'bad' should be used.  Perhaps a separate queue called
| 'raw'?  It is nice that files > MAXSIZE need only be left in 'bad'.
If we're going to do something like this going forward, we can certainly 
change the queue. For this 'debug' effort, I wanted to keep it simple
and use an existing mm_cfg queue name.
Excellent point. A couple of very minor comments on the file, but other than that, it looks great. (I know you copied this from the original file, but still I can't resist. ;)
# Copyright (C) 1998,1999,2000,2001,2002 by the Free Software Foundation, Inc. 

1998-2008



#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 

"""Accept posts to a list and handle them properly.
The main advertised address for a list should be filtered to this program, through the mail wrapper. E.g. for list [EMAIL PROTECTED]', the `test' 
alias would deliver to this script.
Stdin is the mail message, and argv[1] is the name of the target mailing list. 

"""

import os
import sys

import paths
from Mailman import mm_cfg
from Mailman import Utils
from Mailman.i18n import _
from Mailman.Queue.sbcache import get_switchboard
from Mailman.Logging.Utils import LogStdErr

LogStdErr("error", "post")

MAXSIZE = 1000000



def main():
# TBD: If you've configured your list or aliases so poorly as to get # either of these first two errors, there's little that can be done to # save your messages. They will be lost. Minimal testing of new lists 
   # should avoid either of these problems.
   try:
       listname = sys.argv[1]
   except IndexError:
       print >> sys.stderr, _('post script got no listname.')
       sys.exit(1)
   # Make sure the list exists
   if not Utils.list_exists(listname):
print >> sys.stderr, _('post script, list not found: % (listname)s') 
       sys.exit(1)
# Immediately queue the message for the incoming qrunner to process. The # advantage to this approach is that messages should never get lost -- # some MTAs have a hard limit to the time a filter prog can run. Postfix # is a good example; if the limit is hit, the proc is SIGKILL'd giving us 
   # no chance to save the message.
   bdq = get_switchboard(mm_cfg.BADQUEUE_DIR)
   filebase = bdq.enqueue(sys.stdin.read(),
                          listname=listname,
                          tolist=1, _plaintext=1)


Should probably use True there instead of 1.


   frompath= os.path.join(mm_cfg.BADQUEUE_DIR, filebase + '.pck')
   topath= os.path.join(mm_cfg.INQUEUE_DIR, filebase + '.pck')


Space in front of the =




   if os.stat(frompath).st_size < MAXSIZE:
       os.rename(frompath,topath)


Space after the comma.



if __name__ == '__main__':
   main()


- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iEYEARECAAYFAkhsTbQACgkQ2YZpQepbvXGGigCfe+w4Ynz/qvFEp6VmurbySv42
b6cAoJa9wuSaql8dLUo8/VXT/Sxiu9pW
=Ywsy
-----END PGP SIGNATURE-----
------------------------------------------------------
Mailman-Users mailing list
[email protected]
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Reply via email to