[Mailman-Developers] big list

[Fil]

I've tried and installed 50,000 subscribers in one list, to send them
VERP messages (this is weird, but they are boucners from some other mailing
list, managed with sympa).

It seems to take forever to send a message : 'top' gives me

  PID USER PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
  617 mailman   18   0 49860  44M  2576 R85.5  4.3  41:05 qrunner
/home/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s

That's 41 minutes and not one message has gone out yet.

Is that normal ?  (almost current cvs)

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

> It seems to take forever to send a message : 'top' gives me
> 
>   PID USER PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
>   617 mailman   18   0 49860  44M  2576 R85.5  4.3  41:05 qrunner
> /home/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
> 
> That's 41 minutes and not one message has gone out yet.

Well! Now it has eaten up all my memory (1 Gb), and it's swapping like crazy :

182 processes: 175 sleeping, 3 running, 4 zombie, 0 stopped
CPU0 states:  5.4% user, 93.3% system,  0.0% nice,  0.5% idle
CPU1 states: 10.0% user, 84.4% system,  0.0% nice,  5.1% idle
Mem:  1029592K av, 1015496K used,   14096K free,   0K shrd,   21468K buff
Swap:  128484K av,  128484K used,   0K free  371260K cached



-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> That's 41 minutes and not one message has gone out yet.

F> Is that normal ?  (almost current cvs)

Probably not .

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> Well! Now it has eaten up all my memory (1 Gb), and it's
F> swapping like crazy :

This is just a guess (and I'll try to look at it in more detail later
today), that the act of turning a message into 50k VERP-able copies
is the problem.  I.e. the loop in Personalize.py.  Or it might be
IncomingRunner trying to dequeue 50k messages (although that should
only happen one at a time).

Do you see anything in qfiles/in?

I'll try to create a bunch of dummy addrs and send a personalized
message through the system.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> Do you see anything in qfiles/in?

Indeed:

1015601891.96+b81bcd9e6a652f432f5fe592826e5b04161c9675.pck
1015601891.96+b81f4b00e4d37f128f37df4521180e3fa9f29401.db
1015601891.96+b81f4b00e4d37f128f37df4521180e3fa9f29401.pck
1015601891.96+b821bcd0fd18a1a290e592c08f269006f04eb573.db
1015601891.96+b821bcd0fd18a1a290e592c08f269006f04eb573.pck
1015601891.96+b82367106668726b6598101227c9215d58b87f37.db
1015601891.96+b82367106668726b6598101227c9215d58b87f37.pck
1015601891.96+b827213177f651f88b5a37adaf86bcc62e26658c.db
1015601891.96+b827213177f651f88b5a37adaf86bcc62e26658c.pck
1015601891.96+b827ed7dad14baa38a54536829349efb1c9ad15e.db
1015601891.96+b827ed7dad14baa38a54536829349efb1c9ad15e.pck
1015601891.96+b8286e4aecc0c39bae39f488d004893c09d63b38.db
1015601891.96+b8286e4aecc0c39bae39f488d004893c09d63b38.pck
1015601891.96+b82c5c5e0c9c893d651b07f9bba3347239f33e50.db
1015601891.96+b82c5c5e0c9c893d651b07f9bba3347239f33e50.pck

thousands... will I also run out of disk space ?

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Fil <[EMAIL PROTECTED]> :
> Well! Now it has eaten up all my memory (1 Gb), and it's swapping like crazy :

OK, out of disk space now:

/dev/sda6 1.8G  1.8G 0 100% /home


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

>> Do you see anything in qfiles/in?

F> Indeed:

| 1015601891.96+b81bcd9e6a652f432f5fe592826e5b04161c9675.pck
| 1015601891.96+b81f4b00e4d37f128f37df4521180e3fa9f29401.db
| 1015601891.96+b81f4b00e4d37f128f37df4521180e3fa9f29401.pck
| 1015601891.96+b821bcd0fd18a1a290e592c08f269006f04eb573.db
| 1015601891.96+b821bcd0fd18a1a290e592c08f269006f04eb573.pck
| 1015601891.96+b82367106668726b6598101227c9215d58b87f37.db
| 1015601891.96+b82367106668726b6598101227c9215d58b87f37.pck
| 1015601891.96+b827213177f651f88b5a37adaf86bcc62e26658c.db
| 1015601891.96+b827213177f651f88b5a37adaf86bcc62e26658c.pck
| 1015601891.96+b827ed7dad14baa38a54536829349efb1c9ad15e.db
| 1015601891.96+b827ed7dad14baa38a54536829349efb1c9ad15e.pck
| 1015601891.96+b8286e4aecc0c39bae39f488d004893c09d63b38.db
| 1015601891.96+b8286e4aecc0c39bae39f488d004893c09d63b38.pck
| 1015601891.96+b82c5c5e0c9c893d651b07f9bba3347239f33e50.db
| 1015601891.96+b82c5c5e0c9c893d651b07f9bba3347239f33e50.pck

F> thousands... will I also run out of disk space ?

When you turn on VERPing Mailman makes a unique copy of each message
for each recipient, so do the math. :)

There might be a way to defer this until SMTP delivery time, in which
case we'd only create the copies in memory when talking to the smtpd.
These copies would get gc'd after delivery, but we'd still have to
play the game that I think Marc brought up about not blasting too many
messages down the same socket connection.

I'll think on this.
-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> F> thousands... will I also run out of disk space ?

Now I have run out of disk space (wrong partition!)


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Carson Gaspar]

--On Friday, March 08, 2002 11:49 AM -0500 "Barry A. Warsaw" 
<[EMAIL PROTECTED]> wrote:


> There might be a way to defer this until SMTP delivery time, in which
> case we'd only create the copies in memory when talking to the smtpd.

Please! Exploding onto slow disk is bad...

Also, why create a copy at all? Why not just do the substitution in 
real-time during the SMTP conversation? i.e., instead of 
buffer->filter->buffer->tcp, just do buffer->filter->tcp. No GC needed.

-- 
Carson


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> >> Do you see anything in qfiles/in?

I have freed some disk space, 'stopped' Mailman with mailmanctl, and
swapping has receded. Now I have:

$ ls qfiles/in/ -1 | wc
 100199  100199 5861674

And the only Mailman process doing alot of work is
619 mailman   12   0 29052 8500  1740 S 6.7  0.8   9:14 qrunner
/home/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Nigel Metheringham]

On Fri, 2002-03-08 at 16:49, Barry A. Warsaw wrote:
> When you turn on VERPing Mailman makes a unique copy of each message
> for each recipient, so do the math. :)

Ugh...

> There might be a way to defer this until SMTP delivery time, in which
> case we'd only create the copies in memory when talking to the smtpd.

Ideally you would make a copy in memory, ship it down the delivery
pipeline, then work on the next copy... really only need a few copies
actually in python memory at once (pretty much just the number of
outgoing delivery streams).

> These copies would get gc'd after delivery, but we'd still have to
> play the game that I think Marc brought up about not blasting too many
> messages down the same socket connection.

Fun :-)

Of course as the MTA catches these its going to make a queue file on
disk for each one unless you can do this funky passing the VERP stuff
down the pipeline stuff thats been mentioned on the list before.

Nigel.
-- 
[ Nigel Metheringham   [EMAIL PROTECTED] ]
[ - Comments in this message are my own and not ITO opinion/policy - ]


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

Not surprising.  Stop Mailman, wax those files, and I'll work out
something better.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Nigel Metheringham <[EMAIL PROTECTED]> :
> Of course as the MTA catches these its going to make a queue file on
> disk for each one unless you can do this funky passing the VERP stuff
> down the pipeline stuff thats been mentioned on the list before.

Yes, but: my MTA's spool disk is rather large, but I had not anticipated
that for Mailman's spool, which is on a smaller partition...

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "NM" == Nigel Metheringham
> <[EMAIL PROTECTED]> writes:

NM> Ideally you would make a copy in memory, ship it down the
NM> delivery pipeline, then work on the next copy... really only
NM> need a few copies actually in python memory at once (pretty
NM> much just the number of outgoing delivery streams).

That's what I have in mind.

NM> Of course as the MTA catches these its going to make a queue
NM> file on disk for each one unless you can do this funky passing
NM> the VERP stuff down the pipeline stuff thats been mentioned on
NM> the list before.

It would be really cool if we could get a bunch of MTA authors
together (I only care about the open source ones ) to define a
protocol for letting the MTA doing the stitching.  I think Postfix,
and probably Exim support a way to do this for the envelope sender,
but the really interesting bits happen when the body of the message
can be personalized.  The outgoing MTA's the most efficient place to
do this, but you have to get it the information it needs to chew on.

I know there's been some talk about subsuming the outgoing
functionality into MM, but I see such a bulk mailer/stitcher as a
separate project, that could be integrated into MM through a new
DELIVERY_MODULE.  I don't expect to have time to work on that myself,
so there's an opportunity for someone who wants to contribute.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> Not surprising.  Stop Mailman, wax those files, and I'll work out
> something better.

If it can be of interest : I had to 'kill -9' the qrunner that was running
in circles, then just restarted the whole thing, and my personalized
messages are going out at full speed now.

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Jay R. Ashworth]

On Fri, Mar 08, 2002 at 11:49:39AM -0500, Barry A. Warsaw wrote:
> When you turn on VERPing Mailman makes a unique copy of each message
> for each recipient, so do the math. :)
> 
> There might be a way to defer this until SMTP delivery time, in which
> case we'd only create the copies in memory when talking to the smtpd.
> These copies would get gc'd after delivery, but we'd still have to
> play the game that I think Marc brought up about not blasting too many
> messages down the same socket connection.

This is the 'range' dilemma.  We need an 'xrange' solution.

Cheers,
-- jra
-- 
Jay R. Ashworth[EMAIL PROTECTED]
Member of the Technical Staff Baylink RFC 2100
The Suncoast Freenet The Things I Think
Tampa Bay, Floridahttp://baylink.pitas.com +1 727 647 1274

   "If you don't have a dream; how're you gonna have a dream come true?"
 -- Captain Sensible, The Damned (from South Pacific's "Happy Talk")

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[John W Baxter]

At 12:15 -0500 3/8/2002, Barry A. Warsaw wrote:
>It would be really cool if we could get a bunch of MTA authors
>together (I only care about the open source ones ) to define a
>protocol for letting the MTA doing the stitching.  I think Postfix,
>and probably Exim support a way to do this for the envelope sender,
>but the really interesting bits happen when the body of the message
>can be personalized.  The outgoing MTA's the most efficient place to
>do this, but you have to get it the information it needs to chew on.
>
>I know there's been some talk about subsuming the outgoing
>functionality into MM, but I see such a bulk mailer/stitcher as a
>separate project, that could be integrated into MM through a new
>DELIVERY_MODULE.  I don't expect to have time to work on that myself,
>so there's an opportunity for someone who wants to contribute.

I suspect you'll get a lot of resistance from MTA authors to the idea of an
MTA (mail transfer agent) messing with the bodies of the messages.  That's
MUA work.

This is more for a purpose-built tool which knows how to send messages to
the world's MTAs and how to prepare messages, but does not know anything
about receiving messages from general local senders or from the world.  In
other words, a potentially useful different product.

Aside:  how far we've come from the old mainframe LISTSERV, the network of
which carefully sent one copy along with a list to addresses to various
neighbors "near" the addressees for further distribution.  So quite likely,
only one copy crossed the Atlantic...possibly one one went from Chicago to
Florida, etc etc.


As to VERPing a big list with the *current* tool:  don't (if it hurts,
don't do it, and I doubt whether throwing enough more hardware at the
50,000 address list is feasible).  VERP a bunch of little lists roughly
sequentially.  The originator of the thread might try the first 1,000 and
see whether 50 1000 member lists will work.  *Particularly* when nearly
50,000 bounces are expected (the list is bouncers from other lists, per the
initial message).

  --John
-- 
John Baxter   [EMAIL PROTECTED]  Port Ludlow, WA, USA

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ John W Baxter <[EMAIL PROTECTED]> :
> As to VERPing a big list with the *current* tool:  don't (if it hurts,
> don't do it, and I doubt whether throwing enough more hardware at the
> 50,000 address list is feasible).

Sometimes you want to try and stress things, just to give beta feedback to
Barry. I've shown a problem with computing all VERPed messages before
sending (as opposed to *while sending*), I'm quite happy with that being in
a beta (or alpha) moment before the much awaited MM2.1.

> VERP a bunch of little lists roughly sequentially.  The originator of the
> thread might try the first 1,000 and see whether 50 1000 member lists will
> work.  *Particularly* when nearly 50,000 bounces are expected (the list is
> bouncers from other lists, per the initial message).

What I'm witnessing now is also interesting: these personalized messages go
out one after the other, almost 20 seconds between them. The IncomingRunner
is taking up about 100% of one of the CPU. The BounceRunner taking 30% of
the other. I guess both are trying, for each file, to rescan the database
file? I can't understand why the IncomingRunner should check the database...

In conclusion, it's going to take on lng time expelling all these
files... Too bad I'll have to stop that experiment (or spend several days
with an eye on log files).

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[John W Baxter]

At 23:31 +0100 3/8/2002, Fil wrote:
>Sometimes you want to try and stress things, just to give beta feedback to
>Barry. I've shown a problem with computing all VERPed messages before
>sending (as opposed to *while sending*), I'm quite happy with that being in
>a beta (or alpha) moment before the much awaited MM2.1.

Indeed, and that part of the thing you did was excellent.  (I find it
interesting that a second try worked somewhat well, at least initially.)
If you wanted quick results, the approach turned out not to be right...but
it certainly stressed well.

Cheers!
-- 
John Baxter   [EMAIL PROTECTED]  Port Ludlow, WA, USA

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ John W Baxter <[EMAIL PROTECTED]> :
> Indeed, and that part of the thing you did was excellent.  (I find it
> interesting that a second try worked somewhat well, at least initially.)

No: I said "it's going out fast", but I was confused: messages from other
lists were going out fast. Those messages were already very slow, one every
20 seconds... must be the time needed to lock + read the list.db + unlock.

> If you wanted quick results, the approach turned out not to be right...but
> it certainly stressed well.

:-)

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Dan Mick]

> > "NM" == Nigel Metheringham
> > <[EMAIL PROTECTED]> writes:
> 
> NM> Ideally you would make a copy in memory, ship it down the
> NM> delivery pipeline, then work on the next copy... really only
> NM> need a few copies actually in python memory at once (pretty
> NM> much just the number of outgoing delivery streams).
> 
> That's what I have in mind.
> 
> NM> Of course as the MTA catches these its going to make a queue
> NM> file on disk for each one unless you can do this funky passing
> NM> the VERP stuff down the pipeline stuff thats been mentioned on
> NM> the list before.
> 
> It would be really cool if we could get a bunch of MTA authors
> together (I only care about the open source ones ) to define a
> protocol for letting the MTA doing the stitching.  I think Postfix,
> and probably Exim support a way to do this for the envelope sender,
> but the really interesting bits happen when the body of the message
> can be personalized.  The outgoing MTA's the most efficient place to
> do this, but you have to get it the information it needs to chew on.

BTW, I'm doing this for the From VERPing with Postfix, and it's 
working out really well for me.  The changes are really small
and well-contained, and my bounce processing is *really* working
better now (I never ever have a missed bounce):

Index: Mailman/Handlers/SMTPDirect.py
===
RCS file: /cvsroot/mailman/mailman/Mailman/Handlers/SMTPDirect.py,v
retrieving revision 2.11
diff -u -r2.11 SMTPDirect.py
--- Mailman/Handlers/SMTPDirect.py  16 Dec 2001 06:23:14 -  2.11
+++ Mailman/Handlers/SMTPDirect.py  8 Mar 2002 23:28:20 -
@@ -75,6 +75,9 @@
 # do that here?)
 del msg['sender']
 del msg['errors-to']
+# DJM - do this unconditionally since we're VERPing as below
+del msg['sender']
+del msg['errors-to']
 # Get the flat, plain text of the message
 msgtext = msg.as_string(unixfrom=0)
 # Time to split up the recipient list.  If we're VERPing then each chunk
@@ -270,7 +273,9 @@
 if not getattr(conn, 'sock', 0):
 conn.connect(mm_cfg.SMTPHOST, mm_cfg.SMTPPORT)
 # Send the message
-refused = conn.sendmail(envsender, recips, msgtext)
+# DJM - do Postfix VERPing
+refused = conn.sendmail(envsender, recips, msgtext,\
+  mail_options=["XVERP"])
 except smtplib.SMTPRecipientsRefused, e:
 refused = e.recipients
 # MTA not responding, or other socket problems, or any other kind of
Index: Mailman/Queue/BounceRunner.py
===
RCS file: /cvsroot/mailman/mailman/Mailman/Queue/BounceRunner.py,v
retrieving revision 2.8
diff -u -r2.8 BounceRunner.py
--- Mailman/Queue/BounceRunner.py   7 Mar 2002 16:54:39 -   2.8
+++ Mailman/Queue/BounceRunner.py   8 Mar 2002 23:28:21 -
@@ -111,6 +111,8 @@
 continue  # not a bounce to our list
 # All is good
 addr = '%s@%s' % mo.group('mailbox', 'host')
+   syslog('bounce', 'VERP bounce from %s header for addr %s',
+header, addr)
 except IndexError:
 syslog('error',
"VERP_REGEXP doesn't yield the right match groups: %s",


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

Heh, you're gonna hate me then, because I'm doing some significant
surgery on SMTPDirect.py.  I'm probably going to throw out the
threading stuff because I can't keep it all straight in my head, but
I'm up for resurrecting it in an SMTPThreaded.py module if there's
sufficient interest.

I haven't looked at your patch but I will if/when  I get this
new version working.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> I've tried and installed 50,000 subscribers in one list, to
F> send them VERP messages

The good news: thanks for stressing this part of Mailman.  I /think/
cvs now should perform much better.  Please try it out.  Mailman now
defers the final knitting of the message together with its header and
footer until SMTP delivery time.  That means that personalization and
VERPing are added to an in-memory copy.

I expect this to have two benefits: Mailman itself should be much less
hungry for disk I/O when doing VERP, and it should be nice for memory
footprint as well, as I expect the in-memory message copies to be
sufficently garbage collected away.  Please let me know what you
observe.

No promises that there aren't bugs lurking.

Bad new: I won't release a beta1 today.  I'd like for this code to air
out a bit, plus I didn't get to the last few non-related mods I was
planning to get to.  I'm headed off-line for a few hours now, but I'll
check back in tonight and over the weekend, time permitting.

More bad news: threaded delivery is gone.  Was anybody using this,
really?  Oh well, it was always labeled experimental anyway .

breaking-warsaw's-second-law-ly y'rs,
-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> What I'm witnessing now is also interesting: these personalized
F> messages go out one after the other, almost 20 seconds between
F> them. The IncomingRunner is taking up about 100% of one of the
F> CPU. The BounceRunner taking 30% of the other. I guess both are
F> trying, for each file, to rescan the database file? I can't
F> understand why the IncomingRunner should check the database...

This part /should/ perform better too.  OutgoingRunner usually[1]
doesn't need to lock the list so there's much less contention on the
list lock.  IncomingRunner is doing much less work now too.

-Barry

[1] It only locks it occasionally if it got failures when talking to
the MTA.  It collects these failures and periodically cleans them out,
doing a bounce registration for each.  "Periodically" is defined in
DEAL_WITH_PERMFAILURES_EVERY in OutgoingRunner.py <1 wink>.

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Graham TerMarsch]

On Friday 08 March 2002 15:07, you wrote:
> No: I said "it's going out fast", but I was confused: messages from
> other lists were going out fast. Those messages were already very slow,
> one every 20 seconds... must be the time needed to lock + read the
> list.db + unlock.

I'd expect that this is likely the problem.  I manage a list for a 
customer who has ~180k addresses in Mailman, and after solving MTA 
problems we found that the next biggest bottleneck was the Mailman data 
store.  It generally took 8-10 seconds on a Dual-Athlon-1600Mhz machine to 
process a single bounce message, just about all of which was chewed up in 
doing locking and IO (ok, probably just IO).

The actual size of the "config.db" file was close to 12MB when we started 
having major problems with it, never mind that when slurped in QRunner 
generally ate up 50-70MB of RAM while processing the queue.  With the CPU 
pegged at 95+% usage from QRunner, I'd totally expect that most of our CPU 
time was just spent spinning while slurping in and spitting out the list 
after each and every update was made to the list.

For our particular case, we found that by splitting out the larger list 
into a series of smaller lists (e.g. one for each letter of the alphabet), 
we were able to _substantially_ reduce the overhead involved and got 
processing back down to 5-6msgs/sec again.  I wouldn't necessarily say 
that what we've done is the "be all and end all" answer to this, though, 
it just pushed the problem back to a later date for us, giving us enough 
time to look at other things before it rears its ugly head again as some 
lists grow faster than others ("s" is always a big list).

For point of reference, we're running this on Mailman-2.0.8, using 
Postfix-1.1.3 as the MTA.  We used to have it running on Sendmail-8.12, 
which worked quite well and handled the load without any problems, but had 
a tendency to create MUCH more IO than Postfix does.  Being that I haven't 
yet gotten my dream IO system for this machine, the shift to Postfix was 
necessary to get msgs flying out the door faster.  There are still other 
bottlenecks in the system, but after our updates Mailman wasn't one of the 
major ones on the list any more.

I'll also note that our install has a number of the Bounce handlers 
removed, to make the processing chain shorter.  We ran many megs of debug 
logs spitting out information about which Bounce handler processed a given 
msg, and disabled all of those that were at the bottom of the list.  I 
think all we've got left in there right now is 'Postfix', 'DSN', and 
'Catchall', which catches more than 90% of the bounces we get sent back 
through the system.  Isn't perfect for catching and processing all of the 
bounces, but the reduction in pipeline length for the bounce processing 
made a noticable difference in performance.  This, however, I expect is 
related to the performance not of Mailman itself, but of Python in 
general, in the context of the regexes and the MIME libraries that Mailman 
is using.

-- 
Graham TerMarsch
Howling Frog Internet Development, Inc.   http://www.howlingfrog.com

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "GT" == Graham TerMarsch <[EMAIL PROTECTED]> writes:

GT> The actual size of the "config.db" file was close to 12MB when
GT> we started having major problems with it, never mind that when
GT> slurped in QRunner generally ate up 50-70MB of RAM while
GT> processing the queue.  With the CPU pegged at 95+% usage from
GT> QRunner, I'd totally expect that most of our CPU time was just
GT> spent spinning while slurping in and spitting out the list
GT> after each and every update was made to the list.

I suspect that MM2.1 will perform better here for several reasons.
First, the OutgoingRunner almost never needs to acquire the lock to
get the message out the door.  If all your addresses are non-local and
you're not doing synchronous delivery, then you'll never get failures
during the SMTP dialog, so it'll never acquire the list lock.

Second, you can tune your BounceRunner to only process bounces "once
in a while".  If you really don't need to process them as soon as they
come in, just crank up the sleep interval between sweeps of the bounce
queue.

Third, if you decide to go the VERP route, you'll almost never need to
run through the bounce pipeline, since the bouncing address can be
unambiguously culled from the envelope sender.

MM2.1 inherits MM2.0's data store.  That's not going to change until
MM3.0, but I agree that something more efficient is sorely needed to
avoid list lock contention.  I personally think ZODB is a great way to
go, and may be the default, but the system /will/ be architected so
that you could use something else underneath if you want.

GT> I'll also note that our install has a number of the Bounce
GT> handlers removed, to make the processing chain shorter.
[...]
GT> This, however, I expect is related to the performance not of
GT> Mailman itself, but of Python in general, in the context of
GT> the regexes and the MIME libraries that Mailman is using.

I'll bet most of that is simply spent importing all the bounce
pipeline modules.  Try strace'ing Python sometime and doing some
imports.  Fear the number of stats going on. :)

Again, MM2.1 will be much better because we've switched to a
long-running daemon design.  The bounce qrunner will warm up after the
initial set of imports, so if that /is/ the culprit, you'll see better
performance there as well.

Cheers,
-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Ron Jarrell]

At 01:46 PM 3/8/02 -0800, John W Baxter wrote:
>Aside:  how far we've come from the old mainframe LISTSERV, the network of
>which carefully sent one copy along with a list to addresses to various
>neighbors "near" the addressees for further distribution.  So quite likely,
>only one copy crossed the Atlantic...possibly one one went from Chicago to
>Florida, etc etc.

Still does, to some extent.  Listserv passes jobs to other listservs when it
can figure out that's useful.  Also, part of the license fee for running listserv
is access to be able to distribute big jobs to lsofts large server, and let them
deal with delivering it for you.


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> The good news: thanks for stressing this part of Mailman.  I /think/
> cvs now should perform much better.  Please try it out.  Mailman now

I'm trying it now: it's emptying the queue a bit faster, haven't tried to
send a new message yet (I'll have to clear the queue intelligently in order
to make sure no message intended for other lists are deleted). I'll keep you
posted.

However the update gives me this bug on
http://listes.rezo.net/mailman/listinfo/

(if you click it when you wake up, and it's gone, it means I've found
out...)

Traceback (most recent call last):
  File "/home/mailman/scripts/driver", line 82, in run_main
main()
  File "/home/mailman/Mailman/Cgi/listinfo.py", line 42, in main
listinfo_overview()
  File "/home/mailman/Mailman/Cgi/listinfo.py", line 85, in
listinfo_overview
mlist = MailList.MailList(name, lock=0)
  File "/home/mailman/Mailman/MailList.py", line 102, in __init__
self.Load()
  File "/home/mailman/Mailman/MailList.py", line 541, in Load
self.CheckVersion(dict)
  File "/home/mailman/Mailman/MailList.py", line 558, in CheckVersion
self.Lock()
  File "/home/mailman/Mailman/MailList.py", line 148, in Lock
self.Load()
  File "/home/mailman/Mailman/MailList.py", line 541, in Load
self.CheckVersion(dict)
  File "/home/mailman/Mailman/MailList.py", line 561, in CheckVersion
Update(self, stored_state)
  File "/home/mailman/Mailman/versions.py", line 53, in Update
CanonicalizeUserOptions(l)
  File "/home/mailman/Mailman/versions.py", line 349, in
CanonicalizeUserOptions
if l.getMemberOption(k, mm_cfg.DisableDelivery):
  File "/home/mailman/Mailman/OldStyleMemberships.py", line 120, in
getMemberOption
self.__assertIsMember(member)
  File "/home/mailman/Mailman/OldStyleMemberships.py", line 113, in
__assertIsMember
raise Errors.NotAMemberError, member
NotAMemberError: x@x(a real address, of course --Fil)


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Fil <[EMAIL PROTECTED]> :
> @ Barry A. Warsaw <[EMAIL PROTECTED]> :
> > The good news: thanks for stressing this part of Mailman.  I /think/
> > cvs now should perform much better.  Please try it out.  Mailman now

I've tried to send again my 'big list' as personalized messages. The
messages went out fast, but

*First problem:* Although they are personalized (To: address ofg recipient),
the messages sent were not VERPed (from the mail.log I saw only "from
", not "from ").

So I did /etc/init.d/mailman stop, in order to be able to read the list
config: I checked that "personalize" was still On. It was.
And mm_cfg.py contains:
VERP_PERSONALIZED_DELIVERIES = 1
VERP_DELIVERY_INTERVAL = 10
OWNERS_CAN_ENABLE_PERSONALIZATION = 1


*Second problem:* Then I did /etc/init.d/mailman start, but the sending of
this message did not resume.

The message is in qfiles/shunt, but the addresses not yet delivered to are
nowhere to be seen. It seems to mean that, if the computer shuts down while
Mailman is sending a batch of mails, a portion of the recipients will not
receive their copy?

I think the right way to do it would be, alongside the xxx.db and xxx.pck
files in qfiles/shunt/ , to have a file xxx.dest containing all addresses
not yet delived-to (in whatever format), that could be written one every x
seconds to disk, in order to be able to resume sending if the computer has
crashed, the power failed, or some dumb administrator tried to restart
mailman at that moment.


*Third problem:* The "a message awaits your approval" mail is empty
(the Mailman site is configured as French, maybe that's a source of trouble:
I see this strange header :
Content-Type: multipart/mixed; charset="fr";
boundary="===73517001808033933=="
)

-- Fil

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> *First problem:* Although they are personalized (To: address
F> ofg recipient), the messages sent were not VERPed (from the
F> mail.log I saw only "from ", not "from
F> ").

Fixed in cvs.


F> *Second problem:* Then I did /etc/init.d/mailman start, but the
F> sending of this message did not resume.

F> The message is in qfiles/shunt, but the addresses not yet
F> delivered to are nowhere to be seen. It seems to mean that, if
F> the computer shuts down while Mailman is sending a batch of
F> mails, a portion of the recipients will not receive their copy?

F> I think the right way to do it would be, alongside the xxx.db
F> and xxx.pck files in qfiles/shunt/ , to have a file xxx.dest
F> containing all addresses not yet delived-to (in whatever
F> format), that could be written one every x seconds to disk, in
F> order to be able to resume sending if the computer has crashed,
F> the power failed, or some dumb administrator tried to restart
F> mailman at that moment.

Think of shunt as a safety net.  If Mailman has a bug that causes an
uncaught exception to occur while handling the message, it gets
shuffled off to shunt, so that once the bug is fixed, you ought to be
able to move the message back to qfiles/in and have it delivered
again.

I may do two additional things with shunt: first, I should probably
include a key in the metadata to indicate which queue the shunted file
came from, and second, I want to add a bin/unshunt command which will
re-queue shunted messages safely and correctly.

As to your other problem, I'm going to have to think about that.  I
agree that it's not good that if Mailman is shut down that messages
are delivered either twice or not at all.  A solution will likely
require some disk i/o, but the question is, how to do things robustly
without hammering the disk.

F> *Third problem:* The "a message awaits your approval" mail is
F> empty (the Mailman site is configured as French, maybe that's a
F> source of trouble: I see this strange header : Content-Type:
F> multipart/mixed; charset="fr";
F> boundary="===73517001808033933==" )

I'll look into that one too.
-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> F> *Third problem:* The "a message awaits your approval" mail is
> F> empty (the Mailman site is configured as French, maybe that's a
> F> source of trouble: I see this strange header : Content-Type:
> F> multipart/mixed; charset="fr";
> F> boundary="===73517001808033933==" )
> 
> I'll look into that one too.

I think I've got it: the French translation of 
"This is a multi-part message in MIME format.

"

lacks the trailing linefeeds. Hence mutt (at least mutt) does not detect the
part of the multipart message and shows an empty mail.

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Fil <[EMAIL PROTECTED]> :
> @ Barry A. Warsaw <[EMAIL PROTECTED]> :
> > F> *Third problem:* The "a message awaits your approval" mail is
> > F> empty (the Mailman site is configured as French, maybe that's a
> > F> source of trouble: I see this strange header : Content-Type:
> > F> multipart/mixed; charset="fr";
> > F> boundary="===73517001808033933==" )
> > 
> > I'll look into that one too.
> 
> I think I've got it: the French translation of 

Not sure now: I've tried to get the mail through SquirrelMail (a nice php
webmail), and it says:

>>
Body retrieval error. The reason for this is most probably that
the message is malformed. Please help us making future versions
better by submitting this message to the developers knowledgebase!
Submit message
Response: OK
Message: FETCH completed
FETCH line: * OK [PARSE] Ignoring nested encoding of multipart contents 

---
Return-Path: <[EMAIL PROTECTED]>
Received: from miel.brainstorm.fr (localhost [127.0.0.1])
by miel.brainstorm.fr (Postfix) with ESMTP
id 072E51C1AE; Sun, 10 Mar 2002 01:03:55 +0100 (CET)
Received: from miel.brainstorm.fr (localhost [127.0.0.1])
by miel.brainstorm.fr (Postfix) with ESMTP id B1A9E1C1AE
for <[EMAIL PROTECTED]>; Sun, 10 Mar 2002 01:03:53 +0100 (CET)
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: =?iso-8859-1?q?Une_soumission_sur_la_li?=
 =?iso-8859-1?q?ste_spip_=E0_partir_de_mou?=
 =?iso-8859-1?q?stapha=2Ecom=40parisfree=2Ecom?=
 =?iso-8859-1?q?_requiert_une_approbation?=
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Message-ID: <[EMAIL PROTECTED]>
Date: Sun, 10 Mar 2002 01:03:52 +0100
X-BeenThere: [EMAIL PROTECTED]
X-Mailman-Version: 2.1a4+
Precedence: bulk
List-Help: 
List-Post: 
List-Subscribe: ,

List-Id: SPIP : questions/reponses 
List-Unsubscribe: ,

List-Archive: http://listes.rezo.net/archives/spip/
Content-Type: multipart/mixed; charset="fr";
boundary="===021006692614097044=="
Sender: [EMAIL PROTECTED]
Errors-To: [EMAIL PROTECTED]
Content-Length: 3819
Lines: 128

--===021006692614097044==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit


--===021006692614097044==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit


--===021006692614097044==
Content-Type: text/plain; charset="iso-8859-1"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

En tant qu'administrateur de liste, votre autorisation est n=E9cessaire
pour l'envoi du message suivant vers la liste:


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Nick Simicich]

At 02:52 AM 2002-03-09 -0500, Ron Jarrell wrote:
>At 01:46 PM 3/8/02 -0800, John W Baxter wrote:
> >Aside:  how far we've come from the old mainframe LISTSERV, the network of
> >which carefully sent one copy along with a list to addresses to various
> >neighbors "near" the addressees for further distribution.  So quite likely,
> >only one copy crossed the Atlantic...possibly one one went from Chicago to
> >Florida, etc etc.
>
>Still does, to some extent.  Listserv passes jobs to other listservs when it
>can figure out that's useful.  Also, part of the license fee for running 
>listserv
>is access to be able to distribute big jobs to lsofts large server, and 
>let them
>deal with delivering it for you.

Actually, the recent releases say things in their control file like "This 
server has been down since 1999 and it does not look like it will ever be 
brought up again."

And Listserv has gone "open source".  I have decided to migrate away from 
my hand tuned Majordomo 1, and I have been investigating various mailing 
list programs. (Mailman is one of them -- and I am still unsure what 
version I should install --- since this is a new attempt at installation, 
should I install the beta, or should I stick with the stable version? -- I 
really want custom footers...)  I got a copy of the open source listserv, 
and I have compiled it, but so far as I can tell, the distribution is 
missing essential text files that do not come except with a licensed 
prepared binary copy - it is about to get deleted from my system, even 
though it seems to have a bunch of nice features, and I have a kind of 
nostalgic desire to run it.

The folding of network topology knowledge into the mail distribution 
manager made sense when networks were simpler - and when they typically 
were store and forward.  Remember that the network this was made for was an 
"NJE" or "RSCS" store and forward network - the unit of transmission was 
not the packet, it was an e-mail (actually, file, an e-mail was just a file 
with a special name)...the e-mail was not sent directly to a destination, 
in fact, there was no way to get to the destination "directly" unless it 
was local.  The e-mail was sent to a node that was closer to the 
destination.  And that node figured out a route and sent it on forward.  It 
*could* take days for a message to cross the Atlantic if the links were 
saturated.  And it was not unheard for a bunch of messages to be spun off 
to tape, and for that tape to be Fedexed or DHL'd to a node much closer to 
the destination, so as to get around an overloaded link. (We used a bitnet 
like network in IBM --- and sometimes there was only a small pathway 
between countries, with files that could take a day in transit tying it 
up.  Early versions of the protocol could send one thing at a time across a 
link, and short files were preferred, but once a long file started, it 
monopolized the link until it finished.  Later versions could subdivide a 
link and multiplex.)

The point was that the machines like Listserv, or the thing called Toolsrun 
that we used inside of IBM, could "understand" the network topology and 
could shed subscriptions to closer servers.  As well as holding archives 
closer to the end user

But these days, frankly, teaching your mailing list server about network 
topology seems counter-productive.  The knowledge of network topology 
belongs, if anywhere, in the MTA and the database that describes that, in 
our case, the DNS.

For example, the concept of "Florida" is more or less meaningless.  My 
system is in Florida, the one that I am typing this note on.  The packets 
that leave this system go to a router that, I am told, has a "Florida" name 
when I reverse translate it, but which is actually in Chicago 
somewhere.  They actually traverse the link on something that is probably 
ATM, with virtual circuits set up so that it is actually a small piece of a 
bigger pipe -- but the next time that this is surfaced into a router that 
looks at the IP layer is across the country.

So for me to look for an IRC server that is in Chicago is sort of 
silly.  Those packets at least go across the country once, and then they 
might come back to Florida - even if they come back on XO's net, they cross 
the country twice.

Putting this sort of "layer violation" in Listserv was probably essential 
at the time.  A link of 4800 BPS or slower was not uncommon at the time 
that the Bitnet protocols were designed.  1200 was the limit for dial-ups, 
and a lot of mail was moved by store and forwards such as bitnet, or even 
uucpnet.   For a long time, RSCS nodes would not burst - that is, you could 
not drop a piece of mail into the RSCS or NJE networks with multiple 
destinations and have it manage the shipment of copies to end nodes.  Every 
destination had to have its own copy traversing those slow links.

But had that existed from day one, the listserv and toolsrun shedding of 
subscriptions to "closer" nodes might never have ex

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "BAW" == Barry A Warsaw <[EMAIL PROTECTED]> writes:

BAW> Think of shunt as a safety net.  If Mailman has a bug that
BAW> causes an uncaught exception to occur while handling the
BAW> message, it gets shuffled off to shunt, so that once the bug
BAW> is fixed, you ought to be able to move the message back to
BAW> qfiles/in and have it delivered again.

BAW> I may do two additional things with shunt: first, I should
BAW> probably include a key in the metadata to indicate which
BAW> queue the shunted file came from, and second, I want to add a
BAW> bin/unshunt command which will re-queue shunted messages
BAW> safely and correctly.

This is now done.

BAW> As to your other problem, I'm going to have to think about
BAW> that.  I agree that it's not good that if Mailman is shut
BAW> down that messages are delivered either twice or not at all.
BAW> A solution will likely require some disk i/o, but the
BAW> question is, how to do things robustly without hammering the
BAW> disk.

This should be solved too.  First, Mailman already attempts to cleanly
shut down the qrunner loops on receipt of SIGHUP or SIGTERM.  Second,
should an error occur, the message's metadata dictionary (i.e. its .db
file) will containe a list of as-yet-undelivered recipients.  So upon
restart or unshunting, it should continue where it's left off.  To
reduce the impact on the disk, it will only maintain undelivered
addresses per `chunk', which can be SMTP_MAX_RCPTS, or per-address if
VERPing.  I think this is a fine tradeoff (some small number of folks
/might/ get duplicates, but no one should miss a message).

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> I think I've got it: the French translation of 
F> "This is a multi-part message in MIME format.

F> "

F> lacks the trailing linefeeds. Hence mutt (at least mutt) does
F> not detect the part of the multipart message and shows an empty
F> mail.

Ousmane does a very good job of keeping the French translation
up-to-date.  Hopefully he will look into this and fix whatever might
be necessary.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Ousmane Wilane]

> "BAW" == Barry A Warsaw <[EMAIL PROTECTED]> writes:

> "F" == Fil  <[EMAIL PROTECTED]> writes:

  F> I think I've got it: the French translation of "This is a
  F> multi-part message in MIME format."

I didn't find the offending string anywhere in the current Mailman
source tree. Perhaps I'm overlooking something.  
Fil can you help me find it and fix it please!  BTW IIRC, I use to run
msghack/msgfmt on the catalog to make sure that everything is ok.

  F> lacks the trailing linefeeds. Hence mutt (at least mutt) does not
  F> detect the part of the multipart message and shows an empty mail.

  BAW> Ousmane does a very good job of keeping the French translation
  BAW> up-to-date.  Hopefully he will look into this and fix whatever
  BAW> might be necessary.

Thanks!
I'm trying to do my best but as noted by Fil in one of his messages,
French is not my mother tongue, so there are many imperfections laying
around and I'm eager to get more involvement for proofreading and
alike.

Cheers

-- Ousmane Wilane




___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Ousmane Wilane <[EMAIL PROTECTED]> :
> > "F" == Fil  <[EMAIL PROTECTED]> writes:
>   F> I think I've got it: the French translation of "This is a
>   F> multi-part message in MIME format."

I was wrong, sorry Ousmane.

It's something in the way the multi-part message is formed, but I can't find
what (I don't know enough about MIME). However I could "bounce" the
problematic messages to Barry if he wanted. In short, it's every notice from
mailman, and even some messages (at least one) that go through.


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> I was wrong, sorry Ousmane.

F> It's something in the way the multi-part message is formed, but
F> I can't find what (I don't know enough about MIME). However I
F> could "bounce" the problematic messages to Barry if he
F> wanted. In short, it's every notice from mailman, and even some
F> messages (at least one) that go through.

I believe this is fixed now in cvs, but it required some changes to
the email package (now embodied in version 1.2).  Please do a cvs
update and see if the problem is fixed for you.  I could reproduce the
bug in the English version and it looks fixed now.

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> F> I can't find what (I don't know enough about MIME). However I
>
> I believe this is fixed now in cvs, but it required some changes to
> the email package (now embodied in version 1.2).  Please do a cvs
> update and see if the problem is fixed for you.  I could reproduce the
> bug in the English version and it looks fixed now.

Yes, it's working again. Thank you!

However the other bug came back: did you not commit the correction?

Traceback (most recent call last):
  File "/home/mailman/scripts/driver", line 82, in run_main
main()
  File "/home/mailman/Mailman/Cgi/listinfo.py", line 42, in main
listinfo_overview()
  File "/home/mailman/Mailman/Cgi/listinfo.py", line 85, in
listinfo_overview
mlist = MailList.MailList(name, lock=0)
  File "/home/mailman/Mailman/MailList.py", line 102, in __init__
self.Load()
  File "/home/mailman/Mailman/MailList.py", line 541, in Load
self.CheckVersion(dict)
  File "/home/mailman/Mailman/MailList.py", line 558, in CheckVersion
self.Lock()
  File "/home/mailman/Mailman/MailList.py", line 148, in Lock
self.Load()
  File "/home/mailman/Mailman/MailList.py", line 541, in Load
self.CheckVersion(dict)
  File "/home/mailman/Mailman/MailList.py", line 561, in CheckVersion
Update(self, stored_state)
  File "/home/mailman/Mailman/versions.py", line 53, in Update
CanonicalizeUserOptions(l)
  File "/home/mailman/Mailman/versions.py", line 367, in
CanonicalizeUserOptions
if l.getMemberOption(k, mm_cfg.DisableDelivery):
  File "/home/mailman/Mailman/OldStyleMemberships.py", line 120, in
getMemberOption
self.__assertIsMember(member)
  File "/home/mailman/Mailman/OldStyleMemberships.py", line 113, in
__assertIsMember
raise Errors.NotAMemberError, member


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Fil <[EMAIL PROTECTED]> :
> However the other bug came back: did you not commit the correction?
>   File "/home/mailman/Mailman/OldStyleMemberships.py", line 120, in
> getMemberOption
> self.__assertIsMember(member)
>   File "/home/mailman/Mailman/OldStyleMemberships.py", line 113, in
> __assertIsMember
> raise Errors.NotAMemberError, member

In the logs I now have this kind of things re: OldStyleMemberships

==> /home/mailman/logs/error <==
Mar 12 10:28:37 2002 (5350) Bouncer exception while processing module DSN:
Mar 12 10:28:37 2002 (5350) Traceback (most recent call last):
  File "/home/mailman/Mailman/Bouncers/BouncerAPI.py", line 77, in ScanMessages
mlist.registerBounce(addr, msg)
  File "/home/mailman/Mailman/Bouncer.py", line 107, in registerBounce
self.setBounceInfo(member, info)
  File "/home/mailman/Mailman/OldStyleMemberships.py", line 344, in setBounceInfo
assert self.__mlist.Locked() AssertionError


-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> Yes, it's working again. Thank you!

F> However the other bug came back: did you not commit the
F> correction?

>> "/home/mailman/Mailman/OldStyleMemberships.py", line 120, in
F> getMemberOption
>>   self.__assertIsMember(member) File
>> "/home/mailman/Mailman/OldStyleMemberships.py", line 113, in
| __assertIsMember
| raise Errors.NotAMemberError, member

Nope, this one is definitely in versions.py 2.25.  In fact it was
committed in revision 2.22 on 09-Mar-2002.

-Barry


revision 2.22
date: 2002/03/09 19:11:27;  author: bwarsaw;  state: Exp;  lines: +9 -0
CanonicalizeUserOptions(): Two changes.  First, when migrating
user_options, skip the key if its not a member.  This can happen when
upgrading legacy databases because I believe there were bugs long ago
that caused some keys to appear in user_options but not in members or
digest_members.

Also, we now have a mini-version id for the user_options stuff so
CanonicalizeUserOptions() won't try to run every time some other part
of the schema changes.

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Barry A. Warsaw]

> "F" == Fil  <[EMAIL PROTECTED]> writes:

F> In the logs I now have this kind of things re:
F> OldStyleMemberships

Are you positive you're up-to-date in cvs?  No sticky tags?

-Barry

___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> Are you positive you're up-to-date in cvs?  No sticky tags?

Don't know for sure. I had three files in status "Need Patch". I'll checkout
a brand new version, and keep you posted if some bug comes up again.

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Fil <[EMAIL PROTECTED]> :
> Don't know for sure. I had three files in status "Need Patch". I'll checkout
> a brand new version, and keep you posted if some bug comes up again.

I am ashamed: "cvs update" did me no good; some scripts were up-to-date,
others definitely not. Now everything's a clean install, I'll let the
week-end go without a post to the -developpers list.

-- Fil


___
Mailman-Developers mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/mailman-developers

Re: [Mailman-Developers] big list

[Fil]

@ Barry A. Warsaw <[EMAIL PROTECTED]> :
> > "F" == Fil  <[EMAIL PROTECTED]> writes:
> F> I am ashamed: "cvs update" did me no good; some scripts were
> F> up-to-date, others definitely not. Now everything's a clean
> F> install, I'll let the week-end go without a post to the
> F> -developpers list.
> 
> :)


Hi there, my week end's over!

Here are tracebacks of bugs observed with Mailman 2.1b1 (I'm positive this
is a nice and complete install from cvs!) over the week end.

Two bugs seem to be running:

1)   File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address 
AttributeError: address

One of my lists' admin (his list has 13 000 subscribers) has hit this one
several times this weekend (he received the tracebacks by email).


2) A timeout occurs on the 'pending' database and maybe on the list database
also when we send many messages. From observing the processes running on the
server and the flow of emails, it looks like the culprit is the
BounceRunner, which locks these databases for long periods, and does not
release the locks fast enough for the other processes.

Maybe this is because the lists are big (ie long time
opening/saving/checking if the bounce is a member). We don't care much about
the BounceRunner losing its lock, but we should let it block the other
processes, who are responding to the users (be it the Web interface or the
mail command runner).

On my setup, a bounce on the 50 000 subscribers' list is processed in about
3-4 seconds. The BounceRunner takes up 100% of one CPU (I have two) and a
lot of memory (although less than last time I raised that issue).
But count: 3 seconds * 40 000 bounces = 33 hours. So if someone migrates her
big list from some bounce-blind MLM to Mailman, she'll run into that issue:
sub/unsub/listinfo/whatever requests on the list returning exceptions for 33
hours.

This has happened many times also over the week end.

Hope this can be helpful.



SOME TRACEBACKS:

- Original Message -
From: <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Sunday, March 17, 2002 2:09 PM
Subject: Erreur Mailman inattendue


Une erreur Mailman inattendue s'est produite dans
MailCommandHandler.ParseMailCommand(). En voici les raisons:

Traceback (most recent call last):
  File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in
ParseMailCommands
self.__dispatch[cmd](args, line, msg)
  File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in
ProcessConfirmCmd
results = self.ProcessConfirmation(args[0], msg)
  File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address
AttributeError: address
Pour gerer votre abonnement :
http://listes.rezo.net/mailman/listinfo/listname

The error log says about the same thing:
Mar 17 14:09:04 2002 (9618) Unexpected Mailman error:
Traceback (most recent call last):
  File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in ParseMailCom
self.__dispatch[cmd](args, line, msg)
  File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in ProcessConfi
results = self.ProcessConfirmation(args[0], msg)
  File "/home/mailman/Mailman/MailList.py", line 980, in ProcessConfirmation
addr = userdesc.address
AttributeError: address



***

Mar 17 18:52:14 2002 admin(8563):

admin(8563): [- Mailman Version: 2.1b1 -]
admin(8563): [- Traceback --]
admin(8563): Traceback (most recent call last):
admin(8563):   File "/home/mailman/scripts/driver", line 82, in run_main
admin(8563): main()
admin(8563):   File "/home/mailman/Mailman/Cgi/subscribe.py", line 94, in main
admin(8563): process_form(mlist, doc, cgidata, language)
admin(8563):   File "/home/mailman/Mailman/Cgi/subscribe.py", line 176, in proc
admin(8563): mlist.AddMember(userdesc, remote)
admin(8563):   File "/home/mailman/Mailman/MailList.py", line 711, in AddMember
admin(8563): cookie = Pending.new(Pending.SUBSCRIPTION, userdesc)
admin(8563):   File "/home/mailman/Mailman/Pending.py", line 61, in new
admin(8563): lock.lock(timeout=30)
admin(8563):   File "/home/mailman/Mailman/LockFile.py", line 292, in lock
admin(8563): raise TimeOutError
admin(8563): TimeOutError

***

Mar 17 19:11:25 2002 (9618) Unexpected Mailman error:
Traceback (most recent call last):
  File "/home/mailman/Mailman/MailCommandHandler.py", line 273, in
ParseMailCom
self.__dispatch[cmd](args, line, msg)
  File "/home/mailman/Mailman/MailCommandHandler.py", line 727, in
ProcessConfi
results = self.ProcessConfirmation(args[0], msg)
  File "/home/mailman/Mailman/MailList.py", line 968, in ProcessConfirmation
data = Pending.confirm(cookie)
  File "/home/mailman/Mailman/Pending.py", line 94, in confirm
lock.lock(timeout=30)
  File "/home/mailman/Mailma