Re: [Mailman-Users] Debugging mass invitations using web page interface...

[The Mailing List Server Admin]

Mark,


I have updated the script to fix this.


Just a quick note to thank you for the fix.  Muchly appreciated!


--
  Cordially,
  the UNH Mailing List Server Admins
  Bill Costa, Adjunct Admin
  (603) 862-3056
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[Mark Sapiro]

On 8/26/19 3:28 PM, The Mailing List Server Admin wrote:
> Way way back in August 2016, we had a problem with submitting too
> many members via the web interface.  As a solution, the following
> was suggested:
> 
>> You'd need a custom script to do the 'extra text'. I have added an
>> invite_members script I use at
>>  and
>>  This can send
>> invitations including the 'extra text'.
> 
> We used this script with great success in Aug 2017 and 2018.  But
> this year we've run into a problem.  When I submit the script it
> appears to hang waiting for some kind of lock.  I can see this
> when I eventually CTRL-C out.
> 
>     Traceback (most recent call last):
>   File "/home/mladmin/Util/invite_members", line 236, in ?
>     main()
>   File "/home/mladmin/Util/invite_members", line 197, in main
>     mlist = MailList.MailList(listname)
>   File "/usr/local/mailman/Mailman/MailList.py", line 129, in
>   __init__
>     self.Lock()
>   File "/usr/local/mailman/Mailman/MailList.py", line 162, in
>   Lock
>     self.__lock.lock(timeout)
>   File "/usr/local/mailman/Mailman/LockFile.py", line 312, in
>   lock
>     self.__sleep()
>   File "/usr/local/mailman/Mailman/LockFile.py", line 496, in
>   __sleep
>     time.sleep(interval)
>     KeyboardInterrupt


The list is locked. See  for more
about locks and removing stale locks.

I have looked at the script and I see an issue. It can leave the list
locked if you specify a message file and that file cannot be read.

I have updated the script to fix this.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[The Mailing List Server Admin]

Way way back in August 2016, we had a problem with submitting too
many members via the web interface.  As a solution, the following
was suggested:


You'd need a custom script to do the 'extra text'. I have added an
invite_members script I use at 
 and

 This can send
invitations including the 'extra text'.


We used this script with great success in Aug 2017 and 2018.  But
this year we've run into a problem.  When I submit the script it
appears to hang waiting for some kind of lock.  I can see this
when I eventually CTRL-C out.

Traceback (most recent call last):
  File "/home/mladmin/Util/invite_members", line 236, in ?
main()
  File "/home/mladmin/Util/invite_members", line 197, in main
mlist = MailList.MailList(listname)
  File "/usr/local/mailman/Mailman/MailList.py", line 129, in
  __init__
self.Lock()
  File "/usr/local/mailman/Mailman/MailList.py", line 162, in
  Lock
self.__lock.lock(timeout)
  File "/usr/local/mailman/Mailman/LockFile.py", line 312, in
  lock
self.__sleep()
  File "/usr/local/mailman/Mailman/LockFile.py", line 496, in
  __sleep
time.sleep(interval)
KeyboardInterrupt

In the past we've used the script from a non-priv'd account with
no problems.  When it failed this time, I tried running it as
'mailman' but it still hangs on the lock.  Any suggestions?
(BTW, the line numbers will be different from the original script
because of some debug print statements and additional comments
added.)

--
  Cordially,
  the UNH Mailing List Server Admins
  Bill Costa, Adjunct Admin
  (603) 862-3056
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[The Mailing List Server Admin]

Mark Sapiro  recently wrote (in part):


You could use the script ... to list the pending subscriptions. Some
may have been accepted already, so the fact that one isn't there
doesn't mean it wasn't sent, but if you know the order from the
original input, you can probably figure how far it got.


Between being able to see who is now already subscribed from my
original list, and looking at the pending subscription requests, I
should be able to figure out who we've missed.  Thanks for pointing
out yet another useful MM2 utility!

--
  Cordially,
  the UNH Mailing List Server Admins
  Bill Costa, senior admin
  (603) 862-3056
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[Mark Sapiro]

On 08/27/2016 03:16 AM, The Mailing List Server Admin wrote:
> 
> So when Apache timed out, would it have killed the admin CGI on the
> way out?  In other words, would the admin script still run to
> completion and send out all the invitations from that batch?  Or is it
> most likely to have been a partial run?  Am I correct that the only
> place I will find a record of the invitations sent will be in the
> Sendmail mail logs?  I'd like to avoid sending dup invites if
> possible.


When Apache times out, it kills the CGI process. The process meanwhile
has been queueing the invitations to be sent, so a bunch will be sent.

You could use the script at
 to list the pending
subscriptions. Some may have been accepted already, so the fact that one
isn't there doesn't mean it wasn't sent, but if you know the order from
the original input, you can probably figure how far it got.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[The Mailing List Server Admin]

Mark, once again thank you for your valued expertise...

With regards to the failed thousand invitations submission via the Web
UI:


The admin CGI is processing the mass subscribe/invite list. While it
is doing this it is not generating any output. If it takes 'too
long' the web server will time out waiting for output from the
CGI.


Makes perfect sense.  (I wonder if MM3 uses Ajax or other web
techniques to provide some sort of progress indicator to prevent this
from happening.)

So when Apache timed out, would it have killed the admin CGI on the
way out?  In other words, would the admin script still run to
completion and send out all the invitations from that batch?  Or is it
most likely to have been a partial run?  Am I correct that the only
place I will find a record of the invitations sent will be in the
Sendmail mail logs?  I'd like to avoid sending dup invites if
possible.

Thanks for pointer to the command line invitation script.  I'll add it
to my growing arsenal.

--
  Cordially,
  the UNH Mailing List Server Admins
  Bill Costa, senior admin
  (603) 862-3056
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Debugging mass invitations using web page interface...

[Mark Sapiro]

On 08/26/2016 05:24 PM, The Mailing List Server Admin wrote:
> 
> I ended up having the same error happen.  Looking at the Mailman logs,
> I can't see any difference before or after my submission.  In the
> HTTPD logs, I see:
> 
>>> [Fri Aug 26 19:59:23 2016] [warn] [client 132.177.215.132] Timeout
>>> waiting for output from CGI script
>>> /usr/local/mailman/cgi-bin/admin, referer:
>>> https://lists.unh.edu/mailman/admin/campus.connection/members/add
> 
>>> [Fri Aug 26 19:59:23 2016] [error] [client 132.177.215.132]
>>> Premature end of script headers: admin, referer:
>>> https://lists.unh.edu/mailman/admin/campus.connection/members/add
> 
> So is there an inherent limit to the number of invites that can be
> submitted via the web form?


The admin CGI is processing the mass subscribe/invite list. While it is
doing this it is not generating any output. If it takes 'too long' the
web server will time out waiting for output from the CGI. That's what's
happening here.

The amount of time is set by apache's TimeOut directive



> As a work around, how would I do large invites on behalf of the owner
> from the command line, including the 'extra text' that is allowed via
> the web interface?


You'd need a custom script to do the 'extra text'. I have added an
invite_members script I use at
 and
. This can send
invitations including the 'extra text'.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org