[Mailman-Users] Large list import

2005-03-05 Thread lists
I've got a list of a couple hundred thousand subscribers that I need to 
mass subscribe, however the file upload import keeps returning an 
"internal error."  Is there a better way to do this, such as uploading 
the lists to the server manually and cat'ing it onto the end of some 
sort of store that Mailman uses?

-George
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp


Re: [Mailman-Users] Large list import

2005-03-05 Thread Michael Loftis

--On Saturday, March 05, 2005 11:35 PM -0500 [EMAIL PROTECTED] wrote:
I've got a list of a couple hundred thousand subscribers that I need to
mass subscribe, however the file upload import keeps returning an
"internal error."  Is there a better way to do this, such as uploading
the lists to the server manually and cat'ing it onto the end of some sort
of store that Mailman uses?
YEah there are CLI tools to do the additions but I dont' think you'll want 
to do this...

At ~15k subscribers on a list we have it's pudgy and slow, it'd be 
unusuable at 200k.  Mailman rewrites the entire config.pck/list config file 
for every add/remove/change on a list.  Very inefficient, very unscaleable.

I'm poking around at the code lately deciding if I'll tackle the job of 
fixing that or not...but until then you might want to do something with a 
sort of umbrella/sub list approach.

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp


Re: [Mailman-Users] Large list import

2005-03-05 Thread Mark Sapiro
[EMAIL PROTECTED] wrote:

>I've got a list of a couple hundred thousand subscribers that I need to 
>mass subscribe, however the file upload import keeps returning an 
>"internal error."  Is there a better way to do this, such as uploading 
>the lists to the server manually and cat'ing it onto the end of some 
>sort of store that Mailman uses?

That's a LARGE list. See
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.015.htp

It sounds like you're using web mass subscribe. Is that the case? You
might be better off with bin/add_members, but you should probably run
it without notifications or welcome messages. See the entire thread
starting at
http://mail.python.org/pipermail/mailman-users/2005-January/041736.html

--
Mark Sapiro <[EMAIL PROTECTED]>   The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp


Re: [Mailman-Users] Large list import

2005-03-05 Thread lists
Fortunately this person wants the list to be completely moderated so  
that there will not be back and forth traffice, just outbound.  Will  
this help matters and make it snappy enough?  Oh, I just read the rest  
of your message!  :)  It appears that it truly is handling things  
inefficiently in the list subscriber IO.  Wow, this is a real problem.   
What should I do?  Are there other solutions that are better than  
Mailman for this?  Not quite sure what you mean by the umbrella/sub  
list approach.

Thanks for the heads up!

On Mar 5, 2005, at 11:53 PM, Michael Loftis wrote:

--On Saturday, March 05, 2005 11:35 PM -0500 [EMAIL PROTECTED]  
wrote:

I've got a list of a couple hundred thousand subscribers that I need  
to
mass subscribe, however the file upload import keeps returning an
"internal error."  Is there a better way to do this, such as uploading
the lists to the server manually and cat'ing it onto the end of some  
sort
of store that Mailman uses?
YEah there are CLI tools to do the additions but I dont' think you'll  
want to do this...

At ~15k subscribers on a list we have it's pudgy and slow, it'd be  
unusuable at 200k.  Mailman rewrites the entire config.pck/list config  
file for every add/remove/change on a list.  Very inefficient, very  
unscaleable.

I'm poking around at the code lately deciding if I'll tackle the job of  
fixing that or not...but until then you might want to do something with  
a sort of umbrella/sub list approach.

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives:  
http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:  
http://mail.python.org/mailman/options/mailman-users/lists% 
40southernohio.net

Security Policy:  
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp


Re: [Mailman-Users] Large list import

2005-03-05 Thread Michael Loftis

--On Sunday, March 06, 2005 12:07 AM -0500 [EMAIL PROTECTED] wrote:
Fortunately this person wants the list to be completely moderated so
that there will not be back and forth traffice, just outbound.  Will
this help matters and make it snappy enough?  Oh, I just read the rest
of your message!  :)  It appears that it truly is handling things
inefficiently in the list subscriber IO.  Wow, this is a real problem.
What should I do?  Are there other solutions that are better than
Mailman for this?  Not quite sure what you mean by the umbrella/sub  list
approach.
Thanks for the heads up!
Multiple lists each woth ~10k or ~15k subscribers, and then one master list 
with all those sub-lists subscribed.

Sub lists configured to only take postings formt he master listthis 
DOES complicate subscribing though.

so YMWV :)
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp