Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Brad Knowles]

On Oct 29, 2009, at 11:54 AM, Mark Sapiro wrote:

> If I understand the OP's situation, this probably won't help. I think it
> is a case of a single post to a large list. In this case, a single
> OutgoingRunner will handle the message and the other runners will do
> nothing.

If it's a case of a single post to a large list, then the problem is most 
likely going to be Python pickle contention.  Split the large list up into 
multiple smaller sub-lists, with a parent "umbrella" list.  More info on that 
solution is in the FAQ wiki, but the OP should have already found that if they 
searched for "performance".

I can guarantee you that I can build an MTA configuration that is faster than 
Mailman could ever possibly be, for this single reason alone -- even if you run 
both on a pure RAMdisk.


As for the rest, an option would be to use a load-balancing switch (either 
hardware or implemented in software), but there's a heck of a lot of tuning to 
be done on a single machine running a properly configured MTA and a properly 
designed mailing list server infrastructure, before you get to the point where 
load balancing switches and multiple outbound mail relay servers would start to 
make a big difference.

Most of these topics are at least touched on in the FAQ Wiki and the fact that 
the OP hasn't mentioned them tells me that they either didn't do their 
homework, or they don't understand the concepts well enough to be able to do 
the homework.

--
Brad Knowles 
LinkedIn Profile: 

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Mark Sapiro]

Stefan Förster wrote:

> 3. If mail is leaving your MTA at a slow rate but the delivery from
> Mailman to the MTA is fast, try running several queue runners and
> specify an SMTPHOST wit a DNS record pointing to several addresses
> (I'm not really sure about this one, perhaps Mark could clarify).


If I understand the OP's situation, this probably won't help. I think it
is a case of a single post to a large list. In this case, a single
OutgoingRunner will handle the message and the other runners will do
nothing.

The idea of multiple addresses in DNS for SMTPHOST is that multiple
connections to SMTPHOST *may* connect to different servers, but in the
short term, this probably doesn't work, because this depends on the
authoritative DNS returning the A records in different order, but once
we look up SMTPHOST, we may not ask again before TTL expires. Multiple
addresses for a name in DNS works for load balancing of a lot of
incoming connects from multiple sources. I'm not sure it works well or
at all for outgoing connects from a single source.

However, if the multiple A records does work in the short term, it may
help even in the large list scenario because SMTPDirect will close and
reopen the connection to SMTPHOST every SMTP_MAX_SESSIONS_PER_CONNECTION
chunks, so if you set SMTP_MAX_SESSIONS_PER_CONNECTION to a small
positive number, you may connect to a different IP for successive
connects to deliver the one message.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Stefan Förster]

* Hien HUYNH HUU :
>  Could Mailman be configured with multi mail server (at least 2
>  mail server) at the same time ? So that It can transfer huge
>  numbers of emails at the same time . If yes, please let me know
>  how to configure It.

I'm pretty sure you are doing it wrong (configuring your MTA, I mean)
- or perhaps you are simply limited badnwith-wise. You might want to
show us you MTA/Mailman integration so that we can come up with some
helpful hints.

Other than that, I think you have three options:
1. Get a load balancer. Their prices are down to the low five digit
range nowadays.
2. Build yourself a load balancer (helpful Google keywords might be
"ldirector", "linux lvs" and "pacemaker", "openais", "heatbeat" for
availability).
3. If mail is leaving your MTA at a slow rate but the delivery from
Mailman to the MTA is fast, try running several queue runners and
specify an SMTPHOST wit a DNS record pointing to several addresses
(I'm not really sure about this one, perhaps Mark could clarify).


Cheers
Stefan
-- 
Stefan Förster http://www.incertum.net/ Public Key: 0xBBE2A9E9
Wer schläft, sündigt nicht.
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Stephen J. Turnbull]

Hien HUYNH HUU writes:

 > I agree with you that this is no a mailman issue . But I think If
 > mailman can use load balancing MTA server, Its performance will be
 > increased , isn't It ? That's exactly I need now because I can't
 > increase the bandwidth and MTA performance more.

If you are bandwidth-limited, you're stuck.  You have to increase
bandwidth, or use it more efficiently.

If you are not bandwidth-limited, I find it hard to believe that you
*can't* improve the performance of the MTA to at least the level you
require.  It seems likely that whatever bottleneck is slowing your
system down would apply to the "load balancing MTA" just as it does to
your current one.  You need to figure out what is slowing down your
system.

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Mark Sapiro]

Hien HUYNH HUU wrote:

>   That's exactly what I need , could you please tell me more what can I 
> modify on SMTPDirect.py file ?


If I were doing this, I would modify the Connection class constructor
to accept host and port arguments and use them in the __connect()
method instead of mm_cfg.SMTPHOST and mm_cfg.SMTPPORT. Then I would
modify the

conn = Connection()
try:
[...]
finally:
conn.quit()
msgdata['recips'] = origrecips

code in process to instantiate multiple connections to multiple
servers. Maybe make mm_cfg.SMTPHOST and mm_cfg.SMTPPORT lists, and
instantiate one connection for each host/port pair. Then round-robbin
through the connections in the deliveryfunc() call in the while loop.

If the above is not clear, you have to learn Python until it becomes
clear.

Your effort would probably be better spent either learning how to tune
your existing MTA to be more performant or installing a more
performant MTA. In an earlier post, you mentioned delivering 10,000
messages in 10 minutes (a rate of 16.7 messages per second) as an
acceptable rate. On my server using Postfix essentially out-of-the-box
and full Mailman VERP, Mailman delivers on the order of 50 messages
per second to Postfix.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Carl Zwanzig]

Hien HUYNH HUU wrote:

Dear all, Could Mailman be configured with multi mail server (at least 2
mail server) at the same time ? So that It can transfer huge numbers of
emails at the same time . If yes, please let me know how to configure It.


You really don't need to do that. Have you read the Performance section of 
the FAQ? Some -very- large lists manage to do without this 'feature'.  After 
that, it's an MTA and bandwidth issue, not a mailman issue.


z!

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Hien HUYNH HUU]

Dear Sapiro,
Yeah,
   That's exactly what I need , could you please tell me more what can I modify 
on SMTPDirect.py file ?
   Thank you very much for your reply.
Best regards. 
   Huu Hien

From: Mark Sapiro [m...@msapiro.net]
Sent: Friday, October 23, 2009 10:25 PM
To: Hien HUYNH HUU; mailman-users@python.org
Subject: Re: [Mailman-Users] Using Mailman with Multi Mail Server

Hien HUYNH HUU wrote:

>Could Mailman be configured with multi mail server (at least 2 mail 
> server) at the same time ? So that It can transfer huge numbers of emails at 
> the same time . If yes, please let me know how to configure It.


I assume you are talking about outgoing mail. This is not a
'configuration' option. It would require significant modification to
Mailman/Handlers/SMTPDirect.py to implement load sharing using
multiple outgoing MTAs.

--
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Hien HUYNH HUU]

Dear Zwanzig,
   Thank for your reply. I read the Performance section of FAQ and did exactly 
as they guide. I also tunning my mail server but the result is not as my 
expectation.
I agree with you that this is no a mailman issue . But I think If mailman 
can use load balancing MTA server, Its performance will be increased , isn't It 
? That's exactly I need now because I can't increase the bandwidth and MTA 
performance more.
   Regards,
   Huu Hien


From: Carl Zwanzig [...@tuunq.com]
Sent: Friday, October 23, 2009 11:09 PM
To: Hien HUYNH HUU
Cc: mailman-users@python.org
Subject: Re: [Mailman-Users] Using Mailman with Multi Mail Server

Hien HUYNH HUU wrote:
> Dear all, Could Mailman be configured with multi mail server (at least 2
> mail server) at the same time ? So that It can transfer huge numbers of
> emails at the same time . If yes, please let me know how to configure It.

You really don't need to do that. Have you read the Performance section of
the FAQ? Some -very- large lists manage to do without this 'feature'.  After
that, it's an MTA and bandwidth issue, not a mailman issue.

z!

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Using Mailman with Multi Mail Server

[Mark Sapiro]

Hien HUYNH HUU wrote:

>Could Mailman be configured with multi mail server (at least 2 mail 
> server) at the same time ? So that It can transfer huge numbers of emails at 
> the same time . If yes, please let me know how to configure It.


I assume you are talking about outgoing mail. This is not a
'configuration' option. It would require significant modification to
Mailman/Handlers/SMTPDirect.py to implement load sharing using
multiple outgoing MTAs.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Using Mailman with Multi Mail Server

[Hien HUYNH HUU]

Dear all,
Could Mailman be configured with multi mail server (at least 2 mail server) 
at the same time ? So that It can transfer huge numbers of emails at the same 
time . If yes, please let me know how to configure It.
Please let me know about your ideas.
Thank you  and best regards,
Huu Hien

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org