Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
On 9/1/07, Mark Sapiro wrote:
OTOH, it won't be that slow. If your MTA as as blinding fast as you
say, I imagine that for moderate sized messages, say 10 KB, it would
take well under a minute to deliver to the MTA.
It all depends on how many recipients there are, how large the
message is, and how many total deliveries are required to get all
necessary copies of the message transmitted from Mailman to the MTA.
I don't know how many members are on the many lists at python.org, but
many of these lists have personalized and/or VERPed delivery and thus
send an individual message to each subscriber, and it doesn't seem to
be an issue there.
We currently have 128 lists, and the Top Ten largest lists are:
3406 Python-list
3316 Mailman-announce
2997 Tutor
2598 Mailman-Users
1492 Python-announce-list
1309 Python-Dev
874 python-win32
855 Pythonmac-SIG
694 Image-SIG
646 XML-SIG
Going back to May 2005, the Top Ten lists with the most traffic are:
35787 python-list
21220 python-dev
18747 tutor
16636 python-checkins
13810 mailman-users
11241 python-bugs-list
10184 python-3000
7754 python-help
5790 patches
5400 pythonmac-sig
The Top Ten fastest lists with the highest average delivery rates to
the MTA are:
total delivery
List # msgs time (secs) Avg. rate
-- -
Tutor 18629829 / 104001.403 =179.131 msg/s
Baypiggies 747203 / 4413.775 =169.289 msg/s
python-win321649983 / 9783.514 =168.649 msg/s
C++-sig1317540 / 7915.801 =166.444 msg/s
Edu-sig 801584 / 5248.601 =152.723 msg/s
Python-announce-list2002410 / 13363.320 =149.844 msg/s
PythonCE 245065 / 1638.371 =149.578 msg/s
Python-ideas 131066 /883.584 =148.335 msg/s
Image-SIG 465971 / 3271.483 =142.434 msg/s
Distutils-SIG 610489 / 4330.037 =140.989 msg/s
Now, we are a bit of a special case, since we dump all our outgoing
mail on the ISP that provides hosting services for the Python.org
machines (XS4ALL.nl), but as far as the rest of the numbers are
concerned, I think they're fairly representative.
--
Brad Knowles [EMAIL PROTECTED]
LinkedIn Profile: http://tinyurl.com/y8kpxu
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
Security Policy:
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
On 9/1/07, Michael Anderson wrote: So, the question I have is - is there any way to set mailman so that it doesn't send all its copies of a message to yahoo.com for instance all at once? Can you break it up? We have 200 yahoo users which is over their threshold. Search the FAQ Wizard for throttle. In particular, see FAQ 4.51 at http://www.python.org/cgi-bin/faqw-mm.py?req=showfile=faq04.051.htp. Also, is there a way to find out which major blacklists your server is on and also a way to find the addresses you need to request whitelisting? I have a feeling there is a repository for this that you can search, but I don't know where to find it. Google for blacklist checker. Three of the top five hits are: http://www.robtex.com/rbl.html http://www.moensted.dk/spam/ http://www.mxtoolbox.com/blacklists.aspx -- Brad Knowles [EMAIL PROTECTED] LinkedIn Profile: http://tinyurl.com/y8kpxu -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
On 9/1/07, Mark Sapiro wrote: I suppose that's possible, but before going down that road, I would make sure that the domain that the server identifies itself as in SMTP HELO or EHLO is the same domain returned by an rDNS lookup of its IP address. Another factor might be the firewall. Some firewalls (especially certain versions of the Cisco PIX) are known to break SMTP in some pretty horrible ways, and it is entirely possible that large sites might set up their servers to look for certain types of behaviour that are in violation of the protocol. Or maybe they had these problems previously, and then instituted a way to work around them. But when the firewall changed, they didn't un-break whatever else they had to break in a different way, in order to work around the previous bizarre Cisco PIX problems. At the very least, this would be something else to check out. -- Brad Knowles [EMAIL PROTECTED] LinkedIn Profile: http://tinyurl.com/y8kpxu -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
On Sep 1, 2007, at 1:09 PM, Mark Sapiro wrote: I suppose that's possible, but before going down that road, I would make sure that the domain that the server identifies itself as in SMTP HELO or EHLO is the same domain returned by an rDNS lookup of its IP address. Let me add to this. I fully agree that there are DNS related things that can and should be done to make a legitimate mailing list system less likely to be blacklisted. I run a mailing list server at 72.64.118.118. Before I set that up I had to have the rDNS (aka PTR record) for that IP changed from static-72-64-118-118.dllstx.fios.verizon.net. to n118.ewd.goldmark.org. Doing this was the hardest part of the job because it took two painful hours on the phone with Verizon Business Internet support. However painful that was, it really was necessary. I would not have set up a mailing list server until I had that sorted out. I also have SPF records for all of the domains that might appear in HELO or MAIL FROM addresses for anything coming off of my net. Having a proper rDNS and SPF records to match should demonstrate to most reasonable systems that the mail from my IP really is under my (fully traceable) control. Of course I have postmaster and abuse addresses working for all of the domains that mail may come from. Still, with all of this, I got blocked by 1and1.com as being a dynamic address (which of course I'm not). What was also annoying was that 1and1.com rejected the mail with a 4xx, so the mail just sat in my outgoing queue, retrying every now and then until I removed the messages from the queue. It took several attempts to contact 1and1.com support (they don't publish a telephone number for non-customers) before this finally got fixed. But other than that, I've had no problems with comcast, yahoo or hotmail and only minimal problems with AOL. AOL has a document on the web someplace about what they want from mailing list providers. Most of their requirements are pretty much what people should be doing anyway. This is a long winded way of saying that before looking at exotic solutions like throttling, there are other steps you should be taking anyway that may resolve the problems. -j -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
[Mailman-Users] Comcast, Yahoo, Hotmail blocking us
Suddenly, these major ISP's were refusing our connections. We are not on their blacklists. This all started on one day. After a great deal of investigation by my sysadmin and my University IT guys we have come to the conclusion that it might have been caused by updating our mail gateway machine. We upgraded from an antiquated PIX to one that is lightening fast. Plus we upgraded our bandwidth at the same time. So, what is happening is we are delivering the mail so fast that these ISP's are rejecting mail because they have thresholds in place that block large quantities of mail coming in from the same server in a short time. So, the question I have is - is there any way to set mailman so that it doesn't send all its copies of a message to yahoo.com for instance all at once? Can you break it up? We have 200 yahoo users which is over their threshold. We are grasping at straws on this one. Also, is there a way to find out which major blacklists your server is on and also a way to find the addresses you need to request whitelisting? I have a feeling there is a repository for this that you can search, but I don't know where to find it. Thanks for your help. Michael -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
Also, is there a way to find out which major blacklists your server is on Try one of these: http://www.mxtoolbox.com/blacklists.aspx http://www.dnsbl.info/advanced.asp You should probably check spamhaus.org directly here: http://www.spamhaus.org/zen/ If you find yourself on one or more blacklists, visit the blacklist's Web page for information on removal or whitelisting. -- Bob [EMAIL PROTECTED] wrote on 09/01/2007 12:48:06 PM: Suddenly, these major ISP's were refusing our connections. We are not on their blacklists. This all started on one day. After a great deal of investigation by my sysadmin and my University IT guys we have come to the conclusion that it might have been caused by updating our mail gateway machine. We upgraded from an antiquated PIX to one that is lightening fast. Plus we upgraded our bandwidth at the same time. So, what is happening is we are delivering the mail so fast that these ISP's are rejecting mail because they have thresholds in place that block large quantities of mail coming in from the same server in a short time. So, the question I have is - is there any way to set mailman so that it doesn't send all its copies of a message to yahoo.com for instance all at once? Can you break it up? We have 200 yahoo users which is over their threshold. We are grasping at straws on this one. Also, is there a way to find out which major blacklists your server is on and also a way to find the addresses you need to request whitelisting? I have a feeling there is a repository for this that you can search, but I don't know where to find it. Thanks for your help. Michael -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman- users/bbrown%40spsu.edu Security Policy: http://www.python.org/cgi-bin/faqw-mm.py? req=showamp;file=faq01.027.htp -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
Michael Anderson wrote: Suddenly, these major ISP's were refusing our connections. We are not on their blacklists. This all started on one day. After a great deal of investigation by my sysadmin and my University IT guys we have come to the conclusion that it might have been caused by updating our mail gateway machine. We upgraded from an antiquated PIX to one that is lightening fast. Plus we upgraded our bandwidth at the same time. So, what is happening is we are delivering the mail so fast that these ISP's are rejecting mail because they have thresholds in place that block large quantities of mail coming in from the same server in a short time. I suppose that's possible, but before going down that road, I would make sure that the domain that the server identifies itself as in SMTP HELO or EHLO is the same domain returned by an rDNS lookup of its IP address. I.e. if I connect from IP 10.0.0.1 and identify myself in HELO as example.com, an rDNS lookup of 10.0.0.1 should return example.com. If this is not the case, or if the rDNS returns a 'generic' name such as DSL-10-0-0-1.example.com, this is sufficient reason for some ISPs to block you. So, the question I have is - is there any way to set mailman so that it doesn't send all its copies of a message to yahoo.com for instance all at once? Can you break it up? We have 200 yahoo users which is over their threshold. There is no throttling per se in Mailman. You can probably slow things down a bit by setting VERP_DELIVERY_INTERVAL = 1 in mm_cfg.py which will cause Mailman to deliver each recipient to the outgoing MTA in a separate SMTP transaction instead of sending in one transaction with up to 500 recipients. You can accomplish the same thing by setting SMTP_MAX_RCPTS = 1 but if you're sending to 1 recipient per transaction, you might as well get the advantage of VERP like delivery for better bounce detection. If you have default Mailman settings, it is possible that Mailman is delivering all 200 Yahoo recipients to the MTA in a single transaction. Then, the MTA may or may not attempt to deliver all 200 to Yahoo in a single transaction which Yahoo may not like. If so, the above settings may help. If you want to slow things more than that, you need to investigate how to do it in the MTA. -- Mark Sapiro [EMAIL PROTECTED] The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
Michael Anderson wrote: I can't imagine that sending one message at a time to 1500 users on a real busy listserv would be a good idea.. Could it not really be slow? Well, you asked how to slow it down ... OTOH, it won't be that slow. If your MTA as as blinding fast as you say, I imagine that for moderate sized messages, say 10 KB, it would take well under a minute to deliver to the MTA. I don't know how many members are on the many lists at python.org, but many of these lists have personalized and/or VERPed delivery and thus send an individual message to each subscriber, and it doesn't seem to be an issue there. -- Mark Sapiro [EMAIL PROTECTED] The highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us
We changed our Sendmail configuration to send out one message at a time and it seems to be working fine so far. Thanks for all the help. MA From: Mark Sapiro [EMAIL PROTECTED] Organization: Not Very Much Date: Sat, 1 Sep 2007 15:02:12 -0700 To: Michael Anderson [EMAIL PROTECTED] Cc: Mailman Users mailman-users@python.org Subject: Re: [Mailman-Users] Comcast, Yahoo, Hotmail blocking us Michael Anderson wrote: I can't imagine that sending one message at a time to 1500 users on a real busy listserv would be a good idea.. Could it not really be slow? Well, you asked how to slow it down ... OTOH, it won't be that slow. If your MTA as as blinding fast as you say, I imagine that for moderate sized messages, say 10 KB, it would take well under a minute to deliver to the MTA. I don't know how many members are on the many lists at python.org, but many of these lists have personalized and/or VERPed delivery and thus send an individual message to each subscriber, and it doesn't seem to be an issue there. -- Mailman-Users mailing list Mailman-Users@python.org http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp
