Re: claim: qmail uses more bandwidth
On Tue, 12 Jan 1999, Adam D. McKenna wrote: I've posted messages to linux-smp that have taken over 12 hours to get posted (or at least to be returned to me..) The qmail mailing list, on the other hand, usually has a 1-2 second turnaround. Surely you jest? Do you have -any- idea how many deliveries, plus cvs sessions, vger.rutgers.edu deals with daily? Sorry, but the system handling the QMail mailing list doesn't even come close to that level of throughput load. Ask David; he and and the ZMailer developer (who also tunes vger's mail system) posted some stats today on the linux-kernel list regarding what that machine has to put up with. And now that anonymous CVS has been closed, deliveries are back to the 1-2 hour range. Not bad, considering the number of subscribers, the level of traffic, and the spread of exploders. -- Edward S. Marshall [EMAIL PROTECTED] [ What goes up, must come down. ] http://www.logic.net/~emarshal/ [ Ask any system administrator. ] Linux labyrinth 2.2.0-pre4 #1 Sun Jan 3 13:28:42 CST 1999 i586 unknown 11:20pm up 9 days, 8:01, 5 users, load average: 0.13, 0.03, 0.01
Re: claim: qmail uses more bandwidth
From: Edward S. Marshall [EMAIL PROTECTED] :On Tue, 12 Jan 1999, Adam D. McKenna wrote: : I've posted messages to linux-smp that have taken over 12 hours to get : posted (or at least to be returned to me..) : : The qmail mailing list, on the other hand, usually has a 1-2 second : turnaround. : :Surely you jest? Do you have -any- idea how many deliveries, plus cvs :sessions, vger.rutgers.edu deals with daily? Sorry, but the system :handling the QMail mailing list doesn't even come close to that level of :throughput load. Exactly which comment of mine did you have a problem with? recently list.cr.yp.to has sustained 1.5 million deliveries per day over a seven day period, without lagging. Even if vger.rutgers.edu is doing ten times that, (which I seriously doubt), it shouldn't take a matter of hours to get a message posted. This type of thing really affects the flow of conversation. :Ask David; he and and the ZMailer developer (who also tunes vger's mail :system) posted some stats today on the linux-kernel list regarding what :that machine has to put up with. I don't read the linux-kernel list, and the stats have not been posted here. If you intend to argue a point based on these stats, then post them here. :And now that anonymous CVS has been closed, deliveries are back to the 1-2 :hour range. Not bad, considering the number of subscribers, the level :of traffic, and the spread of exploders. I really don't see how the fact that they're running CVS off the same machine is relevant. If the machine is choking that badly, they should get some new hardware. Companies are practically begging to support linux development these days. Say whatever you want about qmail and zmailer, but please don't pretend that vger's current situation is "good enough". It needs to be fixed. If it can't be fixed, then someone with sufficient resources should take over. --Adam
Re: claim: qmail uses more bandwidth
On Wed, 13 Jan 1999, Dax Kelson wrote: If the list has a 20 second turn around like the qmail list, those things wouldn't happen. Actually it is more like 1-2 seconds.
Re: claim: qmail uses more bandwidth
I just had a flashback from every report I did in high school. --Adam - Original Message - From: Dax Kelson [EMAIL PROTECTED] To: Adam D. McKenna [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Wednesday, January 13, 1999 2:01 AM Subject: Re: claim: qmail uses more bandwidth :On Wed, 13 Jan 1999, Adam D. McKenna wrote: : : From: Edward S. Marshall [EMAIL PROTECTED] : :On Tue, 12 Jan 1999, Adam D. McKenna wrote: : : I've posted messages to linux-smp that have taken over 12 hours to get : : posted (or at least to be returned to me..) : : : : The qmail mailing list, on the other hand, usually has a 1-2 second : : turnaround. : : : :Surely you jest? Do you have -any- idea how many deliveries, plus cvs : :sessions, vger.rutgers.edu deals with daily? Sorry, but the system : :handling the QMail mailing list doesn't even come close to that level of : :throughput load. : : Exactly which comment of mine did you have a problem with? : : recently list.cr.yp.to has sustained 1.5 million deliveries per day over a : seven day period, without lagging. : : Even if vger.rutgers.edu is doing ten times that, (which I seriously doubt), : it shouldn't take a matter of hours to get a message posted. This type of : thing really affects the flow of conversation. : :The stats said the linux-kernel list has about 1.5 million deliveries per :day. With anon-cvs turned off the lag of posting is down from 30 hours to :1-2 hours, still totally unacceptable. : :It seriously degrades communication. For example a new pre patch comes :out with some typo in it and 40 people post about the problem because they :haven't received any messages to list mentioning that problem (yet). : :If the list has a 20 second turn around like the qmail list, those :things wouldn't happen. : :But like Russell said, David Miller has been refusing to even *look* at :qmail for 3 years. : :Dax Kelson : : : : :
Re: claim: qmail uses more bandwidth
On Wed, 13 Jan 1999, Dax Kelson wrote: The stats said the linux-kernel list has about 1.5 million deliveries per day. With anon-cvs turned off the lag of posting is down from 30 hours to 1-2 hours, still totally unacceptable. -- trimmed slightly From: Matti Aarnio [EMAIL PROTECTED] To: Brian Gerst [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: vger.rutgers.edu/mailing list Numbers of different recipients per top-domain in all of VGER's lists are listed here (top-10 in count), along with associated fanout relay: 633 ca vger.rutgers.edu 660 se ifi.uio.no 689 it nic.funet.fi 911 au samba.anu.edu.au 1176 org listserv.funet.fi 1226 uk ferret.lmh.ox.ac.uk 2634 de ferret.lmh.ox.ac.uk 2700 edu entropy.muc.muohio.edu 4475 net listserv.funet.fi 8119 com listserv.funet.fi We *tried* to run ORG, COM, and NET thru US located systems, but when those relays failed, we moved the traffic back to FUNET... By all means you are welcome to offer fanout relay service, but are you prepared for the load ? Mind you, VGER is delivering about 1.5 MILLION recipients per 24 hours through the fanouts, and some small (mostly US-based) top-levels by itself. The disk where all this is happening is not hottest possible (or perhaps it literally is that, but not fastest).. I did notice that since the weekend, the list has been flowing much better, presumably because of restricting CVS access on vger. Time to seperate the list and CVS on different machines? At least to separate spindles.. (disks, that is) Brian Gerst /Matti Aarnio [EMAIL PROTECTED] -- It seriously degrades communication. For example a new pre patch comes out with some typo in it and 40 people post about the problem because they haven't received any messages to list mentioning that problem (yet). If the list has a 20 second turn around like the qmail list, those things wouldn't happen. True, but it wouldn't happen with qmail. But like Russell said, David Miller has been refusing to even *look* at qmail for 3 years. qmail on vger would make having exploders rather pointless[1]. Currently, when a mail is received for linux-kernel, vger sends ~200 copies to subscribers and another dozen or so to exploders. If I'm not mistaken (maybe I am) qmail, in this situation would turn those dozen messages to exploders into around 3000. Matthew. [1] Except for dealing down down hosts, c.
Re: claim: qmail uses more bandwidth
On 13 Jan 1999, Russ Allbery wrote: I don't know about bandwidth, but I suspect that's not a big issue. vger's hardware is a SPARCclassic w/40MHz CPU, 128Mb RAM and (I believe) a single 3.5Gb disk. See me other post for information about list size. That's scary. My desktop machine is something like five or ten times faster than that. Admittedly, CPU matters hardly at all for mail delivery, but still. Someone should donate a Pentium with a better disk layout to the poor people. What's even more scary is that uptil a couple of years ago it all used to be run on SunOS 4.. Of course, load was lower then. All that's really needed is a seperate disk for either CVS or the mail queue. Matthew.
Re: claim: qmail uses more bandwidth
On 13-Jan-99 Matthew Kirkwood wrote: qmail on vger would make having exploders rather pointless[1]. Currently, when a mail is received for linux-kernel, vger sends ~200 copies to subscribers and another dozen or so to exploders. If I'm not mistaken (maybe I am) qmail, in this situation would turn those dozen messages to exploders into around 3000. ... [1] Except for dealing down down hosts, c. Ezmlm supports lists and sublists. They work very well and similar to what you are saying. So, yes you are mistaken. --- Pedro Melo [EMAIL PROTECTED] IP - Engenharia http://ip.pt/ Tel: +351-1-3166740 Av. Duque de Avila, 23 Fax: +351-1-3166701 1049-071 LISBOA - PORTUGAL 1:19pm up 8 days, 22:02, 6 users, load average: 0.05, 0.24, 0.34
Re: claim: qmail uses more bandwidth
Dax Kelson [EMAIL PROTECTED] writes on 13 January 1999 at 00:02:27 -0700 On Wed, 13 Jan 1999, Dax Kelson wrote: If the list has a 20 second turn around like the qmail list, those things wouldn't happen. Actually it is more like 1-2 seconds. It doesn't feel nearly that fast to me. I just happen to have the list copy of a message I sent still lying around. Unless I'm overlooking something important, comparing the sent and received timestamps from my qmail (on the same system, so same clock) should be a reasonable measure of the round-trip time on list messages. The headers are attached, but for this one message it was 6 seconds. That's considerably faster than it seems to me, but of course it's also just one data point (and of course your 1-2 seconds could be right on, and my 6 for this message is exceptional for some random reason). Anyway, in the spirit of "measure, don't speculate" I thought I'd contribute this to the bitstream. Received: (qmail 512 invoked from network); 13 Jan 1999 15:30:29 - Received: from muncher.math.uic.edu (131.193.178.181) by gw.ddb.com with SMTP; 13 Jan 1999 15:30:29 - Received: (qmail 15112 invoked by uid 1002); 13 Jan 1999 15:30:15 - Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm Precedence: bulk Delivered-To: mailing list [EMAIL PROTECTED] Received: (qmail 2774 invoked from network); 13 Jan 1999 15:30:15 - Received: from koobera.math.uic.edu ([EMAIL PROTECTED]) by muncher.math.uic.edu with SMTP; 13 Jan 1999 15:30:15 - Received: (qmail 29223 invoked by uid 666); 13 Jan 1999 15:30:47 - Delivered-To: [EMAIL PROTECTED] Received: (qmail 29219 invoked from network); 13 Jan 1999 15:30:46 - Received: from gw.ddb.com ([EMAIL PROTECTED]) by koobera.math.uic.edu with SMTP; 13 Jan 1999 15:30:46 - Received: (qmail 507 invoked by uid 501); 13 Jan 1999 15:30:23 - -- David Dyer-Bennet [EMAIL PROTECTED] http://www.ddb.com/~ddb (photos, sf) Minicon: http://www.mnstf.org/minicon http://ouroboros.demesne.com/ The Ouroboros Bookworms Join the 20th century before it's too late!
Re: claim: qmail uses more bandwidth
[EMAIL PROTECTED] [EMAIL PROTECTED] writes on 13 January 1999 at 09:45:51 -0600 Dax Kelson [EMAIL PROTECTED] writes on 13 January 1999 at 00:02:27 -0700 On Wed, 13 Jan 1999, Dax Kelson wrote: If the list has a 20 second turn around like the qmail list, those things wouldn't happen. Actually it is more like 1-2 seconds. It doesn't feel nearly that fast to me. I just happen to have the list copy of a message I sent still lying around. Unless I'm overlooking something important, comparing the sent and received timestamps from my qmail (on the same system, so same clock) should be a reasonable measure of the round-trip time on list messages. The headers are attached, but for this one message it was 6 seconds. And the turnaround on THAT message was 5 seconds. Received: (qmail 629 invoked from network); 13 Jan 1999 15:45:57 - Received: from muncher.math.uic.edu (131.193.178.181) by gw.ddb.com with SMTP; 13 Jan 1999 15:45:57 - Received: (qmail 24427 invoked by uid 1002); 13 Jan 1999 15:45:42 - Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm Precedence: bulk Delivered-To: mailing list [EMAIL PROTECTED] Received: (qmail 9781 invoked from network); 13 Jan 1999 15:45:42 - Received: from gw.ddb.com ([EMAIL PROTECTED]) by muncher.math.uic.edu with SMTP; 13 Jan 1999 15:45:42 - Received: (qmail 624 invoked by uid 501); 13 Jan 1999 15:45:52 - I promise I'll stop now :-) -- David Dyer-Bennet [EMAIL PROTECTED] http://www.ddb.com/~ddb (photos, sf) Minicon: http://www.mnstf.org/minicon http://ouroboros.demesne.com/ The Ouroboros Bookworms Join the 20th century before it's too late!
Re: claim: qmail uses more bandwidth
What operating system is running on muncher.math.uic.edu 30 emails a second is pretty high. Ken Jones Inter7
Re: claim: qmail uses more bandwidth
What operating system is running on muncher.math.uic.edu 30 emails a second is pretty high. Ken Jones Inter7 Not that it really matters, but: security [344] os muncher.math.uic.edu muncher.math.uic.edu: OpenBSD 2.2 - 2.4 Scott
Re: claim: qmail uses more bandwidth
Just let me drop in here for a german site: I synchronize my system-date at every dial-in with my ISP's. Oops, the turn took over 30 seconds, scandalous ;-)!! But as I dropped to the list just recently, so maybe I am No. 2000 on this list ... Regards Mirko Received: from muncher.math.uic.edu (muncher.math.uic.edu [131.193.178.181]) by mail.inka.de with smtp id 100RwR-0001od-00; Wed, 13 Jan 1999 16:10:43 +0100 Received: (qmail 16720 invoked by uid 1002); 13 Jan 1999 15:10:09 - Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm Precedence: bulk Delivered-To: mailing list [EMAIL PROTECTED] Received: (qmail 2898 invoked from network); 13 Jan 1999 15:10:08 - Received: from quechua.inka.de (HELO mail.inka.de) ([EMAIL PROTECTED]) by muncher.math.uic.edu with SMTP; 13 Jan 1999 15:10:08 - Received: from picard.inka.de (cxxxiii.yapay.inka.de [212.227.15.7]) by mail.inka.de with smtp id 100Rw2-0001oS-00; Wed, 13 Jan 1999 16:10:18 +0100 --
claim: qmail uses more bandwidth
FYI -- Forwarded message -- Date: Tue, 12 Jan 1999 17:45:15 + From: Nigel Metheringham [EMAIL PROTECTED] To: Dax Kelson [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], Aaron Tiensivu [EMAIL PROTECTED] Subject: Re: Qmail for Vger campaign :) Reality check folks. Firstly this is all academic, VGER doesn't run sendmail at all. Its mailing list performance is extraordinarily good considering the hardware base and other load on the system. Second qmail is incredibly unsuitable for an arrangement like this. If you think otherwise you are not in the real world. [Hint consider the current bandwidth used by the current VGER config with distributed exploders, now consider the fact you will multiply that bandwidth use by several orders of magnitude if you use qmail]. Qmail is not a seriously scalable mailer - its a damn good single system performer, but it sucks bandwidth for big lists and does not scale well. Thirdly this has sod all to do with kernel development. Have a private flamefest if you wish (ideally without me), but the mailing list is not benefiting from a time wasting discussion adding to the mailing list load. Nigel. -- [ [EMAIL PROTECTED] - Systems Software Engineer ] [ Tel : +44 113 207 6112 Fax : +44 113 234 6065 ] [ Real life is but a pale imitation of a Dilbert strip ] [ We're recruiting http://www.theplanet.net/profile/recruit.htm ]
Re: claim: qmail uses more bandwidth
On Tue, Jan 12, 1999 at 10:58:09AM -0700, Dax Kelson wrote: Firstly this is all academic, VGER doesn't run sendmail at all. Its mailing list performance is extraordinarily good considering the hardware base and other load on the system. Yep. Vger runs zmailer, which bogs down like all hell when it's queue exceeds about 20k, especially when some addresses are hard to reach. It also suffers from the bogus "optimization" of doing an extra dns lookup to order it's queue. This has been demonstrated to cost more then it gains in real usage (or at least, I've proved it to myself). Second qmail is incredibly unsuitable for an arrangement like this. If you think otherwise you are not in the real world. [Hint consider the current bandwidth used by the current VGER config with distributed exploders, now consider the fact you will multiply that bandwidth use by several orders of magnitude if you use qmail]. This strikes me as false, or at least incomplete. Qmail has additional capabilities that make remote list explosion pretty easy. With serialmail and smtproutes a vger-like hub-exploder setup should be doable with some work. Probably less time would have gone into developing the vger setup had qmail and its packages been around then. Also, using a patched qmail-qmqpc (read the qmqp host from env or file) and a qmqpd on the remote end list explosion could be made fast and cheap - faster and cheaper then using smtp, anyway. I think, anyway. I haven't had a chance to implement this. Qmail is not a seriously scalable mailer - its a damn good single system performer, but it sucks bandwidth for big lists and does not scale well. Neither is zmailer. In fact, zmailer is far less so, but they've stretched it a long way. In fact a lot farther then I'd have guessed it could go. -Peter
Re: claim: qmail uses more bandwidth
- "Peter C. Norton" [EMAIL PROTECTED]: | On Tue, Jan 12, 1999 at 10:58:09AM -0700, Dax Kelson wrote: No, he didn't. He just forwarded a piece of email from Nigel Metheringham. Tricky to get those attributions right sometimes, isn't it? - Harald
Re: claim: qmail uses more bandwidth
On Tue, Jan 12, 1999 at 08:31:39PM +0100, Harald Hanche-Olsen wrote: - "Peter C. Norton" [EMAIL PROTECTED]: | On Tue, Jan 12, 1999 at 10:58:09AM -0700, Dax Kelson wrote: No, he didn't. He just forwarded a piece of email from Nigel Metheringham. Tricky to get those attributions right sometimes, isn't it? *Sigh*, my apologies Dax. I got careless. But have coffee and a second battery for my portable, so I'll do better next time :) -Peter
Re: claim: qmail uses more bandwidth
Peter C. Norton writes: This strikes me as false, or at least incomplete. Qmail has additional capabilities that make remote list explosion pretty easy. With serialmail and smtproutes a vger-like hub-exploder setup should be doable with some work. Probably less time would have gone into developing the vger setup had qmail and its packages been around then. They've been refusing to use qmail for almost three years now: Date: Wed, 24 Jan 96 22:35:14 EST From: nelson (Russell Nelson) To: "David S. Miller" [EMAIL PROTECTED] Subject: Re: fast mailer In-Reply-To: [EMAIL PROTECTED] References: [EMAIL PROTECTED] [EMAIL PROTECTED] David S. Miller writes: Date: Wed, 24 Jan 96 22:32 EST From: [EMAIL PROTECTED] (Russell Nelson) Dan Bernstein has written a new mailer called qmail. It's in ftp://koobera.math.uic.edu/pub/software. It's still beta test software, but it seems to blow the doors off any other mail transport agent I've ever seen. He says that it makes zmailer's speed look pathetic. The mailing list for beta testers is run by qmail, and it's astoundingly fast. I'll look into it as time permits (ie. I wont be able to) I love a man with a little optimism. And you have little optimism. :) -- -russ nelson [EMAIL PROTECTED] http://crynwr.com/~nelson Crynwr supports Open Source(tm) Software| PGPok | There is good evidence 521 Pleasant Valley Rd. | +1 315 268 1925 voice | that freedom is the Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | cause of world peace.
Re: claim: qmail uses more bandwidth
I've posted messages to linux-smp that have taken over 12 hours to get posted (or at least to be returned to me..) The qmail mailing list, on the other hand, usually has a 1-2 second turnaround. --Adam
