Re: High Volume....
On Mon, Jul 02, 2001 at 11:37:03AM +0200, Xavier Pegenaute wrote: Hi all.. i can try any prime number for hashing ..? ( is limited?) You can try any number, but anything over 1/1000th of your queuesize is overkill. The default of 23 is therefore suitable for about 23.000 messages. and, is possible have two qmail over the same queue ..? No. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
Re: High Volume....
On Mon, Jul 02, 2001 at 01:18:12PM +0100, Jose Celestino wrote: http://www-archive.ornl.gov:8000 If you have nothing to say, don't say it. And if you insist on saying it, at least learn how to quote correctly. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
Re: High Volume....
On Tue, Jul 03, 2001 at 09:11:33AM +0200, Peter van Dijk wrote: On Mon, Jul 02, 2001 at 11:37:03AM +0200, Xavier Pegenaute wrote: Hi all.. i can try any prime number for hashing ..? ( is limited?) You can try any number, but anything over 1/1000th of your queuesize is overkill. I thought it would be better to take a number about the square root of the top queue size, which minimizes search length on a classical file system. For 23000 your on the order of 151, which happens to look prime. Jost -- | [EMAIL PROTECTED] Please help stamp out spam! | | Postmaster, JAPH, resident answer machine am RZ der RUB | | Pluralitas non est ponenda sine necessitate | | William of Ockham (1285-1347/49) |
Re: High Volume....
On Tue, Jul 03, 2001 at 12:45:09PM +, Jost Krieger wrote: [snip] I thought it would be better to take a number about the square root of the top queue size, which minimizes search length on a classical file system. For 23000 your on the order of 151, which happens to look prime. Search length is O((number of messages)/(conf-split)+(conf-split)). Why? If qmail wants to access message 12345, which is, for example, in 5/12345, the OS has to: - linear search to find directory 5/ - inside this dir, linear search for 12345 Finding 5/ will take O(conf-split) steps. Finding 12345 within that dir will take O((number of messages)/(conf-split)) steps. This is all on classical filesystems. On filesystems like ReiserFS that use search trees and other fancy stuff, it all basically comes down to O(lg n), or even O(1) with hashing. Greetz, Peter -- Against Free Sex! http://www.dataloss.nl/Megahard_en.html
High Volume....
Hi all.. i can try any prime number for hashing ..? ( is limited?) and, is possible have two qmail over the same queue ..? Thanks for all..
Re: High Volume....
http://www-archive.ornl.gov:8000 Thus spake Xavier Pegenaute, on Mon, Jul 02, 2001 at 11:37:03AM +0200: Hi all.. i can try any prime number for hashing ..? ( is limited?) and, is possible have two qmail over the same queue ..? Thanks for all.. -- Jose Celestino [EMAIL PROTECTED] - The paradox render and the merge in complete, Nothing but the process is infinite -- Borknagar - Colossus
Re: High Availability, High Volume and NFS
Hi, I'm from netapp, but what I can say for you, a netapp is a dedicated box for this kind of usage (high load) if you have any questions don't hesitate Thx Fab. -Original Message- From: John White [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 24, 2001 12:49 AM To: [EMAIL PROTECTED] Subject: Re: High Availability, High Volume and NFS On Wed, May 23, 2001 at 01:40:13PM -0500, Duane Schaub wrote: We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. Others may point out that an observed weakness of the stock linux kernel from RH 6.1 has been shown to have weak NFS performance when compared to some of the BSD O/S family. If you feel comfortable recompiling your kernel, check out http://nfs.sourceforge.net/ The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. Once qmail exceeded about 70 sessions, it was beyond the point of return and would not recover. Have you thought about the stopgap measure of throttling down on the number of concurrent pop3 sessions each machine is allowed? Say you want to cap it at 50 total. Just use 50/n, where n is the number of client machines, as the max concurrency for tcpserver (-c). You can increase the client backlog so all the clients see is a pause (-b). http://cr.yp.to/ucspi-tcp/tcpserver.html The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. It will be tempting to throw more hardware at the problem. Depends on your budget. Right now, I like the the new DDR RAM chipsets for Athlon processors. I like the idea of 3ware hardware IDE RAID which looks like a SCSI controller to the system. Balance the bugetary requirements of upgrading your hardware (without knowing what the effect will be) vs. changing your O/S (with some benchmarking already in hand). Oh, check this out: http://innominate.org/%7Etgr/projects/tuning/ Check out slide 37 for relevant conclusions, but the entire presentation is interesting. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? I've read repeated positive reviews with a netapp, but I still would explore FreeBSD performance first. John White
Re: High Availability, High Volume and NFS
On 23 May 2001, Mark Delany wrote:
I don't want to start an OS war, but if you want to use NFS on an
Intel box, I strongly suggest one of the BSDs. I was in a situation
where I had to use Linux NFS servers - that was until they failed
miserabled. They were replaced with FreeBSD and the problems went
away.
Also, check how your OS supports turning UDP checksumming off, and make
sure it's off. It's of no great help on a local switched segment and
affects NFS performance. Are you using traditional NFS or TCP-based NFS?
If performance is the real desired goal, UDP-based NFS is going to be a
lot faster, if not as secure. But, hey, once you're in NFS-land, you're
going to want to keep it all on a tight, local segment if security's even
vaguely your issue.
Finally, you may want to put the server on a gig connection into the
switch, and the client servers on the same switch on 100Mbit FDX.
-M
Regards.
On Wed, May 23, 2001 at 01:40:13PM -0500, Duane Schaub allegedly wrote:
I want to set up multiple qmail machines to access an NFS backend. We have
about 10,000 users (running maildir) and an average of 5 emails/user/dat and
av. 10K in size. On average, there are 6 simultaneous pop sessions with
approx. 200 new sessions/min.
We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients.
The result was that the qmail machines were BARELY able to keep up. If
there were any pauses on the NFS server, the POP sessions would build to
50-60 very quickly with qmail crashing at about 300 sessions. Once qmail
exceeded about 70 sessions, it was beyond the point of return and would not
recover.
The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a
Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the
performance was abysmal! Performing an ls in a user/new directory took 21
seconds for a response.
I think NFS would work, but I don't really want a Netapp F5 ($50,000). What
NFS experiences are out there?
If you wish - respond privately [EMAIL PROTECTED]
Duane.
Michael Brian Scher [EMAIL PROTECTED] [EMAIL PROTECTED]
Sr. Research Consultant
Attorney, Anthropologist, Part-Time Guru
Mailaise: n, ('mail-aze). See Outlook.
Re: High Availability, High Volume and NFS
On Wed, May 23, 2001 at 01:40:13PM -0500, Duane Schaub wrote: We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. Others may point out that an observed weakness of the stock linux kernel from RH 6.1 has been shown to have weak NFS performance when compared to some of the BSD O/S family. If you feel comfortable recompiling your kernel, check out http://nfs.sourceforge.net/ The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. Once qmail exceeded about 70 sessions, it was beyond the point of return and would not recover. Have you thought about the stopgap measure of throttling down on the number of concurrent pop3 sessions each machine is allowed? Say you want to cap it at 50 total. Just use 50/n, where n is the number of client machines, as the max concurrency for tcpserver (-c). You can increase the client backlog so all the clients see is a pause (-b). http://cr.yp.to/ucspi-tcp/tcpserver.html The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. It will be tempting to throw more hardware at the problem. Depends on your budget. Right now, I like the the new DDR RAM chipsets for Athlon processors. I like the idea of 3ware hardware IDE RAID which looks like a SCSI controller to the system. Balance the bugetary requirements of upgrading your hardware (without knowing what the effect will be) vs. changing your O/S (with some benchmarking already in hand). Oh, check this out: http://innominate.org/%7Etgr/projects/tuning/ Check out slide 37 for relevant conclusions, but the entire presentation is interesting. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? I've read repeated positive reviews with a netapp, but I still would explore FreeBSD performance first. John White
High Availability, High Volume and NFS
I want to set up multiple qmail machines to access an NFS backend. We have about 10,000 users (running maildir) and an average of 5 emails/user/dat and av. 10K in size. On average, there are 6 simultaneous pop sessions with approx. 200 new sessions/min. We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. Once qmail exceeded about 70 sessions, it was beyond the point of return and would not recover. The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? If you wish - respond privately [EMAIL PROTECTED] Duane. President, | Terra World, Inc. Terra World, Inc.| 200 ARCO Place, Suite 252 (888)332-1616| Independence, KS 67301 (620)332-1616| When your work counts, Use www.terraworld.net |T E R R A W O R L D
Re: High Availability, High Volume and NFS
I want to set up multiple qmail machines to access an NFS backend. We have about 10,000 users (running maildir) and an average of 5 emails/user/dat and av. 10K in size. On average, there are 6 simultaneous pop sessions with approx. 200 new sessions/min. I'm working on something similar. The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? Well, our solution was to use an existing EMC Cellera, but I think that's beyond your means :) On the other hand, if you go with a speedy CPU (Athalon 800Mhz+, I think), and at least 1GB of RAM, and a really, really good NIC (I like the NetGear FA310TX, and I think they've come out with a newer one), you should be ok. If you wish - respond privately [EMAIL PROTECTED] Duane. Dave
Re: High Availability, High Volume and NFS
I don't want to start an OS war, but if you want to use NFS on an Intel box, I strongly suggest one of the BSDs. I was in a situation where I had to use Linux NFS servers - that was until they failed miserabled. They were replaced with FreeBSD and the problems went away. Regards. On Wed, May 23, 2001 at 01:40:13PM -0500, Duane Schaub allegedly wrote: I want to set up multiple qmail machines to access an NFS backend. We have about 10,000 users (running maildir) and an average of 5 emails/user/dat and av. 10K in size. On average, there are 6 simultaneous pop sessions with approx. 200 new sessions/min. We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. Once qmail exceeded about 70 sessions, it was beyond the point of return and would not recover. The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? If you wish - respond privately [EMAIL PROTECTED] Duane. President, | Terra World, Inc. Terra World, Inc.| 200 ARCO Place, Suite 252 (888)332-1616| Independence, KS 67301 (620)332-1616| When your work counts, Use www.terraworld.net |T E R R A W O R L D
Re: High Availability, High Volume and NFS
Duane Schaub [EMAIL PROTECTED] wrote: I want to set up multiple qmail machines to access an NFS backend. We have about 10,000 users (running maildir) and an average of 5 emails/user/dat and av. 10K in size. 50,000 messages a day, 10k each? Not a huge load. We handle about a tenth of that on one machine with a P733 and SCSI disks. System load is typically 0.1 (i.e., negligible). On average, there are 6 simultaneous pop sessions with approx. 200 new sessions/min. This could be the big problem. We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. You're not using the default kernel shipped with RH6.1, are you? That would be 2.2.5-something. Early 2.2 kernels are known to have absolutely terrible NFS performance as NFS servers, and client-side isn't a whole lot better. NFS has gotten somewhat better in 2.2.19, but server-side is still not up to snuff. You may want to run OpenBSD or FreeBSD as the NFS server. The NFS server was nothing special (P350/IDE 256Mb RAM). This is your biggest problem. This is not a performance workstation class machine, let alone a high-performance server. You'll notice a huge difference just in switching to SCSI with Linux, even cheap 7200rpm SCSI disks on an average controller. For your load, I would recommend good 15kRPM SCSI disks in a hardware RAID setup (not software RAID). This would probably cure your performance issues on the NFS server. Adding additional RAM would likely benefit as well, as more of the filesystem could be cached. Memory is so cheap these days that you might want to jump right to 1GB. The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. If qmail is crashing (prove it!) due to POP3 load, you've seriously misconfigured your server. You should be using tcpserver's concurrency limits to enforce a maximum limit on concurrent POP3 connections to a level that your system can actually handle. qmail itself doesn't crash for any reason related to POP3 access. If you wish - respond privately [EMAIL PROTECTED] No, I think I'll keep it on the list. That way others can benefit from your experiences and learn from them, rather than asking the same question next week, next month, next year... Charles -- --- Charles Cazabon[EMAIL PROTECTED] GPL'ed software available at: http://www.qcc.sk.ca/~charlesc/software/ Any opinions expressed are just that -- my opinions. ---
RE: High Availability, High Volume and NFS
Unleash the daemon! ;) -Original Message- From: Mark Delany [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 23, 2001 2:12 PM To: Qmail List Subject: Re: High Availability, High Volume and NFS I don't want to start an OS war, but if you want to use NFS on an Intel box, I strongly suggest one of the BSDs. I was in a situation where I had to use Linux NFS servers - that was until they failed miserabled. They were replaced with FreeBSD and the problems went away. Regards. On Wed, May 23, 2001 at 01:40:13PM -0500, Duane Schaub allegedly wrote: I want to set up multiple qmail machines to access an NFS backend. We have about 10,000 users (running maildir) and an average of 5 emails/user/dat and av. 10K in size. On average, there are 6 simultaneous pop sessions with approx. 200 new sessions/min. We have tried a Redhat6.1 backend on the NFS with Redhat 6.1 NFS clients. The result was that the qmail machines were BARELY able to keep up. If there were any pauses on the NFS server, the POP sessions would build to 50-60 very quickly with qmail crashing at about 300 sessions. Once qmail exceeded about 70 sessions, it was beyond the point of return and would not recover. The NFS server was nothing special (P350/IDE 256Mb RAM). We also tried a Dell 2300 (Dual 400/RAID5) NT server running Intergraph NFS But the performance was abysmal! Performing an ls in a user/new directory took 21 seconds for a response. I think NFS would work, but I don't really want a Netapp F5 ($50,000). What NFS experiences are out there? If you wish - respond privately [EMAIL PROTECTED] Duane. President, | Terra World, Inc. Terra World, Inc.| 200 ARCO Place, Suite 252 (888)332-1616| Independence, KS 67301 (620)332-1616| When your work counts, Use www.terraworld.net |T E R R A W O R L D
Re: Handling high volume lists
Andy Bradford [EMAIL PROTECTED] writes: list especially has been known to have delays of up to at least 3 hours before emails that are sent actually show up in my mailbox. I still remember the days where it was nearly always below 30 seconds. Flamewars developed much faster :) Regards, Frank
Re: Handling high volume lists
Andy Bradford [EMAIL PROTECTED] wrote: It wouldn't be hard to configure your Gnus to add the mail-followup-to header for this list. Better yet, create a file containing a list of mailing lists, e.g.: [EMAIL PROTECTED] [EMAIL PROTECTED] ... And set the QMAILMFTFILE environment variable to the name of that file. -Dave
Re: Handling high volume lists (was: Newbies vs. arrogant experts)
On Sun, May 13, 2001 at 10:18:45AM +0200, Robin S. Socha [EMAIL PROTECTED] wrote: · dupes (like, I am on this list and only a complete retard would send me a Cc: (which makes approx. 31 retards per month which, in return, makes me wonder when the prices for anti-personnel ammo will finally drop)). Maybe I missed it, but I didn't see a Mail-Followup-To header in your message. Are people supposed to guess whether or not you are on the list or whether or not you want a cc header listing your address so that your mail filter can handle the message differently. Since you made a point about not wanting cc's I manually removed it from this message.
Re: Handling high volume lists
* Bruno Wolff III [EMAIL PROTECTED]: On Sun, May 13, 2001 at 10:18:45AM +0200, Robin S. Socha [EMAIL PROTECTED] wrote: [dupes] Maybe I missed it, but I didn't see a Mail-Followup-To header in your message. Are people supposed to guess whether or not you are on the list or whether or not you want a cc header listing your address so that your mail filter can handle the message differently. Everyone here is on this list unless otherwise stated. Noone therefore wants Cc:s unless otherwise stated. How long have you been in a technical environment? Since you made a point about not wanting cc's I manually removed it from this message. You are one hell of a luser. ## lists adds a list of mailing lists addresses ## so mutt knows about these for showing them in the folder indes ## and to allow replying to them with the command list-reply. lists qmail -- Robin S. Socha - Your Worst Network Nightmare(tm). `In Germany, they are not referred to as network administrators. They prefer to be called Sons Of The Third Reich.' (Kate: www.katewerk.com)
Re: Handling high volume lists
From: Robin S. Socha [EMAIL PROTECTED] Date: 14 May 2001 22:30:00 +0200 * Bruno Wolff III [EMAIL PROTECTED]: On Sun, May 13, 2001 at 10:18:45AM +0200, Robin S. Socha [EMAIL PROTECTED] et wrote: [dupes] Maybe I missed it, but I didn't see a Mail-Followup-To header in your message. Are people supposed to guess whether or not you are on the list or whether or not you want a cc header listing your address so that your mail filter can handle the message differently. Everyone here is on this list unless otherwise stated. Noone therefore wants Cc:s unless otherwise stated. How long have you been in a technical environment? Since you made a point about not wanting cc's I manually removed it from this message. You are one hell of a luser. Cool... I'm glad I'm not the only one declared one hell of a luser by Robin this week! I'd hate to be the only one...it would make me feel soinadequate. I'd like to state for the record that I'm on the list and I want CC's because if I'm actually providing something useful to a conversation, the sooner I get the other side's messages, the sooner I can say something useful. I leave the headers unedited because that means the default behavior of most mail clients will be to do exactly what I want. I'm not cc'ing Robin even though he hasn't put the appropriate headers on his message because I see no need to accelerate his ability to flame me. (Besides, I'm already a luser, so what more do I have to luse?) Chris -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 4314 Avenue C Austin, TX 78751-3709 +1 512 374 0500 My email address is an experiment in SPAM elimination. For an explanation of what we're doing, see http://www.DeepEddy.Com/tms.html Nobody ever got fired for buying Microsoft, but they could get fired for relying on Microsoft. PGP signature
Re: Handling high volume lists
Everyone here is on this list unless otherwise stated. Noone therefore wants Cc:s unless otherwise stated. How long have you been in a technical environment? http://www.unicom.com/pw/reply-to-harmful.html You are one hell of a luser. pots, kettles, black jason
Re: Handling high volume lists
* Robin S. Socha [EMAIL PROTECTED] [010514 17:13]: * Bruno Wolff III [EMAIL PROTECTED]: On Sun, May 13, 2001 at 10:18:45AM +0200, Robin S. Socha [EMAIL PROTECTED] wrote: [dupes] Maybe I missed it, but I didn't see a Mail-Followup-To header in your message. Are people supposed to guess whether or not you are on the list or whether or not you want a cc header listing your address so that your mail filter can handle the message differently. Everyone here is on this list unless otherwise stated. Noone therefore wants Cc:s unless otherwise stated. How long have you been in a technical environment? Robin, really. Last I checked the qmail mailing list made no such effort to determine whether or not a poster is subscribed. Even if it does, other lists (even technical ones!) don't. Lighten up man, go smoke some bud or something. /pg -- Peter Green : Architekton Internet Services, LLC : [EMAIL PROTECTED] --- I'd rather be rich than stupid. (Jack Handey)
Re: Handling high volume lists
Thus said Robin S. Socha on 14 May 2001 22:30:00 +0200: Maybe I missed it, but I didn't see a Mail-Followup-To header in your message. Are people supposed to guess whether or not you are on the list or whether or not you want a cc header listing your address so that your mail filter can handle the message differently. Everyone here is on this list unless otherwise stated. Noone therefore wants Cc:s unless otherwise stated. How long have you been in a technical environment? I disagree completely. There are many cases in which I have greatly appreciated a Cc on mailing lists on which I participate because they generally arrive much sooner than the mailing list answer does. This list especially has been known to have delays of up to at least 3 hours before emails that are sent actually show up in my mailbox. That doesn't help much when the critical problem has escalated and my hair has fallen out. It wouldn't be hard to configure your Gnus to add the mail-followup-to header for this list. Even though it isn't quite a standard and probably never will be, I know a lot of users on this list have software that respects it. I know mine does. Andy -- [---[system uptime]] 8:14pm up 5 days, 22:51, 5 users, load average: 1.11, 1.18, 1.20
Handling high volume lists (was: Newbies vs. arrogant experts)
* Russell Nelson [EMAIL PROTECTED] [010512 20:57]: Chris Garrigues writes: As it is, I consider unsubscribing several times a week (and it's not because of the newbies). I send qmail list traffic into its own mailbox, and read it once a day. It's kind of handy, because I can see the questions which don't get answered, and sometimes I answer them if I feel so led. There are several ways of dealing with high volume lists (which in the Microsoft Outlook (Express) age equals high amount of lusers messing with stuff they should never be exposed to). I've been getting between 500 and 2500 messages (mail and news) per day for years, and I've found that there is only one tool that does the job effectively: Gnus http://gnus.org/.· Why? There are several problems with high volume lists. In no particular order: · lusers not reading the minimum amount of documentation; · broken software (why did you not cut the (was: (which should have been (Was: in the first place) from your quote?). Examples: - removing References: headers in order to, well, what? break threads (a concept alien to the morons in Redmond, but extremely useful - if you have a good newsreader, you know what I'm talking about); - encouraging *wrong* behaviour like message on top, full quote below (eh, bandwidth is unlimited, right?), excessive signatures (yes, 4 lines *is* enough, no, I will *never* ring you up or visit you) without sigdashes (yes, that's ^-- $, and yes, it *does* make sense, because various tools can cut your sig from replies or (even better) make it invisible if it sucks), sending HTML (what is this - the web?) or VCards (as I said, I will *not* phone or visit you, and no, I'm not interested in your birthday, favourite sexual deviation or daughter's name either (if she's willing and blonde, you can send me some bikini shots, though); · off topic threads (like this one); · idiots and trolls (usually mutually interchangable, but stupid trolls are even more annoying (greetings to Michael T. Babcock while we're at it (remember kids: no lame flame without at least one ad hominem attack))); · dupes (like, I am on this list and only a complete retard would send me a Cc: (which makes approx. 31 retards per month which, in return, makes me wonder when the prices for anti-personnel ammo will finally drop)). Solutions: == · Encourage people who are obviously lost to adapt a behaviour fitting for a technical environment: tell them to read and follow http://learn.to/edit_messgages/ and the links therein; · Tell them why using Outlook and similar programs is not only bad for themselves but also destroys threads, which means archives, which means shared knowledge databases; · Use software that lives up to the challenge of a Microsoft-luser infested environment: Gnus. - Gnus introduces the concept of adaptive scoring. Killfiles are for lusers, real men score: each article is assigned a value for From:, Subject:, References:, quote/text ratio, etc. and the summary for the group the article is filtered into is sorted according to these values: Gods first, lusers last. - Gnus automagically de-moronizes messages (while reading and in replies): * convert RE: AW: SE: to Re:; * cut (Was:); * wrap long lines; * nuke HTML, VCards etc.; * remove spurious blank lines; * ... - Gnus works with IMAP (including mail splitting) and Maildir (courtesy of Paul Jarc's nnmaildir.el - *great* work!). Ummm... are we on topic, yet? Good. - Gnus lets you merge mailing lists into thematically similar groups. - Gnus lets you read mail like news (including expiry). Possible additional weapons include procmail[1] and the BBDB (for marking posters known to you). Anyway, I sometimes wonder what makes people think they deserve help if they don't invest a minimum amount of time and diligence into writing their mails to technical mailing lists. The qmail list is a particularly sad example with a group of *extremely* competent and helpful (I've /never/ had to wait for more than 6h for an answer to my problems (*all* of which were solved)) people being swamped by a tidal wave of morons asking FAQs, not knowing their way around the Internet or using orthography and grammar in a way that indicates a grasp of the English language that will effectively bar them from a deeper understanding of qmail, anyway. Yes, people like me who are fairly new to qmail and had rather have these people answer meaningful (i.e. I can learn something from this rather than wow, those are either extremely nice or masochistic people) questions get very frustrated over this. But it's much easier to whine and groan, Chris, instead of taking some action, isn't it? After all, your post was brought to us via softmail written by the same k3wl D00d3 who invented the Internet (no, not Al Treehugger Gore, but the Great Chairman Gill Bates
Re: Handling high volume lists (was: Newbies vs. arrogant experts)
From: Robin S. Socha [EMAIL PROTECTED] Date: 13 May 2001 10:18:45 +0200 * Russell Nelson [EMAIL PROTECTED] [010512 20:57]: Chris Garrigues writes: As it is, I consider unsubscribing several times a week (and it's not because of the newbies). I send qmail list traffic into its own mailbox, and read it once a day. It's kind of handy, because I can see the questions which don't get answered, and sometimes I answer them if I feel so led. ... But it's much easier to whine and groan, Chris, instead of taking some action, isn't it? After all, your post was brought to us via softmail written by the same k3wl D00d3 who invented the Internet (no, not Al Treehugger Gore, but the Great Chairman Gill Bates himself), so it *has* to be okay, eh? Not. [ Since you mentioned me by name, I guess i won't ignore your generally useful message with a generally negative tone. ] I have no idea what you mean by softmail written by [Bill Gates]. My post was written in exmh which is layered on top of mh running under Linux and delivered using qmail via a firewall/mailrelay also running qmail to a mailing list running ezmlm and qmail to you. If there was anything written by Bill Gates, it's not on my end. Clearly you didn't look at the headers of my original message, and in fact, I'll assume you didn't even read it since you apparently decided for no good reason that I must be a user of microsoft software. Allow me to quote myself from the message that you ignored: I've been on this list now since late 1996 and in recent times, it's become almost intolerable with all the flamage. Somehow on other lists people manage to co-exist with newbies without having to extract a pound of flesh with every question. I suspect that if this list had been this rude in 1996, I would have stuck with sendmail. Most likely you looked at the one sentence that Russell quoted and thought to your self: Hey! An opportunity to prove to some whinny little twit how much smarter he'd be if he used the cool tools that I use. It's exactly this attitude that I was referring to in the parenthetical comment that Russell quoted. Can't we try to give the benefit of a doubt instead of looking at every post as an opportunity to prove that we have bigger opensource balls than the next guy? -- Chris Garrigues http://www.DeepEddy.Com/~cwg/ virCIO http://www.virCIO.Com 4314 Avenue C Austin, TX 78751-3709 +1 512 374 0500 My email address is an experiment in SPAM elimination. For an explanation of what we're doing, see http://www.DeepEddy.Com/tms.html Nobody ever got fired for buying Microsoft, but they could get fired for relying on Microsoft. PGP signature
Re: Recommended patches for high-volume ezmlm server
On Fri, Mar 09, 2001 at 10:24:11AM -0600, Mate Wierdl wrote: On Thu, Mar 08, 2001 at 09:48:22PM +0100, Peter van Dijk wrote: On Thu, Mar 08, 2001 at 12:16:52PM -0800, Don Rose wrote: I am building a server to house several very high volume mailing lists (2-4M users each) and wanted to know which patches were recommended for use with ezmlm and ezmlm-idx, as well as qmail itself. I have read about the big-concurrency patch and that seems relevant, but I'm not sure about the others. No others are. I am theorizing: having millions of users means lots of bad addresses. So now, when ezmlm-warn sends out its gripes, it may mean a few hundred thousand separate messages in the queue. Is that a load in the queue not to worry about? Those don't get send out all at the same time. And your theory is broken - ezmlm subscriber lists are almost by definition high-quality. Greetz, Peter.
Re: Recommended patches for high-volume ezmlm server
On Fri, Mar 09, 2001 at 08:40:53AM -0800, Don Rose wrote: Actually the lists are only announcement-type lists so users won't be posting, so the only messages that should be queued are like you said the bounce probes, and the messages the mods post to it. I was thinking of setting up 2 or 3 QMQP servers on a LocalDirector to handle the actual sending of the messages, so the original machine isn't too busy to recieve new mail. Using a LocalDirector for QMQP is overkill. If one of your QMQP servers is down, the 'clients' will use another one automatically. This just means a slight delay, and nothing to worry about. Greetz, Peter.
RE: Recommended patches for high-volume ezmlm server
So having multiple QMQP machines as opposed to a single machine wouldn't deliver the mail faster? We've already got the LocalDirector in place doing other things and we wanted to utilize it for this if we could. For this usage, its not a matter of redundant failover, but more a matter of load balancing, so that 3 machines can deliver millions of emails faster than a single one could. Am I wrong in my thinking here? -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Saturday, March 10, 2001 3:53 AM To: '[EMAIL PROTECTED]' Subject: Re: Recommended patches for high-volume ezmlm server On Fri, Mar 09, 2001 at 08:40:53AM -0800, Don Rose wrote: Actually the lists are only announcement-type lists so users won't be posting, so the only messages that should be queued are like you said the bounce probes, and the messages the mods post to it. I was thinking of setting up 2 or 3 QMQP servers on a LocalDirector to handle the actual sending of the messages, so the original machine isn't too busy to recieve new mail. Using a LocalDirector for QMQP is overkill. If one of your QMQP servers is down, the 'clients' will use another one automatically. This just means a slight delay, and nothing to worry about. Greetz, Peter.
Re: Recommended patches for high-volume ezmlm server
On Thu, Mar 08, 2001 at 09:48:22PM +0100, Peter van Dijk wrote: On Thu, Mar 08, 2001 at 12:16:52PM -0800, Don Rose wrote: I am building a server to house several very high volume mailing lists (2-4M users each) and wanted to know which patches were recommended for use with ezmlm and ezmlm-idx, as well as qmail itself. I have read about the big-concurrency patch and that seems relevant, but I'm not sure about the others. No others are. I am theorizing: having millions of users means lots of bad addresses. So now, when ezmlm-warn sends out its gripes, it may mean a few hundred thousand separate messages in the queue. Is that a load in the queue not to worry about? Mate
RE: Recommended patches for high-volume ezmlm server
Actually the lists are only announcement-type lists so users won't be posting, so the only messages that should be queued are like you said the bounce probes, and the messages the mods post to it. I was thinking of setting up 2 or 3 QMQP servers on a LocalDirector to handle the actual sending of the messages, so the original machine isn't too busy to recieve new mail. -Original Message- From: Mate Wierdl [mailto:[EMAIL PROTECTED]] Sent: Friday, March 09, 2001 8:24 AM To: [EMAIL PROTECTED] Subject: Re: Recommended patches for high-volume ezmlm server On Thu, Mar 08, 2001 at 09:48:22PM +0100, Peter van Dijk wrote: On Thu, Mar 08, 2001 at 12:16:52PM -0800, Don Rose wrote: I am building a server to house several very high volume mailing lists (2-4M users each) and wanted to know which patches were recommended for use with ezmlm and ezmlm-idx, as well as qmail itself. I have read about the big-concurrency patch and that seems relevant, but I'm not sure about the others. No others are. I am theorizing: having millions of users means lots of bad addresses. So now, when ezmlm-warn sends out its gripes, it may mean a few hundred thousand separate messages in the queue. Is that a load in the queue not to worry about? Mate
Recommended patches for high-volume ezmlm server
I am building a server to house several very high volume mailing lists (2-4M users each) and wanted to know which patches were recommended for use with ezmlm and ezmlm-idx, as well as qmail itself. I have read about the big-concurrency patch and that seems relevant, but I'm not sure about the others. Can someone give me a heads up? Thank you.
Re: Recommended patches for high-volume ezmlm server
On Thu, Mar 08, 2001 at 12:16:52PM -0800, Don Rose wrote: I am building a server to house several very high volume mailing lists (2-4M users each) and wanted to know which patches were recommended for use with ezmlm and ezmlm-idx, as well as qmail itself. I have read about the big-concurrency patch and that seems relevant, but I'm not sure about the others. No others are. Greetz, Peter.
Re: high volume server configurations
The reason I have 200 in the conf-split so 200 sub queue directories will be created to increases file access time by reducing inode table seek time. I also went ahead a made the file system of /var/qmail/queue xfs.. Peter van Dijk wrote: On Wed, Feb 07, 2001 at 01:41:17PM -0600, Sid Wilroy wrote: Can someone point me to a doc of how to optimize qmail for relaying only? I don't need local delivery - nada... We are trying to get 25 mailings per second.. What I have done thus far is: 200 in conf-split conf-split should be prime. I don't think you know what conf-split means. Higher is not always better. I also applied these patches: qmail-1.03]# patch -p1 /usr/local/src/big-todo.103.patch qmail-1.03]# patch -p1 /usr/local/src/big-concurrency.patch I got this error: 1 out of 5 hunks FAILED -- saving rejects to file spawn.c.rej Try -p0, perhaps. Greetz, Peter.
Re: high volume server configurations
On Thu, Feb 15, 2001 at 02:12:14PM -0600, Sid Wilroy wrote: The reason I have 200 in the conf-split so 200 sub queue directories will be created to increases file access time by reducing inode table seek time. conf-split should be a *prime* number. Also, a large conf-split only makes sense if you have more than 20.000 messages *in your queue*. This won't usually happen. I also went ahead a made the file system of /var/qmail/queue xfs.. That might be a good idea indeed. It also takes away most of the reasons for a big conf-split. Greetz, Peter.
why prime? [was high volume server configurations)
How come the conf-split should be prime? I've read it and (unfortunately) repeatedly ignored. And does it hamper things greatly by it not being so (yet)? -- Michael Boyiazis [EMAIL PROTECTED] Mail Architect, NetZero, Inc. -Original Message- From: Peter van Dijk [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 15, 2001 2:22 PM To: [EMAIL PROTECTED] Subject: Re: high volume server configurations On Thu, Feb 15, 2001 at 02:12:14PM -0600, Sid Wilroy wrote: The reason I have 200 in the conf-split so 200 sub queue directories will be created to increases file access time by reducing inode table seek time. conf-split should be a *prime* number. Also, a large conf-split only makes sense if you have more than 20.000 messages *in your queue*. This won't usually happen. I also went ahead a made the file system of /var/qmail/queue xfs.. That might be a good idea indeed. It also takes away most of the reasons for a big conf-split. Greetz, Peter.
Re: why prime? [was high volume server configurations)
On Thu, Feb 15, 2001 at 02:58:48PM -0800, Michael Boyiazis wrote: How come the conf-split should be prime? That's how hashing works. Read up on it :) Greetz, Peter.
Envelope extraction on high-volume site
We currently manage a qmail setup with close to 1 million virtual accounts handling 3+ million incomming messages daily. Everything if fully redudant (minimum of 3 servers for everything) having several SMTP, POP, IMAP, SQL, LDAP and NFS servers all hooked up via fiber connections. This setup works great and handles the load great but several of our customers have lately been complaining about the speed of mailbox access. We have narrowed the problem down to the amount of time needed to extract the headers from thousands of messages stored within a single folder. One customer has 2gigs of mail and it is taking more then 10 minutes to generate data for the headers request. (telling them to delete some is _NOT_ a valid responce) This message was written to see if there is currently a project underway (or to compile a listing of ideas for such a beast) to integrate an envelope (meaning user_delivered_to, folder, to, cc, from, subject, size, delivery date, attachments flag [yes/no], normal maildir flags) extraction "subservice" into qmail. This would also need to tie into both POP and IMAP to maintain it. It should update a SQL database with the envelope information for each and every message delivered localy. If POP or IMAP does anything, this should also be reflected in the SQL database. If they need to do sorts or listings, they should do so via the database (unless it is eaiser and faster to do it by its self). The reason I would like to see this information extracted is primarly do to a propritary webmail infastructure being developed. It is much faster for us to issue SQL commands to generate listings then IMAP or POP commnads. I relize that this will make a mess when it comes to sync'ing everything but I am unable to think of a better way to approach this. I would like to hear what you think reguarding this matter. Do you have a different approach: tell me. Do you know of a way of doing this: tell me. I really do want to hear from you. Mike
Re: high volume server configurations
I also applied these patches: qmail-1.03]# patch -p1 /usr/local/src/big-todo.103.patch qmail-1.03]# patch -p1 /usr/local/src/big-concurrency.patch I got this error: 1 out of 5 hunks FAILED -- saving rejects to file spawn.c.rej Try -p0, perhaps. Greetz, Peter. I have Errors, too with Solaris patch Tool but with GNU patch it's ok -- Michael..
high volume server configurations
Can someone point me to a doc of how to optimize qmail for relaying only? I don't need local delivery - nada... We are trying to get 25 mailings per second.. What I have done thus far is: 200 in conf-split 255 in conf-spawn 255 in concurrencyremote I also applied these patches: qmail-1.03]# patch -p1 /usr/local/src/big-todo.103.patch qmail-1.03]# patch -p1 /usr/local/src/big-concurrency.patch I got this error: 1 out of 5 hunks FAILED -- saving rejects to file spawn.c.rej begin:vcard n:Wilroy;Sid tel;pager:[EMAIL PROTECTED] tel;fax:408-732-8100 tel;work:408-732-8800 x232 x-mozilla-html:FALSE adr:;; version:2.1 email;internet:[EMAIL PROTECTED] fn:Sid Wilroy end:vcard
Re: high volume server configurations
On Wed, Feb 07, 2001 at 01:41:17PM -0600, Sid Wilroy wrote: Can someone point me to a doc of how to optimize qmail for relaying only? I don't need local delivery - nada... We are trying to get 25 mailings per second.. What I have done thus far is: 200 in conf-split conf-split should be prime. I don't think you know what conf-split means. Higher is not always better. I also applied these patches: qmail-1.03]# patch -p1 /usr/local/src/big-todo.103.patch qmail-1.03]# patch -p1 /usr/local/src/big-concurrency.patch I got this error: 1 out of 5 hunks FAILED -- saving rejects to file spawn.c.rej Try -p0, perhaps. Greetz, Peter.
Re: qmail or postfix for high volume mailing list?
* Sam Trenholme [EMAIL PROTECTED] writes: Philip Mak writes: Can someone tell me: Should I use qmail or postfix to run this discussion list? Oh boy, since this is cross-posted to both the qmail and to the Postfix list, this could become a holy war. Since noone's mentioned s*ndm**l yet, this seems avoidable. I myself have never used Postfix, but have used Qmail. I've used both and would not mind using one over the other, but... My general sense: * Qmail apprently has slightly better performance for mailing list stuff, Postfix has slightly more performance for indivudal mailboxes. ezmlm(-idx) is the best MLM, full stop. All other MLMs pale in comparison. That's one hell of a good argument if you're looking into an MTA for mailing lists. ezmlm/qmail running under tcpserver under a softupdate'd *BSD... well... And since this is x-mailed to the postfix list, where not everyone might be familiar with this document: , | Mailing list management is one of qmail's strengths. Notable features: | | * qmail lets each user handle his own mailing lists. The delivery | instructions for user-whatever go into ~user/.qmail-whatever. | | * qmail makes it really easy to set up mailing list owners. If the user | touches ~user/.qmail-whatever-owner, all bounces will come back to him. | | * qmail supports VERPs, which permit completely reliable automated | bounce handling for mailing lists of any size. | | * SPEED---qmail blasts through mailing lists an order of magnitude | faster than sendmail. For example, one message was successfully | delivered to 150 hosts around the world in just 70 seconds, with qmail's | out-of-the-box configuration. | | * qmail automatically prevents mailing list loops, even across hosts. | | * qmail allows inconceivably gigantic mailing lists. No random limits. | | * qmail handles aliasing and forwarding with the same simple mechanism. | For example, Postmaster is controlled by ~alias/.qmail-postmaster. This | means that cross-host loop detection also applies to aliases. | | * qmail supports the ezmlm mailing list manager, which easily and | automatically handles bounces, subscription requests, and archives. ` I run a few mailing lists (4 of them would definitly count as "high volume") and where MajorDomo and MailMan were hitting really bad on the CPU with Sendmail, ezmlm-idx and qmail are hardly noticeable. On a reasonably fast machine. * Postfix is more open-source than Qmail Arguable. * Postfix is easier to configure than Qmail Highly arguable. * Qmail is more flexible than Postfix Unfortunately, also arguable. You will be happy with whatever choice you make. I'd say that heavily depends on the MLM chosen. And oh, I would up your RAM to 128 megs. ... per slot. As per usual. -- Robin S. Socha http://socha.net/
qmail or postfix for high volume mailing list?
Hello, I am looking into hosting a high volume discussion list (~3000 users, 20 MB of messages per month). The available hardware will probably be a RaQ3 server with 32 MB of RAM (should I pay for more RAM? if so, how much?), so I wouldn't have much system resources to spare. My preferred MLM is Listar. I'm looking into MTAs; from the various mailing list archives I've read on the web, it seems that qmail and postfix are the top MTAs. I could not find information to tell me which one would work better for my situation, however. Can someone tell me: Should I use qmail or postfix to run this discussion list? I am not very concerned about configuration difficulties since I only have to set it up one time, but performance will be important. -Philip Mak ([EMAIL PROTECTED])
Re: qmail or postfix for high volume mailing list?
Oh boy, since this is cross-posted to both the qmail and to the Postfix list, this could become a holy war. I myself have never used Postfix, but have used Qmail. My general sense: * Postfix and Qmail both are very hi-performance MTAs * Qmail apprently has slightly better performance for mailing list stuff, Postfix has slightly more performance for indivudal mailboxes. * Postfix is more open-source than Qmail * Postfix is easier to configure than Qmail * Qmail is more flexible than Postfix You will be happy with whatever choice you make. And oh, I would up your RAM to 128 megs. - Sam Can someone tell me: Should I use qmail or postfix to run this discussion list?
Re: Configuration for high volume qmail box
"Max" [EMAIL PROTECTED] wrote: Primary box - accept incoming e-mail's for organization, users will be connecting via Pop only. All e-mail will be removed from the server and downloaded onto the users PC. Maildir+qmail-pop3d. I do not need console/telnet mail for my users, only the root account for convenience (I am flexible what program I will use). mutt is good. This box needs to selectively relay e-mail based on network address. (or if possible authenticate the sending user). Selective relaying is no problem. There's limited add-on authentication support available (see www.qmail.org), and various pop-before-smtp mechanisms. I have 50 local users, and 40 remote users (these numbers grow quickly). We maintain mailing lists for the different office locations as well as the different departments. No problem. ezmlm is great for lists, but if you're already using something else, it can probably be made to work pretty easily. Secondary box - just in case the primary is down. This box needs to accept e-mail and relay to the primary, as well as selectively relaying based on network address. This machine will also host some announcement mailing lists. The lists have a large amount of users but are low volume. Backup MX's are falling out of favor, at least among the vocal members of this list. Why not just let mail queue up at the sending site? Mailstats from the current Linux/Sendmail machine (no office or external mailing lists are running on this box) are in the attached file. They indicate a pretty light load for the hardware/software you're considering. I need to provide LDAP to my users as well. I have been reading from http://www.openldap.org but will take a while for me to understand the config. This is not a big concern. There is an ldap add-on for qmail. I intend to install a web interface to the mail system in the future, again I have not done an evaluation yet so I am really flexible to the configuration. Haven't used one with qmail, but they're available. -Dave
Configuration for high volume qmail box
I am looking at moving my company's mail system from Linux/Sendmail to FreeBSD/Qmail. I have already installed the package from the ports collection. My question is given that this machine will be processing 5,000 + 10,000 e-mails on a standard day, and up to 100,000 e-mails when we use our lists.. 1) What configuration options should I use with qmail (i.e. should I use maildirs?) 2) Are there tuning/performance options I should change? 3) Which Pop server should I use (I am using the Washington EDU Imap package now) 4) How do I record all sent and received e-mails that the box processes to a separate file? (I would like to record all incoming e-mail to "in-email.log", and all outbound e-mail to "out-email.log, as well as deliver the e-mail to the correct user/domain. These logs will be used for historical backup purposes.) 3) etc... I know these questions are very basic, but I have zero qmail experience. What I have read about the product is very exciting and I hope it fits well with the organization. Thanks in advance. Max e. [EMAIL PROTECTED]
Re: Configuration for high volume qmail box
"Max" [EMAIL PROTECTED] writes: 1) What configuration options should I use with qmail (i.e. should I use maildirs?) Yes, definitely use maildirs. 2) Are there tuning/performance options I should change? Look on http://www.qmail.org/ for the patches targeted to big servers. Be sure to use the ucspi tools (tcpserver, etc) and daemontools. See the Life With Qmail page (referenced off the above qmail site). 3) Which Pop server should I use (I am using the Washington EDU Imap package now) qmail ships with qmail-pop3d. If that doesn't work for you, come back and ask the list -- there are lots of solutions. 4) How do I record all sent and received e-mails that the box processes to a separate file? (I would like to record all incoming e-mail to "in-email.log", and all outbound e-mail to "out-email.log, as well as deliver the e-mail to the correct user/domain. These logs will be used for historical backup purposes.) The qmail FAQ has an entry which describes how you can store a copy of all mail passing through your server. You could write a script that separates incoming and outgoing mail to different folders. I know these questions are very basic, but I have zero qmail experience. What I have read about the product is very exciting and I hope it fits well with the organization. Hey, it's worked everywhere I've worked! Faried.
Re: Configuration for high volume qmail box
How do I apply these patches? - Original Message - From: Faried Nawaz [EMAIL PROTECTED] To: Max [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Friday, January 21, 2000 11:35 AM Subject: Re: Configuration for high volume qmail box "Max" [EMAIL PROTECTED] writes: 1) What configuration options should I use with qmail (i.e. should I use maildirs?) Yes, definitely use maildirs. 2) Are there tuning/performance options I should change? Look on http://www.qmail.org/ for the patches targeted to big servers. Be sure to use the ucspi tools (tcpserver, etc) and daemontools. See the Life With Qmail page (referenced off the above qmail site). 3) Which Pop server should I use (I am using the Washington EDU Imap package now) qmail ships with qmail-pop3d. If that doesn't work for you, come back and ask the list -- there are lots of solutions. 4) How do I record all sent and received e-mails that the box processes to a separate file? (I would like to record all incoming e-mail to "in-email.log", and all outbound e-mail to "out-email.log, as well as deliver the e-mail to the correct user/domain. These logs will be used for historical backup purposes.) The qmail FAQ has an entry which describes how you can store a copy of all mail passing through your server. You could write a script that separates incoming and outgoing mail to different folders. I know these questions are very basic, but I have zero qmail experience. What I have read about the product is very exciting and I hope it fits well with the organization. Hey, it's worked everywhere I've worked! Faried.
Re: Configuration for high volume qmail box
Max writes: How do I apply these patches? If you don't know how to apply a patch, you shouldn't be applying patches. Please trust me; knowing this will save you much wasted time down the road. -- -russ nelson [EMAIL PROTECTED] http://russnelson.com Crynwr sells support for free software | PGPok | "Ask not what your country 521 Pleasant Valley Rd. | +1 315 268 1925 voice | can force other people to Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | do for you..." -Perry M.
Re: Configuration for high volume qmail box
Russell Nelson [EMAIL PROTECTED] wrote: Max writes: How do I apply these patches? If you don't know how to apply a patch, you shouldn't be applying patches. Please trust me; knowing this will save you much wasted time down the road. That's generally good advice, but at some point one has to learn how it's done. See: http://Web.InfoAve.Net/~dsill/lwq.html#patches -Dave
Re: Configuration for high volume qmail box
"Max" [EMAIL PROTECTED] wrote: I am looking at moving my company's mail system from Linux/Sendmail to FreeBSD/Qmail. I have already installed the package from the ports collection. My question is given that this machine will be processing 5,000 + 10,000 e-mails on a standard day, and up to 100,000 e-mails when we use our lists.. 1) What configuration options should I use with qmail (i.e. should I use maildirs?) Insufficient data. Just to answer the maildir question, we need to know something about how the system will be used. Will all mailbox access be via POP or IMAP, or will some users log in to read mail using pine/mutt/etc? If the latter, will these people also need POP/IMAP, or will they exclusively use a local MUA? Some MUA's don't grok maildirs. E.g., pine requires patches. Mutt has excellent maildir support. For POP/IMAP users, maildir works well. For other configuration options, we'd need to know more details of your set-up. It's probably best to go with a basic configuration and tweak it as necessary. 2) Are there tuning/performance options I should change? Raise concurrencylocal and concurrencyremote. Run dnscache. 3) Which Pop server should I use (I am using the Washington EDU Imap package now) qmail-pop3d. There are imapd patches for maildir support. See: http://Web.InfoAve.Net/~dsill/lwq.html#imap-maildir -Dave
Re: Configuration for high volume qmail box
I am going to structure this in the following way. Primary box - accept incoming e-mail's for organization, users will be connecting via Pop only. All e-mail will be removed from the server and downloaded onto the users PC. I do not need console/telnet mail for my users, only the root account for convenience (I am flexible what program I will use). This box needs to selectively relay e-mail based on network address. (or if possible authenticate the sending user). I have 50 local users, and 40 remote users (these numbers grow quickly). We maintain mailing lists for the different office locations as well as the different departments. Secondary box - just in case the primary is down. This box needs to accept e-mail and relay to the primary, as well as selectively relaying based on network address. This machine will also host some announcement mailing lists. The lists have a large amount of users but are low volume. Both machines will be a Dell PC w/ a 400MHz celeron processor, 96MB of ram, and a 6GB hard drive. We connect to the Internet via a full point-to-point T1. Mailstats from the current Linux/Sendmail machine (no office or external mailing lists are running on this box) are in the attached file. I need to provide LDAP to my users as well. I have been reading from http://www.openldap.org but will take a while for me to understand the config. This is not a big concern. I intend to install a web interface to the mail system in the future, again I have not done an evaluation yet so I am really flexible to the configuration. Thanks for the help - Max - Original Message - From: Dave Sill [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, January 21, 2000 12:23 PM Subject: Re: Configuration for high volume qmail box "Max" [EMAIL PROTECTED] wrote: I am looking at moving my company's mail system from Linux/Sendmail to FreeBSD/Qmail. I have already installed the package from the ports collection. My question is given that this machine will be processing 5,000 + 10,000 e-mails on a standard day, and up to 100,000 e-mails when we use our lists.. 1) What configuration options should I use with qmail (i.e. should I use maildirs?) Insufficient data. Just to answer the maildir question, we need to know something about how the system will be used. Will all mailbox access be via POP or IMAP, or will some users log in to read mail using pine/mutt/etc? If the latter, will these people also need POP/IMAP, or will they exclusively use a local MUA? Some MUA's don't grok maildirs. E.g., pine requires patches. Mutt has excellent maildir support. For POP/IMAP users, maildir works well. For other configuration options, we'd need to know more details of your set-up. It's probably best to go with a basic configuration and tweak it as necessary. 2) Are there tuning/performance options I should change? Raise concurrencylocal and concurrencyremote. Run dnscache. 3) Which Pop server should I use (I am using the Washington EDU Imap package now) qmail-pop3d. There are imapd patches for maildir support. See: http://Web.InfoAve.Net/~dsill/lwq.html#imap-maildir -Dave Statistics from Fri Jan 7 09:55:08 2000 M msgsfr bytes_from msgstobytes_to msgsrej msgsdis Mailer 312943 851594K294671386329K0 0 local 5 13 74K2 2K 20 0 fax 84 4K 15 68K7 0 suucp 13 9586 208522K 4751 297473K0 0 esmtp = T225461060194K342351683872K 27 0
High Volume server??
hi I'm arisandy from Indonesia we have Free POP3 mail service now we have 2+ members using Dell PowerEdge 4300 with 1 Gig RAM and 17 Gig HDD Redhat Linux 6.0, Qmail 1.03+patches7(Bruce Guenter)+Vmailmgr 0.93 but now we have problem with delivery system it almost time out and write this to log files /var/log/smtpd: 941182078.995173 tcpserver: status: 79/100 941182079.192038 tcpserver: status: 80/100 941182079.192104 tcpserver: warning: dropping connection, unable to fork: temporary failure but our local/remote connection is low: 941182127.835350 status: local 3/120 remote 18/120 the configuraion are: concurrencylocal=120 concurrencylocal=120 concurrencypop3=100 concurrencysmtp=100 I already set in startup files...using ulimit command to use high task process and high open file handleusing "ulimit" but stilll dropping connection happen...:( is there anything wrong with our system...??? thanks
Re: High Volume server??
On Fri, Oct 29, 1999 at 03:18:25PM +0700, Arisandy wrote: I'm arisandy from Indonesia we have Free POP3 mail service now we have 2+ members using Dell PowerEdge 4300 with 1 Gig RAM and 17 Gig HDD Redhat Linux 6.0, Qmail 1.03+patches7(Bruce Guenter)+Vmailmgr 0.93 but now we have problem with delivery system it almost time out and write this to log files /var/log/smtpd: 941182078.995173 tcpserver: status: 79/100 941182079.192038 tcpserver: status: 80/100 941182079.192104 tcpserver: warning: dropping connection, unable to fork: temporary failure but our local/remote connection is low: 941182127.835350 status: local 3/120 remote 18/120 This has nothing to do with concurrency of INCOMING smtp connections. the configuraion are: concurrencylocal=120 concurrencylocal=120 concurrencypop3=100 concurrencysmtp=100 I don't know which software you are using, but I don't know of any config files named concurrencypop3 or concurrencysmtp Check your startup scripts. There should be a line like: /usr/local/bin/tcpserver -R -v -u 101 -g 101 -c 50 0 smtp \ /usr/local/sbin/tcpcontrol /usr/local/etc/popmgmt/etc/smtp_allow.cdb \ /var/qmail/bin/qmail-smtpd 21 | /var/qmail/bin/splogger smtpd that starts up the tcpserver for incoming SMTP connections. The "-c 50" limits in this case the number of concurrent tcpservers allowed. You probably have "-c 80", change that to e.g. "-c 120". You could do the same in the startup script that starts the qmail-pop3d if you experience problems there, too. \Maex -- SpaceNet GmbH | http://www.Space.Net/ | Yeah, yo mama dresses Research Development| mailto:[EMAIL PROTECTED] | you funny and you need Joseph-Dollinger-Bogen 14 | Tel: +49 (89) 32356-0| a mouse to delete files D-80807 Muenchen | Fax: +49 (89) 32356-299 |
Re: High Volume server??
I'm normally a digest subscriber and I saw this come through the digest. We have approx 15,000 accounts on our server. I solved the "unable to fork" problem using sh's limit (not the bash(?), csh(?) ulimit) and by increasing the value of the -c option in tcpserver. I have the following lines in my init script: # Start qmail for local deliveries. csh -cf '/var/qmail/rc ' # Start qmail SMTP. csh -cf '/root/bin/qmailsmtprc ' # Start qmail POP3. csh -cf '/root/bin/qmailpop3drc ' and, for example, here's my qmailsptprc script using limit: #!/bin/sh limit maxproc 240 /usr/contrib/bin/tcpserver -c 100 -x /etc/tcp.smtp.cdb \ -u 13985 -g 105 0 smtp \ sh -c ' /usr/contrib/bin/fixcr | /var/qmail/bin/qmail-smtpd cd /var/qmail/autoturn exec /usr/contrib/bin/setlock -nx $TCPREMOTEIP/seriallock \ /usr/contrib/bin/maildirsmtp $TCPREMOTEIP \ autoturn-$TCPREMOTEIP- $TCPREMOTEIP AutoTURN ' 21 | /var/qmail/bin/splogger smtpd 3 P. Pirn - Sys Admin - see complete headers for more info
High volume - help!
Hello, For some time we have been offering a free email redirection service. Due to some recent press it has suddenly picked up and we are registering more than 100 users / day. The way we have it set up is using a database lookup of the evelope recipient and then redirecting via |/var/qmail/bin/forward to the users real email address. Everything was running great until a couple weeks ago. We now have around 20 thousand accounts and at times email rerouting is getting slow. Yesterday was a good example. I noticed that the queue was getting over the 2000 mark and kept growing. When I sent email to a test account I maintain (that delivers to the same network) my email would experience tremendous delays (two hours) before being delivered. My basic question is how can I increase the throughput of my Qmail 1.03 installation? What I have done: I have both concurrencylocal and concurrencyremote set to 120. But most of the time qmail has around 10 qmail-remote processes going. Every now and again I give it the ALRM signal and that shoots up the processes but then eventually things back down to the around 10 level. A week or so ago I added the big-todo.patch although I am not really sure what it does. I didn't really see an improvement. I have tcpserver at -c80 for qmail-smtpd. One thing I find discouraging is when the queue inflates and a new email (my test email) gets sent in there is a delay. It is as if the queue really is a queue in the FIFO sense. In a moment of desperation I installed zmailer to handle the outbound transmission of new emails. That seemed to solve some queue problems but zmailer soon brought my machine to its knees by eating all the memory, despite resource limits in its config files. Is there anything more than I can do or strategies I could persue? Something I missed? Advice would be greatly appreciated! Many thanks, Alex Panagides Ceará, Brazil PS. Platform: Debian/Linux 2.1, (2.2.10) Pentium 233, 160Mb SDRAM
Re: High volume - help!
"Alexis S. Panagides" [EMAIL PROTECTED] wrote: My basic question is how can I increase the throughput of my Qmail 1.03 installation? Easy: locate the bottleneck and remove it. :-) I have both concurrencylocal and concurrencyremote set to 120. But most of the time qmail has around 10 qmail-remote processes going. If you only have 10 qmail-remotes running, but 2000+ queued remote messages, something is seriously wrong. You need to open your sysadmin toolkit and get under the hood. Are you CPU bound? Not likely. I/O bound? Probably. N/W bound? Possibly? Run top, vmstat, iostat, etc. until you can identify the critical resource. Every now and again I give it the ALRM signal and that shoots up the processes but then eventually things back down to the around 10 level. Huh. From that, it sounds like you don't really have a problem. But you said a test message took 2 hours, so something is definitely not right. Did you trace your test message through the logs? A week or so ago I added the big-todo.patch although I am not really sure what it does. I didn't really see an improvement. Oops. One shouldn't patch qmail, or any other system, for that matter, unless one knows what the patch does. I have tcpserver at -c80 for qmail-smtpd. Your problem is outgoing throughput, not incoming. One thing I find discouraging is when the queue inflates and a new email (my test email) gets sent in there is a delay. It is as if the queue really is a queue in the FIFO sense. It *is* a queue. qmail-send can't do infinite work instantaneously. If there's more work than it can handle, waiting jobs queue up. Have you run qmailanalog? -Dave
High volume performance questions
I am setting up a web site that will generate customized daily mail messages for subscribers. In other words, we will be sending different messages to each person, rather than bulk-mailing the same message to many people. I would like to hear from qmail experts on these questions: 1. How much hardware? Initially we want enough capacity to deliver a million mails a day. We plan to use a series of Linux boxes, and multiplex the messages across them. How many will we need, and how powerful? 2. Is qmail the right choice, or should we use Postfix? My impression is that qmail's VERPs will let us handle the inevitable flood of bounces more effectively, but that Postfix is more efficient. What are people's experiences in this regard? Thanks in advance, Luke Blanshard
