[EMAIL PROTECTED]> <[EMAIL PROTECTED]> Subject: Re: qpopper high load average Date: Fri, 23 Jan 2004 10:15:20 +1100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Hi Bart, Average size of our mailboxes is around 800kb per mailbox. of the 297,000 about 250,000 of them are active and live. 50,000 of them are inactive/suspended holding up 55GB between them. Our total spool is currently 240GB including the 55GB for the suspend/inactive users. We just haven't deleted their old spool data yet. What do you use for your SMTP server? ----- Original Message ----- From: "Bart Dumon" <[EMAIL PROTECTED]> To: "Paul" <[EMAIL PROTECTED]> Cc: "Subscribers of Qpopper" <[EMAIL PROTECTED]> Sent: Thursday, January 22, 2004 9:58 PM Subject: Re: qpopper high load average > Hi Paul, > > We have 320k mailboxes, 4 dual xeon mailservers. the setup was > highly influenced by the requirement of redundancy, therefor we're > using multiple machines with a spool on NFS. at least 2 boxes > should be able to handle all the traffic, so i'm quite certain > there is an underlying problem which affects our performance. > > The NFS connection is 100mbit FD for each box, the NAS itself > is 1Gbit FD. during business hours one box can easily get > to 30-40mbit/sec. resulting in 100-120mbit/sec of throughput > on the NAS. > You're considering maildir/nfs to be a potential slow down, > but i don't see it that way because disk I/O is usually the > bottleneck on smtp/pop servers. both ways have their advantages > and disadvantages. > > Anyway, you seem to be doing pretty well with your single system, > but do you have any idea of the average size of the mailboxes > on your system and the percentage of active users? > > > bart > > On Thu, Jan 22, 2004 at 10:51:39AM +1100, Paul wrote: > > HI Bart, > > > > We run a mailserver for a 297,000 odd mailboxes. We normally during peak see > > a load average of about 15, this is sustained and doesn't flucate that much. > > The box is a dual 3.06ghz xeon with 3gb of DDR and 525GB of Ultra320 raid5 > > which is used for spool. On the server we have qpopper auth'ing via mysql > > and exim as the local smtp server. > > Our mail is stored in a double hash array like /var/spool/mail/e/b/ebadine > > and we just use flatfile, instead of Maildir. The riad should be raid10 but > > it's not always realistic, we use raid5 for example, its a bit of slow down > > compared to raid10 but meets our needs > > > > Maildir would be a potential slow down, especially over NFS. What speed > > Mb/sec do you get over the NFS connection? I'm assuming its 100mbit full > > duplex > > When we originally had everyone in /var/spool/mail/$username our system > > bogged down to an insane level, once we double hashed it, it was fantasic. > > > > Do run any form of performance monitoring on the server? We use mrtg and > > graph cpu, network, memory etc so we can easily spot bottlenecks. > > > > ----- Original Message ----- > > From: "Bart Dumon" <[EMAIL PROTECTED]> > > To: "Paul" <[EMAIL PROTECTED]> > > Cc: "Subscribers of Qpopper" <[EMAIL PROTECTED]> > > Sent: Thursday, January 22, 2004 8:41 AM > > Subject: Re: qpopper high load average > > > > > > > > > > On Wed, Jan 21, 2004 at 11:24:12AM +1100, Paul wrote: > > > > How many connections/sec for the server during peak? What does the load > > avg > > > > get to? What storage is it (the mail spool)? What filesystem? > > > > > > during peaks we get about 15 connections/sec per server, the load > > > gets up to 800 if we do not interfere. once the popper gets > > > restarted the load will decrease. the mail spool is kept on a NAS > > > and is accessed using nfs. > > > > > > > > > bart > > > -- > > > > >
