Re: Upgrade a mail server
Good procedure! - Original Message - From: "I. Forbes" <[EMAIL PROTECTED]> To: Sent: Tuesday, February 19, 2002 11:35 PM Subject: Re: Upgrade a mail server > Hello Craig > > On 19 Feb 2002, at 10:38, Craig Sanders wrote: > > > i'd love to convert it over to Maildir/ but haven't yet found any way > > that doesn't involve many hours of downtime while converting the > > mailboxes from mbox format to Maildir. > > I did this a while back. It is possible with very little apparent > downtime. (We are using Exim and Courier): > > - create Maildirs for all users. (This is important if both your > POP3/IMAP software and MDA are not configured to create missing > Maildirs "on the fly"). > > - change your MDA to deliver into the new maildirs > > (At this stage new mail is not visible to users when it arrives - but > they can still see their old mail. The downtime for this phase should > be short) > > - change your POP3/IMAP programs to pick up mail from the > maildirs. > > (At this stage old mail is not visible to users, but new mail is. This > should not be too much of a problem - if users have left MB worth of > mail in their boxes, they can't want it too badly, it is when new mail > is not available that people complain.) > > - run your script which reads the mbox files, and delivers to > maildirs. My script renamed the mailbox files just after they > had been converted, so I could restart the script without > incurring duplicate deliveries if (when) the script crashed. > > - By the time the script finishes, all mail is visible again. > > - Keep the old mbox files around for a few days just in case you > discover a problem ... > > No corruption, no duplication, no mail lost, no file locking, no error > messages on client desktops, not too much loss of service and very > few support calls. > > Have fun! > > > Ian > > - > Ian Forbes ZSD > http://www.zsd.co.za > Office: +27 21 683-1388 Fax: +27 21 674-1106 > Snail Mail: P.O. Box 46827, Glosderry, 7702, South Africa > - > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > >
Re: Upgrade a mail server
Good procedure! - Original Message - From: "I. Forbes" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, February 19, 2002 11:35 PM Subject: Re: Upgrade a mail server > Hello Craig > > On 19 Feb 2002, at 10:38, Craig Sanders wrote: > > > i'd love to convert it over to Maildir/ but haven't yet found any way > > that doesn't involve many hours of downtime while converting the > > mailboxes from mbox format to Maildir. > > I did this a while back. It is possible with very little apparent > downtime. (We are using Exim and Courier): > > - create Maildirs for all users. (This is important if both your > POP3/IMAP software and MDA are not configured to create missing > Maildirs "on the fly"). > > - change your MDA to deliver into the new maildirs > > (At this stage new mail is not visible to users when it arrives - but > they can still see their old mail. The downtime for this phase should > be short) > > - change your POP3/IMAP programs to pick up mail from the > maildirs. > > (At this stage old mail is not visible to users, but new mail is. This > should not be too much of a problem - if users have left MB worth of > mail in their boxes, they can't want it too badly, it is when new mail > is not available that people complain.) > > - run your script which reads the mbox files, and delivers to > maildirs. My script renamed the mailbox files just after they > had been converted, so I could restart the script without > incurring duplicate deliveries if (when) the script crashed. > > - By the time the script finishes, all mail is visible again. > > - Keep the old mbox files around for a few days just in case you > discover a problem ... > > No corruption, no duplication, no mail lost, no file locking, no error > messages on client desktops, not too much loss of service and very > few support calls. > > Have fun! > > > Ian > > - > Ian Forbes ZSD > http://www.zsd.co.za > Office: +27 21 683-1388 Fax: +27 21 674-1106 > Snail Mail: P.O. Box 46827, Glosderry, 7702, South Africa > - > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Tue, Feb 19, 2002 at 07:40:32PM +1100, Craig Sanders wrote: > On Tue, Feb 19, 2002 at 05:51:20PM +1100, Russell Coker wrote: > > When I configured it to use hashed indexes to /etc/passwd (this was on > > AIX which does it much better than Linux) the load average dropped to > > between 0.5 and 4 (depending on the time of day), CPU usage was often > > less than 100%, and everything was responsive! > > i was thinking of hashed passwd/shadow files when i said it would scale > up to at least 1 users. Well, I've done 70% of that without even sweating. I'm sure it could scale well above that. (db files are -fast-... heck, INN uses a slightly hacked variant of the db files to handle -millions- of records quickly). > > The problem is that the utility programs like passwd(1) aren't smart > > enough to update the hash, and the libraries that read from the hashes > > aren't smart enough to check the time stamps and use /etc/passwd if > > it's more recent. AIX does this much better. they aren't? Haven't looked at PAM? # # The PAM configuration file for the Shadow `passwd' service # password required pam_make.so /var/lib/misc Magic. (You'll also need it in chsh and chfn and any other ways you have to edit the passwd file.) See ftp://people.redhat.com/misa/old-devel/pam/modules/pam_make-0.1.tar.gz > i found (when i was using hashed passwd files) that the following in > root's crontab is adequate: > > 0/5 * * * cd /var/lib/misc ; make > > or run it every minute if you prefer. I run it every 15. ... and /etc/passwd and /etc/shadow are rsync'd between machines. So changing the password on the master will run that makefile and update the slaves.
Re: Upgrade a mail server
On Tue, Feb 19, 2002 at 07:40:32PM +1100, Craig Sanders wrote: > On Tue, Feb 19, 2002 at 05:51:20PM +1100, Russell Coker wrote: > > When I configured it to use hashed indexes to /etc/passwd (this was on > > AIX which does it much better than Linux) the load average dropped to > > between 0.5 and 4 (depending on the time of day), CPU usage was often > > less than 100%, and everything was responsive! > > i was thinking of hashed passwd/shadow files when i said it would scale > up to at least 1 users. Well, I've done 70% of that without even sweating. I'm sure it could scale well above that. (db files are -fast-... heck, INN uses a slightly hacked variant of the db files to handle -millions- of records quickly). > > The problem is that the utility programs like passwd(1) aren't smart > > enough to update the hash, and the libraries that read from the hashes > > aren't smart enough to check the time stamps and use /etc/passwd if > > it's more recent. AIX does this much better. they aren't? Haven't looked at PAM? # # The PAM configuration file for the Shadow `passwd' service # password required pam_make.so /var/lib/misc Magic. (You'll also need it in chsh and chfn and any other ways you have to edit the passwd file.) See ftp://people.redhat.com/misa/old-devel/pam/modules/pam_make-0.1.tar.gz > i found (when i was using hashed passwd files) that the following in > root's crontab is adequate: > > 0/5 * * * cd /var/lib/misc ; make > > or run it every minute if you prefer. I run it every 15. ... and /etc/passwd and /etc/shadow are rsync'd between machines. So changing the password on the master will run that makefile and update the slaves. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
Hello Craig On 19 Feb 2002, at 10:38, Craig Sanders wrote: > i'd love to convert it over to Maildir/ but haven't yet found any way > that doesn't involve many hours of downtime while converting the > mailboxes from mbox format to Maildir. I did this a while back. It is possible with very little apparent downtime. (We are using Exim and Courier): - create Maildirs for all users. (This is important if both your POP3/IMAP software and MDA are not configured to create missing Maildirs "on the fly"). - change your MDA to deliver into the new maildirs (At this stage new mail is not visible to users when it arrives - but they can still see their old mail. The downtime for this phase should be short) - change your POP3/IMAP programs to pick up mail from the maildirs. (At this stage old mail is not visible to users, but new mail is. This should not be too much of a problem - if users have left MB worth of mail in their boxes, they can't want it too badly, it is when new mail is not available that people complain.) - run your script which reads the mbox files, and delivers to maildirs. My script renamed the mailbox files just after they had been converted, so I could restart the script without incurring duplicate deliveries if (when) the script crashed. - By the time the script finishes, all mail is visible again. - Keep the old mbox files around for a few days just in case you discover a problem ... No corruption, no duplication, no mail lost, no file locking, no error messages on client desktops, not too much loss of service and very few support calls. Have fun! Ian - Ian Forbes ZSD http://www.zsd.co.za Office: +27 21 683-1388 Fax: +27 21 674-1106 Snail Mail: P.O. Box 46827, Glosderry, 7702, South Africa -
Re: Upgrade a mail server
Hello Craig On 19 Feb 2002, at 10:38, Craig Sanders wrote: > i'd love to convert it over to Maildir/ but haven't yet found any way > that doesn't involve many hours of downtime while converting the > mailboxes from mbox format to Maildir. I did this a while back. It is possible with very little apparent downtime. (We are using Exim and Courier): - create Maildirs for all users. (This is important if both your POP3/IMAP software and MDA are not configured to create missing Maildirs "on the fly"). - change your MDA to deliver into the new maildirs (At this stage new mail is not visible to users when it arrives - but they can still see their old mail. The downtime for this phase should be short) - change your POP3/IMAP programs to pick up mail from the maildirs. (At this stage old mail is not visible to users, but new mail is. This should not be too much of a problem - if users have left MB worth of mail in their boxes, they can't want it too badly, it is when new mail is not available that people complain.) - run your script which reads the mbox files, and delivers to maildirs. My script renamed the mailbox files just after they had been converted, so I could restart the script without incurring duplicate deliveries if (when) the script crashed. - By the time the script finishes, all mail is visible again. - Keep the old mbox files around for a few days just in case you discover a problem ... No corruption, no duplication, no mail lost, no file locking, no error messages on client desktops, not too much loss of service and very few support calls. Have fun! Ian - Ian Forbes ZSD http://www.zsd.co.za Office: +27 21 683-1388 Fax: +27 21 674-1106 Snail Mail: P.O. Box 46827, Glosderry, 7702, South Africa - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Tue, Feb 19, 2002 at 05:51:20PM +1100, Russell Coker wrote:
> When I configured it to use hashed indexes to /etc/passwd (this was on
> AIX which does it much better than Linux) the load average dropped to
> between 0.5 and 4 (depending on the time of day), CPU usage was often
> less than 100%, and everything was responsive!
i was thinking of hashed passwd/shadow files when i said it would scale
up to at least 1 users.
> > you don't need nscd, just hashed map files for
> > /etc/{passwd,group,shadow). there's a Makefile in /var/lib/misc
> > which handles that.
>
> The problem is that the utility programs like passwd(1) aren't smart
> enough to update the hash, and the libraries that read from the hashes
> aren't smart enough to check the time stamps and use /etc/passwd if
> it's more recent. AIX does this much better.
i found (when i was using hashed passwd files) that the following in
root's crontab is adequate:
0/5 * * * cd /var/lib/misc ; make
or run it every minute if you prefer.
> > i'd love to convert it over to Maildir/ but haven't yet found any way
> > that doesn't involve many hours of downtime while converting the
> > mailboxes from mbox format to Maildir.
> >
> > one of these days i'll have the time to sit down and work out a good
> > solution to the problem. i've got some ideas but no time to work them
> > out.
>
> Conversion doesn't require system down-time.
see comments in previous messages. whether it's real downtime or just
apparent downtime to the user doesn't really matter - the users are
going to complain anyway.
> Firstly you create Maildir directories. Then you can convert the mail
> server to Maildir delivery in a matter of minutes, then you run a
> conversion script to convert mbox files to Maildir (may take hours or
> even a day depending on system speed and size of the spool). When you
> think it's about half done you change the POP and IMAP servers to read
> from Maildir instead of mbox. The end result is that there is a
> matter of minutes of actual down time,
no, that is "hours or even a day" of down time.
craig
--
craig sanders <[EMAIL PROTECTED]>
Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch
RE: Upgrade a mail server
Title: RE: Upgrade a mail server
very interesing your comments.I am installing a box with postfix+courier-pop3+courier-imap. When woody gets stable state, I will upgrade my mailserver, anf of course, I will post you the results and problems ..
Thanks to all.
Un saludo.
-Mensaje original-
De: Russell Coker [mailto:[EMAIL PROTECTED]]
Enviado el: martes, 19 de febrero de 2002 7:51
Para: Craig Sanders
CC: Javier Castillo Alcibar; debian-isp@lists.debian.org
Asunto: Re: Upgrade a mail server
On Tue, 19 Feb 2002 10:38, Craig Sanders wrote:
> > > I have a debian 2.1 email server, with sendmail as smtp server, and
> > > qpopper as POP3 server. This box hosts several domains (about 100
> > > different domains, using virtual hosting), with a total of 1300
> > > users. Because the auth method is based on /etc/passwd, I have 1300
> > > unix user accounts, one per mailbox, of course.
> > >
> > > This is becomming not scalable, and I want to start to use a mail
> > > server with auth based on sql, using MySQL for example. I am
> > > thinking in use a debian woody( I'll wait till woody becomes
> > > stable), with sendmail as smtp server, but I am not sure about the
> > > pop3 daemon I have to use.
> >
> > /etc/passwd should scale to 1300 users, using nscd should solve any
> > problems there. But chances are that changing how you look up user
> > names won't make any noticable difference in performance.
>
> yes, /etc/passwd scales up to several thousand accounts - at least 5000
> or 1 depending on how good your hardware is.
I expect that /etc/passwd would cause performance problems before 10K users.
I used to run a machine with 27K users, it ran like a dog (load average was
always >30, the system was not responsive, commands like "ls -l /tmp" would
take noticable amounts of time to complete). When I configured it to use
hashed indexes to /etc/passwd (this was on AIX which does it much better than
Linux) the load average dropped to between 0.5 and 4 (depending on the time
of day), CPU usage was often less than 100%, and everything was responsive!
> you don't need nscd,
> just hashed map files for /etc/{passwd,group,shadow). there's a
> Makefile in /var/lib/misc which handles that.
The problem is that the utility programs like passwd(1) aren't smart enough
to update the hash, and the libraries that read from the hashes aren't smart
enough to check the time stamps and use /etc/passwd if it's more recent. AIX
does this much better.
> i'd love to convert it over to Maildir/ but haven't yet found any way
> that doesn't involve many hours of downtime while converting the
> mailboxes from mbox format to Maildir.
>
> one of these days i'll have the time to sit down and work out a good
> solution to the problem. i've got some ideas but no time to work them
> out.
Conversion doesn't require system down-time. Firstly you create Maildir
directories. Then you can convert the mail server to Maildir delivery in a
matter of minutes, then you run a conversion script to convert mbox files to
Maildir (may take hours or even a day depending on system speed and size of
the spool). When you think it's about half done you change the POP and IMAP
servers to read from Maildir instead of mbox. The end result is that there
is a matter of minutes of actual down time, and for some users it will appear
that their mail has been delayed (there are no guarantees about delivery time
anyway).
--
Signatures >4 lines are rude. If you send email to me or to a mailing list
that I am subscribed to which has >4 lines of legalistic junk at the end
then you are specifically authorizing me to do whatever I wish with the
message (the sig won't be read).
Re: Upgrade a mail server
On Tue, 19 Feb 2002 10:38, Craig Sanders wrote:
> > > I have a debian 2.1 email server, with sendmail as smtp server, and
> > > qpopper as POP3 server. This box hosts several domains (about 100
> > > different domains, using virtual hosting), with a total of 1300
> > > users. Because the auth method is based on /etc/passwd, I have 1300
> > > unix user accounts, one per mailbox, of course.
> > >
> > > This is becomming not scalable, and I want to start to use a mail
> > > server with auth based on sql, using MySQL for example. I am
> > > thinking in use a debian woody( I'll wait till woody becomes
> > > stable), with sendmail as smtp server, but I am not sure about the
> > > pop3 daemon I have to use.
> >
> > /etc/passwd should scale to 1300 users, using nscd should solve any
> > problems there. But chances are that changing how you look up user
> > names won't make any noticable difference in performance.
>
> yes, /etc/passwd scales up to several thousand accounts - at least 5000
> or 1 depending on how good your hardware is.
I expect that /etc/passwd would cause performance problems before 10K users.
I used to run a machine with 27K users, it ran like a dog (load average was
always >30, the system was not responsive, commands like "ls -l /tmp" would
take noticable amounts of time to complete). When I configured it to use
hashed indexes to /etc/passwd (this was on AIX which does it much better than
Linux) the load average dropped to between 0.5 and 4 (depending on the time
of day), CPU usage was often less than 100%, and everything was responsive!
> you don't need nscd,
> just hashed map files for /etc/{passwd,group,shadow). there's a
> Makefile in /var/lib/misc which handles that.
The problem is that the utility programs like passwd(1) aren't smart enough
to update the hash, and the libraries that read from the hashes aren't smart
enough to check the time stamps and use /etc/passwd if it's more recent. AIX
does this much better.
> i'd love to convert it over to Maildir/ but haven't yet found any way
> that doesn't involve many hours of downtime while converting the
> mailboxes from mbox format to Maildir.
>
> one of these days i'll have the time to sit down and work out a good
> solution to the problem. i've got some ideas but no time to work them
> out.
Conversion doesn't require system down-time. Firstly you create Maildir
directories. Then you can convert the mail server to Maildir delivery in a
matter of minutes, then you run a conversion script to convert mbox files to
Maildir (may take hours or even a day depending on system speed and size of
the spool). When you think it's about half done you change the POP and IMAP
servers to read from Maildir instead of mbox. The end result is that there
is a matter of minutes of actual down time, and for some users it will appear
that their mail has been delayed (there are no guarantees about delivery time
anyway).
--
Signatures >4 lines are rude. If you send email to me or to a mailing list
that I am subscribed to which has >4 lines of legalistic junk at the end
then you are specifically authorizing me to do whatever I wish with the
message (the sig won't be read).
Re: Upgrade a mail server
On Tue, Feb 19, 2002 at 05:51:20PM +1100, Russell Coker wrote:
> When I configured it to use hashed indexes to /etc/passwd (this was on
> AIX which does it much better than Linux) the load average dropped to
> between 0.5 and 4 (depending on the time of day), CPU usage was often
> less than 100%, and everything was responsive!
i was thinking of hashed passwd/shadow files when i said it would scale
up to at least 1 users.
> > you don't need nscd, just hashed map files for
> > /etc/{passwd,group,shadow). there's a Makefile in /var/lib/misc
> > which handles that.
>
> The problem is that the utility programs like passwd(1) aren't smart
> enough to update the hash, and the libraries that read from the hashes
> aren't smart enough to check the time stamps and use /etc/passwd if
> it's more recent. AIX does this much better.
i found (when i was using hashed passwd files) that the following in
root's crontab is adequate:
0/5 * * * cd /var/lib/misc ; make
or run it every minute if you prefer.
> > i'd love to convert it over to Maildir/ but haven't yet found any way
> > that doesn't involve many hours of downtime while converting the
> > mailboxes from mbox format to Maildir.
> >
> > one of these days i'll have the time to sit down and work out a good
> > solution to the problem. i've got some ideas but no time to work them
> > out.
>
> Conversion doesn't require system down-time.
see comments in previous messages. whether it's real downtime or just
apparent downtime to the user doesn't really matter - the users are
going to complain anyway.
> Firstly you create Maildir directories. Then you can convert the mail
> server to Maildir delivery in a matter of minutes, then you run a
> conversion script to convert mbox files to Maildir (may take hours or
> even a day depending on system speed and size of the spool). When you
> think it's about half done you change the POP and IMAP servers to read
> from Maildir instead of mbox. The end result is that there is a
> matter of minutes of actual down time,
no, that is "hours or even a day" of down time.
craig
--
craig sanders <[EMAIL PROTECTED]>
Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: Upgrade a mail server
Title: RE: Upgrade a mail server
very interesing your comments.I am installing a box with postfix+courier-pop3+courier-imap. When woody gets stable state, I will upgrade my mailserver, anf of course, I will post you the results and problems ..
Thanks to all.
Un saludo.
-Mensaje original-
De: Russell Coker [mailto:[EMAIL PROTECTED]]
Enviado el: martes, 19 de febrero de 2002 7:51
Para: Craig Sanders
CC: Javier Castillo Alcibar; [EMAIL PROTECTED]
Asunto: Re: Upgrade a mail server
On Tue, 19 Feb 2002 10:38, Craig Sanders wrote:
> > > I have a debian 2.1 email server, with sendmail as smtp server, and
> > > qpopper as POP3 server. This box hosts several domains (about 100
> > > different domains, using virtual hosting), with a total of 1300
> > > users. Because the auth method is based on /etc/passwd, I have 1300
> > > unix user accounts, one per mailbox, of course.
> > >
> > > This is becomming not scalable, and I want to start to use a mail
> > > server with auth based on sql, using MySQL for example. I am
> > > thinking in use a debian woody( I'll wait till woody becomes
> > > stable), with sendmail as smtp server, but I am not sure about the
> > > pop3 daemon I have to use.
> >
> > /etc/passwd should scale to 1300 users, using nscd should solve any
> > problems there. But chances are that changing how you look up user
> > names won't make any noticable difference in performance.
>
> yes, /etc/passwd scales up to several thousand accounts - at least 5000
> or 1 depending on how good your hardware is.
I expect that /etc/passwd would cause performance problems before 10K users.
I used to run a machine with 27K users, it ran like a dog (load average was
always >30, the system was not responsive, commands like "ls -l /tmp" would
take noticable amounts of time to complete). When I configured it to use
hashed indexes to /etc/passwd (this was on AIX which does it much better than
Linux) the load average dropped to between 0.5 and 4 (depending on the time
of day), CPU usage was often less than 100%, and everything was responsive!
> you don't need nscd,
> just hashed map files for /etc/{passwd,group,shadow). there's a
> Makefile in /var/lib/misc which handles that.
The problem is that the utility programs like passwd(1) aren't smart enough
to update the hash, and the libraries that read from the hashes aren't smart
enough to check the time stamps and use /etc/passwd if it's more recent. AIX
does this much better.
> i'd love to convert it over to Maildir/ but haven't yet found any way
> that doesn't involve many hours of downtime while converting the
> mailboxes from mbox format to Maildir.
>
> one of these days i'll have the time to sit down and work out a good
> solution to the problem. i've got some ideas but no time to work them
> out.
Conversion doesn't require system down-time. Firstly you create Maildir
directories. Then you can convert the mail server to Maildir delivery in a
matter of minutes, then you run a conversion script to convert mbox files to
Maildir (may take hours or even a day depending on system speed and size of
the spool). When you think it's about half done you change the POP and IMAP
servers to read from Maildir instead of mbox. The end result is that there
is a matter of minutes of actual down time, and for some users it will appear
that their mail has been delayed (there are no guarantees about delivery time
anyway).
--
Signatures >4 lines are rude. If you send email to me or to a mailing list
that I am subscribed to which has >4 lines of legalistic junk at the end
then you are specifically authorizing me to do whatever I wish with the
message (the sig won't be read).
Re: Upgrade a mail server
On Tue, 19 Feb 2002, Craig Sanders wrote: > the particular mail server that needs conversion has about 6000 users. > most of whom have very small mailboxes. some have obscenely large > mailboxes (200MB or more). last time i checked there was a total of > about 6GB in /var/spool/mail If you have a similar machine around, just copy the whole /var/spool/mail over and try your conversion scripts on the dry dock :-) Alex -- "Life is what happens to you while you're busy making other plans." John Lennon
Re: Upgrade a mail server
On Tue, 19 Feb 2002 10:38, Craig Sanders wrote:
> > > I have a debian 2.1 email server, with sendmail as smtp server, and
> > > qpopper as POP3 server. This box hosts several domains (about 100
> > > different domains, using virtual hosting), with a total of 1300
> > > users. Because the auth method is based on /etc/passwd, I have 1300
> > > unix user accounts, one per mailbox, of course.
> > >
> > > This is becomming not scalable, and I want to start to use a mail
> > > server with auth based on sql, using MySQL for example. I am
> > > thinking in use a debian woody( I'll wait till woody becomes
> > > stable), with sendmail as smtp server, but I am not sure about the
> > > pop3 daemon I have to use.
> >
> > /etc/passwd should scale to 1300 users, using nscd should solve any
> > problems there. But chances are that changing how you look up user
> > names won't make any noticable difference in performance.
>
> yes, /etc/passwd scales up to several thousand accounts - at least 5000
> or 1 depending on how good your hardware is.
I expect that /etc/passwd would cause performance problems before 10K users.
I used to run a machine with 27K users, it ran like a dog (load average was
always >30, the system was not responsive, commands like "ls -l /tmp" would
take noticable amounts of time to complete). When I configured it to use
hashed indexes to /etc/passwd (this was on AIX which does it much better than
Linux) the load average dropped to between 0.5 and 4 (depending on the time
of day), CPU usage was often less than 100%, and everything was responsive!
> you don't need nscd,
> just hashed map files for /etc/{passwd,group,shadow). there's a
> Makefile in /var/lib/misc which handles that.
The problem is that the utility programs like passwd(1) aren't smart enough
to update the hash, and the libraries that read from the hashes aren't smart
enough to check the time stamps and use /etc/passwd if it's more recent. AIX
does this much better.
> i'd love to convert it over to Maildir/ but haven't yet found any way
> that doesn't involve many hours of downtime while converting the
> mailboxes from mbox format to Maildir.
>
> one of these days i'll have the time to sit down and work out a good
> solution to the problem. i've got some ideas but no time to work them
> out.
Conversion doesn't require system down-time. Firstly you create Maildir
directories. Then you can convert the mail server to Maildir delivery in a
matter of minutes, then you run a conversion script to convert mbox files to
Maildir (may take hours or even a day depending on system speed and size of
the spool). When you think it's about half done you change the POP and IMAP
servers to read from Maildir instead of mbox. The end result is that there
is a matter of minutes of actual down time, and for some users it will appear
that their mail has been delayed (there are no guarantees about delivery time
anyway).
--
Signatures >4 lines are rude. If you send email to me or to a mailing list
that I am subscribed to which has >4 lines of legalistic junk at the end
then you are specifically authorizing me to do whatever I wish with the
message (the sig won't be read).
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
Craig Sanders wrote: > > - configure postfix so that "deferred_transports = local" - this leaves >ALL local deliveries in the queue until i flush it manually while >still allowing the mail server to relay outbound mail to other >servers for my users. > > or > > - create Maildir/ and ~/.conversion_in_progress in each home directory >before starting the conversion script, and have a procmail or >maildrop rule which exited with a temporary failure code (EX_TEMPFAIL >from /usr/include/sysexits.h) if ~/.conversion_in_progress exists. > >this would effectively defer incoming mail only for those users whose >mailbox hasn't been converted yet. > But as long as you have the Maildir created (along with cur, new, and tmp), you can safely deliver even if the user's mailbox has not been converted, as long as you're delivering to the Maildir. Although, to be extra careful, the temp failure should be cool, since it will spool locally. One thought is that you may want to check your SMTP server's setting for how long a message can sit in the queue before it generates a warning message. I know Sendmail defaults to 4 hours, not sure about postfix. Doensn't sound like a problem with the first option above though. >the conversion script would "rm -f ~/.conversion_in_progress" as soon >as the conversion was done. > > this solves one major problem - i don't like the idea of having SMTP > down at all...i won't tolerate it and my users certainly wouldn't. > Actually, external SMTP should be fine, as long as it's less than 4 hours (typical default setting for queue warning). Local users' SMTP connections are the tricky part. ... -- _ Rich Puhek ETN Systems Inc. _
Re: Upgrade a mail server
On Mon, Feb 18, 2002 at 09:21:16PM -0600, Rich Puhek wrote: > You didn't state if you're an ISP or another institution. ISP.24/7 operation required. more precisely: it's not my ISP, i'm the senior system admin at an ISP. > > (*) it involves using semaphore files in each users homedir to > > indicate whether they are maildir or still mbox, plus > > procmail/maildrop rules to deliver accordingly, and a pop proxy > > which chooses whether to connect to the mbox or Maildir capable pop > > daemon. when that's done it is possible to safely convert all the > > mboxes to maildirs > > Yow. That might be overkill. I'd try some testing to determine how > long the migration process will take. If you're talking about anything > under 4 hours, you can probably count on running from, say 2AM to 6AM. > Set your, ahem, pickier users to be transfered first. Safety isn't > going to be an issue with Maildirs, the only concern I'd have is > making sure that before you run your script to convert the mboxes you > have disabled any other process that will touch an mbox. Last > convertion I did our customer (an ISP with ~3000 users) had accepted a > maintenance window where POP and SMTP services would be unavailable, > so I simply made sure sendmail and qpopper were dead, then for good > measure I believe I changed permissions, ownership, or the directory > name of /var/spool/mail, just in case. yes, that's another variation. if downtime was only 4-6 hours and if i was willing for users to be unable to fetch mail with POP or IMAP for that time then i could: - configure postfix so that "deferred_transports = local" - this leaves ALL local deliveries in the queue until i flush it manually while still allowing the mail server to relay outbound mail to other servers for my users. or - create Maildir/ and ~/.conversion_in_progress in each home directory before starting the conversion script, and have a procmail or maildrop rule which exited with a temporary failure code (EX_TEMPFAIL from /usr/include/sysexits.h) if ~/.conversion_in_progress exists. this would effectively defer incoming mail only for those users whose mailbox hasn't been converted yet. the conversion script would "rm -f ~/.conversion_in_progress" as soon as the conversion was done. this solves one major problem - i don't like the idea of having SMTP down at all...i won't tolerate it and my users certainly wouldn't. doing this, both sending and fetching mail would work perfectly for users whose mailbox has already been converted. users who have yet to be converted could send mail but not fetch it. that's not a bad compromise...if the entire job could be done in only a few hours. the particular mail server that needs conversion has about 6000 users. most of whom have very small mailboxes. some have obscenely large mailboxes (200MB or more). last time i checked there was a total of about 6GB in /var/spool/mail > If preliminary testing indicates you can do the changeover is a matter > of very few hours (which wouldn't surprise me with decent hardware), you that's the key issue. can the entire system be converted in less than 6 hours. > Here's another angle to consider: perhaps a watchdog script to fire > off a convertion script as soon as the user logs in (PPP-wise, this > does assume you're in a dialup-ISP type environment). By the time they > finish connecting, negotiating, and resolving the POP server IP > address, most of their mail will be converted. On the downside... not > sure I trust a script to run that unattended for such a critical > process. i've already had this idea. it has some merit, but it's more complicated than the one i mentioned in my last message, and certainly more complicated than the above. in general, i disapprove of unneccesary complication - i'm a strong believer in the KISS principle. > Best of luck when you finally convert. You're on the right track to > make sure it doesn't end in disaster. it will happen, but not until i am 100% certain that it's going to work without a single problem. craig -- craig sanders <[EMAIL PROTECTED]> Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch
Re: Upgrade a mail server
Craig Sanders wrote: > > On Mon, Feb 18, 2002 at 06:02:38PM -0600, Rich Puhek wrote: > > Craig Sanders wrote: > > > i'd love to convert it over to Maildir/ but haven't yet found any way > > > that doesn't involve many hours of downtime while converting the > > > mailboxes from mbox format to Maildir. > > > > > > one of these days i'll have the time to sit down and work out a good > > > solution to the problem. i've got some ideas but no time to work them > > > out. > > > > Look at qmail's site. They've got some nice tools for scripting the > > switch. Some will need a little bit of cooking for your specific > > implementaion, but basicly it's there. > > yes, i've seen all that. > > none of the available scripts do the job in what i consider to be a safe > manner while the system is STILL RUNNING as a mail server (handling both > incoming mail AND pop & imap connections from hundreds of users). > Yea, there are some limitations. Remember though, once you switch to Maildir, your script can safely plop data into the Maildirs while MDAs are dropping mail in and MUAs are removing it. That's a key to avoiding disaster, and one of my favorite Maildir features. > > Downtime will be the time it takes to install procmail (or other > > Maildir happy LDA) and restart sendmail (assuming home directories are > > setup properly.) Also need to restart your POP3 daemon of course. > > Then, you can fire off a script to convert the mailboxes. > > no, there is a LOT more to it than that...and the fact that NONE of the > existing conversion scripts take that into account is why i don't > consider any of them safe enough to use as is. > There's definately more to it, but your downtime (in terms of not being reachable via SMTP or POP) is of short duration. Downtime in terms of mail not being accessable will obviously depend on the quantity of data and the machine performance. I agree with the safety issue of the convertion scripts. That's why I added a few wrinkles to the scripts I used. 1) Cooked in some sanity checking (does $HOME/Maildir exist? who owns it?, etc. skip mbox on failure, print warning so I can handle it later). 2) Added a lot of verbosity to the scripts. 3) ran on small chunks of the user list (~100 users at a crack, just so I had less mess to repair if the stuff hit the fan) 4) Before shutting down SMTP and POP: made a copy of /var/spool/mail 5) Immediately after shutting down SMTP and POP: rsynced the copy (less downtime than full copy at that point). > you might want to risk it on a home system with only a few dozen > accounts or less, but nobody in their right mind would convert a > professional mail server with thousands of accounts without first > planning out every step of the transition and taking the time to think > of (and work around) every little thing that *could* go wrong. > Definately true, that's what I did on the last cutovers I did (~1500 users on the last one, about ~3000 users on the one prior). And I got to cut over from Post.Office on NT to Linux/Sendmail/courier-pop. Ugh! > end-users care about their mail, and any fault with the mail system is > highly-visiblethey WILL call the helpdesk if they can't get their > mail. > > there are several race-conditions which must be avoided, and oddities > that WILL cause users to call the helpdesk and whine that they can't > read their mail (e.g. when the pop daemon is Maildir/ but their mailbox > hasn't been converted yet because it takes a long time to convert > several gigabytes of other users mbox files to maildir. users will not > tolerate being unable to access their mail for an hour, let alone a day > or several days) > You definately will get some calls if mail is not showing up. My most recent convertion (the P.O. to Sendmail) had to haul ~2GB of mail up over a 3MB link (moved the mail server from one POP to another while we were at it). Not only did I have to transfer the mail, I got to strip the damn ^Ms out (did I mention how much I miss that Post.Office server?) Did have some complaints the next morning, but everyone was happy when we told them that an upgrade was in progress and the mail was on its way. That (lag in convertion) is the biggest hurdle (assuming you already have user accounts on the machine). You should be alright if you have a reasonably fast machine. Also worth considering is exploiting any maintenance windows you may have. You didn't state if you're an ISP or another institution. If you're a business, university, etc, you most likely have a defined window when "the system may be down". Even if you're an ISP (for some reason ISPs got the shaft, and are supposed to have everything working at full capacity, latest version of everything, but are never allowed to work on anything), you will have an overnight period of very low activity. This time could be perfect for such a cutover. > the problem isn't unsolvable. it's just tedious to get it right. i've > already mapped out ways around thes
Re: Upgrade a mail server
On Tue, 19 Feb 2002, Craig Sanders wrote: > the particular mail server that needs conversion has about 6000 users. > most of whom have very small mailboxes. some have obscenely large > mailboxes (200MB or more). last time i checked there was a total of > about 6GB in /var/spool/mail If you have a similar machine around, just copy the whole /var/spool/mail over and try your conversion scripts on the dry dock :-) Alex -- "Life is what happens to you while you're busy making other plans." John Lennon -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Mon, Feb 18, 2002 at 06:02:38PM -0600, Rich Puhek wrote: > Craig Sanders wrote: > > i'd love to convert it over to Maildir/ but haven't yet found any way > > that doesn't involve many hours of downtime while converting the > > mailboxes from mbox format to Maildir. > > > > one of these days i'll have the time to sit down and work out a good > > solution to the problem. i've got some ideas but no time to work them > > out. > > Look at qmail's site. They've got some nice tools for scripting the > switch. Some will need a little bit of cooking for your specific > implementaion, but basicly it's there. yes, i've seen all that. none of the available scripts do the job in what i consider to be a safe manner while the system is STILL RUNNING as a mail server (handling both incoming mail AND pop & imap connections from hundreds of users). > Downtime will be the time it takes to install procmail (or other > Maildir happy LDA) and restart sendmail (assuming home directories are > setup properly.) Also need to restart your POP3 daemon of course. > Then, you can fire off a script to convert the mailboxes. no, there is a LOT more to it than that...and the fact that NONE of the existing conversion scripts take that into account is why i don't consider any of them safe enough to use as is. you might want to risk it on a home system with only a few dozen accounts or less, but nobody in their right mind would convert a professional mail server with thousands of accounts without first planning out every step of the transition and taking the time to think of (and work around) every little thing that *could* go wrong. end-users care about their mail, and any fault with the mail system is highly-visiblethey WILL call the helpdesk if they can't get their mail. there are several race-conditions which must be avoided, and oddities that WILL cause users to call the helpdesk and whine that they can't read their mail (e.g. when the pop daemon is Maildir/ but their mailbox hasn't been converted yet because it takes a long time to convert several gigabytes of other users mbox files to maildir. users will not tolerate being unable to access their mail for an hour, let alone a day or several days) the problem isn't unsolvable. it's just tedious to get it right. i've already mapped out ways around these problems (*) - or at least the problems that i've thought of - one of these days when i get the time i'll implement it on a test system, and then run it on my real system when it is working to my satisfaction. in short: if you plan the transition right then the users won't even notice that you've changed anything, it will all happen seamlessly behind the scenes. if you don't plan it properly then your helpdesk will be overloaded with calls about mail problems. (*) it involves using semaphore files in each users homedir to indicate whether they are maildir or still mbox, plus procmail/maildrop rules to deliver accordingly, and a pop proxy which chooses whether to connect to the mbox or Maildir capable pop daemon. when that's done it is possible to safely convert all the mboxes to maildirs craig -- craig sanders <[EMAIL PROTECTED]> Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch
Re: Upgrade a mail server
Craig Sanders wrote: > > - configure postfix so that "deferred_transports = local" - this leaves >ALL local deliveries in the queue until i flush it manually while >still allowing the mail server to relay outbound mail to other >servers for my users. > > or > > - create Maildir/ and ~/.conversion_in_progress in each home directory >before starting the conversion script, and have a procmail or >maildrop rule which exited with a temporary failure code (EX_TEMPFAIL >from /usr/include/sysexits.h) if ~/.conversion_in_progress exists. > >this would effectively defer incoming mail only for those users whose >mailbox hasn't been converted yet. > But as long as you have the Maildir created (along with cur, new, and tmp), you can safely deliver even if the user's mailbox has not been converted, as long as you're delivering to the Maildir. Although, to be extra careful, the temp failure should be cool, since it will spool locally. One thought is that you may want to check your SMTP server's setting for how long a message can sit in the queue before it generates a warning message. I know Sendmail defaults to 4 hours, not sure about postfix. Doensn't sound like a problem with the first option above though. >the conversion script would "rm -f ~/.conversion_in_progress" as soon >as the conversion was done. > > this solves one major problem - i don't like the idea of having SMTP > down at all...i won't tolerate it and my users certainly wouldn't. > Actually, external SMTP should be fine, as long as it's less than 4 hours (typical default setting for queue warning). Local users' SMTP connections are the tricky part. ... -- _ Rich Puhek ETN Systems Inc. _ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Mon, Feb 18, 2002 at 09:21:16PM -0600, Rich Puhek wrote: > You didn't state if you're an ISP or another institution. ISP.24/7 operation required. more precisely: it's not my ISP, i'm the senior system admin at an ISP. > > (*) it involves using semaphore files in each users homedir to > > indicate whether they are maildir or still mbox, plus > > procmail/maildrop rules to deliver accordingly, and a pop proxy > > which chooses whether to connect to the mbox or Maildir capable pop > > daemon. when that's done it is possible to safely convert all the > > mboxes to maildirs > > Yow. That might be overkill. I'd try some testing to determine how > long the migration process will take. If you're talking about anything > under 4 hours, you can probably count on running from, say 2AM to 6AM. > Set your, ahem, pickier users to be transfered first. Safety isn't > going to be an issue with Maildirs, the only concern I'd have is > making sure that before you run your script to convert the mboxes you > have disabled any other process that will touch an mbox. Last > convertion I did our customer (an ISP with ~3000 users) had accepted a > maintenance window where POP and SMTP services would be unavailable, > so I simply made sure sendmail and qpopper were dead, then for good > measure I believe I changed permissions, ownership, or the directory > name of /var/spool/mail, just in case. yes, that's another variation. if downtime was only 4-6 hours and if i was willing for users to be unable to fetch mail with POP or IMAP for that time then i could: - configure postfix so that "deferred_transports = local" - this leaves ALL local deliveries in the queue until i flush it manually while still allowing the mail server to relay outbound mail to other servers for my users. or - create Maildir/ and ~/.conversion_in_progress in each home directory before starting the conversion script, and have a procmail or maildrop rule which exited with a temporary failure code (EX_TEMPFAIL from /usr/include/sysexits.h) if ~/.conversion_in_progress exists. this would effectively defer incoming mail only for those users whose mailbox hasn't been converted yet. the conversion script would "rm -f ~/.conversion_in_progress" as soon as the conversion was done. this solves one major problem - i don't like the idea of having SMTP down at all...i won't tolerate it and my users certainly wouldn't. doing this, both sending and fetching mail would work perfectly for users whose mailbox has already been converted. users who have yet to be converted could send mail but not fetch it. that's not a bad compromise...if the entire job could be done in only a few hours. the particular mail server that needs conversion has about 6000 users. most of whom have very small mailboxes. some have obscenely large mailboxes (200MB or more). last time i checked there was a total of about 6GB in /var/spool/mail > If preliminary testing indicates you can do the changeover is a matter > of very few hours (which wouldn't surprise me with decent hardware), you that's the key issue. can the entire system be converted in less than 6 hours. > Here's another angle to consider: perhaps a watchdog script to fire > off a convertion script as soon as the user logs in (PPP-wise, this > does assume you're in a dialup-ISP type environment). By the time they > finish connecting, negotiating, and resolving the POP server IP > address, most of their mail will be converted. On the downside... not > sure I trust a script to run that unattended for such a critical > process. i've already had this idea. it has some merit, but it's more complicated than the one i mentioned in my last message, and certainly more complicated than the above. in general, i disapprove of unneccesary complication - i'm a strong believer in the KISS principle. > Best of luck when you finally convert. You're on the right track to > make sure it doesn't end in disaster. it will happen, but not until i am 100% certain that it's going to work without a single problem. craig -- craig sanders <[EMAIL PROTECTED]> Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
Craig Sanders wrote: > > On Mon, Feb 18, 2002 at 06:02:38PM -0600, Rich Puhek wrote: > > Craig Sanders wrote: > > > i'd love to convert it over to Maildir/ but haven't yet found any way > > > that doesn't involve many hours of downtime while converting the > > > mailboxes from mbox format to Maildir. > > > > > > one of these days i'll have the time to sit down and work out a good > > > solution to the problem. i've got some ideas but no time to work them > > > out. > > > > Look at qmail's site. They've got some nice tools for scripting the > > switch. Some will need a little bit of cooking for your specific > > implementaion, but basicly it's there. > > yes, i've seen all that. > > none of the available scripts do the job in what i consider to be a safe > manner while the system is STILL RUNNING as a mail server (handling both > incoming mail AND pop & imap connections from hundreds of users). > Yea, there are some limitations. Remember though, once you switch to Maildir, your script can safely plop data into the Maildirs while MDAs are dropping mail in and MUAs are removing it. That's a key to avoiding disaster, and one of my favorite Maildir features. > > Downtime will be the time it takes to install procmail (or other > > Maildir happy LDA) and restart sendmail (assuming home directories are > > setup properly.) Also need to restart your POP3 daemon of course. > > Then, you can fire off a script to convert the mailboxes. > > no, there is a LOT more to it than that...and the fact that NONE of the > existing conversion scripts take that into account is why i don't > consider any of them safe enough to use as is. > There's definately more to it, but your downtime (in terms of not being reachable via SMTP or POP) is of short duration. Downtime in terms of mail not being accessable will obviously depend on the quantity of data and the machine performance. I agree with the safety issue of the convertion scripts. That's why I added a few wrinkles to the scripts I used. 1) Cooked in some sanity checking (does $HOME/Maildir exist? who owns it?, etc. skip mbox on failure, print warning so I can handle it later). 2) Added a lot of verbosity to the scripts. 3) ran on small chunks of the user list (~100 users at a crack, just so I had less mess to repair if the stuff hit the fan) 4) Before shutting down SMTP and POP: made a copy of /var/spool/mail 5) Immediately after shutting down SMTP and POP: rsynced the copy (less downtime than full copy at that point). > you might want to risk it on a home system with only a few dozen > accounts or less, but nobody in their right mind would convert a > professional mail server with thousands of accounts without first > planning out every step of the transition and taking the time to think > of (and work around) every little thing that *could* go wrong. > Definately true, that's what I did on the last cutovers I did (~1500 users on the last one, about ~3000 users on the one prior). And I got to cut over from Post.Office on NT to Linux/Sendmail/courier-pop. Ugh! > end-users care about their mail, and any fault with the mail system is > highly-visiblethey WILL call the helpdesk if they can't get their > mail. > > there are several race-conditions which must be avoided, and oddities > that WILL cause users to call the helpdesk and whine that they can't > read their mail (e.g. when the pop daemon is Maildir/ but their mailbox > hasn't been converted yet because it takes a long time to convert > several gigabytes of other users mbox files to maildir. users will not > tolerate being unable to access their mail for an hour, let alone a day > or several days) > You definately will get some calls if mail is not showing up. My most recent convertion (the P.O. to Sendmail) had to haul ~2GB of mail up over a 3MB link (moved the mail server from one POP to another while we were at it). Not only did I have to transfer the mail, I got to strip the damn ^Ms out (did I mention how much I miss that Post.Office server?) Did have some complaints the next morning, but everyone was happy when we told them that an upgrade was in progress and the mail was on its way. That (lag in convertion) is the biggest hurdle (assuming you already have user accounts on the machine). You should be alright if you have a reasonably fast machine. Also worth considering is exploiting any maintenance windows you may have. You didn't state if you're an ISP or another institution. If you're a business, university, etc, you most likely have a defined window when "the system may be down". Even if you're an ISP (for some reason ISPs got the shaft, and are supposed to have everything working at full capacity, latest version of everything, but are never allowed to work on anything), you will have an overnight period of very low activity. This time could be perfect for such a cutover. > the problem isn't unsolvable. it's just tedious to get it right. i've > already mapped out ways around the
Re: Upgrade a mail server
On Mon, Feb 18, 2002 at 06:02:38PM -0600, Rich Puhek wrote: > Craig Sanders wrote: > > i'd love to convert it over to Maildir/ but haven't yet found any way > > that doesn't involve many hours of downtime while converting the > > mailboxes from mbox format to Maildir. > > > > one of these days i'll have the time to sit down and work out a good > > solution to the problem. i've got some ideas but no time to work them > > out. > > Look at qmail's site. They've got some nice tools for scripting the > switch. Some will need a little bit of cooking for your specific > implementaion, but basicly it's there. yes, i've seen all that. none of the available scripts do the job in what i consider to be a safe manner while the system is STILL RUNNING as a mail server (handling both incoming mail AND pop & imap connections from hundreds of users). > Downtime will be the time it takes to install procmail (or other > Maildir happy LDA) and restart sendmail (assuming home directories are > setup properly.) Also need to restart your POP3 daemon of course. > Then, you can fire off a script to convert the mailboxes. no, there is a LOT more to it than that...and the fact that NONE of the existing conversion scripts take that into account is why i don't consider any of them safe enough to use as is. you might want to risk it on a home system with only a few dozen accounts or less, but nobody in their right mind would convert a professional mail server with thousands of accounts without first planning out every step of the transition and taking the time to think of (and work around) every little thing that *could* go wrong. end-users care about their mail, and any fault with the mail system is highly-visiblethey WILL call the helpdesk if they can't get their mail. there are several race-conditions which must be avoided, and oddities that WILL cause users to call the helpdesk and whine that they can't read their mail (e.g. when the pop daemon is Maildir/ but their mailbox hasn't been converted yet because it takes a long time to convert several gigabytes of other users mbox files to maildir. users will not tolerate being unable to access their mail for an hour, let alone a day or several days) the problem isn't unsolvable. it's just tedious to get it right. i've already mapped out ways around these problems (*) - or at least the problems that i've thought of - one of these days when i get the time i'll implement it on a test system, and then run it on my real system when it is working to my satisfaction. in short: if you plan the transition right then the users won't even notice that you've changed anything, it will all happen seamlessly behind the scenes. if you don't plan it properly then your helpdesk will be overloaded with calls about mail problems. (*) it involves using semaphore files in each users homedir to indicate whether they are maildir or still mbox, plus procmail/maildrop rules to deliver accordingly, and a pop proxy which chooses whether to connect to the mbox or Maildir capable pop daemon. when that's done it is possible to safely convert all the mboxes to maildirs craig -- craig sanders <[EMAIL PROTECTED]> Fabricati Diem, PVNC. -- motto of the Ankh-Morpork City Watch -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
Craig Sanders wrote: > i'd love to convert it over to Maildir/ but haven't yet found any way > that doesn't involve many hours of downtime while converting the > mailboxes from mbox format to Maildir. > > one of these days i'll have the time to sit down and work out a good > solution to the problem. i've got some ideas but no time to work them > out. > Look at qmail's site. They've got some nice tools for scripting the switch. Some will need a little bit of cooking for your specific implementaion, but basicly it's there. Downtime will be the time it takes to install procmail (or other Maildir happy LDA) and restart sendmail (assuming home directories are setup properly.) Also need to restart your POP3 daemon of course. Then, you can fire off a script to convert the mailboxes. Good luck! --Rich _ Rich Puhek ETN Systems Inc. _
Re: Upgrade a mail server
On Thu, Feb 14, 2002 at 10:22:08PM +1100, Russell Coker wrote:
> On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote:
> > I have a debian 2.1 email server, with sendmail as smtp server, and
> > qpopper as POP3 server. This box hosts several domains (about 100
> > different domains, using virtual hosting), with a total of 1300
> > users. Because the auth method is based on /etc/passwd, I have 1300
> > unix user accounts, one per mailbox, of course.
> >
> > This is becomming not scalable, and I want to start to use a mail
> > server with auth based on sql, using MySQL for example. I am
> > thinking in use a debian woody( I'll wait till woody becomes
> > stable), with sendmail as smtp server, but I am not sure about the
> > pop3 daemon I have to use.
>
> /etc/passwd should scale to 1300 users, using nscd should solve any
> problems there. But chances are that changing how you look up user
> names won't make any noticable difference in performance.
yes, /etc/passwd scales up to several thousand accounts - at least 5000
or 1 depending on how good your hardware is. you don't need nscd,
just hashed map files for /etc/{passwd,group,shadow). there's a
Makefile in /var/lib/misc which handles that.
if you need to scale beyond that then use LDAP rather than mysql for
account info.
and dump sendmail. it's performance is lousy. use a decent mailer like
postfix (converting from sendmail to postfix is very easy as postfix is
backwards compatible with all but the most obscurely weird sendmail
configurations).
alternatively, use the decent-but-nonfree mailer qmail...but it won't be
a smooth upgrade, you'd have to throw away everything you ever learned
about running an MTA and redo your entire mail system from scratch.
> The best way of killing performance on a POP server is using
> /var/spool/mail. Use Maildir storage and you'll get huge performance
> gains.
>
> I am guessing that you use /var/spool/mail as I can't imagine any
> other way of getting performance problems with only 1300 users.
absolutely true.
POP/IMAP on /var/spool/mail mailboxes is the thing that really kills
performance on one of my mail servers (~6000 accounts in /etc/passwd).
this is especially true for those accounts which have 10s of megabytes
in their mailbox.
i'd love to convert it over to Maildir/ but haven't yet found any way
that doesn't involve many hours of downtime while converting the
mailboxes from mbox format to Maildir.
one of these days i'll have the time to sit down and work out a good
solution to the problem. i've got some ideas but no time to work them
out.
craig
--
craig sanders <[EMAIL PROTECTED]>
Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch
Re: Upgrade a mail server
Craig Sanders wrote: > i'd love to convert it over to Maildir/ but haven't yet found any way > that doesn't involve many hours of downtime while converting the > mailboxes from mbox format to Maildir. > > one of these days i'll have the time to sit down and work out a good > solution to the problem. i've got some ideas but no time to work them > out. > Look at qmail's site. They've got some nice tools for scripting the switch. Some will need a little bit of cooking for your specific implementaion, but basicly it's there. Downtime will be the time it takes to install procmail (or other Maildir happy LDA) and restart sendmail (assuming home directories are setup properly.) Also need to restart your POP3 daemon of course. Then, you can fire off a script to convert the mailboxes. Good luck! --Rich _ Rich Puhek ETN Systems Inc. _ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Thu, Feb 14, 2002 at 10:22:08PM +1100, Russell Coker wrote:
> On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote:
> > I have a debian 2.1 email server, with sendmail as smtp server, and
> > qpopper as POP3 server. This box hosts several domains (about 100
> > different domains, using virtual hosting), with a total of 1300
> > users. Because the auth method is based on /etc/passwd, I have 1300
> > unix user accounts, one per mailbox, of course.
> >
> > This is becomming not scalable, and I want to start to use a mail
> > server with auth based on sql, using MySQL for example. I am
> > thinking in use a debian woody( I'll wait till woody becomes
> > stable), with sendmail as smtp server, but I am not sure about the
> > pop3 daemon I have to use.
>
> /etc/passwd should scale to 1300 users, using nscd should solve any
> problems there. But chances are that changing how you look up user
> names won't make any noticable difference in performance.
yes, /etc/passwd scales up to several thousand accounts - at least 5000
or 1 depending on how good your hardware is. you don't need nscd,
just hashed map files for /etc/{passwd,group,shadow). there's a
Makefile in /var/lib/misc which handles that.
if you need to scale beyond that then use LDAP rather than mysql for
account info.
and dump sendmail. it's performance is lousy. use a decent mailer like
postfix (converting from sendmail to postfix is very easy as postfix is
backwards compatible with all but the most obscurely weird sendmail
configurations).
alternatively, use the decent-but-nonfree mailer qmail...but it won't be
a smooth upgrade, you'd have to throw away everything you ever learned
about running an MTA and redo your entire mail system from scratch.
> The best way of killing performance on a POP server is using
> /var/spool/mail. Use Maildir storage and you'll get huge performance
> gains.
>
> I am guessing that you use /var/spool/mail as I can't imagine any
> other way of getting performance problems with only 1300 users.
absolutely true.
POP/IMAP on /var/spool/mail mailboxes is the thing that really kills
performance on one of my mail servers (~6000 accounts in /etc/passwd).
this is especially true for those accounts which have 10s of megabytes
in their mailbox.
i'd love to convert it over to Maildir/ but haven't yet found any way
that doesn't involve many hours of downtime while converting the
mailboxes from mbox format to Maildir.
one of these days i'll have the time to sit down and work out a good
solution to the problem. i've got some ideas but no time to work them
out.
craig
--
craig sanders <[EMAIL PROTECTED]>
Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Thu, Feb 14, 2002 at 10:33:49AM +0100, Javier Castillo Alcibar wrote: > Hi all!! > > I have a debian 2.1 email server, with sendmail as smtp server, and > qpopper as POP3 server. This box hosts several domains (about 100 different > domains, using virtual hosting), with a total of 1300 users. Because the > auth method is based on /etc/passwd, I have 1300 unix user accounts, one per > mailbox, of course. > > This is becomming not scalable, and I want to start to use a mail > server with auth based on sql, using MySQL for example. I am thinking in use > a debian woody( I'll wait till woody becomes stable), with sendmail as smtp > server, but I am not sure about the pop3 daemon I have to use. Why isn't it scalable? If the getpwent() calls are getting slow look at using libnss-db: This package supports using the "db" mechanism for Name Service Switch. Glibc 2.1.92 and before included this module, but it has been since been split and is now a separate package. If you use .db files for your passwd, shadow, group or other naming databases, then you need this module. Looking things up in db files is simple, fast, and reliable. (An SQL server is complex, slow for small data sets [ie, has a far higher startup cost than a flat file], and the complexity can jeapordize the reliability.) Keeping the db file sync'd is done via 'make'. (See the 'pam_make' module for fun.) > My questions are: > > 1? what POP3 server do you recomend me to use, in order to use sql > method for auth.?? If you still insist on SQL, use something like nss-mysql: This package provides a Name Service Switch that allows your MySQL server act as a name service. It currently features full account support (shadow,passwd and group). Installing nscd will make lookups much faster. (Ie, nscd may cover up some of the startup costs by caching results.) Then you can use whatever servers you want. > 2? what's is the way to "transfer" the info (users, passwords, > emails, etc) from my old box to my new box and database?? A bit of perl to parse /etc/shadow into the database? I'd still (and do) use libnss-db. Simple is good. I love MySQL, but things like /etc/(passwd|shadow) are too critical to trust to something as complex as MySQL when the db libraries are much much more reliable.
Re: Upgrade a mail server
On Thu, Feb 14, 2002 at 10:33:49AM +0100, Javier Castillo Alcibar wrote: > Hi all!! > > I have a debian 2.1 email server, with sendmail as smtp server, and > qpopper as POP3 server. This box hosts several domains (about 100 different > domains, using virtual hosting), with a total of 1300 users. Because the > auth method is based on /etc/passwd, I have 1300 unix user accounts, one per > mailbox, of course. > > This is becomming not scalable, and I want to start to use a mail > server with auth based on sql, using MySQL for example. I am thinking in use > a debian woody( I'll wait till woody becomes stable), with sendmail as smtp > server, but I am not sure about the pop3 daemon I have to use. Why isn't it scalable? If the getpwent() calls are getting slow look at using libnss-db: This package supports using the "db" mechanism for Name Service Switch. Glibc 2.1.92 and before included this module, but it has been since been split and is now a separate package. If you use .db files for your passwd, shadow, group or other naming databases, then you need this module. Looking things up in db files is simple, fast, and reliable. (An SQL server is complex, slow for small data sets [ie, has a far higher startup cost than a flat file], and the complexity can jeapordize the reliability.) Keeping the db file sync'd is done via 'make'. (See the 'pam_make' module for fun.) > My questions are: > > 1? what POP3 server do you recomend me to use, in order to use sql > method for auth.?? If you still insist on SQL, use something like nss-mysql: This package provides a Name Service Switch that allows your MySQL server act as a name service. It currently features full account support (shadow,passwd and group). Installing nscd will make lookups much faster. (Ie, nscd may cover up some of the startup costs by caching results.) Then you can use whatever servers you want. > 2? what's is the way to "transfer" the info (users, passwords, > emails, etc) from my old box to my new box and database?? A bit of perl to parse /etc/shadow into the database? I'd still (and do) use libnss-db. Simple is good. I love MySQL, but things like /etc/(passwd|shadow) are too critical to trust to something as complex as MySQL when the db libraries are much much more reliable. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Fri, 15 Feb 2002 03:53, Alexander List wrote: > > > This is becomming not scalable, and I want to start to use a mail > > > server with auth based on sql, using MySQL for example. I am thinking > > > in use a debian woody( I'll wait till woody becomes stable), with > > > sendmail as smtp server, but I am not sure about the pop3 daemon I have > > > to use. > > The problem I encountered is that if you want to have users with shell > access, you need some mapping from the (whatever: LDAP, MySQL...) > database to the uid/gid. That is needed for file permissions etc. This is easy to do, libpam-ldap and libnss-ldap. > So, if you want to have shell access, there's no way round /etc/passwd I > guess. AFAIK there's some PAM modules for Novell that will auto-adduser if > a user doesn't exist in the passwd database yet. No. Just use LDAP for all of it. It's not difficult to setup and it works well. > If you want to have a sealed e-mail box, you could probably use > Cyrus-IMAPd which also provides a pop server. I know that it is being used > in several universities in Austria, with 1 or more users. So I guess > it scales quite well. Of course installing Cyrus for the first time is not > as plug-and-pray as just throwing in your favourite pop3 server, but > there's a script included that will convert the mailboxes to Cyrus' > format. Cyrus does not scale that well. Cyrus does not work on distributed servers, it does not work well with NFS, and it does not have good support for multiple domains. Cyrus does work well if you want to do exactly what CMU does, have ~25K mail boxes on a single machine in a single domain. > > The best way of killing performance on a POP server is using > > /var/spool/mail. Use Maildir storage and you'll get huge performance > > gains. > > I can only support this opinion. I had a user from the publishing > business, using Outlook on a Macintosh. His Inbox was ~280MB, the mail > server machine had 256MB of RAM and he was coming in from abroad, so a > quite unstable and slow network connection. It was a real nightmare, > because he couldn't delete messages with Outlook, and ssh connections > started to die because of the high packet loss... A large company was doing some tests with my Postal program because of their poor performance for mail serving. They had a Sun E450 machine with 4 UltraSPARC CPUs, 1.25G of RAM, and hardware RAID to support 1000 users, but the performance still sucked because they used mbox. With Maildir any Pentium class machine with 256M of RAM and a couple of IDE drive could deliver better performance. -- http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/projects.html Projects I am working on http://www.coker.com.au/~russell/ My home page
Re: Upgrade a mail server
On Fri, 15 Feb 2002 03:53, Alexander List wrote: > > > This is becomming not scalable, and I want to start to use a mail > > > server with auth based on sql, using MySQL for example. I am thinking > > > in use a debian woody( I'll wait till woody becomes stable), with > > > sendmail as smtp server, but I am not sure about the pop3 daemon I have > > > to use. > > The problem I encountered is that if you want to have users with shell > access, you need some mapping from the (whatever: LDAP, MySQL...) > database to the uid/gid. That is needed for file permissions etc. This is easy to do, libpam-ldap and libnss-ldap. > So, if you want to have shell access, there's no way round /etc/passwd I > guess. AFAIK there's some PAM modules for Novell that will auto-adduser if > a user doesn't exist in the passwd database yet. No. Just use LDAP for all of it. It's not difficult to setup and it works well. > If you want to have a sealed e-mail box, you could probably use > Cyrus-IMAPd which also provides a pop server. I know that it is being used > in several universities in Austria, with 1 or more users. So I guess > it scales quite well. Of course installing Cyrus for the first time is not > as plug-and-pray as just throwing in your favourite pop3 server, but > there's a script included that will convert the mailboxes to Cyrus' > format. Cyrus does not scale that well. Cyrus does not work on distributed servers, it does not work well with NFS, and it does not have good support for multiple domains. Cyrus does work well if you want to do exactly what CMU does, have ~25K mail boxes on a single machine in a single domain. > > The best way of killing performance on a POP server is using > > /var/spool/mail. Use Maildir storage and you'll get huge performance > > gains. > > I can only support this opinion. I had a user from the publishing > business, using Outlook on a Macintosh. His Inbox was ~280MB, the mail > server machine had 256MB of RAM and he was coming in from abroad, so a > quite unstable and slow network connection. It was a real nightmare, > because he couldn't delete messages with Outlook, and ssh connections > started to die because of the high packet loss... A large company was doing some tests with my Postal program because of their poor performance for mail serving. They had a Sun E450 machine with 4 UltraSPARC CPUs, 1.25G of RAM, and hardware RAID to support 1000 users, but the performance still sucked because they used mbox. With Maildir any Pentium class machine with 256M of RAM and a couple of IDE drive could deliver better performance. -- http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/projects.html Projects I am working on http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
RE: Upgrade a mail server
Title: RE: Upgrade a mail server Hi, thanks to all of you for your good answers. The reason to using MySQL is because is more easy to admin users and to implementing in the future web interfaces.. -Mensaje original- De: Jason Lim [mailto:[EMAIL PROTECTED]] Enviado el: jueves, 14 de febrero de 2002 18:34 Para: Russell Coker; Javier Castillo Alcibar; debian-isp@lists.debian.org Asunto: Re: Upgrade a mail server > > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. > > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. > > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. Or he is running this on a 486 ;-)
Re: Upgrade a mail server
> > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. > > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. > > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. Or he is running this on a 486 ;-)
Re: Upgrade a mail server
On Thu, 14 Feb 2002, Russell Coker wrote: > > This is becomming not scalable, and I want to start to use a mail > > server with auth based on sql, using MySQL for example. I am thinking in > > use a debian woody( I'll wait till woody becomes stable), with sendmail as > > smtp server, but I am not sure about the pop3 daemon I have to use. The problem I encountered is that if you want to have users with shell access, you need some mapping from the (whatever: LDAP, MySQL...) database to the uid/gid. That is needed for file permissions etc. So, if you want to have shell access, there's no way round /etc/passwd I guess. AFAIK there's some PAM modules for Novell that will auto-adduser if a user doesn't exist in the passwd database yet. If you want to have a sealed e-mail box, you could probably use Cyrus-IMAPd which also provides a pop server. I know that it is being used in several universities in Austria, with 1 or more users. So I guess it scales quite well. Of course installing Cyrus for the first time is not as plug-and-pray as just throwing in your favourite pop3 server, but there's a script included that will convert the mailboxes to Cyrus' format. > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. I can only support this opinion. I had a user from the publishing business, using Outlook on a Macintosh. His Inbox was ~280MB, the mail server machine had 256MB of RAM and he was coming in from abroad, so a quite unstable and slow network connection. It was a real nightmare, because he couldn't delete messages with Outlook, and ssh connections started to die because of the high packet loss... Alex -- "Forgive me, but I'm talking to a politician." John Simpson, BBC World
Re: Upgrade a mail server
On Thu, 14 Feb 2002, Russell Coker wrote: > On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote: > > I have a debian 2.1 email server, with sendmail as smtp server, and > > qpopper as POP3 server. This box hosts several domains (about 100 different > > domains, using virtual hosting), with a total of 1300 users. Because the > > auth method is based on /etc/passwd, I have 1300 unix user accounts, one > > per mailbox, of course. > > > > This is becomming not scalable, and I want to start to use a mail > > server with auth based on sql, using MySQL for example. I am thinking in > > use a debian woody( I'll wait till woody becomes stable), with sendmail as > > smtp server, but I am not sure about the pop3 daemon I have to use. > > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. I don't think performance is the main issue. It is the username clashes; for example: [EMAIL PROTECTED], [EMAIL PROTECTED], ... [EMAIL PROTECTED] Using virtusertable or other mapping begins to become messy and confusing. In addition, email only accounts don't really need any Unix UID (or wasted home directories). > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. qpopper configured to copy mailboxes can be a performance problem when the mailboxes are huge and clients connect often. Javier, Exim can do the MySQL for authentication; do some searches on the exim-users mailing list archive to find some examples and ideas for POP3 servers that work with it. (It may also be in the Exim FAQ.) Personally, I don't think the extra baggage of any SQL server is needed or wise for just email user authentication. (I simply use vm-pop3d which someday may support db(3) database files.) Jeremy C. Reed ... ISP-FAQ.com -- find answers to your questions http://www.isp-faq.com/
RE: Upgrade a mail server
Title: RE: Upgrade a mail server Hi, thanks to all of you for your good answers. The reason to using MySQL is because is more easy to admin users and to implementing in the future web interfaces.. -Mensaje original- De: Jason Lim [mailto:[EMAIL PROTECTED]] Enviado el: jueves, 14 de febrero de 2002 18:34 Para: Russell Coker; Javier Castillo Alcibar; [EMAIL PROTECTED] Asunto: Re: Upgrade a mail server > > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. > > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. > > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. Or he is running this on a 486 ;-)
Re: Upgrade a mail server
> > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. > > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. > > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. Or he is running this on a 486 ;-) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Thu, 14 Feb 2002, Russell Coker wrote: > > This is becomming not scalable, and I want to start to use a mail > > server with auth based on sql, using MySQL for example. I am thinking in > > use a debian woody( I'll wait till woody becomes stable), with sendmail as > > smtp server, but I am not sure about the pop3 daemon I have to use. The problem I encountered is that if you want to have users with shell access, you need some mapping from the (whatever: LDAP, MySQL...) database to the uid/gid. That is needed for file permissions etc. So, if you want to have shell access, there's no way round /etc/passwd I guess. AFAIK there's some PAM modules for Novell that will auto-adduser if a user doesn't exist in the passwd database yet. If you want to have a sealed e-mail box, you could probably use Cyrus-IMAPd which also provides a pop server. I know that it is being used in several universities in Austria, with 1 or more users. So I guess it scales quite well. Of course installing Cyrus for the first time is not as plug-and-pray as just throwing in your favourite pop3 server, but there's a script included that will convert the mailboxes to Cyrus' format. > The best way of killing performance on a POP server is using /var/spool/mail. > Use Maildir storage and you'll get huge performance gains. I can only support this opinion. I had a user from the publishing business, using Outlook on a Macintosh. His Inbox was ~280MB, the mail server machine had 256MB of RAM and he was coming in from abroad, so a quite unstable and slow network connection. It was a real nightmare, because he couldn't delete messages with Outlook, and ssh connections started to die because of the high packet loss... Alex -- "Forgive me, but I'm talking to a politician." John Simpson, BBC World -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Thu, 14 Feb 2002, Russell Coker wrote: > On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote: > > I have a debian 2.1 email server, with sendmail as smtp server, and > > qpopper as POP3 server. This box hosts several domains (about 100 different > > domains, using virtual hosting), with a total of 1300 users. Because the > > auth method is based on /etc/passwd, I have 1300 unix user accounts, one > > per mailbox, of course. > > > > This is becomming not scalable, and I want to start to use a mail > > server with auth based on sql, using MySQL for example. I am thinking in > > use a debian woody( I'll wait till woody becomes stable), with sendmail as > > smtp server, but I am not sure about the pop3 daemon I have to use. > > /etc/passwd should scale to 1300 users, using nscd should solve any problems > there. But chances are that changing how you look up user names won't make > any noticable difference in performance. I don't think performance is the main issue. It is the username clashes; for example: info@domain1, info@domain2, ... info@domain100. Using virtusertable or other mapping begins to become messy and confusing. In addition, email only accounts don't really need any Unix UID (or wasted home directories). > I am guessing that you use /var/spool/mail as I can't imagine any other way > of getting performance problems with only 1300 users. qpopper configured to copy mailboxes can be a performance problem when the mailboxes are huge and clients connect often. Javier, Exim can do the MySQL for authentication; do some searches on the exim-users mailing list archive to find some examples and ideas for POP3 servers that work with it. (It may also be in the Exim FAQ.) Personally, I don't think the extra baggage of any SQL server is needed or wise for just email user authentication. (I simply use vm-pop3d which someday may support db(3) database files.) Jeremy C. Reed ... ISP-FAQ.com -- find answers to your questions http://www.isp-faq.com/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Upgrade a mail server
On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote: > I have a debian 2.1 email server, with sendmail as smtp server, and > qpopper as POP3 server. This box hosts several domains (about 100 different > domains, using virtual hosting), with a total of 1300 users. Because the > auth method is based on /etc/passwd, I have 1300 unix user accounts, one > per mailbox, of course. > > This is becomming not scalable, and I want to start to use a mail > server with auth based on sql, using MySQL for example. I am thinking in > use a debian woody( I'll wait till woody becomes stable), with sendmail as > smtp server, but I am not sure about the pop3 daemon I have to use. /etc/passwd should scale to 1300 users, using nscd should solve any problems there. But chances are that changing how you look up user names won't make any noticable difference in performance. The best way of killing performance on a POP server is using /var/spool/mail. Use Maildir storage and you'll get huge performance gains. I am guessing that you use /var/spool/mail as I can't imagine any other way of getting performance problems with only 1300 users. -- http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/projects.html Projects I am working on http://www.coker.com.au/~russell/ My home page
Upgrade a mail server
Title: Upgrade a mail server Hi all!! I have a debian 2.1 email server, with sendmail as smtp server, and qpopper as POP3 server. This box hosts several domains (about 100 different domains, using virtual hosting), with a total of 1300 users. Because the auth method is based on /etc/passwd, I have 1300 unix user accounts, one per mailbox, of course. This is becomming not scalable, and I want to start to use a mail server with auth based on sql, using MySQL for example. I am thinking in use a debian woody( I'll wait till woody becomes stable), with sendmail as smtp server, but I am not sure about the pop3 daemon I have to use. My questions are: 1º what POP3 server do you recomend me to use, in order to use sql method for auth.?? 2º what's is the way to "transfer" the info (users, passwords, emails, etc) from my old box to my new box and database?? Any helps would be welcome. Best Regards. un saludo. Javier Castillo Alcíbar - [EMAIL PROTECTED] Alhambra Systems, S.A. - www.alhsys.com c/Albasanz 14, 28037 Madrid Tel.: +34 91 787 23 00 Fax.: +34 91 787 23 01
Re: Upgrade a mail server
On Thu, 14 Feb 2002 20:33, Javier Castillo Alcibar wrote: > I have a debian 2.1 email server, with sendmail as smtp server, and > qpopper as POP3 server. This box hosts several domains (about 100 different > domains, using virtual hosting), with a total of 1300 users. Because the > auth method is based on /etc/passwd, I have 1300 unix user accounts, one > per mailbox, of course. > > This is becomming not scalable, and I want to start to use a mail > server with auth based on sql, using MySQL for example. I am thinking in > use a debian woody( I'll wait till woody becomes stable), with sendmail as > smtp server, but I am not sure about the pop3 daemon I have to use. /etc/passwd should scale to 1300 users, using nscd should solve any problems there. But chances are that changing how you look up user names won't make any noticable difference in performance. The best way of killing performance on a POP server is using /var/spool/mail. Use Maildir storage and you'll get huge performance gains. I am guessing that you use /var/spool/mail as I can't imagine any other way of getting performance problems with only 1300 users. -- http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/projects.html Projects I am working on http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Upgrade a mail server
Title: Upgrade a mail server Hi all!! I have a debian 2.1 email server, with sendmail as smtp server, and qpopper as POP3 server. This box hosts several domains (about 100 different domains, using virtual hosting), with a total of 1300 users. Because the auth method is based on /etc/passwd, I have 1300 unix user accounts, one per mailbox, of course. This is becomming not scalable, and I want to start to use a mail server with auth based on sql, using MySQL for example. I am thinking in use a debian woody( I'll wait till woody becomes stable), with sendmail as smtp server, but I am not sure about the pop3 daemon I have to use. My questions are: 1º what POP3 server do you recomend me to use, in order to use sql method for auth.?? 2º what's is the way to "transfer" the info (users, passwords, emails, etc) from my old box to my new box and database?? Any helps would be welcome. Best Regards. un saludo. Javier Castillo Alcíbar - [EMAIL PROTECTED] Alhambra Systems, S.A. - www.alhsys.com c/Albasanz 14, 28037 Madrid Tel.: +34 91 787 23 00 Fax.: +34 91 787 23 01
