qmail Digest 7 Aug 1999 10:00:01 -0000 Issue 721
Topics (messages 28650 through 28682):
HELP: Distributed mail system
28650 by: Jason Brown <[EMAIL PROTECTED]>
28651 by: Russell Nelson <[EMAIL PROTECTED]>
28658 by: <[EMAIL PROTECTED]>
Qmail throughput
28652 by: Jim Arnott <[EMAIL PROTECTED]>
28653 by: Victor Tavares <[EMAIL PROTECTED]>
28654 by: "Pieckiel, Kevin A" <[EMAIL PROTECTED]>
28655 by: "Johannes Erdfelt" <[EMAIL PROTECTED]>
28656 by: Jim Arnott <[EMAIL PROTECTED]>
28659 by: <[EMAIL PROTECTED]>
28662 by: <[EMAIL PROTECTED]>
28664 by: Jim Arnott <[EMAIL PROTECTED]>
Backward Alias System
28657 by: "Timothy L. Mayo" <[EMAIL PROTECTED]>
28666 by: Russell Nelson <[EMAIL PROTECTED]>
Performance
28660 by: "Cris Daniluk" <[EMAIL PROTECTED]>
28663 by: "Tim Hunter" <[EMAIL PROTECTED]>
28665 by: <[EMAIL PROTECTED]>
28667 by: Mark Delany <[EMAIL PROTECTED]>
28668 by: Jim Arnott <[EMAIL PROTECTED]>
28669 by: "Pieckiel, Kevin A" <[EMAIL PROTECTED]>
28670 by: Mark Delany <[EMAIL PROTECTED]>
28671 by: Mark Delany <[EMAIL PROTECTED]>
28672 by: "David Dyer-Bennet" <[EMAIL PROTECTED]>
28673 by: "Tim Hunter" <[EMAIL PROTECTED]>
28674 by: "Racer X" <[EMAIL PROTECTED]>
Qmail newbie POP problem..
28661 by: "Hawke Robinson" <[EMAIL PROTECTED]>
RH 6.0 compile problem...
28675 by: [EMAIL PROTECTED] (Ed Weinberg)
28676 by: Ken Jones <[EMAIL PROTECTED]>
28677 by: [EMAIL PROTECTED] (Ed Weinberg)
28678 by: Ryan Cleary <[EMAIL PROTECTED]>
28679 by: [EMAIL PROTECTED] (Ed Weinberg)
$HOME must be owned by user?
28680 by: "Scott D. Yelich" <[EMAIL PROTECTED]>
28681 by: Russell Nelson <[EMAIL PROTECTED]>
How to backup ?
28682 by: "Gururaj H.S" <[EMAIL PROTECTED]>
Administrivia:
To subscribe to the digest, e-mail:
[EMAIL PROTECTED]
To unsubscribe from the digest, e-mail:
[EMAIL PROTECTED]
To bug my human owner, e-mail:
[EMAIL PROTECTED]
To post to the list, e-mail:
[EMAIL PROTECTED]
----------------------------------------------------------------------
Hello,
I have a very interesting situation here. I have two servers,
host1.mydomain.com and host2.mydomain.com. I have host1 as the primary
mail server in the DNS-MX record. Both the hosts have qmail-1.03
installed in it.
I have user mailboxes in both the server:
in HOST1: user1, user2, user3
in HOST2: user4, user5, user6
Now i would like to make qmail in my whole system work in such a way
that any mail to [EMAIL PROTECTED] or [EMAIL PROTECTED] or
[EMAIL PROTECTED] be delivered to user1, user2, user3 in HOST1
whereas mail to [EMAIL PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED] be delivered to mailboxes user4, user5, user6 in
HOST2.
Later i would like to implement these concepts to my hosts in
different cities (POP) so that every user of our network will have
same template of e-mail address ([EMAIL PROTECTED]) without any
mention of the hostname of the server in his/her city.
If anybody has any idea making this distributed system, please help.
Thanks in advance.
Jason
Jason Brown writes:
> in HOST1: user1, user2, user3
> in HOST2: user4, user5, user6
>
> Now i would like to make qmail in my whole system work in such a way
> that any mail to [EMAIL PROTECTED] or [EMAIL PROTECTED] or
> [EMAIL PROTECTED] be delivered to user1, user2, user3 in HOST1
> whereas mail to [EMAIL PROTECTED] [EMAIL PROTECTED]
> [EMAIL PROTECTED] be delivered to mailboxes user4, user5, user6 in
> HOST2.
echo '|forward "$LOCAL@HOST2"' >host1:~alias/.qmail-default
echo '|forward "$LOCAL@HOST1"' >host2:~alias/.qmail-default
A little more complication will let you generate a better bounce
message if someone sends mail to [EMAIL PROTECTED]
--
-russ nelson <[EMAIL PROTECTED]> http://crynwr.com/~nelson
Crynwr sells support for free software | PGPok | Government schools are so
521 Pleasant Valley Rd. | +1 315 268 1925 voice | bad that any rank amateur
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | can outdo them. Homeschool!
On Fri, 6 Aug 1999, Jason Brown wrote:
> Hello,
>
> I have a very interesting situation here. I have two servers,
> host1.mydomain.com and host2.mydomain.com. I have host1 as the primary
> mail server in the DNS-MX record. Both the hosts have qmail-1.03
> installed in it.
>
> I have user mailboxes in both the server:
>
> in HOST1: user1, user2, user3
> in HOST2: user4, user5, user6
>
> Now i would like to make qmail in my whole system work in such a way
> that any mail to [EMAIL PROTECTED] or [EMAIL PROTECTED] or
> [EMAIL PROTECTED] be delivered to user1, user2, user3 in HOST1
> whereas mail to [EMAIL PROTECTED] [EMAIL PROTECTED]
> [EMAIL PROTECTED] be delivered to mailboxes user4, user5, user6 in
> HOST2.
>
> Later i would like to implement these concepts to my hosts in
> different cities (POP) so that every user of our network will have
> same template of e-mail address ([EMAIL PROTECTED]) without any
> mention of the hostname of the server in his/her city.
put the maildirs on a high-performance server using RAID and a logging
filesystem (eg sun's online disksuite). cross-mount the filestores between
the servers. Users then read mail from their maildirs usinf qmail-pop3d
off any of the servers.
front-end the servers with a number of qmail systems which deliver over
NFS.
you can then play around with secondary interface addresses on the popmail
servers to publish the same IP address at each city which is assosiated
with (say) pop.mydomain.com and is then routed to the nearest pop-server
RjL
All,
Thanks for all your help and pointers. Looks like its the disk bandwith. When I
take out the fsync's in qmail-queue it drops down to 24 seconds (41.6/sec).
SUMMARY:
Could only inject a 1000 byte file into the queue at a rate of 13.6/sec.
On a ~433 Alpha with Digital UNIX 4.0.
CONCLUSION:
The queue is very disk intensive and all writes are fsync'ed.
Things to try:
(thanks to David Dyer-Bennet,Daemeon Reiydelle, [EMAIL PROTECTED],
[EMAIL PROTECTED])
Remove the fsync call in qmail-queue.c
Use a battery backed ram drive.
Send directly before queing and queue those that fail (does qmail
support this)
Performance tune the system
SCSI controller with write-back cache
Run qmail-queue directly (qmail-queue does the fsync's though)
thanks again,
Jim Arnott
How do you do this with Qmail? Isn't that what Sendmail does?
> Another way is to make an initial attempt to deliver the message
> directly, before queueing it. Since some fairly large proportion of
> messages go through on the first try (depends a lot on your address
> mix, but 50% to 95%), queueing only the ones that fail of initial
> delivery can also be a big win.
--vic
Another option, call qmail-remote directly. You'll require a
preformatted message, but ultimately THAT is the program that actually
delivers the message to a machine. But on the other hand, you'd have to
check for errors if it can't be delivered...
Kevin
-----Original Message-----
From: Jim Arnott [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 06, 1999 10:02 AM
To: [EMAIL PROTECTED]
Subject: Re: Qmail throughput
All,
Thanks for all your help and pointers. Looks like its the disk bandwith.
When I
take out the fsync's in qmail-queue it drops down to 24 seconds
(41.6/sec).
SUMMARY:
Could only inject a 1000 byte file into the queue at a rate of 13.6/sec.
On a ~433 Alpha with Digital UNIX 4.0.
CONCLUSION:
The queue is very disk intensive and all writes are fsync'ed.
Things to try:
(thanks to David Dyer-Bennet,Daemeon Reiydelle,
[EMAIL PROTECTED],
[EMAIL PROTECTED])
Remove the fsync call in qmail-queue.c
Use a battery backed ram drive.
Send directly before queing and queue those that fail (does
qmail
support this)
Performance tune the system
SCSI controller with write-back cache
Run qmail-queue directly (qmail-queue does the fsync's though)
thanks again,
Jim Arnott
On Fri, Aug 06, 1999, Jim Arnott <[EMAIL PROTECTED]> wrote:
>
> Thanks for all your help and pointers. Looks like its the disk bandwith. When I
> take out the fsync's in qmail-queue it drops down to 24 seconds (41.6/sec).
>
> SUMMARY:
>
> Could only inject a 1000 byte file into the queue at a rate of 13.6/sec.
> On a ~433 Alpha with Digital UNIX 4.0.
>
> CONCLUSION:
>
> The queue is very disk intensive and all writes are fsync'ed.
>
> Things to try:
>
> (thanks to David Dyer-Bennet,Daemeon Reiydelle, [EMAIL PROTECTED],
> [EMAIL PROTECTED])
>
> Remove the fsync call in qmail-queue.c
> Use a battery backed ram drive.
> Send directly before queing and queue those that fail (does qmail
> support this)
> Performance tune the system
> SCSI controller with write-back cache
> Run qmail-queue directly (qmail-queue does the fsync's though)
This still seems VERY slow, esp for Digital Unix. Which patch version of
Digital Unix are you using? What kind of partition are you writing to?
AdvFS or UFS?
JE
> On Fri, Aug 06, 1999, Jim Arnott <[EMAIL PROTECTED]> wrote:
> >
> > Thanks for all your help and pointers. Looks like its the disk bandwith.
When I
> > take out the fsync's in qmail-queue it drops down to 24 seconds (41.6/sec).
> >
> > SUMMARY:
> >
> > Could only inject a 1000 byte file into the queue at a rate of 13.6/sec.
> > On a ~433 Alpha with Digital UNIX 4.0.
> >
> > CONCLUSION:
> >
> > The queue is very disk intensive and all writes are fsync'ed.
> >
> > Things to try:
> >
> > (thanks to David Dyer-Bennet,Daemeon Reiydelle, [EMAIL PROTECTED],
> > [EMAIL PROTECTED])
> >
> > Remove the fsync call in qmail-queue.c
> > Use a battery backed ram drive.
> > Send directly before queing and queue those that fail (does qmail
> > support this)
> > Performance tune the system
> > SCSI controller with write-back cache
> > Run qmail-queue directly (qmail-queue does the fsync's though)
>
> This still seems VERY slow, esp for Digital Unix. Which patch version of
> Digital Unix are you using? What kind of partition are you writing to?
> AdvFS or UFS?
>
> JE
>
# uname -a
OSF1 uws001.vms.bridge.com V4.0 878 alpha
/var/qmail is on the usr_domain#usr partition and its AdvFS.
-jim
On Fri, 6 Aug 1999, Jim Arnott wrote:
> CONCLUSION:
> The queue is very disk intensive and all writes are fsync'ed.
they are supposed to be! anything else risks you loosing messages if the
system crashes
> Things to try:
>
> (thanks to David Dyer-Bennet,Daemeon Reiydelle, [EMAIL PROTECTED],
> [EMAIL PROTECTED])
>
> Remove the fsync call in qmail-queue.c
> Use a battery backed ram drive.
> Send directly before queing and queue those that fail (does qmail
> support this)
> Performance tune the system
> SCSI controller with write-back cache
> Run qmail-queue directly (qmail-queue does the fsync's though)
+++++++ stop using splogger
Richard
On Fri, 6 Aug 1999, Victor Tavares wrote:
>
> How do you do this with Qmail? Isn't that what Sendmail does?
>
>
> > Another way is to make an initial attempt to deliver the message
> > directly, before queueing it. Since some fairly large proportion of
> > messages go through on the first try (depends a lot on your address
> > mix, but 50% to 95%), queueing only the ones that fail of initial
> > delivery can also be a big win.
basically. assuming your mail message is in a file called mail-out.txt and
includes all of the necessary headers (To, From, Date, Subject, etc), and
you want to send it to [EMAIL PROTECTED] from [EMAIL PROTECTED]
qmail-remote example.com [EMAIL PROTECTED] [EMAIL PROTECTED] \
<mail-out.txt >reports.txt
you then parse reports.txt and decide if the deliverd failed -- if it
did then you invoke qmail-queue to put it in qmail-send's queue to get it
sent sometime later.
Obviously, from a perl script one uses a bi-directional pipe to
communicate with the sub-processes for efficiency, or one could have a
large number of processes doing things, which could give unlimited
concurrency (subject to memory...)
RjL
> On Fri, 6 Aug 1999, Jim Arnott wrote:
>
> > CONCLUSION:
> > The queue is very disk intensive and all writes are fsync'ed.
>
> they are supposed to be! anything else risks you loosing messages if the
> system crashes
> > Things to try:
> >
> > (thanks to David Dyer-Bennet,Daemeon Reiydelle, [EMAIL PROTECTED],
> > [EMAIL PROTECTED])
> >
> > Remove the fsync call in
> > Use a battery backed ram drive.
> > Send directly before queing and queue those that fail (does qmail
> > support this)
> > Performance tune the system
> > SCSI controller with write-back cache
> > Run qmail-queue directly (qmail-queue does the fsync's though)
>
> +++++++ stop using splogger
I dont use the splogger, I use cyclog.
Also qmail-queue does not log (as far as I know) and in this "experiment"
I did not evaen have qmail-send running as stated in my original message.
-jim
>
> Richard
>
As soon as you put the domains into control/locals you made the entries in
control/virtualdomains totally useless. Domains must go in EITHER
control/locals or control/virtualdomains. THEY CANNOT BE PUT IN BOTH
FILES. If they are the domain is considered local and the virtualdomains
entry is IGNORED.
On Thu, 5 Aug 1999, Daniel Callan wrote:
> Hi Magnus,
>
> Thanks for your reply.
>
> At 08:21 5/08/99 +0200, you wrote:
> >> I know that the archive is full of alias problems but so far I cannot
> >> seem to find someone mentioning the simplest (and worst) problem of
> >> them all: If an account exists that is the same name as a virtual-domain
> >> specific alias (or even any alias for that matter), the account takes
> >> preference over the alias and the mail goes straight to the account.
> >
> >This is not correct.
>
> Which part?
>
> >I don't know what you mean by "virtual-domain
> >specific alias",
>
> I meant (in /etc/aliases):
>
> [EMAIL PROTECTED]: account1 (domain specific alias)
> [EMAIL PROTECTED]: account2
>
> AS OPPOSED TO
>
> design: account1 (gloabal alias)
>
> >but only domains that are local - i.e. exists in
> >control/locals will result in an immediate delivery to the local user if
> >the user exists.
> > ALL local deliveries are done first.
>
> All of the virtual domains I'm using are in control/locals AND
> in control/virtdomains. If they weren't, I couldn't use them as domains
> for all the local accounts. None of the domains are soley for either
> purpose, they are all used for BOTH accounts and aliases.
>
> Point is, Sendmail was able to have virtual domains apply to all
> known accounts and aliases (WITH ALIASES CHECKED BEFORE ACCOUNTS).
>
> >
> >> Am I the only who thinks this is completey backwards???
> >
> >Unfortunately not.
>
> Glad to hear that at least ;-)
>
> >
> >> Has anyone managed to get a sane alias system going under Qmail?
> >
> >yes. indeed.
> >
> >Check out
> >http://x42.com/doc/qmail/vdomains1.txt
> >for an old posting to this list.
> >
> >There will be a section in LWQ about this.
> >
>
> I did and strangely enough it doesn't vary from our setup very much
> at all. More particularly, we have already achieved the goals of that
> page. Only we DO have them in locals (to make the normal accounts respond
> to that domain too) and we run the aliases straight from the /etc/aliases
> (to keep the lusers from wiping their .qmail* files under FTP)
>
> ie:
> >>I am trying to set up multiple domains with multiple aliases. For
> > example:
> > Joe B. has a login of joeb and an email address of - [EMAIL PROTECTED] Yet
> > Joe H. has a login in as joeh and an email address of - [EMAIL PROTECTED]
> >
>
> I can already do this quite effectively :-)
>
> What I CAN'T do is:
>
> Joe B. has a login of "joeb" and an email address of - [EMAIL PROTECTED]
> (WORKS OK)
> Joe H. has a login in as "joeh" and an email address of - [EMAIL PROTECTED]
> (STILL WORKS)
> Joe ? comes along and gets the login "joe" for email of - [EMAIL PROTECTED]
> (DOH! THIS ACCOUNT GETS MAIL FOR ALL THREE OF THEM)
>
> This was not the case in Sendmail. Why change?
>
> As I mentioned earlier, I definately require BOTH kinds of
> functionality for each domain (ie: local aliases AND local accounts).
> We are a Virtual ISP / Web Domain hosting service and have about
> 200 virtual domains on this mailserver (about 50 of which have 100-500
> local email accounts for each of that clients' users AND a gaggle of aliases
> for themselves, some of which shorter than 8 chars eg: design, info).
>
> This is why solutions such as "Domain only works for local aliases" or
> "Domain only works for local accounts" are not even an option for us.
> I need both. :-\
>
> It just seems so silly to check accounts before aliases, for any reason
> at all. I really can't fathom a single benefit from that order.
>
>
> Thanks for the help anyway,
> (sorry if I'm sounding out of sorts,
> I am having the day from hell here)
>
> Regards,
> -Daniel
>
>
> Daniel Callan
> System Engineer/
> Senior Programmer
>
> [EMAIL PROTECTED]
> -- DataLine.net.au --
> http://dataline.net.au
>
---------------------------------
Timothy L. Mayo mailto:[EMAIL PROTECTED]
Senior Systems Administrator
localconnect(sm)
http://www.localconnect.net/
The National Business Network Inc. http://www.nb.net/
One Monroeville Center, Suite 850
Monroeville, PA 15146
(412) 810-8888 Phone
(412) 810-8886 Fax
Perhaps it's a good time to point people to
http://www.qmail.org/qmail-lint-0.53 . It catches problems like these.
Timothy L. Mayo writes:
> As soon as you put the domains into control/locals you made the entries in
> control/virtualdomains totally useless. Domains must go in EITHER
> control/locals or control/virtualdomains. THEY CANNOT BE PUT IN BOTH
> FILES. If they are the domain is considered local and the virtualdomains
> entry is IGNORED.
--
-russ nelson <[EMAIL PROTECTED]> http://crynwr.com/~nelson
Crynwr sells support for free software | PGPok | Government schools are so
521 Pleasant Valley Rd. | +1 315 268 1925 voice | bad that any rank amateur
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | can outdo them. Homeschool!
Well, we're starting into our testing of qmail so that we can transition
away from the garbage-polluted ms smtp server that we had such a long thread
about earlier this week.
Basically, we constructed a temporary system for benchmarking. It is a dual
p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
cheetahs on a raid 0 stripe. The system has 160mb ram.
The first benchmark we ran was to see how fast we could populate the queue.
I made a script to sequentially fill the queue with 20kb messages. It was
able to do 2000 20kb messages in approximately 16 seconds (precision was
only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
is more than adequate for our needs. I was very impressed by the fact that
qmail could populate the queue so quickly. I think that definitely goes to
show the scalability of qmail.
The next test we're going to do is to use it as a mail relay, relaying from
the message generator machines out to the net. For the short term, we are
going to run 4 separate qmails with 4 separate queues. Each instance will be
on a separate ip, though. What needs to be done to qmail to make it bind to
a specific IP? This is pretty vital that we bind to separate ips because
eventually we will be putting in 4 network cards (one for each queue).
To further increase our hardware aresenal, once we find the optimal
performance setup, we're going to build 5 of them. We'll have 5 machines
generating mail, 5 sending, and we hope to be able to send upward of 10
million or more per day. At that time we'll also have a 256mb cache on the
raid controller so that the queue can run much more efficiently.
I think that everyone on the qmail list deserves a big thanks from all of us
for the valuable information and insight. It appears that qmail will be a
successful solution for us, and ironically, thousands of dollars cheaper
than the Big Hardware Big Software microsoft solution that we were using
before.
Once we get the network cards in and binded, we'll be on our way to a
wonderful solution...
Cris Daniluk
MicroStrategy
Makes me amazed at the machines that people have to run mail anymore.
My company mail server is a p75, 32M, 2G IDE
I have only 30 users but alot of throughput, never had any problems with
queuing or ever had to reboot for any reason other than a kernel update.
Maybe I should run some benchmarks just to show how great qmail is on a
piece of dirt machine.
-----Original Message-----
From: Cris Daniluk [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 06, 1999 4:31 PM
To: QMail
Subject: Performance
Well, we're starting into our testing of qmail so that we can transition
away from the garbage-polluted ms smtp server that we had such a long thread
about earlier this week.
Basically, we constructed a temporary system for benchmarking. It is a dual
p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
cheetahs on a raid 0 stripe. The system has 160mb ram.
The first benchmark we ran was to see how fast we could populate the queue.
I made a script to sequentially fill the queue with 20kb messages. It was
able to do 2000 20kb messages in approximately 16 seconds (precision was
only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
is more than adequate for our needs. I was very impressed by the fact that
qmail could populate the queue so quickly. I think that definitely goes to
show the scalability of qmail.
The next test we're going to do is to use it as a mail relay, relaying from
the message generator machines out to the net. For the short term, we are
going to run 4 separate qmails with 4 separate queues. Each instance will be
on a separate ip, though. What needs to be done to qmail to make it bind to
a specific IP? This is pretty vital that we bind to separate ips because
eventually we will be putting in 4 network cards (one for each queue).
To further increase our hardware aresenal, once we find the optimal
performance setup, we're going to build 5 of them. We'll have 5 machines
generating mail, 5 sending, and we hope to be able to send upward of 10
million or more per day. At that time we'll also have a 256mb cache on the
raid controller so that the queue can run much more efficiently.
I think that everyone on the qmail list deserves a big thanks from all of us
for the valuable information and insight. It appears that qmail will be a
successful solution for us, and ironically, thousands of dollars cheaper
than the Big Hardware Big Software microsoft solution that we were using
before.
Once we get the network cards in and binded, we'll be on our way to a
wonderful solution...
Cris Daniluk
MicroStrategy
On Fri, 6 Aug 1999, Cris Daniluk wrote:
> on a separate ip, though. What needs to be done to qmail to make it bind to
> a specific IP? This is pretty vital that we bind to separate ips because
> eventually we will be putting in 4 network cards (one for each queue).
tcpserver takes the listen IP address ('host') as an argument -- that gets
your message into one of the queues. There is a patch on www.qmail.org
which allows qmail-remote to bind to a specific local address
Richard
At 04:36 PM Friday 8/6/99, you wrote:
>Makes me amazed at the machines that people have to run mail anymore.
>My company mail server is a p75, 32M, 2G IDE
>I have only 30 users but alot of throughput, never had any problems with
Right. But presumably Cris is talking about something a "tad" bigger. I
found that I needed around 4 X Sparc 5s for 60,000-100,000 users.
That's a system that has to deal with over 2,000 times your load and there
are plenty bigger...
Mark.
>queuing or ever had to reboot for any reason other than a kernel update.
>Maybe I should run some benchmarks just to show how great qmail is on a
>piece of dirt machine.
>
>-----Original Message-----
>From: Cris Daniluk [mailto:[EMAIL PROTECTED]]
>Sent: Friday, August 06, 1999 4:31 PM
>To: QMail
>Subject: Performance
>
>
>Well, we're starting into our testing of qmail so that we can transition
>away from the garbage-polluted ms smtp server that we had such a long thread
>about earlier this week.
>
>Basically, we constructed a temporary system for benchmarking. It is a dual
>p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
>cheetahs on a raid 0 stripe. The system has 160mb ram.
>
>The first benchmark we ran was to see how fast we could populate the queue.
>I made a script to sequentially fill the queue with 20kb messages. It was
>able to do 2000 20kb messages in approximately 16 seconds (precision was
>only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
>is more than adequate for our needs. I was very impressed by the fact that
>qmail could populate the queue so quickly. I think that definitely goes to
>show the scalability of qmail.
>
>The next test we're going to do is to use it as a mail relay, relaying from
>the message generator machines out to the net. For the short term, we are
>going to run 4 separate qmails with 4 separate queues. Each instance will be
>on a separate ip, though. What needs to be done to qmail to make it bind to
>a specific IP? This is pretty vital that we bind to separate ips because
>eventually we will be putting in 4 network cards (one for each queue).
>
>To further increase our hardware aresenal, once we find the optimal
>performance setup, we're going to build 5 of them. We'll have 5 machines
>generating mail, 5 sending, and we hope to be able to send upward of 10
>million or more per day. At that time we'll also have a 256mb cache on the
>raid controller so that the queue can run much more efficiently.
>
>I think that everyone on the qmail list deserves a big thanks from all of us
>for the valuable information and insight. It appears that qmail will be a
>successful solution for us, and ironically, thousands of dollars cheaper
>than the Big Hardware Big Software microsoft solution that we were using
>before.
>
>Once we get the network cards in and binded, we'll be on our way to a
>wonderful solution...
>
>Cris Daniluk
>MicroStrategy
Wow, thats very impressive throughput.
-jim
>
> Well, we're starting into our testing of qmail so that we can transition
> away from the garbage-polluted ms smtp server that we had such a long thread
> about earlier this week.
>
> Basically, we constructed a temporary system for benchmarking. It is a dual
> p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
> cheetahs on a raid 0 stripe. The system has 160mb ram.
>
> The first benchmark we ran was to see how fast we could populate the queue.
> I made a script to sequentially fill the queue with 20kb messages. It was
> able to do 2000 20kb messages in approximately 16 seconds (precision was
> only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
> is more than adequate for our needs. I was very impressed by the fact that
> qmail could populate the queue so quickly. I think that definitely goes to
> show the scalability of qmail.
>
> The next test we're going to do is to use it as a mail relay, relaying from
> the message generator machines out to the net. For the short term, we are
> going to run 4 separate qmails with 4 separate queues. Each instance will be
> on a separate ip, though. What needs to be done to qmail to make it bind to
> a specific IP? This is pretty vital that we bind to separate ips because
> eventually we will be putting in 4 network cards (one for each queue).
>
> To further increase our hardware aresenal, once we find the optimal
> performance setup, we're going to build 5 of them. We'll have 5 machines
> generating mail, 5 sending, and we hope to be able to send upward of 10
> million or more per day. At that time we'll also have a 256mb cache on the
> raid controller so that the queue can run much more efficiently.
>
> I think that everyone on the qmail list deserves a big thanks from all of us
> for the valuable information and insight. It appears that qmail will be a
> successful solution for us, and ironically, thousands of dollars cheaper
> than the Big Hardware Big Software microsoft solution that we were using
> before.
>
> Once we get the network cards in and binded, we'll be on our way to a
> wonderful solution...
>
> Cris Daniluk
> MicroStrategy
>
Hehe... mine is a 486/66 with 16 megs of RAM. We have about 50 users
and no problems. I thought about doing some benchmarking, but haven't
gotten around to it yet.
Kevin
-----Original Message-----
From: Tim Hunter [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 06, 1999 4:37 PM
To: Qmail
Subject: RE: Performance
Makes me amazed at the machines that people have to run mail anymore.
My company mail server is a p75, 32M, 2G IDE
I have only 30 users but alot of throughput, never had any problems with
queuing or ever had to reboot for any reason other than a kernel update.
Maybe I should run some benchmarks just to show how great qmail is on a
piece of dirt machine.
The advise about using tcpserver is very sound, not just because of the
binding benefits.
The other benefits are that you can constrain resource consumption nicely.
Eg, on a 256M system that is, say, just doing inbound smtp, you might use
tcpserver to constrain the concurrency to 250 or so. You might also set
resource limits for tcpserver (which are then inherited by qmail-smtpd) so
that no single smtp session can consume more than, say 1M of memory.
This way, the worst thing that can happen is that an smtp session will be
deferred. The benefit is that you know that the system cannot be overloaded
by external connections - malicious or otherwise.
I tend to log and graph the tcpserver output that way I can see peak session
usage trends over time, which tell you how close you are to running out of
resources.
Mark.
At 04:31 PM Friday 8/6/99, Cris Daniluk wrote:
>Well, we're starting into our testing of qmail so that we can transition
>away from the garbage-polluted ms smtp server that we had such a long thread
>about earlier this week.
>
>Basically, we constructed a temporary system for benchmarking. It is a dual
>p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
>cheetahs on a raid 0 stripe. The system has 160mb ram.
>
>The first benchmark we ran was to see how fast we could populate the queue.
>I made a script to sequentially fill the queue with 20kb messages. It was
>able to do 2000 20kb messages in approximately 16 seconds (precision was
>only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
>is more than adequate for our needs. I was very impressed by the fact that
>qmail could populate the queue so quickly. I think that definitely goes to
>show the scalability of qmail.
>
>The next test we're going to do is to use it as a mail relay, relaying from
>the message generator machines out to the net. For the short term, we are
>going to run 4 separate qmails with 4 separate queues. Each instance will be
>on a separate ip, though. What needs to be done to qmail to make it bind to
>a specific IP? This is pretty vital that we bind to separate ips because
>eventually we will be putting in 4 network cards (one for each queue).
>
>To further increase our hardware aresenal, once we find the optimal
>performance setup, we're going to build 5 of them. We'll have 5 machines
>generating mail, 5 sending, and we hope to be able to send upward of 10
>million or more per day. At that time we'll also have a 256mb cache on the
>raid controller so that the queue can run much more efficiently.
>
>I think that everyone on the qmail list deserves a big thanks from all of us
>for the valuable information and insight. It appears that qmail will be a
>successful solution for us, and ironically, thousands of dollars cheaper
>than the Big Hardware Big Software microsoft solution that we were using
>before.
>
>Once we get the network cards in and binded, we'll be on our way to a
>wonderful solution...
>
>Cris Daniluk
>MicroStrategy
At 03:38 PM Friday 8/6/99, Jim Arnott wrote:
>Wow, thats very impressive throughput.
>-jim
I just hope that the queue was actually processed at that rate. Sure you can
invoke qmail-inject and have all the messages in todo, but they really need
to be processed by qmail-send before you can say the queue injection is
complete.
Cris? Did you actually check the output of qmail-qstat at the end of that
test to ensure that
"messages in queue but not yet preprocessed" was zero?
Mark.
> >
> > Well, we're starting into our testing of qmail so that we can transition
> > away from the garbage-polluted ms smtp server that we had such a long
> thread
> > about earlier this week.
> >
> > Basically, we constructed a temporary system for benchmarking. It is a dual
> > p2 400 with a dpt smartraid5 controller and 64mb cache. We put in 2 9.1gb
> > cheetahs on a raid 0 stripe. The system has 160mb ram.
> >
> > The first benchmark we ran was to see how fast we could populate the queue.
> > I made a script to sequentially fill the queue with 20kb messages. It was
> > able to do 2000 20kb messages in approximately 16 seconds (precision was
> > only to the second, so 15.1-16.9 are valid ranges). 125 messages per second
> > is more than adequate for our needs. I was very impressed by the fact that
> > qmail could populate the queue so quickly. I think that definitely goes to
> > show the scalability of qmail.
> >
> > The next test we're going to do is to use it as a mail relay, relaying from
> > the message generator machines out to the net. For the short term, we are
> > going to run 4 separate qmails with 4 separate queues. Each instance
> will be
> > on a separate ip, though. What needs to be done to qmail to make it bind to
> > a specific IP? This is pretty vital that we bind to separate ips because
> > eventually we will be putting in 4 network cards (one for each queue).
> >
> > To further increase our hardware aresenal, once we find the optimal
> > performance setup, we're going to build 5 of them. We'll have 5 machines
> > generating mail, 5 sending, and we hope to be able to send upward of 10
> > million or more per day. At that time we'll also have a 256mb cache on the
> > raid controller so that the queue can run much more efficiently.
> >
> > I think that everyone on the qmail list deserves a big thanks from all
> of us
> > for the valuable information and insight. It appears that qmail will be a
> > successful solution for us, and ironically, thousands of dollars cheaper
> > than the Big Hardware Big Software microsoft solution that we were using
> > before.
> >
> > Once we get the network cards in and binded, we'll be on our way to a
> > wonderful solution...
> >
> > Cris Daniluk
> > MicroStrategy
> >
>
Tim Hunter <[EMAIL PROTECTED]> writes on 6 August 1999 at 16:36:48 -0400
> Makes me amazed at the machines that people have to run mail anymore.
> My company mail server is a p75, 32M, 2G IDE
> I have only 30 users but alot of throughput, never had any problems with
> queuing or ever had to reboot for any reason other than a kernel update.
> Maybe I should run some benchmarks just to show how great qmail is on a
> piece of dirt machine.
I'd be interested in seeing that sort of statistics too -- closer to
my real world, for one thing :-) .
Note that the person whose configuration you were marveling at is
looking to move 10 MILLION customized individual email messages a day.
His configuration is much bigger than I use for moving mail -- but
he's aiming at moving several orders of magnitude more mail than I
move, too, so that seems fair.
(And gee it's smart to build a test configuration and do some
benchmarking before committing to something that seems likely to push
the limits of hardware / software technology!)
--
David Dyer-Bennet ***NOTE ADDRESS CHANGES*** [EMAIL PROTECTED]
http://dd-b.lighthunters.net/ (photos) Minicon: http://www.mnstf.org/minicon
http://www.dd-b.net/dd-b (sf) http://ouroboros.demesne.com/ Ouroboros Bookworms
Join the 20th century before it's too late!
Should someone setup some kind of standard benchmarking? I'm interested, as
I'm sure that 1/2 the list is now on how well our systems perform. I think
that this should be easily duplicated on any mail system (Unix at least) so
we can prove to the masses how scaleable qmail is.
BTW I am glad I don't have 10 Million emails to go out everyday. I like my
cushy Sysadmin Job =)
-----Original Message-----
From: David Dyer-Bennet [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 06, 1999 5:14 PM
To: Qmail
Subject: RE: Performance
Tim Hunter <[EMAIL PROTECTED]> writes on 6 August 1999 at 16:36:48 -0400
> Makes me amazed at the machines that people have to run mail anymore.
> My company mail server is a p75, 32M, 2G IDE
> I have only 30 users but alot of throughput, never had any problems with
> queuing or ever had to reboot for any reason other than a kernel update.
> Maybe I should run some benchmarks just to show how great qmail is on a
> piece of dirt machine.
I'd be interested in seeing that sort of statistics too -- closer to
my real world, for one thing :-) .
Note that the person whose configuration you were marveling at is
looking to move 10 MILLION customized individual email messages a day.
His configuration is much bigger than I use for moving mail -- but
he's aiming at moving several orders of magnitude more mail than I
move, too, so that seems fair.
(And gee it's smart to build a test configuration and do some
benchmarking before committing to something that seems likely to push
the limits of hardware / software technology!)
--
David Dyer-Bennet ***NOTE ADDRESS CHANGES*** [EMAIL PROTECTED]
http://dd-b.lighthunters.net/ (photos) Minicon: http://www.mnstf.org/minicon
http://www.dd-b.net/dd-b (sf) http://ouroboros.demesne.com/ Ouroboros
Bookworms
Join the 20th century before it's too late!
just to throw in my 2 cents-
we've built a linux toaster setup here that we call our "cluster". right
now it's 4 boxes sitting behind a layer 4 switch from foundry networks. the
switch does automatic load balancing over the machines (right now it's round
robin, but it can give weightings to different machines), and it will also
remove a machine from the cluster if it stops responding.
anyway, the 4 machines are kind of a mix, but they're all p2-300's or better
with plenty of ram. what we've found from our testing is that the backend
disk storage is almost always going to be the bottleneck and the single
point of failure.
our cluster does SMTP inbound and outbound, storing the messages on a
central spool (currently NFS, gack), and it also does POP access. load
testing proved that qmail could queue up the local disk substantially faster
than things could be delivered to the final endpoint over NFS. still, we
estimated that these 4 machines could probably handle around 3 million SMTP
messages a day (that's from SMTP session start until message is queued),
around 500k deliveries/day, or some combination thereof, but that's
dependent on NFS.
from the SMTP side, the cluster appears to scale almost linearly. on the
local delivery and POP side, we're looking into some high end storage
systems to better manage the load. EMC has a product called Celerra that
looks neat; if anyone has any experience with it i'd love to hear about it.
one thing we've done is to use separate network interface cards for the
"front side" and "back side". that is, all internet traffic goes over one
card, and all the NFS traffic goes over another. this keeps traffic from
going over the same wire more than once (ie, a large message will go over
the internet card once, and over the NFS card once, rather than a single
card twice). anecdotal and empirical evidence seems to suggest it's faster,
but YMMV.
shag
=====
Judd Bourgeois | CNM Network +1 (805) 520-7170
Software Architect | 1900 Los Angeles Avenue, 2nd Floor
[EMAIL PROTECTED] | Simi Valley, CA 93065
...yours is not the less noble because no drum beats before you when
you go out into your daily battlefields, and no crowds shout about your
coming when you return from your daily victory or defeat.
--Robert Louis Stevenson
----- Original Message -----
From: David Dyer-Bennet <[EMAIL PROTECTED]>
To: Qmail <[EMAIL PROTECTED]>
Sent: Fri 6 Aug 1999 14.14
Subject: RE: Performance
> Tim Hunter <[EMAIL PROTECTED]> writes on 6 August 1999 at 16:36:48 -0400
> > Makes me amazed at the machines that people have to run mail anymore.
> > My company mail server is a p75, 32M, 2G IDE
> > I have only 30 users but alot of throughput, never had any problems
with
> > queuing or ever had to reboot for any reason other than a kernel
update.
> > Maybe I should run some benchmarks just to show how great qmail is on a
> > piece of dirt machine.
>
> I'd be interested in seeing that sort of statistics too -- closer to
> my real world, for one thing :-) .
>
> Note that the person whose configuration you were marveling at is
> looking to move 10 MILLION customized individual email messages a day.
> His configuration is much bigger than I use for moving mail -- but
> he's aiming at moving several orders of magnitude more mail than I
> move, too, so that seems fair.
>
> (And gee it's smart to build a test configuration and do some
> benchmarking before committing to something that seems likely to push
> the limits of hardware / software technology!)
> --
> David Dyer-Bennet ***NOTE ADDRESS CHANGES***
[EMAIL PROTECTED]
> http://dd-b.lighthunters.net/ (photos) Minicon:
http://www.mnstf.org/minicon
> http://www.dd-b.net/dd-b (sf) http://ouroboros.demesne.com/ Ouroboros
Bookworms
> Join the 20th century before it's too late!
>
Sure could use a hint as to what's hanging me up on this,any suggestions are
appreciated:
I am running Linux 2.2.5-15 (RH6) and qmail 1.03 and daemontools 0.53 and
checkpassword 0.81.
I can send email fine (smtp is ok), and my server is getting the emails and
holding them for me (I can read my emails if I ssh (or telnet) into the
server with my account and type "mail" and do what I wish with it. but if I
start up netscape on that server and try to get email I get "ERR-
authorization failed" error. (of course it does the same remotely as well).
One concern I have is the server doesn't seem to be putting my email into
the maildir(S) (As I thought I had configured, but must have done wrong
somewhere?), it just puts it in the mailbox instead (I think?). But even
though that may be wrong, wouldn't I still be able to check pop3 without an
error? I just wouldn't actually\get any emai right? I thought maybe I had
screwed up the checkpassword somehow (simple though it may be), so
reinstalled it, but made no difference. If i test checkpassword run as su
(or root) root I can test the user and pass just fine, (but of course if I'm
running as a regular user and run the test steps it doens't matter which
user I pass to it, I get the ERR- authentication failed). Probably one
simple piece I missed, I appreeciate any insight.
Thanks
-Hawke Robinson
CNA, MCP+I, MCSE, ASin CS, more on the way...
Sun Solaris Admin, Java Programmer, CCNA.
not sure if I am missing a lib or something, but when I type:
#make setup check
linux responds:
./compile qmail-local.c
qmail-local.c:1: sys/types.h: No such file or directory
qmail-local.c:2: sys/stat.h: No such file or directory
make: *** [qmail-local.o] Error 1
any ideas?
-- Ed Weinberg,
Detel, Inc., An Internet Presence Provider
[EMAIL PROTECTED]
Ed Weinberg wrote:
>
> not sure if I am missing a lib or something, but when I type:
> #make setup check
> linux responds:
> ./compile qmail-local.c
> qmail-local.c:1: sys/types.h: No such file or directory
> qmail-local.c:2: sys/stat.h: No such file or directory
> make: *** [qmail-local.o] Error 1
>
> any ideas?
>
> -- Ed Weinberg,
> Detel, Inc., An Internet Presence Provider
> [EMAIL PROTECTED]
You didn't install the linux headers. You missed a package
--
Ken Jones
mailto:[EMAIL PROTECTED]
http://www.inter7.com/qmailadmin/ - web based qmail adminstration
On Fri, 06 Aug 1999 18:26:09 -0500, Ken Jones <[EMAIL PROTECTED]> wrote:
>Ed Weinberg wrote:
>>
>> not sure if I am missing a lib or something, but when I type:
>> #make setup check
>> linux responds:
>> ./compile qmail-local.c
>> qmail-local.c:1: sys/types.h: No such file or directory
>> qmail-local.c:2: sys/stat.h: No such file or directory
>> make: *** [qmail-local.o] Error 1
>You didn't install the linux headers. You missed a package
I think I did:
# rpm -qa|grep headers
kernel-headers-2.2.5-22
-- Ed Weinberg,
Detel, Inc., An Internet Presence Provider
[EMAIL PROTECTED]
On Fri, 6 Aug 1999, Ed Weinberg wrote:
> not sure if I am missing a lib or something, but when I type:
> #make setup check
> linux responds:
> ./compile qmail-local.c
> qmail-local.c:1: sys/types.h: No such file or directory
> qmail-local.c:2: sys/stat.h: No such file or directory
> make: *** [qmail-local.o] Error 1
These two headers are part of package glibc-devel-2.1.1-6, which
apparently you don't have installed. To install this package, go to
wherever you installed redhat 6.0 from, go to the RedHat/RPMS directory,
and do the command
rpm -ivh glibc-devel-*
(Replace i386 with whatever architecture you are running, if it's
not Intel.)
--Ryan Cleary
_______________________________________________________________________
Ryan Cleary <[EMAIL PROTECTED]>
On Fri, 6 Aug 1999 19:46:06 -0400 (EDT), Ryan Cleary
<[EMAIL PROTECTED]> wrote:
>These two headers are part of package glibc-devel-2.1.1-6, which
>apparently you don't have installed. To install this package, go to
I found this out minutes before I received this email!
Thanks.
-- Ed Weinberg,
Detel, Inc., An Internet Presence Provider
[EMAIL PROTECTED]
-----BEGIN PGP SIGNED MESSAGE-----
Perhaps DJB could answer this best.... but I'm wondering why
the home directory has to be owned by the user and if it's
not, .qmail files don't seem to be used.
I would like to have the home directory for a user to be owned by root
so the user can not modify the files, yet qmail seems to think this is a
bad thing. I noticed it doesn't seem to have a problem with the actual
.qmail* files owned by root -- but the current dir needs to be owned by
the user.
Any thoughts?
Scott
-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
iQCVAwUBN6uZ7B4PLs9vCOqdAQHtlAQAigaZ4t4kEz976jgpVaFgCMjHIgt8ZYhb
InWVnlBDbGv+allTlvqCMXmA3/ZxguZPuH5Z3BBPAy5Mj3iwqMUfpfuXBJaV53bM
+AIsRldFuCYfliLCVnGErC9UyKNcDc4pDt6IXaOF1sO2Rsk6xVdCdg5+sxC6oQnN
O5x/7xV39GY=
=FP1j
-----END PGP SIGNATURE-----
Scott D. Yelich writes:
> Perhaps DJB could answer this best.... but I'm wondering why
> the home directory has to be owned by the user and if it's
> not, .qmail files don't seem to be used.
Given /etc/passwd and the user's home directory, how do you decide
whether an entry in /etc/passwd should receive mail? Look for a
.qmail file and give up otherwise? That's a reasonable choice, but it
involves creating a bunch of .qmail files, all of which are empty.
But also consider that qmail-local runs as the user, and if the user
doesn't own their home directory, they can't modify their mailbox.
--
-russ nelson <[EMAIL PROTECTED]> http://crynwr.com/~nelson
Crynwr sells support for free software | PGPok | Government schools are so
521 Pleasant Valley Rd. | +1 315 268 1925 voice | bad that any rank amateur
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | can outdo them. Homeschool!
|
Hi,
We are using a linux 5.2
server as our Mail Server with Qmail as Mail
software. I would like to know how to take
backup of these and also what
all needs to be backed up in order to restore in
case of a failure with minimum
time.
Thanks in advance
Gururaj
|
