Re: Sendmail or not?
On Sunday, April 8, 2001, at 04:16 PM, Ask Bjoern Hansen wrote: > On Sun, 8 Apr 2001, Tom Mornini wrote: > > I don't understand why anyone would do anything else than just > forking to qmail-inject. > > At least on Linux and FreeBSD that goes reasonably fast (you're not > sending hundreds and hundreds of mails per minute, are you?), is > really simple and the mail will be delivered faster than your mail > program can login to the mail server. Well, actually, yes, we were sending a HUGE amount of mail for various reasons, none of which were spam-related. :-) The site that I worked on went from a single box with no traffic to 3M fully dynamic, DB transaction, language-translate on the fly, page views per day. The site also generated 16Mbs AVERAGE traffic out of our co-location facility. In addition we were routing over 20M SMS messages per month, many of which were delivered via SMTP. This traffic was on top of the regular account verification and password forgotten e-mails. And in addition to that, we were hosting forward to SMTP and forward to SMS mailboxes. As the site grew, we learned better and better ways of doing things. I'll never again think in terms of "this will work for a while" if I can help it, because when sites get popular, they get popular FAST, and it's a real pain to scale things after the fact. :-) -- -- Tom Mornini -- ICQ 113526784
Re: Sendmail or not?
On Sun, 8 Apr 2001, Tom Mornini wrote: I don't understand why anyone would do anything else than just forking to qmail-inject. At least on Linux and FreeBSD that goes reasonably fast (you're not sending hundreds and hundreds of mails per minute, are you?), is really simple and the mail will be delivered faster than your mail program can login to the mail server. And it supports VERP so you easily can do something about all those bounces. Try it. Try it. - ask -- ask bjoern hansen, http://ask.netcetera.dk/ !try; do();
Re: Sendmail or not?
> - Original Message - > From: "Sean Chittenden" <[EMAIL PROTECTED]> > To: "Per Einar" <[EMAIL PROTECTED]> > Cc: <[EMAIL PROTECTED]> > Sent: Sunday, April 08, 2001 11:42 AM > Subject: Re: Sendmail or not? > >> if you | to sendmail, then you still fork. Mail::Sendmail, I >> bet dime to dollar, forks and execs a process. The only way to avoid >> forking is to open a socket and send the message via SMTP (or QMQP if >> you want to pimp your system out with qmail). Unless you're on AIX or >> solaris and performance is an issue, fork and get the job done w/ >> minimal headaches. -sc > > No, Mail::Sendmail sends through SMTP. That's why I talked about it. > So, you're saying that I should just fork anyway? Will there be a big > performance loss on Linux? And should I use Apache::Subprocess for > this, or > just go ahead with a standard pipe? Oh man, have I been here. After starting with forking, and then graduating to Mail::Sendmail, we finally realized that you could have your cake (queuing) and eat it too (no forking) by using Mail::Sendmail to deliver messages for relay to a mailhost (even if it's the same machine). Performance is best the mailhost is a dedicated machine, or load balanced group of systems. On Sunday, April 8, 2001, at 02:52 AM, Per Einar wrote: -- -- Tom Mornini -- ICQ 113526784
Re: Sendmail or not?
- Original Message - From: "barries" <[EMAIL PROTECTED]> To: "Per Einar" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, April 08, 2001 4:24 PM Subject: Re: Sendmail or not? > On Sun, Apr 08, 2001 at 12:07:01PM +0200, Per Einar wrote: > > > > What I was suggesting was that I could do the SMTP connection myself. But > > you're right, that might involve queuing problems. > > If you do this, make sure that the SMTP connection is to your local MTA, > since 'net delays could mean a mod_perl process hanging around for a > long time waiting for name resolution or a distant or slow MTA to respond. I will consider doing this, yes. > > Also, if you do fork or connect to your local MTA, make sure it's in > queueing mode and not immediate delivery mode, so your mod_perl process > doesn't wait for it to deliver the mail. > > > I don't know anything about named pipes, but isn't it possible to write to > > some sort of intermediate file that is monitored by a daemon process which > > then sends it off to sendmail? > > If you're planning on writing a daemon, a named pipe or Unix domain > socket is good. You'll need a multiprocessing/multithreading daemon, > though, if you don't want to risk blocking while a previous message gets > attended to. Intermediate files mean simpler code, a cron job or simple > sleep(10) polling loop can be used. > > If you're sending new account info, password reminders or other things > that the user might want to immediately check his in box for, you want > them to get to the users ASAP, so forking or SMTPing to a local MTA > makes a lot of sense. Yes, this is exactly my case. That's why I didn't want to wait too long before processing the message. > > If you're already using a database for your app, it'll probably have the > locking issues dealt with, popping messages of the queue and handing > them to an MTA qould be trivial. If you're using a database that > supports triggers you might even be able to get away without perceivable > delays. And you might be able to move the mail handling off to another > machine easily. Thanks a lot to you and to anyone else who contributed: you have helped me a lot into finding solutions for my problem. Per Einar Ellefsen [EMAIL PROTECTED]
Re: Sendmail or not?
On Sun, Apr 08, 2001 at 12:07:01PM +0200, Per Einar wrote: > > What I was suggesting was that I could do the SMTP connection myself. But > you're right, that might involve queuing problems. If you do this, make sure that the SMTP connection is to your local MTA, since 'net delays could mean a mod_perl process hanging around for a long time waiting for name resolution or a distant or slow MTA to respond. Also, if you do fork or connect to your local MTA, make sure it's in queueing mode and not immediate delivery mode, so your mod_perl process doesn't wait for it to deliver the mail. > I don't know anything about named pipes, but isn't it possible to write to > some sort of intermediate file that is monitored by a daemon process which > then sends it off to sendmail? If you're planning on writing a daemon, a named pipe or Unix domain socket is good. You'll need a multiprocessing/multithreading daemon, though, if you don't want to risk blocking while a previous message gets attended to. Intermediate files mean simpler code, a cron job or simple sleep(10) polling loop can be used. If you're sending new account info, password reminders or other things that the user might want to immediately check his in box for, you want them to get to the users ASAP, so forking or SMTPing to a local MTA makes a lot of sense. If you're already using a database for your app, it'll probably have the locking issues dealt with, popping messages of the queue and handing them to an MTA qould be trivial. If you're using a database that supports triggers you might even be able to get away without perceivable delays. And you might be able to move the mail handling off to another machine easily. - Barrie
Re: Sendmail or not?
> No, Mail::Sendmail sends through SMTP. That's why I talked about it. My bad, next time I'll check out the readme. > So, you're saying that I should just fork anyway? If you have an SMTP server that's available for relaying from your host, I'd go for Mail::Sendmail because it seems like a simple interface. > Will there be a big performance loss on Linux? Naw, Linux and FreeBSD are extremely quick at forking. > And should I use Apache::Subprocess for this, or just go ahead with > a standard pipe? Standard pipe is probably okay for your site. Unless you've got a load of .40 or higher, I wouldn't worry about this (and .4 is probably even little low to be worrying about this). Motto: Get it done. If there's a problem, fix it later. Don't prevent yourself from fixing a problem if one arrises. -sc -- Sean Chittenden PGP signature
Re: Sendmail or not?
You might want to actually try it out and see what kind of delays are involved in firing off the MDA. Under peak loads where the bottle neck would too many MDA's running at one time (most of the MDA code I assume would be shared though) and under low loads when the bottle neck would be of executing the MDA. By the way sendmail is not the only MDA available - you would get many MDA's with quite small footprint in case that becomes a bottleneck. What you could also try out is use SMTP to deliver to your own mailserver and then it can do the queueing. Ofcourse here your benchmark gets complicated by the fact that you need to stress test under peak loads for your webserver and mailserver both. Mithun PS: This has been discussed in the past and you might want to look at the archives too.
Re: Sendmail or not?
- Original Message - From: "Mithun Bhattacharya" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, April 08, 2001 11:56 AM Subject: Re: Sendmail or not? > Am I correct in assuming that you are writing to a file based on user > inputs in real time ?? Doesn't that imply you have to go around and > implement some sort of file locking or queuing procedure in the handler > itself ?? Well, if I follow G.W. Haywood's, yes that would be to write the mails to files. But these files wouldn't really need to be locked, as they could be generated in a particular folder, with a name regrouping the time and process id. What I was suggesting was that I could do the SMTP connection myself. But you're right, that might involve queuing problems. > Personally I would rather take the overhead of firing off the > MDA and let the mail subsytem worry about queuing timeouts and whatever > else might come up. That's what I thought too. The queuing is important, and I couldn't possibly handle that myself. This is why I like G.W. Haywood's idea of avoiding the overhead _and_ handing it off to sendmail. My only worry with this system is the fact that it would make the waiting time before receiving the e-mail pretty long. I don't know anything about named pipes, but isn't it possible to write to some sort of intermediate file that is monitored by a daemon process which then sends it off to sendmail? Per Einar Ellefsen [EMAIL PROTECTED]
Re: Sendmail or not?
Am I correct in assuming that you are writing to a file based on user inputs in real time ?? Doesn't that imply you have to go around and implement some sort of file locking or queuing procedure in the handler itself ?? Personally I would rather take the overhead of firing off the MDA and let the mail subsytem worry about queuing timeouts and whatever else might come up. Mithun
Re: Sendmail or not?
- Original Message - From: "Sean Chittenden" <[EMAIL PROTECTED]> To: "Per Einar" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, April 08, 2001 11:42 AM Subject: Re: Sendmail or not? > if you | to sendmail, then you still fork. Mail::Sendmail, I > bet dime to dollar, forks and execs a process. The only way to avoid > forking is to open a socket and send the message via SMTP (or QMQP if > you want to pimp your system out with qmail). Unless you're on AIX or > solaris and performance is an issue, fork and get the job done w/ > minimal headaches. -sc No, Mail::Sendmail sends through SMTP. That's why I talked about it. So, you're saying that I should just fork anyway? Will there be a big performance loss on Linux? And should I use Apache::Subprocess for this, or just go ahead with a standard pipe? Per Einar Ellefsen [EMAIL PROTECTED]
Re: Sendmail or not?
if you | to sendmail, then you still fork. Mail::Sendmail, I bet dime to dollar, forks and execs a process. The only way to avoid forking is to open a socket and send the message via SMTP (or QMQP if you want to pimp your system out with qmail). Unless you're on AIX or solaris and performance is an issue, fork and get the job done w/ minimal headaches. -sc On Sun, Apr 08, 2001 at 08:52:38AM +0200, Per Einar wrote: > There is still one subject about subprocesses under mod_perl which is > unclear to me: should I avoid pipes ot other programs or not? > > What I have understood is that I should avoid forking subprocesses. > > Now, here is my case: I want to send an e-mail to somebody. Should I open a > pipe to sendmail (with open my $fh = gensym(), '|sendmail -t -oi'), or > should I avoid this and use a Perl module like Mail::Sendmail. > > While I think Mail::Sendmail could provide what I need, I guess that > sendmail is better at handling message queues etc, so it will be preferable > to use that. > > What is the best choice in my case? > > Per Einar Ellefsen > [EMAIL PROTECTED] > > > -- Sean Chittenden PGP signature
Re: Sendmail or not?
- Original Message - From: "G.W. Haywood" <[EMAIL PROTECTED]> To: "Per Einar" <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Sunday, April 08, 2001 10:27 AM Subject: Re: Sendmail or not? > Hi there, > > On Sun, 8 Apr 2001, Per Einar wrote: > > > What I have understood is that I should avoid forking subprocesses. > > The reason is (loosely speaking) performance. If you haven't any > worries about that, go ahead and fork. If it were my system I'd > probably just write the mail pieces to files and have a cron job hand > the files to sendmail every hour. Well, I am worried about performance. I like your idea of writing the mails to files. Seems like a handy idea. > > > What is the best choice in my case? > > Only you can have enough information to answer that. Thank you, Per Einar Ellefsen [EMAIL PROTECTED]
Re: Sendmail or not?
Hi there, On Sun, 8 Apr 2001, Per Einar wrote: > What I have understood is that I should avoid forking subprocesses. The reason is (loosely speaking) performance. If you haven't any worries about that, go ahead and fork. If it were my system I'd probably just write the mail pieces to files and have a cron job hand the files to sendmail every hour. > What is the best choice in my case? Only you can have enough information to answer that. 73, Ged.
Sendmail or not?
There is still one subject about subprocesses under mod_perl which is unclear to me: should I avoid pipes ot other programs or not? What I have understood is that I should avoid forking subprocesses. Now, here is my case: I want to send an e-mail to somebody. Should I open a pipe to sendmail (with open my $fh = gensym(), '|sendmail -t -oi'), or should I avoid this and use a Perl module like Mail::Sendmail. While I think Mail::Sendmail could provide what I need, I guess that sendmail is better at handling message queues etc, so it will be preferable to use that. What is the best choice in my case? Per Einar Ellefsen [EMAIL PROTECTED]
