Hi, I was surfing through google's archive, and some reason my USENET reader didn't catch this reply, so here goes. . .(my comments are embedded below)
"Manuel Lemos" <[EMAIL PROTECTED]> wrote in message [EMAIL PROTECTED]">news:[EMAIL PROTECTED]... > Hello, > > On 12/22/2002 02:52 PM, Jonathan Chum wrote: > >>>I was considering of writing the mass mailing application in PHP instead > >>>though. > >>> > >>>If anyone has eperience writing such applications with this amount of > >>>emails, I'd like to know what you've done. > >> > >>If you do not need to send personalized messages (messages that differ > >>for each recipient), just put all recipients in a BCc: header and send a > >>single message to the local mailer queue (not via SMTP). > >> > >>If you do not care for the users that bounce messages, just make the > >>return path be black hole email address. OTOH, if you care about bounces > >>(you should if you mailing list is large or is not clean), consider > >>using ezmlm, which is a mailing list manager than among other things > >>takes care of bounce messages thanks to qmail VERP. I was told that is > >>the one that eGroups hacked to use in the now known YahooGroups site. > >> > >>Once I built a small Web interface for ezmlm. It was meant just to > >>create and edit several mailing lists meant to be used as newsletter for > >>a portal with many sites. Is simple but it already comes with a SOAP > >>interface to manage the mailing list subscribers remotely. > >> > >>http://www.phpclasses.org/ezmlmmanager > > > > > > I heard that using BCC, it can only handle a certain amount of receipients. > > No, if you look in the archives of this list you will notice that this > was explained several times. What happens is that some ISP of shared > hosting limit the number of recipients of the messages you can send > because they do not want you to do mass mailing at all. You should check > your ISP Acceptable Use Policy before trying to do any mass mailing. > Trying to queue individual messages to each recipient may fool the > server limitations but you may still be against the AUP. > > > > Also, messages that are BCC'd are also tend to be blocked by mail servers > > and clients. > > Any server owners that do that are dumb because while they think they > are blocking spammers, what they get is they are simply blocking opt-in > mailing lists for instance like this one you are getting. At the same > time you still keep getting most of the spam because real spammers use > dedicated machines make the recipients addresses show in the messages as > needed to defeat that silly anti-spam criteria. So, what they get is to > block solicited e-mail. > > Anyway, the problem of personalizing messages is that makes your > mailings take much longer to generate and queue because you need to > create separate message for each recipient. I'm weighing down personalization and enveloping for this project, and probably going for the personalization of each email. This rules out BCC and enveloping with SMTP. > > >>No, queuing via SMTP is the slowest way to send messages. Your script > >>should not bother to deliver the messages to the recipients SMTP > >>servers. Delivery can take hours or days to finish due to network > >>congestions and hard to conect SMTP servers. Just queue the messages in > >>the local mailer and let it take care the actual delivery. > >> > >>I would recommend a qmail based system anytime, with or without ezmlm on > >>top. In a production system that I manage, it just takes 3 seconds to > >>queue a alert message to be sent to 50,000 via a local qmail server. > >> > >>You can also use sendmail almost as fast using the queue only mode. Some > >>people think that sendmail is slow and many forks processes because they > >>are not aware of how to configure it to queue the messages the fastest > >>way that is possible. > >> > >>You may want to look into this class that has a sub-classes for > >>delivering with sendmail program directly instead of using the mail(). > >>It lets you configure the sendmail delivery mode. There is also a > >>sub-class for delivering with qmail. > >> > >>http://www.phpclasses.org/mimemessage > > > > > > I was looking at a variety of methods, and seems that everyone thinks for > > best performance, you would inject the email in to Qmail or Sendmail's queue > > That is because it is really the best. :) Sounds good, definately going with Qmail on this project... > > and run it every hour. > > With qmail you do not have to run the queue like sendmail. You just > inject the messages and it will deliver them when possible. If it delivers when possible, won't the server flake if it get's 100,000 emails inject into Qmail? If I was to use your class and loop through 100,000 emails into Qmail, will Qmail attempt all at once? > > As for SMTP, I meant sending the email through your outbound SMTP server > > which in our case, a server we build specifically for handling that sort of > > Of course all e-mail is delivered to each recipient via SMTP, but that > you can relay to your local mailer. > > > > volume. I was reading through Perl's BulkMail module, > > http://mojo.skazat.com/support/documentation/Bulkmail.pm.html > > > > They are using SMTP to delivery their emails. Using envelopes and sorting > > the emails by their host meant even faster delivery. Though the envelope > > That would be a good idea if you have messages to be sent to many > recipients of the same domain but in reality it is more problematic. > Some servers indeed reject messages to be sent many recipients at once. > Although that is not illegal, it is a pattern of spamming. Another > problem, is that you will have an hard time distinguishing which > addresses of a same domain are bouncing and which are not bouncing. Very interesting... I hadn't thought about that. Definately don't want legit emails mistaken as SPAM. > > > method seems to be like BCC as you can't personalized each email To: > > recepient. > > If you want delivery efficiency, forget personalization. > > > > Your Mime Message class seems to be better for sending mass mailings as it > > queues up the email into Sendmail without opening/closing the connection for > > Actually the class provides different delivery methods that maybe > appropriate in different circunstances: mail(), sendmail, qmail and SMTP. > > SMTP is the slowest for queuing but it is the fastest for direct > delivery. My SMTP class supports direct deliveries which is great to > deliver urgent messages as they are not relayed to the local mailer. It > delivers to the recipient SMTP server and you will know if the message > was accepted right away. I use this to deliver password reminder and > other messages that users of my sites are anxious to receive because the > local mailer queue may be loaded at the moment with some long delivery. That makes sense. > > each recepient. phpmailer.sourceforget.net has that problem of opening and > > closing a connection, yet they claim to receive good results of up to 40,000 > > emails per hour. Another software using that class was able to send 500,000 > > emails in 10 hours. > > You should not consider any delivery statistics because all depends on > factors that have nothing to do with the software you use, like the > outbound bandwidth, remote server reachability, anti-spam reception > delays, temporary messages refusals (mailbox full, blocked account, etc..). Right. . . however I was more curious on a comparison between piping into Qmail vs SMTP. So far, you have prooven greatly the differences and has swayed my decision alot :) > > If injecting the emails directly into Qmail (Sounds to me that Qmail is far > > better for email deliver than to be using Sendmail) vs using SMTP, then > > DJB software rules! BTW, consider using DJBDNS cache to reduce DNS > server lookup time. I'll check it out. We've been using Bind and are considering of using TinyDNS, but also heard good things about DJBDNS. Mainly we are looking for PostGreSQL extensions because we built a web hosting control panel that's rounded around PostGresSQL to manage services. > > > that'll be the approach. The only reason I'd would be using SMTP is to keep > > the code and data (on MySQL) stored on one machine and when it's time to > > blast out an email, it'll establish a connection to a SMTP server assigned > > to the list to deliver the email. > > In that case, you may consider QMQP relay which is a protocol that lets > you rely entire mail queues from a server to another. This is > recommended when you have many busy mailing lists served from one > machine and you can use more servers do the actual delivery. I think > only qmail and ezmlm support QMQP. Thanks, I'll check QMQP as well. > > > Though if the code was written to sit on each server that scans for pending > > tasks, it'll pick up it's job from a master database server and then > > directly injecting Qmail locally. > > That is what qmail does. > > > > The software app I'm writing is expected to have many users with small to > > very large lists. I'm trying to spec this out so that it scales well and > > with the fastest delivery. From what I'm hearing from you and from the first > > guy's reply to this thread, inject the email into Qmail is the quickest way. > > Yes, qmail was thought for that. Yahoogroups use it and I have also used > when I worked in a large portal with many subscriber newsletters as I > mentioned. We had some very large like for MTV Brasil that had almost > 400,000 subscribers. > > The greatest problem that made me learn some hard lessons is that it is > very problematic if you start with subscribers list that are not clean > up of no longer valid users. Once you start delivering messages to those > addresses, you get a flood of bounces that pratically stop your server > queue. > > The MTV newsletter was weekly, but that affected the newsletters of > other sites that were stopped during the MTV newsletter bounces. Since > ezmlm does not unsubscribe bouncing addresses right away, the solution > was to remove bouncing subscribers using an external validation > procedure. I used this other class for the cleanup. After that it was a > breeze. We had peaks of 10,000 messages sent per minute. > > http://www.phpclasses.org/emailvalidation We had a 72K list with about 6K bad emails that Mailman didn't unsubscribe until we used MojoMail which unsubscribes them from the database instantly as well as perform validations. Now, blasts are handled alot quicker. So far, you have given me a good case to use Qmail and using your PHP class as a foundation for this application. . . I'll have to check the other software recommendations you made and see how'd I incorporate this into the application. BTW, isn't there any overhead associating with opening/closing a connection to inject each email into Qmail? Or it's very minimal overhead to even worry about? If 100,000 was looped and injected into Qmail, is Qmail fast enough that 100,000 emails are not just sitting on the server filling up disk space? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
