Maximum number of delivery of emails
Hi, I want to send 1 Lacs emails per hour. Please suggest me the steps to achieve this. -- Incase of any further queries, Please feel free to mail me or contact me on the numbers provided below. Thanks Regards, Avinash Pawar Software Engineer. Viva Infomedia Pvt. Ltd. 242, Oshiwara Industrial Centre, New Link Road, Opp. Oshiwara Bus Depot, Goregaon West, Mumbai 400104. Direct: +91.22.40310356 Board: +91.22.40310310 Viva Infomedia: Awarded as Best SME (E-Commerce) at CNBC Emerging India Awards 2009
Re: Maximum number of delivery of emails
On Tue, Sep 07, 2010 at 01:50:30PM +0530, Avinash Pawar // Viva wrote: I want to send 1 Lacs emails per hour. Most readers of this (international) list do not know that 1 lac is 100,000. This usage is largely confined to India. Please suggest me the steps to achieve this. This is approximately ~1700 msgs/min or ~28 msgs/sec. To send at this rate, run ~4-8 parallel threads that inject mail into Postfix via SMTP at approximately this rate. If your network bandwidth is sufficient, your disks are not too slow, the messages are not too large, and the destination systems are willing to accept your mail quickly, Postfix will not only accept mail at this rate, but deliver it with no noticeable latency. I've see real-world throughput that is 10 times higher per machine (commodity server with battery backed RAID controller). -- Viktor.
Re: Maximum number of delivery of emails
* Victor Duchovni victor.ducho...@morganstanley.com: On Tue, Sep 07, 2010 at 01:50:30PM +0530, Avinash Pawar // Viva wrote: I want to send 1 Lacs emails per hour. Most readers of this (international) list do not know that 1 lac is 100,000. This usage is largely confined to India. Ah! I'm reading Sacred games and they talk about Lakhs of Rupees all the time. It's 100k. Ah! -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: Maximum number of delivery of emails
Hi, How many mails can I sent using basic configuration of postfix? Also please give me some idea about postfix performance tuning. On Tue, Sep 7, 2010 at 6:44 PM, Victor Duchovni victor.ducho...@morganstanley.com wrote: On Tue, Sep 07, 2010 at 01:50:30PM +0530, Avinash Pawar // Viva wrote: I want to send 1 Lacs emails per hour. Most readers of this (international) list do not know that 1 lac is 100,000. This usage is largely confined to India. Please suggest me the steps to achieve this. This is approximately ~1700 msgs/min or ~28 msgs/sec. To send at this rate, run ~4-8 parallel threads that inject mail into Postfix via SMTP at approximately this rate. If your network bandwidth is sufficient, your disks are not too slow, the messages are not too large, and the destination systems are willing to accept your mail quickly, Postfix will not only accept mail at this rate, but deliver it with no noticeable latency. I've see real-world throughput that is 10 times higher per machine (commodity server with battery backed RAID controller). -- Viktor. -- Incase of any further queries, Please feel free to mail me or contact me on the numbers provided below. Thanks Regards, Avinash Pawar Software Engineer. Viva Infomedia Pvt. Ltd. 242, Oshiwara Industrial Centre, New Link Road, Opp. Oshiwara Bus Depot, Goregaon West, Mumbai 400104. Direct: +91.22.40310356 Board: +91.22.40310310 Viva Infomedia: Awarded as Best SME (E-Commerce) at CNBC Emerging India Awards 2009
Re: Maximum number of delivery of emails
On Tue, Sep 07, 2010 at 06:50:17PM +0530, Avinash Pawar // Viva wrote: How many mails can I sent using basic configuration of postfix? This question has no answer, except to say that on typical commodity server hardware you are unlikely to send more than ~3,000 msgs/sec per Postfix instance. A queue-manager performance test I ran 2 years ago showed that at near ~3000 msgs/sec, the queue-manager is working non-stop and cannot go any faster (with a queue on RAM disk). One might also add that throughput of 300-400 msgs/sec is realistic for a Postfix system with low CPU overhead (no content filtering) and low disk access latency sending mail to high capacity destinations that are not throttling the traffic. Also please give me some idea about postfix performance tuning. http://www.postfix.org/OVERVIEW.html http://www.postfix.org/QSHAPE_README.html http://www.postfix.org/TUNING_README.html Read these, and then ask specific questions... -- Viktor.
Re: Maximum number of delivery of emails
Zitat von Victor Duchovni victor.ducho...@morganstanley.com: On Tue, Sep 07, 2010 at 06:50:17PM +0530, Avinash Pawar // Viva wrote: How many mails can I sent using basic configuration of postfix? This question has no answer, except to say that on typical commodity server hardware you are unlikely to send more than ~3,000 msgs/sec per Postfix instance. A queue-manager performance test I ran 2 years ago showed that at near ~3000 msgs/sec, the queue-manager is working non-stop and cannot go any faster (with a queue on RAM disk). Just curious: Is it clear where this limit come from eg. is it dependant of the performance which a single core can deliever or is bound by memory latency or some others? Regards Andreas smime.p7s Description: S/MIME Cryptographic Signature
Re: Maximum number of delivery of emails
On Tue, Sep 07, 2010 at 06:13:23PM +0200, lst_ho...@kwsoft.de wrote: This question has no answer, except to say that on typical commodity server hardware you are unlikely to send more than ~3,000 msgs/sec per Postfix instance. A queue-manager performance test I ran 2 years ago showed that at near ~3000 msgs/sec, the queue-manager is working non-stop and cannot go any faster (with a queue on RAM disk). Just curious: Is it clear where this limit come from eg. is it dependant of the performance which a single core can deliever or is bound by memory latency or some others? Single-core CPU limit. The system had 4 CPUs and the load peaked at ~25%. The queue manager is single-threaded, and must do a fair amount of message envelope processing. So the current design tops out at ~2-3k msgs/sec, which is substantially faster than other constraints on real systems, so the queue manager is not your bottleneck in real systems. One could design a queue manager in which message parsing, ... is done in multiple processes, and only the scheduling is done by the central process. Such a queue manager would scale to higher loads, but this is simply not a useful direction at this time. -- Viktor.
Re: Maximum number of delivery of emails
On 09/07/2010 08:07 PM, Victor Duchovni wrote: On Tue, Sep 07, 2010 at 06:13:23PM +0200, lst_ho...@kwsoft.de wrote: This question has no answer, except to say that on typical commodity server hardware you are unlikely to send more than ~3,000 msgs/sec per Postfix instance. A queue-manager performance test I ran 2 years ago showed that at near ~3000 msgs/sec, the queue-manager is working non-stop and cannot go any faster (with a queue on RAM disk). Just curious: Is it clear where this limit come from eg. is it dependant of the performance which a single core can deliever or is bound by memory latency or some others? Single-core CPU limit. The system had 4 CPUs and the load peaked at ~25%. The queue manager is single-threaded, and must do a fair amount of message envelope processing. So the current design tops out at ~2-3k msgs/sec, which is substantially faster than other constraints on real systems, so the queue manager is not your bottleneck in real systems. One could design a queue manager in which message parsing, ... is done in multiple processes, and only the scheduling is done by the central process. Such a queue manager would scale to higher loads, but this is simply not a useful direction at this time. So running 1 instance per CPU (for queue-bound installations) might scale to multi-core setups ? I'm curious which part of the queue process is not thread-safe - or perhaps it's just hard to do so. J.
Re: Maximum number of delivery of emails
Zitat von Victor Duchovni victor.ducho...@morganstanley.com: On Tue, Sep 07, 2010 at 06:13:23PM +0200, lst_ho...@kwsoft.de wrote: This question has no answer, except to say that on typical commodity server hardware you are unlikely to send more than ~3,000 msgs/sec per Postfix instance. A queue-manager performance test I ran 2 years ago showed that at near ~3000 msgs/sec, the queue-manager is working non-stop and cannot go any faster (with a queue on RAM disk). Just curious: Is it clear where this limit come from eg. is it dependant of the performance which a single core can deliever or is bound by memory latency or some others? Single-core CPU limit. The system had 4 CPUs and the load peaked at ~25%. The queue manager is single-threaded, and must do a fair amount of message envelope processing. So the current design tops out at ~2-3k msgs/sec, which is substantially faster than other constraints on real systems, so the queue manager is not your bottleneck in real systems. Ok, so if one wants to really peak out it is more useful to have less cores, but faster ones given that I/O is able to keep up. One could design a queue manager in which message parsing, ... is done in multiple processes, and only the scheduling is done by the central process. Such a queue manager would scale to higher loads, but this is simply not a useful direction at this time. Totally agree. I think the need for pushing 2-3k msgs/sec or more is not that widespread to worry about. Many Thanks for sharing knowledge Andreas smime.p7s Description: S/MIME Cryptographic Signature
Re: Maximum number of delivery of emails
On Tue, Sep 07, 2010 at 09:07:54PM +0200, lst_ho...@kwsoft.de wrote: Single-core CPU limit. The system had 4 CPUs and the load peaked at ~25%. The queue manager is single-threaded, and must do a fair amount of message envelope processing. So the current design tops out at ~2-3k msgs/sec, which is substantially faster than other constraints on real systems, so the queue manager is not your bottleneck in real systems. Ok, so if one wants to really peak out it is more useful to have less cores, but faster ones given that I/O is able to keep up. Purely hypothetical discussion, no real MTA handles 1k messages every second. Yes, a single faster core improves the impractically high ceiling on queue manager throughput when the disk subsystem is so fast that the queue manager is CPU rather than I/O constrained. Today's practical queue managers with queues on spinning disks will get bogged down in I/O first. -- Viktor.
Re: Maximum number of delivery of emails
Victor Duchovni: On Tue, Sep 07, 2010 at 09:07:54PM +0200, lst_ho...@kwsoft.de wrote: Single-core CPU limit. The system had 4 CPUs and the load peaked at ~25%. The queue manager is single-threaded, and must do a fair amount of message envelope processing. So the current design tops out at ~2-3k msgs/sec, which is substantially faster than other constraints on real systems, so the queue manager is not your bottleneck in real systems. Ok, so if one wants to really peak out it is more useful to have less cores, but faster ones given that I/O is able to keep up. Purely hypothetical discussion, no real MTA handles 1k messages every second. Yes, a single faster core improves the impractically high ceiling on queue manager throughput when the disk subsystem is so fast that the queue manager is CPU rather than I/O constrained. Today's practical queue managers with queues on spinning disks will get bogged down in I/O first. Large persistent-memory file systems are becoming available. Some emulate disks and therefore are constrained by disk protocol overhead, while others will run at native speed using file systems that are optimized for FLASH-like technologies. I wonder what the next bottleneck will be: network thoughput/latency, 1000s of mail delivering processes thrashing caches while switching context on many-core systems, or the queue manager. Wietse
