On Wed, May 16, 2001 at 02:38:38PM -0400, John R Levine wrote:
> I have a spam-like application that will be sending out thousands of
> customized single-recipient messages. (It's spam-like because it says
> "you wrote to us about XXXX on YYYY", but unlike spam, they really did
> write and I have the saved messages to prove it.)
>
> Rather than dumping them all into qmail-inject or qmail-queue which would
> cause constipation unless I install the big-todo patch which is a pain, I
> was thinking of calling qmail-remote directly, then qmail-queue if
> qmail-remote didn't work, with a bunch of remotes going at once.
>
> The addresses come out of a database and the customization is trivial, so
> I was planning to write it in perl. (The main bottleneck is the network
> delays for qmail-remote.) But before I do, has someone already written
> this?
I recently did one of these - it was more designed for mass customized
mailings and used a pool of sender servers and a distributed queue -
we're talking millions and millions of email per day here... It's a
complex system and I haven't the code, but I have some experiences
that I can share.
I used IO::select to handle running multiple qmail-remotes at the same
time. qmail-remote has a really small footprint so you can run 1000s
of them concurrently on a modest sized server. It takes a fair amount
of code to manage multiplexed pipes in conjunction with handling
stdout and stderr (execution errors) responses and exit conditions.
(I see that there is an IO::Poll in which case I'd probably use that
in preference to IO::Select because of some of the select limit issues
on some OSes).
The next thing you have to worry about is managing your own queue and
retries for delivery failures. This can be much simpler and faster
than a full qmail-send type queue of course, such as a single flat
file for the whole delivery run with an occassional sync.
Bounces of course you'll handle with some sort of VERP address.
Having said all that, are you talking less than, say, 10,000 mails?
If so, one simple strategy is to inject each mail at the rate of say 1
per second. At that rate 1000 mails are injected in about 16 minutes,
ten thousand in a little less than 3 hours. That sort of injection
rate should not require bigtodo patches so if you don't mind your
delivery script running for 3 hours, then that might be the easiest
strategy.
Regards.
