Hi,
I have been using qmail for the last year and a half and have been closely
following the mailing list at securepoint, and didn't find anything related
to my query, hence I took the liberty of posting it.
The objective is to build a high-volumer server capable of doing mail-merged
email blasts to several lists with 10,000 to 1,000,000 users, provide
detailed reports about the status of emails (sent, bounced, bad email
addresses, opened, forwarded), list management (across multiple lists for
each user) and of course, stability.
Over the period of last 12 months, we explored several options - and finally
settled on qmail (what else?). I am using a Pentium III with Linux Redhat
6.2 installed on it, with 512 MB of RAM, 20 GB HDD and JDK 1.2.2 connected
to a 128 Kbps line.
Following are the topics on which I need your comments/suggestions:-
1. Earlier we used to "Runtime.exec()" qmail-inject and manually give it the
messages. This way, qmail would go on and do the delivery. We would then
parse the log files to find the status of the message.
1.1 We had a unique "from" address for each blast for each user to
uniquely identify each email sent (in maillog). Sometimes, instead of
logging the "From" address, the maillog would have the "replyto" address.
Any ideas why? Is there anything else that can be used to uniquely identify
a message?
1.2 For each blast we want to handle the bounced emails individually (we
would need to update the appropriate table). What do we do for that? We
cannot just "set" environment variables since there will be multiple
mail-merges and blasts happening simultaneously.
1.3 Usually after about 5,000 deliveries, the messages would be stuck in
the queue. We then added the CNAME lookup patch, and this increased to about
10,000. Currently, we "prune" the lists uploaded by the users and send
messages in chunks of 2000, with less than 30 concurrent messages. Any
suggestions what could be the culprit? What can we do to circumvent this
problem?
1.4 What would be the best possible way to handle unsubscribe requests.
Currently we invoke a java program from the .qmail file that updates the
database. Any suggestions how this can be improved upon?
2. We then decided to switch over to using qmail-remote, to circumvent the
queue and the logging problem. This effectively means we will have to do our
own logging. Is there anyway to hand over different messages to qmail-remote
rather than invoking it for each message? We have now decided to change the
implementation so that at any point of time, there will be as many threads
sending messages as the qmail concurrency (say around 100), and the messages
themselves will be broken into chunks of 300 to 500 each. How can we improve
this?
3. Currently, we have our own implementation for checking bad e-mail
addresses, list management, handling bounces and mail-merge. Are there any
guidelines/sample code available (any language), that we can look at?
4 . What other things should we keep in mind to provide stability to the
system? What patches to qmail are advisable to be installed? What should be
the typical server configuration for such a system?
5. On a parallel note, what would be the best algorithm to track forwarded
messages? We make use of cookies right now (but that provides 50% accuracy).
I apologize if I broke some protocol and asked some questions that do not
pertain to this list.
Regards,
manav.
