On 04/01/2010 03:28, Bob Eastbrook wrote:
Hi all,
Thanks for the suggestions. It sounds like the consensus is that I
should avoid polling with POP or IMAP and deal with incoming messages
directly.
I think it depends on your expected load and the results of a few
benchmarks...
The benefit of "push" mail, ie deliver to a command is much faster
processing of requests, with lower latency. The disadvantage is that if
requests come in faster then the backend can process them then they will
either overwhelm your backend or start to queue in (Postfix) and this
will lead to loss of control over the order of processing and fairly
arbitrary delays to certain messages (out of order)
The benefit of deliver to a mailbox and poll is that you can use the
mailbox as a primitive queue system and just pull messages out as fast
as you like and in-order.
Note both methods are pretty inefficient compared with a dedicated
message queue system, eg Amazon SQS, rabbit MQS, etc, etc.
Personally I like the idea of using the mailserver as a basic message
queue system and I use it for several applications. I don't expect high
loads and out of order processing is not an issue so I deliver to a
command which in turn uses curl to deliver to a backend process via
http. I have no real protection against being overwhelmed by quantity
of requests in this architecture, but the expected message speed is only
a few per hour so it's not yet a big deal
I think that while 1,000 messages a sec is not yet a big deal you should
do some serious analysis of your needs. Something which becomes 10-100x
larger will need a serious look at the architecture and you will wish
you did it now and not later
Good luck!
Ed W
