[
https://issues.apache.org/jira/browse/JAMES-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068825#comment-18068825
]
Benoit Tellier commented on JAMES-4192:
---------------------------------------
Another alternative proposed by a coworker:
When a mail needs to be delayed, enqueue it into a dedicated outgoing.waiting
queue instead of outgoing. A lightweight scheduled task periodically moves
messages from outgoing.waiting to outgoing once their JAMES_NEXT_DELIVERY
timestamp is due. The browse(), getSize(), and delete() operations
transparently combine both queues, preserving full mail queue management.
Issues it fixes:
- fixes the issue for real
- do not need to read the scheduler internals, we keep standard JMS...
> ActiveMQ: delayed mails block the queue while dequeuing
> -------------------------------------------------------
>
> Key: JAMES-4192
> URL: https://issues.apache.org/jira/browse/JAMES-4192
> Project: James Server
> Issue Type: Bug
> Components: Queue
> Affects Versions: 3.7.0, 3.8.0, 3.7.1, 3.7.3, 3.7.4, 3.8.1, 3.7.5, 3.8.2,
> 3.8.3
> Reporter: ouvtam
> Priority: Major
>
> We operate James and observed issues when encountering a lot of temporary
> failures on remote delivery. It seemed James could not dequeue the outgoing
> queue.
> *Bug*
> Delayed mails were enqueued into the regular ActiveMQ queue with
> {{{}NEXT_DELIVERY_TIME{}}}.
> Because of ActiveMQ queue paging ({{{}maxPageSize defaults to 200{}}}),
> delayed messages at the head of the queue could block ready messages behind
> them, preventing non-delayed mails from being dequeued.
> *Steps to reproduce*
> * Configure RemoteDelivery with <maxRetries>1</maxRetries> and a high delay
> time (e.g. <delayTime>60 minutes</delayTime>)
> * Send at least 200 mails to a remote server that causes temporary
> exceptions (e.g. 421 greylist). In our case we configured a <gateway> that
> simulated such behaviour (i.e. always sending a temporary exception 421)
> * Sending the 201 mail will never be dequeued (i.e. returns always null),
> because the 200 prior mails are sitting in the queue (ActiveMQ's
> maxPageSize=200). After 60 minutes of delay time the queue will processed
> eventually.
> *Fix*
> Delayed mails are no longer kept in the regular JMS queue before they are due.
> They are stored in the ActiveMQ Scheduler and only appear in the regular
> queue once their delay expires.
> *Result*
> Non-delayed mails are no longer blocked by delayed mails.
> *Implementation Details*
> I will provide a PR that takes advantage of the ActiveMQ's Scheduler Support.
> We already tested this in our local setup.
> Mails with the delayed JMS property set (i.e.
> org.apache.activemq.ScheduledMessage.AMQ_SCHEDULED_DELAY) will be intercepted
> by ActiveMQ's Scheduler and not added to the queue (e.g. outgoing). Instead,
> the scheduler stores the messages in the scheduler store. After the delay has
> passed the scheduler sends the messages into the queue for the consumer.
> *Alternative*
> Another approach would be to increase maxPageSize (e.g. using Per Destination
> Policies). This would increase the memory footprint of ActiveMQ, because it
> would hold the amount of maxPageSize messages in memory. For delayed messages
> this would not be very efficient, because with high delays they just sit in
> memory. In addition, the problem could occur nonetheless (e.g. lot of
> deliveries)
> *Questions*
> Using ** the scheduler introduces an important question:
> * For instance, if 100 mails are delayed, the
> ActiveMQCacheableMailQueue#getSize would return 0 if no mails are in the
> queue. The reason is the delayed mails sit in the scheduler store. What is
> the contract for ManageableMailQueue#getSize? Should it count the total mails
> with the delayed messages? Or only the ones that have no delay? Since we make
> use of WebAdmin this would be changing behaviour.
> ** Unfortunately, we cannot count the size of the scheduled messages using
> the same approach as in normal queues (i.e. using Statistics Plugin).
> *** Either we use a special JMS browse command to browse the messages in the
> scheduler store while filtering the scheduled messages by queue (i.e. only
> scheduled messages for the queue "outgoing").
> *** Or we activate JMX on the Broker to access the ScheduledMessages
> directly and iterate through all jobs, unmarshal the payload, and filter the
> scheduler jobs by queue (i.e. only scheduled messages for the queue
> "outgoing").
> *** Using WebAdmin it would be great to pass params such as
> /mailQueues/outgoing?delayed
> ** An alternative would be to create two manageable queues by
> ActiveMQMailQueueFactory#createCacheableMailQueue (e.g. "outgoing" and
> "outgoing-delayed") that are manageable through "/mailQueues/outgoing" and
> "/mailQueues/outgoing-delayed"
>
> Happy for feedback, so I can create the PR in the next few days.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
