[
https://issues.apache.org/jira/browse/JAMES-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068889#comment-18068889
]
Benoit Tellier commented on JAMES-4192:
---------------------------------------
Not quite precisely.
In his proposal the waiting queue is not using a selector (that would have the
exact same pitfall) but a queue browser, which do not suffer from this
constraint. We could then use message ID selector to move messages that are
needed for delivery from the waiting queue to the new one...
The scheduler based approach to retain correctness on browsing likely is very
low level, tangled to ActiveMQ internals. We likely can get a try at it but IMO
we should not disregard higher level alternatives...
> ActiveMQ: delayed mails block the queue while dequeuing
> -------------------------------------------------------
>
> Key: JAMES-4192
> URL: https://issues.apache.org/jira/browse/JAMES-4192
> Project: James Server
> Issue Type: Bug
> Components: Queue
> Affects Versions: 3.7.0, 3.8.0, 3.7.1, 3.7.3, 3.7.4, 3.8.1, 3.7.5, 3.8.2,
> 3.8.3
> Reporter: ouvtam
> Priority: Major
>
> We operate James and observed issues when encountering a lot of temporary
> failures on remote delivery. It seemed James could not dequeue the outgoing
> queue.
> *Bug*
> Delayed mails were enqueued into the regular ActiveMQ queue with
> {{{}NEXT_DELIVERY_TIME{}}}.
> Because of ActiveMQ queue paging ({{{}maxPageSize defaults to 200{}}}),
> delayed messages at the head of the queue could block ready messages behind
> them, preventing non-delayed mails from being dequeued.
> *Steps to reproduce*
> * Configure RemoteDelivery with <maxRetries>1</maxRetries> and a high delay
> time (e.g. <delayTime>60 minutes</delayTime>)
> * Send at least 200 mails to a remote server that causes temporary
> exceptions (e.g. 421 greylist). In our case we configured a <gateway> that
> simulated such behaviour (i.e. always sending a temporary exception 421)
> * Sending the 201 mail will never be dequeued (i.e. returns always null),
> because the 200 prior mails are sitting in the queue (ActiveMQ's
> maxPageSize=200). After 60 minutes of delay time the queue will processed
> eventually.
> *Fix*
> Delayed mails are no longer kept in the regular JMS queue before they are due.
> They are stored in the ActiveMQ Scheduler and only appear in the regular
> queue once their delay expires.
> *Result*
> Non-delayed mails are no longer blocked by delayed mails.
> *Implementation Details*
> I will provide a PR that takes advantage of the ActiveMQ's Scheduler Support.
> We already tested this in our local setup.
> Mails with the delayed JMS property set (i.e.
> org.apache.activemq.ScheduledMessage.AMQ_SCHEDULED_DELAY) will be intercepted
> by ActiveMQ's Scheduler and not added to the queue (e.g. outgoing). Instead,
> the scheduler stores the messages in the scheduler store. After the delay has
> passed the scheduler sends the messages into the queue for the consumer.
> *Alternative*
> Another approach would be to increase maxPageSize (e.g. using Per Destination
> Policies). This would increase the memory footprint of ActiveMQ, because it
> would hold the amount of maxPageSize messages in memory. For delayed messages
> this would not be very efficient, because with high delays they just sit in
> memory. In addition, the problem could occur nonetheless (e.g. lot of
> deliveries)
> *Questions*
> Using ** the scheduler introduces an important question:
> * For instance, if 100 mails are delayed, the
> ActiveMQCacheableMailQueue#getSize would return 0 if no mails are in the
> queue. The reason is the delayed mails sit in the scheduler store. What is
> the contract for ManageableMailQueue#getSize? Should it count the total mails
> with the delayed messages? Or only the ones that have no delay? Since we make
> use of WebAdmin this would be changing behaviour.
> ** Unfortunately, we cannot count the size of the scheduled messages using
> the same approach as in normal queues (i.e. using Statistics Plugin).
> *** Either we use a special JMS browse command to browse the messages in the
> scheduler store while filtering the scheduled messages by queue (i.e. only
> scheduled messages for the queue "outgoing").
> *** Or we activate JMX on the Broker to access the ScheduledMessages
> directly and iterate through all jobs, unmarshal the payload, and filter the
> scheduler jobs by queue (i.e. only scheduled messages for the queue
> "outgoing").
> *** Using WebAdmin it would be great to pass params such as
> /mailQueues/outgoing?delayed
> ** An alternative would be to create two manageable queues by
> ActiveMQMailQueueFactory#createCacheableMailQueue (e.g. "outgoing" and
> "outgoing-delayed") that are manageable through "/mailQueues/outgoing" and
> "/mailQueues/outgoing-delayed"
>
> Happy for feedback, so I can create the PR in the next few days.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
