[
https://issues.apache.org/jira/browse/QPID-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004097#comment-17004097
]
Alex Rudyy commented on QPID-8385:
----------------------------------
[~rgodfrey], the implement changes look good to me. I will merge them into 7.1
branch and close this JIRA and QPID-8393 after the merge.
> [Broker-J] Improve performance of dequeuing old queue entries on triggering
> ring policy
> ---------------------------------------------------------------------------------------
>
> Key: QPID-8385
> URL: https://issues.apache.org/jira/browse/QPID-8385
> Project: Qpid
> Issue Type: Improvement
> Components: Broker-J
> Reporter: Alex Rudyy
> Priority: Major
> Fix For: qpid-java-broker-8.0.0
>
> Attachments:
> 0001-QPID-8385-Broker-J-Delete-queue-entry-asynchronously.patch,
> 0001-QPID-8385-Broker-J-Utilize-AsyncAutoCommitTransactio.patch,
> QPID-8385_QPID-8393.patch
>
>
> The performance of dequeuing old queue entries with triggering ring policy is
> affected by the following factors:
> * the oldest entry is not deleted immediately. As result, the deletion of
> the same oldest entry can be requested from multiple publishing IO threads
> * only one publishing thread can acquire the message with
> {{QueueEntry#acquireOrSteal}}, whilst the rest of publishing IO threads would
> set a {{DelayedAcquisitionStateListener}}. As result, thousands of
> {{DelayedAcquisitionStateListeners}} can be potentially set on the same queue
> entry
> * Setting of {{DelayedAcquisitionStateListeners}} from concurrent IO threads
> can consume host CPU up to 100% as there is a tight loop in
> {{StateChangeListenerEntry#add(StateChangeListenerEntry<T,E>)}}. There is a
> risk of running into defect {{QPID-8393}} whilst setting a listener.
> * The old message is deleted in IO thread blocking the enqueing of new
> messages
> It seems that high CPU consumption on setting of
> {{DelayedAcquisitionStateListeners}} is the main contributor for the
> perfromance degradation on old entries dequeueing by ring policy.
> The current approach for dequeuing of old queue entries from ring policy
> should be revisited. We need a mechanism to avoid multiple dequeueing of the
> same queue entry from concurrent threads. Only after introduction of such
> mechanism we can look into further performance optimisation for deletion of
> persistent queue entries.
> AutoCommitTransaction is used to dequeue entries on message expiration or
> breaching ring policy threshold. As result, the dequeueing operation for
> persistent queue entry blocks until queue entry record is removed from the
> store and the underlying store transaction is synced to disk. The sequential
> removal of multiple entries can results in unnecessary delays due to syncing
> disk on every dequeue.
> The broker performance for the corner cases described above can be
> significantly improved by using asynchronous transactions in dequeue
> operations.
> Asynchronous transaction does not wait for the store dequeue transaction to
> sync to disk. As result, the performance of removal unneeded queue entries
> can increase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
