wy96f commented on issue #2750: ARTEMIS-2399 Improve performance when there are a lot of subscribers URL: https://github.com/apache/activemq-artemis/pull/2750#issuecomment-518064779 I've added a new commit to reduce file open/close. The log before this commit showed for each queue depage was followed by two deliver after a few seconds, something like this: `2019-07-31 11:24:12,915 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub86 is 4470916 with maxSize = 5242880. Depaged 2000 messages, pendingDelivery=0, intermediateMessageReferences= 2000, queueDelivering=984 2019-07-31 11:24:15,583 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub86, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@2c21ad1e doing deliver. messageReferences=0 2019-07-31 11:24:22,596 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub86, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@2c21ad1e doing deliver. messageReferences=2 2019-07-31 11:24:22,620 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub86 is 4490706 with maxSize = 5242880. Depaged 2000 messages, pendingDelivery=0, intermediateMessageReferences= 2000, queueDelivering=990 2019-07-31 11:24:25,620 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub86, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@2c21ad1e doing deliver. messageReferences=0 2019-07-31 11:24:33,095 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub86, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@2c21ad1e doing deliver. messageReferences=2` `2019-07-31 11:20:34,082 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub49 is 4473860 with maxSize = 5242880. Depaged 2000 messages, pendingDelivery=0, intermediateMessageReferences= 2000, queueDelivering=982 2019-07-31 11:20:34,161 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub49, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@41a2d788 doing deliver. messageReferences=0 2019-07-31 11:20:44,535 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub49, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@41a2d788 doing deliver. messageReferences=2 2019-07-31 11:20:44,802 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub49 is 4473010 with maxSize = 5242880. Depaged 2000 messages, pendingDelivery=0, intermediateMessageReferences= 2000, queueDelivering=971 2019-07-31 11:20:44,999 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub49, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@41a2d788 doing deliver. messageReferences=0 2019-07-31 11:20:55,230 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub49, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=5d09e339-b32d-11e9-b1ac-fa163e6757d7], temp=false]@41a2d788 doing deliver. messageReferences=2` It is likely paged message is cleared when delivering a few seconds after depage causing opening/closing file to read message. After this commit, depage is immediately followed by deliver. The log was like this: `2019-08-02 16:35:11,317 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub72 is 2227083 with maxSize = 5242880. Depaged 509 messages, pendingDelivery=491, intermediateMessageReferences= 509, queueDelivering=387 2019-08-02 16:35:11,317 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub72, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=e0d3e34f-b4df-11e9-956a-fa163e6757d7], temp=false]@36c11804 doing deliver. messageReferences=491 2019-08-02 16:35:16,467 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub72 is 2224399 with maxSize = 5242880. Depaged 1000 messages, pendingDelivery=0, intermediateMessageReferences= 1000, queueDelivering=268 2019-08-02 16:35:16,467 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub72, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=e0d3e34f-b4df-11e9-956a-fa163e6757d7], temp=false]@36c11804 doing deliver. messageReferences=0` `2019-08-02 16:36:03,048 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub53 is 2225927 with maxSize = 5242880. Depaged 1000 messages, pendingDelivery=0, intermediateMessageReferences= 1000, queueDelivering=1501 2019-08-02 16:36:03,048 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub53, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=e0d3e34f-b4df-11e9-956a-fa163e6757d7], temp=false]@2965382d doing deliver. messageReferences=0 2019-08-02 16:36:06,825 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] Queue Memory Size after depage on queue=sub53 is 2228961 with maxSize = 5242880. Depaged 1000 messages, pendingDelivery=0, intermediateMessageReferences= 1000, queueDelivering=1590 2019-08-02 16:36:06,825 DEBUG [org.apache.activemq.artemis.core.server.impl.QueueImpl] QueueImpl[name=sub53, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=e0d3e34f-b4df-11e9-956a-fa163e6757d7], temp=false]@2965382d doing deliver. messageReferences=0` This new added commit reduces file open/close by about 70% - approximately 220 times per second open/close in first test scenario and 1770 times per second in second test scenario.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
