[
https://issues.apache.org/jira/browse/ARTEMIS-3728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870941#comment-17870941
]
Jean-Pascal Briquet edited comment on ARTEMIS-3728 at 8/5/24 6:15 AM:
----------------------------------------------------------------------
Maybe it is worth reopening this issue. I observed it happening last week on
Artemis 2.35.0 in a dual mirror configuration as well.
The stack trace looks similar to what you had seen, but on my side, it occurred
on the "source" broker, not on the "target"/DR side where applications were
consuming and producing messages.
While it was happening, it effectively blocked all messages arriving in the
queue. A broker restart resolved the situation.
There were no errors or warnings visible beforehand on the source broker, it
just appeared suddenly, and there was nothing on the target broker either.
{code:java}
2024-07-30 05:27:07,636 ERROR [org.apache.activemq.artemis.core.server]
AMQ224041: Failed to deliver
java.lang.NullPointerException: Cannot assign field "next" because "this.tail"
is null
at
org.apache.activemq.artemis.utils.collections.LinkedListImpl.addTail(LinkedListImpl.java:192)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl.addTail(PriorityLinkedListImpl.java:84)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.internalAddTail(QueueImpl.java:3025)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.doInternalPoll(QueueImpl.java:3092)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3148)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4461)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
~[artemis-commons-2.35.0.jar:2.35.0]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
[artemis-commons-2.35.0.jar:2.35.0] {code}
was (Author: JIRAUSER303376):
Maybe it is worth reopening this issue. I observed it happening last week on
Artemis 2.35.0 in a dual mirror configuration as well., I have seen it
happening last week on Artemis 2.35.0 on dual mirror configuration type too.
The stack trace looks similar to what you had seen, but on my side, it occurred
on the "source" broker, not on the "target"/DR side where applications were
consuming and producing messages.
While it was happening, it effectively blocked all messages arriving in the
queue. A broker restart resolved the situation.
There were no errors or warnings visible beforehand on the source broker, it
just appeared suddenly, and there was nothing on the target broker either.
{code:java}
2024-07-30 05:27:07,636 ERROR [org.apache.activemq.artemis.core.server]
AMQ224041: Failed to deliver
java.lang.NullPointerException: Cannot assign field "next" because "this.tail"
is null
at
org.apache.activemq.artemis.utils.collections.LinkedListImpl.addTail(LinkedListImpl.java:192)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl.addTail(PriorityLinkedListImpl.java:84)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.internalAddTail(QueueImpl.java:3025)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.doInternalPoll(QueueImpl.java:3092)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3148)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4461)
~[artemis-server-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
~[artemis-commons-2.35.0.jar:2.35.0]
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
~[artemis-commons-2.35.0.jar:2.35.0]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
[artemis-commons-2.35.0.jar:2.35.0] {code}
> Failed to deliver: java.lang.NullPointerException
> -------------------------------------------------
>
> Key: ARTEMIS-3728
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3728
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.20.0
> Environment: centos 7
> Corretto JDK 11
> Artemis 2.20.0 (with epoll module)
>
> Reporter: Stephen Baker
> Priority: Minor
> Attachments: artemis-1.profile-artms1, broker-1.xml-artms1
>
>
> Seeing the following error in the logs on one of our artemis servers
> repeatedly since last night:
> {noformat}
> 2022-03-17 00:02:20,434 ERROR [org.apache.activemq.artemis.core.server]
> AMQ224041: Failed to deliver: java.lang.NullPointerException
> at
> org.apache.activemq.artemis.utils.collections.LinkedListImpl.addTail(LinkedListImpl.java:141)
> [artemis-commons-2.20.0.jar:]
> at
> org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl.addTail(PriorityLinkedListImpl.java:84)
> [artemis-commons-2.20.0.jar:]
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.internalAddTail(QueueImpl.java:2877)
> [artemis-server-2.20.0.jar:2.20.0]
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.doInternalPoll(QueueImpl.java:2938)
> [artemis-server-2.20.0.jar:2.20.0]
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2963)
> [artemis-server-2.20.0.jar:2.20.0]
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4205)
> [artemis-server-2.20.0.jar:2.20.0]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> [artemis-commons-2.20.0.jar:]
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> [artemis-commons-2.20.0.jar:]
> at
> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
> [artemis-commons-2.20.0.jar:]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> [java.base:]
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> [java.base:]
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [artemis-commons-2.20.0.jar:]{noformat}
> I don't know how it started or how to reproduce, but I can say that we
> switched to dual mirroring last night per the [current
> documentation|https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html].
> In the process we renamed our mirrors (before they had the same name on each
> side), and deleted the old mirror queues through the management console after
> all of the servers were up. We did this on 4 other pairs (all separate
> clusters) without running into this issue.
> These errors are coming only from the live server on the disaster recovery
> site, which has no consumers except the mirror connection.
> Prior to these errors we did have:
> {noformat}
> 2022-03-16 23:52:29,482 WARN
> [org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget]
> Queue $ACTIVEMQ_ARTEMIS_MIRROR_Mirror not found on mirror target, ignoring
> ack for queue=$ACTIVEMQ_ARTEMIS_MIRROR_Mirror, messageID=63165878649,
> nodeID=dea32b83-efd5-11eb-b5b1-0050568fe3b2{noformat}
> in our logs, where that mirror name is the old shared mirror name (not part
> of the current broker.xml).
> Error does not appear to have come back on a restart of the server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@activemq.apache.org
For additional commands, e-mail: issues-h...@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
