jbonofre opened a new issue, #1907: URL: https://github.com/apache/activemq/issues/1907
I observed a race condition on active/passive mode where the passive broker can start the Jetty server (and so the webconsole) where waiting for the lock on KahaDB. This is a behavior change as the Jetty/webconsole is not supposed to start on the passive (slave). I investigated and it's due to Spring 6.2.x update. Before Spring 6.2.x, Spring used a strict locking: during `preInstantiateSingletons`, bean creation was strictly serialized on a single thread. The broker's `@PostConstruct` in `XBeanBrokerService` calls `start()` which blocks on the store lock (in `LockableServiceSupport.preStart()`). Since everything was single threaded, the Jetty `invokeStart` bean was never reached (the context was stuck waiting for the broker to acquire the lock). So on a slave, Jetty and webconsole were not started at all. Spring 6.2.x introduced two key changes: 1. Lenient locking: non-bootstrap threads can now proceed with bean creation without waiting for the main thread. 2. Pre-instantiation skip (see https://github.com/spring-projects/spring-framework/issues/34349): `preInstantiateSingletons` now skips beans already in creation by another thread and moves on to the next bean. So now, while the broker's `@PostConstruct` is blocking on the store lock, Spring 6.2.x can proceed to instantiate and start Jetty. It means the user can access the webconsole but get an error: `getBrokerAdmin()` (in the webconsole) returns `null` because the broker hasn't finished starting, causing this JSP error: ``` 2026-04-08 18:21:36,725 | WARN | /admin/ | org.eclipse.jetty.server.HttpChannel | qtp1825662648-26 jakarta.servlet.ServletException: jakarta.servlet.ServletException: org.apache.jasper.JasperException: An exception occurred processing [/decorators/head.jsp] at line [20] 17: limitations under the License. 18: --%> 19: 20: <title><c:out value="${requestContext.brokerQuery.brokerAdmin.brokerName} : ${pageTitle}" /></title> 21: 22: <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> 23: Stacktrace: at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:157) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.Server.handle(Server.java:563) at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: jakarta.servlet.ServletException: org.apache.jasper.JasperException: An exception occurred processing [/decorators/head.jsp] at line [20] ``` Users have a workaround by setting `-Dspring.locking.strict=true` in `ACTIVEMQ_OPTS`. This restores the behavior previous from Spring 6.2.x. A real "enforcement" fix should be done, either: 1. Fix in the webconsole (`ApplicationContextFilter`) to detect slave and redirect to a "slave waiting" page. 2. Add `depends-on="brokerBean"` to the Jetty `invokeStart` bean in `jetty.xml`, to make Jetty waits for the broker. I'm investigating the best approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://activemq.apache.org/contact
