[
https://issues.apache.org/jira/browse/QPID-8727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomas Vavricka updated QPID-8727:
---------------------------------
Description:
After upgrading to Jetty 12, sometimes an uncaught {{IllegalStateException:
channel already completed}} is observed:
2025-12-02T06:51:16,271Z ERROR [jetty-immediate-executor] (o.a.q.s.Main) -
Uncaught exception, shutting down.
java.lang.IllegalStateException: channel already completed
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1264)
at org.eclipse.jetty.server.Response.writeError(Response.java:647)
at org.eclipse.jetty.server.Response.writeError(Response.java:550)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at
org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
at java.base/java.lang.Thread.run(Thread.java:840)
Suppressed: java.util.concurrent.TimeoutException: Idle timeout
expired: 30074/30000 ms
at
org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:167)
at
org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 common frames omitted
Suppressed: java.lang.IllegalStateException: channel already
completed
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getHttpChannelState(HttpChannelState.java:864)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getComponents(HttpChannelState.java:851)
at
org.eclipse.jetty.server.Request$Wrapper.getComponents(Request.java:853)
at
org.eclipse.jetty.server.handler.ErrorHandler.generateAcceptableResponse(ErrorHandler.java:203)
at
org.eclipse.jetty.server.handler.ErrorHandler.generateResponse(ErrorHandler.java:163)
at
org.eclipse.jetty.server.handler.ErrorHandler.handle(ErrorHandler.java:118)
at
org.eclipse.jetty.server.Response.writeError(Response.java:635)
at
org.eclipse.jetty.server.Response.writeError(Response.java:550)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at
org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
... 1 common frames omitted
This exception bubbles out of the Jetty executor thread
({{{}jetty-immediate-executor{}}}) and becomes an uncaught exception in the
broker, leading to broker shutdown.
To fix the issue with the exception being thrown from Jetty web server threads,
creation of the Jetty server instance in management-http plugin is changed to
pass an UncaughtExceptionHandler, which will log exceptions without letting
them "bubble" to the broker level (with the except of java.lang.Error or qpid
ServerScopedException).
was:
After upgrading to Jetty 12, sometimes an uncaught {{IllegalStateException:
channel already completed}} is observed:
2025-12-02T06:51:16,271Z ERROR [jetty-immediate-executor] (o.a.q.s.Main) -
Uncaught exception, shutting down.
java.lang.IllegalStateException: channel already completed
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1264)
at org.eclipse.jetty.server.Response.writeError(Response.java:647)
at org.eclipse.jetty.server.Response.writeError(Response.java:550)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at
org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
at java.base/java.lang.Thread.run(Thread.java:840)
Suppressed: java.util.concurrent.TimeoutException: Idle timeout
expired: 30074/30000 ms
at
org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:167)
at
org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 common frames omitted
Suppressed: java.lang.IllegalStateException: channel already
completed
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getHttpChannelState(HttpChannelState.java:864)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getComponents(HttpChannelState.java:851)
at
org.eclipse.jetty.server.Request$Wrapper.getComponents(Request.java:853)
at
org.eclipse.jetty.server.handler.ErrorHandler.generateAcceptableResponse(ErrorHandler.java:203)
at
org.eclipse.jetty.server.handler.ErrorHandler.generateResponse(ErrorHandler.java:163)
at
org.eclipse.jetty.server.handler.ErrorHandler.handle(ErrorHandler.java:118)
at
org.eclipse.jetty.server.Response.writeError(Response.java:635)
at
org.eclipse.jetty.server.Response.writeError(Response.java:550)
at
org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
at
org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
... 1 common frames omitted
This exception bubbles out of the Jetty executor thread
({{{}jetty-immediate-executor{}}}) and becomes an uncaught exception in the
broker, leading to broker shutdown.
> [Broker-J] Jetty IllegalStateException leads to broker shutdown
> ---------------------------------------------------------------
>
> Key: QPID-8727
> URL: https://issues.apache.org/jira/browse/QPID-8727
> Project: Qpid
> Issue Type: Bug
> Components: Broker-J
> Affects Versions: qpid-java-broker-10.0.0
> Reporter: Daniil Kirilyuk
> Priority: Major
> Fix For: qpid-java-broker-10.0.1
>
>
> After upgrading to Jetty 12, sometimes an uncaught {{IllegalStateException:
> channel already completed}} is observed:
>
> 2025-12-02T06:51:16,271Z ERROR [jetty-immediate-executor] (o.a.q.s.Main) -
> Uncaught exception, shutting down.
> java.lang.IllegalStateException: channel already completed
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1264)
> at org.eclipse.jetty.server.Response.writeError(Response.java:647)
> at org.eclipse.jetty.server.Response.writeError(Response.java:550)
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
> at
> org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
> at java.base/java.lang.Thread.run(Thread.java:840)
> Suppressed: java.util.concurrent.TimeoutException: Idle timeout
> expired: 30074/30000 ms
> at
> org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:167)
> at
> org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> ... 1 common frames omitted
> Suppressed: java.lang.IllegalStateException: channel already
> completed
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.lockedGetHttpChannelState(HttpChannelState.java:872)
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getHttpChannelState(HttpChannelState.java:864)
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelRequest.getComponents(HttpChannelState.java:851)
> at
> org.eclipse.jetty.server.Request$Wrapper.getComponents(Request.java:853)
> at
> org.eclipse.jetty.server.handler.ErrorHandler.generateAcceptableResponse(ErrorHandler.java:203)
> at
> org.eclipse.jetty.server.handler.ErrorHandler.generateResponse(ErrorHandler.java:163)
> at
> org.eclipse.jetty.server.handler.ErrorHandler.handle(ErrorHandler.java:118)
> at
> org.eclipse.jetty.server.Response.writeError(Response.java:635)
> at
> org.eclipse.jetty.server.Response.writeError(Response.java:550)
> at
> org.eclipse.jetty.server.internal.HttpChannelState$ChannelCallback.failed(HttpChannelState.java:1639)
> at
> org.eclipse.jetty.server.internal.HttpChannelState.lambda$onFailure$1(HttpChannelState.java:458)
> ... 1 common frames omitted
> This exception bubbles out of the Jetty executor thread
> ({{{}jetty-immediate-executor{}}}) and becomes an uncaught exception in the
> broker, leading to broker shutdown.
> To fix the issue with the exception being thrown from Jetty web server
> threads, creation of the Jetty server instance in management-http plugin is
> changed to pass an UncaughtExceptionHandler, which will log exceptions
> without letting them "bubble" to the broker level (with the except of
> java.lang.Error or qpid ServerScopedException).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
