Where you able to find anything useful in the data dump? I can provide more info if needed.
On Tue, Aug 11, 2015 at 10:33 PM, Mack Gerhardt <[email protected]> wrote: > Here is some detailed info we were able to extract. The endpoint > 172.22.4.102 is a haproxy server > > [1] > wsSession.connection = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > aws = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > endPoint = > SelectChannelEndPoint@41ce2b7a{/172.22.4.102:60455 > <->8080,Open,in,out,R,-,1656/30000,WebSocketServerConnection}{io=1,kio=1,kro=1} > endPoint.isOpen() = true > endPoint.isOutputShutdown() = false > endPoint.isInputShutdown() = false > wsSession.isOpen() = true > wsSession.isFailed() = false > wsSession.isRunning() = true > wsSession.isStarted() = true > wsSession.isStarting() = false > wsSession.isStopped() = false > wsSession.isStopping() = false > wsSession.getConnection().getIOState() = IOState@74964a1f > [OPEN,in,out]] > > WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],behavior=SERVER,connection=WebSocketServerConnection@599eae20 > {FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002 > [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62 > [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077 > [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] > [1] > wsSession.connection = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > aws = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > endPoint = > SelectChannelEndPoint@41ce2b7a{/172.22.4.102:60455 > <->8080,Open,in,out,R,-,1662/30000,WebSocketServerConnection}{io=1,kio=1,kro=1} > endPoint.isOpen() = true > endPoint.isOutputShutdown() = false > endPoint.isInputShutdown() = false > wsSession.isOpen() = true > wsSession.isFailed() = false > wsSession.isRunning() = true > wsSession.isStarted() = true > wsSession.isStarting() = false > wsSession.isStopped() = false > wsSession.isStopping() = false > wsSession.getConnection().getIOState() = IOState@74964a1f > [OPEN,in,out]] > > WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],behavior=SERVER,connection=WebSocketServerConnection@599eae20 > {FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002 > [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62 > [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077 > [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] > [1] > wsSession.connection = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > aws = > org.eclipse.jetty.websocket.server.WebSocketServerConnection > endPoint = > SelectChannelEndPoint@41ce2b7a{/172.22.4.102:60455 > <->8080,Open,in,out,R,-,1663/30000,WebSocketServerConnection}{io=1,kio=1,kro=1} > endPoint.isOpen() = true > endPoint.isOutputShutdown() = false > endPoint.isInputShutdown() = false > wsSession.isOpen() = true > wsSession.isFailed() = false > wsSession.isRunning() = true > wsSession.isStarted() = true > wsSession.isStarting() = false > wsSession.isStopped() = false > wsSession.isStopping() = false > wsSession.getConnection().getIOState() = IOState@74964a1f > [OPEN,in,out]] > > WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],behavior=SERVER,connection=WebSocketServerConnection@599eae20 > {FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002 > [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62 > [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077 > [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7 > ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] > > > On Tue, Aug 11, 2015 at 6:37 PM, Joakim Erdfelt <[email protected]> > wrote: > >> I'm trying to understand how IOState thinks its OPEN, but the >> wsSession.isOpen() doesn't comply. >> I'm needing to know some state from the wsSession.connection.endpoint to >> get a better idea on the state your scenario is in. >> >> Joakim Erdfelt / [email protected] >> >> On Tue, Aug 11, 2015 at 3:34 PM, Alex Jeannopoulos <[email protected]> >> wrote: >> >>> closed is - !wsSession.isOpen() >>> >>> For now, I was able to install code into production dynamically to call >>> >>> if ( !wsSession.isOpen() ){ >>> factory.onSessionClosed(wsSession); >>> closed++; >>> } >>> >>> I am letting it run in production under load, to see how memory usage >>> is, so far in about 10 mins the memory freed up and is running at the >>> levels of a traditional embedded jetty server doing http polling only no >>> websockets. >>> >>> On Tue, Aug 11, 2015 at 6:16 PM, Joakim Erdfelt <[email protected]> >>> wrote: >>> >>>> I take it "open" is WebSocketSession.isOpen() >>>> What is "closed" in your labels? >>>> >>>> >>>> Joakim Erdfelt / [email protected] >>>> >>>> On Tue, Aug 11, 2015 at 2:33 PM, Mack Gerhardt <[email protected]> >>>> wrote: >>>> >>>>> I will take care of that now, some other details from my dump, I >>>>> iterated teh set of openSessions (35359), and I tallied up the following >>>>> from methods on WebSocketSession >>>>> >>>>> cnt = 35359 >>>>> open = 374 >>>>> closed = 34985 >>>>> failed = 0 >>>>> running = 35359 >>>>> started = 35359 >>>>> starting = 0 >>>>> stopped = 0 >>>>> stopping = 0 >>>>> >>>>> On Tue, Aug 11, 2015 at 5:15 PM, Joakim Erdfelt <[email protected]> >>>>> wrote: >>>>> >>>>>> man, i've got to clean that output up. too much noise/duplicates. >>>>>> >>>>>> The raw connection details are missing *groan*, and the ioState is >>>>>> also not present *double-groan*. >>>>>> Would you mind running a snapshot with some updates to the >>>>>> WebSocketSession.toString() to help use figure this out. >>>>>> I'm really curious about the raw connection state (will let me know >>>>>> what java/jetty thinks the physical connection is doing), as well as the >>>>>> higher level websocket ioState (let me know what the websocket level >>>>>> thinks >>>>>> is reality). >>>>>> >>>>>> Joakim Erdfelt / [email protected] >>>>>> >>>>>> On Tue, Aug 11, 2015 at 1:54 PM, Mack Gerhardt <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I know this looks messy, its the first 10 sessions that I dumped. >>>>>>> >>>>>>> >>>>>>> [0] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@616c1084 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@6b96034a >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [2] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@7bbf0fac >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [3] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@739caef5 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498 >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [4] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@257b2516 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [5] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@bf4ec91 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [6] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@31509b57 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [7] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@cea32af >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963 >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [8] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@24694625 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625 >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b >>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033 >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> [9] >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> - STARTED >>>>>>> +- incomingHandler : >>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ] >>>>>>> +- outgoingHandler : >>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection] >>>>>>> - STARTED >>>>>>> +- Stack >>>>>>> +- Network : WebSocketServerConnection@4e1f21fa >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]} >>>>>>> +- Websocket: >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa >>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde >>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803 >>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec >>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17 >>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]] >>>>>>> >>>>>>> >>>>>>> On Tue, Aug 11, 2015 at 3:57 PM, Joakim Erdfelt <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> If you can access the session in that openSession list, can you >>>>>>>> iterate through them for state information? >>>>>>>> >>>>>>>> A simple toString() on the individual objects in that list should >>>>>>>> produce reams of useful information. >>>>>>>> >>>>>>>> I'm curious about the various connection and ioState values atm. >>>>>>>> >>>>>>>> >>>>>>>> Joakim Erdfelt / [email protected] >>>>>>>> >>>>>>>> On Tue, Aug 11, 2015 at 12:42 PM, Mack Gerhardt < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> I am seeing this in 9.2.11 but I also saw this in 9.3.0. I do see >>>>>>>>> sessions closing and removed but I am seeing a leak where my embedded >>>>>>>>> jetty >>>>>>>>> engine will die on a OOM after about 12 hours of heave use in >>>>>>>>> production. >>>>>>>>> If there is anything I could do or provide please let me know. I was >>>>>>>>> even >>>>>>>>> thinking of recompiling the websocket jar exposing the openSessions >>>>>>>>> so I >>>>>>>>> could attempt to introspect the sessions. >>>>>>>>> >>>>>>>>> Alex >>>>>>>>> >>>>>>>>> On Tue, Aug 11, 2015 at 3:26 PM, Joakim Erdfelt < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Sounds like a bug. >>>>>>>>>> What version of Jetty are you using? >>>>>>>>>> >>>>>>>>>> Joakim Erdfelt / [email protected] >>>>>>>>>> >>>>>>>>>> On Tue, Aug 11, 2015 at 12:02 PM, Mack Gerhardt < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> I am experiencing an issue with a leak of WebSocketSession >>>>>>>>>>> objects. I believe I might not be closing down sessions properly. I >>>>>>>>>>> was >>>>>>>>>>> using the plumbr.eu tool and it detected the following. Any >>>>>>>>>>> ideas of what should I look for. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> At the time the leak was detected, 30724 objects existed of class >>>>>>>>>>> >>>>>>>>>>> org.eclipse.jetty.websocket.common.WebSocketSession >>>>>>>>>>> >>>>>>>>>>> These objects had been created at >>>>>>>>>>> >>>>>>>>>>> org.eclipse.jetty.websocket.common.WebSocketSessionFactory.createSession(java.net.URI, >>>>>>>>>>> org.eclipse.jetty.websocket.common.events.EventDriver, >>>>>>>>>>> org.eclipse.jetty.websocket.common.LogicalConnection):48 >>>>>>>>>>> >>>>>>>>>>> and were being held >>>>>>>>>>> >>>>>>>>>>> in array of java.util.concurrent.CopyOnWriteArrayList >>>>>>>>>>> >>>>>>>>>>> in al of java.util.concurrent.CopyOnWriteArraySet >>>>>>>>>>> >>>>>>>>>>> in openSessions of >>>>>>>>>>> org.eclipse.jetty.websocket.server.WebSocketServerFactory >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> jetty-users mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> To change your delivery options, retrieve your password, or >>>>>>>>>>> unsubscribe from this list, visit >>>>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> jetty-users mailing list >>>>>>>>>> [email protected] >>>>>>>>>> To change your delivery options, retrieve your password, or >>>>>>>>>> unsubscribe from this list, visit >>>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> jetty-users mailing list >>>>>>>>> [email protected] >>>>>>>>> To change your delivery options, retrieve your password, or >>>>>>>>> unsubscribe from this list, visit >>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> jetty-users mailing list >>>>>>>> [email protected] >>>>>>>> To change your delivery options, retrieve your password, or >>>>>>>> unsubscribe from this list, visit >>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> jetty-users mailing list >>>>>>> [email protected] >>>>>>> To change your delivery options, retrieve your password, or >>>>>>> unsubscribe from this list, visit >>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> jetty-users mailing list >>>>>> [email protected] >>>>>> To change your delivery options, retrieve your password, or >>>>>> unsubscribe from this list, visit >>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> jetty-users mailing list >>>>> [email protected] >>>>> To change your delivery options, retrieve your password, or >>>>> unsubscribe from this list, visit >>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> jetty-users mailing list >>>> [email protected] >>>> To change your delivery options, retrieve your password, or unsubscribe >>>> from this list, visit >>>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>>> >>> >>> >>> _______________________________________________ >>> jetty-users mailing list >>> [email protected] >>> To change your delivery options, retrieve your password, or unsubscribe >>> from this list, visit >>> https://dev.eclipse.org/mailman/listinfo/jetty-users >>> >> >> >> _______________________________________________ >> jetty-users mailing list >> [email protected] >> To change your delivery options, retrieve your password, or unsubscribe >> from this list, visit >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> > >
_______________________________________________ jetty-users mailing list [email protected] To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
