Here is some detailed info we were able to extract. The endpoint
172.22.4.102 is a haproxy server
[1]
wsSession.connection =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
aws =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
endPoint = SelectChannelEndPoint@41ce2b7a
{/172.22.4.102:60455
<->8080,Open,in,out,R,-,1656/30000,WebSocketServerConnection}{io=1,kio=1,kro=1}
endPoint.isOpen() = true
endPoint.isOutputShutdown() = false
endPoint.isInputShutdown() = false
wsSession.isOpen() = true
wsSession.isFailed() = false
wsSession.isRunning() = true
wsSession.isStarted() = true
wsSession.isStarting() = false
wsSession.isStopped() = false
wsSession.isStopping() = false
wsSession.getConnection().getIOState() = IOState@74964a1f[OPEN,in,out]]
WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],behavior=SERVER,connection=WebSocketServerConnection@599eae20
{FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002
[ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62
[behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077
[batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
[1]
wsSession.connection =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
aws =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
endPoint = SelectChannelEndPoint@41ce2b7a
{/172.22.4.102:60455
<->8080,Open,in,out,R,-,1662/30000,WebSocketServerConnection}{io=1,kio=1,kro=1}
endPoint.isOpen() = true
endPoint.isOutputShutdown() = false
endPoint.isInputShutdown() = false
wsSession.isOpen() = true
wsSession.isFailed() = false
wsSession.isRunning() = true
wsSession.isStarted() = true
wsSession.isStarting() = false
wsSession.isStopped() = false
wsSession.isStopping() = false
wsSession.getConnection().getIOState() = IOState@74964a1f[OPEN,in,out]]
WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],behavior=SERVER,connection=WebSocketServerConnection@599eae20
{FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002
[ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62
[behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077
[batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
[1]
wsSession.connection =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
aws =
org.eclipse.jetty.websocket.server.WebSocketServerConnection
endPoint = SelectChannelEndPoint@41ce2b7a
{/172.22.4.102:60455
<->8080,Open,in,out,R,-,1663/30000,WebSocketServerConnection}{io=1,kio=1,kro=1}
endPoint.isOpen() = true
endPoint.isOutputShutdown() = false
endPoint.isInputShutdown() = false
wsSession.isOpen() = true
wsSession.isFailed() = false
wsSession.isRunning() = true
wsSession.isStarted() = true
wsSession.isStarting() = false
wsSession.isStopped() = false
wsSession.isStopping() = false
wsSession.getConnection().getIOState() = IOState@74964a1f[OPEN,in,out]]
WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],behavior=SERVER,connection=WebSocketServerConnection@599eae20
{FILL_INTERESTED}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@5e6de002
[ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@18ed8f62
[behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@597fa077
[batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@276cd7e7
],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
On Tue, Aug 11, 2015 at 6:37 PM, Joakim Erdfelt <[email protected]> wrote:
> I'm trying to understand how IOState thinks its OPEN, but the
> wsSession.isOpen() doesn't comply.
> I'm needing to know some state from the wsSession.connection.endpoint to
> get a better idea on the state your scenario is in.
>
> Joakim Erdfelt / [email protected]
>
> On Tue, Aug 11, 2015 at 3:34 PM, Alex Jeannopoulos <[email protected]>
> wrote:
>
>> closed is - !wsSession.isOpen()
>>
>> For now, I was able to install code into production dynamically to call
>>
>> if ( !wsSession.isOpen() ){
>> factory.onSessionClosed(wsSession);
>> closed++;
>> }
>>
>> I am letting it run in production under load, to see how memory usage is,
>> so far in about 10 mins the memory freed up and is running at the levels of
>> a traditional embedded jetty server doing http polling only no websockets.
>>
>> On Tue, Aug 11, 2015 at 6:16 PM, Joakim Erdfelt <[email protected]>
>> wrote:
>>
>>> I take it "open" is WebSocketSession.isOpen()
>>> What is "closed" in your labels?
>>>
>>>
>>> Joakim Erdfelt / [email protected]
>>>
>>> On Tue, Aug 11, 2015 at 2:33 PM, Mack Gerhardt <[email protected]>
>>> wrote:
>>>
>>>> I will take care of that now, some other details from my dump, I
>>>> iterated teh set of openSessions (35359), and I tallied up the following
>>>> from methods on WebSocketSession
>>>>
>>>> cnt = 35359
>>>> open = 374
>>>> closed = 34985
>>>> failed = 0
>>>> running = 35359
>>>> started = 35359
>>>> starting = 0
>>>> stopped = 0
>>>> stopping = 0
>>>>
>>>> On Tue, Aug 11, 2015 at 5:15 PM, Joakim Erdfelt <[email protected]>
>>>> wrote:
>>>>
>>>>> man, i've got to clean that output up. too much noise/duplicates.
>>>>>
>>>>> The raw connection details are missing *groan*, and the ioState is
>>>>> also not present *double-groan*.
>>>>> Would you mind running a snapshot with some updates to the
>>>>> WebSocketSession.toString() to help use figure this out.
>>>>> I'm really curious about the raw connection state (will let me know
>>>>> what java/jetty thinks the physical connection is doing), as well as the
>>>>> higher level websocket ioState (let me know what the websocket level
>>>>> thinks
>>>>> is reality).
>>>>>
>>>>> Joakim Erdfelt / [email protected]
>>>>>
>>>>> On Tue, Aug 11, 2015 at 1:54 PM, Mack Gerhardt <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> I know this looks messy, its the first 10 sessions that I dumped.
>>>>>>
>>>>>>
>>>>>> [0]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@616c1084
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@616c1084
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@1e934e02
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@775db967
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@28cce9de
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@3f4c0042
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@6b96034a
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@6b96034a
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@74ce2e72
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1f38eda1
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@25d0d803
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@25b3fc31
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [2]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@7bbf0fac
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@7bbf0fac
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@23b2b09f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@1642a
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@465d6d89
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@7e0703a0
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [3]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@739caef5
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@739caef5
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7837a498
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@552bbf32
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@7650bdc6
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@451cd881
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [4]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@257b2516
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@257b2516
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2da1dca4
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5e6cea5b
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@543c8ddf
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@60ad4eff
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [5]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@bf4ec91
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@bf4ec91
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@32f3e186
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@291c072
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@303e5472
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@289351db
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [6]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@31509b57
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@31509b57
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@3d4e4f5f
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7f911140
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@762e1e84
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@1c02b680
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [7]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@cea32af
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@cea32af
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@7b063963
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@7ddac2b9
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@334c9426
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@4391b935
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [8]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@24694625
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@24694625
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@f8d971b
>>>>>> [ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@5837cf68
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@437f7033
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@6e6ea7aa
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>> [9]
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>> - STARTED
>>>>>> +- incomingHandler :
>>>>>> JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ]
>>>>>> +- outgoingHandler :
>>>>>> ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]
>>>>>> - STARTED
>>>>>> +- Stack
>>>>>> +- Network : WebSocketServerConnection@4e1f21fa
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]}
>>>>>> +- Websocket:
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> WebSocketSession[websocket=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ],behavior=SERVER,connection=WebSocketServerConnection@4e1f21fa
>>>>>> {IDLE}{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@2835fbde
>>>>>> [ExtensionStack,s=START,c=0,len=5,f=null,p=WebSocketPolicy@10f3f803
>>>>>> [behavior=SERVER,maxTextMessageSize=65535,maxTextMessageBufferSize=32768,maxBinaryMessageSize=65536,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=10000,idleTimeout=30000,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@77ad35ec
>>>>>> [batching=true],incoming=JettyAnnotatedEventDriver[com.backpocket.servlet.jetty9.ChatWebSocket@2dc95e17
>>>>>> ],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]]
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 11, 2015 at 3:57 PM, Joakim Erdfelt <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> If you can access the session in that openSession list, can you
>>>>>>> iterate through them for state information?
>>>>>>>
>>>>>>> A simple toString() on the individual objects in that list should
>>>>>>> produce reams of useful information.
>>>>>>>
>>>>>>> I'm curious about the various connection and ioState values atm.
>>>>>>>
>>>>>>>
>>>>>>> Joakim Erdfelt / [email protected]
>>>>>>>
>>>>>>> On Tue, Aug 11, 2015 at 12:42 PM, Mack Gerhardt <[email protected]
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> I am seeing this in 9.2.11 but I also saw this in 9.3.0. I do see
>>>>>>>> sessions closing and removed but I am seeing a leak where my embedded
>>>>>>>> jetty
>>>>>>>> engine will die on a OOM after about 12 hours of heave use in
>>>>>>>> production.
>>>>>>>> If there is anything I could do or provide please let me know. I was
>>>>>>>> even
>>>>>>>> thinking of recompiling the websocket jar exposing the openSessions so
>>>>>>>> I
>>>>>>>> could attempt to introspect the sessions.
>>>>>>>>
>>>>>>>> Alex
>>>>>>>>
>>>>>>>> On Tue, Aug 11, 2015 at 3:26 PM, Joakim Erdfelt <[email protected]
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Sounds like a bug.
>>>>>>>>> What version of Jetty are you using?
>>>>>>>>>
>>>>>>>>> Joakim Erdfelt / [email protected]
>>>>>>>>>
>>>>>>>>> On Tue, Aug 11, 2015 at 12:02 PM, Mack Gerhardt <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> I am experiencing an issue with a leak of WebSocketSession
>>>>>>>>>> objects. I believe I might not be closing down sessions properly. I
>>>>>>>>>> was
>>>>>>>>>> using the plumbr.eu tool and it detected the following. Any
>>>>>>>>>> ideas of what should I look for.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> At the time the leak was detected, 30724 objects existed of class
>>>>>>>>>>
>>>>>>>>>> org.eclipse.jetty.websocket.common.WebSocketSession
>>>>>>>>>>
>>>>>>>>>> These objects had been created at
>>>>>>>>>>
>>>>>>>>>> org.eclipse.jetty.websocket.common.WebSocketSessionFactory.createSession(java.net.URI,
>>>>>>>>>> org.eclipse.jetty.websocket.common.events.EventDriver,
>>>>>>>>>> org.eclipse.jetty.websocket.common.LogicalConnection):48
>>>>>>>>>>
>>>>>>>>>> and were being held
>>>>>>>>>>
>>>>>>>>>> in array of java.util.concurrent.CopyOnWriteArrayList
>>>>>>>>>>
>>>>>>>>>> in al of java.util.concurrent.CopyOnWriteArraySet
>>>>>>>>>>
>>>>>>>>>> in openSessions of
>>>>>>>>>> org.eclipse.jetty.websocket.server.WebSocketServerFactory
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> jetty-users mailing list
>>>>>>>>>> [email protected]
>>>>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>>>>> unsubscribe from this list, visit
>>>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> jetty-users mailing list
>>>>>>>>> [email protected]
>>>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>>>> unsubscribe from this list, visit
>>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> jetty-users mailing list
>>>>>>>> [email protected]
>>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>>> unsubscribe from this list, visit
>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> jetty-users mailing list
>>>>>>> [email protected]
>>>>>>> To change your delivery options, retrieve your password, or
>>>>>>> unsubscribe from this list, visit
>>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jetty-users mailing list
>>>>>> [email protected]
>>>>>> To change your delivery options, retrieve your password, or
>>>>>> unsubscribe from this list, visit
>>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jetty-users mailing list
>>>>> [email protected]
>>>>> To change your delivery options, retrieve your password, or
>>>>> unsubscribe from this list, visit
>>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jetty-users mailing list
>>>> [email protected]
>>>> To change your delivery options, retrieve your password, or unsubscribe
>>>> from this list, visit
>>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>>
>>>
>>>
>>> _______________________________________________
>>> jetty-users mailing list
>>> [email protected]
>>> To change your delivery options, retrieve your password, or unsubscribe
>>> from this list, visit
>>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>>
>>
>>
>> _______________________________________________
>> jetty-users mailing list
>> [email protected]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>
>
> _______________________________________________
> jetty-users mailing list
> [email protected]
> To change your delivery options, retrieve your password, or unsubscribe
> from this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
_______________________________________________
jetty-users mailing list
[email protected]
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
