Re: Upgrade fom Tomat 8.0.33 to 8.5.9 - stuck clients

[Enrico Olivelli]

After rolling back to old Nio all clients works very well as expected.


I'm sorry, I don't know how to reproduce this kind of issue on a test case,
I have tried to write some simple multi-thread tests but I cannot break
Tomcat Nio2 connector, it happens only on my QA environment.

At this point I'm not able to submit any issue to your bug tracker.

Thank you


2017-01-13 0:43 GMT+01:00 Mark Thomas :

> On 12/01/2017 21:09, Enrico Olivelli wrote:
> > Thank you Chris
> >
> > Il gio 12 gen 2017, 18:04 Christopher Schultz <
> ch...@christopherschultz.net>
> > ha scritto:
> >
> > Enrico,
> >
> > On 1/12/17 8:53 AM, Enrico Olivelli wrote:
>  I'm upgrading from Tomcat 8.0.33. I see that after a period of work
>  requests remains "pending", for instance I get all clients
>  remaining waiting for a response (parsing HTTP Response header) and
>  no active thread on my Tomcat.
> 
>  This happens in my QA environment where I start several
>  WebDriver/Unit tests againts my web application. I have many cases,
>  the most simple I this is the following:
> 
>  on  the client side (a JAX-WS client):
> 
>  "main" #1 prio=5 os_prio=0 tid=0x7f7cf0009000 nid=0x7ddc
>  runnable [0x7f7cf6e3f000]
> 
>  java.lang.Thread.State: RUNNABLE
> 
>  at java.net.SocketInputStream.socketRead0(Native Method)
> 
>  at
>  java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> 
>  at java.net.SocketInputStream.read(SocketInputStream.java:170)
> 
>  at java.net.SocketInputStream.read(SocketInputStream.java:141)
> 
>  at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> 
>  at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> 
>  at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> 
>  - locked <0xfd159a18> (a java.io.BufferedInputStream)
> 
>  at
>  sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
> >
> > Okay, client is waiting.
> >
> >
> >> From server side application logs I see that the webapp has processed
> the
> >> request. In this case I am using Metro JAX-WS.
> >> It seems that the reponse is waiting in some buffer.
> >> Is there any way to inspect tomcat buffers? Network buffers? I am using
> >> linux.
> >
> >> The strange thing is that with Tomcat 8.0.33 there is no problem.
> >>  The same problem happens even with other servlets of different
> frameworks
> >> and other webapps.
> >
> >> On 8.0.x I was using Nio connector, on 8.5 I am using Nio2.
> >> The problem is both on http and jsse https.
>
> Try NIO.
>
> Mark
>
>
> >
> >> Is any new relevant default value changed from 8.0 and 8.5? I can't find
> >> any idea on changelogs or documentation
> >
> >
> >
> >
> >
>  in the server side: Full thread dump Java HotSpot(TM) 64-Bit Server
>  VM (25.92-b14 mixed mode):
> 
> 
>  "anInnocuousThread" #861 daemon prio=5 os_prio=0
>  tid=0x7ff23c0de800 nid=0xdb26 runnable [0x7ff1ad379000]
>  java.lang.Thread.State: RUNNABLE at
>  sun.nio.ch.EPoll.epollWait(Native Method) at
>  sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194) at
>  sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268) at
>  java.lang.Thread.run(Thread.java:745) at
>  sun.misc.InnocuousThread.run(InnocuousThread.java:74)
> 
>  (lots of this kind) (HTTPS Connector I think)
> >
> > Without the rest of the stack trace, it's hard to tell if those are okay
> >
> >
> >> I have cut all the other threads which had the same stacktrace.
> >> I have stripped out GC threads, JMX...
> >> My application is not present in any thread but I have several clients
> >> stuck at waiting for a response.
> >
> >> As soon as I can I will attach the full dump
> >
> >> Thank you very much
> >> I appreciate your help
> >
> >> Enrico
> >
> >> .
> >
>  "https-jsse-nio2-10.168.10.55-8443-exec-130" #572 daemon prio=5
>  os_prio=0 tid=0x7ff23c0a nid=0xd4a2 waiting on condition
>  [0x7ff1b2bbf000] java.lang.Thread.State: WAITING (parking) at
>  sun.misc.Unsafe.park(Native Method) - parking to wait for
>  <0x80f1d420> (a
>  java.util.concurrent.locks.AbstractQueuedSynchronizer$
> ConditionObject)
> >
> > These
> 
> > are request-processing threads, waiting for work. They are idle.
> >
>  (lots of this kind) (HTTP Connector I think)
> 
>  "http-nio2-10.168.10.55-8080-exec-21" #281 daemon prio=5 os_prio=0
>  tid=0x7ff2340e7000 nid=0xd262 waiting on condition
>  [0x7ff1af995000] java.lang.Thread.State: WAITING (parking) at
>  sun.misc.Unsafe.park(Native Method) - parking to wait for
>  <0x80f0cdb0> (a
>  java.util.concurrent.locks.AbstractQueuedSynchronizer$
> ConditionObject)
> >
> > Same
> 
> > here.
> >
> > The others look okay to me.
> >
> > -chris
> >>
> >> 

Re: Upgrade fom Tomat 8.0.33 to 8.5.9 - stuck clients

[Mark Thomas]

On 12/01/2017 21:09, Enrico Olivelli wrote:
> Thank you Chris
> 
> Il gio 12 gen 2017, 18:04 Christopher Schultz 
> ha scritto:
> 
> Enrico,
> 
> On 1/12/17 8:53 AM, Enrico Olivelli wrote:
 I'm upgrading from Tomcat 8.0.33. I see that after a period of work
 requests remains "pending", for instance I get all clients
 remaining waiting for a response (parsing HTTP Response header) and
 no active thread on my Tomcat.

 This happens in my QA environment where I start several
 WebDriver/Unit tests againts my web application. I have many cases,
 the most simple I this is the following:

 on  the client side (a JAX-WS client):

 "main" #1 prio=5 os_prio=0 tid=0x7f7cf0009000 nid=0x7ddc
 runnable [0x7f7cf6e3f000]

 java.lang.Thread.State: RUNNABLE

 at java.net.SocketInputStream.socketRead0(Native Method)

 at
 java.net.SocketInputStream.socketRead(SocketInputStream.java:116)

 at java.net.SocketInputStream.read(SocketInputStream.java:170)

 at java.net.SocketInputStream.read(SocketInputStream.java:141)

 at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)

 at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)

 at java.io.BufferedInputStream.read(BufferedInputStream.java:345)

 - locked <0xfd159a18> (a java.io.BufferedInputStream)

 at
 sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
> 
> Okay, client is waiting.
> 
> 
>> From server side application logs I see that the webapp has processed the
>> request. In this case I am using Metro JAX-WS.
>> It seems that the reponse is waiting in some buffer.
>> Is there any way to inspect tomcat buffers? Network buffers? I am using
>> linux.
> 
>> The strange thing is that with Tomcat 8.0.33 there is no problem.
>>  The same problem happens even with other servlets of different frameworks
>> and other webapps.
> 
>> On 8.0.x I was using Nio connector, on 8.5 I am using Nio2.
>> The problem is both on http and jsse https.

Try NIO.

Mark


> 
>> Is any new relevant default value changed from 8.0 and 8.5? I can't find
>> any idea on changelogs or documentation
> 
> 
> 
> 
> 
 in the server side: Full thread dump Java HotSpot(TM) 64-Bit Server
 VM (25.92-b14 mixed mode):


 "anInnocuousThread" #861 daemon prio=5 os_prio=0
 tid=0x7ff23c0de800 nid=0xdb26 runnable [0x7ff1ad379000]
 java.lang.Thread.State: RUNNABLE at
 sun.nio.ch.EPoll.epollWait(Native Method) at
 sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194) at
 sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268) at
 java.lang.Thread.run(Thread.java:745) at
 sun.misc.InnocuousThread.run(InnocuousThread.java:74)

 (lots of this kind) (HTTPS Connector I think)
> 
> Without the rest of the stack trace, it's hard to tell if those are okay
> 
> 
>> I have cut all the other threads which had the same stacktrace.
>> I have stripped out GC threads, JMX...
>> My application is not present in any thread but I have several clients
>> stuck at waiting for a response.
> 
>> As soon as I can I will attach the full dump
> 
>> Thank you very much
>> I appreciate your help
> 
>> Enrico
> 
>> .
> 
 "https-jsse-nio2-10.168.10.55-8443-exec-130" #572 daemon prio=5
 os_prio=0 tid=0x7ff23c0a nid=0xd4a2 waiting on condition
 [0x7ff1b2bbf000] java.lang.Thread.State: WAITING (parking) at
 sun.misc.Unsafe.park(Native Method) - parking to wait for
 <0x80f1d420> (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 
> These

> are request-processing threads, waiting for work. They are idle.
> 
 (lots of this kind) (HTTP Connector I think)

 "http-nio2-10.168.10.55-8080-exec-21" #281 daemon prio=5 os_prio=0
 tid=0x7ff2340e7000 nid=0xd262 waiting on condition
 [0x7ff1af995000] java.lang.Thread.State: WAITING (parking) at
 sun.misc.Unsafe.park(Native Method) - parking to wait for
 <0x80f0cdb0> (a
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 
> Same

> here.
> 
> The others look okay to me.
> 
> -chris
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>> --
> 
> 
> -- Enrico Olivelli
> 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Upgrade fom Tomat 8.0.33 to 8.5.9 - stuck clients

[Enrico Olivelli]

Thank you Chris

Il gio 12 gen 2017, 18:04 Christopher Schultz 
ha scritto:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Enrico,
>
> On 1/12/17 8:53 AM, Enrico Olivelli wrote:
> > I'm upgrading from Tomcat 8.0.33. I see that after a period of work
> > requests remains "pending", for instance I get all clients
> > remaining waiting for a response (parsing HTTP Response header) and
> > no active thread on my Tomcat.
> >
> > This happens in my QA environment where I start several
> > WebDriver/Unit tests againts my web application. I have many cases,
> > the most simple I this is the following:
> >
> > on  the client side (a JAX-WS client):
> >
> > "main" #1 prio=5 os_prio=0 tid=0x7f7cf0009000 nid=0x7ddc
> > runnable [0x7f7cf6e3f000]
> >
> > java.lang.Thread.State: RUNNABLE
> >
> > at java.net.SocketInputStream.socketRead0(Native Method)
> >
> > at
> > java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> >
> > at java.net.SocketInputStream.read(SocketInputStream.java:170)
> >
> > at java.net.SocketInputStream.read(SocketInputStream.java:141)
> >
> > at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> >
> > at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> >
> > at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> >
> > - locked <0xfd159a18> (a java.io.BufferedInputStream)
> >
> > at
> > sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
>
> Okay, client is waiting.
>

>From server side application logs I see that the webapp has processed the
request. In this case I am using Metro JAX-WS.
It seems that the reponse is waiting in some buffer.
Is there any way to inspect tomcat buffers? Network buffers? I am using
linux.

The strange thing is that with Tomcat 8.0.33 there is no problem.
 The same problem happens even with other servlets of different frameworks
and other webapps.

On 8.0.x I was using Nio connector, on 8.5 I am using Nio2.
The problem is both on http and jsse https.

Is any new relevant default value changed from 8.0 and 8.5? I can't find
any idea on changelogs or documentation





> > in the server side: Full thread dump Java HotSpot(TM) 64-Bit Server
> > VM (25.92-b14 mixed mode):
> >
> >
> > "anInnocuousThread" #861 daemon prio=5 os_prio=0
> > tid=0x7ff23c0de800 nid=0xdb26 runnable [0x7ff1ad379000]
> > java.lang.Thread.State: RUNNABLE at
> > sun.nio.ch.EPoll.epollWait(Native Method) at
> > sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194) at
> > sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268) at
> > java.lang.Thread.run(Thread.java:745) at
> > sun.misc.InnocuousThread.run(InnocuousThread.java:74)
> >
> > (lots of this kind) (HTTPS Connector I think)
>
> Without the rest of the stack trace, it's hard to tell if those are okay
>

I have cut all the other threads which had the same stacktrace.
I have stripped out GC threads, JMX...
My application is not present in any thread but I have several clients
stuck at waiting for a response.

As soon as I can I will attach the full dump

Thank you very much
I appreciate your help

Enrico

.
>
> > "https-jsse-nio2-10.168.10.55-8443-exec-130" #572 daemon prio=5
> > os_prio=0 tid=0x7ff23c0a nid=0xd4a2 waiting on condition
> > [0x7ff1b2bbf000] java.lang.Thread.State: WAITING (parking) at
> > sun.misc.Unsafe.park(Native Method) - parking to wait for
> > <0x80f1d420> (a
> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> These
> >
> are request-processing threads, waiting for work. They are idle.
>
> > (lots of this kind) (HTTP Connector I think)
> >
> > "http-nio2-10.168.10.55-8080-exec-21" #281 daemon prio=5 os_prio=0
> > tid=0x7ff2340e7000 nid=0xd262 waiting on condition
> > [0x7ff1af995000] java.lang.Thread.State: WAITING (parking) at
> > sun.misc.Unsafe.park(Native Method) - parking to wait for
> > <0x80f0cdb0> (a
> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>
> Same
> >
> here.
>
> The others look okay to me.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJYd7cuAAoJEBzwKT+lPKRYjz4QAMdOEYMtJjMP3ZNmwNgUtFiv
> EGjupmPN4xYrcqmLGC3uBQUTSPHpwCnSz7rdunqmMsludgKeNdFUrljcWtMjNqHG
> Kacv+epNlT60L+Uqfeld4XDiADoXuVfxtX3KwGgNnSCsDNP/qgeSfP/61ucpmiqi
> rq5UF36CdGfp351Tw+6UEJyD86VSgRq+0URgDojkCvT9wwSJhmKzsFFG96QCzF4t
> dgbZEJTDPmg1y25dk4bi3GBmBbSMpM3o4s9/0HaZ8OUk/JrwCEFdvPbo8OWSZgmR
> C7JlAu+l4yN3rcj6fa58IyfxcyK8q68FSlck0bMP3ZCbBKNikY0ENj6GDY3s33/N
> ih1hWLyRtVnqrw2w2551Zi8gMj1dp7MOXogM4cp+hy+R13ihWKcwoshteGrX6mOb
> 4B1tLUtgweFeCiFLFC2Lno2M4OtiSSBbI6d5Dr4kIX7h9rCkDBxZHkTvZEGK82z3
> +1S4w8QLceRiyoJplPhb+81S2PrLedG7fbSpcwRAv/lHZzxFFaRgrcyxNw8wn5/o
> LDpV0+MRaf/VgZSiTB99Yn+bisXfKsaGLSbKytsM9Ts2LgvBP0C/W/0Ja86ghDUc
> D+WygQyikdpKN6UX04ZKJ2FUQJ+7/W6gXupHlzvvUvxy71iHSLaArmv/3I2UNgcv

Re: Upgrade fom Tomat 8.0.33 to 8.5.9 - stuck clients

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Enrico,

On 1/12/17 8:53 AM, Enrico Olivelli wrote:
> I'm upgrading from Tomcat 8.0.33. I see that after a period of work
> requests remains "pending", for instance I get all clients
> remaining waiting for a response (parsing HTTP Response header) and
> no active thread on my Tomcat.
> 
> This happens in my QA environment where I start several
> WebDriver/Unit tests againts my web application. I have many cases,
> the most simple I this is the following:
> 
> on  the client side (a JAX-WS client):
> 
> "main" #1 prio=5 os_prio=0 tid=0x7f7cf0009000 nid=0x7ddc
> runnable [0x7f7cf6e3f000]
> 
> java.lang.Thread.State: RUNNABLE
> 
> at java.net.SocketInputStream.socketRead0(Native Method)
> 
> at
> java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> 
> at java.net.SocketInputStream.read(SocketInputStream.java:170)
> 
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> 
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> 
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> 
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> 
> - locked <0xfd159a18> (a java.io.BufferedInputStream)
> 
> at
> sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)

Okay, client is waiting.

> in the server side: Full thread dump Java HotSpot(TM) 64-Bit Server
> VM (25.92-b14 mixed mode):
> 
> 
> "anInnocuousThread" #861 daemon prio=5 os_prio=0
> tid=0x7ff23c0de800 nid=0xdb26 runnable [0x7ff1ad379000] 
> java.lang.Thread.State: RUNNABLE at
> sun.nio.ch.EPoll.epollWait(Native Method) at
> sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194) at
> sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268) at
> java.lang.Thread.run(Thread.java:745) at
> sun.misc.InnocuousThread.run(InnocuousThread.java:74)
> 
> (lots of this kind) (HTTPS Connector I think)

Without the rest of the stack trace, it's hard to tell if those are okay
.

> "https-jsse-nio2-10.168.10.55-8443-exec-130" #572 daemon prio=5
> os_prio=0 tid=0x7ff23c0a nid=0xd4a2 waiting on condition
> [0x7ff1b2bbf000] java.lang.Thread.State: WAITING (parking) at
> sun.misc.Unsafe.park(Native Method) - parking to wait for
> <0x80f1d420> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

These
> 
are request-processing threads, waiting for work. They are idle.

> (lots of this kind) (HTTP Connector I think)
> 
> "http-nio2-10.168.10.55-8080-exec-21" #281 daemon prio=5 os_prio=0 
> tid=0x7ff2340e7000 nid=0xd262 waiting on condition
> [0x7ff1af995000] java.lang.Thread.State: WAITING (parking) at
> sun.misc.Unsafe.park(Native Method) - parking to wait for
> <0x80f0cdb0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)

Same
> 
here.

The others look okay to me.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYd7cuAAoJEBzwKT+lPKRYjz4QAMdOEYMtJjMP3ZNmwNgUtFiv
EGjupmPN4xYrcqmLGC3uBQUTSPHpwCnSz7rdunqmMsludgKeNdFUrljcWtMjNqHG
Kacv+epNlT60L+Uqfeld4XDiADoXuVfxtX3KwGgNnSCsDNP/qgeSfP/61ucpmiqi
rq5UF36CdGfp351Tw+6UEJyD86VSgRq+0URgDojkCvT9wwSJhmKzsFFG96QCzF4t
dgbZEJTDPmg1y25dk4bi3GBmBbSMpM3o4s9/0HaZ8OUk/JrwCEFdvPbo8OWSZgmR
C7JlAu+l4yN3rcj6fa58IyfxcyK8q68FSlck0bMP3ZCbBKNikY0ENj6GDY3s33/N
ih1hWLyRtVnqrw2w2551Zi8gMj1dp7MOXogM4cp+hy+R13ihWKcwoshteGrX6mOb
4B1tLUtgweFeCiFLFC2Lno2M4OtiSSBbI6d5Dr4kIX7h9rCkDBxZHkTvZEGK82z3
+1S4w8QLceRiyoJplPhb+81S2PrLedG7fbSpcwRAv/lHZzxFFaRgrcyxNw8wn5/o
LDpV0+MRaf/VgZSiTB99Yn+bisXfKsaGLSbKytsM9Ts2LgvBP0C/W/0Ja86ghDUc
D+WygQyikdpKN6UX04ZKJ2FUQJ+7/W6gXupHlzvvUvxy71iHSLaArmv/3I2UNgcv
0A17RtwfpdoJAWH0mFH6
=NKZe
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Upgrade fom Tomat 8.0.33 to 8.5.9 - stuck clients

[Enrico Olivelli]

Hi,
I'm upgrading from Tomcat 8.0.33.
I see that after a period of work requests remains "pending", for instance
I get all clients remaining
waiting for a response (parsing HTTP Response header) and no active thread
on my Tomcat.

This happens in my QA environment where I start several WebDriver/Unit
tests againts my web application. I have many cases, the most simple I this
is the following:

on  the client side (a JAX-WS client):

"main" #1 prio=5 os_prio=0 tid=0x7f7cf0009000 nid=0x7ddc runnable
[0x7f7cf6e3f000]

   java.lang.Thread.State: RUNNABLE

at java.net.SocketInputStream.socketRead0(Native Method)

at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)

at java.net.SocketInputStream.read(SocketInputStream.java:170)

at java.net.SocketInputStream.read(SocketInputStream.java:141)

at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)

at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)

at java.io.BufferedInputStream.read(BufferedInputStream.java:345)

- locked <0xfd159a18> (a java.io.BufferedInputStream)

at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)

at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)

at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)

- locked <0xfcecee78> (a
sun.net.www.protocol.http.HttpURLConnection)

at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)

- locked <0xfcecee78> (a
sun.net.www.protocol.http.HttpURLConnection)

at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)

at
com.sun.xml.ws.transport.http.client.HttpClientTransport.readResponseCodeAndMessage(HttpClientTransport.java:209)

at
com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:162)

at
com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:94)

at
com.sun.xml.ws.transport.DeferredTransportPipe.processRequest(DeferredTransportPipe.java:116)

at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:598)

at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:557)

at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:542)

at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)

- locked <0xfcbb7720> (a com.sun.xml.ws.api.pipe.Fiber)

at
com.sun.xml.ws.api.pipe.helper.AbstractTubeImpl.process(AbstractTubeImpl.java:112)

at com.sun.xml.xwss.XWSSClientPipe.process(XWSSClientPipe.java:160)

at
com.sun.xml.ws.api.pipe.helper.PipeAdapter.processRequest(PipeAdapter.java:115)

at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:598)

at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:557)

at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:542)

at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)

- locked <0xfcbb7720> (a com.sun.xml.ws.api.pipe.Fiber)

at com.sun.xml.ws.client.Stub.process(Stub.java:222)

at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:135)

at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:109)

at
com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)

at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:118)


in the server side:
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.92-b14 mixed mode):


"anInnocuousThread" #861 daemon prio=5 os_prio=0 tid=0x7ff23c0de800
nid=0xdb26 runnable [0x7ff1ad379000]
   java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPoll.epollWait(Native Method)
at sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194)
at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268)
at java.lang.Thread.run(Thread.java:745)
at sun.misc.InnocuousThread.run(InnocuousThread.java:74)

(lots of this kind) (HTTPS Connector I think)

"https-jsse-nio2-10.168.10.55-8443-exec-130" #572 daemon prio=5 os_prio=0
tid=0x7ff23c0a nid=0xd4a2 waiting on condition [0x7ff1b2bbf000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x80f1d420> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
at