Re: Severe errors in log on Tomcat 8.5.5

2016-09-17 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Andreas,

On 9/17/16 11:55 AM, Andreas Røsdal wrote:
> Freeciv-web runs Tomcat on https://play.freeciv.org/ and I have
> some questions about some error messages that I get in the Tomcat
> logs.

(Cool! I had never heard of freeciv-web. "No more turns no more
turns... no more turns! :)

> I recently upgraded from Tomcat 8.0.37 to Tomcat 8.5.5. I think
> that there are some stability issues which came as a result of the
> upgrade to Tomcat 8.5.5.
> 
> The source code of the Java web application can be found here: 
> https://github.com/freeciv/freeciv-web/tree/develop/freeciv-web
> 
> Nginx runs in front of Tomcat as a HTTP 2 proxy.
> 
> Server version:Apache Tomcat/8.5.5 Server built:
> Aug 31 2016 19:51:16 UTC Server number: 8.5.5.0 OS Name:
> Linux OS Version:4.4.0-36-generic Architecture:
> amd64 Java Home: /opt/jdk/jdk1.8.0_73/jre JVM Version:
> 1.8.0_73-b02 JVM Vendor:Oracle Corporation
> 
> 
> These are some of the errors that I see in the Tomcat logs:
> 
> 17-Sep-2016 17:44:27.241 INFO [http-nio-8080-exec-10] 
> org.apache.coyote.http11.Http11Processor.service Error parsing HTTP
> request header Note: further occurrences of HTTP header parsing
> errors will be logged at DEBUG level. 
> java.lang.IllegalArgumentException: Invalid character found in
> method name. HTTP method names must be tokens at
> org.apache.coyote.http11.Http11InputBuffer.parseRequestLine( 
> Http11InputBuffer.java:462) at
> org.apache.coyote.http11.Http11Processor.service( 
> Http11Processor.java:667) at
> org.apache.coyote.AbstractProcessorLight.process( 
> AbstractProcessorLight.java:66) at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process( 
> AbstractProtocol.java:802) at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor. 
> doRun(NioEndpoint.java:1410) at
> org.apache.tomcat.util.net.SocketProcessorBase.run( 
> SocketProcessorBase.java:49) at
> java.util.concurrent.ThreadPoolExecutor.runWorker( 
> ThreadPoolExecutor.java:1142) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run( 
> ThreadPoolExecutor.java:617) at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( 
> TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

This looks like a client error. Can you get a byte-level protocol
trace of the request/response?

> java.lang.NullPointerException at
> org.apache.coyote.http11.Http11OutputBuffer.commit( 
> Http11OutputBuffer.java:332) at
> org.apache.coyote.http11.Http11Processor.prepareResponse( 
> Http11Processor.java:1288) at
> org.apache.coyote.AbstractProcessor.action( 
> AbstractProcessor.java:261) at
> org.apache.coyote.http11.Http11Processor.endRequest( 
> Http11Processor.java:1457) at
> org.apache.coyote.http11.Http11Processor.service( 
> Http11Processor.java:823) at
> org.apache.coyote.AbstractProcessorLight.process( 
> AbstractProcessorLight.java:66) at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process( 
> AbstractProtocol.java:802) at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor. 
> doRun(NioEndpoint.java:1410) at
> org.apache.tomcat.util.net.SocketProcessorBase.run( 
> SocketProcessorBase.java:49) at
> java.util.concurrent.ThreadPoolExecutor.runWorker( 
> ThreadPoolExecutor.java:1142) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run( 
> ThreadPoolExecutor.java:617) at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( 
> TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

That error is that the socketWrapper object is null. :(

http://svn.apache.org/viewvc/tomcat/tc8.5.x/tags/TOMCAT_8_5_5/java/org/a
pache/coyote/http11/Http11OutputBuffer.java?revision=1758670&view=markup
#l332

Looks like the response object or facade has been trashed.

> 17-Sep-2016 17:45:17.466 SEVERE [http-nio-8080-exec-20] 
> org.apache.coyote.http11.Http11Processor.service Error processing
> request java.lang.NullPointerException at
> org.apache.catalina.connector.CoyoteAdapter.service( 
> CoyoteAdapter.java:389) at
> org.apache.coyote.http11.Http11Processor.service( 
> Http11Processor.java:784) at
> org.apache.coyote.AbstractProcessorLight.process( 
> AbstractProcessorLight.java:66) at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process( 
> AbstractProtocol.java:802) at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor. 
> doRun(NioEndpoint.java:1410) at
> org.apache.tomcat.util.net.SocketProcessorBase.run( 
> SocketProcessorBase.java:49) at
> java.util.concurrent.ThreadPoolExecutor.runWorker( 
> ThreadPoolExecutor.java:1142) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run( 
> ThreadPoolExecutor.java:617) at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run( 
> TaskThread.java:61) at java.lang.Thread.run(Thread.java:745)

Any of the request, its mapping data, or the context it was mapped to
has been trashed.

http://svn.apache.org/viewvc/tomcat/tc8.5.x/tags/TOMCAT_8_5_5/java/org/a
pache/catalina

Re: Threadlocal leaks while Tomcat shutdown

2016-09-17 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Amit,

On 9/17/16 9:41 AM, Amit Pande wrote:
> This might not be the right forum to ask this question.  Yet wanted
> to if anyone faced this issue.
> 
> Our application uses Jacorb library to talk to legacy daemons over
> CORBA.
> 
> However while stopping the Tomcat, observing following errors. They
> are from the jacrob.jar ..but not sure how to prevent this. Any
> thoughts ?
> 
> 
> SEVERE: The web application [] created a ThreadLocal with key of
> type [org.jacorb.orb.Delegate$1] (value
> [org.jacorb.orb.Delegate$1@62305af6]) and a value of type
> [java.lang.Boolean] (value [false]) but failed to remove it when
> the web application was stopped. Threads are going to be renewed
> over time to try and avoid a probable memory leak. Sep 17, 2016
> 7:04:53 PM org.apache.catalina.loader.WebappClassLoaderBase
> checkThreadLocalMapForLeaks SEVERE: The web application [] created
> a ThreadLocal with key of type [org.jacorb.orb.Delegate$1] (value
> [org.jacorb.orb.Delegate$1@62305af6]) and a value of type
> [java.lang.Boolean] (value [false]) but failed to remove it when
> the web application was stopped. Threads are going to be renewed
> over time to try and avoid a probable memory leak.

Your best bet is to ask the Jacorb group how to clean their
ThreadLocal values since you are using this library in a
servlet-container context.

The good news is that Tomcat will slowly retire all of the threads in
the thread pool so that those ThreadLocals will eventually be GC'd and
you won't have a ClassLoader leak. So it would be best to find out how
to use Jacorb correctly (or, if they have a bug, get them to fix it),
but in the meantime you should be safe from memory leaks -- at least
from this particular kind of ThreadLocal leak.

Hope that helps,
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJX3XLBAAoJEBzwKT+lPKRYcg0P/13NUKxmR3U6i0e7HcSAMDbL
eBA5R+5SRGi0VAn9TnDOxLfedkjQnNaWk9k9lUrhm+pytJAb88r/Cbf2ijtrTK2H
N3Lva5sIFjnFT0Qx7b0dRYBTjKmmeiWxpRbwYbgYgO8ktJxU7I/DnpCV6rejViT5
7aLj/UhiMeQo+xWunoW3JDh8f3xRm3hxglAvwg/ah+V99K8HNBaT1HzHYX4j2+3e
uxPqouviZe1zJG4XCn737+MJaG7fVMROnSQEiccnB/W6Zyr8kIf1DsoUhtTi0N7O
vYcSbUIGQYIDh8q6sxxHOvc/t5a5Hul2toNnB+SCHpXw5EwKkLzA5euG1j78BLcP
R9ctvV1/1SIiyOew712T31bw563F6MdGOReXxvotz6veYCUiXVaTt7JPe2tGxl7p
qy7RjJzWTrTSIIaKToLDBFgvkNUjkNrRSuNMs/Mupr5VOfIloFVw4dMRRVwA2BrX
A0kKbAyZ0EcT5e9lTMHuD6wyqe0yuvZQd1r732WGsq+0ArlGcTGJgONuDOrKm8kl
HPAz3jKx40kTfW7RHr6cZABu0QL0AbnVWghZMXL6hHrDNMBkQwJYXbISrToq+ujH
6HIDm7BqsPrI94+ETY2dLLkYQkSILr4PREzuKVlya762t92unT42k5CfgzXw5QpF
LzCJjhDrRYICb3ETCIh7
=gwpo
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: SQLExceptions after stopping Tomcat

2016-09-17 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Yuval,

On 9/16/16 2:33 PM, Yuval Schwartz wrote:
> Hey Chris,
> 
> You have a line in your link that reads:
> 
> // if you want to issue multiple queries and/or // work with
> multiple result sets, either declare more Statement // and
> ResultSet locals above and duplicate all cleanup logic for them, //
> or make sure you close each object along the way before you // try
> to re-use the reference
> 
> I have the following code in one of my data accessing methods:
> 
> [same ps and rs instantiating as you]
> 
> try { ps = connection.prepareStatement(query1); rs =
> ps.executeQuery();
> 
> ...[process rs. (I dont close the ps or the rs at this point)]
> 
> ps = connection.prepareStatement(query2); rs = ps.executeQuery(); 
>  }
> 
> [catch and finally blocks as you specify]
> 
> As you can see I simply reuse the ps and rs variables, without
> closing them beforehand. Is this a problem?

That depends upon your database driver. I seem to recall Oracle
specifically being very unforgiving when it comes to failing to close
resources. I would add explicit "close" calls to every object you are
expecting to be finished with.

> The reason this never raised a flag before is because I have
> another database accessing method that executes two queries in the
> same fashion and it never caused any issues.

Are there any other factors involved? Maybe that other accessing
method doesn't get called very often? Is it in a separate process
where cleanup happens when the process completes? Do you have some
other kind of pooling mechanism that will auto-close resources?

I think it's always a good idea to follow the API's recommendations
about closing resources when you are done with them. In the worst
case, the close() method is a no-op and it will eventually be JIT'd
away. In the best case, you get clean resource management.

- -chris

-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJX3XIPAAoJEBzwKT+lPKRYhKsQAJo/+GhhRAQ2y6Mqkr54Ufb5
Wz22WHvJ15er8z/LdZ8FLzGchD7WsmgBZj8v3yeEptzoxK7rhjvoxrO7w6ZTtz7x
6Qogj+qa+/HeUqELnr2SkQdyrMp+aal/DEXQvMvFp+VAKrEq+x0OKeMGBvzaimsf
xt9AWHeD5YsfRyGqvW/RFDIPFcSq83VDha2owQEcvvzNaw/DRTPTi/OubHoDuFtF
+MCkLjXRkbYThg/ljElWivEDd2NZJP5yDtwTBAsnQeydvKVpN7vzA47tQOznbxHN
eVhM1aPxQim1Jt3++jXT+ByGHNpiHLuhKkxwVPm8ZD6WyWgFGHp7R6w9eddnMLth
mUhzS5HPCLYOS0x3XcuheuhybEQdWNiBh4d6dQS59+rkdU3CASPWrbwFsdEPwMq1
0OdDJAZ436t+EP+XJjObnT8KV9wBWFzEN3ywxhjHWdus6bmJR+O6dxlbFXYHoYdX
fOemuIsJvjb4vl0mw9nql4kKm1MsU46gAqjP89fiCaIVPXGJjGTyqLeD48HUiNNy
YwhSCEFcj1ucQp6vG+MU+AnvQbxwnnrymjcuDWsgMehG772zWmRsg5mT3NaK4EZr
rbPnfyZSMkNMgGMjwCZQl8GlKlrsyIgnS/5zozPiSmldqJ0dEyKhPgxIfWpAIJiG
K0/k2szfVtoSBaje5eIG
=+Il1
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Severe errors in log on Tomcat 8.5.5

2016-09-17 Thread Andreas Røsdal
Hello!

Freeciv-web runs Tomcat on https://play.freeciv.org/ and I have some
questions about some error messages that I get in the Tomcat logs. I
recently upgraded from Tomcat 8.0.37 to Tomcat 8.5.5. I think that there
are some stability issues which came as a result of the upgrade to Tomcat
8.5.5.

The source code of the Java web application can be found here:
https://github.com/freeciv/freeciv-web/tree/develop/freeciv-web

Nginx runs in front of Tomcat as a HTTP 2 proxy.

 Server version:Apache Tomcat/8.5.5
 Server built:  Aug 31 2016 19:51:16 UTC
 Server number: 8.5.5.0
 OS Name:   Linux
 OS Version:4.4.0-36-generic
 Architecture:  amd64
 Java Home: /opt/jdk/jdk1.8.0_73/jre
 JVM Version:   1.8.0_73-b02
 JVM Vendor:Oracle Corporation


These are some of the errors that I see in the Tomcat logs:

17-Sep-2016 17:44:27.241 INFO [http-nio-8080-exec-10]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request
header
 Note: further occurrences of HTTP header parsing errors will be logged at
DEBUG level.
 java.lang.IllegalArgumentException: Invalid character found in method
name. HTTP method names must be tokens
at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(
Http11InputBuffer.java:462)
at org.apache.coyote.http11.Http11Processor.service(
Http11Processor.java:667)
at org.apache.coyote.AbstractProcessorLight.process(
AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(
AbstractProtocol.java:802)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.
doRun(NioEndpoint.java:1410)
at org.apache.tomcat.util.net.SocketProcessorBase.run(
SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(
TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)



java.lang.NullPointerException
at org.apache.coyote.http11.Http11OutputBuffer.commit(
Http11OutputBuffer.java:332)
at org.apache.coyote.http11.Http11Processor.prepareResponse(
Http11Processor.java:1288)
at org.apache.coyote.AbstractProcessor.action(
AbstractProcessor.java:261)
at org.apache.coyote.http11.Http11Processor.endRequest(
Http11Processor.java:1457)
at org.apache.coyote.http11.Http11Processor.service(
Http11Processor.java:823)
at org.apache.coyote.AbstractProcessorLight.process(
AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(
AbstractProtocol.java:802)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.
doRun(NioEndpoint.java:1410)
at org.apache.tomcat.util.net.SocketProcessorBase.run(
SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(
TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

17-Sep-2016 17:45:17.466 SEVERE [http-nio-8080-exec-20]
org.apache.coyote.http11.Http11Processor.service Error processing request
 java.lang.NullPointerException
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:389)
at org.apache.coyote.http11.Http11Processor.service(
Http11Processor.java:784)
at org.apache.coyote.AbstractProcessorLight.process(
AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(
AbstractProtocol.java:802)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.
doRun(NioEndpoint.java:1410)
at org.apache.tomcat.util.net.SocketProcessorBase.run(
SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(
TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

17-Sep-2016 17:45:17.467 SEVERE [http-nio-8080-exec-20]
org.apache.coyote.http11.Http11Processor.endRequest Error finishing response
 java.lang.NullPointerException
at org.apache.coyote.http11.Http11OutputBuffer.flushBuffer(
Http11OutputBuffer.java:514)
at org.apache.coyote.http11.Http11OutputBuffer.finishResponse(
Http11OutputBuffer.java:301)
at org.apache.coyote.http11.Http11Processor.endRequest(
Http11Processor.java:1458)
at org.apache.coyote.http11.Http11Processor.service(
Http11Processor.java:823)
at org.apache.coyote.AbstractProcessorLight.process(
AbstractProcessorLight.java:66

Threadlocal leaks while Tomcat shutdown

2016-09-17 Thread Amit Pande
This might not be the right forum to ask this question.  Yet wanted to if 
anyone faced this issue.

Our application uses Jacorb library to talk to legacy daemons over CORBA.

However while stopping the Tomcat, observing following errors. They are from 
the jacrob.jar ..but not sure how to prevent this. Any thoughts ?


SEVERE: The web application [] created a ThreadLocal with key of type 
[org.jacorb.orb.Delegate$1] (value [org.jacorb.orb.Delegate$1@62305af6]) and a 
value of type [java.lang.Boolean] (value [false]) but failed to remove it when 
the web application was stopped. Threads are going to be renewed over time to 
try and avoid a probable memory leak.
Sep 17, 2016 7:04:53 PM org.apache.catalina.loader.WebappClassLoaderBase 
checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type 
[org.jacorb.orb.Delegate$1] (value [org.jacorb.orb.Delegate$1@62305af6]) and a 
value of type [java.lang.Boolean] (value [false]) but failed to remove it when 
the web application was stopped. Threads are going to be renewed over time to 
try and avoid a probable memory leak.


Thanks,
Amit