NPE in SecureNioChannel (TC 8.5.5)
Hello, A thread from a day or two ago mentioned a NPE in SecureNioChannel when the connector is configured with Http11NioProtocol. OP mentioned using Http11Nio2Protocol resolves the issue. I am also seeing this exception and it is resolved by switching to the Nio2 protocol implementation. Is this a reasonable workaround until TC 8.5.6? Are there dis/advantages to using Nio2 instead of Nio? Thanks, Colin Ubuntu 12.04.5 LTS (GNU/Linux 3.0.0-32-virtual x86_64) java version "1.8.0_101" Tomcat 8.5.5 14-Sep-2016 17:26:44.052 SEVERE [https-jsse-nio-443-exec-9] org.apache.coyote.AbstractProtocol$ConnectionHandler.process Error reading request, ignored java.lang.NullPointerException at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.getSslSupport(NioEndpoint.java:1329) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:792) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1410) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) (Sorry I wasn't subscribed to the list so I can't reply to the original thread.) - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to create an alias for a context name
On Fri, Jun 13, 2014 at 4:39 PM, Christopher Schultz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > James, > > On 6/13/14, 4:11 PM, James H. H. Lampert wrote: >> When this thread first came up, I tried one obvious approach: >> creating a symbolic link in the file system, mapping a new >> directory name into an existing context's directory. >> >> It didn't work. > > If it did work, what you'd end up with is double-deployment, which is > something the OP wanted to avoid. > > If the lb is the problem, the solution is to make the canonical "name" > of the application one that does not include an underscore. Then, use > something like url-rewrite (or, even better, do it at the lb) to > *redirect* users permanently from the "old" URL space to the new URL > space. > > Or, just get a load-balancer that doesn't fall-over when you have > underscores in your URLs. That's just plain bad software. :( > > - -chris I couldn't agree more re "bad software" and I'll go ahead and name names. It's an Amazon "elastic load balancer". It's configured to provide session affinity using the JSESSIONID cookie. When tomcat sets the JSESSIONID it uses "path=/the_context". The ELB then sets its AWSELB cookie with "path=/the%5Fcontext". I don't know if that's valid per the cookie "spec", but that doesn't matter b/c neither Firefox nor Chrome will return the AWSELB cookie w/ the underscore encoded like that. This of course breaks the session affinity causing all sorts of annoying problems. I may open an issue w/ Amazon about it, but there is no telling when (if) they'd bother to fix it. So I'm pursing other options as well. Thanks again for the suggestions, Colin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to create an alias for a context name
On Fri, Jun 13, 2014 at 8:37 AM, Konstantin Kolinko wrote: > 2014-06-12 21:56 GMT+04:00 Colin Ingarfield : >> Hello, >> >> Is it possible in Tomcat 6/7 to create an alias for a context? We >> have a context that includes an underscore and that is unfortunately >> causing problems w/ a load balancer and cookie paths. Our URLs >> resemble this: >> https://example.com/the_context/login.jsp >> >> Where we have deployed a war file named "the_context.war". >> >> I'd like to somehow alias the context name so both of these urls would >> effectively be the same: >> https://example.com/the_context/login.jsp >> https://example.com/thecontext/login.jsp >> >> Ideally this would be done w/o deploying the application twice. Is >> this possible in Tomcat 6 and/or 7 ? >> > > > While it may be possible to create an "alias" my hacking into Tomcat's > mapper component (that maps requests to Contexts), you will have > problem if your web application calls ServletContext.getContextPath() > as that information does not depend on the incoming request. > > There are two typical solutions: > a) Name one URL as "the canonical" one and redirect your users from > the old URL to the new one. That is to respond with HTTP status code > 301 and a redirection. > b) Rewrite URLs in the incoming request and in response (if needed). > > a) can be done by a simple Filter or by using a standard "rewrite" one > b) needs some mode advanced rewrite implementation > > You may use UrlRewriteFilter [1] (in any version of Tomcat), > RewriteValve (in Tomcat 8), > mod_rewrite in Apache HTTPD server (if your Tomcat runs behind that server). > > [1] http://wiki.apache.org/tomcat/AddOns#UrlRewrite > > Best regards, > Konstantin Kolinko > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > OK, that makes sense and I can see how this could be done w/ just a Filter. And thank you for the link to UrlRewrite... I was not aware of it and it looks very powerful/flexible. Thanks! - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
How to create an alias for a context name
Hello, Is it possible in Tomcat 6/7 to create an alias for a context? We have a context that includes an underscore and that is unfortunately causing problems w/ a load balancer and cookie paths. Our URLs resemble this: https://example.com/the_context/login.jsp Where we have deployed a war file named "the_context.war". I'd like to somehow alias the context name so both of these urls would effectively be the same: https://example.com/the_context/login.jsp https://example.com/thecontext/login.jsp Ideally this would be done w/o deploying the application twice. Is this possible in Tomcat 6 and/or 7 ? Thanks, Colin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Deadlock when using jetty 8 JDBCSessionManager and Tomcat 7 JDBC Connector
First of all, thank you this detailed analysis of the situation. Based on the bug report I suspected the driver, but it now seems clear it was (at least part of) the problem. re: Why was it necessary for the abandoned connection thread to act? I'm not sure.. I looked into the JDBC Session manager class and it appeared to be simply reading/deserializing a session from the database when the deadlock occurred. I would expect that to be a very fast query -- this is a production system but under very light load. I looked into the pool configuration and 'removed abandoned timeout' was 60 seconds. That timeout seems a bit short, but if it takes > 60 sec to load a session from the database then there is a serious performance issue. I've never observed delays like that in the application. My guess at this point is perhaps the db (Amazon RDS/MySql) was unavailable/very slow/down in some way, causing this connection to be held beyond the timeout. I don't have great visibility into performance or outage information for RDS, tho I may reach out to DBAs and try to get it. Thanks again for your help. Regards, Colin On Mon, Mar 18, 2013 at 6:39 AM, Mark Thomas wrote: > On 15/03/2013 21:36, Colin Ingarfield wrote: > > Short version: > Your upgrade to the latest Connector/J will have fixed this particular > problem. > > > Long version: > > > >>>>> Found one Java-level deadlock: = > > > > Thread 1: > >>>>> Here are the stack traces: Thread 12820: (state = BLOCKED) - >>>>> com.mysql.jdbc.ConnectionImpl.getCharacterSetMetadata() @bci=0, >>>>> line=2851 (Compiled frame) - >>>>> com.mysql.jdbc.Field.getStringFromBytes(int, int) @bci=37, >>>>> line=717 (Compiled frame) - com.mysql.jdbc.Field.getName() @bci=17, >>>>> line=631 (Interpreted frame) - >>>>> com.mysql.jdbc.ResultSetImpl.buildIndexMapping() @bci=78, line=752 >>>>> (Compiled frame) - >>>>> com.mysql.jdbc.ResultSetImpl.findColumn(java.lang.String) @bci=12, >>>>> line=1110 (Interpreted frame) - >>>>> com.mysql.jdbc.ResultSetImpl.getString(java.lang.String) @bci=3, >>>>> line=5609 (Interpreted frame) - >>>>> org.eclipse.jetty.server.session.JDBCSessionManager$1.run() >>>>> @bci=111, line=844 (Interpreted frame) - > > > > Thread 2: > >>>>> Thread 890: (state = BLOCKED) - >>>>> com.mysql.jdbc.ResultSetImpl.realClose(boolean) @bci=0, line=7195 >>>>> (Interpreted frame) - com.mysql.jdbc.ResultSetImpl.close() @bci=2, >>>>> line=909 (Interpreted frame) - >>>>> com.mysql.jdbc.StatementImpl.realClose(boolean, boolean) @bci=126, >>>>> line=2478 (Interpreted frame) - >>>>> com.mysql.jdbc.PreparedStatement.realClose(boolean, boolean) >>>>> @bci=71, line=3098 (Interpreted frame) - >>>>> com.mysql.jdbc.ConnectionImpl.closeAllOpenStatements() @bci=90, >>>>> line=1628 (Interpreted frame) - >>>>> com.mysql.jdbc.ConnectionImpl.realClose(boolean, boolean, boolean, >>>>> java.lang.Throwable) @bci=176, line=4388 (Interpreted frame) - >>>>> com.mysql.jdbc.ConnectionImpl.close() @bci=32, line=1601 >>>>> (Interpreted frame) - >>>>> org.apache.tomcat.jdbc.pool.PooledConnection.disconnect(boolean) >>>>> @bci=47, line=330 (Interpreted frame) - > > > >>>>> Once I dug up these stack traces I started to wonder if the mysql >>>>> driver was the problem (or contributing to the problem.) I was >>>>> using Connector/J version 5.1.19 when the deadlock occurred. I >>>>> found this bug: http://bugs.mysql.com/bug.php?id=61247 which >>>>> sounds a lot like what appears to have happened. I'm interested in >>>>> your thoughts on this. > > The problem is related to two threads accessing the same connection. > Given the scenario - the pool spots a potentially abandoned connection > and tries to close it - I don't view this as unreasonable behaviour by > the pool since there are no other options available. > > The problem is that Thread 1 (above) results in this call order: > ResultSetImpl.findColumn(java.lang.String) - Syncs on ResultSetImpl > ... > ConnectionImpl.getCharacterSetMetadata()- Syncs on ConnectionImpl > > while Thread 2 results in this call order: > ConnectionImpl.close()- Syncs on ConnectionImpl > ... > ResultSetImpl.realClose(boolean) - Syncs on ResultSetImpl > > Two threads obtaining the same locks in a different order will result i
Re: Deadlock when using jetty 8 JDBCSessionManager and Tomcat 7 JDBC Connector
On Fri, Mar 15, 2013 at 2:21 PM, Christopher Schultz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Colin, > > On 3/14/13 3:41 PM, Colin Ingarfield wrote: >> (Sorry I cannot reply correctly b/c I was on the digest list) >> >> The deadlocked threads: Deadlock Detection: >> >> Found one Java-level deadlock: = >> >> "qtp1840392480-3740": waiting to lock Monitor@0x7f4350001fd0 >> (Object@0x0006c01a0e88, a com/mysql/jdbc/JDBC4Connection), >> which is held by "PoolCleaner[2009981184:1363034108768]" >> "PoolCleaner[2009981184:1363034108768]": waiting to lock >> Monitor@0x7f4350001f28 (Object@0x0006c1ed5738, a >> com/mysql/jdbc/JDBC4ResultSet), which is held by >> "qtp1840392480-3740" >> >> Found a total of 1 deadlock. >> >> Here are the stack traces: Thread 12820: (state = BLOCKED) - >> com.mysql.jdbc.ConnectionImpl.getCharacterSetMetadata() @bci=0, >> line=2851 (Compiled frame) - >> com.mysql.jdbc.Field.getStringFromBytes(int, int) @bci=37, >> line=717 (Compiled frame) - com.mysql.jdbc.Field.getName() @bci=17, >> line=631 (Interpreted frame) - >> com.mysql.jdbc.ResultSetImpl.buildIndexMapping() @bci=78, line=752 >> (Compiled frame) - >> com.mysql.jdbc.ResultSetImpl.findColumn(java.lang.String) @bci=12, >> line=1110 (Interpreted frame) - >> com.mysql.jdbc.ResultSetImpl.getString(java.lang.String) @bci=3, >> line=5609 (Interpreted frame) - >> org.eclipse.jetty.server.session.JDBCSessionManager$1.run() >> @bci=111, line=844 (Interpreted frame) - >> org.eclipse.jetty.server.handler.ContextHandler.handle(java.lang.Runnable) >> >> > @bci=53, line=1119 (Interpreted frame) >> - >> org.eclipse.jetty.server.session.JDBCSessionManager.loadSession(java.lang.String, >> >> > java.lang.String, java.lang.String) @bci=61, line=884 (Interpreted >> frame) - >> org.eclipse.jetty.server.session.JDBCSessionManager.getSession(java.lang.String) >> >> > @bci=345, line=518 (Interpreted frame) >> - >> org.eclipse.jetty.server.session.JDBCSessionManager.getSession(java.lang.String) >> >> > @bci=2, line=69 (Interpreted frame) >> - >> org.eclipse.jetty.server.session.AbstractSessionManager.getHttpSession(java.lang.String) >> >> > @bci=13, line=272 (Interpreted frame) >> - >> org.eclipse.jetty.server.session.SessionHandler.checkRequestedSessionId(org.eclipse.jetty.server.Request, >> >> > javax.servlet.http.HttpServletRequest) @bci=192, line=277 (Interpreted >> frame) - >> org.eclipse.jetty.server.session.SessionHandler.doScope(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=47, line=158 >> (Interpreted frame) - >> org.eclipse.jetty.server.handler.ContextHandler.doScope(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=416, line=999 >> (Interpreted frame) - >> org.eclipse.jetty.server.handler.ScopedHandler.handle(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=13, line=117 >> (Interpreted frame) - >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=399, line=250 >> (Interpreted frame) - >> org.eclipse.jetty.server.handler.HandlerCollection.handle(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=42, line=149 >> (Compiled frame) - >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(java.lang.String, >> >> > org.eclipse.jetty.server.Request, >> javax.servlet.http.HttpServletRequest, >> javax.servlet.http.HttpServletResponse) @bci=23, line=111 >> (Compiled frame) - >> org.eclipse.jetty.server.Server.handle(org.eclipse.jetty.server.AbstractHttpConnection) >> >> > @bci=134, line=350 (Compiled frame) >> - org.eclipse.jetty.server.AbstractHttpConnection.handleRequest() >> @bci=228, line=454 (Compiled frame) - >> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete() >> @bci=448, line=890 (Interpreted frame) - >> java.util.HashMap.get(java
Deadlock when using jetty 8 JDBCSessionManager and Tomcat 7 JDBC Connector
Hello, We're using Jetty 8.1.3.v20120416 w/ JDBCSessionManager as our application server with Tomcat 7's JDBC Connection pool 7.0.28. We've run this particular combination in production since at least Sept 2012. (Java 6 64 bit/Ubuntu, Amazon RDS/MySQL db.) This morning one Jetty instance stopped accepting requests and the problem appears to be a deadlock btw a jetty connection handler thread and the conn pool's "PoolCleaner" thread. From jstack trace: Deadlock Detection: Found one Java-level deadlock: = "qtp1840392480-3740": waiting to lock Monitor@0x7f4350001fd0 (Object@0x0006c01a0e88, a com/mysql/jdbc/JDBC4Connection), which is held by "PoolCleaner[2009981184:1363034108768]" "PoolCleaner[2009981184:1363034108768]": waiting to lock Monitor@0x7f4350001f28 (Object@0x0006c1ed5738, a com/mysql/jdbc/JDBC4ResultSet), which is held by "qtp1840392480-3740" Once this happened all other worker threads blocked at various points within JDBCSession manager. Obviously restarting the instance resolved the issue for now. How does one go about diagnosing / resolving such an issue? It's not clear to me if either component is "at fault" here...I have the full jstack output as well as a full heap dump of the java process, if that would help. Regards, Colin
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
On Friday I tried to recreate this issue using iptables (to block all outgoing traffic to 3306) but was still unable to recreate it. After unblocking 3306 the pool would eventually recover and create new connections. So I still do not understand why it was necessary to restart Tomcat to resolve the original problem. Since I cannot recreate the problem the best I can do is adjust some settings to help evaluate it if (when) it happens again in the future. 1. Turn on the 'abanonded' settings per your suggestions so I can see if the app is actually leaking connections. 2. Turn up pool logging to FINE. When a connection attempt times out or fails it logs at this level. 3. Possibly enable connect and TCP read timeouts on the mysql jdbc driver. Per the docs they are 'infinite' by default, but I think lower timeouts would help to detect network/firewall problems more quickly. And I'm writing a script to perform stack traces, heap dumps, lsof for open files, etc., to run on the jvm process before restarting in the event this happens again. Who knows, maybe the problem was 1 open sockets or something. Thank you for your time looking into this. I appreciate it. Regards, Colin Thank you Filip for all your help. On Thu, Mar 22, 2012 at 9:39 PM, Filip Hanik Mailing Lists < devli...@hanik.com> wrote: > Pretty much you're guaranteed to have a network problem at that point. You > see Java caches DNS translations forever, and yanking VPN like that may > change around IPs but the JVM is not aware of that. Wireshark would tell > you that. Now relying in VPN is never a good thing, but maybe it's > required. You could try > 1. Use IP instead of host name in your jdbc URL > 2. Configure the JRE to not cache dns lookups, (network.properties) > > > The error you see tells you that: > 1. The pool doesn't have any idle established connections idle=0 > 2. The pool doesn't have any connections used by other threads busy=0 > 3. There is currently 1 thread trying to activate a connection size=1. The > size is an atomic counter to protect against overuse in a lock free way. > > > Filip > > > > Hi Filip, > > Today I have been trying to recreate the issue by disconnecting from the > vpn, as: > 1. Start app. Pool creates some connections via the vpn. > 2. Test app a bit to execute sql queries. > 3. Shut down the vpn > 4. Force some more queries. Predictably, connections fail and exceptions > show up in the logs. > 5. Restore vpn connection > 6. Check if pool creates new connections, which it does not. > > I also upgraded to the latest pool available in maven > central: tomcat-jdbc-7.0.26.jar > > I understand this could still be a connection leak in my application. But > the new pool version logs an error I don't understand: > ... stack trace ... > Caused by: java.sql.SQLException: [scheduler-low-1] Timeout: Pool empty. > Unable to fetch a connection in 10 seconds, none available[size:1; busy:0; > idle:0; lastwait:1]. > ... more trace ... > > The relevant part of my current pool DataSource configuration: > removeAbandonedTimeout="10" > removeAbandoned="true" > logAbandoned="true" > > defaultAutoCommit="false" > maxActive="1" maxIdle="1" minIdle="1" maxWait="1" > testOnBorrow="true" > validationQuery="SELECT 1" > > I also have yet to see any "abandoned" log messages. > > Should the pool always have at least 1 busy or idle connection? If not > would it create another? > > Thanks, > Colin > > > > > On Thu, Mar 22, 2012 at 11:11 AM, Filip Hanik Mailing Lists < > devli...@hanik.com> wrote: > > > > Ultimately tho I'd still like to see some debug logging from the pool > > > itself. Is there a simple way to turn it on? > > > > not to the problem you are looking at. if a connection got taken out of > > the pool, and it passed validation, then everything is ok. > > at this point the SQLException you get has all the data, and the problem > > is probably at the network level > > > > the fact that you see that for 2 hours and problem goes away with > restart, > > that can only be the app holding on to the flawed connection, cause there > > would have been several validations during the 2 hour period :) I think > > there is a loop somewhere that when it fails it just retries and retries, > > logAbandoned will show that though. > > > > Filip > > > > > > > > > > - Original Message - > > > From: "Colin Ingarfield" > > > To: "Tomcat Users List" > > > Sen
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
Hi Filip, Today I have been trying to recreate the issue by disconnecting from the vpn, as: 1. Start app. Pool creates some connections via the vpn. 2. Test app a bit to execute sql queries. 3. Shut down the vpn 4. Force some more queries. Predictably, connections fail and exceptions show up in the logs. 5. Restore vpn connection 6. Check if pool creates new connections, which it does not. I also upgraded to the latest pool available in maven central: tomcat-jdbc-7.0.26.jar I understand this could still be a connection leak in my application. But the new pool version logs an error I don't understand: ... stack trace ... Caused by: java.sql.SQLException: [scheduler-low-1] Timeout: Pool empty. Unable to fetch a connection in 10 seconds, none available[size:1; busy:0; idle:0; lastwait:1]. ... more trace ... The relevant part of my current pool DataSource configuration: removeAbandonedTimeout="10" removeAbandoned="true" logAbandoned="true" defaultAutoCommit="false" maxActive="1" maxIdle="1" minIdle="1" maxWait="1" testOnBorrow="true" validationQuery="SELECT 1" I also have yet to see any "abandoned" log messages. Should the pool always have at least 1 busy or idle connection? If not would it create another? Thanks, Colin On Thu, Mar 22, 2012 at 11:11 AM, Filip Hanik Mailing Lists < devli...@hanik.com> wrote: > > Ultimately tho I'd still like to see some debug logging from the pool > > itself. Is there a simple way to turn it on? > > not to the problem you are looking at. if a connection got taken out of > the pool, and it passed validation, then everything is ok. > at this point the SQLException you get has all the data, and the problem > is probably at the network level > > the fact that you see that for 2 hours and problem goes away with restart, > that can only be the app holding on to the flawed connection, cause there > would have been several validations during the 2 hour period :) I think > there is a loop somewhere that when it fails it just retries and retries, > logAbandoned will show that though. > > Filip > > > > > - Original Message - > > From: "Colin Ingarfield" > > To: "Tomcat Users List" > > Sent: Thursday, March 22, 2012 8:06:14 AM > > Subject: Re: how to enable debug logging for Tomcat jdbc pool (Tomcat > 6.0.32) > > > > Ah, Wireshark. My friend calls it the "universal debugger". :) > > > > I will set the validation interval to 1 and keep an eye on the > > network to > > see what's going on. I may also install MySql locally so I can kill > > it > > easily to try and simulation connection timeouts. I won't really > > feel this > > is resolved until I can recreate the original issue. > > > > > > > Thanks, > > Colin > > > > On Wed, Mar 21, 2012 at 11:20 AM, Filip Hanik Mailing Lists < > > devli...@hanik.com> wrote: > > > > > it will take a while to see the abandoned log. I'm not implying > > > every > > > request hogs the connection, but that you could have ended up in a > > > scenario > > > where that did happen. > > > otherwise, you would have not seen the problem for 2 hours and to > > > go away > > > when the system was restarted, as it should have failed on > > > validation. > > > > > > You can enable validation every single time by doing > > > > > > validationInterval="1" > > > > > > after that, if it was me, I'd start pulling in something like > > > Wireshark to > > > see what is going on > > > > > > Filip > > > > > > - Original Message - > > > > From: "Colin Ingarfield" > > > > To: "Tomcat Users List" > > > > Sent: Wednesday, March 21, 2012 10:11:43 AM > > > > Subject: Re: how to enable debug logging for Tomcat jdbc pool > > > > (Tomcat > > > 6.0.32) > > > > > > > > I added the 3 abandoned settings but I don't see any indication > > > > in > > > > the > > > > tomcat log that connections are being abandoned. I also made the > > > > max > > > > pool > > > > size pretty small.. my application would have failed quickly if > > > > all > > > > the > > > > connections we're being incorrectly held up. > > > > > > > > Anything else I can try? Thanks again for your help. > > > > > > >
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
Ah, Wireshark. My friend calls it the "universal debugger". :) I will set the validation interval to 1 and keep an eye on the network to see what's going on. I may also install MySql locally so I can kill it easily to try and simulation connection timeouts. I won't really feel this is resolved until I can recreate the original issue. Ultimately tho I'd still like to see some debug logging from the pool itself. Is there a simple way to turn it on? Thanks, Colin On Wed, Mar 21, 2012 at 11:20 AM, Filip Hanik Mailing Lists < devli...@hanik.com> wrote: > it will take a while to see the abandoned log. I'm not implying every > request hogs the connection, but that you could have ended up in a scenario > where that did happen. > otherwise, you would have not seen the problem for 2 hours and to go away > when the system was restarted, as it should have failed on validation. > > You can enable validation every single time by doing > > validationInterval="1" > > after that, if it was me, I'd start pulling in something like Wireshark to > see what is going on > > Filip > > - Original Message - > > From: "Colin Ingarfield" > > To: "Tomcat Users List" > > Sent: Wednesday, March 21, 2012 10:11:43 AM > > Subject: Re: how to enable debug logging for Tomcat jdbc pool (Tomcat > 6.0.32) > > > > I added the 3 abandoned settings but I don't see any indication in > > the > > tomcat log that connections are being abandoned. I also made the max > > pool > > size pretty small.. my application would have failed quickly if all > > the > > connections we're being incorrectly held up. > > > > Anything else I can try? Thanks again for your help. > > > > -- Colin > > > > On Wed, Mar 21, 2012 at 10:41 AM, Filip Hanik Mailing Lists < > > devli...@hanik.com> wrote: > > > > > Got it, thank you. > > > The other way this can happen is if the application checks out a > > > connection and then never returns it, and expects it to be used. > > > For this you will want to enable > > > > > > removeAbandonedTimeout="60" > > > removeAbandoned="true" > > > logAbandoned="true" > > > > > > this should tell you pretty quickly if you got a component that is > > > hogging > > > the connection. So test that first. Now if that is the case, there > > > is a way > > > to fix that: > > > > > > 1. remove the above settings > > > 2. compile and configure the interceptor described in: > > > https://issues.apache.org/bugzilla/show_bug.cgi?id=52024 > > > In this interceptor, when a failure occurs, it automatically > > > reconnects > > > and retries the operation. And that is the only way to get around > > > the > > > problem (assuming my assumption is correct) > > > > > > > > > Filip > > > - Original Message - > > > > From: "Colin Ingarfield" > > > > To: users@tomcat.apache.org > > > > Sent: Wednesday, March 21, 2012 9:30:46 AM > > > > Subject: Re: how to enable debug logging for Tomcat jdbc pool > > > > (Tomcat > > > 6.0.32) > > > > > > > > My configuration: > > > > > > > > > > > name="jdbc/cdb.mysql" > > > > defaultAutoCommit="false" > > > > driverClassName="com.mysql.jdbc.Driver" > > > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" > > > > url="jdbc:mysql://X.com/_dev?sessionVariables=TRANSACTION > > > > ISOLATION LEVEL READ COMMITTED" > > > > username="X" > > > > password="X" > > > > > > > > maxActive="100" > > > > maxIdle="100" > > > > minIdle="10" > > > > initialSize="10" > > > > maxWait="1" > > > > testOnBorrow="true" > > > > type="javax.sql.DataSource" > > > > validationQuery="SELECT 1"/> > > > > > > > > > > > > I have testOnBorrow and validationQuery set as you suggest, so I > > > > do > > > > not > > > > think that is the issue. > > > > > > > > Thanks, > > > > Colin > > > > > > > > > > - > > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
Chris, On Thu, Mar 22, 2012 at 7:40 AM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Colin, > > On 3/21/12 12:11 PM, Colin Ingarfield wrote: > > I added the 3 abandoned settings but I don't see any indication in > > the tomcat log that connections are being abandoned. I also made > > the max pool size pretty small.. my application would have failed > > quickly if all the connections we're being incorrectly held up. > > In development, I recommend setting your max connection pool size to > "1": you'll find potential deadlocks that way, too. > > > http://blog.christopherschultz.net/index.php/2009/03/16/properly-handling-pooled-jdbc-connections/ > > - -chris > -BEGIN PGP SIGNATURE- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk9rHbgACgkQ9CaO5/Lv0PCHaQCgnv5/vPGuULmZDHk2/H4/TNcr > 3nkAmgOKjma3jVulg56+UaZIHFquEsgB > =8YkY > -END PGP SIGNATURE- > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > Setting the pool size to 1 for dev is a good idea. I'll try that. Nice blog post re: jdbc. Thankfully I use Spring 3 JDBC and it takes care of all that jdbc grunt work for me. It's possible my code is doing something strange that prevents Spring from cleaning up, but I'm not sure what that would be. Thanks for the suggestions, Colin
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
I added the 3 abandoned settings but I don't see any indication in the tomcat log that connections are being abandoned. I also made the max pool size pretty small.. my application would have failed quickly if all the connections we're being incorrectly held up. Anything else I can try? Thanks again for your help. -- Colin On Wed, Mar 21, 2012 at 10:41 AM, Filip Hanik Mailing Lists < devli...@hanik.com> wrote: > Got it, thank you. > The other way this can happen is if the application checks out a > connection and then never returns it, and expects it to be used. > For this you will want to enable > > removeAbandonedTimeout="60" > removeAbandoned="true" > logAbandoned="true" > > this should tell you pretty quickly if you got a component that is hogging > the connection. So test that first. Now if that is the case, there is a way > to fix that: > > 1. remove the above settings > 2. compile and configure the interceptor described in: > https://issues.apache.org/bugzilla/show_bug.cgi?id=52024 > In this interceptor, when a failure occurs, it automatically reconnects > and retries the operation. And that is the only way to get around the > problem (assuming my assumption is correct) > > > Filip > - Original Message - > > From: "Colin Ingarfield" > > To: users@tomcat.apache.org > > Sent: Wednesday, March 21, 2012 9:30:46 AM > > Subject: Re: how to enable debug logging for Tomcat jdbc pool (Tomcat > 6.0.32) > > > > My configuration: > > > > > name="jdbc/cdb.mysql" > > defaultAutoCommit="false" > > driverClassName="com.mysql.jdbc.Driver" > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" > > url="jdbc:mysql://X.com/_dev?sessionVariables=TRANSACTION > > ISOLATION LEVEL READ COMMITTED" > > username="X" > > password="X" > > > > maxActive="100" > > maxIdle="100" > > minIdle="10" > > initialSize="10" > > maxWait="1" > > testOnBorrow="true" > > type="javax.sql.DataSource" > > validationQuery="SELECT 1"/> > > > > > > I have testOnBorrow and validationQuery set as you suggest, so I do > > not > > think that is the issue. > > > > Thanks, > > Colin > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
My configuration: I have testOnBorrow and validationQuery set as you suggest, so I do not think that is the issue. Thanks, Colin
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
>> Hello, >> >> I'm using the new Tomcat jdbc pool (1.1.0.1) with Tomcat 6.0.32, Ubuntu >> x86_64. I would like to increase the logging from the pool to try and >> chase down connection timeouts. > >Where your "1.1.0.1" comes from? > >It is not an official release. >Released versions of jdbc-pool come with Tomcat 7 and share its version numbers. > >Best regards, >Konstantin Kolinko iirc I copied the version number from the tomcat-jdbc.jar/META-INF/MANIFEST.MF file and renamed the jar myself. (Looking in the manifest now I see Bundle-Version: 1.1.0.1). I prefer to avoid unversioned jar files in my project as it can cause confusion. But I don't recall which version of Tomcat 7 I got the jar from. Is the best policy to always use the tomcat-jdbc jar from the latest version of Tomcat 7? Thank you, Colin
Re: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
ct.GeneratedMethodAccessor22.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29] at net.bull.javamelody.JdbcWrapper$4.invoke(JdbcWrapper.java:660) ~[javamelody-core-1.34.0.jar:1.34.0] at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:232) ~[javamelody-core-1.34.0.jar:1.34.0] at $Proxy21.getConnection(Unknown Source) ~[na:na] at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.getTargetConnection(LazyConnectionDataSourceProxy.java:403) ~[spring-jdbc-3.0.5.RELEASE.jar:3.0.5.RELEASE] at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:376) ~[spring-jdbc-3.0.5.RELEASE.jar:3.0.5.RELEASE] at $Proxy28.prepareStatement(Unknown Source) ~[na:na] at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29] at net.bull.javamelody.JdbcWrapper$ConnectionInvocationHandler.invoke(JdbcWrapper.java:176) ~[javamelody-core-1.34.0.jar:1.34.0] at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:232) ~[javamelody-core-1.34.0.jar:1.34.0] at $Proxy29.prepareStatement(Unknown Source) ~[na:na] at org.springframework.jdbc.core.JdbcTemplate$SimplePreparedStatementCreator.createPreparedStatement(JdbcTemplate.java:1375) ~[spring-jdbc-3.0.5.RELEASE.jar:3.0.5.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:580) ~[spring-jdbc-3.0.5.RELEASE.jar:3.0.5.RELEASE] ... 22 common frames omitted Caused by: java.net.ConnectException: Connection timed out at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.6.0_29] at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351) ~[na:1.6.0_29] at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213) ~[na:1.6.0_29] at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200) ~[na:1.6.0_29] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) ~[na:1.6.0_29] at java.net.Socket.connect(Socket.java:529) ~[na:1.6.0_29] at java.net.Socket.connect(Socket.java:478) ~[na:1.6.0_29] at java.net.Socket.(Socket.java:375) ~[na:1.6.0_29] at java.net.Socket.(Socket.java:218) ~[na:1.6.0_29] at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:257) ~[mysql-connector-java-5.1.17-bin.jar:na] at com.mysql.jdbc.MysqlIO.(MysqlIO.java:294) ~[mysql-connector-java-5.1.17-bin.jar:na] ... 68 common frames omitted On 03/20/2012 10:29 AM, Filip Hanik (mailing lists) wrote: Define "connection timeouts" so that we can understand your problem to suggest for how to trace it down. What are you trying to search for. Errors would be logged as errors, and should show up with the standard configuration Filip -Original Message- From: Colin Ingarfield [mailto:colin...@gmail.com ] Sent: Monday, March 19, 2012 1:51 PM To: users@tomcat.apache.org Subject: how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32) Hello, I'm using the new Tomcat jdbc pool (1.1.0.1) with Tomcat 6.0.32, Ubuntu x86_64. I would like to increase the logging from the pool to try and chase down connection timeouts. I added the following line to $CATALINA_BASE/conf/logging.properties: org.apache.tomcat.jdbc.pool.level=FINE (the rest of the logging.properties file is unchanged.) But I don't see any pool debug logging output on the console. I thought this setting would enable debug logging for all classes in that package. Is there something else I need to do? Thank you, Colin
how to enable debug logging for Tomcat jdbc pool (Tomcat 6.0.32)
Hello, I'm using the new Tomcat jdbc pool (1.1.0.1) with Tomcat 6.0.32, Ubuntu x86_64. I would like to increase the logging from the pool to try and chase down connection timeouts. I added the following line to $CATALINA_BASE/conf/logging.properties: org.apache.tomcat.jdbc.pool.level=FINE (the rest of the logging.properties file is unchanged.) But I don't see any pool debug logging output on the console. I thought this setting would enable debug logging for all classes in that package. Is there something else I need to do? Thank you, Colin
