On Tue, Feb 25, 2014 at 11:37 AM, Daniel Mikusa <dmik...@gopivotal.com>wrote:
> On Feb 25, 2014, at 10:14 AM, Charles Richard < > charle...@thelearningbar.com> wrote: > > > Hi, > > > > On Tue, Feb 25, 2014 at 1:26 AM, Christopher Schultz < > > ch...@christopherschultz.net> wrote: > > > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA256 > >> > >> Charles, > >> > >> On 2/24/14, 10:15 AM, Charles Richard wrote: > >>> Sorry if this is not the right forum for this kind of inquiry. I > >>> figure the best candidates would be in this forum from personal > >>> experience. > >>> > >>> Our company is having production issues which I believe are either > >>> due to application inefficiencies or a bug somewhere in our > >>> software stack. > >>> > >>> We are having production issues with our Tomcat connection pool > >>> using c3p0 and while my knowledge in this area has improved, I lack > >>> the Java developer background that might help in this area and we > >>> are at a point where we need this solved quickly. > >> > >> I've never gotten the sense that c3p0 was production-ready. What made > >> you deploy with c2p0 instead of either of the two connection pools > >> that ship with Tomcat? (Note that c3p0 has nothing to do with Tomcat, > >> other than that Tomcat can be configured to use c3p0 as its > >> connection-pool). > >> > >> > > That is good to know that c3p0 might not be commonly used in production > by > > companies using tomcats. I was under the impression it was the most > > commonly used. > > > > > >> The problems could be related to leaked connections which I'm quite > >>> sure we have. I have turned on c3p0 debugging and identified this > >>> in the past and the ideal consultant could identify in our code > >>> where those are happening and fix them. > >> > >> Both pools Tomcat provides can help you track-down so-called > >> "abandoned" connections by providing stack traces that point to the > >> line of code that obtained the connection (or even Statement or > >> ResultSet). > >> > >> C3P0 also allows you to track down abandoned connections and my gut > > feeling tells me how problem is not tied down exclusively to this as our > > Tomcat connection pools go from 1 used connection to 150 in a minute and > I > > know we are not losing that many leaked connections in a minute. > > Regardless, I will check with the manager here to see if this has been > > tried in the past before I started this job and potentially try this as > > unfortunately, the problem is not happening in our staging environment. > > When you are experiencing the problem next, take some thread dumps. Try > to get two or three, with 10 to 15 seconds between each one. You can then > look at the thread dumps to see what's going on here. > > > https://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F > It's not easy to get good thread dumps as one minute things are fine and the next minute, the site is going downhill. I have bash scripts that monitor c3p0 connections and take an automatic thread dump as soon as the number exceeds what's normal. > > There is information about interpreting them here or if you need help with > it, you could post it to the list. > > > https://wiki.apache.org/tomcat/HowTo#How_do_I_read_a_Java_thread_dump_.3F > > I have tried to analyze the thread dumps I have gotten so far without much luck but this could just be that I don't understand enough what I'm looking for. I will look at the link provided. Thanks for taking the time to write this! Dan > > Cheers, Charles > > > > > >> I would highly recommend that you read this blog post I wrote several > >> years ago that can help you look for obvious errors by providing > >> examples for what JDBC code should look like -- if you are managing > >> your own JDBC calls of course: > >> > >> > http://blog.christopherschultz.net/index.php/2009/03/16/properly-handling-pooled-jdbc-connections/ > >> > >>> We are looking to hire a consultant that would come to Fredericton, > >>> NB, Canada to work with us on this problem. Serious inquiries only. > >>> I will be looking for proof that you have extensive experience with > >>> Tomcat, Java Spring and c3p0. > >>> > >>> If you are interested, send me your resume (through your company > >>> or individually) and send me as much proof as possible of your > >>> experience with the specific technologies mentioned. > >> > >> While there is certainly no prohibition against doing so, this isn't > >> really a "help wanted" message board. We are happy to help you -- for > >> free! -- via email to solve your own problems. If you find this free > >> forum helpful, please consider staying to be a part of the community. > >> You may help others or learn things yourself. > >> > >> I find this forum for the most part helpful and I've posted a few times > > before. In general though, I haven't felt that comfortable to post here. > > > > Before you hire anyone, you might want to try the following: > >> > >> 1. Swap c3p0 for something more reliable such as Tomcat's default pool > >> (based upon commons-dbcp) or the newer, higher-performance > >> tomcat-pool. You can find documentation for both of those online in > >> the Users' Guide. If the pool is performing okay, this may not be > >> necessary, but it's nearly free to try so it may be a good first-step. > >> > > > > I might have to wait a month to try this so our peak traffic subsides as > I > > could test the config on our staging environment but since the problem is > > only happening on production, I would be nervous of not having the > optimal > > configuration and make things worse. > > > >> > >> 2. Check your own code for resource-handling errors. Take some time to > >> read my blog post, or have your engineers do it (better yet, have > >> everyone read it). > >> > >> Will do. > > > > > >> 3. Run a static-analysis tool such as FindBugs (which is free) against > >> your code. It can detect resource leaks you may not have known you > >> had. Running the tool takes almost no setup and/or skill, and you can > >> read the report entirely for free. If you have a million errors, it > >> might take you a lot of staff to identify and fix the problems, but > >> it's likely to point-out some problem areas -- and point you to the > >> exact line numbers where problems are likely to be. > >> > >> Thanks for the recommendation, I've never heard of this tool. > > > > > >> You might get a lot of mileage (kilometerage?) out of the above > >> without having to hire anyone. > >> > >> Thanks for taking the time to write this and trying to help! > > > >> - -chris > >> > > > > Cheers! > > Charles > > > > > >> -----BEGIN PGP SIGNATURE----- > >> Version: GnuPG v1 > >> Comment: GPGTools - http://gpgtools.org > >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > >> > >> iQIcBAEBCAAGBQJTDCmHAAoJEBzwKT+lPKRYUTAP/iNhYjUQqaYIXCEn7WrKEpVW > >> PCn4+ZLmzlmm1r7GSVbSc0+632JKA+YSOz+c8LplGnMAmUa6eU7sz2KgrunsnCCm > >> y5UiNySJ3nQEtKcHn0c7TpNmfGUGn61Y0GoK9A+JID8jubxBNMmt4nvH7FBfRrGA > >> GjB93hwIsJ0pK0InmejXmcoBxRYIzaVavV7nW7I/63SURv+TnXmuriHtrC5KBB39 > >> /o5V9VNaDZqsm7zsi3qqEYoxY2/yEMt+W9++bYajXFhhhUevWdRw2NFgihyewRuL > >> xc4GWV+5OIvSlsr0JtKXlJha8ooz1BVPigRd6vRQDaw2DFNzXy0Jl30kGK5ECuwI > >> SmldUzYjZZGEJBhN2RGNsJgDi0zpvOIBmAciUCWZFNy7xzhubx+DTK1AHUYjW15j > >> H1ujwZAkXfSsfY9h+vP9ATntn2YBQ3R3L7pKtjd+d7T5C9ULDIglH4o3+2kK4TEg > >> zV+bUhyLXM/sf9IXMOTUZa9TsF8EovbaT5sRx2gXWqbMKOgyy3lJuzMaOg3110Cd > >> P5tOfypeevWE0NY5+nUXDtmgKz+GHAkaOkdhlvDBjQ0UwNL/yV9WU5ui1pev2KLi > >> Ct7DHj92K//ROM6RerkMjHPCcLPX5c6bbBzXl5R/unfprkPoRZ0RwgxQK/2ARAst > >> ZDSf1VD+AUL82qVQy1rf > >> =VgOW > >> -----END PGP SIGNATURE----- > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
