Re: Tomcat 5 pause (free gmail account for fix :-)
Yoav, Would you be interested in working on a contract basis to assist us with this problem? If so what would your per hour charge be? Thanks, Sam On Thu, 14 Oct 2004 16:06:37 -0400, Shapira, Yoav <[EMAIL PROTECTED]> wrote: > > Hi, > Excellent post, that one from Matt Dale -- good advice. > > Yoav Shapira http://www.yoavshapira.com > > > > > >-Original Message- > >From: Dale, Matt [mailto:[EMAIL PROTECTED] > >Sent: Thursday, October 14, 2004 4:04 PM > >To: Tomcat Users List; Sam Gallant > >Subject: RE: Tomcat 5 pause (free gmail account for fix :-) > > > >I reckon this still might be a garbage collection problem. > > > >I would download jvmstat from sun and use visualgc to monitor this. It > >gives a good graphical representation of when the box is garbage > collecting > >and how much stuff is in each of the generations. > > > >We had a similar situation and this arose because the app created lots > of > >very short lived objects. Increasing the young generation size > alleviated > >this problem and also reducing tenuring so medium term objects made it > to > >the old generation. > > > >There are also a few more garbage collection parameters you can play > with. > >If you have multiple processors you could try enabling the parallel > >collector. There is also the Concurrent Mark Sweep collector which > locks > >the JVM only in a small part of it's collection. You could try using > the > >aggressive heap settings which will try to use all the memory in the > >machine that it can. > > > >I think more visibility of your garbage collection cycles is the first > port > >of call in any event. > > > >-Original Message- > >From: Sam Gallant [mailto:[EMAIL PROTECTED] > >Sent: 14 October 2004 20:17 > >To: [EMAIL PROTECTED] > >Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > > >Everyone, > >Thanks in advance for any help. Also I have a gmail invite for the > >person who has a fix for this if they are interested. > > > >My company has been using Tomcat for several years, but a problem has > >crept up that we have not been able to solve. Basically, tomcat will > >stop processing requests for 2-60 second period several times a day. > > > >Here is a list of software that we have tried. (Note we have tried > >changing each key componant to see if we isolate the componant that is > >the problem, but no luck yet) > > > >OS: RedHat 9 & AS3 > >Threading model: linux threads & nptl > >JVM: sun 1.4.2_4 & latest ibm > >Http connector: ajp w/apache 2 and coyote connector > >JBDC connector 1.0 > > > > > >1. Doesn't always happen durning old gen garbage collection, but does > >sometimes > >2. Before switching to incremental gc we received out of memory errors > >which resulted in Tomcat completly hanging > >3. After switching to incremental gc the effect changed to 2-60 second > >periods of time that Tomcat won't process request, but it does resume > >on its' own. > >4. Cpu usage for most of the day is less than 20% utiliztion, but when > >the problem occurs the cpu spikes to 100% utiliztion briefly. > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > This e-mail, including any attachments, is a confidential business communication, > and may contain information that is confidential, proprietary and/or privileged. > This e-mail is intended only for the individual(s) to whom it is addressed, and may > not be saved, copied, printed, disclosed or used by anyone else. If you are not > the(an) intended recipient, please immediately delete this e-mail from your computer > system and notify the sender. Thank you. > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
> From: "Cox, Charlie" <[EMAIL PROTECTED]> > Sent: Thursday, October 14, 2004 12:54 PM > Is your machine swapping? 512MB can go quickly... It sounds like swapping to me. The problem is that even though you say "256MB" on the Java Heap, the heap consumes a LOT more than that in Real Ram, and could easily be bouncing off of the 512MB RAM limit of the machine. If there is anything else on this machine I'd be extra concerned. GC and Swap DO NOT mix. Try lowering your Xmx size, and try to get a better grasp on the real image size of your Java process. You could also try added RAM to the machine as well. Regards, Will Hartung ([EMAIL PROTECTED]) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
We are going to run a test this evening with an application that someone else wrote other than our own developers to see if the problem still exists. When the problem occurs we hit 100% cpu util with jsvc (tomcat deamon) taking all of the cpu. Also we switched back to the standard collector with 128 mb min / 256 mb max which resolves the out of memory problem, but we still have the pauses every so often. We have watched Visualgc and went through the jvmstat logs that showed us that new and old gen had plenty of memory when the problem exibited itself. We didn't have any gc for a period of time before and after the problem occured. Again thanks for all of the help and suggestions. On Thu, 14 Oct 2004 21:42:52 +0100, Dale, Matt <[EMAIL PROTECTED]> wrote: > > My next line of attack would be at the application itself. Is it possible that the > application is locking somewhere or is poorly threaded in some respect? > > What does top say when the cpu spikes? ie what is the load? what is the top process? > > If you experienced out of memory errors before you used the incremental collector > have you tried switching back to the standard collector but increasing the Xmx > setting to what is required by the app so as not to experience out of memory > exceptions. > > > > > -Original Message- > From: Sam Gallant [mailto:[EMAIL PROTECTED] > Sent: 14 October 2004 21:25 > To: Tomcat Users List > Subject: Re: Tomcat 5 pause (free gmail account for fix :-) > > Matt, > We have used jvmstat/visualgc which is how we found out that the > problem happens at times when no gc is taking place. We are actually > logging every second with jvmstat everyday at this point. The problem > we are stuck at now is trying to figure out how to get more infomation > out of the jvm when we can't catch the problem with a profiler and > can't run verbose gc for a long period of time. > > On Thu, 14 Oct 2004 16:06:37 -0400, Shapira, Yoav <[EMAIL PROTECTED]> wrote: > > > > Hi, > > Excellent post, that one from Matt Dale -- good advice. > > > > Yoav Shapira http://www.yoavshapira.com > > > > > > > > > > >-----Original Message----- > > >From: Dale, Matt [mailto:[EMAIL PROTECTED] > > >Sent: Thursday, October 14, 2004 4:04 PM > > >To: Tomcat Users List; Sam Gallant > > >Subject: RE: Tomcat 5 pause (free gmail account for fix :-) > > > > > >I reckon this still might be a garbage collection problem. > > > > > >I would download jvmstat from sun and use visualgc to monitor this. It > > >gives a good graphical representation of when the box is garbage > > collecting > > >and how much stuff is in each of the generations. > > > > > >We had a similar situation and this arose because the app created lots > > of > > >very short lived objects. Increasing the young generation size > > alleviated > > >this problem and also reducing tenuring so medium term objects made it > > to > > >the old generation. > > > > > >There are also a few more garbage collection parameters you can play > > with. > > >If you have multiple processors you could try enabling the parallel > > >collector. There is also the Concurrent Mark Sweep collector which > > locks > > >the JVM only in a small part of it's collection. You could try using > > the > > >aggressive heap settings which will try to use all the memory in the > > >machine that it can. > > > > > >I think more visibility of your garbage collection cycles is the first > > port > > >of call in any event. > > > > > >-Original Message- > > >From: Sam Gallant [mailto:[EMAIL PROTECTED] > > >Sent: 14 October 2004 20:17 > > >To: [EMAIL PROTECTED] > > >Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > > > > > >Everyone, > > >Thanks in advance for any help. Also I have a gmail invite for the > > >person who has a fix for this if they are interested. > > > > > >My company has been using Tomcat for several years, but a problem has > > >crept up that we have not been able to solve. Basically, tomcat will > > >stop processing requests for 2-60 second period several times a day. > > > > > >Here is a list of software that we have tried. (Note we have tried > > >changing each key componant to see if we isolate the componant that is > > >the problem, but no luck yet) > > > > > >OS: RedHat 9 & AS3 > > >Threading model: linux threads &a
RE: Tomcat 5 pause (free gmail account for fix :-)
My next line of attack would be at the application itself. Is it possible that the application is locking somewhere or is poorly threaded in some respect? What does top say when the cpu spikes? ie what is the load? what is the top process? If you experienced out of memory errors before you used the incremental collector have you tried switching back to the standard collector but increasing the Xmx setting to what is required by the app so as not to experience out of memory exceptions. -Original Message- From: Sam Gallant [mailto:[EMAIL PROTECTED] Sent: 14 October 2004 21:25 To: Tomcat Users List Subject: Re: Tomcat 5 pause (free gmail account for fix :-) Matt, We have used jvmstat/visualgc which is how we found out that the problem happens at times when no gc is taking place. We are actually logging every second with jvmstat everyday at this point. The problem we are stuck at now is trying to figure out how to get more infomation out of the jvm when we can't catch the problem with a profiler and can't run verbose gc for a long period of time. On Thu, 14 Oct 2004 16:06:37 -0400, Shapira, Yoav <[EMAIL PROTECTED]> wrote: > > Hi, > Excellent post, that one from Matt Dale -- good advice. > > Yoav Shapira http://www.yoavshapira.com > > > > > >-Original Message- > >From: Dale, Matt [mailto:[EMAIL PROTECTED] > >Sent: Thursday, October 14, 2004 4:04 PM > >To: Tomcat Users List; Sam Gallant > >Subject: RE: Tomcat 5 pause (free gmail account for fix :-) > > > >I reckon this still might be a garbage collection problem. > > > >I would download jvmstat from sun and use visualgc to monitor this. It > >gives a good graphical representation of when the box is garbage > collecting > >and how much stuff is in each of the generations. > > > >We had a similar situation and this arose because the app created lots > of > >very short lived objects. Increasing the young generation size > alleviated > >this problem and also reducing tenuring so medium term objects made it > to > >the old generation. > > > >There are also a few more garbage collection parameters you can play > with. > >If you have multiple processors you could try enabling the parallel > >collector. There is also the Concurrent Mark Sweep collector which > locks > >the JVM only in a small part of it's collection. You could try using > the > >aggressive heap settings which will try to use all the memory in the > >machine that it can. > > > >I think more visibility of your garbage collection cycles is the first > port > >of call in any event. > > > >-Original Message- > >From: Sam Gallant [mailto:[EMAIL PROTECTED] > >Sent: 14 October 2004 20:17 > >To: [EMAIL PROTECTED] > >Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > > >Everyone, > >Thanks in advance for any help. Also I have a gmail invite for the > >person who has a fix for this if they are interested. > > > >My company has been using Tomcat for several years, but a problem has > >crept up that we have not been able to solve. Basically, tomcat will > >stop processing requests for 2-60 second period several times a day. > > > >Here is a list of software that we have tried. (Note we have tried > >changing each key componant to see if we isolate the componant that is > >the problem, but no luck yet) > > > >OS: RedHat 9 & AS3 > >Threading model: linux threads & nptl > >JVM: sun 1.4.2_4 & latest ibm > >Http connector: ajp w/apache 2 and coyote connector > >JBDC connector 1.0 > > > > > >1. Doesn't always happen durning old gen garbage collection, but does > >sometimes > >2. Before switching to incremental gc we received out of memory errors > >which resulted in Tomcat completly hanging > >3. After switching to incremental gc the effect changed to 2-60 second > >periods of time that Tomcat won't process request, but it does resume > >on its' own. > >4. Cpu usage for most of the day is less than 20% utiliztion, but when > >the problem occurs the cpu spikes to 100% utiliztion briefly. > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > This e-mail, including any attachments, is a confidential business communication, > and may contain information that is confidential, proprietary and/or privileged. > This e-mail is intended only for the individual(s) to whom it is addressed, and may > not be saved, copied, printed, disclose
Re: Tomcat 5 pause (free gmail account for fix :-)
Nope not swapping. Based on what we see in jvmstat/visualgc we typically have 64mb of data in old gen, 12mb of app data, and about 12 mb of new gen data. The OS still has free memory as well. On Thu, 14 Oct 2004 15:54:59 -0400, Cox, Charlie <[EMAIL PROTECTED]> wrote: > Is your machine swapping? 512MB can go quickly... > > > > > -Original Message- > > From: Sam Gallant [mailto:[EMAIL PROTECTED] > > Sent: Thursday, October 14, 2004 3:40 PM > > To: Montz, James C. (James Tower) > > Cc: Tomcat Users List > > Subject: Re: Tomcat 5 pause (free gmail account for fix :-) > > > > We have done default gc with 8mb/256mb min/max, default gc with > > 128mb/256mb min/max, and incremental with the same settings. The > > machine has 512mb ram. When we turn on verbose gc the gc logging just > > stops after a period of time both on the sun jvm and the ibm. When > > there is 100% cpu util jsvc is using 99% of the cpu. > > > > > > On Thu, 14 Oct 2004 14:33:13 -0500, Montz, James C. (James Tower) > > <[EMAIL PROTECTED]> wrote: > > > Are you defining any Memory Min/Max (-Xms/-Xmx) constraints on the JVM? > > > Are you collecting any verbose GC information (-verbose:gc)? > > > > > > How much physical memory available on the machine? > > > > > > At times of 100% CPU Utilization, is it java that is consumnig the > > > majority of it? > > > > > > > > > > > > -Original Message- > > > From: Sam Gallant [mailto:[EMAIL PROTECTED] > > > Sent: Thursday, October 14, 2004 2:17 PM > > > To: [EMAIL PROTECTED] > > > Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > > > Everyone, > > > Thanks in advance for any help. Also I have a gmail invite for the > > > person who has a fix for this if they are interested. > > > > > > My company has been using Tomcat for several years, but a problem has > > > crept up that we have not been able to solve. Basically, tomcat will > > > stop processing requests for 2-60 second period several times a day. > > > > > > Here is a list of software that we have tried. (Note we have tried > > > changing each key componant to see if we isolate the componant that is > > > the problem, but no luck yet) > > > > > > OS: RedHat 9 & AS3 > > > Threading model: linux threads & nptl > > > JVM: sun 1.4.2_4 & latest ibm > > > Http connector: ajp w/apache 2 and coyote connector > > > JBDC connector 1.0 > > > > > > 1. Doesn't always happen durning old gen garbage collection, but does > > > sometimes > > > 2. Before switching to incremental gc we received out of memory errors > > > which resulted in Tomcat completly hanging > > > 3. After switching to incremental gc the effect changed to 2-60 second > > > periods of time that Tomcat won't process request, but it does resume > > > on its' own. > > > 4. Cpu usage for most of the day is less than 20% utiliztion, but when > > > the problem occurs the cpu spikes to 100% utiliztion briefly. > > > > > > - > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > - > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
Matt, We have used jvmstat/visualgc which is how we found out that the problem happens at times when no gc is taking place. We are actually logging every second with jvmstat everyday at this point. The problem we are stuck at now is trying to figure out how to get more infomation out of the jvm when we can't catch the problem with a profiler and can't run verbose gc for a long period of time. On Thu, 14 Oct 2004 16:06:37 -0400, Shapira, Yoav <[EMAIL PROTECTED]> wrote: > > Hi, > Excellent post, that one from Matt Dale -- good advice. > > Yoav Shapira http://www.yoavshapira.com > > > > > >-Original Message- > >From: Dale, Matt [mailto:[EMAIL PROTECTED] > >Sent: Thursday, October 14, 2004 4:04 PM > >To: Tomcat Users List; Sam Gallant > >Subject: RE: Tomcat 5 pause (free gmail account for fix :-) > > > >I reckon this still might be a garbage collection problem. > > > >I would download jvmstat from sun and use visualgc to monitor this. It > >gives a good graphical representation of when the box is garbage > collecting > >and how much stuff is in each of the generations. > > > >We had a similar situation and this arose because the app created lots > of > >very short lived objects. Increasing the young generation size > alleviated > >this problem and also reducing tenuring so medium term objects made it > to > >the old generation. > > > >There are also a few more garbage collection parameters you can play > with. > >If you have multiple processors you could try enabling the parallel > >collector. There is also the Concurrent Mark Sweep collector which > locks > >the JVM only in a small part of it's collection. You could try using > the > >aggressive heap settings which will try to use all the memory in the > >machine that it can. > > > >I think more visibility of your garbage collection cycles is the first > port > >of call in any event. > > > >-Original Message- > >From: Sam Gallant [mailto:[EMAIL PROTECTED] > >Sent: 14 October 2004 20:17 > >To: [EMAIL PROTECTED] > >Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > > >Everyone, > >Thanks in advance for any help. Also I have a gmail invite for the > >person who has a fix for this if they are interested. > > > >My company has been using Tomcat for several years, but a problem has > >crept up that we have not been able to solve. Basically, tomcat will > >stop processing requests for 2-60 second period several times a day. > > > >Here is a list of software that we have tried. (Note we have tried > >changing each key componant to see if we isolate the componant that is > >the problem, but no luck yet) > > > >OS: RedHat 9 & AS3 > >Threading model: linux threads & nptl > >JVM: sun 1.4.2_4 & latest ibm > >Http connector: ajp w/apache 2 and coyote connector > >JBDC connector 1.0 > > > > > >1. Doesn't always happen durning old gen garbage collection, but does > >sometimes > >2. Before switching to incremental gc we received out of memory errors > >which resulted in Tomcat completly hanging > >3. After switching to incremental gc the effect changed to 2-60 second > >periods of time that Tomcat won't process request, but it does resume > >on its' own. > >4. Cpu usage for most of the day is less than 20% utiliztion, but when > >the problem occurs the cpu spikes to 100% utiliztion briefly. > > > >- > >To unsubscribe, e-mail: [EMAIL PROTECTED] > >For additional commands, e-mail: [EMAIL PROTECTED] > > > This e-mail, including any attachments, is a confidential business communication, > and may contain information that is confidential, proprietary and/or privileged. > This e-mail is intended only for the individual(s) to whom it is addressed, and may > not be saved, copied, printed, disclosed or used by anyone else. If you are not > the(an) intended recipient, please immediately delete this e-mail from your computer > system and notify the sender. Thank you. > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat 5 pause (free gmail account for fix :-)
Hi, Excellent post, that one from Matt Dale -- good advice. Yoav Shapira http://www.yoavshapira.com >-Original Message- >From: Dale, Matt [mailto:[EMAIL PROTECTED] >Sent: Thursday, October 14, 2004 4:04 PM >To: Tomcat Users List; Sam Gallant >Subject: RE: Tomcat 5 pause (free gmail account for fix :-) > >I reckon this still might be a garbage collection problem. > >I would download jvmstat from sun and use visualgc to monitor this. It >gives a good graphical representation of when the box is garbage collecting >and how much stuff is in each of the generations. > >We had a similar situation and this arose because the app created lots of >very short lived objects. Increasing the young generation size alleviated >this problem and also reducing tenuring so medium term objects made it to >the old generation. > >There are also a few more garbage collection parameters you can play with. >If you have multiple processors you could try enabling the parallel >collector. There is also the Concurrent Mark Sweep collector which locks >the JVM only in a small part of it's collection. You could try using the >aggressive heap settings which will try to use all the memory in the >machine that it can. > >I think more visibility of your garbage collection cycles is the first port >of call in any event. > >-Original Message- >From: Sam Gallant [mailto:[EMAIL PROTECTED] >Sent: 14 October 2004 20:17 >To: [EMAIL PROTECTED] >Subject: Tomcat 5 pause (free gmail account for fix :-) > > >Everyone, >Thanks in advance for any help. Also I have a gmail invite for the >person who has a fix for this if they are interested. > >My company has been using Tomcat for several years, but a problem has >crept up that we have not been able to solve. Basically, tomcat will >stop processing requests for 2-60 second period several times a day. > >Here is a list of software that we have tried. (Note we have tried >changing each key componant to see if we isolate the componant that is >the problem, but no luck yet) > >OS: RedHat 9 & AS3 >Threading model: linux threads & nptl >JVM: sun 1.4.2_4 & latest ibm >Http connector: ajp w/apache 2 and coyote connector >JBDC connector 1.0 > > >1. Doesn't always happen durning old gen garbage collection, but does >sometimes >2. Before switching to incremental gc we received out of memory errors >which resulted in Tomcat completly hanging >3. After switching to incremental gc the effect changed to 2-60 second >periods of time that Tomcat won't process request, but it does resume >on its' own. >4. Cpu usage for most of the day is less than 20% utiliztion, but when >the problem occurs the cpu spikes to 100% utiliztion briefly. > >- >To unsubscribe, e-mail: [EMAIL PROTECTED] >For additional commands, e-mail: [EMAIL PROTECTED] This e-mail, including any attachments, is a confidential business communication, and may contain information that is confidential, proprietary and/or privileged. This e-mail is intended only for the individual(s) to whom it is addressed, and may not be saved, copied, printed, disclosed or used by anyone else. If you are not the(an) intended recipient, please immediately delete this e-mail from your computer system and notify the sender. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat 5 pause (free gmail account for fix :-)
I reckon this still might be a garbage collection problem. I would download jvmstat from sun and use visualgc to monitor this. It gives a good graphical representation of when the box is garbage collecting and how much stuff is in each of the generations. We had a similar situation and this arose because the app created lots of very short lived objects. Increasing the young generation size alleviated this problem and also reducing tenuring so medium term objects made it to the old generation. There are also a few more garbage collection parameters you can play with. If you have multiple processors you could try enabling the parallel collector. There is also the Concurrent Mark Sweep collector which locks the JVM only in a small part of it's collection. You could try using the aggressive heap settings which will try to use all the memory in the machine that it can. I think more visibility of your garbage collection cycles is the first port of call in any event. -Original Message- From: Sam Gallant [mailto:[EMAIL PROTECTED] Sent: 14 October 2004 20:17 To: [EMAIL PROTECTED] Subject: Tomcat 5 pause (free gmail account for fix :-) Everyone, Thanks in advance for any help. Also I have a gmail invite for the person who has a fix for this if they are interested. My company has been using Tomcat for several years, but a problem has crept up that we have not been able to solve. Basically, tomcat will stop processing requests for 2-60 second period several times a day. Here is a list of software that we have tried. (Note we have tried changing each key componant to see if we isolate the componant that is the problem, but no luck yet) OS: RedHat 9 & AS3 Threading model: linux threads & nptl JVM: sun 1.4.2_4 & latest ibm Http connector: ajp w/apache 2 and coyote connector JBDC connector 1.0 1. Doesn't always happen durning old gen garbage collection, but does sometimes 2. Before switching to incremental gc we received out of memory errors which resulted in Tomcat completly hanging 3. After switching to incremental gc the effect changed to 2-60 second periods of time that Tomcat won't process request, but it does resume on its' own. 4. Cpu usage for most of the day is less than 20% utiliztion, but when the problem occurs the cpu spikes to 100% utiliztion briefly. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Any opinions expressed in this E-mail may be those of the individual and not necessarily the company. This E-mail and any files transmitted with it are confidential and solely for the use of the intended recipient. If you are not the intended recipient or the person responsible for delivering to the intended recipient, be advised that you have received this E-mail in error and that any use or copying is strictly prohibited. If you have received this E-mail in error please notify the beCogent postmaster at [EMAIL PROTECTED] Unless expressly stated, opinions in this email are those of the individual sender and not beCogent Ltd. You must take full responsibility for virus checking this email and any attachments. Please note that the content of this email or any of its attachments may contain data that falls within the scope of the Data Protection Acts and that you must ensure that any handling or processing of such data by you is fully compliant with the terms and provisions of the Data Protection Act 1984 and 1998. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
We have tried using a profiler, but it hasn't worked well because the problem only happens a few times over an eight hour day. When we run the profiler the we get a different effect which may or may not be the same problem plus the new effect happens quite often. Has anyone else run into problem running the verbose gc for several hours? On Thu, 14 Oct 2004 14:33:34 -0500, QM <[EMAIL PROTECTED]> wrote: > On Thu, Oct 14, 2004 at 02:16:56PM -0500, Sam Gallant wrote: > : My company has been using Tomcat for several years, but a problem has > : crept up that we have not been able to solve. Basically, tomcat will > : stop processing requests for 2-60 second period several times a day. > > It still may be some sort of GC activity. A profiler will show you > exactly what's going on, and (better still) whether it's a particular > type of object or place of object creation that's causing a GC spike. > > Be sure to give the profiler the Gmail invite. ;) > > -QM > > -- > > software -- http://www.brandxdev.net > tech news -- http://www.RoarNetworX.com > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
I have a similar problem(TC 3.3.1) but it is related to our pooling solution running out of connections and/or it trying to reap not-checked-in connections. The lockups users reported were during a reap cycle. After a few seconds up to a minute(sound familiar) it would "go away" and the app would start servicing requests again(checkout "netstat -a" and count your connections before and after your hicup). It only happens when someone "hits" the place where I didn't checkin enough times to deplete the pool. I'm feverishly trying to finish implementation so as a bandaid I've increased my pools depth to 100 connections for alpha testing...working for now. Question: Anyone have any ideas on how to track what/where a checkout doesn't get checked-in(better pooling solution? I'm using bitmechanic now). What benefits does TC 4/5 built-in pooling mechanism have over my current solution? Also, when my TC ran out of memory(out of Resources Exception) I just increased the Heap size using TOMCAT_OPTS="$TOMCAT_OPTS -Xmx300m" placed in the tomcat.sh script(TC 3.3.1 remember) and it's never happened again(no object loitering, just lots of objects!). If you grow too large(and/or never stop growing!) I'd look into indirectly referenced objects..here's a link to an article about this behavior(good performance analysis info): http://www.opensourcetutorials.com/tutorials/Server-Side-Coding/Java/java-garbage-collection-performance/page4.html --Jonathan Sam Gallant wrote: Everyone, Thanks in advance for any help. Also I have a gmail invite for the person who has a fix for this if they are interested. My company has been using Tomcat for several years, but a problem has crept up that we have not been able to solve. Basically, tomcat will stop processing requests for 2-60 second period several times a day. Here is a list of software that we have tried. (Note we have tried changing each key componant to see if we isolate the componant that is the problem, but no luck yet) OS: RedHat 9 & AS3 Threading model: linux threads & nptl JVM: sun 1.4.2_4 & latest ibm Http connector: ajp w/apache 2 and coyote connector JBDC connector 1.0 1. Doesn't always happen durning old gen garbage collection, but does sometimes 2. Before switching to incremental gc we received out of memory errors which resulted in Tomcat completly hanging 3. After switching to incremental gc the effect changed to 2-60 second periods of time that Tomcat won't process request, but it does resume on its' own. 4. Cpu usage for most of the day is less than 20% utiliztion, but when the problem occurs the cpu spikes to 100% utiliztion briefly. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat 5 pause (free gmail account for fix :-)
Is your machine swapping? 512MB can go quickly... > -Original Message- > From: Sam Gallant [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 14, 2004 3:40 PM > To: Montz, James C. (James Tower) > Cc: Tomcat Users List > Subject: Re: Tomcat 5 pause (free gmail account for fix :-) > > We have done default gc with 8mb/256mb min/max, default gc with > 128mb/256mb min/max, and incremental with the same settings. The > machine has 512mb ram. When we turn on verbose gc the gc logging just > stops after a period of time both on the sun jvm and the ibm. When > there is 100% cpu util jsvc is using 99% of the cpu. > > > On Thu, 14 Oct 2004 14:33:13 -0500, Montz, James C. (James Tower) > <[EMAIL PROTECTED]> wrote: > > Are you defining any Memory Min/Max (-Xms/-Xmx) constraints on the JVM? > > Are you collecting any verbose GC information (-verbose:gc)? > > > > How much physical memory available on the machine? > > > > At times of 100% CPU Utilization, is it java that is consumnig the > > majority of it? > > > > > > > > -Original Message- > > From: Sam Gallant [mailto:[EMAIL PROTECTED] > > Sent: Thursday, October 14, 2004 2:17 PM > > To: [EMAIL PROTECTED] > > Subject: Tomcat 5 pause (free gmail account for fix :-) > > > > Everyone, > > Thanks in advance for any help. Also I have a gmail invite for the > > person who has a fix for this if they are interested. > > > > My company has been using Tomcat for several years, but a problem has > > crept up that we have not been able to solve. Basically, tomcat will > > stop processing requests for 2-60 second period several times a day. > > > > Here is a list of software that we have tried. (Note we have tried > > changing each key componant to see if we isolate the componant that is > > the problem, but no luck yet) > > > > OS: RedHat 9 & AS3 > > Threading model: linux threads & nptl > > JVM: sun 1.4.2_4 & latest ibm > > Http connector: ajp w/apache 2 and coyote connector > > JBDC connector 1.0 > > > > 1. Doesn't always happen durning old gen garbage collection, but does > > sometimes > > 2. Before switching to incremental gc we received out of memory errors > > which resulted in Tomcat completly hanging > > 3. After switching to incremental gc the effect changed to 2-60 second > > periods of time that Tomcat won't process request, but it does resume > > on its' own. > > 4. Cpu usage for most of the day is less than 20% utiliztion, but when > > the problem occurs the cpu spikes to 100% utiliztion briefly. > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
We have done default gc with 8mb/256mb min/max, default gc with 128mb/256mb min/max, and incremental with the same settings. The machine has 512mb ram. When we turn on verbose gc the gc logging just stops after a period of time both on the sun jvm and the ibm. When there is 100% cpu util jsvc is using 99% of the cpu. On Thu, 14 Oct 2004 14:33:13 -0500, Montz, James C. (James Tower) <[EMAIL PROTECTED]> wrote: > Are you defining any Memory Min/Max (-Xms/-Xmx) constraints on the JVM? > Are you collecting any verbose GC information (-verbose:gc)? > > How much physical memory available on the machine? > > At times of 100% CPU Utilization, is it java that is consumnig the > majority of it? > > > > -Original Message- > From: Sam Gallant [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 14, 2004 2:17 PM > To: [EMAIL PROTECTED] > Subject: Tomcat 5 pause (free gmail account for fix :-) > > Everyone, > Thanks in advance for any help. Also I have a gmail invite for the > person who has a fix for this if they are interested. > > My company has been using Tomcat for several years, but a problem has > crept up that we have not been able to solve. Basically, tomcat will > stop processing requests for 2-60 second period several times a day. > > Here is a list of software that we have tried. (Note we have tried > changing each key componant to see if we isolate the componant that is > the problem, but no luck yet) > > OS: RedHat 9 & AS3 > Threading model: linux threads & nptl > JVM: sun 1.4.2_4 & latest ibm > Http connector: ajp w/apache 2 and coyote connector > JBDC connector 1.0 > > 1. Doesn't always happen durning old gen garbage collection, but does > sometimes > 2. Before switching to incremental gc we received out of memory errors > which resulted in Tomcat completly hanging > 3. After switching to incremental gc the effect changed to 2-60 second > periods of time that Tomcat won't process request, but it does resume > on its' own. > 4. Cpu usage for most of the day is less than 20% utiliztion, but when > the problem occurs the cpu spikes to 100% utiliztion briefly. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 5 pause (free gmail account for fix :-)
On Thu, Oct 14, 2004 at 02:16:56PM -0500, Sam Gallant wrote: : My company has been using Tomcat for several years, but a problem has : crept up that we have not been able to solve. Basically, tomcat will : stop processing requests for 2-60 second period several times a day. It still may be some sort of GC activity. A profiler will show you exactly what's going on, and (better still) whether it's a particular type of object or place of object creation that's causing a GC spike. Be sure to give the profiler the Gmail invite. ;) -QM -- software -- http://www.brandxdev.net tech news -- http://www.RoarNetworX.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tomcat 5 pause (free gmail account for fix :-)
Are you defining any Memory Min/Max (-Xms/-Xmx) constraints on the JVM? Are you collecting any verbose GC information (-verbose:gc)? How much physical memory available on the machine? At times of 100% CPU Utilization, is it java that is consumnig the majority of it? -Original Message- From: Sam Gallant [mailto:[EMAIL PROTECTED] Sent: Thursday, October 14, 2004 2:17 PM To: [EMAIL PROTECTED] Subject: Tomcat 5 pause (free gmail account for fix :-) Everyone, Thanks in advance for any help. Also I have a gmail invite for the person who has a fix for this if they are interested. My company has been using Tomcat for several years, but a problem has crept up that we have not been able to solve. Basically, tomcat will stop processing requests for 2-60 second period several times a day. Here is a list of software that we have tried. (Note we have tried changing each key componant to see if we isolate the componant that is the problem, but no luck yet) OS: RedHat 9 & AS3 Threading model: linux threads & nptl JVM: sun 1.4.2_4 & latest ibm Http connector: ajp w/apache 2 and coyote connector JBDC connector 1.0 1. Doesn't always happen durning old gen garbage collection, but does sometimes 2. Before switching to incremental gc we received out of memory errors which resulted in Tomcat completly hanging 3. After switching to incremental gc the effect changed to 2-60 second periods of time that Tomcat won't process request, but it does resume on its' own. 4. Cpu usage for most of the day is less than 20% utiliztion, but when the problem occurs the cpu spikes to 100% utiliztion briefly. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
