If Task Manager is that High it is a sure fire case that your app is using that amount of Memory!
Again, you will get a far clearer picture of things if you run the VisualGC - I wouldn't trust JRun metrics at all. N -----Original Message----- From: Douglas Knudsen [mailto:[EMAIL PROTECTED] Sent: 01 March 2005 14:50 To: CF-Server Subject: Re: memory use and CF ok On Tue, 1 Mar 2005 14:11:21 -0000, Robertson-Ravo, Neil (RX) <[EMAIL PROTECTED]> wrote: > Hey, > > All of this really stems on how your application interacts with Java/JVM. > There is no really technotes from Macromedia or otherwise which will provide > you with a best fit setting. Whatever else has been said ConcMarkSweep will > give you worse performance than a Parallel GC - for sure. However f you are > not sure, then simply remove any setting of the GC type and the JVM will > take / use Parallel - if you have multiple CPU's you need to use them ;-) > Good point! danke. > What metrics have you ran before to give you memory consumption? (DO NOT > rely on Task manager!) I have the JRun metrics loggin enabled. Example entries during load DateTime,,listenTh,delayTh,idleTh,busyTh,totalTh,delayRq,droppedRq,delayMs,T otalMem,FreeMem,J2EESessions 03/01 09:20:41,metrics ,3,0,6,2,11,0,0,0,655296,445742,1 03/01 09:21:14,metrics ,3,1,1,6,11,0,0,0,655296,487605,1 03/01 09:21:44,metrics ,3,0,6,3,12,0,0,0,655296,421309,1 03/01 09:22:17,metrics ,3,0,2,10,15,0,0,0,655296,506024,1 03/01 09:22:47,metrics ,2,0,10,2,14,0,0,0,655296,480495,1 03/01 09:23:19,metrics ,2,0,6,6,14,0,0,0,655296,537053,1 03/01 09:23:49,metrics ,2,0,8,4,14,0,0,0,655296,481596,1 03/01 09:24:21,metrics ,3,0,1,10,14,0,0,0,655296,532518,1 03/01 09:24:51,metrics ,5,0,9,2,16,0,0,0,655296,457535,1 03/01 09:25:23,metrics ,5,0,7,4,16,0,0,0,655296,528284,1 > > In Task manager ensure that you have the VM column visible, What you are > probably seeing is timedout requests or queued requests being buffered into > the system. Do you have a lot of Connection Reset by Peer's in the runtime > logs? If you do, then CF is backlogging processing of the requests which > will consume memory. Doh! I need a refresher course in using task manager. JVM heap is set to 640MB. Task mangers VM reports 850,000K and mem use is at 754,000k. what is the discrepency? > > I would certainly update your JVM as the GC in later versions is greatly > improved. ok, on the task list. and thanks again for your help on this. > > Thanks > > Neil > > > -----Original Message----- > From: Douglas Knudsen [mailto:[EMAIL PROTECTED] > Sent: 01 March 2005 14:04 > To: CF-Server > Subject: Re: memory use and CF > > thanks for the reply Neil, very helpful. Info out there on this is > scarce. I'm going to reply inline below. > > On Tue, 1 Mar 2005 10:03:52 -0000, Robertson-Ravo, Neil (RX) > <[EMAIL PROTECTED]> wrote: > > OK, > > > > Sorry I was delayed, damn timezones! OK, it looks as though you have a > > fairly beefy machine...A few things you can do here (and these are never > > documents by MM as they are no doubt out of scope of CF!) > > > > I would remove the -XX:+UseConcMarkSweepGC from your arguments and replace > > it with -XX:+UseParallelGC. If you have a 4 processor box then you going > to > > suffer some performance problems with a ConcMarkSweep. > > This is interesting, Robi Sen's blog mentions the opposite and I think > B Purcell's does too > http://www.robisen.com/index.cfm?mode=entry&entry=FD4BE2FC-55DC-F2B1-FED0717 > CC1C7E0AF > we are using the 1.4.2 JVM shipping with CFMX. Interesting. > > > > > Your Heap size looks scarily low - if its too low you will begin to get a > > lot of GC's happening. For a 2GB box IO would set the heap to 1GB for Min > > and Max. One thing to note is that there is no ideal setting for the heap > - > > it is entirely based on how much memory your application uses - and each > > application is different. If you have not already I suggest you go and > run > > the Sun VisualGC application which is part of the JVMStat tool. > > > > http://java.sun.com/performance/jvmstat/ > > > > I am working on getting this tool to work. I'm not a JAR head yet :) > Hmm, I raised the heap to 1GB before and got the same issues. From > the metrics monitoring, free heap space rarely goes below 400MB. > Increasing the heap too big is also bad correct? Causing larger GCs? > What I can't figure oiut is why windows task manager says the JRun > instance is taking up more memory then it si though, see my first post > in this thread. > > > Your symptoms all point to the fact that your app is a memory hogger - > > probably due to largish queries (as MX uses around 4x more memory for a > > query than CF5 does) or indeed just extensive use of compels procedures. > > > > Yes, this is what I suspect too. One of the apps on this box is a > heavy query laden piece of work that I have fought with teh developer > on since inception. The developer sees no issue in a web page running > 30 to 60 seconds if that's what the user wants. geesh. I'm > segregating his app to a seperate instance soon. I expect some of > these issues to go away then. In fact, I can sometimes see a large > chunk, ~100MB, of free heap space get used at the same time this app > has a long running request. Seconds later teh heap space is returned > though, so it seems GC works. > > > I would redefined the following values (settings inline) > -XX:MaxPermSize=64m > > -XX:PermSize=48m -XX:NewSize=256m > > Ok, the MaxPermSize conflicts with this > www.macromedia.com/support/coldfusion/ts/documents/java_lang_outofmemory.htm > any reason to set it lower? What's your thoughts? Setting NewSize > this high reduces GC frequency, correct? > > > > > As for your XML settings - there a few things you can do. First off, what > > is your Simultaneous connections set to? Within your proxy service the > > activeHandlerThreads setting should be the same as your Simultaneous > > connections value. And within your Scheduler service (this is sometimes > > totally missing in CFMX) you should have > > > > <service class="jrunx.scheduler.SchedulerService" name="SchedulerService"> > > <attribute name="bindToJNDI">true</attribute> > > <attribute name="activeHandlerThreads">20</attribute> > > <attribute name="maxHandlerThreads">1000</attribute> > > <attribute name="minHandlerThreads">20</attribute> > > <attribute name="threadWaitTimeout">60</attribute> > > <attribute name="timeout">300</attribute> > > </service> > > > > Where min and activehandler threads are equal to the value set in > > simultaneous threads. > > yup, have these set, I read much about this. I failed to list them > before. I have no simultaneous thread setting in CFAdmin as we are > using the J2EE install on JRun. > > > > > Also consider that it is IIS which is causing the slowdown. We have had > > some serious issues with IIS5 which has the IIS Lockdown tool applied. > > Sometimes it would just hang the server - CF would look to be serving > pages > > OK and the IIS service is running but not pages get served....the only way > > to get them back (not always succcessful) is to perform an IISReset from > the > > command line. What to do is go download IISState tool here: > > > > http://www.iisfaq.com/Default.aspx?tabid=2513 > > > > When you are getting slow//blocked pages then run this and it will > generate > > a log which you can then review to see if any IIS processes are blocking > CF. > > You can also post it to the Microsoft Newsgroup to get an MVP input. > > > > ah, thanks, I never saw this tool before. I have been looking at > IISTracer too, a nice tool. We are thinking about buying seefusion > too (www.seefusion.com) > > > I see you are using large query sets. Are you running these via CFQUERY > or > > in an SP? I would suggest that any of these large result sets are set to > be > > dirty reads in order to stop blocking/locks. > > Yup, using cfquery for most everything. Kind of a pain to use an SP > to return a result set in Oracle/CF. > > > > > You may also want to consider creating separate CF Datasource connections > > (to the same Database) for different processes - i.e. if you have a report > > which runs ad-hoc and takes time you can increase performance and speed by > > running this in its own connection pool. > > > > We have about 15 apps on this box. Each app gets its own DSN. Except > we have 'maintain connections' off for every one at the moment, > thinking this hanging server issue was related to JDBC drivers. > > > Let me know how you get on, if you need to contact me offline you can via > > MSN...let me know if you want it offline. > > > > N > > > > > > -----Original Message----- > > From: Douglas Knudsen [mailto:[EMAIL PROTECTED] > > Sent: 28 February 2005 21:04 > > To: CF-Server > > Subject: Re: memory use and CF > > > > thankyou Neil! > > > > We are using win2K Server, IIS5, CFMX6.1 UPD 1, 4 Xeon P4s, and 2GB of RAM > > We use Oracle 9i and we use the JDBC drivers that come with CF > > I have this as teh JVM settings, > > java.args=-server -Xms640m -Xmx640m -Dsun.io.useCanonCaches=false > > > -Xbootclasspath/a:"{application.home}/servers/cfusion/cfusion-ear/cfusion-wa > > r/WEB-INF/cfusion/lib/webchartsJava2D.jar" > > -XX:MaxPermSize=128m -XX:PermSize=64m -XX:+UseConcMarkSweepGC > > -XX:NewSize=48m -DJINTEGRA_NATIVE_MODE -DJINTEGRA_PREFETCH_ENUMS > > > > I have metrics logging on, here is a sample > > > DateTime,,listenTh,delayTh,idleTh,busyTh,totalTh,delayRq,droppedRq,delayMs,T > > otalMem,FreeMem,J2EESessions > > 02/28 08:51:11,metrics ,2,0,7,4,13,0,0,0,655296,499693,1 > > 02/28 08:51:43,metrics ,2,0,9,2,13,0,0,0,655296,541514,1 > > 02/28 08:52:13,metrics ,2,0,11,0,13,0,0,0,655296,511963,1 > > 02/28 08:52:45,metrics ,11,0,5,2,18,0,0,0,655296,571183,1 > > When the server hangs, the metrics are similar, no indication of > > nastiness at all > > > > have some logging of GCs on, a sample here > > 02/28 15:35:36,debug MM GC collected 0 messages, 0 remaining. > > 02/28 15:43:36,debug MM GC collected 0 messages, 0 remaining. > > 02/28 15:51:36,debug MM GC collected 0 messages, 0 remaining. > > > > We have disabled 'maintain connections' for all DSNs thinking its > > related to JDBC stuffs. > > > > relevant jrun.xml settings > > <service class="jrun.servlet.jrpp.JRunProxyService" name="ProxyService"> > > <attribute name="activeHandlerThreads">25</attribute> > > <attribute name="backlog">500</attribute> > > <attribute name="deactivated">false</attribute> > > <attribute name="interface">*</attribute> > > <attribute name="maxHandlerThreads">1000</attribute> > > <attribute name="minHandlerThreads">20</attribute> > > <attribute name="port">51002</attribute> > > <attribute name="threadWaitTimeout">180</attribute> > > <attribute name="timeout">300</attribute> > > considering teh metrics logging, there does not appear anything wrong > > with these. > > > > At this poitn, the only thing I can see to do is stack/thread traces. > > But time constraints, I'm supposed to be developing!, are leading us > > to think of a schedule restart of the cf instance. > > > > We have one app that continually gets page request times in the 15 to > > 20 second range, but the metrics logging seems ok with that. We are > > planning on moving it to its own instance, thinking its the problem > > child. > > > > hope this helps some. > > > > Doug > > > > On Mon, 28 Feb 2005 19:59:01 -0000, Robertson-Ravo, Neil (RX) > > <[EMAIL PROTECTED]> wrote: > > > Sorry, coming in latee on this one...what are your Java settings (Heap > > Size, > > > Perm Size, New etc..). What are the specs of your server? This looks > > like > > > a GC issue. > > > > > > Have you ran any extensive monitoring? What SQL backend do you have? > What > > > drivers? > > > > > > I can help you on this issue as I have had 6-8 months of memory/cf > > debugging > > > under my belt - believe me, I have tweaked and set thousands of values > for > > a > > > decent performance setup. > > > > > > Give me as much info as you can on settings/configs etc..and we can > > > hopefully resolve this. > > > > > > Neil > > > Member of TMM CF > > > > > > > > > -----Original Message----- > > > From: Douglas Knudsen > > > To: CF-Server > > > Sent: 28/02/2005 19:10 > > > Subject: Re: memory use and CF > > > > > > thinking about this....we do use CFLDAP for all authentication. I > > > have read somewhere that this tag can be an issue. Anyone experience > > > this? > > > > > > On Mon, 28 Feb 2005 13:20:27 -0500, Douglas Knudsen > > > <[EMAIL PROTECTED]> wrote: > > > > nope, debugging is off in our production environments, something I > > > > speak LOUDLY about amongst my team. Thanks for the input. > > > > > > > > > > > > On Mon, 28 Feb 2005 10:06:26 -0800, Cameron Childress > > > > <[EMAIL PROTECTED]> wrote: > > > > > I've seen several reports lately that leaving Debugging turned on > > > will > > > > > also cause memory leaks in CF. Is it turned on on these boxes? > > > > > > > > > > -Cameron > > > > > > > > > > On Mon, 28 Feb 2005 08:42:08 -0500, Douglas Knudsen > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > Running CFMX6.1 UPD1 on JRun with J2EE install. > > > > > > What is the difference between memory use as seen in Windows Task > > > > > > Manager and memory use in the metrics logging via JRun? I got the > > > JVM > > > > > > memory set to 640m in the jvm.config file. According to metrics > > > > > > logging I see many entries like > > > > > > > > > > > > TotalMem,FreeMem > > > > > > 655296,545928 > > > > > > > > > > > > with the FreeMem rarely less than 400000. > > > > > > > > > > > > At the same time I see that according to task manager, the jrun > > > > > > instance is using about 750,000k. This figure slowy climbs from > > > about > > > > > > 350,000k when the JRun service is started. When it reachs a bit > > > over > > > > > > 800,000k the server starts to hang. > > > > > > > > > > > > Any thoughts? > > > > > > > > > > > > -- > > > > > > Douglas Knudsen > > > > > > http://www.cubicleman.com > > > > > > this is my signature, like it? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:10:5193 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/10 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:10 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.10 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
