Sorry for taking time to reply Chris. Are you periodically hot-re-deploying your application in production? If so, you may want to stop doing that, as it appears that you have a ClassLoader-pinning leak in your application or some dependent library
I do not do hot deployment in production.But somehow I feel its leaking memory.There are some threads which for some reasons remains hanging. But I think I will focus on it later on. I do have copy of your presentation.I will go through it and may be post couple of question if I have. You should be able to run a reasonable service in half a gig of heap space. Just be aware that caching information in your application is going to significantly increase the amount of heap space required by your application in a steady-state. Yup I am aware of this limitation. I know cache is often stored in local heap and probably that's reason it consumes heap space. Currently I am checking if ehcache or jcs meets my need. Ehcache for some reasons fills my heap very fast.I tried storing some 100 json strings and retrieving them. My OOM's started once I integrated this library in my web inf /lib folder.After I removed it, frequency of OOM has substantially decreased.Probably I am not using it correctly somewhere. Do you know what open source caching people frequently use for java web apps ? Fetching a lot of data isn't usually a big deal. Just make sure that your "fetch size" is set to something reasonable. There are some JDBC drivers (MySQL Connector/J in particular) that will load the entire ResultSet into memory before allowing you to traverse it unless you make arrangements to limit that memory usage. But bringing-back thousands of records from a db isn't a problem -- unless you don't *have to* in which case you might want to optimize your queries to avoid pulling data you don't actually need. But too-many-records would be a performance problem, not a memory proble m. haha.Its fetch size which my autocomplete was missing...I am not sure how I missed it but it was missing. Thanks Mark. On Tue, Jun 30, 2015 at 1:17 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Kiran, > > On 6/29/15 10:21 AM, Kiran Badi wrote: > > Christopher Schultz wrote: > >> The number of users shouldn't impact your PermGen space. Perhaps > >> only once you get to that stage are you hitting enough of your > >> features to load classes into PermGen. (Or maybe you are using > >> String.intern a lot...) > > > > I analysed some logs and I could see that users query features > > which makes DB calls, so those calls do have 1000's of rows in > > it.But some calls also fetch empty result set and some error out, > > partly because code for those calls are broken( Some of those dao > > classes have hard coded DB parameter which I am cleaning it out > > now). As far as I know I do not do any string cancat, those calls > > are all simple list fetch calls to views. > > > > I am trying to implement some caching using either ehcache or JCS > > but I think it has to wait for some time, till I gain some > > understanding on how these works.( I think I need to serialize lot > > of model classes for that probably will require some code changes > > again).i know I have lot of work to do ,maybe I one at a time > > change :) > > > >> PermGen failures will effect the whole JVM. There is no way to > >> protect App B from App A unless they are in different JVMs. > > > > I can understand this. so doing daily restart now to manage issue > > till I figure out some solution to it. > > > >> What makes you say that? It seems that you have more information > >> than you are giving us. > > > > Its not hardened code so I think it still has some issues with it. > > Also during development I can see similar errors on local dev box, > > If I do deploy and redeploy at least 8 to 10 times, I start seeing > > those perm gen errors,its just that it references a new class > > file every time,maybe I can share it with you all once I get it > > again. > > Stop right there. > > Are you periodically hot-re-deploying your application in production? > If so, you may want to stop doing that, as it appears that you have a > ClassLoader-pinning leak in your application or some dependent library. > > See this presentation for more information: > http://people.apache.org/~markt/presentations/2010-11-04-Memory-Leaks-60 > mins.pdf > > > Also it's I have written this code and I am not that fantastic > > coder yet, but I will reach there short span :) > > > > > >> Usually, PermGen doesn't have to be enormous. What's your memory > >> cap with your hosting provider? > > > > I have private tomcat 7x but I remember hosting provider > > mentioning that 512mb is final,but I will check with them again > > later this week. > > You should be able to run a reasonable service in half a gig of heap > space. Just be aware that caching information in your application is > going to significantly increase the amount of heap space required by > your application in a steady-state. > > > Below is what I see in catalina logs when I do restart of tomcat, > > > > Picked up _JAVA_OPTIONS: -Xms20m -Xmx128m -XX:MinHeapFreeRatio=20 > > -XX:MaxHeapFreeRatio=40 -XX:NewSize=10m -XX:MaxNewSize=10m > > -XX:SurvivorRatio=6 -XX:TargetSurvivorRatio=80 > > -XX:+CMSClassUnloadingEnabled -XX:+CMSClassUnloadingEnabled > > So you have a 128MiB max heap space, but you aren't specifying a > PermGen size. For some frameworks (particularly Spring), you'll need > to increase the size of the PermGen heap space because it really just > does need to load 3e8 classes or whatever. > > Or, you could upgrade to Java 8 which does not have a PermGen space. > You'll just eventually run out of "regular" heap space in that case, > if you really do have a memory leak. > > > I think was thinking CMSClassUnloadingEnabled should fix my perm > > gen issues, but I think its not the case. > > Class unloading should pretty much always be enabled, so that setting > likely does nothing for you. It doesn't look like you are enabling the > CMS GC, so those settings probably don't make a bit of difference. > > >> You either need more PermGen space, or you need to locate some > >> kind of leak in your application and fix it. IIRC, there are > >> some RMI-related leaks and Proxy-related leaks in PermGen > >> depending upon your exact circumstances. > >> > >> It would be good to know what's in PermGen when it hits its > >> limit. > >> > >> What are your current heap settings, including PermGen? What > >> JVM? > >> > >> Try: $ jmap -heap <pid> > >> > >> and > >> > >> $ jmap -permstat <pid> > > > > I will try to get those dumps but I do not use any RMI or generate > > some kind of proxies . Mine is simple app with lot of forms in it. > > Though there are few calls which fetches lot of data from > > servers.Sometimes few autocomplete calls fetch 1000's of records.I > > am trying to remove those calls. > > Fetching a lot of data isn't usually a big deal. Just make sure that > your "fetch size" is set to something reasonable. There are some JDBC > drivers (MySQL Connector/J in particular) that will load the entire > ResultSet into memory before allowing you to traverse it unless you > make arrangements to limit that memory usage. > > But bringing-back thousands of records from a db isn't a problem -- > unless you don't *have to* in which case you might want to optimize > your queries to avoid pulling data you don't actually need. > > But too-many-records would be a performance problem, not a memory proble > m. > > > Below is my jvm details > > > > Apache Tomcat/7.0.50 1.7.0_17-b02 Oracle Corporation Linux > > 2.6.32-531.29.2.lve1.3.11.1.el6.x86_64 amd64 > > > > Thanks Chris for reply. > > Hope that helps, > - -chris > -----BEGIN PGP SIGNATURE----- > Comment: GPGTools - http://gpgtools.org > > iQIcBAEBCAAGBQJVks88AAoJEBzwKT+lPKRYOh8P/2DwjRn0hH/R3Xsa0Sf+FFKh > 5/dzcnRf7C1PvCVQ8IrBTIsKuMCBY6dbS9PxJe7+hWo/UIaScF62B6S4xwd4WCsa > 5ojVDKslt/BcNGk/6bXyK2uDFXJRFB3j3654XgsU9/T+GvNOF9F6rNQYnRDZVuuv > UMbphYOEHfFBplIEJnu5Q6SiLD219aKLa3Xmbl1+WHn1tQyoPivFmdBBkAkUO+92 > WN6vwRKiI0VgKZ9mbH2pgI1BVq3qDxKksPx8nv1ju3nDb5MRCWgbNGeJFhl87B5b > 78XmiNdmYQA4K1PSk4tGXergTDpgJ4PtYhBSYvZC8xqFQnEaiUWtqFwKCqAyFL+S > FX1Tef/Rc/20XYnxqUm9Wi5XX2Nq2KCBxs/ob7DdwC52TyB9LQDNP7eUvYPZQv13 > cuEW+eDozfYaao9Io+rparQ3Wz95y/1bwgEP5BjxK0b5rqDuDn84HsC6FrSLBPJc > qp1q6vs4JYB3jIwdNKN+JbGoPTPYfwcI2ESuviVbAAKO+nYQKj0lIgERQBp2UDCN > RSVSlXW64W//LFuDZzfN+v099vj0BOh1EEWoM+lXDeT9PSpam+gEcaze/gTK5L06 > 0x/0gXgiQGNNgDFmP6ovTpcBA59IA6o8cfYTEbovnJnBr/Bx10pET9D8/QR515od > XJyzmMIXwXltaOzdAiEe > =p9EI > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
