[Dspace-tech] java.lang.OutOfMemoryError: PermGen space
Hi, We've experiencing severe failure on our production enviroment of Dspace. The system crashes and goes down constantly (every 2 or 3 hours approximately). The logs show this message: 14-Jan-2015 09:11:50.035 SEVERE [http-nio-80-exec-8] org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun java.lang.OutOfMemoryError: PermGen space The characteristics of our server are the following: Processor: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz RAM: 8 GB SO: CentOS Linux release 7.0.1406 Dspace version: 3.3 Apache Tomcat: 8.0.15 Java: 1.7.0_71 After several hours finding a solution, some changes to the JAVA_OPTS environment variable were made. This is how it is set right now: JAVA_OPTS="-Xmx4096M -Xms4096M -Xmn128m -Dfile.encoding=UTF-8 -XX:NewSize=128m -XX:MaxNewSize=128m -XX:PermSize=256m -XX:MaxPermSize=2048m -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:NewRatio=16 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/logs-dspace/tomcat -Dcom.sun.management.jmxremote" Is anyone have experienced this problem? And how can we resolve it? Thanks in advance for any help. Bender -- New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] java.lang.OutOfMemoryError: PermGen space
We have this documentation page [1] that explains JVM memory allocation, although not in much more detail than you already know. The important part is that you should start monitoring how your memory grows over time (two suggestions are psi probe and jconsole) and tweak the values in your JAVA_OPTS accordingly. Also don't forget that the memory assigned to Tomcat must not exceed total RAM available on the system (you don't want it to start swapping). Check how much memory you have available when Tomcat is off (because there are also other services on the system which require a small amount of memory). Keep some memory unallocated, otherwise your CLI jobs may start crashing - they have a default of 256 MB [2], but some tasks may need more. Ultimately, your periodic problem may be a scheduled job that fails to run because it doesn't have enough memory allocated. Or it runs and crashes Tomcat by starving it from memory. Check when your DSpace CLI tasks are scheduled to run. [1] https://wiki.duraspace.org/display/DSDOC5x/Performance+Tuning+DSpace [2] https://github.com/DSpace/DSpace/blob/dspace-4_x/dspace/bin/dspace#L75 Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] java.lang.OutOfMemoryError: PermGen space
On Wed, Jan 14, 2015 at 3:34 PM, bender wrote: > Besides the actions that you recommend to take, is there any possibility > that a software component in particular, like Java or Tomcat, could be the > origin of the problem? I wonder if the version of each software could > present some sort of incompatibility. > That's always possible, but it wouldn't be my first guess. After looking at the graphs you will see how each of the memory spaces grow, that should at the very least pinpoint the problem. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] java.lang.OutOfMemoryError: PermGen space
Helix, Thanks for your advice! I'll update any advance on solving this problem. Bender 2015-01-14 11:45 GMT-03:00 helix84 : > On Wed, Jan 14, 2015 at 3:34 PM, bender > wrote: > >> Besides the actions that you recommend to take, is there any possibility >> that a software component in particular, like Java or Tomcat, could be the >> origin of the problem? I wonder if the version of each software could >> present some sort of incompatibility. >> > > That's always possible, but it wouldn't be my first guess. After looking > at the graphs you will see how each of the memory spaces grow, that should > at the very least pinpoint the problem. > > > Regards, > ~~helix84 > > Compulsory reading: DSpace Mailing List Etiquette > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > > > > -- > New Year. New Location. New Benefits. New Data Center in Ashburn, VA. > GigeNET is offering a free month of service with a new server in Ashburn. > Choose from 2 high performing configs, both with 100TB of bandwidth. > Higher redundancy.Lower latency.Increased capacity.Completely compliant. > http://p.sf.net/sfu/gigenet > ___ > DSpace-tech mailing list > DSpace-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dspace-tech > List Etiquette: > https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette > -- New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] java.lang.OutOfMemoryError: PermGen space
Hi, Finally the solution to this problem has come with an update of Java. We've updated from Java 7 to 8. Other software versions and configurations remains the same. The system have been stable since last wednesday. Anyway, it still is kept under observation. Regards, Bender 2015-01-14 11:59 GMT-03:00 bender : > Helix, > > Thanks for your advice! > > I'll update any advance on solving this problem. > > Bender > > 2015-01-14 11:45 GMT-03:00 helix84 : > >> On Wed, Jan 14, 2015 at 3:34 PM, bender >> wrote: >> >>> Besides the actions that you recommend to take, is there any possibility >>> that a software component in particular, like Java or Tomcat, could be the >>> origin of the problem? I wonder if the version of each software could >>> present some sort of incompatibility. >>> >> >> That's always possible, but it wouldn't be my first guess. After looking >> at the graphs you will see how each of the memory spaces grow, that should >> at the very least pinpoint the problem. >> >> >> Regards, >> ~~helix84 >> >> Compulsory reading: DSpace Mailing List Etiquette >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> >> >> >> -- >> New Year. New Location. New Benefits. New Data Center in Ashburn, VA. >> GigeNET is offering a free month of service with a new server in Ashburn. >> Choose from 2 high performing configs, both with 100TB of bandwidth. >> Higher redundancy.Lower latency.Increased capacity.Completely compliant. >> http://p.sf.net/sfu/gigenet >> ___ >> DSpace-tech mailing list >> DSpace-tech@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dspace-tech >> List Etiquette: >> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette >> > > -- New Year. New Location. New Benefits. New Data Center in Ashburn, VA. GigeNET is offering a free month of service with a new server in Ashburn. Choose from 2 high performing configs, both with 100TB of bandwidth. Higher redundancy.Lower latency.Increased capacity.Completely compliant. http://p.sf.net/sfu/gigenet___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
