Re: [Dspace-tech] another unscheduled shutdown
Thank you to Dr.Andrea, we downgraded the jvm (to 6u45) and the tomcat have been running for almost a week without forced shutdown (previously 3-4 times a day). on a side note, aren't there suppose any documentation about jvm compatibility to each OS version? Best Regards. AdyWP On 07/03/2013 12:01 PM, Andrea Schweer wrote: Hi, On 03/07/13 14:20, Ady Wahyudi Paundu wrote: is it a known problem? because i've read several article about bugs on java 7, something about loops? so, did you downgrade your jvm? We downgraded the jvm; I haven't had time to do a root cause investigation. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
I,ve done several hardware test these couple of days using memtest86+ and`IBM DSA Memory stress test. IBM DSA Memory Stress Test gave me 'Passed' result, but memtest86+ showed errors. I don't know which one to believe, so i think i will try Dr.Andrea's option. Dr.Andrea, which jvm version now you are running? Regards, AdyWP On Wed, 03 Jul 2013 16:01:33 +1200, Andrea Schweer wrote: Hi, On 03/07/13 14:20, Ady Wahyudi Paundu wrote: is it a known problem? because i've read several article about bugs on java 7, something about loops? so, did you downgrade your jvm? We downgraded the jvm; I haven't had time to do a root cause investigation. cheers, Andrea Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
is it a known problem? because i've read several article about bugs on java 7, something about loops? so, did you downgrade your jvm? as for my problem, i still haven't pinpoint the source problem, but now i'm investigating the probability of hardware (RAM) problem Regards, AdyWP On 06/27/2013 10:56 AM, Andrea Schweer wrote: I've had problems with Java 7 on a RHEL box (RHEL 6.4), maybe that's it here too? cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Hi, On 03/07/13 14:20, Ady Wahyudi Paundu wrote: is it a known problem? because i've read several article about bugs on java 7, something about loops? so, did you downgrade your jvm? We downgraded the jvm; I haven't had time to do a root cause investigation. cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Hiy'all unfortunately my system still experiencing unscheduled tomcat shutdown (even after i increase the heap and permgen). this is the error message i got from catalina.out: # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2afaab76955d, pid=6604, tid=1081977152 # # JRE version: 7.0_21-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x69b55d] MachNode::in_RegMask(unsigned int) const+0x3d # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try ulimit -c unlimited before starting Java again # # An error report file with more information is saved as: # /home/dzpac3/hs_err_pid6604.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp i know this is JVM problem (and not dspace's), but i'm asking this question anyway: can anybody help me for a solution? TIA AdyWP On 06/13/2013 05:12 PM, helix84 wrote: There's really no hard science behind determining those numbers. For starters, you can increase them to 2x their current value. I recommend deploying the Psi probe [1] webapp and watching utilization of the different memory spaces over time to get some idea how much they need. You can find a bit of information on here, but no exact values: [2] After giving Tomcat more memory, you should notice significant improvement in page load times. Especially Solr is memory-hungry, so this will be most visible if you're using Discovery. [1] http://code.google.com/p/psi-probe/ [2] https://wiki.duraspace.org/display/DSDOC3x/Performance+Tuning+DSpace Regards, ~~helix84 Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Can you send the following system specs: 1. Server operating system. 2. Server hardware, with RAM and CPU details. 3. DSpace version and UI used. 4. Details of guides or links used to setup the IR. Cheers hg On 26 June 2013 12:04, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: Hiy'all unfortunately my system still experiencing unscheduled tomcat shutdown (even after i increase the heap and permgen). this is the error message i got from catalina.out: # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2afaab76955d, pid=6604, tid=1081977152 # # JRE version: 7.0_21-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x69b55d] MachNode::in_RegMask(unsigned int) const+0x3d # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try ulimit -c unlimited before starting Java again # # An error report file with more information is saved as: # /home/dzpac3/hs_err_pid6604.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp i know this is JVM problem (and not dspace's), but i'm asking this question anyway: can anybody help me for a solution? TIA AdyWP On 06/13/2013 05:12 PM, helix84 wrote: There's really no hard science behind determining those numbers. For starters, you can increase them to 2x their current value. I recommend deploying the Psi probe [1] webapp and watching utilization of the different memory spaces over time to get some idea how much they need. You can find a bit of information on here, but no exact values: [2] After giving Tomcat more memory, you should notice significant improvement in page load times. Especially Solr is memory-hungry, so this will be most visible if you're using Discovery. [1] http://code.google.com/p/psi-probe/ [2] https://wiki.duraspace.org/display/DSDOC3x/Performance+Tuning+DSpace Regards, ~~helix84 Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- *Hilton Gibson* Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
here is my machine info: - dual cpu (xeon 2.2G) IBM machine with 2x8 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - Current JAVA_OPTS=-Xmx8192M -Xms8192M -XX:MaxPermSize=1024M -XX:PermSize=1024M -XX:+UseParallelGC -XX:-UseLoopPredicate -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote=true - OS: RedHat Enterprise 5.6 64 bit - DSpace 3.1 Thanks AdyWP On 06/26/2013 06:36 PM, Hilton Gibson wrote: Can you send the following system specs: 1. Server operating system. 2. Server hardware, with RAM and CPU details. 3. DSpace version and UI used. 4. Details of guides or links used to setup the IR. Cheers hg On 26 June 2013 12:04, Ady Wahyudi Paundu ad...@unhas.ac.id mailto:ad...@unhas.ac.id wrote: Hiy'all unfortunately my system still experiencing unscheduled tomcat shutdown (even after i increase the heap and permgen). this is the error message i got from catalina.out: # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2afaab76955d, pid=6604, tid=1081977152 # # JRE version: 7.0_21-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x69b55d] MachNode::in_RegMask(unsigned int) const+0x3d # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try ulimit -c unlimited before starting Java again # # An error report file with more information is saved as: # /home/dzpac3/hs_err_pid6604.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp i know this is JVM problem (and not dspace's), but i'm asking this question anyway: can anybody help me for a solution? TIA AdyWP Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Ok. Have you tried this: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S08#Step_8.3_Increase_number_of_open_files_available Cheers hg On 26 June 2013 12:47, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: ** here is my machine info: - dual cpu (xeon 2.2G) IBM machine with 2x8 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - Current JAVA_OPTS=-Xmx8192M -Xms8192M -XX:MaxPermSize=1024M -XX:PermSize=1024M -XX:+UseParallelGC -XX:-UseLoopPredicate -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote=true - OS: RedHat Enterprise 5.6 64 bit - DSpace 3.1 Thanks AdyWP On 06/26/2013 06:36 PM, Hilton Gibson wrote: Can you send the following system specs: 1. Server operating system. 2. Server hardware, with RAM and CPU details. 3. DSpace version and UI used. 4. Details of guides or links used to setup the IR. Cheers hg On 26 June 2013 12:04, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: Hiy'all unfortunately my system still experiencing unscheduled tomcat shutdown (even after i increase the heap and permgen). this is the error message i got from catalina.out: # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x2afaab76955d, pid=6604, tid=1081977152 # # JRE version: 7.0_21-b11 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.21-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x69b55d] MachNode::in_RegMask(unsigned int) const+0x3d # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try ulimit -c unlimited before starting Java again # # An error report file with more information is saved as: # /home/dzpac3/hs_err_pid6604.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp i know this is JVM problem (and not dspace's), but i'm asking this question anyway: can anybody help me for a solution? TIA AdyWP -- *Hilton Gibson* Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Not yet, but i'm implementing it now... i'll let you know the result in the mean time, can you explain what it is for? TIA AdyWP On 06/26/2013 07:27 PM, Hilton Gibson wrote: Ok. Have you tried this: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S08#Step_8.3_Increase_number_of_open_files_available Cheers hg On 26 June 2013 12:47, Ady Wahyudi Paundu ad...@unhas.ac.id mailto:ad...@unhas.ac.id wrote: here is my machine info: - dual cpu (xeon 2.2G) IBM machine with 2x8 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - Current JAVA_OPTS=-Xmx8192M -Xms8192M -XX:MaxPermSize=1024M -XX:PermSize=1024M -XX:+UseParallelGC -XX:-UseLoopPredicate -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote=true - OS: RedHat Enterprise 5.6 64 bit - DSpace 3.1 Thanks AdyWP Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
It should allow you to get a proper java dump the next time. The dump should tell you what happened. I am not a java expert but it seems like the direction to go. Personally I have better results regularly using Ubuntu as a server. Perhaps you should reconsider Redhat EL, I believe it has very old versions of software. On 26 June 2013 13:25, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: ** Not yet, but i'm implementing it now... i'll let you know the result in the mean time, can you explain what it is for? TIA AdyWP On 06/26/2013 07:27 PM, Hilton Gibson wrote: Ok. Have you tried this: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S08#Step_8.3_Increase_number_of_open_files_available Cheers hg On 26 June 2013 12:47, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: here is my machine info: - dual cpu (xeon 2.2G) IBM machine with 2x8 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - Current JAVA_OPTS=-Xmx8192M -Xms8192M -XX:MaxPermSize=1024M -XX:PermSize=1024M -XX:+UseParallelGC -XX:-UseLoopPredicate -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote=true - OS: RedHat Enterprise 5.6 64 bit - DSpace 3.1 Thanks AdyWP -- *Hilton Gibson* Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Hi, I think you need more info before you do much more performance tuning, but, if it helps at all, here are the settings we are using in production with 32bit RHEL5: http://lso.umsystem.edu/~pottingerhj/article/25/dspace-3-tomcat-tweaks --Hardy Sent from my iPad On Jun 26, 2013, at 7:17 AM, Hilton Gibson hilton.gib...@gmail.commailto:hilton.gib...@gmail.com wrote: It should allow you to get a proper java dump the next time. The dump should tell you what happened. I am not a java expert but it seems like the direction to go. Personally I have better results regularly using Ubuntu as a server. Perhaps you should reconsider Redhat EL, I believe it has very old versions of software. On 26 June 2013 13:25, Ady Wahyudi Paundu ad...@unhas.ac.idmailto:ad...@unhas.ac.id wrote: Not yet, but i'm implementing it now... i'll let you know the result in the mean time, can you explain what it is for? TIA AdyWP On 06/26/2013 07:27 PM, Hilton Gibson wrote: Ok. Have you tried this: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S08#Step_8.3_Increase_number_of_open_files_available Cheers hg On 26 June 2013 12:47, Ady Wahyudi Paundu ad...@unhas.ac.idmailto:ad...@unhas.ac.id wrote: here is my machine info: - dual cpu (xeon 2.2G) IBM machine with 2x8 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - Current JAVA_OPTS=-Xmx8192M -Xms8192M -XX:MaxPermSize=1024M -XX:PermSize=1024M -XX:+UseParallelGC -XX:-UseLoopPredicate -Dfile.encoding=UTF-8 -Dcom.sun.management.jmxremote=true - OS: RedHat Enterprise 5.6 64 bit - DSpace 3.1 Thanks AdyWP -- Hilton Gibson Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- This SF.nethttp://SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.netmailto:DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
I've had problems with Java 7 on a RHEL box (RHEL 6.4), maybe that's it here too? cheers, Andrea -- Dr Andrea Schweer IRR Technical Specialist, ITS Information Systems The University of Waikato, Hamilton, New Zealand -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Dear Mr.Wood and Mr.Gibson, Thank you for the insight, our repo. system hasn't been crashed for a week now. I just follow Mr.Gibson suggestion to reduce log file (change all instances of INFO to ERROR) Best Regards, Ady W. Paundu -- View this message in context: http://dspace.2283337.n4.nabble.com/another-unscheduled-shutdown-tp4664613p4664850.html Sent from the DSpace - Tech mailing list archive at Nabble.com. -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
On Thu, Jun 13, 2013 at 10:02 AM, ady ad...@unhas.ac.id wrote: Dear Mr.Wood and Mr.Gibson, Thank you for the insight, our repo. system hasn't been crashed for a week now. I just follow Mr.Gibson suggestion to reduce log file (change all instances of INFO to ERROR) Please, note that by removing the INFO lines from log files, you'll be unable to generate some statistics later. However, Solr statistics will still work if you use them. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
ouch... but i don't know what else to do... here is my machine info: - dual processor IBM machine with 16 GB memory - work only for repository (plus postgredb) - with modjk to work with apache - using only xmlui-mirage - JAVA_OPTS=-Xmx1024M -Xms512M -XX:MaxPermSize=256M -Dfile.encoding=UTF-8 - OS: RedHat Enterprise 5.6 64 bit well... i think i will settle with this configuration for now until i found a better solution Thanks Helix Warm regards, Ady W. Paundu On 06/13/2013 04:12 PM, helix84 wrote: On Thu, Jun 13, 2013 at 10:02 AM, adyad...@unhas.ac.id wrote: Dear Mr.Wood and Mr.Gibson, Thank you for the insight, our repo. system hasn't been crashed for a week now. I just follow Mr.Gibson suggestion to reduce log file (change all instances of INFO to ERROR) Please, note that by removing the INFO lines from log files, you'll be unable to generate some statistics later. However, Solr statistics will still work if you use them. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
On Thu, Jun 13, 2013 at 10:23 AM, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: - dual processor IBM machine with 16 GB memory - JAVA_OPTS=-Xmx1024M -Xms512M -XX:MaxPermSize=256M -Dfile.encoding=UTF-8 Since you have 16 GB of memory, you may definitely want to assign majority of it to Tomcat. You're giving it less than 2 GBs. I recommend giving it about 14 GB, the rest should be more than enough for postgres, apache, disk cache and any other processes. If disk space for log files is a problem, you may want to compress them or move them to another machine. Most syslog implementations allow you to do both. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Oh.. can I? lol.. all the reference I read never mention anything above 2G :) how about the other two? (Xms and MaxPermSize) Regards, Ady W. Paundu On 06/13/2013 04:47 PM, helix84 wrote: On Thu, Jun 13, 2013 at 10:23 AM, Ady Wahyudi Paunduad...@unhas.ac.id wrote: - dual processor IBM machine with 16 GB memory - JAVA_OPTS=-Xmx1024M -Xms512M -XX:MaxPermSize=256M -Dfile.encoding=UTF-8 Since you have 16 GB of memory, you may definitely want to assign majority of it to Tomcat. You're giving it less than 2 GBs. I recommend giving it about 14 GB, the rest should be more than enough for postgres, apache, disk cache and any other processes. If disk space for log files is a problem, you may want to compress them or move them to another machine. Most syslog implementations allow you to do both. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
We use 8GB. See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S08#Java_environment_settings_used_for_SUNScholar On 13 June 2013 10:46, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: Oh.. can I? lol.. all the reference I read never mention anything above 2G :) how about the other two? (Xms and MaxPermSize) Regards, Ady W. Paundu On 06/13/2013 04:47 PM, helix84 wrote: On Thu, Jun 13, 2013 at 10:23 AM, Ady Wahyudi Paunduad...@unhas.ac.id wrote: - dual processor IBM machine with 16 GB memory - JAVA_OPTS=-Xmx1024M -Xms512M -XX:MaxPermSize=256M -Dfile.encoding=UTF-8 Since you have 16 GB of memory, you may definitely want to assign majority of it to Tomcat. You're giving it less than 2 GBs. I recommend giving it about 14 GB, the rest should be more than enough for postgres, apache, disk cache and any other processes. If disk space for log files is a problem, you may want to compress them or move them to another machine. Most syslog implementations allow you to do both. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette Powered by PTIK UNHAS ~ http://unhas.ac.id/ -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- *Hilton Gibson* Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
On Wed, Jun 5, 2013 at 7:18 AM, Ady Wahyudi Paundu ad...@unhas.ac.id wrote: Today i had another automatic shutdown. This is the log last lines taken from catalina, dspace and cocoon: Sorry, there is nothing in these logs that indicates what happened. It never happened to me that Tomcat shut down spontaneously, although I have plenty of experience with other cases where it could not respond to queries. I can give you only general recommendations - check your diskspace on all used partitions, especially where you store your log files; check memory usage for each individual JVM memory spaces/generations and see if any of them fills up after prolonged use; check file system permissions - it's easiest to run dspace under the same user as you run tomcat. Regards, ~~helix84 Compulsory reading: DSpace Mailing List Etiquette https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
On the assumption that the affected site runs on Linux, one thing to check is the OOM Killer. When all memory and swap space is committed and the kernel is desperate to free some memory, it will pick a large process and terminate it to reclaim its memory. Tomcat tends to be large and in many ways look like a good candidate for reclamation. When the OutOfMemory Killer strikes, the process has no warning and its logs won't show anything. I don't recall whether the kernel logs such events. You can get some useful numbers from 'top'. If this is happening, then you need one or more of several things: o reduce Tomcat's memory demand significantly. There is a lot of advice out there about sizing Tomcat's memory, some of it better than others. If Tomcat's heap bloats up to tremendous size and then sheds much committed space during infrequent garbage collections, try reducing the heap size until you get small garbage collections at intervals of a few minutes. Any more heap than that is probably unnecessary, unless your load is *very* bursty. Small GCs are cheap and we can afford to have them frequently. I try to keep the heap startup size slightly larger than the steady-state committed size and the heap maximum about twice that. A monitor such as Psi Probe helps to get these numbers, and is good to glance at regularly to understand loads and trends. o migrate some other large services to some other machine, to free up memory. o install more memory. You can temporarily alleviate the problem with more swap space, but swapping is slow. Normally the swap space should be very little used. Add memory until every essential process fits, you have a comfortable margin in free space and I/O buffers, and swap space is mostly or entirely unused. Other things to look for would be hardware errors that destroy the process somehow. Is the kernel logging oopses, severe I/O errors, or the like? You might have memory or interconnect issues. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. signature.asc Description: Digital signature -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
Re: [Dspace-tech] another unscheduled shutdown
Here are some of the optimisations I did on an Ubuntu 10.04 LTS server: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Optimisations Hope it helps. On 5 June 2013 15:04, Mark H. Wood mw...@iupui.edu wrote: On the assumption that the affected site runs on Linux, one thing to check is the OOM Killer. When all memory and swap space is committed and the kernel is desperate to free some memory, it will pick a large process and terminate it to reclaim its memory. Tomcat tends to be large and in many ways look like a good candidate for reclamation. When the OutOfMemory Killer strikes, the process has no warning and its logs won't show anything. I don't recall whether the kernel logs such events. You can get some useful numbers from 'top'. If this is happening, then you need one or more of several things: o reduce Tomcat's memory demand significantly. There is a lot of advice out there about sizing Tomcat's memory, some of it better than others. If Tomcat's heap bloats up to tremendous size and then sheds much committed space during infrequent garbage collections, try reducing the heap size until you get small garbage collections at intervals of a few minutes. Any more heap than that is probably unnecessary, unless your load is *very* bursty. Small GCs are cheap and we can afford to have them frequently. I try to keep the heap startup size slightly larger than the steady-state committed size and the heap maximum about twice that. A monitor such as Psi Probe helps to get these numbers, and is good to glance at regularly to understand loads and trends. o migrate some other large services to some other machine, to free up memory. o install more memory. You can temporarily alleviate the problem with more swap space, but swapping is slow. Normally the swap space should be very little used. Add memory until every essential process fits, you have a comfortable margin in free space and I/O buffers, and swap space is mostly or entirely unused. Other things to look for would be hardware errors that destroy the process somehow. Is the kernel logging oopses, severe I/O errors, or the like? You might have memory or interconnect issues. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Machines should not be friendly. Machines should be obedient. -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette -- *Hilton Gibson* Linux Systems Administrator JS Gericke Library Room 1025C Stellenbosch University Private Bag X5036 Stellenbosch 7599 South Africa Tel: +27 21 808 4100 | Cell: +27 84 646 4758 http://library.sun.ac.za http://scholar.sun.ac.za http://www.journals.ac.za -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech List Etiquette: https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette