[jira] [Commented] (DERBY-7086) Derby opera DB growing fast

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17247056#comment-17247056
 ] 

Vamsavardhana Reddy commented on DERBY-7086:


Richard,

thank you for your response.  Our "appliance" restart actually kills the derby 
server JVM.  Let me see if a graceful shutdown can be performed.

The doc link you provided about storage and recovery talks about periodic 
checkpoints to keep the log size small.  Is there a way to configure a time 
interval for it?

 

Thanks,

Vamsi

> Derby opera DB growing fast
> ---
>
> Key: DERBY-7086
> URL: https://issues.apache.org/jira/browse/DERBY-7086
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.8.1.2, 10.8.2.2, 10.8.3.0
> Environment: Derby on CentOS8
>Reporter: Gobi
>Priority: Major
>
> Hi Team,
>  
> Greetings to you!
>  
> We use Derby database version 10.8
>  
> Our observation is that there is not much table data but the disk size is too 
> large.  Large space occupied by a /log folder .  We would like to check with 
> Derby team what this could be holding.
>  
> We replaced the customer Opera DB, Our analysis given below which is before 
> and after replacement.
>  BEFORE opening in ij.sh
>  [g0-7540c/Standalone od-issue]# du -h var
>  48K 
> var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC/B3C4E02E5379FEBA0FBB40DB4FE840C0
>  52K var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC
>  4.0K    var/db/opera/files/AFA9712BE7E8196E70F62B08ABDE44B9
>  4.0K    var/db/opera/files/9FDC6EC9F5ECC99A86E8B36E23BA42A8
>  4.0K    var/db/opera/files/9876690C43C63886472E8093CC5040F6
>  4.0K    var/db/opera/files/94D70757CE7B1A374A8D2F05178E48DF
>  40K 
> var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C/A3D7B6B3515E1E97E58CDE057B664EBF
>  44K var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C
>  4.0K    var/db/opera/files/94C9A9659E044849CE9CA52E58E74E0D
>  384K    
> var/db/opera/files/A0A1F400944A760F0345B18540CE/AF0E0325DECF1EB322DECCA2CC9A4D2E
>  388K    var/db/opera/files/A0A1F400944A760F0345B18540CE
>  40K 
> var/db/opera/files/92AFC3101442985C97DB95407FD34F1E/8AAA75548D79A21EC53680FBB23443FB
>  44K var/db/opera/files/92AFC3101442985C97DB95407FD34F1E
>  552K    var/db/opera/files
>  4.0K    var/db/opera/derby/tmp
>  11M var/db/opera/derby/seg0
>  626M    var/db/opera/derby/log
>  636M    var/db/opera/derby
>  637M    var/db/opera
>  637M    var/db
>  637M    var
>  
>  [g0-7540c/Standalone od-issue]# cd var
>  [g0-7540c/Standalone var]# cd db
>  
>  [g0-7540c/Standalone db]# ij.sh
>  ij version 10.8
>  ij> connect 'jdbc:derby:opera/derby';
>  ij> quit;
>  [g0-7540c/Standalone db]# cd ../..
>  
>  AFTER
>  [g0-7540c/Standalone od-issue]# du -h var
>  48K 
> var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC/B3C4E02E5379FEBA0FBB40DB4FE840C0
>  52K var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC
>  4.0K    var/db/opera/files/AFA9712BE7E8196E70F62B08ABDE44B9
>  4.0K    var/db/opera/files/9FDC6EC9F5ECC99A86E8B36E23BA42A8
>  4.0K    var/db/opera/files/9876690C43C63886472E8093CC5040F6
>  4.0K    var/db/opera/files/94D70757CE7B1A374A8D2F05178E48DF
>  40K 
> var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C/A3D7B6B3515E1E97E58CDE057B664EBF
>  44K var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C
>  4.0K    var/db/opera/files/94C9A9659E044849CE9CA52E58E74E0D
>  384K    
> var/db/opera/files/A0A1F400944A760F0345B18540CE/AF0E0325DECF1EB322DECCA2CC9A4D2E
>  388K    var/db/opera/files/A0A1F400944A760F0345B18540CE
>  40K 
> var/db/opera/files/92AFC3101442985C97DB95407FD34F1E/8AAA75548D79A21EC53680FBB23443FB
>  44K var/db/opera/files/92AFC3101442985C97DB95407FD34F1E
>  552K    var/db/opera/files
>  11M var/db/opera/derby/seg0
>  1.1M    var/db/opera/derby/log
>  12M var/db/opera/derby
>  12M var/db/opera
>  12M var/db
>  12M var
>  
>  [g0-7540c/Standalone db]# ij.sh
>  ij version 10.8
>  ij> connect 'jdbc:derby:opera/derby';
>  ij> show tables;
>  TABLE_SCHEM |TABLE_NAME    |REMARKS
>  
>  SYS |SYSALIASES    |
>  SYS |SYSCHECKS |
>  SYS |SYSCOLPERMS   |
>  SYS |SYSCOLUMNS    |
>  SYS |SYSCONGLOMERATES  |
>  SYS |SYSCONSTRAINTS    |
>  SYS |SYSDEPENDS    |
>  SYS |SYSFILES  |
>  SYS |SYSFOREIGNKEYS    |
>  SYS |SYSKEYS   |
>  SYS |SYSPERMS  |
>  SYS |SYS

[jira] [Commented] (DERBY-7086) Derby opera DB growing fast

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245662#comment-17245662
 ] 

Vamsavardhana Reddy commented on DERBY-7086:


Any suggestions on how we can avoid the on disk growth of the database or any 
known issues that have been fixed in derby v10.8.3 ?

> Derby opera DB growing fast
> ---
>
> Key: DERBY-7086
> URL: https://issues.apache.org/jira/browse/DERBY-7086
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.8.1.2, 10.8.2.2, 10.8.3.0
> Environment: Derby on CentOS8
>Reporter: Gobi
>Priority: Major
>
> Hi Team,
>  
> Greetings to you!
>  
> We use Derby database version 10.8
>  
> Our observation is that there is not much table data but the disk size is too 
> large.  Large space occupied by a /log folder .  We would like to check with 
> Derby team what this could be holding.
>  
> We replaced the customer Opera DB, Our analysis given below which is before 
> and after replacement.
>  BEFORE opening in ij.sh
>  [g0-7540c/Standalone od-issue]# du -h var
>  48K 
> var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC/B3C4E02E5379FEBA0FBB40DB4FE840C0
>  52K var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC
>  4.0K    var/db/opera/files/AFA9712BE7E8196E70F62B08ABDE44B9
>  4.0K    var/db/opera/files/9FDC6EC9F5ECC99A86E8B36E23BA42A8
>  4.0K    var/db/opera/files/9876690C43C63886472E8093CC5040F6
>  4.0K    var/db/opera/files/94D70757CE7B1A374A8D2F05178E48DF
>  40K 
> var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C/A3D7B6B3515E1E97E58CDE057B664EBF
>  44K var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C
>  4.0K    var/db/opera/files/94C9A9659E044849CE9CA52E58E74E0D
>  384K    
> var/db/opera/files/A0A1F400944A760F0345B18540CE/AF0E0325DECF1EB322DECCA2CC9A4D2E
>  388K    var/db/opera/files/A0A1F400944A760F0345B18540CE
>  40K 
> var/db/opera/files/92AFC3101442985C97DB95407FD34F1E/8AAA75548D79A21EC53680FBB23443FB
>  44K var/db/opera/files/92AFC3101442985C97DB95407FD34F1E
>  552K    var/db/opera/files
>  4.0K    var/db/opera/derby/tmp
>  11M var/db/opera/derby/seg0
>  626M    var/db/opera/derby/log
>  636M    var/db/opera/derby
>  637M    var/db/opera
>  637M    var/db
>  637M    var
>  
>  [g0-7540c/Standalone od-issue]# cd var
>  [g0-7540c/Standalone var]# cd db
>  
>  [g0-7540c/Standalone db]# ij.sh
>  ij version 10.8
>  ij> connect 'jdbc:derby:opera/derby';
>  ij> quit;
>  [g0-7540c/Standalone db]# cd ../..
>  
>  AFTER
>  [g0-7540c/Standalone od-issue]# du -h var
>  48K 
> var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC/B3C4E02E5379FEBA0FBB40DB4FE840C0
>  52K var/db/opera/files/B88A9E5185C464BF6DEC15CEE45E40AC
>  4.0K    var/db/opera/files/AFA9712BE7E8196E70F62B08ABDE44B9
>  4.0K    var/db/opera/files/9FDC6EC9F5ECC99A86E8B36E23BA42A8
>  4.0K    var/db/opera/files/9876690C43C63886472E8093CC5040F6
>  4.0K    var/db/opera/files/94D70757CE7B1A374A8D2F05178E48DF
>  40K 
> var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C/A3D7B6B3515E1E97E58CDE057B664EBF
>  44K var/db/opera/files/BDE828C23CFC92DCD76249734B9B424C
>  4.0K    var/db/opera/files/94C9A9659E044849CE9CA52E58E74E0D
>  384K    
> var/db/opera/files/A0A1F400944A760F0345B18540CE/AF0E0325DECF1EB322DECCA2CC9A4D2E
>  388K    var/db/opera/files/A0A1F400944A760F0345B18540CE
>  40K 
> var/db/opera/files/92AFC3101442985C97DB95407FD34F1E/8AAA75548D79A21EC53680FBB23443FB
>  44K var/db/opera/files/92AFC3101442985C97DB95407FD34F1E
>  552K    var/db/opera/files
>  11M var/db/opera/derby/seg0
>  1.1M    var/db/opera/derby/log
>  12M var/db/opera/derby
>  12M var/db/opera
>  12M var/db
>  12M var
>  
>  [g0-7540c/Standalone db]# ij.sh
>  ij version 10.8
>  ij> connect 'jdbc:derby:opera/derby';
>  ij> show tables;
>  TABLE_SCHEM |TABLE_NAME    |REMARKS
>  
>  SYS |SYSALIASES    |
>  SYS |SYSCHECKS |
>  SYS |SYSCOLPERMS   |
>  SYS |SYSCOLUMNS    |
>  SYS |SYSCONGLOMERATES  |
>  SYS |SYSCONSTRAINTS    |
>  SYS |SYSDEPENDS    |
>  SYS |SYSFILES  |
>  SYS |SYSFOREIGNKEYS    |
>  SYS |SYSKEYS   |
>  SYS |SYSPERMS  |
>  SYS |SYSROLES  |
>  SYS |SYSROUTINEPERMS   |
>  SYS |SYSSCHEMAS    |
>  SYS |SYSSEQUENCES  |
>  SYS 

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091390#comment-17091390
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


Thanks Richard.  Will try the debug server and post the findings.

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: image-101420-1.png, image-101420-2.png, 
> image-101420-3.png, image-10833-1.png, image-10833-2.png, 
> javaheapdump-oom-derby-101420.zip, javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derbyserver.log 
> -Djava.rmi.serv

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090525#comment-17090525
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


I assume the option for running in debug is the system property  
{{-Dderby.drda.debug=true ?}}

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: image-101420-1.png, image-101420-2.png, 
> image-101420-3.png, image-10833-1.png, image-10833-2.png, 
> javaheapdump-oom-derby-101420.zip, javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derb

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089697#comment-17089697
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


Hi Richard,

I see in our derby.properties that derby.storage.sortBufferMax is set 10240 
(10MB) and to address a specific customer issue!!  Not sure if we can change it 
to 512 as you suggested.  To better understand, can you tell how this value is 
used?  Is it like (a) memory is allocated in chunks for this size multiple 
times to process a query or (b) is it the maximum memory allocated to process 
the query?

 

Thank you

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: image-101420-1.png, image-101420-2.png, 
> image-101420-3.png, image-10833-1.png, image-10833-2.png, 
> javaheapdump-oom-derby-101420.zip, javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 

[jira] [Comment Edited] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088261#comment-17088261
 ] 

Vamsavardhana Reddy edited comment on DERBY-7075 at 4/21/20, 3:59 AM:
--

Hi Richard, thank you very much for your help on this JIRA.  I will try to put 
as much information as possible here to assist you in the investigation.  I 
have uploaded some screenshots.  Not sure if it would be possible for you to 
get into a screenshare to speed things up.  Do let me know.

 

Here is the analysis of heap and javacore from the 10.8.3.3 OOM scenario.



There are two heapdumps  both show same objects on heap (leak suspects)
 Dump Event "systhrow" (0004) Detail "java/lang/OutOfMemoryError" "Java 
heap space" received
 Date: 2020/04/16 at 14:42:23:368
 JRE 1.8.0 Linux amd64-64 (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
 -Xgcpolicy:gencon
 -Xmn192m
 -Xms512m
 -Xmx512m
 -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000
 Total memory free: 5,500,848 (0x0053EFB0) = 5M
 Total memory in use: 531,370,064 (0x1FAC1050) = 507M
 From heapdump review I see below objects (leak suspects)
 1)
 "org.apache.derby.impl.store.access.RAMTransaction" 97,807,424 (22.04%) bytes 
<-- 93.3MB The memory is accumulated in one instance of 
"org.apache.derby.impl.store.access.sort.SortBuffer"
 2)
  "org.apache.derby.impl.store.access.RAMTransaction" 94,323,360 (21.25%) bytes 
<-- 90MB The memory is accumulated in one instance of 
"org.apache.derby.impl.store.access.sort.SortBuffer"
 3) "org.apache.derby.impl.services.cache.ConcurrentCache" (12.48%) bytes <-- 
52.8MB.
 The memory is accumulated in one instance of 
"org.apache.derby.impl.services.cache.ConcurrentCache"
 4)
 org.apache.derby.impl.services.cache.ConcurrentCache" 54,617,784 (12.31%) 
bytes <--52.1MB The memory is accumulated in one instance of 
"org.apache.derby.impl.services.cache.ConcurrentCache"First two leak suspects 
are coming from below current thread in Javacore. "DRDAConnThread_37" 
J9VMThread:0x030C5200, omrthread_t:0x7F97301F7830, 
java/lang/Thread:0xE101CE70, state:R, prio=5
 (java/lang/Thread getId:0x8B, isDaemon:true)
  Java callstack:
 at org/apache/derby/iapi/types/SQLChar.getString(Bytecode PC:18(Compiled Code))
 at org/apache/derby/iapi/types/SQLVarchar.cloneValue(Bytecode PC:5(Compiled 
Code))
 at org/apache/derby/impl/sql/execute/BasicSortObserver.getClone(Bytecode 
PC:22(Compiled Code))
 at 
org/apache/derby/impl/sql/execute/BasicSortObserver.insertNonDuplicateKey(Bytecode
 PC:9(Compiled Code))
 at org/apache/derby/impl/store/access/sort/SortBuffer.insert(Bytecode 
PC:331(Compiled Code))
 at org/apache/derby/impl/store/access/sort/MergeInserter.insert(Bytecode 
PC:13(Compiled Code))
 at org/apache/derby/impl/sql/execute/SortResultSet.loadSorter(Bytecode 
PC:112(Compiled Code))
 at org/apache/derby/impl/sql/execute/SortResultSet.openCore(Bytecode 
PC:90(Compiled Code))
 at 
org/apache/derby/impl/sql/execute/ScrollInsensitiveResultSet.openCore(Bytecode 
PC:14(Compiled Code))
 at org/apache/derby/impl/sql/execute/BasicNoPutResultSetImpl.open(Bytecode 
PC:10(Compiled Code))
 at org/apache/derby/impl/sql/GenericPreparedStatement.executeStmt(Bytecode 
PC:357(Compiled Code))
 at org/apache/derby/impl/sql/GenericPreparedStatement.execute(Bytecode 
PC:5(Compiled Code))
 at org/apache/derby/impl/jdbc/EmbedStatement.executeStatement(Bytecode 
PC:136(Compiled Code))
 (entered lock: org/apache/derby/impl/jdbc/EmbedConnection@0xE6723348, 
entry count: 1)
 at org/apache/derby/impl/jdbc/EmbedPreparedStatement.executeStatement(Bytecode 
PC:16(Compiled Code))
 at org/apache/derby/impl/jdbc/EmbedPreparedStatement.execute(Bytecode 
PC:9(Compiled Code))
 at org/apache/derby/impl/drda/DRDAStatement.execute(Bytecode PC:6(Compiled 
Code))
 at org/apache/derby/impl/drda/DRDAConnThread.processCommands(Bytecode 
PC:611(Compiled Code))
 at org/apache/derby/impl/drda/DRDAConnThread.run(Bytecode PC:115)


was (Author: vamsic):
Hi Richard, thank you very much for your help on this JIRA.  I will try to put 
as much information as possible here to assist you in the investigation.

 

Here is the analysis of heap and javacore from the 10.8.3.3 OOM scenario.



There are two heapdumps  both show same objects on heap (leak suspects)
Dump Event "systhrow" (0004) Detail "java/lang/OutOfMemoryError" "Java heap 
space" received
Date: 2020/04/16 at 14:42:23:368
JRE 1.8.0 Linux amd64-64 (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
-Xgcpolicy:gencon
-Xmn192m
-Xms512m
-Xmx512m
-Xverbosegclog:/var/log/derbyserver_gc.log,10,1000
Total memory free: 5,500,848 (0x0053EFB0) = 5M
Total memory in use: 531,370,064 (0x1FAC1050) = 507M
>From heapdump review I see below objects (leak suspects)
1)
"org.apache.derby.impl.store.access.RAMTransaction" 97,807,424 (22.04%) bytes 
<-- 93.3

[jira] [Updated] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

 [ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-7075:
---
Attachment: image-101420-2.png
image-101420-3.png
image-101420-1.png

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: image-101420-1.png, image-101420-2.png, 
> image-101420-3.png, image-10833-1.png, image-10833-2.png, 
> javaheapdump-oom-derby-101420.zip, javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derbyserver.log 
> -Djava.rmi.server.hostname=9.121.2

[jira] [Updated] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

 [ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-7075:
---
Attachment: image-10833-2.png
image-10833-1.png

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: image-10833-1.png, image-10833-2.png, 
> javaheapdump-oom-derby-101420.zip, javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derbyserver.log 
> -Djava.rmi.server.hostname=9.121.242.177 
> -[Djava.net|https://slack-redir.net/link?url=http%3A%2F%2FDjava.net].preferIPv4Stack=true

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17088261#comment-17088261
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


Hi Richard, thank you very much for your help on this JIRA.  I will try to put 
as much information as possible here to assist you in the investigation.

 

Here is the analysis of heap and javacore from the 10.8.3.3 OOM scenario.



There are two heapdumps  both show same objects on heap (leak suspects)
Dump Event "systhrow" (0004) Detail "java/lang/OutOfMemoryError" "Java heap 
space" received
Date: 2020/04/16 at 14:42:23:368
JRE 1.8.0 Linux amd64-64 (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
-Xgcpolicy:gencon
-Xmn192m
-Xms512m
-Xmx512m
-Xverbosegclog:/var/log/derbyserver_gc.log,10,1000
Total memory free: 5,500,848 (0x0053EFB0) = 5M
Total memory in use: 531,370,064 (0x1FAC1050) = 507M
>From heapdump review I see below objects (leak suspects)
1)
"org.apache.derby.impl.store.access.RAMTransaction" 97,807,424 (22.04%) bytes 
<-- 93.3MB The memory is accumulated in one instance of 
"org.apache.derby.impl.store.access.sort.SortBuffer"
2)
 "org.apache.derby.impl.store.access.RAMTransaction" 94,323,360 (21.25%) bytes 
<-- 90MB The memory is accumulated in one instance of 
"org.apache.derby.impl.store.access.sort.SortBuffer"
3) "org.apache.derby.impl.services.cache.ConcurrentCache" (12.48%) bytes <-- 
52.8MB.
The memory is accumulated in one instance of 
"org.apache.derby.impl.services.cache.ConcurrentCache"
4)
org.apache.derby.impl.services.cache.ConcurrentCache" 54,617,784 (12.31%) bytes 
<--52.1MB The memory is accumulated in one instance of 
"org.apache.derby.impl.services.cache.ConcurrentCache"First two leak suspects 
are coming from below current thread in Javacore. "DRDAConnThread_37" 
J9VMThread:0x030C5200, omrthread_t:0x7F97301F7830, 
java/lang/Thread:0xE101CE70, state:R, prio=5
(java/lang/Thread getId:0x8B, isDaemon:true)
 Java callstack:
at org/apache/derby/iapi/types/SQLChar.getString(Bytecode PC:18(Compiled Code))
at org/apache/derby/iapi/types/SQLVarchar.cloneValue(Bytecode PC:5(Compiled 
Code))
at org/apache/derby/impl/sql/execute/BasicSortObserver.getClone(Bytecode 
PC:22(Compiled Code))
at 
org/apache/derby/impl/sql/execute/BasicSortObserver.insertNonDuplicateKey(Bytecode
 PC:9(Compiled Code))
at org/apache/derby/impl/store/access/sort/SortBuffer.insert(Bytecode 
PC:331(Compiled Code))
at org/apache/derby/impl/store/access/sort/MergeInserter.insert(Bytecode 
PC:13(Compiled Code))
at org/apache/derby/impl/sql/execute/SortResultSet.loadSorter(Bytecode 
PC:112(Compiled Code))
at org/apache/derby/impl/sql/execute/SortResultSet.openCore(Bytecode 
PC:90(Compiled Code))
at 
org/apache/derby/impl/sql/execute/ScrollInsensitiveResultSet.openCore(Bytecode 
PC:14(Compiled Code))
at org/apache/derby/impl/sql/execute/BasicNoPutResultSetImpl.open(Bytecode 
PC:10(Compiled Code))
at org/apache/derby/impl/sql/GenericPreparedStatement.executeStmt(Bytecode 
PC:357(Compiled Code))
at org/apache/derby/impl/sql/GenericPreparedStatement.execute(Bytecode 
PC:5(Compiled Code))
at org/apache/derby/impl/jdbc/EmbedStatement.executeStatement(Bytecode 
PC:136(Compiled Code))
(entered lock: org/apache/derby/impl/jdbc/EmbedConnection@0xE6723348, 
entry count: 1)
at org/apache/derby/impl/jdbc/EmbedPreparedStatement.executeStatement(Bytecode 
PC:16(Compiled Code))
at org/apache/derby/impl/jdbc/EmbedPreparedStatement.execute(Bytecode 
PC:9(Compiled Code))
at org/apache/derby/impl/drda/DRDAStatement.execute(Bytecode PC:6(Compiled 
Code))
at org/apache/derby/impl/drda/DRDAConnThread.processCommands(Bytecode 
PC:611(Compiled Code))
at org/apache/derby/impl/drda/DRDAConnThread.run(Bytecode PC:115)

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" 

[jira] [Comment Edited] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087705#comment-17087705
 ] 

Vamsavardhana Reddy edited comment on DERBY-7075 at 4/20/20, 1:20 PM:
--

Yes, only difference is the OS.  JVM version, parameters and Derby versions are 
same.  With similar application work loads, the combinations/scenarios we tried 
are:
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on{color:#0747a6} *RHELv7* flavour{color} 
with 4GB RAM –- No OOM exceptions – {color:#0747a6}GOOD{color}
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on {color:#ff}*RHELv8*{color} with 
4GB RAM –- OOM error – {color:#ff}BAD{color}
 # Derby v10.14.20 on IBMSDK v8.0.6.0 on RHELv8 with 4GB RAM –- OOM error – BAD

After scenario 2, we tried scenario 3 just in case any already addressed Derby 
JIRA has taken care it.

 


was (Author: vamsic):
Yes, only difference is the OS.  JVM paramters and Derby versions are same.  
With similar application work loads, the combinations/scenarios we tried are:
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on{color:#0747a6} *RHELv7* flavour{color} 
with 4GB RAM –- No OOM exceptions – {color:#0747a6}GOOD{color}
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on {color:#ff}*RHELv8*{color} with 
4GB RAM –- OOM error – {color:#ff}BAD{color}
 # Derby v10.14.20 on IBMSDK v8.0.6.0 on RHELv8 with 4GB RAM –- OOM error – BAD

After scenario 2, we tried scenario 3 just in case any already addressed Derby 
JIRA has taken care it.

 

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..

[jira] [Comment Edited] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087705#comment-17087705
 ] 

Vamsavardhana Reddy edited comment on DERBY-7075 at 4/20/20, 1:20 PM:
--

Yes, only difference is the OS.  JVM paramters and Derby versions are same.  
With similar application work loads, the combinations/scenarios we tried are:
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on{color:#0747a6} *RHELv7* flavour{color} 
with 4GB RAM –- No OOM exceptions – {color:#0747a6}GOOD{color}
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on {color:#ff}*RHELv8*{color} with 
4GB RAM –- OOM error – {color:#ff}BAD{color}
 # Derby v10.14.20 on IBMSDK v8.0.6.0 on RHELv8 with 4GB RAM –- OOM error – BAD

After scenario 2, we tried scenario 3 just in case any already addressed Derby 
JIRA has taken care it.

 


was (Author: vamsic):
With similar application work loads, the combinations/scenarios we tried are:
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on{color:#0747a6} *RHELv7* flavour{color} 
with 4GB RAM –- No OOM exceptions – {color:#0747a6}GOOD{color}
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on {color:#FF}*RHELv8*{color} with 
4GB RAM –- OOM error – {color:#FF}BAD{color}
 # Derby v10.14.20 on IBMSDK v8.0.6.0 on RHELv8 with 4GB RAM –- OOM error – BAD

After scenario 2, we tried scenario 3 just in case any already addressed Derby 
JIRA has taken care it.

 

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087705#comment-17087705
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


With similar application work loads, the combinations/scenarios we tried are:
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on{color:#0747a6} *RHELv7* flavour{color} 
with 4GB RAM –- No OOM exceptions – {color:#0747a6}GOOD{color}
 # Derby v10.8.3.3 on IBMSDK v8.0.6.0 on {color:#FF}*RHELv8*{color} with 
4GB RAM –- OOM error – {color:#FF}BAD{color}
 # Derby v10.14.20 on IBMSDK v8.0.6.0 on RHELv8 with 4GB RAM –- OOM error – BAD

After scenario 2, we tried scenario 3 just in case any already addressed Derby 
JIRA has taken care it.

 

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=f

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087345#comment-17087345
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


Richard, we are seeing a collection of 8 RAMTransaction objects occupying 
around 250MB of heap space mainly.  If you have trouble using the above tool, I 
will be glad to send you static info from my analysis.  But, it is more 
useful/helpful if you can take a look at the analysis in the tool as you can 
quickly switch to different views to draw inference.

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/manage

[jira] [Commented] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

[ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087342#comment-17087342
 ] 

Vamsavardhana Reddy commented on DERBY-7075:


IBM Heap Analyzer is available from 
[https://www.ibm.com/support/pages/ibm-heapanalyzer] 

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue. 
>  
> Exception in thread "DRDAConnThread_39" Exception in thread 
> "DRDAConnThread_41" java/lang/OutOfMemoryError: Java heap space
>  at java/nio/CharBuffer.wrap (CharBuffer.java:373)
>  at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
>  at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
>  at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
>  at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
>  at java/io/PrintStream.write (PrintStream.java:550)
>  at java/io/PrintStream.print (PrintStream.java:693)
>  at java/io/PrintStream.append (PrintStream.java:1089)
>  at java/io/PrintStream.append (PrintStream.java:70)
>  at com/ibm/oti/util/Util.appendTo (Util.java:225)
>  at java/lang/Throwable.printStackTrace (Throwable.java:506)
>  at java/lang/Throwable.printStackTrace (Throwable.java:302)
>  at java/lang/Throwable.printStackTrace (Throwable.java:247)
> Exception in thread "DRDAConnThread_43" 
> java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
>  Java heap space
>  at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
>  at 
> java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
>  Java heap space
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
>  at java/lang/Thread.uncaughtException (Thread.java:1332)
> ---
> :*v7.5.4*
>  OS: RHELv8 with 4GB RAM
>  Java: IBM Java SDK v8.0.6.0
>  Apache Derby v10.8.3.3, Apache Derby v10.14.2.0
> [bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
>  java version "1.8.0_231"
>  Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
>  IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
>  OpenJ9   - f0b6be7
>  OMR  - 18d8f94
>  IBM  - 233dfb5)
>  JCL - 20191016_01 based on Oracle jdk8u231-b10
>  
> /usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derbyserver.log 
> -Djava.rmi.server.hostname=9.121.242.177 
> -[Djava.net|https://slack-redir.net/link?url=ht

[jira] [Updated] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

 [ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-7075:
---
Description: 
We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
memory, java version and java launch command settings for the derby and other 
java processes required for our application in which similar projects/workloads 
are running.  But in one of the environments we are seeing out of memory (OOM) 
exceptions in the derby server java process.  In this environment where OOM is 
occurring, we even refreshed Apache derby to the latest version available for 
Java 8 and still encountered OOM exceptions.  Please help resolve this OOM 
issue. 

 

Exception in thread "DRDAConnThread_39" Exception in thread "DRDAConnThread_41" 
java/lang/OutOfMemoryError: Java heap space
 at java/nio/CharBuffer.wrap (CharBuffer.java:373)
 at sun/nio/cs/StreamEncoder.implWrite (StreamEncoder.java:277)
 at sun/nio/cs/StreamEncoder.write (StreamEncoder.java:137)
 at java/io/OutputStreamWriter.write (OutputStreamWriter.java:244)
 at java/io/BufferedWriter.flushBuffer (BufferedWriter.java:140)
 at java/io/PrintStream.write (PrintStream.java:550)
 at java/io/PrintStream.print (PrintStream.java:693)
 at java/io/PrintStream.append (PrintStream.java:1089)
 at java/io/PrintStream.append (PrintStream.java:70)
 at com/ibm/oti/util/Util.appendTo (Util.java:225)
 at java/lang/Throwable.printStackTrace (Throwable.java:506)
 at java/lang/Throwable.printStackTrace (Throwable.java:302)
 at java/lang/Throwable.printStackTrace (Throwable.java:247)
Exception in thread "DRDAConnThread_43" 
java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryError(?)java.lang.OutOfMemoryError:
 Java heap space
 at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:872)
 at 
java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:8java/lang/OutOfMemoryError:
 Java heap space
 at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:872)
 at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
 at java/lang/ThreadGroup.uncaughtException (ThreadGroup.java:866)
 at java/lang/Thread.uncaughtException (Thread.java:1332)

---

:*v7.5.4*
 OS: RHELv8 with 4GB RAM
 Java: IBM Java SDK v8.0.6.0
 Apache Derby v10.8.3.3, Apache Derby v10.14.2.0

[bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
 java version "1.8.0_231"
 Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
 IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
20191106_432135 (JIT enabled, AOT enabled)
 OpenJ9   - f0b6be7
 OMR  - 18d8f94
 IBM  - 233dfb5)
 JCL - 20191016_01 based on Oracle jdk8u231-b10

 

/usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
-Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
-Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
 
-Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
 -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
-Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
 -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
-XX:+HeapManagementMXBeanCompatibility 
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
-Dderby.system.home=/usr/ironhide/var/db/staging -Dcom.sun.management.jmxremote 
-Dcom.approuter.management.jmxremote.connectorserver.port=31997 
-Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
-Dcom.sun.management.jmxremote.port=1997 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=true 
-Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
 
-Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
 -Dcom.approuter.logging.filename=derbyserver.log 
-Djava.rmi.server.hostname=9.121.242.177 
-[Djava.net|https://slack-redir.net/link?url=http%3A%2F%2FDjava.net].preferIPv4Stack=true
 -Dcom.approuter.sysconf.appliance=true 
-Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog 
-Dcom.mchange.v2.resourcepool.experimental.useScatteredAcquireTask=true 
-Dc3p0.checkoutTimeout=3 -Dc3p0.maxIdleTimeExcessConnections=30 
-Dc3p0.maxStatementsPerConnection=100 -Dc3p0.testConnectionOnCheckin=true 
-Dc3p0.testConnectionOnCheckout=true -Dc3p0.numHelperThreads=10 
-Dosgi.clean=true 
-Dosgi.configuration.area=/usr/ironhide/maestro/bundles/derbyserver 
-Dorg.osgi.framework.bootdelegation=* -Declipse.application.noDefault=true 
-Declipse.consoleLog=true 
-Dcom.approuter.orcmon.purger.compressionmode=sequential 
-DIH_ROOT=/usr/ironhide 
-Djava.util.logging.config.file=/usr/ironhide/maestro/etc/logging.properties 
-

[jira] [Updated] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

 [ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-7075:
---
Description: 
We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
memory, java version and java launch command settings for the derby and other 
java processes required for our application in which similar projects/workloads 
are running.  But in one of the environments we are seeing out of memory (OOM) 
exceptions in the derby server java process.  In this environment where OOM is 
occurring, we even refreshed Apache derby to the latest version available for 
Java 8 and still encountered OOM exceptions.  Please help resolve this OOM 
issue. 

:*v7.5.4*
 OS: RHELv8 with 4GB RAM
 Java: IBM Java SDK v8.0.6.0
 Apache Derby v10.8.3.3, Apache Derby v10.14.2.0

[bceglc385/Standalone ~]# /usr/java/default/jre/bin/java -version
 java version "1.8.0_231"
 Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
 IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
20191106_432135 (JIT enabled, AOT enabled)
 OpenJ9   - f0b6be7
 OMR  - 18d8f94
 IBM  - 233dfb5)
 JCL - 20191016_01 based on Oracle jdk8u231-b10

 

/usr/java/default/bin/java -server -Xdump:heap+system+snap:none 
-Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
-Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
 
-Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
 -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
-Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
 -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
-XX:+HeapManagementMXBeanCompatibility 
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
-Dderby.system.home=/usr/ironhide/var/db/staging -Dcom.sun.management.jmxremote 
-Dcom.approuter.management.jmxremote.connectorserver.port=31997 
-Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
-Dcom.sun.management.jmxremote.port=1997 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=true 
-Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
 
-Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
 -Dcom.approuter.logging.filename=derbyserver.log 
-Djava.rmi.server.hostname=9.121.242.177 
-[Djava.net|https://slack-redir.net/link?url=http%3A%2F%2FDjava.net].preferIPv4Stack=true
 -Dcom.approuter.sysconf.appliance=true 
-Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog 
-Dcom.mchange.v2.resourcepool.experimental.useScatteredAcquireTask=true 
-Dc3p0.checkoutTimeout=3 -Dc3p0.maxIdleTimeExcessConnections=30 
-Dc3p0.maxStatementsPerConnection=100 -Dc3p0.testConnectionOnCheckin=true 
-Dc3p0.testConnectionOnCheckout=true -Dc3p0.numHelperThreads=10 
-Dosgi.clean=true 
-Dosgi.configuration.area=/usr/ironhide/maestro/bundles/derbyserver 
-Dorg.osgi.framework.bootdelegation=* -Declipse.application.noDefault=true 
-Declipse.consoleLog=true 
-Dcom.approuter.orcmon.purger.compressionmode=sequential 
-DIH_ROOT=/usr/ironhide 
-Djava.util.logging.config.file=/usr/ironhide/maestro/etc/logging.properties 
-jar /usr/ironhide/maestro/bundles/org.eclipse.osgi_3.10.1.v20140909-1633.jar

 

[bceglc390/Standalone ~]# cat /proc/meminfo
 MemTotal:    4057920 kB
 MemFree:  152896 kB
 Buffers:  342624 kB
 Cached:   729112 kB
 SwapCached: 3428 kB
 Active:  2644068 kB
 Inactive:    1066404 kB
 Active(anon):    1951704 kB
 Inactive(anon):   712696 kB
 Active(file): 692364 kB
 Inactive(file):   353708 kB
 Unevictable:  20 kB
 Mlocked:  20 kB
 SwapTotal:   3999736 kB
 SwapFree:    3988760 kB
 Dirty:   720 kB
 Writeback: 0 kB
 AnonPages:   2636560 kB
 Mapped:    38508 kB
 Shmem: 25648 kB
 Slab: 129600 kB
 SReclaimable:  48320 kB
 SUnreclaim:    81280 kB
 KernelStack:    6568 kB
 PageTables:    6 kB
 NFS_Unstable:  0 kB
 Bounce:    0 kB
 WritebackTmp:  0 kB
 CommitLimit: 6028696 kB
 Committed_AS:    1658244 kB
 VmallocTotal:   34359738367 kB
 VmallocUsed:  152396 kB
 VmallocChunk:   34359582800 kB
 AnonHugePages:   2433024 kB
 HugePages_Total:   0
 HugePages_Free:    0
 HugePages_Rsvd:    0
 HugePages_Surp:    0
 Hugepagesize:   2048 kB
 DirectMap4k:    4096 kB
 DirectMap2M: 4190208 kB

 

--
 *v7.5.3*
 OS: Bedrock v5 which is custom OS based on RHELv7 with 4GB RAM
 Java: IBM Java SDK v8.0.6.0
 Apache Derby v10.8.3.3

 

[bceglc390/Standalone ~]# /us

[jira] [Updated] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

 [ 
https://issues.apache.org/jira/browse/DERBY-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-7075:
---
Attachment: javaheapdump-oom-derby-v10833.zip
javaheapdump-oom-derby-101420.zip

> OutOfMemory Exceptions in derby server process
> --
>
> Key: DERBY-7075
> URL: https://issues.apache.org/jira/browse/DERBY-7075
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.3.0, 10.14.2.0
>Reporter: Vamsavardhana Reddy
>Priority: Major
> Attachments: javaheapdump-oom-derby-101420.zip, 
> javaheapdump-oom-derby-v10833.zip
>
>
> We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
> memory, java version and java launch command settings for the derby and other 
> java processes required for our application in which similar 
> projects/workloads are running.  But in one of the environments we are seeing 
> out of memory (OOM) exceptions in the derby server java process.  In this 
> environment where OOM is occurring, we even refreshed Apache derby to the 
> latest version available for Java 8 and still encountered OOM exceptions.  
> Please help resolve this OOM issue.  :*v7.5.4*
> OS: RHELv8 with 4GB RAM
> Java: IBM Java SDK v8.0.6.0
> Apache Derby v10.8.3.3, Apache Derby v10.14.2.0[bceglc385/Standalone ~]# 
> /usr/java/default/jre/bin/java -version
> java version "1.8.0_231"
> Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
> IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
> 20191106_432135 (JIT enabled, AOT enabled)
> OpenJ9   - f0b6be7
> OMR  - 18d8f94
> IBM  - 233dfb5)
> JCL - 20191016_01 based on Oracle jdk8u231-b10/usr/java/default/bin/java 
> -server -Xdump:heap+system+snap:none 
> -Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
> -Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
>  
> -Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
>  -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
> -Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
>  -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
> -XX:+HeapManagementMXBeanCompatibility 
> -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
> -Dderby.system.home=/usr/ironhide/var/db/staging 
> -Dcom.sun.management.jmxremote 
> -Dcom.approuter.management.jmxremote.connectorserver.port=31997 
> -Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
> -Dcom.sun.management.jmxremote.port=1997 
> -Dcom.sun.management.jmxremote.local.only=false 
> -Dcom.sun.management.jmxremote.ssl=false 
> -Dcom.sun.management.jmxremote.authenticate=true 
> -Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
>  
> -Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
>  -Dcom.approuter.logging.filename=derbyserver.log 
> -Djava.rmi.server.hostname=9.121.242.177 
> -[Djava.net|https://slack-redir.net/link?url=http%3A%2F%2FDjava.net].preferIPv4Stack=true
>  -Dcom.approuter.sysconf.appliance=true 
> -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog 
> -Dcom.mchange.v2.resourcepool.experimental.useScatteredAcquireTask=true 
> -Dc3p0.checkoutTimeout=3 -Dc3p0.maxIdleTimeExcessConnections=30 
> -Dc3p0.maxStatementsPerConnection=100 -Dc3p0.testConnectionOnCheckin=true 
> -Dc3p0.testConnectionOnCheckout=true -Dc3p0.numHelperThreads=10 
> -Dosgi.clean=true 
> -Dosgi.configuration.area=/usr/ironhide/maestro/bundles/derbyserver 
> -Dorg.osgi.framework.bootdelegation=* -Declipse.application.noDefault=true 
> -Declipse.consoleLog=true 
> -Dcom.approuter.orcmon.purger.compressionmode=sequential 
> -DIH_ROOT=/usr/ironhide 
> -Djava.util.logging.config.file=/usr/ironhide/maestro/etc/logging.properties 
> -jar 
> /usr/ironhide/maestro/bundles/org.eclipse.osgi_3.10.1.v20140909-1633.jar[bceglc390/Standalone
>  ~]# cat /proc/meminfo
> MemTotal:    4057920 kB
> MemFree:  152896 kB
> Buffers:  342624 kB
> Cached:   729112 kB
> SwapCached: 3428 kB
> Active:  2644068 kB
> Inactive:    1066404 kB
> Active(anon):    1951704 kB
> Inactive(anon):   712696 kB
> Active(file): 692364 kB
> Inactive(file):   353708 kB
> Unevictable:  20 kB
> Mlocked:  20 kB
> SwapTotal:   3999736 kB
> SwapFree:    3988760 kB
> Dirty:   720 kB
> Writeback: 0 kB
> AnonPages:   2636560 kB
> Mapped:    38508 kB
> Shmem: 25648 kB
> Slab: 129600 kB
> SReclaimable:  48320 kB

[jira] [Created] (DERBY-7075) OutOfMemory Exceptions in derby server process

[Vamsavardhana Reddy (Jira)]

Vamsavardhana Reddy created DERBY-7075:
--

 Summary: OutOfMemory Exceptions in derby server process
 Key: DERBY-7075
 URL: https://issues.apache.org/jira/browse/DERBY-7075
 Project: Derby
  Issue Type: Bug
  Components: Network Server
Affects Versions: 10.14.2.0, 10.8.3.0
Reporter: Vamsavardhana Reddy


We have two runtime environments v7.5.4 and v7.5.3 similar in terms of OS 
memory, java version and java launch command settings for the derby and other 
java processes required for our application in which similar projects/workloads 
are running.  But in one of the environments we are seeing out of memory (OOM) 
exceptions in the derby server java process.  In this environment where OOM is 
occurring, we even refreshed Apache derby to the latest version available for 
Java 8 and still encountered OOM exceptions.  Please help resolve this OOM 
issue.  :*v7.5.4*
OS: RHELv8 with 4GB RAM
Java: IBM Java SDK v8.0.6.0
Apache Derby v10.8.3.3, Apache Derby v10.14.2.0[bceglc385/Standalone ~]# 
/usr/java/default/jre/bin/java -version
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 8.0.6.0 - pxa6480sr6-20191107_01(SR6))
IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64-Bit Compressed References 
20191106_432135 (JIT enabled, AOT enabled)
OpenJ9   - f0b6be7
OMR  - 18d8f94
IBM  - 233dfb5)
JCL - 20191016_01 based on Oracle jdk8u231-b10/usr/java/default/bin/java 
-server -Xdump:heap+system+snap:none 
-Xdump:heap:events=user,file=/var/log/javaheapdump-user-derby.ph  
-Xdump:heap:events=systhrow,filter=java/lang/OutOfMemoryError,file=/var/log/javaheapdump-oom-derby.phd,range=1..1
 
-Xdump!https://emoji.slack-edge.com/T27SFGS2W/java/c0a6f7d2f3faab62.png!defaults:file=/var/log/javacore-derby.txt
 -Xmx512m -Xms512m -Xmn192m -agentlib:CastIron 
-Xbootclasspath/p:/usr/ironhide/maestro/bundles/com.approuter.management.jmxremote-1.0.0.jar:
 -Xgcpolicy:gencon -Xverbosegclog:/var/log/derbyserver_gc.log,10,1000 
-XX:+HeapManagementMXBeanCompatibility 
-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger 
-Dderby.system.home=/usr/ironhide/var/db/staging -Dcom.sun.management.jmxremote 
-Dcom.approuter.management.jmxremote.connectorserver.port=31997 
-Djmx.remote.protocol.provider.pkgs=com.approuter.management.jmxremote 
-Dcom.sun.management.jmxremote.port=1997 
-Dcom.sun.management.jmxremote.local.only=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.authenticate=true 
-Dcom.sun.management.jmxremote.password.file=/usr/ironhide/maestro/management/jmxremote.password
 
-Dcom.sun.management.jmxremote.access.file=/usr/ironhide/maestro/management/jmxremote.access
 -Dcom.approuter.logging.filename=derbyserver.log 
-Djava.rmi.server.hostname=9.121.242.177 
-[Djava.net|https://slack-redir.net/link?url=http%3A%2F%2FDjava.net].preferIPv4Stack=true
 -Dcom.approuter.sysconf.appliance=true 
-Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog 
-Dcom.mchange.v2.resourcepool.experimental.useScatteredAcquireTask=true 
-Dc3p0.checkoutTimeout=3 -Dc3p0.maxIdleTimeExcessConnections=30 
-Dc3p0.maxStatementsPerConnection=100 -Dc3p0.testConnectionOnCheckin=true 
-Dc3p0.testConnectionOnCheckout=true -Dc3p0.numHelperThreads=10 
-Dosgi.clean=true 
-Dosgi.configuration.area=/usr/ironhide/maestro/bundles/derbyserver 
-Dorg.osgi.framework.bootdelegation=* -Declipse.application.noDefault=true 
-Declipse.consoleLog=true 
-Dcom.approuter.orcmon.purger.compressionmode=sequential 
-DIH_ROOT=/usr/ironhide 
-Djava.util.logging.config.file=/usr/ironhide/maestro/etc/logging.properties 
-jar 
/usr/ironhide/maestro/bundles/org.eclipse.osgi_3.10.1.v20140909-1633.jar[bceglc390/Standalone
 ~]# cat /proc/meminfo
MemTotal:    4057920 kB
MemFree:  152896 kB
Buffers:  342624 kB
Cached:   729112 kB
SwapCached: 3428 kB
Active:  2644068 kB
Inactive:    1066404 kB
Active(anon):    1951704 kB
Inactive(anon):   712696 kB
Active(file): 692364 kB
Inactive(file):   353708 kB
Unevictable:  20 kB
Mlocked:  20 kB
SwapTotal:   3999736 kB
SwapFree:    3988760 kB
Dirty:   720 kB
Writeback: 0 kB
AnonPages:   2636560 kB
Mapped:    38508 kB
Shmem: 25648 kB
Slab: 129600 kB
SReclaimable:  48320 kB
SUnreclaim:    81280 kB
KernelStack:    6568 kB
PageTables:    6 kB
NFS_Unstable:  0 kB
Bounce:    0 kB
WritebackTmp:  0 kB
CommitLimit: 6028696 kB
Committed_AS:    1658244 kB
VmallocTotal:   34359738367 kB
VmallocUsed:  152396 kB
VmallocChunk:   34359582800 kB
AnonHugePages:   2433024 kB
HugePages_Total:   0
HugePages_Free:    0
HugePages_Rsvd:    0
HugePages_Surp:    0
Hugepagesize:   2048 kB
DirectMap4k:    4096 kB
DirectMap2M: 4190208 kB--
*v7.5.3*
OS: Bedro

[jira] [Commented] (DERBY-6766) java.sql.SQLException: nospc.U Unexpected no space error while attempting to update a row on page

[Vamsavardhana Reddy (JIRA)]

[ 
https://issues.apache.org/jira/browse/DERBY-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064571#comment-16064571
 ] 

Vamsavardhana Reddy commented on DERBY-6766:


I have tried the procedure given in 
https://db.apache.org/derby/docs/10.7/adminguide/cadminspace21579.html to 
release the unused space on all application tables.  But the error occurred 
again within a few hours!!

> java.sql.SQLException: nospc.U  Unexpected no space error while attempting to 
> update a row on page
> --
>
> Key: DERBY-6766
> URL: https://issues.apache.org/jira/browse/DERBY-6766
> Project: Derby
>  Issue Type: Bug
>  Components: Store
>Affects Versions: 10.10.2.0
> Environment: Derby 10.10.2.0 - (1582446), AIX 6.1.0.0 
> (6100-08-03-1339), Hibernate 3.5.6-Final, Java 1.6.0 IBM J9 VM (build 2.4, 
> JRE 1.6.0 IBM J9 2.4 AIX ppc64-64)
>Reporter: Kai Duckwitz
>Priority: Critical
>  Labels: nospc
> Attachments: console.zip, derby.log, IDD.DDL
>
>
> We already had this error with version 10.8.2.2 an hoped that it would be 
> fixed in 10.10.2.0 due to [DERBY-4923] - but it is not.
> This error occurs sporadically on updating or inserting objects via 
> hibernate. The application runs for weeks without a problem (20-50 object 
> updates/inserts per second) and then the error occures up to 10 times in 2 
> hours; after that the error can be absent for days or weeks.
> We use only these data types:
> NUMERIC,SMALLINT,TIMESTAMP,VARCHAR
> Stacktrace:
> Caused by: java.sql.SQLException: Unexpected no space error while attempting 
> to update a row on page Page(94,Container(0, 1632)). Values of internal 
> fields at time of error are as follows: slot = 301, recordId = 28, 
> newColumnList = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13}, nextColumn = -1, 
> mode = 0, nextPortionHandle = null, page dump = 
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   ... 51 more
> Caused by: java.sql.SQLException: nospc.U
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown 
> Source)
>   at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown 
> Source)
>   at 
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
> Source)
>   ... 48 more
> Caused by: ERROR nospc: nospc.U
>   at org.apache.derby.impl.store.raw.data.StoredPage.logRow(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.UpdateOperation.writeOptionalDataToBuffer(Unknown
>  Source)
>   at org.apache.derby.impl.store.raw.data.UpdateOperation.(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.LoggableActions.actionUpdate(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.StoredPage.doUpdateAtSlot(Unknown Source)
>   at org.apache.derby.impl.store.raw.data.BasePage.updateAtSlot(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.replace(Unknown
>  Source)
>   at org.apache.derby.impl.sql.execute.RowChangerImpl.updateRow(Unknown 
> Source)
>   at 
> org.apache.derby.impl.sql.execute.UpdateResultSet.collectAffectedRows(Unknown 
> Source)
>   at org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown 
> Source)
>   at 
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
>   at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown 
> Source)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (DERBY-6766) java.sql.SQLException: nospc.U Unexpected no space error while attempting to update a row on page

[Vamsavardhana Reddy (JIRA)]

[ 
https://issues.apache.org/jira/browse/DERBY-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030712#comment-16030712
 ] 

Vamsavardhana Reddy commented on DERBY-6766:


Does this database corruption happen due to load on the database or is it 
because of specific data that is handled by the database?  Anything specific 
that causes this corruption and any suggestions to avoid this issue?

> java.sql.SQLException: nospc.U  Unexpected no space error while attempting to 
> update a row on page
> --
>
> Key: DERBY-6766
> URL: https://issues.apache.org/jira/browse/DERBY-6766
> Project: Derby
>  Issue Type: Bug
>  Components: Store
>Affects Versions: 10.10.2.0
> Environment: Derby 10.10.2.0 - (1582446), AIX 6.1.0.0 
> (6100-08-03-1339), Hibernate 3.5.6-Final, Java 1.6.0 IBM J9 VM (build 2.4, 
> JRE 1.6.0 IBM J9 2.4 AIX ppc64-64)
>Reporter: Kai Duckwitz
>Priority: Critical
>  Labels: nospc
> Attachments: console.zip, derby.log, IDD.DDL
>
>
> We already had this error with version 10.8.2.2 an hoped that it would be 
> fixed in 10.10.2.0 due to [DERBY-4923] - but it is not.
> This error occurs sporadically on updating or inserting objects via 
> hibernate. The application runs for weeks without a problem (20-50 object 
> updates/inserts per second) and then the error occures up to 10 times in 2 
> hours; after that the error can be absent for days or weeks.
> We use only these data types:
> NUMERIC,SMALLINT,TIMESTAMP,VARCHAR
> Stacktrace:
> Caused by: java.sql.SQLException: Unexpected no space error while attempting 
> to update a row on page Page(94,Container(0, 1632)). Values of internal 
> fields at time of error are as follows: slot = 301, recordId = 28, 
> newColumnList = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13}, nextColumn = -1, 
> mode = 0, nextPortionHandle = null, page dump = 
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   ... 51 more
> Caused by: java.sql.SQLException: nospc.U
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown 
> Source)
>   at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown 
> Source)
>   at 
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
> Source)
>   ... 48 more
> Caused by: ERROR nospc: nospc.U
>   at org.apache.derby.impl.store.raw.data.StoredPage.logRow(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.UpdateOperation.writeOptionalDataToBuffer(Unknown
>  Source)
>   at org.apache.derby.impl.store.raw.data.UpdateOperation.(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.LoggableActions.actionUpdate(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.raw.data.StoredPage.doUpdateAtSlot(Unknown Source)
>   at org.apache.derby.impl.store.raw.data.BasePage.updateAtSlot(Unknown 
> Source)
>   at 
> org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.replace(Unknown
>  Source)
>   at org.apache.derby.impl.sql.execute.RowChangerImpl.updateRow(Unknown 
> Source)
>   at 
> org.apache.derby.impl.sql.execute.UpdateResultSet.collectAffectedRows(Unknown 
> Source)
>   at org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown 
> Source)
>   at 
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
>   at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown 
> Source)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (DERBY-6766) java.sql.SQLException: nospc.U Unexpected no space error while attempting to update a row on page

[Vamsavardhana Reddy (JIRA)]

[ 
https://issues.apache.org/jira/browse/DERBY-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939088#comment-15939088
 ] 

Vamsavardhana Reddy commented on DERBY-6766:


I am seeing a similar error with Derby 10.8.3.3 and using hibernate for 
persistence.  Does this error indicate DB is corrupted?

Caused by: java.sql.SQLException: Unexpected no space error while attempting to 
update a row on page Page(87,Container(0, 1328)). Values of internal fields at 
time of error are as follows: slot = 0, recordId = 863, newColumnList = {3}, 
nextColumn = 3, mode = 24, nextPortionHandle = null, page dump = 

at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
 Source)
... 49 more
Caused by: java.sql.SQLException: nospc.U
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
 Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown 
Source)
at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)
... 46 more
Caused by: ERROR nospc: nospc.U
at org.apache.derby.impl.store.raw.data.StoredPage.logRow(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.InsertOperation.writeOptionalDataToBuffer(Unknown
 Source)
at org.apache.derby.impl.store.raw.data.InsertOperation.(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.LoggableActions.actionInsert(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.BasePage.insertAllowOverflow(Unknown 
Source)
at 
org.apache.derby.impl.store.raw.data.StoredPage.doUpdateAtSlot(Unknown Source)
at org.apache.derby.impl.store.raw.data.BasePage.updateAtSlot(Unknown 
Source)
at 
org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.replace(Unknown
 Source)
at org.apache.derby.impl.sql.execute.RowChangerImpl.updateRow(Unknown 
Source)
at 
org.apache.derby.impl.sql.execute.UpdateResultSet.collectAffectedRows(Unknown 
Source)
at org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown 
Source)
at 
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown 
Source)
... 42 more
java.sql.SQLException: nospc.U


> java.sql.SQLException: nospc.U  Unexpected no space error while attempting to 
> update a row on page
> --
>
> Key: DERBY-6766
> URL: https://issues.apache.org/jira/browse/DERBY-6766
> Project: Derby
>  Issue Type: Bug
>  Components: Store
>Affects Versions: 10.10.2.0
> Environment: Derby 10.10.2.0 - (1582446), AIX 6.1.0.0 
> (6100-08-03-1339), Hibernate 3.5.6-Final, Java 1.6.0 IBM J9 VM (build 2.4, 
> JRE 1.6.0 IBM J9 2.4 AIX ppc64-64)
>Reporter: Kai Duckwitz
>Priority: Critical
>  Labels: nospc
> Attachments: console.zip, derby.log, IDD.DDL
>
>
> We already had this error with version 10.8.2.2 an hoped that it would be 
> fixed in 10.10.2.0 due to [DERBY-4923] - but it is not.
> This error occurs sporadically on updating or inserting objects via 
> hibernate. The application runs for weeks without a problem (20-50 object 
> updates/inserts per second) and then the error occures up to 10 times in 2 
> hours; after that the error can be absent for days or weeks.
> We use only these data types:
> NUMERIC,SMALLINT,TIMESTAMP,VARCHAR
> Stacktrace:
> Caused by: java.sql.SQLException: Unexpected no space error while attempting 
> to update a row on page Page(94,Container(0, 1632)). Values of internal 
> fields at time of error are as follows: slot = 301, recordId = 28, 
> newColumnList = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13}, nextColumn = -1, 
> mode = 0, nextPortionHandle = null, page dump = 
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   ... 51 more
> Caused by: java.sql.SQLException: nospc.U
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
>  Source)
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown 
> Source)
>   at org.apache.derby.impl.jdbc.Util.gene

[jira] [Updated] (DERBY-6622) Derby server process hitting OutOfMemoryErrors and taking up 100% cpu

[Vamsavardhana Reddy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DERBY-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-6622:
---

Attachment: derby.log

> Derby server process hitting OutOfMemoryErrors and taking up 100% cpu 
> --
>
> Key: DERBY-6622
> URL: https://issues.apache.org/jira/browse/DERBY-6622
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.2.2
> Environment: Linux
>Reporter: Vamsavardhana Reddy
> Attachments: derby.log, derbyserver.all.out.zip
>
>
> We are using Derby Network Server in DataPower appliance.  Underlying OS is a 
> Linux based.  Derby Server is accessed only by java processes running on the 
> same appliance.  We are noticing that the Derby server process is running 
> into OutOfMemoryErrors.  Post the OOM errors, connection requests to the 
> server are failing and the CPU usage reaches 100%.  We are also noticing 
> deadlocks reported in the logs.  Please help identify the cause and how the 
> issue can be resolved.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (DERBY-6622) Derby server process hitting OutOfMemoryErrors and taking up 100% cpu

[Vamsavardhana Reddy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DERBY-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vamsavardhana Reddy updated DERBY-6622:
---

Attachment: derbyserver.all.out.zip

System.out output from server process.

> Derby server process hitting OutOfMemoryErrors and taking up 100% cpu 
> --
>
> Key: DERBY-6622
> URL: https://issues.apache.org/jira/browse/DERBY-6622
> Project: Derby
>  Issue Type: Bug
>  Components: Network Server
>Affects Versions: 10.8.2.2
> Environment: Linux
>Reporter: Vamsavardhana Reddy
> Attachments: derbyserver.all.out.zip
>
>
> We are using Derby Network Server in DataPower appliance.  Underlying OS is a 
> Linux based.  Derby Server is accessed only by java processes running on the 
> same appliance.  We are noticing that the Derby server process is running 
> into OutOfMemoryErrors.  Post the OOM errors, connection requests to the 
> server are failing and the CPU usage reaches 100%.  We are also noticing 
> deadlocks reported in the logs.  Please help identify the cause and how the 
> issue can be resolved.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (DERBY-6622) Derby server process hitting OutOfMemoryErrors and taking up 100% cpu

[Vamsavardhana Reddy (JIRA)]

Vamsavardhana Reddy created DERBY-6622:
--

 Summary: Derby server process hitting OutOfMemoryErrors and taking 
up 100% cpu 
 Key: DERBY-6622
 URL: https://issues.apache.org/jira/browse/DERBY-6622
 Project: Derby
  Issue Type: Bug
  Components: Network Server
Affects Versions: 10.8.2.2
 Environment: Linux
Reporter: Vamsavardhana Reddy


We are using Derby Network Server in DataPower appliance.  Underlying OS is a 
Linux based.  Derby Server is accessed only by java processes running on the 
same appliance.  We are noticing that the Derby server process is running into 
OutOfMemoryErrors.  Post the OOM errors, connection requests to the server are 
failing and the CPU usage reaches 100%.  We are also noticing deadlocks 
reported in the logs.  Please help identify the cause and how the issue can be 
resolved.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] Created: (DERBY-602) SELECT statement does not return the first colum data correctly

[Vamsavardhana Reddy (JIRA)]

SELECT statement does not return the first colum data correctly
---

 Key: DERBY-602
 URL: http://issues.apache.org/jira/browse/DERBY-602
 Project: Derby
Type: Bug
Versions: 10.1.1.0
 Environment: Win XP, Sun JDK 142
Reporter: Vamsavardhana Reddy


SELECT statement does not return the first colum data correctly.  Here is the 
scenario.  I have used "ij" for executing SQL.

I have used the EmbeddedDriver and created a database.   JDBC URL: 
'jdbc:derby:mydb;create=true'

I have created one table using the SQL given below:

 CREATE TABLE "USERS" ("USER" CHAR(10), "PASSWORD" CHAR(10));

I have inserted one row into this table using the SQL given below:

   INSERT INTO USERS VALUES ('user1', 'password1');

Retrieving  rows from this table using the SQL statement "SELECT * FROM USERS"  
returns the data properly.  Where as "SELECT USER, PASSWORD FROM USERS" retuns 
a value 'APP' for USER column data.


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira