[ http://issues.apache.org/jira/browse/DERBY-1713?page=all ]
John H. Embretsen updated DERBY-1713:
-------------------------------------
Attachment: Derby1713repro.java
I have not been able to try Ibrahim's Test1.java, since I am not able to
decompress the database needed (my version of bzip2 tells me it (test.zip) is
not a valid bzip2 archive - and I'm not using Windows (at the moment)).
Instead, I have created a fully reproducible test case which does not require
having a database in advance. See attachment 'Derby1713repro.java'. To run, put
derby.jar in your classpath and run:
$ java Derby1713repro
If you already have a DB to test with, you can run with the "haveDB" argument:
$ java Derby1713repro haveDB
The test case inserts 40k rows into a table (same DDL as Ibrahim described in
previous comments), which results in a 30 MB database (unless "haveDB" is
specified, in which case the test case only performs a query against an already
existing database).
I think one important thing to do is to reduce as many variables as possible
when trying to reproduce or debug a problem. With this repro, I do get an
OutOfMemoryError (OOME) if the max heap size is 32 MB (-Xmx32M). This happens
without using an inner class (as is done in Test1.java), and it even happens
without executing a 'SELECT count(*)' statement. I have also tried commenting
out the setFetchSize(60000) call, and the OOME still happens (Sun's jvm version
1.5.0_06).
The problem seems to be the huge SELECT, FROM, WHERE statement. At start-up
(when using the "haveDB" option), jstat reports a tenured space usage of 512
bytes. When executing the query, tenured space usage increases to as much as
30272 bytes, which is almost the whole heap. In addition, other parts of the
heap need some space, hence we get an OutOfMemoryError. Feel free to use this
repro as basis for creating a minimalistic test showing the problem.
I think Mike is right when he says that 32 MB max heap is not suitable ("safe")
for this kind of database and this kind of usage. I therefore recommend
lowering the Priority and Urgency of this Jira issue somewhat.
I also recommend changing the summary of this issue once the real problem has
been positively identified. The fact that the JVM is not able to free memory
following an OutOfMemoryEvent is as far as I can tell not a bug in Derby.
> Memory do not return to the system after Shuting down derby 10.2.1.0,
> following an out of memory event
> ------------------------------------------------------------------------------------------------------
>
> Key: DERBY-1713
> URL: http://issues.apache.org/jira/browse/DERBY-1713
> Project: Derby
> Issue Type: Bug
> Components: Performance
> Affects Versions: 10.2.1.0
> Environment: Windows XP SP2
> JRE 1.6 beta2
> Reporter: Ibrahim
> Priority: Critical
> Attachments: Derby1713repro.java, test.zip, Test1.java
>
>
> I face a problem when querying large tables. I run the below SQL and it stuck
> in this query and throws java heap exception OutOfMemory:
> SELECT count(*) FROM <table> WHERE .....
> N.B. I'm using a database of more than 90,000 records (40 MB). I set the
> maxHeap to 32 MB (all other settings have the default value, pageCache ...
> etc ).
> Then, I shutdown the database but the memory is not returned to the system
> (and remain 32 MB [max threshold]). I tried to increase the maxHeap to 128 MB
> in which it works and releases the memory, so I think the problem is when it
> reaches the maxHeap then it seems to not respond to anything such as closing
> the connection or shutting down the database. How can I get rid of this?
> (because i cannot increase the maxHeap as the database increases, I want to
> throw an exception and release the memory)
> I'm using this to shutdown the DB:
> try{DriverManager.getConnection("jdbc:derby:;shutdown=true");}
> catch(SQLException ex){System.err.println("SQLException: " +
> ex.getMessage());}
> I'm using a memory Profiler for monitoring the memory usage.
> Thanks in advanced.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
