[
https://issues.apache.org/jira/browse/DERBY-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894399#comment-16894399
]
Marco edited comment on DERBY-7049 at 7/27/19 10:34 AM:
--------------------------------------------------------
OK. Again thanks a lot. This was really helpful: I saw that indeed many
statements are created using "?", but there are also many that are not. Here's
an example:
{code:sql}
SELECT
"a0"."changed","a0"."created","a0"."id","a0"."lastmodified",
"a0"."lastsyncfromrepositoryid","a0"."localrevision","a0"."name",
"a0"."inprogress","a0"."length","a0"."sha1","a0"."lengthwithpadding",
"a0"."discriminator"
FROM "repofile" "a0"
WHERE
"a0"."discriminator" = 'SsNormalFile' AND
"a0"."id" IN(626,627,628,629,630,631,632,633,634,635,636,637,638)
{code}
It seems to me that DataNucleus creates an explicit new statement with inlined
arguments whenever "IN(...)" is required.
Before I open an issue with DataNucleus, I'd like to know: Is there a way in
JDBC to pass a List<Long> as prepared-statement-argument so that this could be
written as "SELECT ... WHERE ... "a0"."id" IN(?)" or is this not possible,
anyway?
If this is not possible, then I assume the only way to improve things at least
a bit would be to write "SELECT ... WHERE ... "a0"."id"
IN(?,?,?,?,?,?,?,?,?,?,?,?,?)" -- this would still cause many many more
statements to be created than actually needed, but at least every query with
the same number of IDs would reuse a cached statement. This might already be
enough to solve the OOM.
If we have to do it this way, is there a limit of how many
prepared-statement-arguments can be passed? Integer.MAX_VALUE, probably? Or
less?
was (Author: nlmarco):
OK. Again thanks a lot. This was really helpful: I saw that indeed many
statements are created using "?", but there are also many that are not. Here's
an example:
{code:sql}
SELECT
"a0"."changed","a0"."created","a0"."id","a0"."lastmodified","a0"."lastsyncfromrepositoryid","a0"."localrevision","a0"."name","a0"."inprogress","a0"."length","a0"."sha1","a0"."lengthwithpadding","a0"."discriminator"
FROM "repofile" "a0" WHERE "a0"."discriminator" = 'SsNormalFile' AND "a0"."id"
IN(626,627,628,629,630,631,632,633,634,635,636,637,638)
{code}
It seems to me that DataNucleus creates an explicit new statement with inlined
arguments whenever "IN(...)" is required.
Before I open an issue with DataNucleus, I'd like to know: Is there a way in
JDBC to pass a List<Long> as prepared-statement-argument so that this could be
written as "SELECT ... WHERE ... "a0"."id" IN(?)" or is this not possible,
anyway?
If this is not possible, then I assume the only way to improve things at least
a bit would be to write "SELECT ... WHERE ... "a0"."id"
IN(?,?,?,?,?,?,?,?,?,?,?,?,?)" -- this would still cause many many more
statements to be created than actually needed, but at least every query with
the same number of IDs would reuse a cached statement. This might already be
enough to solve the OOM.
If we have to do it this way, is there a limit of how many
prepared-statement-arguments can be passed? Integer.MAX_VALUE, probably? Or
less?
> OutOfMemoryError: Compressed class space
> ----------------------------------------
>
> Key: DERBY-7049
> URL: https://issues.apache.org/jira/browse/DERBY-7049
> Project: Derby
> Issue Type: Bug
> Components: SQL
> Affects Versions: 10.13.1.1
> Reporter: Marco
> Priority: Major
> Attachments: StatementLogReadingVTI.java
>
>
> After a few days of working with an embedded Derby database (currently
> version 10.13.1.1 on Oracle Java 1.8.0_201-b09), the following error occurs:
> *java.lang.OutOfMemoryError: Compressed class space*
> {code:java}
> java.lang.OutOfMemoryError: Compressed class space
> at java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_201]
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763) ~[na:1.8.0_201]
> at java.lang.ClassLoader.defineClass(ClassLoader.java:642) ~[na:1.8.0_201]
> at
> org.apache.derby.impl.services.reflect.ReflectLoaderJava2.loadGeneratedClass(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at
> org.apache.derby.impl.services.reflect.ReflectClassesJava2.loadGeneratedClassFromData(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at
> org.apache.derby.impl.services.reflect.DatabaseClasses.loadGeneratedClass(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at
> org.apache.derby.impl.services.bytecode.GClass.getGeneratedClass(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at
> org.apache.derby.impl.sql.compile.ExpressionClassBuilder.getGeneratedClass(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.sql.compile.StatementNode.generate(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
> ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> ~[derby-10.13.1.1.jar:na]
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.<init>(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
> Source) ~[derby-10.13.1.1.jar:na]
> at
> org.datanucleus.store.rdbms.datasource.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:259)
> ~[datanucleus-rdbms-4.0.12.jar:na]{code}
> I tried to solve the problem by periodically shutting down the database,
> because I read that the generated classes as well as all other allocated
> resources should be released when the DB is shut-down.
> I thus perform the following code once per roughly 20 minutes:
> {code:java}
> String shutdownConnectionURL = connectionURL + ";shutdown=true";
> try {
> DriverManager.getConnection(shutdownConnectionURL);
> } catch (SQLException e) {
> int errorCode = e.getErrorCode();
> if (DERBY_ERROR_CODE_SHUTDOWN_DATABASE_SUCCESSFULLY != errorCode &&
> DERBY_ERROR_CODE_SHUTDOWN_DATABASE_WAS_NOT_RUNNING != errorCode) {
> throw new RuntimeException(e);
> }
> }
> {code}
> Unfortunately, this has no effect :( The OutOfMemoryError still occurs after
> about 2 days. Do I assume correctly that the above code _should_ properly
> shut-down the database? And do I assume correctly that this shut-down should
> release the generated classes?
> IMHO, it is already a bug in Derby that I need to shut-down the database at
> all in order to prevent it from piling up generated classes. Shouldn't it
> already release the generated classes at the end of each transaction? But
> even if I really have to shut-down the DB, it is certainly a bug that the
> classes are still kept in "compressed class space" even after the shut-down.
> I searched the release notes and the existing bugs (here in JIRA) and did not
> find anything related to this {{OutOfMemoryError}}. Hence, I open this
> bug-report, now.
> This issue was originally reported in
> [subshare#74|https://github.com/subshare/subshare/issues/74], but it is IMHO
> clearly a Derby bug.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
