Re: NPE from InternalTriggerExecutionContext.cleanup()
Tim Dudgeon tdudg...@informaticsmatters.com writes: On 03/09/2014 13:21, Knut Anders Hatlen wrote: I don't think so. Derby doesn't allow UPDATE statements in BEFORE triggers I was meaning to just set the column value from the sequence into the new values, not using an insert statement. In my Oracle version its like this: CREATE OR REPLACE TRIGGER samples_code_trg BEFORE INSERT ON samples FOR EACH ROW BEGIN SELECT TO_CHAR(seq_sample_code.nextval) INTO :NEW.sample_code FROM dual; END; I've never had much joy with BEFORE INSERT triggers on Derby. Derby doesn't allow you to change the new values in triggers, I'm afraid. As you've noticed, there's not a whole lot you can do with BEFORE triggers in Derby... -- Knut Anders
Re: NPE from InternalTriggerExecutionContext.cleanup()
Tim Dudgeon tdudg...@informaticsmatters.com writes:
On 02/09/2014 13:37, Knut Anders Hatlen wrote:
Tim Dudgeon tdudg...@informaticsmatters.com writes:
No joy. Seems like nothing extra is being logged.
From the stack trace, cleanupOnError() seems to be called from this
catch block in GenericPreparedStatement.executeStatement():
} catch (StandardException se) {
/* Cann't handle recompiling SPS action recompile here */
if
(!se.getMessageId().equals(SQLState.LANG_STATEMENT_NEEDS_RECOMPILE)
|| spsAction)
throw se;
statementContext.cleanupOnError(se);
continue recompileOutOfDatePlan;
}
It looks like the INSERT statement got a recompilation request. That's
not really an error, but an exception is raised to signal the condition
internally. The statement is supposed to be recompiled and retried
automatically after cleaning up the execution state. Unfortunately, the
trigger execution context doesn't seem to be prepared for this scenario,
and it fails with a NullPointerException in cleanup().
I've managed to reproduce the NPE in my environment, and I've filed
https://issues.apache.org/jira/browse/DERBY-6724 to track the issue.
Thanks,
Interestingly I'm seeing a similar (but not identical) error with
another trigger. This time the stack trace is this:
Tue Sep 02 13:39:09 BST 2014 Thread[SQLExecution,1,system] (XID =
62693), (SESSIONID = 1), (DATABASE =
C:/Users/timbo/Documents/IJCProjects/mini-regs/Vanilla
Oracle/.config/derby-minireg-01-sep/db), (DRDAID = null), Failed
Statement is: UPDATE samples SET sample_code = 'S123456' WHERE
sample_id = CAST
(org.apache.derby.iapi.db.Factory::getTriggerExecutionContext().getNewRow().getObject(1)
AS INTEGER)
java.lang.NullPointerException
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getTriggerActionString(Unknown
Source)
at
org.apache.derby.iapi.sql.dictionary.TriggerDescriptor.getActionSPS(Unknown
Source)
at
org.apache.derby.impl.sql.execute.GenericTriggerExecutor.getAction(Unknown
Source)
at
org.apache.derby.impl.sql.execute.RowTriggerExecutor.fireTrigger(Unknown
Source)
at
org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown
Source)
at
org.apache.derby.impl.sql.execute.UpdateResultSet.fireAfterTriggers(Unknown
Source)
at org.apache.derby.impl.sql.execute.UpdateResultSet.open(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.executeSubStatement(Unknown
Source)
at
org.apache.derby.impl.sql.execute.GenericTriggerExecutor.executeSPS(Unknown
Source)
at
org.apache.derby.impl.sql.execute.RowTriggerExecutor.fireTrigger(Unknown
Source)
at
org.apache.derby.impl.sql.execute.TriggerEventActivator.notifyEvent(Unknown
Source)
at
org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown
Source)
at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
The trigger definition is this:
CREATE TRIGGER samples_code_trg
AFTER INSERT ON samples
REFERENCING NEW AS newrow FOR EACH ROW MODE DB2SQL
UPDATE samples SET sample_code = 'S123456'
WHERE samples.sample_id = newrow.sample_id;
Is there also a trigger that is AFTER UPDATE ON samples? From the
stack trace, it looks like the error happens when an INSERT statement
triggers an UPDATE that again causes some other AFTER trigger to fire.
Unfortunately I'm unable to turn this into a simple reproducible test case.
Is it reproducible in your environment? If so, there might be more clues
(at least line numbers in the stack trace) if you run with debug-enabled
jar files. They're available from the download page, look for the
downloads named *-lib-debug.zip or *-lib-debug.tar.gz.
It might be worth a bug report even if you don't have a simple
reproducible test case, so that we have a place to collect information
about the bug.
This is with Derby 10.10.1.1
Some trigger bugs were found and fixed in 10.11 during the work on
DERBY-534 (WHEN clause in triggers). Most of them were backported and
included in the 10.10.2.0 release. I don't recall any bugs with the
stack trace you're seeing, though.
On a related front, is it possible to do this with BEFORE INSERT
trigger (the actual value for sample_code needs to come from a
sequence - the value 'S123456' is just there to simplify things)?
I don't think so. Derby doesn't
Re: NPE from InternalTriggerExecutionContext.cleanup()
On 03/09/2014 13:21, Knut Anders Hatlen wrote: Is there also a trigger that is AFTER UPDATE ON samples? From the stack trace, it looks like the error happens when an INSERT statement triggers an UPDATE that again causes some other AFTER trigger to fire. Yes there is so you're probably on the right track Unfortunately I'm unable to turn this into a simple reproducible test case. Is it reproducible in your environment? If so, there might be more clues (at least line numbers in the stack trace) if you run with debug-enabled jar files. They're available from the download page, look for the downloads named *-lib-debug.zip or *-lib-debug.tar.gz. It might be worth a bug report even if you don't have a simple reproducible test case, so that we have a place to collect information about the bug. Unfortunately I rebuilt all the tables and triggers and am now not seeing this any more. I added it to JIRA with the stacktrace: https://issues.apache.org/jira/browse/DERBY-6726 Tim
Re: NPE from InternalTriggerExecutionContext.cleanup()
Tim Dudgeon tdudg...@informaticsmatters.com writes:
No joy. Seems like nothing extra is being logged.
From the stack trace, cleanupOnError() seems to be called from this
catch block in GenericPreparedStatement.executeStatement():
} catch (StandardException se) {
/* Cann't handle recompiling SPS action recompile here */
if (!se.getMessageId().equals(SQLState.LANG_STATEMENT_NEEDS_RECOMPILE)
|| spsAction)
throw se;
statementContext.cleanupOnError(se);
continue recompileOutOfDatePlan;
}
It looks like the INSERT statement got a recompilation request. That's
not really an error, but an exception is raised to signal the condition
internally. The statement is supposed to be recompiled and retried
automatically after cleaning up the execution state. Unfortunately, the
trigger execution context doesn't seem to be prepared for this scenario,
and it fails with a NullPointerException in cleanup().
I've managed to reproduce the NPE in my environment, and I've filed
https://issues.apache.org/jira/browse/DERBY-6724 to track the issue.
Thanks,
--
Knut Anders
Re: NPE from InternalTriggerExecutionContext.cleanup()
Tim Dudgeon tdudg...@informaticsmatters.com writes: Thanks for the response. So its seems I'm probably right that my trigger is the ultimate cause. Any ideas on how to work out what is failing here as the real culprit seem to being masked. Hi Tim, If you set the system property derby.stream.error.logSeverityLevel=0, so that all errors are written to derby.log, you may get some more information. Hopefully the original error is printed before the NPE is raised. Tim On 29/08/2014 18:20, Rick Hillegas wrote: On 8/29/14 8:17 AM, Tim Dudgeon wrote: Using Derby 10.10.1.1 I'm seeing strange behaviour when doing a series of inserts that have triggers associated. About 2000 rows get inserted fine, then it blows up completely. The derby.log file contains stuff like this: Database Class Loader started - derby.database.classpath='APP.ExtraFunctions' Loaded chemaxon.misc.derby.functions.MiniregFunctions from database jar APP.EXTRAFUNCTIONS Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\timbo\Documents\IJCProjects\mini-regs\Vanilla Oracle/.config/derby-minireg-11-jun/db), (DRDAID = null), Cleanup action starting Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\ ... ), (DRDAID = null), Failed Statement is: INSERT INTO APP.COMPOUNDS ( ... ) VALUES (?,?,?,?,?,?,?,CURRENT_TIMESTAMP,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) with 31 parameters begin parameter #1: BLOB:Length=1658 :end ... java.lang.NullPointerException at org.apache.derby.impl.sql.execute.InternalTriggerExecutionContext.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.cleanUp(Unknown Source) at org.apache.derby.impl.sql.conn.GenericStatementContext.cleanupOnError(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) ... (somethings cut out for brevity) The trigger involved uses a custom java function that's been added to the DB, so the real problem might well be there, but the stack trace doesn't give much of a pointer. Any suggestions for where to look? Tim Hi Tim, This looks like a weakness in GenericStatementContext.cleanupOnError(). The header comment for that method indicates that some more work needs to be done to protect the method when it trips across subsequent errors during statement cleanup. I have filed https://issues.apache.org/jira/browse/DERBY-6722 to track this defect. If I were tracking this down, I would instrument that method to make it print out the initial error before it calls topResultSet.cleanUp(). Thanks for bringing this weakness to our attention... -Rick -- Knut Anders
Re: NPE from InternalTriggerExecutionContext.cleanup()
Thanks for the response. So its seems I'm probably right that my trigger is the ultimate cause. Any ideas on how to work out what is failing here as the real culprit seem to being masked. Tim On 29/08/2014 18:20, Rick Hillegas wrote: On 8/29/14 8:17 AM, Tim Dudgeon wrote: Using Derby 10.10.1.1 I'm seeing strange behaviour when doing a series of inserts that have triggers associated. About 2000 rows get inserted fine, then it blows up completely. The derby.log file contains stuff like this: Database Class Loader started - derby.database.classpath='APP.ExtraFunctions' Loaded chemaxon.misc.derby.functions.MiniregFunctions from database jar APP.EXTRAFUNCTIONS Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\timbo\Documents\IJCProjects\mini-regs\Vanilla Oracle/.config/derby-minireg-11-jun/db), (DRDAID = null), Cleanup action starting Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\ ... ), (DRDAID = null), Failed Statement is: INSERT INTO APP.COMPOUNDS ( ... ) VALUES (?,?,?,?,?,?,?,CURRENT_TIMESTAMP,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) with 31 parameters begin parameter #1: BLOB:Length=1658 :end ... java.lang.NullPointerException at org.apache.derby.impl.sql.execute.InternalTriggerExecutionContext.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.cleanUp(Unknown Source) at org.apache.derby.impl.sql.conn.GenericStatementContext.cleanupOnError(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) ... (somethings cut out for brevity) The trigger involved uses a custom java function that's been added to the DB, so the real problem might well be there, but the stack trace doesn't give much of a pointer. Any suggestions for where to look? Tim Hi Tim, This looks like a weakness in GenericStatementContext.cleanupOnError(). The header comment for that method indicates that some more work needs to be done to protect the method when it trips across subsequent errors during statement cleanup. I have filed https://issues.apache.org/jira/browse/DERBY-6722 to track this defect. If I were tracking this down, I would instrument that method to make it print out the initial error before it calls topResultSet.cleanUp(). Thanks for bringing this weakness to our attention... -Rick
Re: NPE from InternalTriggerExecutionContext.cleanup()
On 8/29/14 8:17 AM, Tim Dudgeon wrote: Using Derby 10.10.1.1 I'm seeing strange behaviour when doing a series of inserts that have triggers associated. About 2000 rows get inserted fine, then it blows up completely. The derby.log file contains stuff like this: Database Class Loader started - derby.database.classpath='APP.ExtraFunctions' Loaded chemaxon.misc.derby.functions.MiniregFunctions from database jar APP.EXTRAFUNCTIONS Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\timbo\Documents\IJCProjects\mini-regs\Vanilla Oracle/.config/derby-minireg-11-jun/db), (DRDAID = null), Cleanup action starting Fri Aug 29 15:37:56 BST 2014 Thread[Default RequestProcessor,1,system] (XID = 448992), (SESSIONID = 1), (DATABASE = C:\Users\ ... ), (DRDAID = null), Failed Statement is: INSERT INTO APP.COMPOUNDS ( ... ) VALUES (?,?,?,?,?,?,?,CURRENT_TIMESTAMP,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) with 31 parameters begin parameter #1: BLOB:Length=1658 :end ... java.lang.NullPointerException at org.apache.derby.impl.sql.execute.InternalTriggerExecutionContext.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.TriggerEventActivator.cleanup(Unknown Source) at org.apache.derby.impl.sql.execute.InsertResultSet.cleanUp(Unknown Source) at org.apache.derby.impl.sql.conn.GenericStatementContext.cleanupOnError(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown Source) at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) ... (somethings cut out for brevity) The trigger involved uses a custom java function that's been added to the DB, so the real problem might well be there, but the stack trace doesn't give much of a pointer. Any suggestions for where to look? Tim Hi Tim, This looks like a weakness in GenericStatementContext.cleanupOnError(). The header comment for that method indicates that some more work needs to be done to protect the method when it trips across subsequent errors during statement cleanup. I have filed https://issues.apache.org/jira/browse/DERBY-6722 to track this defect. If I were tracking this down, I would instrument that method to make it print out the initial error before it calls topResultSet.cleanUp(). Thanks for bringing this weakness to our attention... -Rick
Re: NPE
Hi Pavel,
speculating a bit, it could be that somehow the Oracle query eats up
memory so that the in-memory Derby doesn't get enough, and that somehow,
when it tries to instantiate its result set this fails. Your workaround
could work because this allows Derby to instantiate it while there is
still space on the heap. Obviously, Derby should have reported OOM if
that is indeed the problem.. Id you try to monitor the app and see if
you're ever close to the max heap size? Mind you, just guessing here...
Dag
On 11.12.2012 17:53, Pavel Bortnovskiy wrote:
Dag, et al. if I may add a bit of color to what's happening...
I've done lots of various tests (unfortunately, still unable to distill it
down to an easy-to-reproduce tester).
And what I discovered is baffling...
The issue seems to occur (with 100% certainty) under the following
circumstances:
(1) two db connections - one to Derby (in-memory only mode) and Oracle
(2) a prepared statement is executed (such as select * from table)
against Derby
(3) Oracle is queried for tables with:
final ResultSet resultSet = databaseMetaData.getTables(null, null, null, new
String[]{TABLE});
(4) if the result set from (2) is iterated over AFTER the (3) then Derby
fails with the described NPE; however, if it is iterated over BEFORE (3),
then OK!
It may also be worth noting that my resultsets are made rewindable, i.e.
with:
statement = connection.prepareStatement(
sql,
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
Finally: although I am completely perplexed as to what's causing this
problem, I found a work-around which, in my estimation, will have minimal
impact: at a particular point in my code (but effectively before calling
getTables() in (3)), the result set from (2) is iterated to the first record
and then rewound:
(not exact code, but that's what effectively is occurring):
If (resultSet.next()) {
resultSet.beforeFirst();
}
As odd as it sounds, it seems to have fixed the issue...
I am planning to return to this as time allows. Then I may be able to create
a stand-alone tester.
But if you have any thoughts as to what might be going on, I would be most
appreciative.
Regards,
Pavel.
-Original Message-
From: Dag Wanvik [mailto:dag.wan...@oracle.com]
Sent: Monday, December 10, 2012 3:15 PM
To: derby-user@db.apache.org
Subject: Re: NPE
On 10.12.2012 19:21, Pavel Bortnovskiy wrote:
I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my
previous message).
Mon Dec 10 13:19:44 EST 2012
Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5),
(DATABASE = memory:test), (DRDAID = null), Failed Statement is: select *
from READER java.lang.NullPointerException
at
org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048)
at
It looks like container is null, cf. the line 1048:
case SCAN_HOLD_INIT:
*** reusableRecordIdSequenceNumber =
container.getReusableRecordIdSequenceNumber();
i.e. has been closed, cf the assignment just above:
ContainerHandle container = open_conglom.reopen();
The call to reopen just returns null if OpenConglomerate has been closed
somehow.
Smells like a race condition of sorts, maybe some missing synchronization in
the Derby code.
If you are able to post a self contained repro program, although this might
be tricky if this is indeed a race, this would increase your chances of
having this fixed soon..
Thanks,
Dag
In any case the NullPointerException shoudln't be seen by the application
code, so this is a bug.
org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622)
at
org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322)
at
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353)
at
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308)
at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261)
at
org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801)
at
org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518)
at
org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478)
at
org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432)
at
org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376)
at QueryResults$1.hasNext(QueryResults.java:268
RE: NPE
Dag, et al. if I may add a bit of color to what's happening...
I've done lots of various tests (unfortunately, still unable to distill it down
to an easy-to-reproduce tester).
And what I discovered is baffling...
The issue seems to occur (with 100% certainty) under the following
circumstances:
(1) two db connections - one to Derby (in-memory only mode) and Oracle
(2) a prepared statement is executed (such as select * from table) against
Derby
(3) Oracle is queried for tables with:
final ResultSet resultSet = databaseMetaData.getTables(null, null, null, new
String[]{TABLE});
(4) if the result set from (2) is iterated over AFTER the (3) then Derby fails
with the described NPE; however, if it is iterated over BEFORE (3), then OK!
It may also be worth noting that my resultsets are made rewindable, i.e. with:
statement = connection.prepareStatement(
sql,
ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
Finally: although I am completely perplexed as to what's causing this problem,
I found a work-around which, in my estimation, will have minimal impact: at a
particular point in my code (but effectively before calling getTables() in
(3)), the result set from (2) is iterated to the first record and then rewound:
(not exact code, but that's what effectively is occurring):
If (resultSet.next()) {
resultSet.beforeFirst();
}
As odd as it sounds, it seems to have fixed the issue...
I am planning to return to this as time allows. Then I may be able to create a
stand-alone tester.
But if you have any thoughts as to what might be going on, I would be most
appreciative.
Regards,
Pavel.
-Original Message-
From: Dag Wanvik [mailto:dag.wan...@oracle.com]
Sent: Monday, December 10, 2012 3:15 PM
To: derby-user@db.apache.org
Subject: Re: NPE
On 10.12.2012 19:21, Pavel Bortnovskiy wrote:
I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my
previous message).
Mon Dec 10 13:19:44 EST 2012
Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5),
(DATABASE = memory:test), (DRDAID = null), Failed Statement is: select *
from READER java.lang.NullPointerException
at
org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048)
at
It looks like container is null, cf. the line 1048:
case SCAN_HOLD_INIT:
*** reusableRecordIdSequenceNumber =
container.getReusableRecordIdSequenceNumber();
i.e. has been closed, cf the assignment just above:
ContainerHandle container = open_conglom.reopen();
The call to reopen just returns null if OpenConglomerate has been closed
somehow.
Smells like a race condition of sorts, maybe some missing synchronization in
the Derby code.
If you are able to post a self contained repro program, although this might be
tricky if this is indeed a race, this would increase your chances of having
this fixed soon..
Thanks,
Dag
In any case the NullPointerException shoudln't be seen by the application
code, so this is a bug.
org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622)
at
org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322)
at
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353)
at
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308)
at
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261)
at
org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801)
at
org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518)
at
org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478)
at
org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432)
at
org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376)
at QueryResults$1.hasNext(QueryResults.java:268)
at DatabasePublisher.process(DatabasePublisher.java:318)
at java.lang.Thread.run(Thread.java:662)
-Original Message-
From: Rick Hillegas [mailto:rick.hille...@oracle.com]
Sent: Monday, December 10, 2012 1:05 PM
To: derby-user@db.apache.org
Subject: Re: NPE
Hi Pavel,
Can you reproduce this problem with the debug version of Derby? That would
give us line numbers in the stack trace.
Thanks,
-Rick
On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote:
Hello:
While testing my application, which uses Derby in memory-only mode, I
get NPE inside Derby under mysterious circumstances
Re: NPE
Hi Pavel, Can you reproduce this problem with the debug version of Derby? That would give us line numbers in the stack trace. Thanks, -Rick On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote: Hello: While testing my application, which uses Derby in memory-only mode, I get NPE inside Derby under “mysterious” circumstances. At this point, I can’t even figure out what’s triggering it and where this could happen, but let me preempt my troubleshooting with some stack traces. Perhaps someone, with good knowledge of Derby’s code, may offer some explanation. Here are some stack traces: Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main] (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) or 2012-12-10 12:12:36,537 [DatabasePublisher[WRITER]] ERROR QueryResults - Unable to execute ResultSet.next() for Statement select * from READER [DERBY] java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 14 more Caused by: java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 6 more One more thing I can share is that this seems to also be a timing problem – if the code is executed slowly, line-by-line in the debugger, then this problem doesn’t occur. But if run at full speed, then these exceptions are thrown. The problem is not intermittent and easily reproduced 100% of the time on my end. Any thoughts would be greatly appreciated. Pavel. Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the
RE: NPE
Hello, Rick: Thank you for your prompt response. I'm currently using Derby 10.8.2.2. Should I use the same version or upgrade? Can you please also advise which debug jars I can use? Thx, Pavel. -Original Message- From: Rick Hillegas [mailto:rick.hille...@oracle.com] Sent: Monday, December 10, 2012 1:05 PM To: derby-user@db.apache.org Subject: Re: NPE Hi Pavel, Can you reproduce this problem with the debug version of Derby? That would give us line numbers in the stack trace. Thanks, -Rick On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote: Hello: While testing my application, which uses Derby in memory-only mode, I get NPE inside Derby under mysterious circumstances. At this point, I can't even figure out what's triggering it and where this could happen, but let me preempt my troubleshooting with some stack traces. Perhaps someone, with good knowledge of Derby's code, may offer some explanation. Here are some stack traces: Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main] (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController. reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController. fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknow n Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(U nknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCor e(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowC ore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(U nknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) or 2012-12-10 12:12:36,537 [DatabasePublisher[WRITER]] ERROR QueryResults - Unable to execute ResultSet.next() for Statement select * from READER [DERBY] java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unkno wn Source) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException( Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unk nown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unkn own Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportA crossDRDA(Unknown Source) ... 14 more Caused by: java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController. reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController. fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknow n Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(U nknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCor e(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowC ore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(U nknown Source) ... 6 more One more thing I can share is that this seems to also be a timing problem - if the code is executed slowly, line-by-line in the debugger
RE: NPE
I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my previous message). Mon Dec 10 13:19:44 EST 2012 Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432) at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) -Original Message- From: Rick Hillegas [mailto:rick.hille...@oracle.com] Sent: Monday, December 10, 2012 1:05 PM To: derby-user@db.apache.org Subject: Re: NPE Hi Pavel, Can you reproduce this problem with the debug version of Derby? That would give us line numbers in the stack trace. Thanks, -Rick On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote: Hello: While testing my application, which uses Derby in memory-only mode, I get NPE inside Derby under mysterious circumstances. At this point, I can't even figure out what's triggering it and where this could happen, but let me preempt my troubleshooting with some stack traces. Perhaps someone, with good knowledge of Derby's code, may offer some explanation. Here are some stack traces: Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main] (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController. reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController. fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknow n Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(U nknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCor e(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowC ore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(U nknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) or 2012-12-10 12:12:36,537 [DatabasePublisher[WRITER]] ERROR QueryResults - Unable to execute ResultSet.next() for Statement select * from READER [DERBY] java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unkno wn Source) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException( Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unk nown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unkn own Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source
Re: NPE
On 10.12.2012 19:21, Pavel Bortnovskiy wrote: I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my previous message). Mon Dec 10 13:19:44 EST 2012 Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048) at It looks like container is null, cf. the line 1048: case SCAN_HOLD_INIT: *** reusableRecordIdSequenceNumber = container.getReusableRecordIdSequenceNumber(); i.e. has been closed, cf the assignment just above: ContainerHandle container = open_conglom.reopen(); The call to reopen just returns null if OpenConglomerate has been closed somehow. Smells like a race condition of sorts, maybe some missing synchronization in the Derby code. If you are able to post a self contained repro program, although this might be tricky if this is indeed a race, this would increase your chances of having this fixed soon.. Thanks, Dag In any case the NullPointerException shoudln't be seen by the application code, so this is a bug. org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432) at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) -Original Message- From: Rick Hillegas [mailto:rick.hille...@oracle.com] Sent: Monday, December 10, 2012 1:05 PM To: derby-user@db.apache.org Subject: Re: NPE Hi Pavel, Can you reproduce this problem with the debug version of Derby? That would give us line numbers in the stack trace. Thanks, -Rick On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote: Hello: While testing my application, which uses Derby in memory-only mode, I get NPE inside Derby under mysterious circumstances. At this point, I can't even figure out what's triggering it and where this could happen, but let me preempt my troubleshooting with some stack traces. Perhaps someone, with good knowledge of Derby's code, may offer some explanation. Here are some stack traces: Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main] (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController. reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController. fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknow n Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(U nknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCor e(Unknown Source) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowC ore(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wFromSource(Unknown Source) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRo wCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(U nknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) or 2012-12-10 12:12:36,537 [DatabasePublisher[WRITER]] ERROR QueryResults - Unable to execute ResultSet.next() for Statement select * from READER [DERBY] java.sql.SQLException: Java exception
RE: NPE
Thanks, Dag. Yeah, I wish I could figure out how to make a self-contained test prog which can reproduce it. I will try. Although I do agree with you that it looks like a race condition, I can reproduce it (within my program) with 100% certainty - every time. However, it might be worth stating the following: my program is written as plug-n-play modules. When I plug in any other module, the error never occurs (or is well-masked). It's only with this newly written module, that it started to fail. In the meantime, what might be causing the container to be NULL? Is it possible that there is a bug in my program which sets it to NULL (or does not assign it) in such a way that other parts of Derby do not fail nor notice that it is NULL? -Original Message- From: Dag Wanvik [mailto:dag.wan...@oracle.com] Sent: Monday, December 10, 2012 3:15 PM To: derby-user@db.apache.org Subject: Re: NPE On 10.12.2012 19:21, Pavel Bortnovskiy wrote: I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my previous message). Mon Dec 10 13:19:44 EST 2012 Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048) at It looks like container is null, cf. the line 1048: case SCAN_HOLD_INIT: *** reusableRecordIdSequenceNumber = container.getReusableRecordIdSequenceNumber(); i.e. has been closed, cf the assignment just above: ContainerHandle container = open_conglom.reopen(); The call to reopen just returns null if OpenConglomerate has been closed somehow. Smells like a race condition of sorts, maybe some missing synchronization in the Derby code. If you are able to post a self contained repro program, although this might be tricky if this is indeed a race, this would increase your chances of having this fixed soon.. Thanks, Dag In any case the NullPointerException shoudln't be seen by the application code, so this is a bug. org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801) at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432) at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376) at QueryResults$1.hasNext(QueryResults.java:268) at DatabasePublisher.process(DatabasePublisher.java:318) at java.lang.Thread.run(Thread.java:662) -Original Message- From: Rick Hillegas [mailto:rick.hille...@oracle.com] Sent: Monday, December 10, 2012 1:05 PM To: derby-user@db.apache.org Subject: Re: NPE Hi Pavel, Can you reproduce this problem with the debug version of Derby? That would give us line numbers in the stack trace. Thanks, -Rick On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote: Hello: While testing my application, which uses Derby in memory-only mode, I get NPE inside Derby under mysterious circumstances. At this point, I can't even figure out what's triggering it and where this could happen, but let me preempt my troubleshooting with some stack traces. Perhaps someone, with good knowledge of Derby's code, may offer some explanation. Here are some stack traces: Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main] (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID = null), Failed Statement is: select * from READER java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericScanController. reopenAfterEndTransaction(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController. fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unkno w n Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray( U nknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo r e
Re: NPE when compressing large table
I agree should log a JIRA then all the information and suggestions can
go there. Having a line number would help a lot for exact cause of the
null pointer.
Historically SYSCS_COMPRESS_TABLE has been the most stable of the 2
compression techniques, and is the one I most recommend.
The lowest part of the stack looks kind of wierd, but hard to know
exactly what is going on as part of the stack is missing. It sort of
looks like it is failing while setting up a connection rather than in
your code. Can you tell from the line numbers where in your code it
is failing?
Also include full derby.log, sometimes that can give more information.
I can't tell from your program fragment. Is the connection autocommit
on or off? If it is on, there might be a problem with your resultset.
On 9/4/2012 7:39 PM, Brett Wooldridge wrote:
This looks like a bug to me, I recommend you open one. You might also
try SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE. If you can copy the DB to
your dev environment, reproduce the issue, and run a derby 10.8.2.2
built with line numbers, it would probably help the devs pinpoint the issue.
Brett
On Wed, Sep 5, 2012 at 5:11 AM, Erick Lichtas elich...@lichtasweb.com
mailto:elich...@lichtasweb.com wrote:
Hi Everyone,
__ __
I am having an issue with the SYSCS_UTIL.SYSCS_COMPRESS_TABLE
operation on a large table in Derby 10.8.2.2.
__ __
Statement stmt = *null*;
CallableStatement cs = *null*;
*try*{
String sql = select schemaname, tablename from
sys.sysschemas s,
+ sys.systables t where s.schemaid=t.schemaid and
t.tabletype='T';
stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(sql);
cs = con.prepareCall(call
SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?));
*while*(rs.next()) {
String schema = rs.getString(1).trim();
String table = rs.getString(2).trim();
log(Compressing + schema + .+ table + ...);
cs.setString(1, schema);
cs.setString(2, table);
cs.setShort(3, (*short*) 1);
cs.execute();
}
}
*finally*{
*if*(stmt != *null*) {
stmt.close();
}
*if*(cs != *null*) {
cs.close();
}
}
__ __
The above code, successfully runs over 3 of 20 some tables, then
hits the 4^th table with over 2 million records. After working on
this table for about 5 minutes, the process terminates with an
SQLException wrapping a NPE.
This happened in a production instance and I have not yet set up a
reproduction in a development environment. I’m wondering if anyone
has any thoughts based on the stacktrace below?
__ __
com.linoma.gaservices.upgrader.UpgradeException: Java exception: ':
java.lang.NullPointerException'.
at
com.linoma.gaservices.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.java:227)
at
com.linoma.gaservices.upgrader.UnixUpgrader.upgrade(UnixUpgrader.java:32)
at
com.linoma.gaservices.upgrader.UpgradeStarter.startUpgrade(UpgradeStarter.java:25)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at my.app.upgrader.Startup.main(Startup.java:72)
Caused by: java.sql.SQLException: Java exception: ':
java.lang.NullPointerException'.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at
org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.init(Unknown Source)
at
org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown Source)
at
org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
Re: NPE when compressing large table
This looks like a bug to me, I recommend you open one. You might also try
SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE. If you can copy the DB to your
dev environment, reproduce the issue, and run a derby 10.8.2.2 built with
line numbers, it would probably help the devs pinpoint the issue.
Brett
On Wed, Sep 5, 2012 at 5:11 AM, Erick Lichtas elich...@lichtasweb.comwrote:
Hi Everyone,
** **
I am having an issue with the SYSCS_UTIL.SYSCS_COMPRESS_TABLE operation on
a large table in Derby 10.8.2.2.
** **
Statement stmt = *null*;
CallableStatement cs = *null*;
*try* {
String sql = select schemaname, tablename from sys.sysschemas s,
+ sys.systables t where s.schemaid=t.schemaid and
t.tabletype='T';
stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(sql);
cs = con.prepareCall(call SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?,
?));
*while* (rs.next()) {
String schema = rs.getString(1).trim();
String table = rs.getString(2).trim();
log(Compressing + schema + . + table + ...);
cs.setString(1, schema);
cs.setString(2, table);
cs.setShort(3, (*short*) 1);
cs.execute();
}
}
*finally* {
*if* (stmt != *null*) {
stmt.close();
}
*if* (cs != *null*) {
cs.close();
}
}
** **
The above code, successfully runs over 3 of 20 some tables, then hits the 4
th table with over 2 million records. After working on this table for
about 5 minutes, the process terminates with an SQLException wrapping a
NPE.
This happened in a production instance and I have not yet set up a
reproduction in a development environment. I’m wondering if anyone has any
thoughts based on the stacktrace below?
** **
com.linoma.gaservices.upgrader.UpgradeException: Java exception: ':
java.lang.NullPointerException'.
at
com.linoma.gaservices.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.java:227)
at
com.linoma.gaservices.upgrader.UnixUpgrader.upgrade(UnixUpgrader.java:32)*
***
at
com.linoma.gaservices.upgrader.UpgradeStarter.startUpgrade(UpgradeStarter.java:25)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at my.app.upgrader.Startup.main(Startup.java:72)
Caused by: java.sql.SQLException: Java exception: ':
java.lang.NullPointerException'.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.init(Unknown Source)
at
org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown
Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown
Source)
at
java.sql.DriverManager.getConnection(DriverManager.java:572)
at
java.sql.DriverManager.getConnection(DriverManager.java:218)
at
my.app.upgrader.db.DerbyDatabaseUpgrader.closeDB(DerbyDatabaseUpgrader.java:99)
at
my.app.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.java:223)
... 7 more
Caused by: java.lang.NullPointerException
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
Source)
at
org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)***
*
at
org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown
Source)
at
org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
at
Re: NPE in BasicSortObserver.getClone() when performing query with HAVING clause
Hi Knut, Ok I will. Any ideas on the cause so I can work-around it? Thanks, -- Matt (from phone) On 13/06/2010, at 3:05 AM, Knut Anders Hatlen knut.hat...@sun.com wrote: On 06/12/10 09:32 AM, Matt Doran wrote: Hi there, I've just tested this with 10.5.3.0 and it also fails. This time with a different stack-trace (see below). This time the NPE is in ColumnReference.remapColumnReferencesToExpressions().I've also attached the derby log for this crash. Given this looks like a serious bug in the latest release, I've uploaded a zipped copy of the database to a free file sharing site ... you can get it at the link below: http://www.mediafire.com/file/id0ljd5davj/derby.zip java.lang.NullPointerException at Hi Matt, I'm able to reproduce this with 10.6.1.0 and with the latest development sources. Please file a bug report at https://issues.apache.org/jira/browse/DERBY so that we don't lose track of it. Thanks, -- Knut Anders
Re: NPE in BasicSortObserver.getClone() when performing query with HAVING clause
Hi there, I've just tested this with 10.5.3.0 and it also fails. This time with a different stack-trace (see below). This time the NPE is in ColumnReference.remapColumnReferencesToExpressions().I've also attached the derby log for this crash. Given this looks like a serious bug in the latest release, I've uploaded a zipped copy of the database to a free file sharing site ... you can get it at the link below: http://www.mediafire.com/file/id0ljd5davj/derby.zip java.lang.NullPointerException at org.apache.derby.impl.sql.compile.ColumnReference.remapColumnReferencesToExpressions(Unknown Source) at org.apache.derby.impl.sql.compile.AggregateNode.getNewExpressionResultColumn(Unknown Source) at org.apache.derby.impl.sql.compile.GroupByNode.addAggregateColumns(Unknown Source) at org.apache.derby.impl.sql.compile.GroupByNode.addNewColumnsForAggregation(Unknown Source) at org.apache.derby.impl.sql.compile.GroupByNode.addAggregates(Unknown Source) at org.apache.derby.impl.sql.compile.GroupByNode.init(Unknown Source) at org.apache.derby.iapi.sql.compile.NodeFactory.getNode(Unknown Source) at org.apache.derby.impl.sql.compile.SelectNode.genProjectRestrict(Unknown Source) at org.apache.derby.impl.sql.compile.SelectNode.modifyAccessPaths(Unknown Source) at org.apache.derby.impl.sql.compile.DMLStatementNode.optimizeStatement(Unknown Source) at org.apache.derby.impl.sql.compile.CursorNode.optimizeStatement(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source) at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.tools.ij.ij.executeImmediate(Unknown Source) at org.apache.derby.impl.tools.ij.utilMain.doCatch(Unknown Source) at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(Unknown Source) at org.apache.derby.impl.tools.ij.utilMain.go(Unknown Source) at org.apache.derby.impl.tools.ij.Main.go(Unknown Source) at org.apache.derby.impl.tools.ij.Main.mainCore(Unknown Source) at org.apache.derby.impl.tools.ij.Main.main(Unknown Source) at org.apache.derby.tools.ij.main(Unknown Source) PS: The sql got a bit munged last time because my email used HTML email ... so here it is again: SELECT user0_.user_id AS col_0_0_, SUM(account2_.balance) AS col_1_0_ FROM tbl_user user0_ INNER JOIN tbl_user_account accountlin1_ ON user0_.user_id = accountlin1_.user_id INNER JOIN tbl_account account2_ ON accountlin1_.account_id = account2_.account_id WHERE user0_.deleted = 'N' AND ( account2_.account_type IN ( 'USER-01', 'USER' ) ) GROUP BY user0_.user_id HAVING SUM(account2_.balance) = 100.0 Regards, Matt On 12/06/2010 5:04 PM, Matt Doran wrote: Hi there, I have a relatively simple query that triggers a NPE in derby. The query is generated out of Hibernate, but it's pretty easy to understand (calculating users with total balances above 100). See below. SELECT user0_.user_id AS col_0_0_, /SUM/(account2_.balance) AS col_1_0_ FROM tbl_user user0_ INNER JOIN tbl_user_account accountlin1_ ON user0_.user_id = accountlin1_.user_id INNER JOIN tbl_account account2_ ON accountlin1_.account_id = account2_.account_id WHERE user0_.deleted = 'N' AND ( account2_.account_type IN ( 'USER-01', 'USER' ) ) GROUP BY user0_.user_id HAVING /SUM/(account2_.balance) = 100.0 When I run this I get the following a nasty NPE ... see stack trace below. If I remove the HAVING clause the query executes successfully.I've attached a derby log with the query plans on showing firstly the successful query without the HAVING clause and then the stacktrace running the above query. I'm running 10.4.2.0.And can provider anyone interested a copy of the DB if that would help (it's only small - 500kb). Any clues on this one? java.lang.NullPointerException at org.apache.derby.impl.sql.execute.BasicSortObserver.getClone(Unknown Source) at org.apache.derby.impl.sql.execute.BasicSortObserver.insertNonDuplicateKey(Unknown Source) at org.apache.derby.impl.sql.execute.AggregateSortObserver.insertNonDuplicateKey(Unknown Source) at org.apache.derby.impl.store.access.sort.SortBuffer.insert(Unknown Source) at org.apache.derby.impl.store.access.sort.MergeInserter.insert(Unknown Source) at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.loadSorter(Unknown
Re: NPE in BasicSortObserver.getClone() when performing query with HAVING clause
On 06/12/10 09:32 AM, Matt Doran wrote: Hi there, I've just tested this with 10.5.3.0 and it also fails. This time with a different stack-trace (see below). This time the NPE is in ColumnReference.remapColumnReferencesToExpressions().I've also attached the derby log for this crash. Given this looks like a serious bug in the latest release, I've uploaded a zipped copy of the database to a free file sharing site ... you can get it at the link below: http://www.mediafire.com/file/id0ljd5davj/derby.zip java.lang.NullPointerException at Hi Matt, I'm able to reproduce this with 10.6.1.0 and with the latest development sources. Please file a bug report at https://issues.apache.org/jira/browse/DERBY so that we don't lose track of it. Thanks, -- Knut Anders
Antwort: Re: NPE in JBitSet
Hi Bryan, thanks for the reply. I submitted the bug as #4549. https://issues.apache.org/jira/browse/DERBY-4549 Regards, Murat Von: Bryan Pendleton bpendle...@amberpoint.com An: Derby Discussion derby-user@db.apache.org Datum: 16.02.2010 02:39 Betreff: Re: NPE in JBitSet I'm getting a NPE with Derby 10.5.3.0. Is there a known workaround or can I post this as a bug at JIRA? Hi Murat, I see the same problem with the current Derby trunk, so this seems like a new bug to me. If you could post it as a bug at JIRA that would be very much appreciated. Please include your test case, and the Derby stack trace, just as you did in your email; it was very clearly presented, thank you! Regarding a workaround, I'm afraid I don't know of any. Something has gone wrong in the data structures in the SQL compiler, but it will take some study to figure out exactly what. thanks, bryan
Re: NPE in JBitSet
I'm getting a NPE with Derby 10.5.3.0. Is there a known workaround or can I post this as a bug at JIRA? Hi Murat, I see the same problem with the current Derby trunk, so this seems like a new bug to me. If you could post it as a bug at JIRA that would be very much appreciated. Please include your test case, and the Derby stack trace, just as you did in your email; it was very clearly presented, thank you! Regarding a workaround, I'm afraid I don't know of any. Something has gone wrong in the data structures in the SQL compiler, but it will take some study to figure out exactly what. thanks, bryan
Re: NPE from NoRowsResultSetImpl.close()
MartinEW wrote:
I am receiving a NPE when attempting to delete records from tables. I have
attached a simplified SQL script that replicates my problem -
http://www.nabble.com/file/p21725256/script.sql script.sql . Can anyone
else replicate this issue?
Hello Martin,
I was unable to reproduce when I tried to run the script with Derby
trunk, 10.4.2.1 and 10.3.3.0. I did this on OpenSolaris with Java SE 6.
I connected to an empty database with the network driver, using ij and
then ran the script.
Can you specify the Derby version you used, and on which operating
system you saw the problem?
--
Kristian
The exception is:
2009-01-29 11:22:51.346 GMT Thread[DRDAConnThread_20,5,derby.daemons] (XID =
2515), (SESSIONID = 35), (DATABASE = test), (DRDAID =
NF01.E18A-522134848570106079{35}), Cleanup action starting
2009-01-29 11:22:51.346 GMT Thread[DRDAConnThread_20,5,derby.daemons] (XID =
2515), (SESSIONID = 35), (DATABASE = test), (DRDAID =
NF01.E18A-522134848570106079{35}), Failed Statement is: delete from
collectives
java.lang.NullPointerException
at
org.apache.derby.impl.sql.execute.NoRowsResultSetImpl.close(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.open(Unknown
Source)
at
org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLIMM(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
2009-01-29 11:22:51.346 GMT Thread[DRDAConnThread_20,5,derby.daemons]
Equally severe exception raised during cleanup (ignored) null
java.lang.NullPointerException
at
org.apache.derby.impl.sql.execute.NoRowsResultSetImpl.close(Unknown Source)
at org.apache.derby.impl.sql.execute.DeleteResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.execute.DeleteCascadeResultSet.cleanUp(Unknown
Source)
at
org.apache.derby.impl.sql.conn.GenericStatementContext.cleanupOnError(Unknown
Source)
at
org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.cleanupOnError(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at
org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLIMM(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Cleanup action completed
Thanks
Martin
Re: NPE from NoRowsResultSetImpl.close()
Kristian Waagan wrote: I was unable to reproduce when I tried to run the script with Derby trunk, 10.4.2.1 and 10.3.3.0. I did this on OpenSolaris with Java SE 6. I connected to an empty database with the network driver, using ij and then ran the script. Where did you get 10.4.2.1 from? I can only find 10.4.2.0 on the Derby site. Thanks, -- Alan Burlison --
Re: NPE from NoRowsResultSetImpl.close()
Alan Burlison wrote: Kristian Waagan wrote: I was unable to reproduce when I tried to run the script with Derby trunk, 10.4.2.1 and 10.3.3.0. I did this on OpenSolaris with Java SE 6. I connected to an empty database with the network driver, using ij and then ran the script. Where did you get 10.4.2.1 from? I can only find 10.4.2.0 on the Derby site. Hi Alan, Sorry, I used Derby trunk and Derby 10.3.3.0. The last one I tried with was Java DB 10.4.2.1, which can be found on the Sun Developer Network site. Regards, -- Kristian Thanks,
Re: NPE from NoRowsResultSetImpl.close()
MartinEW martin.wa...@sun.com writes: I am receiving a NPE when attempting to delete records from tables. I have attached a simplified SQL script that replicates my problem - http://www.nabble.com/file/p21725256/script.sql script.sql . Can anyone else replicate this issue? Thanks for the script, I can reproduce it when I set derby.language.logQueryPlan=true. I think this is https://issues.apache.org/jira/browse/DERBY-2353, which is closed as Cannot Reproduce. I'll reopen it and post a pointer to this discussion. The workaround is to not set derby.language.logQueryPlan. -- Knut Anders
Re: NPE from NoRowsResultSetImpl.close()
Hi Kristian, Can you specify the Derby version you used, and on which operating system you saw the problem? Derby - 10.4.2.0 OS - Solaris snv103 J2SE 6 When creating a fresh DB and running the script it worked for me also. I was using a DB that was already there. But once I applied the following properties it starts to fail again. I am trying to identify which property is causing the issue now. -- Only use properties stored in the database. call syscs_util.syscs_set_database_property( 'derby.database.propertiesOnly', 'true'); -- Set access permissions. call syscs_util.syscs_set_database_property( 'derby.authentication.provider', 'BUILTIN'); call syscs_util.syscs_set_database_property( 'derby.connection.requireAuthentication', 'true'); call syscs_util.syscs_set_database_property( 'derby.database.fullAccessUsers', 'auth'); call syscs_util.syscs_set_database_property( 'derby.database.readOnlyAccessUsers', 'readonly'); -- Create the readonly user. call syscs_util.syscs_set_database_property( 'derby.user.readonly', 'readonly'); -- Drop the default schema. drop schema app restrict; -- Change the page size for the database. -- call syscs_util.syscs_set_database_property( -- 'derby.storage.pageSize', '4096'); -- Save query plans - only for development. call syscs_util.syscs_set_database_property( 'derby.language.logQueryPlan', 'true'); commit; Martin
Re: NPE from NoRowsResultSetImpl.close()
Martin Walsh wrote: Hi Kristian, Can you specify the Derby version you used, and on which operating system you saw the problem? Derby - 10.4.2.0 OS - Solaris snv103 J2SE 6 When creating a fresh DB and running the script it worked for me also. I was using a DB that was already there. But once I applied the following properties it starts to fail again. I am trying to identify which property is causing the issue now. -- Only use properties stored in the database. call syscs_util.syscs_set_database_property( 'derby.database.propertiesOnly', 'true'); -- Set access permissions. call syscs_util.syscs_set_database_property( 'derby.authentication.provider', 'BUILTIN'); call syscs_util.syscs_set_database_property( 'derby.connection.requireAuthentication', 'true'); call syscs_util.syscs_set_database_property( 'derby.database.fullAccessUsers', 'auth'); call syscs_util.syscs_set_database_property( 'derby.database.readOnlyAccessUsers', 'readonly'); -- Create the readonly user. call syscs_util.syscs_set_database_property( 'derby.user.readonly', 'readonly'); -- Drop the default schema. drop schema app restrict; -- Change the page size for the database. -- call syscs_util.syscs_set_database_property( -- 'derby.storage.pageSize', '4096'); -- Save query plans - only for development. call syscs_util.syscs_set_database_property( 'derby.language.logQueryPlan', 'true'); Thanks Martin, This seems to confirm what Knut Anders reported; https://issues.apache.org/jira/browse/DERBY-2353 -- Kristian commit; Martin
Re: NPE from NoRowsResultSetImpl.close()
Kristian Waagan wrote: Where did you get 10.4.2.1 from? I can only find 10.4.2.0 on the Derby site. Sorry, I used Derby trunk and Derby 10.3.3.0. The last one I tried with was Java DB 10.4.2.1, which can be found on the Sun Developer Network site. Interesting, I didn't know there was a later version on there. I'll keep an eye on SDN for future updates, thanks. -- Alan Burlison --
Re: NPE on an innocent query
Hi Stanislav,
I'm also seeing an NPE when I run this script. I have logged
https://issues.apache.org/jira/browse/DERBY-3904 to track this issue.
Thanks for finding this,
-Rick
Stanislav Bryzgalov wrote:
Hello,
I am getting an NPE when trying to run a pretty innocent query. I
suspect this might be because of the platform I am using which is JDK
1.5 diablo on FreeBSD. I got it first on 10.3.2.1 http://10.3.2.1,
then upgraded to 10.4.2.0 http://10.4.2.0 but the problem did not go
away.
After some playing, I narrowed the problem down do a pretty simple
example. Can someone run it please and share results.
-- create two simple tables
CREATE TABLE T1( D1 DATE NOT NULL PRIMARY KEY, N1 VARCHAR( 10 ) );
CREATE TABLE T2( D2 DATE NOT NULL PRIMARY KEY, N2 VARCHAR( 10 ) );
-- insert some data, two recs in T1 and one in T2
INSERT INTO T1 VALUES( DATE( '2008-10-01' ), 'something' ), ( DATE(
'2008-10-02' ), 'something' );
INSERT INTO T2 VALUES( DATE( '2008-10-01' ), 'something' );
-- this runs fine, gives one record '2008-10-02'
SELECT T1.D1
FROM T1
LEFT JOIN T2
ON T1.D1 = T2.D2
WHERE T2.D2 IS NULL;
-- this runs fine too, gives one record '2008-10-02'
SELECT MAX( T1.D1 ) as D
FROM T1
WHERE T1.D1 NOT IN ( SELECT T2.D2 FROM T2 );
-- this one breaks!!!
-- SQL State = XJ001 SQL Code = -1 SQL Message = DERBY SQL error:
SQLCODE: -1, SQLSTATE: XJ001, SQLERRMC: java.lang.NullPointerException
SELECT MAX( T1.D1 ) AS D
FROM T1
LEFT JOIN T2
ON T1.D1 = T2.D2
WHERE T2.D2 IS NULL;
My derby.log has this:
2008-10-09 17:11:58.357 GMT Thread[DRDAConnThread_3,5,main] (XID =
7676), (SESSIONID = 0), (DATABASE = coco2), (DRDAID =
NF01.E6C3-4327113593400890529{19}), Failed Statement is: SELECT
MAX( T1.D1 ) AS D
FROM T1
LEFT JOIN T2
ON T1.D1 = T2.D2
WHERE T2.D2 IS NULL
java.lang.NullPointerException
at
org.apache.derby.impl.sql.compile.GroupByNode.considerPostOptimizeOptimizations(Unknown
Source)
at
org.apache.derby.impl.sql.compile.SelectNode.genProjectRestrict(Unknown
Source)
at
org.apache.derby.impl.sql.compile.SelectNode.modifyAccessPaths(Unknown
Source)
at
org.apache.derby.impl.sql.compile.DMLStatementNode.optimizeStatement(Unknown
Source)
at
org.apache.derby.impl.sql.compile.CursorNode.optimizeStatement(Unknown
Source)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.init(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.init(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.init(Unknown
Source)
at org.apache.derby.jdbc.Driver30.newEmbedPreparedStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
at org.apache.derby.impl.drda.DRDAStatement.prepare(Unknown Source)
at org.apache.derby.impl.drda.DRDAStatement.explicitPrepare(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parsePRPSQLSTT(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
WBR,
Stanislav
Re: NPE getting length of Blob
Daniel Noll wrote: Hi all. I have someone seeing the following error under both 10.3.1.4 and 10.3.2.1: java.lang.NullPointerException at org.apache.derby.client.am.PreparedStatement.setIntX(Unknown Source) at org.apache.derby.client.am.CallableLocatorProcedures.blobGetLength(Unknown Source) at org.apache.derby.client.am.Blob.getLocatorLength(Unknown Source) at org.apache.derby.client.am.Lob.sqlLength(Unknown Source) at org.apache.derby.client.am.Blob.length(Unknown Source) Has anyone seen something like this before? What is the cause of the problem? The exact same code on the exact same version of Derby doesn't cause the same problem over here; could it be something which only happens when unusual network conditions are happening, or something which only happens when the database meets a certain condition? Daniel Can you give more information about the code that uses this? Can you check what version is the derby network server? What version is the network client? This information will be very useful if we try to single step the code to find out the problem source. From the attached information I find it a difficult to speculate on the source of the failure :( Narayanan
Re: NPE getting length of Blob
Daniel Noll wrote: Hi all. I have someone seeing the following error under both 10.3.1.4 and 10.3.2.1: java.lang.NullPointerException at org.apache.derby.client.am.PreparedStatement.setIntX(Unknown Source) at org.apache.derby.client.am.CallableLocatorProcedures.blobGetLength(Unknown Source) at org.apache.derby.client.am.Blob.getLocatorLength(Unknown Source) at org.apache.derby.client.am.Lob.sqlLength(Unknown Source) at org.apache.derby.client.am.Blob.length(Unknown Source) Has anyone seen something like this before? What is the cause of the problem? The exact same code on the exact same version of Derby doesn't cause the same problem over here; could it be something which only happens when unusual network conditions are happening, or something which only happens when the database meets a certain condition? Daniel I have tried getting a blob from a result set and tried the length() method on it after the following Scenarios 1) Closing the connection 2) committing the transaction 3) Rolling back the transaction In the above three I got the expected SQLState. 4) Tried shutting down the network server just before blob.length I get the following Exception in thread main java.sql.SQLNonTransientConnectionException: Insufficient data while reading from the network - expected a minimum of 6 bytes and received only -1 bytes. The connection has been terminated. at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:70) at org.apache.derby.client.am.SqlException.getSQLException(SqlException.java:362) at org.apache.derby.client.am.Blob.length(Blob.java:141) at javaapplication2.Main.main(Main.java:31) None of the above scenarios seem to be giving the exception pointed out. Narayanan
Re: NPE getting length of Blob
On Tuesday 04 March 2008 21:14:43 Narayanan wrote:
Can you give more information about the code that uses this?
It looks like this. I've omitted some of the code which was present as it was
calling framework methods anyway.
private long id;
public int read(byte[] b, int off, int len) {
Connection conn = (get from pool)
try {
PreparedStatement ps = conn.prepareStatement(SELECT data FROM binaries
WHERE id = ?);
ResultSet rs = null;
try {
ps.setLong(1, id);
rs = ps.executeQuery();
if (rs.next()) {
Blob blob = rs.getBlob(data);
int pos = pointer + 1;
int length = (int) Math.min(blob.length() - pointer, len);//throws NPE
if (length == 0) {
return -1;
} else {
byte[] tmp = blob.getBytes(pos, length);
System.arraycopy(tmp, 0, b, off, length);
pointer += length;
return length;
}
}
} finally {
if (rs != null) rs.close();
ps.close();
}
} finally {
conn.commit();
conn.close();
}
}
This is part of an InputStream implementation so it's being called multiple
times in a row with different offset and length.
If I try this on a fresh connection by itself, it doesn't occur. The only
thing that really seems unique about running it in the application itself is
that we're accessing it through a connection pool. I'm still attempting to
get it to happen standalone so I can open a proper bug record for it.
Can you check what version is the derby network server? What version is
the network client?
They are the versions I specified in the previous mail. In both situations I
used the same version for the server and the client; I haven't tried one
version of the server against the other version of the client (nor vice
versa.)
This information will be very useful if we try to single step the code
to find out
the problem source.
I was able to finally reproduce the problem on my own machine so it isn't
dependent on the network, and I have confirmed that it isn't dependent on the
type of data in the database either although since I have only managed to
reproduce it in our own application, all databases tested have more or less
the same schema.
The same code on the embedded version does work. Additionally the same code
on 10.2.2.0 client/server works fine too.
Daniel
Re: NPE getting length of Blob
On Tuesday 04 March 2008 22:36:17 Narayanan wrote: I have tried getting a blob from a result set and tried the length() method on it after the following Scenarios 1) Closing the connection The underlying problem must be something to do with Connection reuse. We discovered that am.PreparedStatement.parameterMetaData_ was being set to null in markClosed(), and yet the same PreparedStatement object was being reused for subsequent blob length queries. So it would work once and then not for any subsequent attempts. Subsequent attempts for the same blob would probably work because the result is cached elsewhere but attempts for a different blob would use the same cached, invalid PreparedStatement object. Here's who's setting it to null: at org.apache.derby.client.am.PreparedStatement.markClosed(PreparedStatement.java:2526); at org.apache.derby.client.am.Statement.markClosed(Statement.java:1755) at org.apache.derby.client.am.Connection.markStatementsClosed(Connection.java:879) at org.apache.derby.client.am.Connection.markClosed(Connection.java:864) at org.apache.derby.client.am.Connection.markClosedForReuse(Connection.java:872) at org.apache.derby.client.am.Connection.closeForReuse(Connection.java:833) at org.apache.derby.client.am.NetConnection.closeForReuse(NetConnection.java:1829) at org.apache.derby.client.am.LogicalConnection.close(LogicalConnection.java:80) We're calling Connection.close() but it's a connection from a pool, and you're supposed to close those. Potential fixes: - Don't set things to null during close methods (it's inherently dangerous.) - Check if the statement is closed after checking whether it's null (possibly defeats the purpose of caching the statement for the connection.) Anyway now that we know roughly where the problem is, some Derby developer will presumably know better than us how to fix it without breaking anything else. Daniel
Re: NPE getting length of Blob
Hi Daniel, Which pool/version are you using? Also how are you configuring the pool? You want to make sure your library is wrapping the connection or if not; is interfacing with the pooled connection/data source object. Raymond On Tue, Mar 4, 2008 at 4:36 PM, Daniel Noll [EMAIL PROTECTED] wrote: On Tuesday 04 March 2008 22:36:17 Narayanan wrote: I have tried getting a blob from a result set and tried the length() method on it after the following Scenarios 1) Closing the connection The underlying problem must be something to do with Connection reuse. We discovered that am.PreparedStatement.parameterMetaData_ was being set to null in markClosed(), and yet the same PreparedStatement object was being reused for subsequent blob length queries. So it would work once and then not for any subsequent attempts. Subsequent attempts for the same blob would probably work because the result is cached elsewhere but attempts for a different blob would use the same cached, invalid PreparedStatement object. Here's who's setting it to null: at org.apache.derby.client.am.PreparedStatement.markClosed( PreparedStatement.java:2526); at org.apache.derby.client.am.Statement.markClosed(Statement.java :1755) at org.apache.derby.client.am.Connection.markStatementsClosed(Connection.java :879) at org.apache.derby.client.am.Connection.markClosed(Connection.java :864) at org.apache.derby.client.am.Connection.markClosedForReuse(Connection.java :872) at org.apache.derby.client.am.Connection.closeForReuse( Connection.java:833) at org.apache.derby.client.am.NetConnection.closeForReuse(NetConnection.java :1829) at org.apache.derby.client.am.LogicalConnection.close(LogicalConnection.java :80) We're calling Connection.close() but it's a connection from a pool, and you're supposed to close those. Potential fixes: - Don't set things to null during close methods (it's inherently dangerous.) - Check if the statement is closed after checking whether it's null (possibly defeats the purpose of caching the statement for the connection.) Anyway now that we know roughly where the problem is, some Derby developer will presumably know better than us how to fix it without breaking anything else. Daniel -- - Raymond Kroeker
Re: NPE getting length of Blob
On Wednesday 05 March 2008 12:43:44 Raymond Kroeker wrote:
Hi Daniel,
Which pool/version are you using? Also how are you configuring the
pool? You want to make sure your library is wrapping the connection or
if not; is interfacing with the pooled connection/data source object.
Data source configuration is... (parameters is our own bean for storing this
stuff...)
ClientConnectionPoolDataSource dataSource =
new ClientConnectionPoolDataSource();
dataSource.setServerName(parameters.getHost());
dataSource.setPortNumber(parameters.getPort());
dataSource.setDatabaseName(parameters.getPath());
We're using MCPM as the pool manager, it's just a simple connection pool
manager in a single Java class which uses PooledConnection to do the hard
stuff.
Relevant code is probably:
private synchronized Connection getConnection2() throws SQLException {
PooledConnection pconn;
if (!recycledConnections.empty()) {
pconn = recycledConnections.pop();
} else {
pconn = dataSource.getPooledConnection();
}
Connection conn = pconn.getConnection();
activeConnections++;
return conn;
}
As it's going via PooledConnection I was expecting it to be reusable after
closing. This works in practice for the embedded version anyway.
I've checked the MCPM code and close() is only called during disposeConnection
and certainly not during recycleConnection.
Daniel
Re: NPE getting length of Blob
Hi Daniel,
What appears to be happening is that the same connection is being
returned by getConnection2(). Basically the implementation of the pool is
not thread safe. The NPE is thrown when the statement attempts to access
its internal parameter meta data which is nulled out when the underlying
physical connection is reset (flagged for reuse). The conclusion is that
the pool implementation is not safely borrowing/returning connections.
I've had luck using commons-dbcp in conjunction with commons-pool for
server side connection pooling when connecting to a derby database server.
http://commons.apache.org/dbcp
http://commons.apache.org/pool
On Tue, Mar 4, 2008 at 5:52 PM, Daniel Noll [EMAIL PROTECTED] wrote:
On Wednesday 05 March 2008 12:43:44 Raymond Kroeker wrote:
Hi Daniel,
Which pool/version are you using? Also how are you configuring the
pool? You want to make sure your library is wrapping the connection or
if not; is interfacing with the pooled connection/data source object.
Data source configuration is... (parameters is our own bean for storing
this
stuff...)
ClientConnectionPoolDataSource dataSource =
new ClientConnectionPoolDataSource();
dataSource.setServerName(parameters.getHost());
dataSource.setPortNumber(parameters.getPort());
dataSource.setDatabaseName(parameters.getPath());
We're using MCPM as the pool manager, it's just a simple connection pool
manager in a single Java class which uses PooledConnection to do the hard
stuff.
Relevant code is probably:
private synchronized Connection getConnection2() throws SQLException {
PooledConnection pconn;
if (!recycledConnections.empty()) {
pconn = recycledConnections.pop();
} else {
pconn = dataSource.getPooledConnection();
}
Connection conn = pconn.getConnection();
activeConnections++;
return conn;
}
As it's going via PooledConnection I was expecting it to be reusable after
closing. This works in practice for the embedded version anyway.
I've checked the MCPM code and close() is only called during
disposeConnection
and certainly not during recycleConnection.
Daniel
--
-
Raymond Kroeker
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison [EMAIL PROTECTED] writes: Knut Anders Hatlen wrote: You shouldn't load the client driver unless your application uses jdbc:derby://localhost/mydb. The reason why the client driver seems to work in your case, is that the server already has loaded the embedded driver. So the code that runs inside the same JVM as the server should use jdbc:derby:mydb and not load the client driver, whereas the external scripts should load the client driver and use jdbc:derby://localhost/mydb. Unfortunately it's not that simple. If you use the embedded driver with STRONG_PASSWORD_SUBSTITUTE_SECURITY securityMechanism=8, you get a NPE is you open, close and then reopen the database - which is necessary if you want to be sure that the static database properties you set are used. OK, I think I see the problem now. It seems like if you connect to a database which has derby.connection.requireAuthentication=true using the embedded driver, Derby throws a NPE if the securityMechanism connection attribute is specified. Specifying the securityMechanism attribute for an embedded connection doesn't really make any sense, I think, since it says how to send passwords over a network connection, and the embedded driver doesn't use the network. This means you will have to connect without securityMechanism when you connect from within Tomcat (with the embedded driver), and with securityMechanism when you connect from the external scripts (with the client driver). Of course, you should have got an exception with a message explaining the problem, not a NPE. HTH, -- Knut Anders
Re: NPE when connecting to database with securityMechanism=8
Knut Anders Hatlen wrote: OK, I think I see the problem now. It seems like if you connect to a database which has derby.connection.requireAuthentication=true using the embedded driver, Derby throws a NPE if the securityMechanism connection attribute is specified. Specifying the securityMechanism attribute for an embedded connection doesn't really make any sense, I think, since it says how to send passwords over a network connection, and the embedded driver doesn't use the network. Mekes sense, thanks. This means you will have to connect without securityMechanism when you connect from within Tomcat (with the embedded driver), and with securityMechanism when you connect from the external scripts (with the client driver). Right. Of course, you should have got an exception with a message explaining the problem, not a NPE. Hmm, aren't unrecognised properties just silently ignored? Shouldn't this be the same? -- Alan Burlison --
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison [EMAIL PROTECTED] writes: Knut Anders Hatlen wrote: Of course, you should have got an exception with a message explaining the problem, not a NPE. Hmm, aren't unrecognised properties just silently ignored? Shouldn't this be the same? Right, that would indeed be more consistent. (Silently ignoring misspelled properties is not among the features I appreciate most, though. A little too helpful...) -- Knut Anders
Re: NPE when connecting to database with securityMechanism=8
Knut Anders Hatlen wrote: You shouldn't load the client driver unless your application uses jdbc:derby://localhost/mydb. The reason why the client driver seems to work in your case, is that the server already has loaded the embedded driver. So the code that runs inside the same JVM as the server should use jdbc:derby:mydb and not load the client driver, whereas the external scripts should load the client driver and use jdbc:derby://localhost/mydb. Unfortunately it's not that simple. If you use the embedded driver with STRONG_PASSWORD_SUBSTITUTE_SECURITY securityMechanism=8, you get a NPE is you open, close and then reopen the database - which is necessary if you want to be sure that the static database properties you set are used. This all seems as fragile as hell, to be honest... -- Alan Burlison --
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison [EMAIL PROTECTED] writes: I'm getting a NPE when I try to connect to a database with securityMechanism=8. This is with Derby 10.3.1.4. I'm setting up the database by creating it, then running the following script from inside my app using the org.apache.derby.tools.ij class: -- call syscs_util.syscs_set_database_property( 'derby.database.propertiesOnly', 'true'); call syscs_util.syscs_set_database_property( 'derby.authentication.provider', 'BUILTIN'); call syscs_util.syscs_set_database_property( 'derby.connection.requireAuthentication', 'true'); call syscs_util.syscs_set_database_property( 'derby.drda.securityMechanism', 'STRONG_PASSWORD_SUBSTITUTE_SECURITY'); -- Hi Alan, I don't think you can set derby.drda.securityMechanism for a single database. You would have to set it for the entire server process, either by starting the Java process with -Dderby.drda.securityMechanism=XXX, or by putting derby.drda.securityMechanism=XXX into derby.properties. -- Knut Anders
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison [EMAIL PROTECTED] writes: Knut Anders Hatlen wrote: I don't think you can set derby.drda.securityMechanism for a single database. You would have to set it for the entire server process, either by starting the Java process with -Dderby.drda.securityMechanism=XXX, or by putting derby.drda.securityMechanism=XXX into derby.properties. Ah right, that certainly would explain the problems I've been having. I assume I could also use System.setProperty() before using NetworkServerControl.start() the derby network server. Yes, that should also do the trick. I'm using Derby inside a tomcat instance, but I want it to run in network server mode rather than embedded mode so I can use external scripts for doing things such as database backups, rather than having to do everything inside tomcat. The NPE still looks like a derby bug though, irrespective of my mistake ;-) I agree, that shouldn't have happened. It would be great if you could file a bug report at http://issues.apache.org/jira/browse/DERBY. I'm also a little confused on how to access a database once the network server is is running, from inside the same JVM that contains the network server. By mistake I forgot to change the URL from the one used by the embedded driver: db = DriverManager.getConnection(jdbc:derby:mydb, props); to the one needed by the network driver db = DriverManager.getConnection(jdbc:derby://localhost/mydb, props); but the database still seemed to open OK, which is rather odd... The network server actually uses the embedded driver to perform database operations, so any application running in the same JVM as the server could have access to the database directly through the embedded driver. In your setup, I think it would make sense to use Derby this way. The servlets could access the database without doing any network I/O, and thereby improving the performance, whereas the external scripts could use the network client driver and don't involve Tomcat. Derby's manuals call this configuration an Embedded Server (see http://db.apache.org/derby/docs/10.3/adminguide/cadminov17524.html). -- Knut Anders
Re: NPE when connecting to database with securityMechanism=8
Knut Anders Hatlen wrote: I assume I could also use System.setProperty() before using NetworkServerControl.start() the derby network server. Yes, that should also do the trick. It does, thanks :-) The NPE still looks like a derby bug though, irrespective of my mistake ;-) I agree, that shouldn't have happened. It would be great if you could file a bug report at http://issues.apache.org/jira/browse/DERBY. Done, DERBY-3025 The network server actually uses the embedded driver to perform database operations, so any application running in the same JVM as the server could have access to the database directly through the embedded driver. In your setup, I think it would make sense to use Derby this way. The servlets could access the database without doing any network I/O, and thereby improving the performance, whereas the external scripts could use the network client driver and don't involve Tomcat. Derby's manuals call this configuration an Embedded Server (see http://db.apache.org/derby/docs/10.3/adminguide/cadminov17524.html). I knew it worked, but I didn't know if it was correct to use the embedded driver in that situation. After following the link you sent I found the following in: http://db.apache.org/derby/docs/10.3/adminguide/radminembeddedserverex.html The program that starts the Network Server can access the database by using either the embedded driver or the Network Client driver. Thanks! -- Alan Burlison --
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison wrote: The network server actually uses the embedded driver to perform database operations, so any application running in the same JVM as the server could have access to the database directly through the embedded driver. In your setup, I think it would make sense to use Derby this way. The servlets could access the database without doing any network I/O, and thereby improving the performance, whereas the external scripts could use the network client driver and don't involve Tomcat. Derby's manuals call this configuration an Embedded Server (see http://db.apache.org/derby/docs/10.3/adminguide/cadminov17524.html). I knew it worked, but I didn't know if it was correct to use the embedded driver in that situation. After following the link you sent I found the following in: http://db.apache.org/derby/docs/10.3/adminguide/radminembeddedserverex.html The program that starts the Network Server can access the database by using either the embedded driver or the Network Client driver. One other question: If I am attaching to a derby instance that is running inside the same JVM, should I be using the client driver or the embedded driver? From observation the client driver works with both jdbc:derby://locahost/mydb and jdbc:derby:mydb, but should I be using the client driver only for jdbc:derby://locahost/mydb and the embedded driver for jdbc:derby:mydb? -- Alan Burlison --
Re: NPE when connecting to database with securityMechanism=8
Alan Burlison [EMAIL PROTECTED] writes: Alan Burlison wrote: The network server actually uses the embedded driver to perform database operations, so any application running in the same JVM as the server could have access to the database directly through the embedded driver. In your setup, I think it would make sense to use Derby this way. The servlets could access the database without doing any network I/O, and thereby improving the performance, whereas the external scripts could use the network client driver and don't involve Tomcat. Derby's manuals call this configuration an Embedded Server (see http://db.apache.org/derby/docs/10.3/adminguide/cadminov17524.html). I knew it worked, but I didn't know if it was correct to use the embedded driver in that situation. After following the link you sent I found the following in: http://db.apache.org/derby/docs/10.3/adminguide/radminembeddedserverex.html The program that starts the Network Server can access the database by using either the embedded driver or the Network Client driver. One other question: If I am attaching to a derby instance that is running inside the same JVM, should I be using the client driver or the embedded driver? From observation the client driver works with both jdbc:derby://locahost/mydb and jdbc:derby:mydb, but should I be using the client driver only for jdbc:derby://locahost/mydb and the embedded driver for jdbc:derby:mydb? You shouldn't load the client driver unless your application uses jdbc:derby://localhost/mydb. The reason why the client driver seems to work in your case, is that the server already has loaded the embedded driver. So the code that runs inside the same JVM as the server should use jdbc:derby:mydb and not load the client driver, whereas the external scripts should load the client driver and use jdbc:derby://localhost/mydb. -- Knut Anders
Re: NPE when connecting to database with securityMechanism=8
Knut Anders Hatlen wrote: You shouldn't load the client driver unless your application uses jdbc:derby://localhost/mydb. The reason why the client driver seems to work in your case, is that the server already has loaded the embedded driver. So the code that runs inside the same JVM as the server should use jdbc:derby:mydb and not load the client driver, whereas the external scripts should load the client driver and use jdbc:derby://localhost/mydb. That makes sense :-) Thanks for all your help! -- Alan Burlison --
Re: NPE on CreateTrigger
Doh! It looks like if I had looked at the log and searched nabble instead of searching nabble for the NPE and then the log, I would have found that I was experiencing: http://issues.apache.org/jira/browse/DERBY-85 I can create a non-empty default table in the schema and it works as a workaround. Brian bkbonner wrote: I'm trying to create the following trigger: CREATE TRIGGER PID.DELCONGT AFTER DELETE ON PID.PIDPDTCO REFERENCING OLD AS OROW FOR EACH ROWMODE DB2SQL DELETE FROM PID.PIDCONGT WHERE CON_UPC_NO = OROW.CON_UPC_NO; For all intents and purposes, it looks like this should work, however I get: ERROR XJ001: Java exception: ': java.lang.NullPointerException'. The log shows: java.lang.NullPointerException at org.apache.derby.impl.sql.catalog.SYSSTATEMENTSRowFactory.makeSYSSTATEMENTSrow(Unknown Source) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.addSPSDescriptor(Unknown Source) at org.apache.derby.impl.sql.execute.CreateTriggerConstantAction.createSPS(Unknown Source) at org.apache.derby.impl.sql.execute.CreateTriggerConstantAction.executeConstantAction(Unknown Source) at org.apache.derby.impl.sql.execute.MiscResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLIMM(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source) at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source) I'm running 10.1.2.1 Any thoughts? -- View this message in context: http://www.nabble.com/NPE-on-CreateTrigger-tf2624509.html#a7323703 Sent from the Apache Derby Users mailing list archive at Nabble.com.
Re: NPE in setTimestamp(int,Timestamp,Calendar)
Jose de Castro wrote: A NullPointerException occurs if you pass in a null Timestamp into org.apache.derby.client.am.PreparedStatement.setTimestamp(int,Timestamp,Calendar) Does anyone know if this is being addressed in 10.3? I have included a suggested fix. Is this something I should add to JIRA? Please enter a Jira bug and please attach a patch to that issue. Thanks, Dan.
Re: NPE in Derby
Noel J. Bergman wrote: Any idea what would cause: 17/12/05 11:26:36 ERROR mailstore: Error retrieving pending messages java.lang.NullPointerException at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateControlle r.fetch(Unknown Source) at [stack snipped] Our code is the loadPendingMessages method in JDBCSpoolRepository, which executes the listMessagesSQL query found in sqlResources.xml. ref: http://svn.apache.org/repos/asf/james/server/trunk/src/java/org/apache/james /mailrepository/JDBCSpoolRepository.java http://svn.apache.org/repos/asf/james/server/trunk/src/conf/sqlResource s.xml I don't see anything the SQL statement that I would expect to cause a problem. Internal race condition? Most likely a race condition, a JDBC request to Derby should never receive a NPE. I looked at the code briefly and nothing jumped out as wrong. I'l see if I can reproduce it in my messing around with James. Dan.
