[jira] Commented: (DERBY-1279) Bad column name in DatabaseMetaData.getColumns()
[ http://issues.apache.org/jira/browse/DERBY-1279?page=comments#action_12416715 ] Knut Anders Hatlen commented on DERBY-1279: --- As far as I can see, getColums() doesn't return any column named SCOPE_CATLOG or SCOPE_CATALOG (but it should return SCOPE_CATLOG according to the spec). getAttributes() returns SCOPE_CATALOG, but the spec spells 'catalog' correctly for that method. Bad column name in DatabaseMetaData.getColumns() Key: DERBY-1279 URL: http://issues.apache.org/jira/browse/DERBY-1279 Project: Derby Type: Bug Components: JDBC Versions: 10.2.0.0 Reporter: Rick Hillegas Fix For: 10.2.0.0 This method returns a column named SCOPE_CATLOG. Derby has corrected this typo to SCOPE_CATALOG. To conform to the spec, Derby should faithfully mis-spell this column name. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1156) allow the encrypting of an existing unencrypted db and allow the re-encrypting of an existing encrypted db
[ http://issues.apache.org/jira/browse/DERBY-1156?page=all ] Suresh Thalamati updated DERBY-1156: Attachment: reencrypt_3.diff DERBY -1156 (partial) This patch adds some code required to support reconfigure(rencryption) of an already existing encrypted database with a new password(secret key) or an external user specified encryption key. Two new attributes newBootPassword and newEncryptionkey are introduced to support this functionality. -- modified the code to support two have instance of two cipher factories to exist. So that the existing data can decrypted with the old encryption key using one cipher factory and rewrite the data with new encryption keys using another cipher factory. -- reenryption of the database with new keys is similar to encrypting an already existing database. All the container data is read through the page cache and rewritten using the new encryption keys. -- Added test case to test the re-encryption of an encrypted database. TESTS : derbyall test suite passed on Windows XP/JDK142 It would be great if some can review this patch. svn stat: M java\engine\org\apache\derby\impl\services\jce\JCECipherFactory.java A java\engine\org\apache\derby\impl\services\jce\JCECipherFactoryBuilder.java M java\engine\org\apache\derby\impl\store\raw\RawStore.java A java\engine\org\apache\derby\iapi\services\crypto\CipherFactoryBuilder.java M java\engine\org\apache\derby\iapi\reference\Attribute.java M java\engine\org\apache\derby\iapi\reference\Module.java M java\engine\org\apache\derby\modules.properties M java\testing\org\apache\derbyTesting\unitTests\crypto\T_Cipher.java M java\testing\org\apache\derbyTesting\functionTests\tests\store\encryptDatabaseTest1.sql M java\testing\org\apache\derbyTesting\functionTests\master\encryptDatabaseTest1.out allow the encrypting of an existing unencrypted db and allow the re-encrypting of an existing encrypted db -- Key: DERBY-1156 URL: http://issues.apache.org/jira/browse/DERBY-1156 Project: Derby Type: Improvement Components: Store Versions: 10.1.2.3 Reporter: Mike Matrigali Assignee: Suresh Thalamati Priority: Minor Fix For: 10.2.0.0 Attachments: encryptspec.html, reencrypt_1.diff, reencrypt_2.diff, reencrypt_3.diff encrypted database to be re-encrypted with a new password. Here are some ideas for an initial implementation. The easiest way to do this is to make sure we have exclusive access to the data and that no log is required in the new copy of the db. I want to avoid the log as it also is encrypted. Here is my VERY high level plan: 1) Force exclusive access by putting all the work in the low level store, offline boot method. We will do redo recovery as usual, but at the end there will be an entry point to do the copy/encrypt operation. copy/encrypt process: 0) The request to encrypt/re-encrypt the db will be handled with a new set of url flags passed into store at boot time. The new flags will provide the same inputs as the current encrypt flags. So at high level the request will be connect db old_encrypt_url_flags; new_encrypt_url_flags. TODO - provide exact new flag syntax. 1) Open a transaction do all logged work to do the encryption. All logging will be done with existing encryption. 2) Copy and encrypt every db file in the database. The target files will be in the data directory. There will be a new suffix to track the new files, similar to the current process used for handling drop table in a transaction consistent manner without logging the entire table to the log. Entire encrypted destination file is guaranteed synced to disk before transaction commits. I don't think this part needs to be logged. Files will be read from the cache using existing mechanism and written directly into new encrypted files (new encrypted data does not end up in the cache). 3) Switch encrypted files for old files. Do this under a new log operation so the process can be correctly rolled back if the encrypt db operation transaction fails. Rollback will do file at a time switches, no reading of encrypted data is necessary. 4) log a change encryption of db log record, but do not update system.properties with the change. 5) commit transaction. 6) update system.properties and sync changes. 7) TODO - need someway to handle crash between steps 5 and 6. 6) checkpoint all data, at this point guaranteed that there is no outstanding transaction, so after checkpoint is done there is no need for the log. ISSUES: o there probably should be something that catches a request to encrypt to whatever db was already encrypted with. -- This message is automatically
Regression Test Failure! - Derby 415160 - Sun DBTG
[Auto-generated mail] *Derby* 415160/2006-06-18 19:46:38 CEST *derbyall* Failed TestsOK Skip Duration Platform --- *Jvm: 1.6* 15715700 0 106.20% SunOS-5.10_i86pc-i386 Details in http://www.multinet.no/~solberg/public/Apache/DerbyJDK16Jvm1.6/Limited/testSummary-415160.html Attempted failure analysis in http://www.multinet.no/~solberg/public/Apache/Failures/DerbyJDK16Jvm1.6/415160.html *Jvm: 1.5* 4675671 2 110.49% CYGWIN_NT-5.1_i686-unknown 3675672 2 115.57% CYGWIN_NT-5.2_i686-unknown 0675675 2 110.56% Linux-2.6.14-1.1644_FC4_i686-i686 0675675 2 105.30% Linux-2.6.9-34.ELsmp_x86_64-x86_64 0675675 2 176.69% SunOS-5.10_i86pc-i386 0675675 2 136.79% SunOS-5.10_sun4u-sparc NA NA NANA SunOS-5.11_i86pc-i386 0675675 2 113.45% SunOS-5.9_sun4u-sparc Details in http://www.multinet.no/~solberg/public/Apache/Derby/Limited/testSummary-415160.html Attempted failure analysis in http://www.multinet.no/~solberg/public/Apache/Failures/Derby/415160.html *Jvm: 1.4* 3669666 4 104.59% CYGWIN_NT-5.1_i686-unknown 0669669 4 111.38% Linux-2.6.14-1.1644_FC4_i686-i686 1669668 4 101.63% Linux-2.6.9-34.ELsmp_x86_64-x86_64 0669669 4 211.08% SunOS-5.10_i86pc-i386 Details in http://www.multinet.no/~solberg/public/Apache/DerbyJvm1.4/Limited/testSummary-415160.html Attempted failure analysis in http://www.multinet.no/~solberg/public/Apache/Failures/DerbyJvm1.4/415160.html Changes in http://www.multinet.no/~solberg/public/Apache/Derby/UpdateInfo/415160.txt ( All results in http://www.multinet.no/~solberg/public/Apache/index.html )
[jira] Created: (DERBY-1427) sysinfo does not write Java SE and JDBC version when running under JDK 1.6
sysinfo does not write Java SE and JDBC version when running under JDK 1.6 -- Key: DERBY-1427 URL: http://issues.apache.org/jira/browse/DERBY-1427 Project: Derby Type: Bug Components: Tools Versions: 10.2.0.0 Environment: Sun JDK 1.6 Reporter: Olav Sandstaa Priority: Minor When running sysinfo with jdk 1.6 information about which Java SE and JDBC versions that is supported is missing in the output. The following is written: - Derby Information JRE - JDBC: ?-? The complete output from sysinfo: -- Java Information -- Java Version:1.6.0-beta2 Java Vendor: Sun Microsystems Inc. Java home: /usr/local/java/jdk1.6.0_b86/jre Java classpath: jars/sane/derby.jar OS name: SunOS OS architecture: x86 OS version: 5.10 Java user name: os136802 Java user home: /home/os136802 Java user dir: /home/os136802/derby/develop/jdk16/trunk java.specification.name: Java Platform API Specification java.specification.version: 1.6 - Derby Information JRE - JDBC: ?-? [/home/os136802/derby/develop/jdk16/trunk/jars/sane/derby.jar] 10.2.0.4 alpha - (415258M) -- - Locale Information - Current Locale : [English/United States [en_US]] Found support for locale: [de_DE] version: 10.2.0.4 alpha - (415258M) Found support for locale: [es] version: 10.2.0.4 alpha - (415258M) Found support for locale: [fr] version: 10.2.0.4 alpha - (415258M) Found support for locale: [it] version: 10.2.0.4 alpha - (415258M) Found support for locale: [ja_JP] version: 10.2.0.4 alpha - (415258M) Found support for locale: [ko_KR] version: 10.2.0.4 alpha - (415258M) Found support for locale: [pt_BR] version: 10.2.0.4 alpha - (415258M) Found support for locale: [zh_CN] version: 10.2.0.4 alpha - (415258M) Found support for locale: [zh_TW] version: 10.2.0.4 alpha - (415258M) -- -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1341) LOB setBytes method(s) are currently no supported, but part of the Java 1.4 JDBC interface
[ http://issues.apache.org/jira/browse/DERBY-1341?page=all ] Anurag Shekhar updated DERBY-1341: -- Attachment: derby-1341.diff This diff is only for review of aprocah I am taking to resolve this issue. I have introduced 3 new class in this patch. LOBStreamControl, LOBOutputStream and LOBInputStream. LOBStreamControl acts as wrpper on top of bytes stored in memory and the temporary file on the file system. The read and write methods exposed by LOBStreamControl are used by EmbeddedBlob and LOBStream. when the total data size is less than 4k its stored in array once either the setBytes on blob or write on Stream incheses the size more than 4k the data is written into a temporrary file and subsequent operations are performed on the file. If call to truncate reduces the size of the file below 4k the data is again moved back to memory. Chanes made in EmbeddedBlob Changed EmbeddedBlob to pass all the calls which were priviously acessing the bytes array to LOBStreamControl. LOB setBytes method(s) are currently no supported, but part of the Java 1.4 JDBC interface -- Key: DERBY-1341 URL: http://issues.apache.org/jira/browse/DERBY-1341 Project: Derby Type: Bug Components: JDBC Versions: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0, 10.2.0.0, 10.1.2.0, 10.1.1.1, 10.1.1.2, 10.1.2.1, 10.1.3.0, 10.1.2.2, 10.1.2.3, 10.3.0.0, 10.1.2.4, 10.1.2.5 Environment: Windows 2000 Reporter: Keith McFarlane Assignee: Anurag Shekhar Attachments: derby-1341.diff JDBC LOB . getBtypes methods are not implemented in any Derby version to date: there is a place-holder method that throws a SQLException reporting that the methods are not implemented. It would be excellent to have any efficient Derby implementation of the getBytes LOB methods that provide random-access to the binary // character content of database large objects. The specific context is implementing a Lucene Directory interface that stores indexing data (index files) and other binary data in a local encrypted Derby instance. A work around is to write an encrypted RandomAccessFile implementation as a file-sdystem buffer, perhaps writing to the database on closure. An efficient Derby implementation of LOB . getBytes would avoid this an make for a clean design. I can think of several reasons why random-access to LOBs would be valuable in a hostile client environment. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (DERBY-1428) Generating derby properties on the fly can lead to non-deterministic engine startup behavior
Generating derby properties on the fly can lead to non-deterministic engine startup behavior Key: DERBY-1428 URL: http://issues.apache.org/jira/browse/DERBY-1428 Project: Derby Type: Bug Components: Store Versions: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0, 10.2.0.0, 10.1.2.1, 10.1.3.0, 10.1.2.2, 10.1.2.3, 10.3.0.0, 10.1.2.4, 10.1.2.5, 10.1.4.0, 10.1.3.1 Reporter: Rick Hillegas A Heisenbug can arise if more than one embedded Derby application runs in the same VM and at least one of them generates derby properties on the fly. Here's the problem scenario: o The customer runs two embedded Derby apps in the same VM: EmbeddedApp and OtherApp. o EmbeddedApp generates derby properties on the fly before connecting to a database and triggering engine startup. o Whether the engine picks up those generated properties depends on whether EmbeddedApp or OtherApp runs first. Here are two workarounds for the problem: 1) Don't generate derby properties on the fly. Instead, specify them on the VM startup command. Or specify them in a $DERBY_HOME/derby.properties file which is generated before the VM comes up and which does not change during the lifetime of the VM. 2) If you can't do (1), then modify the VM startup script so that it forces the applications to run in a deterministic order. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (DERBY-1384) Increase default BLOB/CLOB length to maximum supported (2G?)
[ http://issues.apache.org/jira/browse/DERBY-1384?page=all ] Bernt M. Johnsen resolved DERBY-1384: - Resolution: Fixed Committed revision 415328. Increase default BLOB/CLOB length to maximum supported (2G?) Key: DERBY-1384 URL: http://issues.apache.org/jira/browse/DERBY-1384 Project: Derby Type: Improvement Components: SQL Reporter: Bernt M. Johnsen Assignee: Bernt M. Johnsen Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1384-code.diff, derby-1384-code.stat, derby-1384-docs.diff, derby-1384-docs.stat Default BLOB/CLOB length should be the maximum length supported by Derby (2G?) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-1384) Increase default BLOB/CLOB length to maximum supported (2G?)
[ http://issues.apache.org/jira/browse/DERBY-1384?page=comments#action_12416763 ] Bernt M. Johnsen commented on DERBY-1384: - Docs Committed revision 415329. Increase default BLOB/CLOB length to maximum supported (2G?) Key: DERBY-1384 URL: http://issues.apache.org/jira/browse/DERBY-1384 Project: Derby Type: Improvement Components: SQL Reporter: Bernt M. Johnsen Assignee: Bernt M. Johnsen Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1384-code.diff, derby-1384-code.stat, derby-1384-docs.diff, derby-1384-docs.stat Default BLOB/CLOB length should be the maximum length supported by Derby (2G?) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-1374) compatibility test fails with 'PROTOCOL Data Stream Syntax Error'
[ http://issues.apache.org/jira/browse/DERBY-1374?page=comments#action_12416770 ] Andreas Korneliussen commented on DERBY-1374: - I will run tests on the patch and commit it. compatibility test fails with 'PROTOCOL Data Stream Syntax Error' - Key: DERBY-1374 URL: http://issues.apache.org/jira/browse/DERBY-1374 Project: Derby Type: Bug Components: Regression Test Failure Versions: 10.2.0.0 Environment: OS: ALL JVM: server: JVM-1.3, client: JVM-1.3 Derby: server: 10.1.2.0, client: Trunk Reporter: Ole Solberg Assignee: Fernanda Pizzorno Priority: Critical Attachments: derby-1374.diff, derby-1374.stat compatibility/serverVM-1.3_server10.1.2.0_clientVM-1.3_clientTrunk: VM Version = 1.3 Could not find com.ibm.db2.jcc.DB2Driver Could not find org.apache.derby.jdbc.EmbeddedDriver Driver org.apache.derby.jdbc.ClientDriver Version = 10.2 java.sql.SQLException: A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected. Reason: 0x8,544. at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:512) at java.sql.DriverManager.getConnection(DriverManager.java:141) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(Unknown Source) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(Unknown Source) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(Unknown Source) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(Unknown Source) Exception in thread main java.lang.Exception: Error lookup up server info: A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected. Reason: 0x8,544. at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(Unknown Source) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(Unknown Source) The compatibility test aborts on failure, so lots of combinations were not tested. The following combinations succeeded: compatibility/embedded_VM-1.3 compatibility/embedded_VM-1.4 compatibility/embedded_VM-1.5 compatibility/serverVM-1.3_serverTrunk_clientVM-1.3_clientTrunk compatibility/serverVM-1.3_serverTrunk_clientVM-1.3_clientDerby10.1.2.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.3_clientDerby10.1.1.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.3_clientDB2JCC compatibility/serverVM-1.3_serverTrunk_clientVM-1.4_clientTrunk compatibility/serverVM-1.3_serverTrunk_clientVM-1.4_clientDerby10.1.2.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.4_clientDerby10.1.1.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.4_clientDB2JCC compatibility/serverVM-1.3_serverTrunk_clientVM-1.5_clientTrunk compatibility/serverVM-1.3_serverTrunk_clientVM-1.5_clientDerby10.1.2.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.5_clientDerby10.1.1.0 compatibility/serverVM-1.3_serverTrunk_clientVM-1.5_clientDB2JCC -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (DERBY-1429) Additional vulnerability to non-deterministic startup behavior when applications generate derby properties on the fly
Additional vulnerability to non-deterministic startup behavior when applications generate derby properties on the fly - Key: DERBY-1429 URL: http://issues.apache.org/jira/browse/DERBY-1429 Project: Derby Type: Bug Components: Store Versions: 10.2.0.0 Reporter: Rick Hillegas JDBC4 driver-autoloading, introduced by DERBY-930, increases the exposure to non-deterministic startup behavior described in DERBY-1428. With the introduction of driver-autloading, DERBY-1428 can be triggered if OtherApp is any application which uses a JDBC driver. That is, OtherApp could use a Derby client driver, the DB2JCC driver, the Oracle client driver, etc.. The extra exposure arises because, with driver auto-loading, all JDBC drivers are registered and the Derby engine boots the first time some application asks for a Connection. The issues are summarized in an email thread http://mail-archives.apache.org/mod_mbox/db-derby-dev/200606.mbox/browser and bug report DERBY-1399. Workarounds are similar to those for DERBY-1428: 1) Determine the derby properties BEFORE the VM starts. 2) If that is not possible, then force the self-configuring embedded application to run first. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1428) Generating derby properties on the fly can lead to non-deterministic engine startup behavior
[ http://issues.apache.org/jira/browse/DERBY-1428?page=all ] Rick Hillegas updated DERBY-1428: - Description: A Heisenbug can arise if more than one embedded Derby application runs in the same VM and at least one of them generates derby properties on the fly. Here's the problem scenario: o The customer runs two embedded Derby apps in the same VM: EmbeddedApp and OtherApp. o EmbeddedApp generates derby properties on the fly before connecting to a database and triggering engine startup. o Whether the engine picks up those generated properties depends on whether EmbeddedApp or OtherApp runs first. Here are two workarounds for the problem: 1) Don't generate derby properties on the fly. Instead, specify them on the VM startup command. Or specify them in a $DERBY_HOME/derby.properties file which is generated before the VM comes up and which does not change during the lifetime of the VM. 2) If you can't do (1), then modify the VM startup script so that the self-configuring EmbeddedApp runs first. was: A Heisenbug can arise if more than one embedded Derby application runs in the same VM and at least one of them generates derby properties on the fly. Here's the problem scenario: o The customer runs two embedded Derby apps in the same VM: EmbeddedApp and OtherApp. o EmbeddedApp generates derby properties on the fly before connecting to a database and triggering engine startup. o Whether the engine picks up those generated properties depends on whether EmbeddedApp or OtherApp runs first. Here are two workarounds for the problem: 1) Don't generate derby properties on the fly. Instead, specify them on the VM startup command. Or specify them in a $DERBY_HOME/derby.properties file which is generated before the VM comes up and which does not change during the lifetime of the VM. 2) If you can't do (1), then modify the VM startup script so that it forces the applications to run in a deterministic order. In workaround (2) be explicit about the order in which the applications have to run. Generating derby properties on the fly can lead to non-deterministic engine startup behavior Key: DERBY-1428 URL: http://issues.apache.org/jira/browse/DERBY-1428 Project: Derby Type: Bug Components: Store Versions: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0, 10.2.0.0, 10.1.2.1, 10.1.3.0, 10.1.2.2, 10.1.2.3, 10.3.0.0, 10.1.2.4, 10.1.2.5, 10.1.4.0, 10.1.3.1 Reporter: Rick Hillegas A Heisenbug can arise if more than one embedded Derby application runs in the same VM and at least one of them generates derby properties on the fly. Here's the problem scenario: o The customer runs two embedded Derby apps in the same VM: EmbeddedApp and OtherApp. o EmbeddedApp generates derby properties on the fly before connecting to a database and triggering engine startup. o Whether the engine picks up those generated properties depends on whether EmbeddedApp or OtherApp runs first. Here are two workarounds for the problem: 1) Don't generate derby properties on the fly. Instead, specify them on the VM startup command. Or specify them in a $DERBY_HOME/derby.properties file which is generated before the VM comes up and which does not change during the lifetime of the VM. 2) If you can't do (1), then modify the VM startup script so that the self-configuring EmbeddedApp runs first. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-930) Add support for autoloading of Derby client drivers
[ http://issues.apache.org/jira/browse/DERBY-930?page=comments#action_12416775 ] Rick Hillegas commented on DERBY-930: - Adding a release note for this issue: PROBLEM If an embedded Derby application generates its own Derby properties on the fly, and that Derby application runs in the same VM with other JDBC applications, then you are not guaranteed that the Derby engine will pick up the custom properties when it boots. SYMPTOM Derby startup behavior will be non-deterministic. Sometimes the engine will pick up the custom properties and sometimes it won't. CAUSE JDBC4 introduced driver-autoloading. This causes all JDBC drivers visible to the VM to register themselves the first time some application requests a Connection. When the embedded Derby driver registers itself, it also boots the Derby engine and the engine picks up whatever Derby properties are currently visible. An embedded Derby application may want to configure the engine properties before asking for a Connection. That embedded Derby application will not get a chance to configure engine properties if some other JDBC application in the same VM runs first and requests a Connection. Two related bugs describe this issue in greater detail: DERBY-1428 and DERBY-1429. SOLUTION There is no general solution to the problem. If two self-configuring embedded Derby applications run in the same VM, then only one of them can win. WORKAROUND The following workarounds may be useful: 1) Don't configure Derby properties inside your applications. Instead, specify Derby properties either on the VM startup line or in a $DERBY_HOME/derby.properties which remains constant for the VM's lifetime. 2) If (1) is not possible, then make the self-configuring Derby application run first. Add support for autoloading of Derby client drivers --- Key: DERBY-930 URL: http://issues.apache.org/jira/browse/DERBY-930 Project: Derby Type: New Feature Components: Build tools, JDBC Reporter: Rick Hillegas Assignee: Rick Hillegas Fix For: 10.2.0.0 Attachments: ClassloadingTest.jar, bug930.diff, bug930_problem.diff, bug930_problem2.diff, bug930_rev2.diff, bug930_rev3.diff Write Derby's driver names into the correct spot in derby.jar and derbyclient.jar so that the 1.6 vm autoloads Derby drivers. Section 10.2.1 of the JDBC4 spec describes the details. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-1429) Additional vulnerability to non-deterministic startup behavior when applications generate derby properties on the fly
[ http://issues.apache.org/jira/browse/DERBY-1429?page=comments#action_12416778 ] Rick Hillegas commented on DERBY-1429: -- We could separate out the booting of the engine from the registration of the embedded driver. This would eliminate this bug in the case that there is only one embedded Derby application in the VM. However, we can't solve the general problem of two self-configuring Derby applications running in the same VM. Only one of them can win. The user guide needs to strongly recommend that application designers avoid generating Derby properties on the fly. Additional vulnerability to non-deterministic startup behavior when applications generate derby properties on the fly - Key: DERBY-1429 URL: http://issues.apache.org/jira/browse/DERBY-1429 Project: Derby Type: Bug Components: Store Versions: 10.2.0.0 Reporter: Rick Hillegas JDBC4 driver-autoloading, introduced by DERBY-930, increases the exposure to non-deterministic startup behavior described in DERBY-1428. With the introduction of driver-autloading, DERBY-1428 can be triggered if OtherApp is any application which uses a JDBC driver. That is, OtherApp could use a Derby client driver, the DB2JCC driver, the Oracle client driver, etc.. The extra exposure arises because, with driver auto-loading, all JDBC drivers are registered and the Derby engine boots the first time some application asks for a Connection. The issues are summarized in an email thread http://mail-archives.apache.org/mod_mbox/db-derby-dev/200606.mbox/browser and bug report DERBY-1399. Workarounds are similar to those for DERBY-1428: 1) Determine the derby properties BEFORE the VM starts. 2) If that is not possible, then force the self-configuring embedded application to run first. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-930) Add support for autoloading of Derby client drivers
[ http://issues.apache.org/jira/browse/DERBY-930?page=comments#action_12416781 ] Rick Hillegas commented on DERBY-930: - Here is a second rev of the release note. This attempts to clarify the extra exposure introduced by jdk1.6. PROBLEM If an embedded Derby application generates its own Derby properties on the fly, and that Derby application runs in the same 1.6 VM with other JDBC applications, then you are not guaranteed that the Derby engine will pick up the custom properties when it boots. SYMPTOM Derby startup behavior will be non-deterministic. Sometimes the engine will pick up the custom properties and sometimes it won't. CAUSE JDBC4 introduced driver-autoloading. This causes all JDBC drivers visible to the 1.6 VM to register themselves the first time some application requests a Connection. When the embedded Derby driver registers itself, it also boots the Derby engine and the engine picks up whatever Derby properties are currently visible. An embedded Derby application may want to configure the engine properties before asking for a Connection. That embedded Derby application will not get a chance to configure engine properties if some other JDBC application in the same 1.6 VM runs first and requests a Connection. Two related bugs describe this issue in greater detail: DERBY-1428 and DERBY-1429. DERBY-1428 affects existing customer installations running old versions of Derby on the 1.3, 1.4, and 1.5 VMs. DERBY-1429 describes the extra exposure introduced with the 1.6 VM. SOLUTION There is no general solution to the problem. If two self-configuring embedded Derby applications run in the same VM, then only one of them can win. WORKAROUND The following workarounds may be useful: 1) Don't configure Derby properties inside your applications. Instead, specify Derby properties either on the VM startup line or in a $DERBY_HOME/derby.properties which remains constant for the VM's lifetime. 2) Don't run on the 1.6 VM. This eliminates the problem provided that you are not running more than one embedded Derby application in the same VM. 3) If (1) and (2) are not possible, then make the self-configuring Derby application run first. Add support for autoloading of Derby client drivers --- Key: DERBY-930 URL: http://issues.apache.org/jira/browse/DERBY-930 Project: Derby Type: New Feature Components: Build tools, JDBC Reporter: Rick Hillegas Assignee: Rick Hillegas Fix For: 10.2.0.0 Attachments: ClassloadingTest.jar, bug930.diff, bug930_problem.diff, bug930_problem2.diff, bug930_rev2.diff, bug930_rev3.diff Write Derby's driver names into the correct spot in derby.jar and derbyclient.jar so that the 1.6 vm autoloads Derby drivers. Section 10.2.1 of the JDBC4 spec describes the details. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-930) Add support for autoloading of Derby client drivers
Hi Bryan, Thanks for the quick response. I have revised the release note. Hopefully, it now describes the VM-specific issues better. Regards, -Rick Bryan Pendleton wrote: Rick Hillegas (JIRA) wrote: Adding a release note for this issue: Hi Rick, thanks for writing the release note. It wasn't clear to me from reading the release note whether this issue only affects JDK 1.6 environments, or whether it can arise under older VMs as well. If it only affects JDK 1.6 environments, perhaps we could change the release note to read: If an embedded Derby application runs in a JDK 1.6/JDBC4 environment, and generates its own Derby properties on the fly, and... thanks, bryan
Re: Driver autoloading and engine booting
Kathey Marsden wrote: David Van Couvering wrote: - Log a bug - Work on fixing it by booting on first connection rather than when the driver is loaded. It would be good to get input from the user community as well. One possible approach would be to log the bug, mark it as a regression, existing application impact, and release note needed. and have a formal release note. Then send a note to derby-user with a pointer to the bug and Ricks summary to see if any users feel there will be impact. Unfortunately, my gut instinct is that this is the kind of issue that lays dormant and then hits someone hard on upgrade or when integrated with another product.It might broadside users unaware, like it did the nist tests and DerbyNetAutostart tests. It will be really helpful to have the bug and the release note to present to users to help them understand impact. Then we can close DERBY-930 back up too until it is decided whether this Heisenbug is a show stopper or not. I have logged two new bugs: DERBY-1428 describes the existing, pre-JDBC4 shape of the problem. DERBY-1429 describes the extra exposure introduced by JDBC4. I have added a release note to DERBY-930, which summarizes the problem and its workarounds. In addition, I have asked the user community to let us know how broad the extra impact is. -1 on removing the autoloading feature, unless people have evidence that this is going to cause real problems for our users. I'll analyze some usage scenarios that I am aware of and see what the impact might be. The most important thing is that this be a deliberate choice and that we not rush new functionality and introduce a regression where there is a reasonable technical alternative, where we can have autoloading and avoid the regression. Thanks. Your analysis will be very helpful. I also have a couple of questions: 1) How much work is it and what are the risks to moving the boot to the first connection ? 2) What are the implications to derby.drdra.startNetworkServer with the DERBY-930 change. Olav mentioned that even if the boot was moved, DerbyNetAutostart would still need to be changed. 3) If derby booted at the first connection or the first instantiation of the driver (whichever came first) , would that solve the derby.drda.startNetworkServer problem. Hi Olav, could you address these questions? If we could get DerbyNetAutoStart running without changes and nist put back the way it was, I think we would be ok. The fact that these tests require changes after DERBY-930 is a good indicator that users will be impacted. I'm afraid I don't understand why we would want to revert our tests to the old behavior. It seems to me that our tests are practicing stunts which we strongly discourage: changing Derby properties on the fly. In general, there will always be Heisenbugs for applications which do this and we should discourage this practice in the strongest possible terms. What is the benefit of reverting the test behavior? Thanks, -Rick Kathey Kathey
Re: interested in working on DERBY-713
Hi Brent, Sounds like you're off to a good start. From the initial bug report, it looks like there's a good idea about which heuristic is being mis-applied here. Once you've studied the optimizer papers, I recommend that you post some high-level candidate solutions. Try to avoid optimizer jargon and concentrate on simple descriptions: o What query plan would you rather see? o What heuristic would the optimizer apply that would lead it to your preferred plan? o How would the optimizer decide to apply the new heuristic rather than the old one? I think you'll get some good feedback if your post contains the phrase Attention, optimizer experts. I think that the optimizer enthusiasts on the list will give you good feedback: o Maybe they can think of a better query plan or heuristic. o Maybe they can see some awkward corner cases in your heuristic. o They can advise you on whether your heuristic will short-circuit other optimizer choices. o They can advise you on whether your heuristic will cause an explosion in the time that the optimizer takes. Thanks for wanting to scratch this itch! Regards, -Rick Brent Verner wrote: Hi, I've recently found need for an embedded java db and only Derby seems even close to handling the task, however the broken query planning for IN clauses makes it unusable :-(. I've decided to eschew an embedded db in favor of PG for now, but I'd really like to be able to use Derby in the near(ish) future for deployment. I'd like to try to fix the query planning around the IN clause. I'm reviewing the internals papers right now, but I'd appreciate any input that might point me in the right direction :-). cheers! Brent
[jira] Assigned: (DERBY-1241) logmirror.ctrl is getting accessed outside the privileged block when the checkpoint instant is invalid on log factory boot method.
[ http://issues.apache.org/jira/browse/DERBY-1241?page=all ] Suresh Thalamati reassigned DERBY-1241: --- Assign To: Suresh Thalamati logmirror.ctrl is getting accessed outside the privileged block when the checkpoint instant is invalid on log factory boot method. Key: DERBY-1241 URL: http://issues.apache.org/jira/browse/DERBY-1241 Project: Derby Type: Bug Components: Store Reporter: Suresh Thalamati Assignee: Suresh Thalamati Priority: Minor This problem was reported on the derby-dev list by Olav Sandstaa , filing jira entry for it. Olav Sandstaa wrote: Rick Hillegas [EMAIL PROTECTED] wrote: java.sql.SQLException: Java exception: 'access denied (java.io.FilePermission /export/home/tmp/derbyjdbc4/DerbyNetClient/TestConnectionMethods/wombat/log/logmirror.ctrl read): java.security.AccessControlException'. at java.security.AccessControlContext.checkPermission(AccessControlContext.java:321) at java.security.AccessController.checkPermission(AccessController.java:546) at java.lang.SecurityManager.checkPermission(SecurityManager.java:532) at java.lang.SecurityManager.checkRead(SecurityManager.java:871) at java.io.File.exists(File.java:731) at org.apache.derby.impl.store.raw.log.LogToFile.boot(LogToFile.java:2940) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1996) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:290) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:542) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:418) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.bootLogFactory(BaseDataFileFactory.java:1762) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.setRawStoreFactory(BaseDataFileFactory.java:1218) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:250) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1996) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:290) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:542) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:418) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:987) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1996) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:290) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:542) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:418) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:738) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:178) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1996) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:290) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1831) at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1697) at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1577) at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:990) at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:541) at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:1586) at org.apache.derby.impl.jdbc.EmbedConnection.init(EmbedConnection.java:216) at org.apache.derby.impl.jdbc.EmbedConnection30.init(EmbedConnection30.java:72) at org.apache.derby.impl.jdbc.EmbedConnection40.init(EmbedConnection40.java:48) at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:62) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:199) at org.apache.derby.impl.drda.Database.makeConnection(Database.java:231) at org.apache.derby.impl.drda.DRDAConnThread.getConnFromDatabaseName(DRDAConnThread.java:1147) at org.apache.derby.impl.drda.DRDAConnThread.verifyUserIdPassword(DRDAConnThread.java:1125) at org.apache.derby.impl.drda.DRDAConnThread.parseSECCHK(DRDAConnThread.java:2709) at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(DRDAConnThread.java:987) at
Re: Driver autoloading and engine booting
Jean T. Anderson wrote: Rick Hillegas wrote: Kathey Marsden wrote: David Van Couvering wrote: - Log a bug - Work on fixing it by booting on first connection rather than when the driver is loaded. It would be good to get input from the user community as well. One possible approach would be to log the bug, mark it as a regression, existing application impact, and release note needed. and have a formal release note. Then send a note to derby-user with a pointer to the bug and Ricks summary to see if any users feel there will be impact. Unfortunately, my gut instinct is that this is the kind of issue that lays dormant and then hits someone hard on upgrade or when integrated with another product.It might broadside users unaware, like it did the nist tests and DerbyNetAutostart tests. It will be really helpful to have the bug and the release note to present to users to help them understand impact. Then we can close DERBY-930 back up too until it is decided whether this Heisenbug is a show stopper or not. I have logged two new bugs: DERBY-1428 describes the existing, pre-JDBC4 shape of the problem. DERBY-1429 describes the extra exposure introduced by JDBC4. I have added a release note to DERBY-930, which summarizes the problem and its workarounds. In addition, I have asked the user community to let us know how broad the extra impact is. Does the precedence for properties documentation need to be updated to mention jdbc 4/jdk 1.6 behavior? http://db.apache.org/derby/docs/dev/tuning/ctunsetprop23308.html -jean Hi Jean, I don't see that the precedence of properties has changed. What has changed is when we read properties during the lifetime of a vm. Driver-autoloading does require many other edits to the user guides. They are summarized in DERBY-1271. Regards, -Rick
Re: Driver autoloading and engine booting
Kathey Marsden wrote: Rick Hillegas wrote: I'm afraid I don't understand why we would want to revert our tests to the old behavior. It seems to me that our tests are practicing stunts which we strongly discourage: changing Derby properties on the fly. In general, there will always be Heisenbugs for applications which do this and we should discourage this practice in the strongest possible terms. What is the benefit of reverting the test behavior? I think the scope of the problem is beyond generating derby.properties on the fly. I think the greater risk is anyone setting system properties to affect derby behaviour. The biggest trouble maker I am guessing would be derby.system.home. The new requirement (I think) is that you set derby.system.home before any driver is autoloaded and would have the symptoms described in this mail. Is that description correct? http://www.nabble.com/Re%3A--jira--Commented%3A-%28DERBY-930%29-Add-support-for-autoloading-of-Derby-client-drivers-p4909085.html I find it not uncommon to see code like this: System.setProperty(derby.system.home, MY_DERBY_DIR); Class.forName(org.apache.derby.jdbc.EmbeddedDriver).newInstance(); You are right, that there are multiple ways to get to the bug. One way is to regenerate derby.properties itself in the directory indicated by derby.system.home. Another way to trigger the behavior is to reset the value of derby.system.home so that it points to a directory with a different derby.properties. I defer to your experience as to which of these is the more travelled route. If I understand the problem correctly, this will no longer reliably work if any other code in the same JVM (or is it classloader context?) makes a database connection to any database. Is that correct? Right. It does not reliably work today if the VM runs more than one embedded derby application, one of which is self-configuring. That's bug DERBY-1428. JDBC4 introduces a new scenario for the bug, as you describe: the bug can turn up if jdk1.6 runs a self-configuring Derby application together with another application, which connects to some database. That's DERBY-1429. The point of seeing if we can keep the tests the same is to see if we can avoid user impact. We should olnly change the tests if and when we decide the behavior change is ok. If later we add better tests that more accurately reflect actual user scemarios like the one above instead, that would be great, but the bottom line is just like the tests, USERS DO STRANGE THINGS and expect things to still work. If they upgrade and things suddenly don't work, they want the old behaviour back and we can't give it to them because new users depend on the new behaviour. We trap ourselves into a very ugly corner.and this is a very bad and messy thing. It is better to keep things working if we can. Every time we change a test to adapt to some new usage requirement it should be a huge red flag that someone is going to hit the problem. Kathey Thanks for the explanation. I'm looking forward to your analysis and to more feedback from the user community so that we can decide what to do here.
How to best handle variations in output from ij in sql tests?
I am working on reducing the number of tests failing when running with
JDK 1.6. Some of the tests are now failing due to jdk16 specific
master files that have not been updated after introduction of textual
changes. To solve some of these failing tests I try eliminate the need
for jdk16 specific master files.
To avoid creating to more sed rules for filtering some of the
differences I wonder if some of the changes better had been fixed by
making sure that the output is more consistent between client and
embedded driver and between JDK 1.5 and JDK 1.6.
I would like to get feedback on whether the following example should
best be solved by fixing the difference between the output from ij or
by continue to have multiple master files. This example is from the
ieptests.sql which now fails with JDK 1.6 with the client driver:
SQL that makes the test fail:
=
ij call SYSCS_UTIL.SYSCS_EXPORT_TABLE ('inventory', 'ORDERTABLE' ,
'extinout/order.dat', null, null, null) ;
Java 1.5 Embedded output:
=
ERROR 38000: The exception 'SQL Exception: Schema 'inventory' does not
exist' was thrown while evaluating an expression.
ERROR 42Y07: Schema 'inventory' does not exist
Java 1.6 Embedded output:
=
ERROR 38000: The exception 'java.sql.SQLSyntaxErrorException: Schema
'inventory' does not exist' was thrown while evaluating an expression.
ERROR 42Y07: Schema 'inventory' does not exist
Java 1.5 Client output:
===
ERROR 38000: The exception 'SQL Exception: Schema 'inventory' does not
exist' was thrown while evaluating an expression. SQLSTATE: 42Y07:
Schema 'inventory' does not exist
Java 1.6 Client output:
===
ERROR 38000: The exception 'java.sql.SQLSyntaxErrorException: Schema
'inventory' does not exist' was thrown while evaluating an expression.
The main difference is:
-embedded: with JDK 1.5 the output writes SQL Exception while on
JDK 1.6 it is java.sql.SQLSyntaxErrorException (more exact, but
less read-friendly?). This difference is handled by seding the
output.
-client: with JDK 1.5 the SQL state is included in the output, while
with JDK 1.6 the SQL state is not included. Is this a bug?
-embedded vs server: embedded writes the output split over two lines,
while the client has everything on one line. Could this be changed?
In addition to the detailed question about the specific output making
this test fail, I wonder if we should have some general rules that we
should attempt the output (error/warnings/responses) from ij be as
similar as possible for (a) between client and server and (b) between
different versions of the JVM?
Any feedback is appreciated!
Thanks,
Olav
Re: Driver autoloading and engine booting
Rick Hillegas wrote: Jean T. Anderson wrote: Rick Hillegas wrote: ... Does the precedence for properties documentation need to be updated to mention jdbc 4/jdk 1.6 behavior? http://db.apache.org/derby/docs/dev/tuning/ctunsetprop23308.html I don't see that the precedence of properties has changed. What has changed is when we read properties during the lifetime of a vm. Driver-autoloading does require many other edits to the user guides. They are summarized in DERBY-1271. ok, good, thanks for the clarification. -jean
[jira] Commented: (DERBY-578) Grouped select from temporary table raises null pointer exception in byte code generator
[ http://issues.apache.org/jira/browse/DERBY-578?page=comments#action_12416803 ] Rick Hillegas commented on DERBY-578: - Thanks, Manish. As you said, the fix looks pretty simple. I'm running regression tests now. I have a couple questions: 1) I don't understand the change to xaHelper. How is that related to this bugfix? 2) Can you explain the test diffs? On the surface, it looks like a number of index scans have been replaced with table scans. However, it's hard to trust the original claim that an index was being used because there doesn't seem to be a usable index. Please help me muddle through these diffs. -Rick Grouped select from temporary table raises null pointer exception in byte code generator Key: DERBY-578 URL: http://issues.apache.org/jira/browse/DERBY-578 Project: Derby Type: Bug Components: SQL, Services Reporter: Rick Hillegas Assignee: Manish Khettry Attachments: 578.diff.new.txt, 578.diff.txt, predicatePushdown.out.patch The last statement in the following script: autocommit off; declare global temporary table session.ztemp ( orderID varchar( 50 ) ) not logged; select orderID from session.ztemp group by orderID; raises the following stack trace in the trunk: java.lang.NullPointerException at java.util.Hashtable.get(Hashtable.java:333) at org.apache.derby.iapi.services.classfile.ClassHolder.findMatchingEntry(ClassHolder.java:656) at org.apache.derby.iapi.services.classfile.ClassHolder.addUtf8Entry(ClassHolder.java:482) at org.apache.derby.iapi.services.classfile.ClassHolder.addString(ClassHolder.java:506) at org.apache.derby.iapi.services.classfile.ClassHolder.addConstant(ClassHolder.java:302) at org.apache.derby.impl.services.bytecode.BCMethod.push(BCMethod.java:452) at org.apache.derby.impl.sql.compile.FromBaseTable.generateDistinctScan(FromBaseTable.java:3216) at org.apache.derby.impl.sql.compile.FromBaseTable.generateResultSet(FromBaseTable.java:3032) at org.apache.derby.impl.sql.compile.FromBaseTable.generate(FromBaseTable.java:2978) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(ProjectRestrictNode.java:1196) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(ProjectRestrictNode.java:1145) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(ProjectRestrictNode.java:1196) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(ProjectRestrictNode.java:1145) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(ProjectRestrictNode.java:1196) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(ProjectRestrictNode.java:1145) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generateMinion(ProjectRestrictNode.java:1196) at org.apache.derby.impl.sql.compile.ProjectRestrictNode.generate(ProjectRestrictNode.java:1145) at org.apache.derby.impl.sql.compile.ScrollInsensitiveResultSetNode.generate(ScrollInsensitiveResultSetNode.java:109) at org.apache.derby.impl.sql.compile.ReadCursorNode.generate(ReadCursorNode.java:118) at org.apache.derby.impl.sql.compile.CursorNode.generate(CursorNode.java:546) at org.apache.derby.impl.sql.compile.StatementNode.generate(StatementNode.java:232) at org.apache.derby.impl.sql.GenericStatement.prepMinion(GenericStatement.java:433) at org.apache.derby.impl.sql.GenericStatement.prepare(GenericStatement.java:107) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(GenericLanguageConnectionContext.java:704) at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:533) at org.apache.derby.impl.jdbc.EmbedStatement.execute(EmbedStatement.java:480) at org.apache.derby.impl.tools.ij.ij.executeImmediate(ij.java:299) at org.apache.derby.impl.tools.ij.utilMain.doCatch(utilMain.java:433) at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:310) at org.apache.derby.impl.tools.ij.Main.go(Main.java:203) at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:169) at org.apache.derby.impl.tools.ij.Main14.main(Main14.java:55) at org.apache.derby.tools.ij.main(ij.java:60) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: How to best handle variations in output from ij in sql tests?
Hi Olav,
Thanks for digging into this. The differences in the embedded output are
caused by JDBC4's introduction of SQLException subclasses. These now
replace Derby's hand-rolled subclass of SQLException. The old Derby
subclass overrode the getMessage() method. That's why you used to see
SQL Exception:... and now you see
java.sql.SQLSyntaxErrorException:... I suppose that we could get
close to the old message text if we introduced a Derby subclass for each
of the JDBC4 exception subclasses and then override the getMessage() in
each of these. This seems like a lot of code bloat without much payback.
I think that handling these discrepancies with a sed script is fine.
I don't understand why the trailing SQLSTATE info has been stripped off
the network messages. Perhaps network serialization of chained
exceptions needs a closer look.
Regards,
-Rick
Olav Sandstaa wrote:
I am working on reducing the number of tests failing when running with
JDK 1.6. Some of the tests are now failing due to jdk16 specific
master files that have not been updated after introduction of textual
changes. To solve some of these failing tests I try eliminate the need
for jdk16 specific master files.
To avoid creating to more sed rules for filtering some of the
differences I wonder if some of the changes better had been fixed by
making sure that the output is more consistent between client and
embedded driver and between JDK 1.5 and JDK 1.6.
I would like to get feedback on whether the following example should
best be solved by fixing the difference between the output from ij or
by continue to have multiple master files. This example is from the
ieptests.sql which now fails with JDK 1.6 with the client driver:
SQL that makes the test fail:
=
ij call SYSCS_UTIL.SYSCS_EXPORT_TABLE ('inventory', 'ORDERTABLE' ,
'extinout/order.dat', null, null, null) ;
Java 1.5 Embedded output:
=
ERROR 38000: The exception 'SQL Exception: Schema 'inventory' does not
exist' was thrown while evaluating an expression.
ERROR 42Y07: Schema 'inventory' does not exist
Java 1.6 Embedded output:
=
ERROR 38000: The exception 'java.sql.SQLSyntaxErrorException: Schema
'inventory' does not exist' was thrown while evaluating an expression.
ERROR 42Y07: Schema 'inventory' does not exist
Java 1.5 Client output:
===
ERROR 38000: The exception 'SQL Exception: Schema 'inventory' does not
exist' was thrown while evaluating an expression. SQLSTATE: 42Y07:
Schema 'inventory' does not exist
Java 1.6 Client output:
===
ERROR 38000: The exception 'java.sql.SQLSyntaxErrorException: Schema
'inventory' does not exist' was thrown while evaluating an expression.
The main difference is:
-embedded: with JDK 1.5 the output writes SQL Exception while on
JDK 1.6 it is java.sql.SQLSyntaxErrorException (more exact, but
less read-friendly?). This difference is handled by seding the
output.
-client: with JDK 1.5 the SQL state is included in the output, while
with JDK 1.6 the SQL state is not included. Is this a bug?
-embedded vs server: embedded writes the output split over two lines,
while the client has everything on one line. Could this be changed?
In addition to the detailed question about the specific output making
this test fail, I wonder if we should have some general rules that we
should attempt the output (error/warnings/responses) from ij be as
similar as possible for (a) between client and server and (b) between
different versions of the JVM?
Any feedback is appreciated!
Thanks,
Olav
[jira] Commented: (DERBY-1428) Generating derby properties on the fly can lead to non-deterministic engine startup behavior
[ http://issues.apache.org/jira/browse/DERBY-1428?page=comments#action_12416810 ] Rick Hillegas commented on DERBY-1428: -- Kathey adds the following: I think DERBY-1428 and DERBY-149 are not really related to generating derby.properties on the fly they are related to when Derby boots and when derby.system.home is recognized. derby.system.home *has* to be set with a system property. Anyone who sets derby.system.home or static derby properties programatically with System.setProperty() may be affected if they do not have control over their calling context. Another driver might cause derby to be autoloaded before these properties are set. Perhaps these issues would be better worded as: Derby can have non-deterministic behaviour when derby.system.home or other static properties are set with System.setProperty() More common reported symptoms will likely be after upgrading to Derby 10.2 and JDK 1.6 users will report the following systems because derby.system.home is not picked up properly. - I get XJ004.C=Database ''...'' not found.(Derby is looking in the wrong place for the db) - There is nothing in my database. (create=true was on the url and so app created a new db in the wrong place) - My database was created in the wrong place. - Properties set in derby.properties are not being picked up. Some more info: http://www.nabble.com/Re%3A--jira--Commented%3A-%28DERBY-930%29-Add-support-for-autoloading-of-Derby-client-drivers-p4909085.html Generating derby properties on the fly can lead to non-deterministic engine startup behavior Key: DERBY-1428 URL: http://issues.apache.org/jira/browse/DERBY-1428 Project: Derby Type: Bug Components: Store Versions: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0, 10.2.0.0, 10.1.2.1, 10.1.3.0, 10.1.2.2, 10.1.2.3, 10.3.0.0, 10.1.2.4, 10.1.2.5, 10.1.4.0, 10.1.3.1 Reporter: Rick Hillegas A Heisenbug can arise if more than one embedded Derby application runs in the same VM and at least one of them generates derby properties on the fly. Here's the problem scenario: o The customer runs two embedded Derby apps in the same VM: EmbeddedApp and OtherApp. o EmbeddedApp generates derby properties on the fly before connecting to a database and triggering engine startup. o Whether the engine picks up those generated properties depends on whether EmbeddedApp or OtherApp runs first. Here are two workarounds for the problem: 1) Don't generate derby properties on the fly. Instead, specify them on the VM startup command. Or specify them in a $DERBY_HOME/derby.properties file which is generated before the VM comes up and which does not change during the lifetime of the VM. 2) If you can't do (1), then modify the VM startup script so that the self-configuring EmbeddedApp runs first. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Driver autoloading and engine booting
Jean T. Anderson wrote: Rick Hillegas wrote: Jean T. Anderson wrote: Rick Hillegas wrote: ... Does the precedence for properties documentation need to be updated to mention jdbc 4/jdk 1.6 behavior? http://db.apache.org/derby/docs/dev/tuning/ctunsetprop23308.html I don't see that the precedence of properties has changed. What has changed is when we read properties during the lifetime of a vm. Driver-autoloading does require many other edits to the user guides. They are summarized in DERBY-1271. ok, good, thanks for the clarification. I take the ok back based on Kathey's post to derby-user [1]. The actual precedence for a static variable will be whatever was visible when another driver caused derby to be autoloaded, possibly before the application set that property. Kathey's derby.system.home example is probably the most likely confusion to occur. Let's say my application connects to MySQL to migrate a database to Derby. When I connect to that MySQL database, the embedded derby driver gets loaded and the embedded engine gets booted. Later, my application calls System.setProperty(derby.system.home, /home/jta/DerbyDatabases) and creates the new derby database. But it doesn't get created where I expected -- the derby.system.home I set is ignored because the derby engine was already booted. -jean [1] http://mail-archives.apache.org/mod_mbox/db-derby-user/200606.mbox/[EMAIL PROTECTED]
[jira] Commented: (DERBY-1338) Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir
[ http://issues.apache.org/jira/browse/DERBY-1338?page=comments#action_12416817 ] Dag H. Wanvik commented on DERBY-1338: -- From: Bryan Pendleton, Date: Tue, 23 May 2006 16:17:59 -0700 The shutdown of the server *has* interrupted the thread, in NetworkServerStarter#blockingStart. So, is this the reason we get this exception? This is *very* interesting -- thank you for doing this great research! This sort of thing is exactly why I've been so scared to add more calls to interrupt() as part of DERBY-1219 and DERBY-1326 -- interrupt() is a very scary call to make. The good news is that this does mean that this is purely a shutdown problem, as the only place that we interrupt the DRDAConnThread instances right now is at the very end of the blockingStart() method. But it certainly means we'll have to be even more careful about whatever fix we eventually make for DERBY-1326, as we definitely don't want to casually add any more interrupt() calls, given the possibility of horrible side effects like this. As I mentioned before, we can make sure the classes in question (DRDAProtocolExceptionInfo and DRDAProtocolException) are already loaded (by creating a dummy object) before the thread is interrupted, thus sidestepping the issue. Alternatively, we could remove the interrupt() call in Network Server shutdown. Since we are shutting down the server anyway, I don't know how crucial it is to clean up those background threads. We have called close() on them already, so if they try to run again, they will quickly detect that they have been closed and exit themselves. And, so long as they have been marked as daemon threads, their mere presence won't prevent the server from shutting down. But I'm also fine with pre-loading the classes if that seems to help. These sorts of startup/shutdown bugs always seem very hard to me: trying to shut down a complex piece of server software while all sorts of complicated processing may be still underway has always been a dicey business. thanks, bryan Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir - Key: DERBY-1338 URL: http://issues.apache.org/jira/browse/DERBY-1338 Project: Derby Type: Bug Components: Regression Test Failure Versions: 10.2.0.0 Environment: Sun JDK 1.4.2, Sun JDK 1.5 Reporter: Dag H. Wanvik Assignee: Dag H. Wanvik Priority: Minor Fix For: 10.2.0.0 When run from Sun JDK 1.4.2 and Sun JDK 1.5, these two tests fail when run from the classes directory. They work, though, when run from jars. Running NSinSameVM in my sandbox: bash-3.00$ java -Dframework=DerbyNetClient org.apache.derbyTesting.functionTests.harness.RunTest derbynet/NSinSameJVM.java *** Start: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:05 *** Initialize for framework: DerbyNetClient startServer = false. Bypass server startup 6 add java.lang.NoClassDefFoundError: org/apache/derby/impl/drda/DRDAProtocolExceptionInfo Test Failed. *** End: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:27 *** I get similar behavior for DerbyNetNewServer. I ran this on a Solaris 10/i86x box. Davis has seen it too, see initial discusson on this thread: http://www.nabble.com/forum/ViewPost.jtp?post=4477600framed=y -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1338) Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir
[ http://issues.apache.org/jira/browse/DERBY-1338?page=all ] Dag H. Wanvik updated DERBY-1338: - Attachment: derby-1338.stat derby-1338.diff Here is a simple patch which preloads the class causing the problem. It makes the tests run correctly - derbyall ran without errors on the classes directory on my Solaris 10/x86 box). I don't have to time to work on getting rid of the thread interrupts right now; although I would prefer that solution. I can leave this issue open or we could file another one. Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir - Key: DERBY-1338 URL: http://issues.apache.org/jira/browse/DERBY-1338 Project: Derby Type: Bug Components: Regression Test Failure Versions: 10.2.0.0 Environment: Sun JDK 1.4.2, Sun JDK 1.5 Reporter: Dag H. Wanvik Assignee: Dag H. Wanvik Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1338.diff, derby-1338.stat When run from Sun JDK 1.4.2 and Sun JDK 1.5, these two tests fail when run from the classes directory. They work, though, when run from jars. Running NSinSameVM in my sandbox: bash-3.00$ java -Dframework=DerbyNetClient org.apache.derbyTesting.functionTests.harness.RunTest derbynet/NSinSameJVM.java *** Start: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:05 *** Initialize for framework: DerbyNetClient startServer = false. Bypass server startup 6 add java.lang.NoClassDefFoundError: org/apache/derby/impl/drda/DRDAProtocolExceptionInfo Test Failed. *** End: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:27 *** I get similar behavior for DerbyNetNewServer. I ran this on a Solaris 10/i86x box. Davis has seen it too, see initial discusson on this thread: http://www.nabble.com/forum/ViewPost.jtp?post=4477600framed=y -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-1108) The test jdbcapi/setTransactionIsolation.java fails with ibm jvm1.5
[
http://issues.apache.org/jira/browse/DERBY-1108?page=comments#action_12416823 ]
A B commented on DERBY-1108:
I'm not very well acquainted with this code, but based on the example of
escape analysis posted by the JVM team, I think I understand why this test is
failing with IBM 1.5.
If we look at the test program again, we can see that it very closely mimics
the example of escape analysis offered by the JVM team. In particular:
System.out.println(done creating table and inserting data.);
conn.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED);
ResultSet rs = stmt.executeQuery(select * from T1);
== Under Java 5.0 rs is eligible for GC here...
System.out.println(current isolation level: +
getIsoLevelName(conn.getTransactionIsolation()));
conn.setTransactionIsolation(java.sql.Connection.TRANSACTION_SERIALIZABLE);
System.out.println(current isolation level: +
getIsoLevelName(conn.getTransactionIsolation()));
== Under previous versions, rs is eligible for GC here...
So with ibm15, rs becomes eligible for GC _prior_ to the call to
setTransactionIsolation because it (rs) is no longer referenced after it is
created. Then the call to gc() in verifyAllHeldResultSetsAreClosed() does the
escape analysis, sees that rs is now eligible for GC and so collects it. As
part of that collection process the JVM calls finalize(), which takes us to
EmbedResultSet.finalize(), and there we see this:
/**
JDBC states that a ResultSet is closed when garbage collected.
We simply mark the activation as unused. Some later use
of the connection will clean everything up.
@exception Throwable Allows any exception to be thrown during
finalize
*/
protected void finalize() throws Throwable {
super.finalize();
if (finalizeActivation != null) {
finalizeActivation.markUnused();
}
}
Thus finalizeActiviation is now marked as unused, which means we end up
ignoring the activation in the verifyAllHeldResultSetsAreClosed() method:
if (!a.isInUse())
{
continue;
}
And therefore we return true from the method and no error is thrown.
To prove this, I added a very simple (albeit meaningless) reference to rs
after the call to setTransactionIsolation:
System.out.println(done creating table and inserting data.);
conn.setTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED);
ResultSet rs = stmt.executeQuery(select * from T1);
System.out.println(current isolation level: +
getIsoLevelName(conn.getTransactionIsolation()));
conn.setTransactionIsolation(java.sql.Connection.TRANSACTION_SERIALIZABLE);
System.out.println(current isolation level: +
getIsoLevelName(conn.getTransactionIsolation()));
System.out.println(rs); // == Added by Army
With this simple line, the escape analysis will *not* mark rs eligible for GC
prior to setTransactionIsolation, which means we won't call finalize(), which
means the activation will remain in use, and therefore we'll get the
exception.
All of that said, I added System.outs to the test program to see what the
object id of rs is before the call to setTransactionIsolation, and I also
added System.outs to see what the object id of the rs is inside
verifyAllHeldResultSetsAreClosed(). Not surprisingly, the objects are NOT the
same. In the test program we see:
[EMAIL PROTECTED]
but in verifyAllHeldResultSetsAreClosed() we see:
[EMAIL PROTECTED]
So as far as I can tell, the test program no longer references the
EmbedResultSet, so it's marked as eligible for GC and is garbage collected by
escape analysis; however, that EmbedResultSet was in turn referencing another
object--BulkTableScanResultSet--that *is* still referenced (by the activation),
and that's the result set that should have caused the error--but since we
bailed on the error checking because the activiation was no longer in use,
we never checked the underlying result set and therefore we didn't throw the
exception.
That's my take on it after looking at it briefly. I still don't know for sure
who's at fault here (the JVM or Derby)--I'd have to look at it more to say
one way or the other. But that's my first take on the issue...
Comments/input/feedback are, as always, much appreciated...
Army
The test jdbcapi/setTransactionIsolation.java fails with ibm jvm1.5
---
Key: DERBY-1108
URL: http://issues.apache.org/jira/browse/DERBY-1108
Project: Derby
Type: Bug
Components: Regression Test Failure
Versions: 10.2.0.0
Environment: java
[jira] Commented: (DERBY-1108) The test jdbcapi/setTransactionIsolation.java fails with ibm jvm1.5
[ http://issues.apache.org/jira/browse/DERBY-1108?page=comments#action_12416825 ] Kathey Marsden commented on DERBY-1108: --- Thanks Army for the comment. I think I understand escape analysis better now and how it relatest to this issue and I think 1) The JVM is ok. It should garbage collect and finalize rs and all its innards because they can never really be used again. 2) Derby is ok. setTransactionIsolation() does not need to consider a ResultSet that can never be used again . 3) The test is not ok. To test what it wants to test, that we can't change the isolation in the middle of retrieving results on a held cursor, it should have code to use rs after setTransactionIsolation, e.g. add an rs.next() after setTransactionIsolation() Does that sound right? The test jdbcapi/setTransactionIsolation.java fails with ibm jvm1.5 --- Key: DERBY-1108 URL: http://issues.apache.org/jira/browse/DERBY-1108 Project: Derby Type: Bug Components: Regression Test Failure Versions: 10.2.0.0 Environment: java version 1.5.0 Java(TM) 2 Runtime Environment, Standard Edition (build pwi32dev-20051104) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-2005110 3 (JIT enabled) J9VM - 20051027_03723_lHdSMR JIT - 20051027_1437_r8 GC - 20051020_AA) JCL - 20051102 Reporter: Manjula Kutty Attachments: test.java The test jdbcapi/setTransactionIsolation.java fails with ibmjvm15. I think the cause of this failure is , the jvm is not throwing an exception while trying to change the transaction isolation when there is holdable cursor on a resultset. Other jvms like sun jdk1.5, ibm142 throws the expected exception. while ibm15 does allow to change the transaction isolation when there is a hold cursor on a result set. Already reported this issue with the ibmjvm people and they are looking in to it -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1338) Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir
[ http://issues.apache.org/jira/browse/DERBY-1338?page=all ] Dag H. Wanvik updated DERBY-1338: - Derby Info: [Patch Available] Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir - Key: DERBY-1338 URL: http://issues.apache.org/jira/browse/DERBY-1338 Project: Derby Type: Bug Components: Regression Test Failure Versions: 10.2.0.0 Environment: Sun JDK 1.4.2, Sun JDK 1.5 Reporter: Dag H. Wanvik Assignee: Dag H. Wanvik Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1338.diff, derby-1338.stat When run from Sun JDK 1.4.2 and Sun JDK 1.5, these two tests fail when run from the classes directory. They work, though, when run from jars. Running NSinSameVM in my sandbox: bash-3.00$ java -Dframework=DerbyNetClient org.apache.derbyTesting.functionTests.harness.RunTest derbynet/NSinSameJVM.java *** Start: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:05 *** Initialize for framework: DerbyNetClient startServer = false. Bypass server startup 6 add java.lang.NoClassDefFoundError: org/apache/derby/impl/drda/DRDAProtocolExceptionInfo Test Failed. *** End: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:27 *** I get similar behavior for DerbyNetNewServer. I ran this on a Solaris 10/i86x box. Davis has seen it too, see initial discusson on this thread: http://www.nabble.com/forum/ViewPost.jtp?post=4477600framed=y -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-1338) Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir
[ http://issues.apache.org/jira/browse/DERBY-1338?page=all ] Dag H. Wanvik updated DERBY-1338: - Component: Network Server (was: Regression Test Failure) Environment: Sun JDK 1.4.2, Sun JDK 1.5 on Solaris was: Sun JDK 1.4.2, Sun JDK 1.5 Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir - Key: DERBY-1338 URL: http://issues.apache.org/jira/browse/DERBY-1338 Project: Derby Type: Bug Components: Network Server Versions: 10.2.0.0 Environment: Sun JDK 1.4.2, Sun JDK 1.5 on Solaris Reporter: Dag H. Wanvik Assignee: Dag H. Wanvik Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1338.diff, derby-1338.stat When run from Sun JDK 1.4.2 and Sun JDK 1.5, these two tests fail when run from the classes directory. They work, though, when run from jars. Running NSinSameVM in my sandbox: bash-3.00$ java -Dframework=DerbyNetClient org.apache.derbyTesting.functionTests.harness.RunTest derbynet/NSinSameJVM.java *** Start: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:05 *** Initialize for framework: DerbyNetClient startServer = false. Bypass server startup 6 add java.lang.NoClassDefFoundError: org/apache/derby/impl/drda/DRDAProtocolExceptionInfo Test Failed. *** End: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:27 *** I get similar behavior for DerbyNetNewServer. I ran this on a Solaris 10/i86x box. Davis has seen it too, see initial discusson on this thread: http://www.nabble.com/forum/ViewPost.jtp?post=4477600framed=y -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Driver autoloading and engine booting
Rick Hillegas wrote: This topic was raised in an earlier email thread (http://comments.gmane.org/gmane.comp.apache.db.derby.devel/20899) but it does not appear that we agreed on a solution. I would like to re-open this topic and hopefully we can converge on how we want to handle this issue. Olav has analyzed a problematic interaction between JDBC4-driver-autoloading and Derby-embedded-engine-booting. In addition to the above email thread, the problem is also discussed in DERBY-1399. Here's a brief sketch of the issue: 1) Remove the JDBC4-driver-autoloading. This removes a useful ease-of-development feature and makes us fail to be JDBC4-compliant. 2) Don't boot the engine when registering the embedded driver. Instead, boot the engine the first time that someone requests an embedded Connection. This approach involves a lot of testing. Thanks for explaing the problem clearly Rick. I believe it is not uncommon for the application to have a code that connects to different database from different vendors and only make connection to one of them at a time as needed in different parts of their code at runtime. I think booting derby (Actually it is just the monitor service modules that gets booted not underlying storage ..etc.) when user is using a different vendor database does not sound right. In my openion, this problem should be fixed if possible. Thanks -suresh
Re: interested in working on DERBY-713
Hiya Rick,
Thanks for you helpful reply! I spent some time over the weekend
snooping around (mostly) InListOperatorNode and OptimizerImpl, but
never found out _where_ the table scan was chosen over an index scan
for finding the IN matches...I really wanted to find it before begging
for someone to point out the obvious :-) Specifically, I got all
tangled up around the dynamically generated classes which implement
the chosen query plan. Also, I didn't ever find any QueryPlan-like
class, but I guess this what the dynamic class(es) implement.
What did I want to do once I got there? Pseudo-code says it best:
// ISTR that the bound IN values are ordered and unique. If not,
// this should be done...
if( column is indexed ){
if( index is unique ){
// this should be a guaranteed win. We know we'll only get
// one match per IN value
do index scan
}
else if( table size SOME_MAGIC_SIZE ){
// SOME_MAGIC_SIZE would represent the size of (the) table
// at which using the index would be reasonably known to
// be more expensive than scanning the table.
do index scan
}
else {
do table scan
}
}
else {
do table scan
}
Is that a workable approach (excepting the SOME_MAGIC_SIZE bit)?
I'll probably have some time later this week to immerse myself in locating
the magic location to apply this logic. A _little_ pointer would be
helpful: maybe a brief overview of how a query plan is determined -- I
suspect his is somewhere around where trulyTheBestTableAccess (or whatever
it's really called :-)) is defined. I'd rather have to bang my head
against this wall a bit, to get a better understanding of things, instead
of being pointed immediately to the solution -- my own version of smelling
the roses, I guess ;-)
Cheers!
Brent
[2006-06-19 11:30] Rick Hillegas said:
| Hi Brent,
|
| Sounds like you're off to a good start. From the initial bug report, it
| looks like there's a good idea about which heuristic is being
| mis-applied here. Once you've studied the optimizer papers, I recommend
| that you post some high-level candidate solutions. Try to avoid
| optimizer jargon and concentrate on simple descriptions:
|
| o What query plan would you rather see?
| o What heuristic would the optimizer apply that would lead it to your
| preferred plan?
| o How would the optimizer decide to apply the new heuristic rather than
| the old one?
|
| I think you'll get some good feedback if your post contains the phrase
| Attention, optimizer experts. I think that the optimizer enthusiasts
| on the list will give you good feedback:
|
| o Maybe they can think of a better query plan or heuristic.
| o Maybe they can see some awkward corner cases in your heuristic.
| o They can advise you on whether your heuristic will short-circuit other
| optimizer choices.
| o They can advise you on whether your heuristic will cause an explosion
| in the time that the optimizer takes.
|
| Thanks for wanting to scratch this itch!
|
| Regards,
| -Rick
|
| Brent Verner wrote:
|
| Hi,
|
| I've recently found need for an embedded java db and only Derby seems
| even close to handling the task, however the broken query planning for IN
| clauses makes it unusable :-(. I've decided to eschew an embedded db
| in favor of PG for now, but I'd really like to be able to use Derby in the
| near(ish) future for deployment.
|
| I'd like to try to fix the query planning around the IN clause. I'm
| reviewing the internals papers right now, but I'd appreciate any input
| that might point me in the right direction :-).
|
| cheers!
| Brent
|
|
|
[jira] Commented: (DERBY-1338) Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir
[ http://issues.apache.org/jira/browse/DERBY-1338?page=comments#action_12416840 ] Bryan Pendleton commented on DERBY-1338: Hi Dag, I looked at your patch. Unless anybody else has a better suggestion, I think we should go ahead and use your patch, since it clearly improves the behavior in your environment. I think that we should put a more complete comment in the code itself, though, so that people aren't required to visit JIRA unless they really need the detailed background. What do you think about the following comment -- is it accurate? // Work around a classloader bug involving interrupt handling during // class loading. If the first request to load the DRDAProtocolException // class occurs during shutdown, the loading of the class may be aborted // when the Network Server calls Thread.interrupt() on the DRDAConnThread. // By including a static reference to the DRDAProtocolException class here, // we ensure that it is loaded as soon as the DRDAConnThread class is loaded, // and therefore we know we won't be trying to load the class during shutdown. // See DERBY-1338 for more background, including pointers to the apparent // classloader bug in the JVM. If this comment seems reasonable to you, and if the community doesn't come up with any better suggestions, I can commit this change later this week. Client tests DerbyNetNewServer and NSinSameVM fail with NoClassDefFoundError: DRDAProtocolExceptionInfo when run from classes dir - Key: DERBY-1338 URL: http://issues.apache.org/jira/browse/DERBY-1338 Project: Derby Type: Bug Components: Network Server Versions: 10.2.0.0 Environment: Sun JDK 1.4.2, Sun JDK 1.5 on Solaris Reporter: Dag H. Wanvik Assignee: Dag H. Wanvik Priority: Minor Fix For: 10.2.0.0 Attachments: derby-1338.diff, derby-1338.stat When run from Sun JDK 1.4.2 and Sun JDK 1.5, these two tests fail when run from the classes directory. They work, though, when run from jars. Running NSinSameVM in my sandbox: bash-3.00$ java -Dframework=DerbyNetClient org.apache.derbyTesting.functionTests.harness.RunTest derbynet/NSinSameJVM.java *** Start: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:05 *** Initialize for framework: DerbyNetClient startServer = false. Bypass server startup 6 add java.lang.NoClassDefFoundError: org/apache/derby/impl/drda/DRDAProtocolExceptionInfo Test Failed. *** End: NSinSameJVM jdk1.4.2_05 DerbyNetClient 2006-05-22 14:49:27 *** I get similar behavior for DerbyNetNewServer. I ran this on a Solaris 10/i86x box. Davis has seen it too, see initial discusson on this thread: http://www.nabble.com/forum/ViewPost.jtp?post=4477600framed=y -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
