Re: org.apache.derbyTesting.functionTests.suites.All is extremely CPU intensive!
Andrew McIntyre wrote (2007-02-15 12:24:58): On 2/15/07, Yip Ng [EMAIL PROTECTED] wrote: Thanks for the info Bernt. I did a svn update, recompile and rerun the junit tests. DERBY-2339 have fixed the problem on the stalling junit regression but I am getting 2 failures in the DRDAProtocolTest... I see these in my nightly tests as well, but only on Windows. Logged as DERBY-2295: https://issues.apache.org/jira/browse/DERBY-2295 DERBY-2295 was reported before DERBY-2108 was committed, so it's not related. andrew -- Bernt Marius Johnsen, Database Technology Group, Staff Engineer, Technical Lead Derby/Java DB Sun Microsystems, Trondheim, Norway pgpplttKch9Jv.pgp Description: PGP signature
[jira] Updated: (DERBY-827) Performance can be improved by re-using language ResultSets across Activation executions.
[ https://issues.apache.org/jira/browse/DERBY-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyre Tjeldvoll updated DERBY-827: - Attachment: rsfromps_prelim.diff I've attached a preliminary (hence no ASF license) patch that creates a new junit test that attempts to fill the mentioned gap in the existing test suites. The new test instantiates all ResultSets returned from GenericResultSetFactory from a PreparedStatement that is executed repeatedly. At this point all ResultSets except InsertVTIResultSet, DeleteVTIResultSet, UpdateVTIResultSet and MaterializedResultSet are created. According to the EMMA reports these ResultSets are never created when running derbyall or the junit tests, so I don't know how to write a test case that does. The preliminary version of this patch adds a println to the ResultSetFactory which shows the ResultSets that are instantiated and the Statement ID and query that did so. This makes it easier to see if the test does what it is supposed to, but would obviously have to be removed in a final version of the patch. Reviews/comments would be much appreciated. Thanks. Performance can be improved by re-using language ResultSets across Activation executions. - Key: DERBY-827 URL: https://issues.apache.org/jira/browse/DERBY-827 Project: Derby Issue Type: Improvement Components: Performance Reporter: Daniel John Debrunner Attachments: derby827_draft_reuse_result_sets.txt, derby827_update920.txt, rsfromps_prelim.diff Shouldn't DistinctScalarAggregateRS implement a close or a finish method (not sure what the difference is) and close the scan controller there. The close() and finish() methods are actually explained in their javadoc in the language org.apache.derby.iapi.sql.ResultSet class. [note this is not a JDBC java.sql.ResultSet object] close() - Tells the system that there will be no more calls to getNextRow() (until the next open() call) finish() - Tells the system that there will be no more access to any database information via this result set So close means the ResultSet may be opened again for more access, while finish means it will not be used again. However, their use in the code always doesn't match that, and that does cause confusion, at least to me. Language ResultSets (not JDBC ones) can be and are opened multiple times, for example when scanning a table multiple times within a join. An Activation, which represents the internal state of java.sql.PreparedStatement object has the lifetime of the java.sql.PreparedStatement, contains a top-level language ResultSet. This top-level language ResultSet provides the execution of the SQL statement, DML, DDL or a query. The top-level ResultSet may contain other ResultSets and could be seen as a tree structure. For the simple case of a primary key lookup query like: select name from customer where id = ? The activation would contain this: top result set ProjectRestrictRS IndexRowToBaseRowRS TableScanRS Now for some reason, even though the api of ResultSet say they can be re-used, and in some cases they are, this result set tree is thrown away after each execution. That is, the top result set has its finish() method called and then the activation removes its reference to it. Then on the next execution a new (identical) tree is set up. There is potential for a huge performance gain if this top level result set and its tree are re-used and have the same lifetime as the Activation. The saving comes in two forms, not having to create many objects on each execution, and not creating short-lived objects for the garbage collector to handle. I made a simple fix, it's a couple of lines of code, just calling close finish at the correct times, and for the above simple primary key lookup query, the performance went from 17,300 to 24,000 selects per second (cached data, single user). I'll post a patch shortly as an indication of the direction, once I can separate it from other changes in my client. However, I'm running the Derby tests and there are some (maybe 25-30) failures, I think because not all the language ResultSet implementations are correctly written to be re-opened. Interestingly, the first failure I saw was in an aggregrate test, which goes back to the issue Manish saw. Even if derbyall passed I would be nervous about submitting this patch for real, because I don't think there's a lot of testing using repeat executions of PreparedStatements in the tests. The ij tests mainly use Statement, this is a single use of an activation so this change would not affect them. Thus such a patch could regress Derby by making it more likely existing bugs would be exposed. Given the performance gains, I think we need to start
Re: org.apache.derbyTesting.functionTests.suites.All is extremely CPU intensive!
On 2/16/07, Bernt M. Johnsen [EMAIL PROTECTED] wrote: Andrew McIntyre wrote (2007-02-15 12:24:58): On 2/15/07, Yip Ng [EMAIL PROTECTED] wrote: Thanks for the info Bernt. I did a svn update, recompile and rerun the junit tests. DERBY-2339 have fixed the problem on the stalling junit regression but I am getting 2 failures in the DRDAProtocolTest... I see these in my nightly tests as well, but only on Windows. Logged as DERBY-2295: https://issues.apache.org/jira/browse/DERBY-2295 DERBY-2295 was reported before DERBY-2108 was committed, so it's not related. Correct. The 2295 failure is not related to 2108. I was just noting that I am also seeing the same test issue, reported as DERBY-2295, that Yip is seeing. The failure in DropDatabaseSetup still appears in my runs of the junit tests on Windows with the latest checkout from the trunk. It does not appear on other platforms. andrew
[jira] Created: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) . . -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
[ https://issues.apache.org/jira/browse/DERBY-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg updated DERBY-2344: --- Attachment: VM_Summary.jpg VM_Summary-1.txt Attaching VM Summary and Overview from a JConsole session of a failing test run. 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg Attachments: VM_Summary-1.txt, VM_Summary.jpg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) . . -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
[ https://issues.apache.org/jira/browse/DERBY-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473655 ] John H. Embretsen commented on DERBY-2344: -- Ole wrote: I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. The Garbage Collection Ergonomics doc for the J2SE 5.0 HotSpot VM [1] indicates that the initial and max heap size is different on server-class machines compared to client-class machines (I believe client-class machines have a default max heap size of 64 MB, while server-class machines use the smaller of 1/4th of the physical memory or 1GB). The server-class machine detection algorithm of the HotSpot VM is described at [2]: For J2SE 5.0, the definition of a server-class machine is one with at least 2 CPUs and at least 2GB of physical memory. [1]: http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html [2]: http://java.sun.com/j2se/1.5.0/docs/guide/vm/server-class.html 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg Attachments: VM_Summary-1.txt, VM_Summary.jpg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) . . -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Regression Test Report - Daily 507623 - Sun DBTG
Problems caused by DERBY-2339 [EMAIL PROTECTED] wrote: [Auto-generated mail] *Daily* 507623/2007-02-14 18:00:11 CET Failed TestsOK Skip Duration Suite --- *Jvm: 1.6* lin 0456456 092.41% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sles NA NA NANA derbyall NA NA NANA suitesAll sol NA NA NANA derbyall NA NA NANA suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc NA NA NANA derbyall NA NA NANA suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/testing/Limited/testSummary-507623.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/FailReports/507623.html *Jvm: 1.5* lin 0450450 292.33% derbyall 025232523 0 1079.08% suitesAll sles 1450449 295.97% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sol 0450450 2 101.18% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll solN+1 0450450 2 122.15% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sparc 0450450 2 101.69% derbyall 025232523 0 2499.65% suitesAll vista 1450449 296.50% derbyall NA NA NANA suitesAll win 0450450 2 112.75% derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/testing/Limited/testSummary-507623.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/FailReports/507623.html *Jvm: 1.4* lin NA NA NANA derbyall NA NA NANA suitesAll sles NA NA NANA derbyall NA NA NANA suitesAll sol NA NA NANA derbyall NA NA NANA suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc NA NA NANA derbyall NA NA NANA suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/testing/Limited/testSummary-507623.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/FailReports/507623.html --- Changes in http://dbtg.thresher.com/derby/test/Daily/UpdateInfo/507623.txt ( All results in http://dbtg.thresher.com/derby/test/ ) -- With regards, Henri van de Scheur, Database Technology Group, Sun Microsystems, Trondheim, Norway
Re: [jira] Created: (DERBY-2339) org.apache.derbyTesting.functionTests.suites.All is extremely CPU intensive since ~svn 507489
Although I no longer see the intensive CPU-use, daily testing hangs on several platforms in this testsuite. More investigation is needed. Will come with an update. Henri Ole Solberg (JIRA) wrote: org.apache.derbyTesting.functionTests.suites.All is extremely CPU intensive since ~svn 507489 - Key: DERBY-2339 URL: https://issues.apache.org/jira/browse/DERBY-2339 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: All Reporter: Ole Solberg Run org.apache.derbyTesting.functionTests.suites.All. See e.g. http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-507489.html http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-507650.html These logs also include thread dumps. -- With regards, Henri van de Scheur, Database Technology Group, Sun Microsystems, Trondheim, Norway
[jira] Updated: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
[ https://issues.apache.org/jira/browse/DERBY-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg updated DERBY-2344: --- Attachment: VM_Summary-serverJvm-PS_Old_Gen.jpg VM_Summary-serverJvm.jpg VM_Summary-serverJvm.txt Attached JConsole output for a run where serverJvm settings are chosen. This shows that mem usage is growing throughout the suites.All run. Observer that the VM_Summary. attachements use Jvm 1.6. 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg Attachments: VM_Summary-1.txt, VM_Summary-serverJvm-PS_Old_Gen.jpg, VM_Summary-serverJvm.jpg, VM_Summary-serverJvm.txt, VM_Summary.jpg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) . . -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (DERBY-2345) truncate on a Blob does not work when the Blob is in memory
truncate on a Blob does not work when the Blob is in memory
---
Key: DERBY-2345
URL: https://issues.apache.org/jira/browse/DERBY-2345
Project: Derby
Issue Type: Bug
Components: JDBC
Reporter: V.Narayanan
Priority: Minor
I tried the following repro. After calling the truncate the Blob object still
returns the length as 29 (its original length) .
import java.sql.*;
public class TruncateBugRepro {
Connection con = null;
public Connection getEmbeddedConnection() throws Exception {
if(con == null) {
Class.forName(org.apache.derby.jdbc.EmbeddedDriver);
con = DriverManager.getConnection
(jdbc:derby:DB1;create=true);
}
return con;
}
public void testTruncate() throws Exception {
//String used to getBytes from and insert into Blob.
String str = new String(I am a Blob!!! I am a Blob!!!);
Connection con = getEmbeddedConnection();
//create the blob
Blob blob = con.createBlob();
//insert bytes
blob.setBytes(1,str.getBytes());
//Retuns the Blob length as 29
System.out.println( + blob.length());
blob.truncate(14);
//returns the Blob length as 29
System.out.println( + blob.length());
}
public static void main(String[] args) throws Exception {
TruncateBugRepro t = new TruncateBugRepro();
t.testTruncate();
}
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-2315) compatibility/testScript.xml should include 10.2.2 in the server/client combinations
[ https://issues.apache.org/jira/browse/DERBY-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen resolved DERBY-2315. --- Resolution: Fixed Fix Version/s: 10.3.0.0 Derby Info: (was: [Patch Available]) Thank you Ole! Committed v2 patch to trunk with revision 508383. compatibility/testScript.xml should include 10.2.2 in the server/client combinations Key: DERBY-2315 URL: https://issues.apache.org/jira/browse/DERBY-2315 Project: Derby Issue Type: Bug Components: Test Affects Versions: 10.3.0.0 Environment: All Reporter: Ole Solberg Assigned To: Ole Solberg Priority: Minor Fix For: 10.3.0.0 Attachments: derby-2315.v1.diff, derby-2315.v1.stat, derby-2315_v2_diff.txt, derby-2315_v2_status.txt Add SERVER_10.2.2.0 and CLIENT_10.2.2.0 in the combinations to be tested. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (DERBY-2329) Compatibility test fails in all combinations with 'serverTrunk', i.e. current trunk revision. (since tinderbox on svn 506771)
[ https://issues.apache.org/jira/browse/DERBY-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen resolved DERBY-2329. --- Resolution: Fixed Fix Version/s: 10.3.0.0 Derby Info: (was: [Patch Available]) Fixed in revision 508383. Compatibility test fails in all combinations with 'serverTrunk', i.e. current trunk revision. (since tinderbox on svn 506771) - Key: DERBY-2329 URL: https://issues.apache.org/jira/browse/DERBY-2329 Project: Derby Issue Type: Bug Components: Test Affects Versions: 10.3.0.0 Environment: OS: Solaris 10 6/06 s10x_u2wos_09a X86 64bits - SunOS 5.10 Generic_118855-14 Reporter: Ole Solberg Assigned To: Ole Solberg Priority: Minor Fix For: 10.3.0.0 Attachments: derby-2329_v1_diff.txt, derby-2329_v1_status.txt Since tinderbox test on svn 506771 the compativility test has failed in all combinations of serverVM-X_serverTrunk_clientVM-Y_clientZ See e.g. http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-506771.html http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-506972.html Some cases VM Version = 1.6 Could not find org.apache.derby.jdbc.ClientDriver Could not find org.apache.derby.jdbc.EmbeddedDriver Driver com.ibm.db2.jcc.DB2Driver Version = 2.4 com.ibm.db2.jcc.c.SqlException: java.net.ConnectException : Error opening socket to server localhost on port 1527 with message : Connection refused at com.ibm.db2.jcc.a.a.init(a.java:135) at com.ibm.db2.jcc.a.b.a(b.java:1542) at com.ibm.db2.jcc.c.o.init(o.java:795) at com.ibm.db2.jcc.a.b.init(b.java:298) at com.ibm.db2.jcc.DB2Driver.connect(DB2Driver.java:162) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:154) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:378) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:357) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:283) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) Exception in thread main java.lang.Exception: Error lookup up server info: java.net.ConnectException : Error opening socket to server localhost on port 1527 with message : Connection refused at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:295) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) VM Version = 1.6 Could not find com.ibm.db2.jcc.DB2Driver Could not find org.apache.derby.jdbc.EmbeddedDriver Driver org.apache.derby.jdbc.ClientDriver Version = 10.3 java.sql.SQLTransientConnectionException: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused. at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:154) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:378) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:357) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:283) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) Caused by: org.apache.derby.client.am.DisconnectException: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused. at org.apache.derby.client.net.NetAgent.init(Unknown Source) at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) at org.apache.derby.client.am.Connection.init(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.NetConnection40.init(Unknown Source) at
[jira] Assigned: (DERBY-2345) truncate on a Blob does not work when the Blob is in memory
[
https://issues.apache.org/jira/browse/DERBY-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anurag Shekhar reassigned DERBY-2345:
-
Assignee: Anurag Shekhar
truncate on a Blob does not work when the Blob is in memory
---
Key: DERBY-2345
URL: https://issues.apache.org/jira/browse/DERBY-2345
Project: Derby
Issue Type: Bug
Components: JDBC
Reporter: V.Narayanan
Assigned To: Anurag Shekhar
Priority: Minor
I tried the following repro. After calling the truncate the Blob object still
returns the length as 29 (its original length) .
import java.sql.*;
public class TruncateBugRepro {
Connection con = null;
public Connection getEmbeddedConnection() throws Exception {
if(con == null) {
Class.forName(org.apache.derby.jdbc.EmbeddedDriver);
con = DriverManager.getConnection
(jdbc:derby:DB1;create=true);
}
return con;
}
public void testTruncate() throws Exception {
//String used to getBytes from and insert into Blob.
String str = new String(I am a Blob!!! I am a Blob!!!);
Connection con = getEmbeddedConnection();
//create the blob
Blob blob = con.createBlob();
//insert bytes
blob.setBytes(1,str.getBytes());
//Retuns the Blob length as 29
System.out.println( + blob.length());
blob.truncate(14);
//returns the Blob length as 29
System.out.println( + blob.length());
}
public static void main(String[] args) throws Exception {
TruncateBugRepro t = new TruncateBugRepro();
t.testTruncate();
}
}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (DERBY-2346) Provide set methods for clob for embedded driver
Provide set methods for clob for embedded driver Key: DERBY-2346 URL: https://issues.apache.org/jira/browse/DERBY-2346 Project: Derby Issue Type: Sub-task Components: JDBC Affects Versions: 10.3.0.0 Reporter: Anurag Shekhar -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (DERBY-2346) Provide set methods for clob for embedded driver
[ https://issues.apache.org/jira/browse/DERBY-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anurag Shekhar reassigned DERBY-2346: - Assignee: Anurag Shekhar Provide set methods for clob for embedded driver Key: DERBY-2346 URL: https://issues.apache.org/jira/browse/DERBY-2346 Project: Derby Issue Type: Sub-task Components: JDBC Affects Versions: 10.3.0.0 Reporter: Anurag Shekhar Assigned To: Anurag Shekhar -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2197) Remove unused code for locking rows while holding a latch
[ https://issues.apache.org/jira/browse/DERBY-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen closed DERBY-2197. - Remove unused code for locking rows while holding a latch - Key: DERBY-2197 URL: https://issues.apache.org/jira/browse/DERBY-2197 Project: Derby Issue Type: Improvement Components: Services, Store, Test Affects Versions: 10.3.0.0 Reporter: Knut Anders Hatlen Assigned To: Knut Anders Hatlen Priority: Minor Fix For: 10.3.0.0 Attachments: derby-2197-engine.diff, derby-2197-engine.stat, derby-2197-v1.diff, derby-2197-v1.stat Some of the code for locking a row while holding a latch doesn't seem to be used outside the unit tests for store. This code should be removed in order to reduce the overall complexity. See the discussion in DERBY-2107 for details. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2047) TestDataSourceFactory doesn't work correctly outside the old harness
[ https://issues.apache.org/jira/browse/DERBY-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen closed DERBY-2047. - TestDataSourceFactory doesn't work correctly outside the old harness Key: DERBY-2047 URL: https://issues.apache.org/jira/browse/DERBY-2047 Project: Derby Issue Type: Bug Components: Test Affects Versions: 10.3.0.0 Reporter: Knut Anders Hatlen Assigned To: Daniel John Debrunner Fix For: 10.3.0.0 Attachments: DataSource.java, derby-2047.diff, derby-2047.stat TestDataSourceFactory uses TestUtil to create DataSource, ConnectionPoolDataSource and XADataSource objects. TestUtil needs to run in the old harness in order to detect which framework it is running under, so it will create embedded data sources for all JUnit tests that are run outside the old harness. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-705) Re-enable test which was disabled in DERBY-506 commit
[ https://issues.apache.org/jira/browse/DERBY-705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen closed DERBY-705. Re-enable test which was disabled in DERBY-506 commit - Key: DERBY-705 URL: https://issues.apache.org/jira/browse/DERBY-705 Project: Derby Issue Type: Test Components: Test Reporter: Oyvind Bakksjo Assigned To: V.Narayanan Priority: Trivial Fix For: 10.2.1.6 Attachments: DERBY-705.diff, DERBY-705.stat Testing of concurrent executions on same connection was disabled in DERBY-506 commit because of DERBY-694 bug. Re-enable when DERBY-694 is fixed. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-694) Statement exceptions cause all the connection's result sets to be closed with the client driver
[ https://issues.apache.org/jira/browse/DERBY-694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen closed DERBY-694. Statement exceptions cause all the connection's result sets to be closed with the client driver --- Key: DERBY-694 URL: https://issues.apache.org/jira/browse/DERBY-694 Project: Derby Issue Type: Bug Components: Network Client Affects Versions: 10.1.2.1 Reporter: Oyvind Bakksjo Assigned To: V.Narayanan Priority: Minor Fix For: 10.2.1.6 Attachments: DERBY-694.html, DERBY-694_upload_v1.diff, DERBY-694_upload_v1.stat, DERBY-694_v2.diff, DERBY-694_v2.stat, DERBY-694_v3.diff, DERBY-694_v3.stat, DERBY-694_v4.diff, DERBY-694_v4.stat, DERBY-694_v5.diff, DERBY-694_v5.stat, DERBY-694_v6.diff, DERBY-694_v6.stat, StatementRollbackTest.java Scenario: Autocommit off. Have two prepared statements, calling executeQuery() on both, giving me two result sets. Can fetch data from both with next(). If one statement gets an exception (say, caused by a division by zero), not only this statement's result set is closed, but also the other open resultset. This happens with the client driver, whereas in embedded mode, the other result set is unaffected by the exception in the first result set (as it should be). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-579) Query timeout set for one statement may affect other statements with the same SQL string
[ https://issues.apache.org/jira/browse/DERBY-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen closed DERBY-579. Query timeout set for one statement may affect other statements with the same SQL string Key: DERBY-579 URL: https://issues.apache.org/jira/browse/DERBY-579 Project: Derby Issue Type: Bug Components: SQL Reporter: Oyvind Bakksjo Assigned To: Oyvind Bakksjo Priority: Minor Fix For: 10.2.1.6 Attachments: DERBY-579.svn.diff, DERBY-579.svn.status The timeout is being set on the class GenericPreparedStatement, but this represents a statement plan and can be shared across multiple connections (through the statement cache). Thus if two connections execute the same statement with different timeouts, they will interfere with each other with the timeout values. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2347) Add code to support request and return of locators over DRDA
[ https://issues.apache.org/jira/browse/DERBY-2347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Øystein Grøvlen updated DERBY-2347: --- Component/s: Network Server Network Client Description: - Make the client able to send DRDA OUTOVR commands to request locators from Network Server. - Make the network server send locators instead of LOB values if locators has been requested. This JIRA will not activate the code that request locators since the client is not yet able to handle locators. Add code to support request and return of locators over DRDA Key: DERBY-2347 URL: https://issues.apache.org/jira/browse/DERBY-2347 Project: Derby Issue Type: Sub-task Components: Network Client, Network Server Reporter: Øystein Grøvlen Assigned To: Øystein Grøvlen - Make the client able to send DRDA OUTOVR commands to request locators from Network Server. - Make the network server send locators instead of LOB values if locators has been requested. This JIRA will not activate the code that request locators since the client is not yet able to handle locators. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-827) Performance can be improved by re-using language ResultSets across Activation executions.
[ https://issues.apache.org/jira/browse/DERBY-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473703 ] Dyre Tjeldvoll commented on DERBY-827: -- I tried to run the new test with latest version of Dan's patch, but only one test case, testLastIndexKeyResultSet fails. When I think about it I think it could be because this test case is one of few where the underlying table/data is changed between each execution of the PreparedStatement. Performance can be improved by re-using language ResultSets across Activation executions. - Key: DERBY-827 URL: https://issues.apache.org/jira/browse/DERBY-827 Project: Derby Issue Type: Improvement Components: Performance Reporter: Daniel John Debrunner Attachments: derby827_draft_reuse_result_sets.txt, derby827_update920.txt, rsfromps_prelim.diff Shouldn't DistinctScalarAggregateRS implement a close or a finish method (not sure what the difference is) and close the scan controller there. The close() and finish() methods are actually explained in their javadoc in the language org.apache.derby.iapi.sql.ResultSet class. [note this is not a JDBC java.sql.ResultSet object] close() - Tells the system that there will be no more calls to getNextRow() (until the next open() call) finish() - Tells the system that there will be no more access to any database information via this result set So close means the ResultSet may be opened again for more access, while finish means it will not be used again. However, their use in the code always doesn't match that, and that does cause confusion, at least to me. Language ResultSets (not JDBC ones) can be and are opened multiple times, for example when scanning a table multiple times within a join. An Activation, which represents the internal state of java.sql.PreparedStatement object has the lifetime of the java.sql.PreparedStatement, contains a top-level language ResultSet. This top-level language ResultSet provides the execution of the SQL statement, DML, DDL or a query. The top-level ResultSet may contain other ResultSets and could be seen as a tree structure. For the simple case of a primary key lookup query like: select name from customer where id = ? The activation would contain this: top result set ProjectRestrictRS IndexRowToBaseRowRS TableScanRS Now for some reason, even though the api of ResultSet say they can be re-used, and in some cases they are, this result set tree is thrown away after each execution. That is, the top result set has its finish() method called and then the activation removes its reference to it. Then on the next execution a new (identical) tree is set up. There is potential for a huge performance gain if this top level result set and its tree are re-used and have the same lifetime as the Activation. The saving comes in two forms, not having to create many objects on each execution, and not creating short-lived objects for the garbage collector to handle. I made a simple fix, it's a couple of lines of code, just calling close finish at the correct times, and for the above simple primary key lookup query, the performance went from 17,300 to 24,000 selects per second (cached data, single user). I'll post a patch shortly as an indication of the direction, once I can separate it from other changes in my client. However, I'm running the Derby tests and there are some (maybe 25-30) failures, I think because not all the language ResultSet implementations are correctly written to be re-opened. Interestingly, the first failure I saw was in an aggregrate test, which goes back to the issue Manish saw. Even if derbyall passed I would be nervous about submitting this patch for real, because I don't think there's a lot of testing using repeat executions of PreparedStatements in the tests. The ij tests mainly use Statement, this is a single use of an activation so this change would not affect them. Thus such a patch could regress Derby by making it more likely existing bugs would be exposed. Given the performance gains, I think we need to start re-using ResultSets from Activation, and devise a way to ensure the testing covers the re-use. The main issue is there is a large number of ResultSet implementations to cover. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-827) Performance can be improved by re-using language ResultSets across Activation executions.
[ https://issues.apache.org/jira/browse/DERBY-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473707 ] Daniel John Debrunner commented on DERBY-827: - Thanks for working on this. I would recommend not using the ASF licence flag as an indicator for if the patch is ready to be used or not. I think a simple comment with the attachment is fine. By not clicking the ASF licence flag you stop others from working on the patch, thus removing the chance for community development. Performance can be improved by re-using language ResultSets across Activation executions. - Key: DERBY-827 URL: https://issues.apache.org/jira/browse/DERBY-827 Project: Derby Issue Type: Improvement Components: Performance Reporter: Daniel John Debrunner Attachments: derby827_draft_reuse_result_sets.txt, derby827_update920.txt, rsfromps_prelim.diff Shouldn't DistinctScalarAggregateRS implement a close or a finish method (not sure what the difference is) and close the scan controller there. The close() and finish() methods are actually explained in their javadoc in the language org.apache.derby.iapi.sql.ResultSet class. [note this is not a JDBC java.sql.ResultSet object] close() - Tells the system that there will be no more calls to getNextRow() (until the next open() call) finish() - Tells the system that there will be no more access to any database information via this result set So close means the ResultSet may be opened again for more access, while finish means it will not be used again. However, their use in the code always doesn't match that, and that does cause confusion, at least to me. Language ResultSets (not JDBC ones) can be and are opened multiple times, for example when scanning a table multiple times within a join. An Activation, which represents the internal state of java.sql.PreparedStatement object has the lifetime of the java.sql.PreparedStatement, contains a top-level language ResultSet. This top-level language ResultSet provides the execution of the SQL statement, DML, DDL or a query. The top-level ResultSet may contain other ResultSets and could be seen as a tree structure. For the simple case of a primary key lookup query like: select name from customer where id = ? The activation would contain this: top result set ProjectRestrictRS IndexRowToBaseRowRS TableScanRS Now for some reason, even though the api of ResultSet say they can be re-used, and in some cases they are, this result set tree is thrown away after each execution. That is, the top result set has its finish() method called and then the activation removes its reference to it. Then on the next execution a new (identical) tree is set up. There is potential for a huge performance gain if this top level result set and its tree are re-used and have the same lifetime as the Activation. The saving comes in two forms, not having to create many objects on each execution, and not creating short-lived objects for the garbage collector to handle. I made a simple fix, it's a couple of lines of code, just calling close finish at the correct times, and for the above simple primary key lookup query, the performance went from 17,300 to 24,000 selects per second (cached data, single user). I'll post a patch shortly as an indication of the direction, once I can separate it from other changes in my client. However, I'm running the Derby tests and there are some (maybe 25-30) failures, I think because not all the language ResultSet implementations are correctly written to be re-opened. Interestingly, the first failure I saw was in an aggregrate test, which goes back to the issue Manish saw. Even if derbyall passed I would be nervous about submitting this patch for real, because I don't think there's a lot of testing using repeat executions of PreparedStatements in the tests. The ij tests mainly use Statement, this is a single use of an activation so this change would not affect them. Thus such a patch could regress Derby by making it more likely existing bugs would be exposed. Given the performance gains, I think we need to start re-using ResultSets from Activation, and devise a way to ensure the testing covers the re-use. The main issue is there is a large number of ResultSet implementations to cover. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Subscription: Derby: JIRA issues with patch available
Issue Subscription Filter: Derby: JIRA issues with patch available (12 issues) Subscriber: derby-dev Key Summary DERBY-2342 convert importExport.java to junit https://issues.apache.org/jira/browse/DERBY-2342 DERBY-2249 Place holder to attach the long running test which will test Derby's improved optimizer https://issues.apache.org/jira/browse/DERBY-2249 DERBY-681 Eliminate the parser's rewriting of the abstract syntax tree for queries with GROUP BY and/or HAVING clauses https://issues.apache.org/jira/browse/DERBY-681 DERBY-2334 Accept the new collation attribute and store it in services.properties file. Will be used for language based collation ordering. https://issues.apache.org/jira/browse/DERBY-2334 DERBY-2087 Clean up of a database server directory after running junit tests. https://issues.apache.org/jira/browse/DERBY-2087 DERBY-2161 MessageBuilder can write the properties file using the wrong encoding. https://issues.apache.org/jira/browse/DERBY-2161 DERBY-2248 Place holder for the NetworkServer system test https://issues.apache.org/jira/browse/DERBY-2248 DERBY-2262 DatabaseMetaData.getTypeInfo returns incorrect MAXIMUM_SCALE value for DECIMAL and NUMERIC types https://issues.apache.org/jira/browse/DERBY-2262 DERBY-2280 DatabaseMetaData.getTypeInfo() UNSIGNED_ATTRIBUTE and AUTO_INCREMENT column returns incorrect information for BLOB CLOB data type https://issues.apache.org/jira/browse/DERBY-2280 DERBY-2031 Convert derbynet/testProtocol.java to JUnit https://issues.apache.org/jira/browse/DERBY-2031 DERBY-2255 ij should indicate that it is waiting for more input in a multi-line interactive statement. https://issues.apache.org/jira/browse/DERBY-2255 DERBY-1620 SQL CASE statement returns ERROR 42X89 when including NULL as a return value https://issues.apache.org/jira/browse/DERBY-1620
[jira] Updated: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
[ https://issues.apache.org/jira/browse/DERBY-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg updated DERBY-2344: --- Attachment: VM_Summary-J507152DK14Jvm15-clientjvm.jpg VM_Summary-507428JDK14Jvm15-clientjvm.jpg VM_Summary-507428JDK6Jvm6-clientjvm.jpg Some observations: VM_Summary-J507152DK14Jvm15-clientjvm.jpg, where no errors problems were reported shows a significant drop in mem usage after ~25 minutes. VM_Summary-507428JDK14Jvm15-clientjvm.jpg, which reported one failure: 1) testPositionAgressive(org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest)java.lang.OutOfMemoryError: Java heap space shows steadily increasing mem usage where 507152 has a drop. VM_Summary-507428JDK6Jvm6-clientjvm.jpg, includes JDBC4 tests and fills memory even faster. It seems like - either the setUp/tearDown for the JUnit tests do not release memory taken per test run, - or may be test cases themselves do not release/close resources? 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg Attachments: VM_Summary-1.txt, VM_Summary-507428JDK14Jvm15-clientjvm.jpg, VM_Summary-507428JDK6Jvm6-clientjvm.jpg, VM_Summary-J507152DK14Jvm15-clientjvm.jpg, VM_Summary-serverJvm-PS_Old_Gen.jpg, VM_Summary-serverJvm.jpg, VM_Summary-serverJvm.txt, VM_Summary.jpg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) .
Regression Test Report - Daily 508011 - Sun DBTG
[Auto-generated mail] *Daily* 508011/2007-02-15 18:00:11 CET Failed TestsOK Skip Duration Suite --- *Jvm: 1.6* lin 0456456 092.20% derbyall 067836783 0 314.41% suitesAll sles 1456455 097.65% derbyall 067836783 0 180.07% suitesAll sol 0456456 097.65% derbyall 067836783 0 298.29% suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc 0456456 0 101.28% derbyall 067836783 0 183.32% suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/FailReports/508011.html *Jvm: 1.5* lin 1450449 292.16% derbyall 025232523 0 252.47% suitesAll sles 0450450 297.09% derbyall 025232523 0 157.81% suitesAll sol 0450450 2 100.39% derbyall 025232523 0 201.60% suitesAll solN+1 1450449 2 121.51% derbyall 025232523 0 221.61% suitesAll sparc 0450450 2 101.29% derbyall 025232523 0 204.18% suitesAll vista 0450450 296.60% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll win 0450450 2 112.37% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/FailReports/508011.html *Jvm: 1.4* lin 0444444 492.84% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sles 0444444 495.13% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sol 0444444 498.69% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc NA NA NANA derbyall NA NA NANA suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/FailReports/508011.html --- Changes in http://dbtg.thresher.com/derby/test/Daily/UpdateInfo/508011.txt ( All results in http://dbtg.thresher.com/derby/test/ )
junit test fails - org.apache.derbyTesting.functionTests.tests.jdbcapi.JDBCHarnessJavaTest
Hi All, The junit test org.apache.derbyTesting.functionTests.tests.jdbcapi.JDBCHarnessJavaTest fails with the following message. Revision #508500. Anybody else seeing this? Julo 1) statementJdbc20(org.apache.derbyTesting.functionTests.tests.jdbcapi.JDBCHarnessJavaTest)junit.framework.ComparisonFailure: Output at line 3 expected:... parameter value '-1[,]000' for Statement.s... but was:... parameter value '-1[??]000' for Statement.s... at org.apache.derbyTesting.functionTests.util.CanonTestCase.compareCanon(CanonTestCase.java:100) at org.apache.derbyTesting.functionTests.util.HarnessJavaTest.runTest(HarnessJavaTest.java:91) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25)
[jira] Commented: (DERBY-2249) Place holder to attach the long running test which will test Derby's improved optimizer
[ https://issues.apache.org/jira/browse/DERBY-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473760 ] A B commented on DERBY-2249: I don't how (or if?) the discussion of where to put the new test and of what to call it ended, but in any event I was able to build it with the build.xml file (after removing the references to mailjdbc and sttest from DERBY-2249_diff.txt) so thank you for providing it. I then started playing around with the test and my initial comments are below. ** 1) In an attempt to get the test to run quickly just so I could see what it does, I changed the values in StaticValues.java to the following: public static int NUM_OF_ROWS=100; //Total number of rows expected in each table public static int NUM_OF_TABLES=8; //Total number of tables to be created public static int ITER=3; //Number of iterations of each query When I then tried to run the test, I saw several failures (see below). Are there any restrictions on what these values can or should be? When I specified them as shown above I saw the following errors: 1) While creating views: SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE16' does not exist. (x2) SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE32' does not exist. (x4) SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE9' does not exist. SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE11' does not exist. SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE13' does not exist. SQLState = 42X05, java.sql.SQLException: Table/View 'MYTABLE15' does not exist. 2) While running query Select from single view: SQLState= 42X05 ErrorCode= 3 Message= Table/View 'V16' does not exist. 3) While printing results: QueryName Using PreparedStatment Using Statement Rows Expected -- Exception in thread main java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.RangeCheck(ArrayList.java(Inlined Compiled Code)) at java.util.ArrayList.get(ArrayList.java(Compiled Code)) at org.apache.derbyTesting.system.langtest.RunLangTest.printResults(RunLangTest.java:141) at org.apache.derbyTesting.system.langtest.RunLangTest.main(RunLangTest.java:121) I tried increasing the number of tables to 32 but I still saw the same errors (just with different table/view names). I then put it back to 64 (the default) and the test ran to completion. ** 2) The R E S U L T S output at the end of the test is very nice. While the test is running, though, a lot of info is printed to the screen, including the queries themselves. All of that info (esp. the queries themselves) is a bit overwhelming (to me, at least ;) I wonder if it might be better to just use query numbers like we do in the results? Or else completely disable the in-progress output and just print the results at the end, since that has all of the relevant information? Perhaps we could add a test parameter that allows for echoing of the query text if the user chooses (verbose)? Certainly not a requirement, though; just a wish item. ** 3) It was not clear to me from the README that specification of -qlist means that the test will search for a file named 'query.list' in the *current* directory (I thought I was supposed to pass the path to query.list in as an argument). It also was not clear to me that if I did *not* specify -qlist the test would _still_ search for a file named 'query.list' in the current directory, with the apparent intent of running the default queries AND the queries in 'query.list'. Is that correct? ** 4) Is there more info available about what is expected in the query.list file if one is specified? I tried creating a query.list file that had a couple of queries pulled from Query2.java but the result was a failure with a syntax error (it looks like maybe the select keyword is being stripped?). I tried putting quotes around the query but that didn't help. Also, if we want to specify more than one query in 'query.list', what's the delimiter? Is that even possible? I think it would be good if all of the relevant information about 'query.list' could be included in the README file. ** 5) I think it would be great if more comments could be added to the various queries in the QueryX.java files. For example, I opened up Query2.java and thought all of the queries were the same. Only after staring at them for a while did I realize that there were identical *except* for the views that they use (v8 vs v16 vs v32, etc). A comment to that effect might save people who look at the queries some time. Similarly for utils/TestViews.java (ex. what's the difference between v8 and v8_2?) Note that since this new test is not going to run as part of derbyall nor as part
[jira] Updated: (DERBY-2196) Run standalone network server with security manager by default
[ https://issues.apache.org/jira/browse/DERBY-2196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rick Hillegas updated DERBY-2196: - Attachment: derby-2196-03-tests-01.diff Attaching derby-2196-03-tests-01.diff. This is a unit test to verify that the network server installs a security manager with the Basic policy as expected. Makes the following changes: M java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/_Suite.java Wires the test into the derbynet suite. A java/testing/org/apache/derbyTesting/functionTests/tests/derbynet/SecureServerTest.java The test itself. Tests that the server, when booted, installs a security manager or not, depending on other command-line settings. M java/testing/org/apache/derbyTesting/junit/NetworkServerTestSetup.java Added some machinery for bringing the server up in a separate process. This test needs a separate process to avoid having to poke special test instrumentation into the network server itself. M java/testing/org/apache/derbyTesting/junit/SecurityManagerSetup.java M java/testing/org/apache/derbyTesting/junit/ServerSetup.java M java/testing/org/apache/derbyTesting/junit/TestConfiguration.java Made some machinery public. Run standalone network server with security manager by default -- Key: DERBY-2196 URL: https://issues.apache.org/jira/browse/DERBY-2196 Project: Derby Issue Type: Improvement Components: Network Server, Security Reporter: Daniel John Debrunner Assigned To: Rick Hillegas Attachments: derby-2196-01-print-01.diff, derby-2196-01-print-02.diff, derby-2196-01-print-03.diff, derby-2196-02-install-01.diff, derby-2196-03-tests-01.diff, secureServer.html, secureServer.html, secureServer.html, secureServer.html, secureServer.html, secureServer.html From an e-mail discussion: ... Derby should match the security provided by typical client server systems such as DB2, Oracle, etc. I think in this case system/database owners are trusting the database system to ensure that their system cannot be attacked. So maybe if Derby is booted as a standalone server with no security manager involved, it should install one with a default security policy. Thus allowing Derby to use Java security manager to manage system privileges but not requiring everyone to become familiar with them. http://mail-archives.apache.org/mod_mbox/db-derby-dev/200612.mbox/[EMAIL PROTECTED] I imagine such a policy would allow any access to databases under derby.system.home and/or user.home. By standalone I mean the network server was started though the main() method (command line). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2249) Place holder to attach the long running test which will test Derby's improved optimizer
[ https://issues.apache.org/jira/browse/DERBY-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473772 ] Sunitha Kambhampati commented on DERBY-2249: I dont think this test falls under the basic directory ( derbyTesting/perf/basic) as it is for simple tests that test a particular issue. As mentioned in an earlier comment by Dan, maybe a better place is to be at the same level as the OE test.. derbyTesting/system/optimizer ? Place holder to attach the long running test which will test Derby's improved optimizer --- Key: DERBY-2249 URL: https://issues.apache.org/jira/browse/DERBY-2249 Project: Derby Issue Type: Test Components: Test Affects Versions: 10.3.0.0 Reporter: Manjula Kutty Assigned To: Manjula Kutty Priority: Trivial Attachments: build.xml, DERBY-2249_diff.txt, LangTest.zip Will be adding a test for the improved optimizer particularly to test the fix for DERBY-805 and DERBY-1205. Once the files are committed will provide a link to both of those issues. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-827) Performance can be improved by re-using language ResultSets across Activation executions.
[ https://issues.apache.org/jira/browse/DERBY-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyre Tjeldvoll updated DERBY-827: - Attachment: rsfromps_prelim2.diff Attaching a modified test with better result checking, and that deletes rows from the underlying tables while executing PreparedStatements. (with ASF license this time, but it still isn't ready for inclusion). Performance can be improved by re-using language ResultSets across Activation executions. - Key: DERBY-827 URL: https://issues.apache.org/jira/browse/DERBY-827 Project: Derby Issue Type: Improvement Components: Performance Reporter: Daniel John Debrunner Attachments: derby827_draft_reuse_result_sets.txt, derby827_update920.txt, rsfromps_prelim.diff, rsfromps_prelim2.diff Shouldn't DistinctScalarAggregateRS implement a close or a finish method (not sure what the difference is) and close the scan controller there. The close() and finish() methods are actually explained in their javadoc in the language org.apache.derby.iapi.sql.ResultSet class. [note this is not a JDBC java.sql.ResultSet object] close() - Tells the system that there will be no more calls to getNextRow() (until the next open() call) finish() - Tells the system that there will be no more access to any database information via this result set So close means the ResultSet may be opened again for more access, while finish means it will not be used again. However, their use in the code always doesn't match that, and that does cause confusion, at least to me. Language ResultSets (not JDBC ones) can be and are opened multiple times, for example when scanning a table multiple times within a join. An Activation, which represents the internal state of java.sql.PreparedStatement object has the lifetime of the java.sql.PreparedStatement, contains a top-level language ResultSet. This top-level language ResultSet provides the execution of the SQL statement, DML, DDL or a query. The top-level ResultSet may contain other ResultSets and could be seen as a tree structure. For the simple case of a primary key lookup query like: select name from customer where id = ? The activation would contain this: top result set ProjectRestrictRS IndexRowToBaseRowRS TableScanRS Now for some reason, even though the api of ResultSet say they can be re-used, and in some cases they are, this result set tree is thrown away after each execution. That is, the top result set has its finish() method called and then the activation removes its reference to it. Then on the next execution a new (identical) tree is set up. There is potential for a huge performance gain if this top level result set and its tree are re-used and have the same lifetime as the Activation. The saving comes in two forms, not having to create many objects on each execution, and not creating short-lived objects for the garbage collector to handle. I made a simple fix, it's a couple of lines of code, just calling close finish at the correct times, and for the above simple primary key lookup query, the performance went from 17,300 to 24,000 selects per second (cached data, single user). I'll post a patch shortly as an indication of the direction, once I can separate it from other changes in my client. However, I'm running the Derby tests and there are some (maybe 25-30) failures, I think because not all the language ResultSet implementations are correctly written to be re-opened. Interestingly, the first failure I saw was in an aggregrate test, which goes back to the issue Manish saw. Even if derbyall passed I would be nervous about submitting this patch for real, because I don't think there's a lot of testing using repeat executions of PreparedStatements in the tests. The ij tests mainly use Statement, this is a single use of an activation so this change would not affect them. Thus such a patch could regress Derby by making it more likely existing bugs would be exposed. Given the performance gains, I think we need to start re-using ResultSets from Activation, and devise a way to ensure the testing covers the re-use. The main issue is there is a large number of ResultSet implementations to cover. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2313) create backup directory relative to the current working dir while running the system test
[ https://issues.apache.org/jira/browse/DERBY-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2313: - Attachment: DERBY-2313_patch2_diff.txt Thanks Mamata for looking in to the patch and the suggestion. I have changed the use of . for the current directory. Please find the attached patch which will fix this issue create backup directory relative to the current working dir while running the system test - Key: DERBY-2313 URL: https://issues.apache.org/jira/browse/DERBY-2313 Project: Derby Issue Type: Sub-task Components: Test Affects Versions: 10.3.0.0 Reporter: Manjula Kutty Assigned To: Manjula Kutty Attachments: DERBY-2313_diff.txt, DERBY-2313_patch2_diff.txt create backup directory relative to the current working dir -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-827) Performance can be improved by re-using language ResultSets across Activation executions.
[ https://issues.apache.org/jira/browse/DERBY-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473801 ] Daniel John Debrunner commented on DERBY-827: - Thanks for marking it asf, makes it easier for others to look at it :-) Just looking at the test I couldn't see any described reason as to why the test breaks the model of a single connection per fixture and instead uses a shared connection and has code to manage a prepared statement cache. Is this addiitonal complexity (and more chance for bugs confusion) needed for the test, or is it an attempt to make the test go faster? I think a cleaner approach to sharing connections across fixtures would be to have a connection pooling decorator, so that the test continues to look feel like other tests. Performance can be improved by re-using language ResultSets across Activation executions. - Key: DERBY-827 URL: https://issues.apache.org/jira/browse/DERBY-827 Project: Derby Issue Type: Improvement Components: Performance Reporter: Daniel John Debrunner Attachments: derby827_draft_reuse_result_sets.txt, derby827_update920.txt, rsfromps_prelim.diff, rsfromps_prelim2.diff Shouldn't DistinctScalarAggregateRS implement a close or a finish method (not sure what the difference is) and close the scan controller there. The close() and finish() methods are actually explained in their javadoc in the language org.apache.derby.iapi.sql.ResultSet class. [note this is not a JDBC java.sql.ResultSet object] close() - Tells the system that there will be no more calls to getNextRow() (until the next open() call) finish() - Tells the system that there will be no more access to any database information via this result set So close means the ResultSet may be opened again for more access, while finish means it will not be used again. However, their use in the code always doesn't match that, and that does cause confusion, at least to me. Language ResultSets (not JDBC ones) can be and are opened multiple times, for example when scanning a table multiple times within a join. An Activation, which represents the internal state of java.sql.PreparedStatement object has the lifetime of the java.sql.PreparedStatement, contains a top-level language ResultSet. This top-level language ResultSet provides the execution of the SQL statement, DML, DDL or a query. The top-level ResultSet may contain other ResultSets and could be seen as a tree structure. For the simple case of a primary key lookup query like: select name from customer where id = ? The activation would contain this: top result set ProjectRestrictRS IndexRowToBaseRowRS TableScanRS Now for some reason, even though the api of ResultSet say they can be re-used, and in some cases they are, this result set tree is thrown away after each execution. That is, the top result set has its finish() method called and then the activation removes its reference to it. Then on the next execution a new (identical) tree is set up. There is potential for a huge performance gain if this top level result set and its tree are re-used and have the same lifetime as the Activation. The saving comes in two forms, not having to create many objects on each execution, and not creating short-lived objects for the garbage collector to handle. I made a simple fix, it's a couple of lines of code, just calling close finish at the correct times, and for the above simple primary key lookup query, the performance went from 17,300 to 24,000 selects per second (cached data, single user). I'll post a patch shortly as an indication of the direction, once I can separate it from other changes in my client. However, I'm running the Derby tests and there are some (maybe 25-30) failures, I think because not all the language ResultSet implementations are correctly written to be re-opened. Interestingly, the first failure I saw was in an aggregrate test, which goes back to the issue Manish saw. Even if derbyall passed I would be nervous about submitting this patch for real, because I don't think there's a lot of testing using repeat executions of PreparedStatements in the tests. The ij tests mainly use Statement, this is a single use of an activation so this change would not affect them. Thus such a patch could regress Derby by making it more likely existing bugs would be exposed. Given the performance gains, I think we need to start re-using ResultSets from Activation, and devise a way to ensure the testing covers the re-use. The main issue is there is a large number of ResultSet implementations to cover. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (DERBY-2348) testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)j failed
testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)j failed -- Key: DERBY-2348 URL: https://issues.apache.org/jira/browse/DERBY-2348 Project: Derby Issue Type: Test Affects Versions: 10.3.0.0 Environment: This test failed on ibm142/ibm15. Reporter: Suresh Thalamati 3) testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)junit.framework.ComparisonFailure: Output at line 26 expected:.. but was:...9 SECMEC=... at org.apache.derbyTesting.functionTests.util.CanonTestCase.compareCanon(CanonTestCase.java:100) at org.apache.derbyTesting.functionTests.util.HarnessJavaTest.runTest(HarnessJavaTest.java:91) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-264) This enhancement to allow expressions in ORDER BY clause will require documentation changes.
[ https://issues.apache.org/jira/browse/DERBY-264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473806 ] Laura Stewart commented on DERBY-264: - Need more information to update the ORDER BY clause topic - http://db.apache.org/derby/docs/dev/ref/ I glanced through the thread that Oystein pointed me to, but did not find what I need. 1. The type of expressions that are allowed. My impression is that they are the Boolean expressions (as opposed to TableExpressions or SelectExpressions)? 2. I need to understand how the Syntax for ORDER BY should appear. Should it just be Expressions added immediately after ColumnPosition (in both places in the Syntax? Are there limitations as to the types of expressions allowed? Boolean, concatenation? 3. Description of the expressions allowed (will appear below the syntax) This will depend on what type of expressions are allowed, but one thing that the other parameters mention is if the column needss to be in the select list. 4. Whatever type of expressions are allowed, that topic in the Derby Ref Guide needs to be updated too. Are there any limitations for ORDER BY? For example, the Boolean expressions topic mentions that there are limitations with check constraints. Thanks! This enhancement to allow expressions in ORDER BY clause will require documentation changes. Key: DERBY-264 URL: https://issues.apache.org/jira/browse/DERBY-264 Project: Derby Issue Type: Sub-task Components: Documentation Affects Versions: 10.1.1.0, 10.2.1.6 Reporter: Satheesh Bandaram Assigned To: Laura Stewart Priority: Minor I can help if needed for documenting this enhancement. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2318) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest) failed on weme6.1 jvm/
[ https://issues.apache.org/jira/browse/DERBY-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473807 ] Suresh Thalamati commented on DERBY-2318: - This test also failed on one of the iibm15 jvm runs: 5) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest)junit.framework.AssertionFailedError: FAILED!! level difference not expected since streams are materialized. expected:50 but was:51 at org.apache.derbyTesting.functionTests.tests.lang.StreamsTest.testStreams(StreamsTest.java:125) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest) failed on weme6.1 jvm/ - Key: DERBY-2318 URL: https://issues.apache.org/jira/browse/DERBY-2318 Project: Derby Issue Type: Test Affects Versions: 10.3.0.0 Environment: Java Version:J2ME Foundation Specification v1.1 Java Vendor: IBM Corporation Java home: c:\jartest\weme6.1 Reporter: Suresh Thalamati Assigned To: Myrna van Lunteren Priority: Minor Fix For: 10.3.0.0 Time: 1,021.188 There was 1 failure: 1) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest)junit.framework.AssertionFailedError: FAILED!! level difference not expected since streams are materialized. expected:48 but was:49 at org.apache.derbyTesting.functionTests.tests.lang.StreamsTest.testStreams(StreamsTest.java:118) at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:205) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) FAILURES!!! Tests run: 1285, Failures: 1, Errors: 0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2318) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest) failed on weme6.1 jvm/
[ https://issues.apache.org/jira/browse/DERBY-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473819 ] Daniel John Debrunner commented on DERBY-2318: -- One guess is that the JIT is changing the stack depth by inlining methods. I think in some ways the test is invalid, I don't think there are no guarantees about the stack depth for subsequent similar operations. testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest) failed on weme6.1 jvm/ - Key: DERBY-2318 URL: https://issues.apache.org/jira/browse/DERBY-2318 Project: Derby Issue Type: Test Affects Versions: 10.3.0.0 Environment: Java Version:J2ME Foundation Specification v1.1 Java Vendor: IBM Corporation Java home: c:\jartest\weme6.1 Reporter: Suresh Thalamati Assigned To: Myrna van Lunteren Priority: Minor Fix For: 10.3.0.0 Time: 1,021.188 There was 1 failure: 1) testStreams(org.apache.derbyTesting.functionTests.tests.lang.StreamsTest)junit.framework.AssertionFailedError: FAILED!! level difference not expected since streams are materialized. expected:48 but was:49 at org.apache.derbyTesting.functionTests.tests.lang.StreamsTest.testStreams(StreamsTest.java:118) at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:205) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) FAILURES!!! Tests run: 1285, Failures: 1, Errors: 0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2315) compatibility/testScript.xml should include 10.2.2 in the server/client combinations
[ https://issues.apache.org/jira/browse/DERBY-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg closed DERBY-2315. -- Verified OK in http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-508484.html (Had not corrected my ~/ant.properties.compatibility file in the 508385 run ) compatibility/testScript.xml should include 10.2.2 in the server/client combinations Key: DERBY-2315 URL: https://issues.apache.org/jira/browse/DERBY-2315 Project: Derby Issue Type: Bug Components: Test Affects Versions: 10.3.0.0 Environment: All Reporter: Ole Solberg Assigned To: Ole Solberg Priority: Minor Fix For: 10.3.0.0 Attachments: derby-2315.v1.diff, derby-2315.v1.stat, derby-2315_v2_diff.txt, derby-2315_v2_status.txt Add SERVER_10.2.2.0 and CLIENT_10.2.2.0 in the combinations to be tested. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2329) Compatibility test fails in all combinations with 'serverTrunk', i.e. current trunk revision. (since tinderbox on svn 506771)
[ https://issues.apache.org/jira/browse/DERBY-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg closed DERBY-2329. -- See DERBY-2315 Compatibility test fails in all combinations with 'serverTrunk', i.e. current trunk revision. (since tinderbox on svn 506771) - Key: DERBY-2329 URL: https://issues.apache.org/jira/browse/DERBY-2329 Project: Derby Issue Type: Bug Components: Test Affects Versions: 10.3.0.0 Environment: OS: Solaris 10 6/06 s10x_u2wos_09a X86 64bits - SunOS 5.10 Generic_118855-14 Reporter: Ole Solberg Assigned To: Ole Solberg Priority: Minor Fix For: 10.3.0.0 Attachments: derby-2329_v1_diff.txt, derby-2329_v1_status.txt Since tinderbox test on svn 506771 the compativility test has failed in all combinations of serverVM-X_serverTrunk_clientVM-Y_clientZ See e.g. http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-506771.html http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-506972.html Some cases VM Version = 1.6 Could not find org.apache.derby.jdbc.ClientDriver Could not find org.apache.derby.jdbc.EmbeddedDriver Driver com.ibm.db2.jcc.DB2Driver Version = 2.4 com.ibm.db2.jcc.c.SqlException: java.net.ConnectException : Error opening socket to server localhost on port 1527 with message : Connection refused at com.ibm.db2.jcc.a.a.init(a.java:135) at com.ibm.db2.jcc.a.b.a(b.java:1542) at com.ibm.db2.jcc.c.o.init(o.java:795) at com.ibm.db2.jcc.a.b.init(b.java:298) at com.ibm.db2.jcc.DB2Driver.connect(DB2Driver.java:162) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:154) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:378) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:357) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:283) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) Exception in thread main java.lang.Exception: Error lookup up server info: java.net.ConnectException : Error opening socket to server localhost on port 1527 with message : Connection refused at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:295) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) VM Version = 1.6 Could not find com.ibm.db2.jcc.DB2Driver Could not find org.apache.derby.jdbc.EmbeddedDriver Driver org.apache.derby.jdbc.ClientDriver Version = 10.3 java.sql.SQLTransientConnectionException: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused. at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:154) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:378) at org.apache.derbyTesting.functionTests.util.DerbyJUnitTest.getConnection(DerbyJUnitTest.java:357) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.findServer(CompatibilitySuite.java:283) at org.apache.derbyTesting.functionTests.tests.junitTests.compatibility.CompatibilitySuite.main(CompatibilitySuite.java:130) Caused by: org.apache.derby.client.am.DisconnectException: java.net.ConnectException : Error connecting to server localhost on port 1527 with message Connection refused. at org.apache.derby.client.net.NetAgent.init(Unknown Source) at org.apache.derby.client.net.NetConnection.newAgent_(Unknown Source) at org.apache.derby.client.am.Connection.init(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.NetConnection40.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl40.newNetConnection(Unknown Source) ... 7 more Caused by: java.net.ConnectException: Connection refused at
Re: Regression Test Report - Daily 508011 - Sun DBTG
Quite some tests had to be stopped: these were 'hanging', maybe also due to what Ole has seen: Derby-2344. Henri [EMAIL PROTECTED] wrote: [Auto-generated mail] *Daily* 508011/2007-02-15 18:00:11 CET Failed TestsOK Skip Duration Suite --- *Jvm: 1.6* lin 0456456 092.20% derbyall 067836783 0 314.41% suitesAll sles 1456455 097.65% derbyall 067836783 0 180.07% suitesAll sol 0456456 097.65% derbyall 067836783 0 298.29% suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc 0456456 0 101.28% derbyall 067836783 0 183.32% suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.6/FailReports/508011.html *Jvm: 1.5* lin 1450449 292.16% derbyall 025232523 0 252.47% suitesAll sles 0450450 297.09% derbyall 025232523 0 157.81% suitesAll sol 0450450 2 100.39% derbyall 025232523 0 201.60% suitesAll solN+1 1450449 2 121.51% derbyall 025232523 0 221.61% suitesAll sparc 0450450 2 101.29% derbyall 025232523 0 204.18% suitesAll vista 0450450 296.60% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll win 0450450 2 112.37% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.5/FailReports/508011.html *Jvm: 1.4* lin 0444444 492.84% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sles 0444444 495.13% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll sol 0444444 498.69% derbyall UNKOWNUNKOWNUNKOWN UNKOWN 0.00% suitesAll solN+1 NA NA NANA derbyall NA NA NANA suitesAll sparc NA NA NANA derbyall NA NA NANA suitesAll vista NA NA NANA derbyall NA NA NANA suitesAll win NA NA NANA derbyall NA NA NANA suitesAll Details in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/testing/Limited/testSummary-508011.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/Daily/jvm1.4/FailReports/508011.html --- Changes in http://dbtg.thresher.com/derby/test/Daily/UpdateInfo/508011.txt ( All results in http://dbtg.thresher.com/derby/test/ )
[jira] Assigned: (DERBY-2348) testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)j failed
[ https://issues.apache.org/jira/browse/DERBY-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew McIntyre reassigned DERBY-2348: -- Assignee: Andrew McIntyre testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)j failed -- Key: DERBY-2348 URL: https://issues.apache.org/jira/browse/DERBY-2348 Project: Derby Issue Type: Test Affects Versions: 10.3.0.0 Environment: This test failed on ibm142/ibm15. Reporter: Suresh Thalamati Assigned To: Andrew McIntyre 3) testProtocol(org.apache.derbyTesting.functionTests.tests.derbynet.NetHarnessJavaTest)junit.framework.ComparisonFailure: Output at line 26 expected:.. but was:...9 SECMEC=... at org.apache.derbyTesting.functionTests.util.CanonTestCase.compareCanon(CanonTestCase.java:100) at org.apache.derbyTesting.functionTests.util.HarnessJavaTest.runTest(HarnessJavaTest.java:91) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:76) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Need help on ORDER BY with expressions
The JIRA issue DERBY-264 is for documenting the updates to the ORDER BY clause which allows expressions ( DERBY-134 ). The comments on DERBY-264 show some examples, but I need additional help understanding what needs to be documented: 1. The type of expressions that are allowed. My impression is that they are the Boolean expressions (as opposed to TableExpressions or SelectExpressions)? 2. I need to understand how the Syntax for ORDER BY should appear. Should it just be Expressions added immediately after ColumnPosition (in both places in the Syntax? Or should it be specific type of expression (TableExpression, SelectExpression, Boolean Expression). Are there limitations as to the types of expressions allowed? Boolean, concatenation? 3. Description of the expressions allowed (will appear below the syntax) This will depend on what type of expressions are allowed, but one thing that the other parameters mention is if the column needs to be in the select list. I need help describing the expressions parameter. 4. Whatever type of expressions are allowed, that topic in the Derby Ref Guide needs to be updated too. Are there any limitations for ORDER BY? For example, the Boolean expressions topic mentions that there are limitations with check constraints. Thanks! -- Laura Stewart
[jira] Created: (DERBY-2349) Accessing a BLOB column twice in an INSERT trigger leads to errors in the value on-disk
Accessing a BLOB column twice in an INSERT trigger leads to errors in the value on-disk --- Key: DERBY-2349 URL: https://issues.apache.org/jira/browse/DERBY-2349 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.3.0.0 Reporter: Daniel John Debrunner Either the BLOB is stored with the incorrect value or the value on disk does not match the stored length on disk and an exception is raised. The BLOB was supplied as a stream value. See this with the new TriggersTest. The text fixture will have a comment with this bug number showing how to reproduce the problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2349) Accessing a BLOB column twice in an INSERT trigger leads to errors in the value on-disk
[ https://issues.apache.org/jira/browse/DERBY-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473830 ] Daniel John Debrunner commented on DERBY-2349: -- Errors seen ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to read more data than was originally stored junit.framework.AssertionFailedError: Blobs have different lengths expected:42879 but was:14 Accessing a BLOB column twice in an INSERT trigger leads to errors in the value on-disk --- Key: DERBY-2349 URL: https://issues.apache.org/jira/browse/DERBY-2349 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.3.0.0 Reporter: Daniel John Debrunner Either the BLOB is stored with the incorrect value or the value on disk does not match the stored length on disk and an exception is raised. The BLOB was supplied as a stream value. See this with the new TriggersTest. The text fixture will have a comment with this bug number showing how to reproduce the problem. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Bay Area Derby Lunch Discussions
It was really great to meet some of the people at the Bay Area Lunch on Wednesday who are involved in Derby! One of the people that I met there was Kim Haase. Kim and I have been contributing documentation to Derby, and naturally we discussed the Derby Library during the lunch. An area that we think might be good to improve is the organization of information which discusses how to setup and start working with Derby. We discussed the possibility of moving the entire content of the Working With Derby manual into the Getting Started Guide. The Working With Derby content is really good, but might be better if it is placed in context with the installation and getting started information. Additionally, the information in the Getting Started Guide that describes SQL syntax might be better in the Reference Manual, where the SQL statements are actually documented. There currently is no information in the Reference Manual that explains the syntax diagrams. How does the rest of the Derby community feel about us analyzing the current organization of the Getting Started Guide and the Working With Derby information, with the intent to propose a reorganization of the information? Are there other areas in the Derby Library that seem to be organized poorly? -- Laura Stewart
[jira] Commented: (DERBY-2344) 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread
[ https://issues.apache.org/jira/browse/DERBY-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473837 ] Daniel John Debrunner commented on DERBY-2344: -- I see this out of heap space on linux running suites.All without derbynet.jar with today's trunk codeline. I tried these combinations: testing.jar from today, derby.jar/derbytools.jar from svn 494119 that fails with the out of memory error testing.jar from svn 494119, derby.jar/derbytools.jar from today that passes. So it would seem it something in the JUnit setup that has changed. I always seem to see the failure in the testPositionAgressive fixture of BlobClob4BlobTest 'java.lang.OutOfMemoryError: Java heap space' in NetworkServerThread Key: DERBY-2344 URL: https://issues.apache.org/jira/browse/DERBY-2344 Project: Derby Issue Type: Bug Affects Versions: 10.3.0.0 Environment: JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Fedora Core release 5 (Bordeaux) 32bits - Linux 2.6.17-1.2174_FC5 #1 Tue Aug 8 15:30:55 EDT 2006 HW: 1 X Intel(R) Pentium(R) 4 CPU 2.66GHz: 2656.261 MHz, 512 KB cache. 773284 kB Total Memory. JVM: Sun Microsystems Inc. 1.5.0_06-b04 OS: Solaris Nevada snv_27a X86 64 bits - SunOS 5.11 snv_27 HW: 1 X x86 AuthenticAMD family 15 model 36 step 2 (AMD Turion(tm) 64 Mobile Technology ML-37): 2000 MHz, unknown cache. 1023 Megabytes Total Memory JVM: Sun Microsystems Inc. - 1.5.0_05-b05 OS: Microsoft Windows XP Professional - 5.1.2600 Service Pack 2 Build 2600 - CYGWIN_NT-5.1 1.5.18(0.132/4/2) 2005-07-02 20:30 Cygwin HW: 1 X Ferrari 4000 x86 Family 15 Model 36 Stepping 2 AuthenticAMD: ~2000 MHz, 2,048 MB cache. 1,022 Total Memory. Reporter: Ole Solberg Attachments: VM_Summary-1.txt, VM_Summary-507428JDK14Jvm15-clientjvm.jpg, VM_Summary-507428JDK6Jvm6-clientjvm.jpg, VM_Summary-J507152DK14Jvm15-clientjvm.jpg, VM_Summary-serverJvm-PS_Old_Gen.jpg, VM_Summary-serverJvm.jpg, VM_Summary-serverJvm.txt, VM_Summary.jpg I see this only on test machines with 1Gb of memory or less. Our lab-machines with 2Gb mem show now problems. See http://dbtg.thresher.com/derby/test/trunk15/jvm1.5/testing/Limited/testSummary-508068.html Extract from a typical thread dump: main prio=1 tid=0x09e2a760 nid=0xef6 runnable [0xbfd89000..0xbfd8a418] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.derby.client.net.Reply.fill(Unknown Source) at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source) at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source) at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readExchangeServerAttributes(Unknown Source) at org.apache.derby.client.net.NetConnection.readServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowServerAttributesAndKeyExchange(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.init(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:525) - locked 0x8cda8a18 (a java.lang.Class) at java.sql.DriverManager.getConnection(DriverManager.java:171) - locked 0x8cda8a18 (a java.lang.Class) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:76) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:764) at org.apache.derbyTesting.junit.BaseJDBCTestCase.openDefaultConnection(BaseJDBCTestCase.java:205) at org.apache.derbyTesting.junit.BaseJDBCTestCase.getConnection(BaseJDBCTestCase.java:90) at org.apache.derbyTesting.functionTests.tests.jdbcapi.BlobClob4BlobTest.setUp(BlobClob4BlobTest.java:56) at junit.framework.TestCase.runBare(TestCase.java:128) . . -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (DERBY-2350) Use of XML values in the action statement of a trigger throw exceptions.
Use of XML values in the action statement of a trigger throw exceptions. Key: DERBY-2350 URL: https://issues.apache.org/jira/browse/DERBY-2350 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.3.0.0 Reporter: Daniel John Debrunner A trigger like, where V, V1 and V2 are columns of type XML will thrown an exception when fired. CREATE TRIGGER AIS AFTER INSERT ON T_MAIN REFERENCING NEW_TABLE AS N FOR EACH STATEMENT INSERT INTO T_ACTION_STATEMENT(A, V1, ID, V2) SELECT 'I', V, ID, V FROM N ERROR 38000: The exception 'java.sql.SQLException: An attempt was made to get a data value of type 'java.lang.Object' from a data value of type 'XML'.' was thrown while evaluating an expression. Most likely because triggers are implementing using VTIs and hence JDBC ResultSets and XML is not supported through JDBC yet. TriggerTest shows this issue, see the comment with the bug number to reproduce. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: Bay Area Derby Lunch Discussions
This sounds like a great idea! Go collaborators! You might want to ask derby-user the same question, though, those are the folks who count :) David Laura Stewart wrote: It was really great to meet some of the people at the Bay Area Lunch on Wednesday who are involved in Derby! One of the people that I met there was Kim Haase. Kim and I have been contributing documentation to Derby, and naturally we discussed the Derby Library during the lunch. An area that we think might be good to improve is the organization of information which discusses how to setup and start working with Derby. We discussed the possibility of moving the entire content of the Working With Derby manual into the Getting Started Guide. The Working With Derby content is really good, but might be better if it is placed in context with the installation and getting started information. Additionally, the information in the Getting Started Guide that describes SQL syntax might be better in the Reference Manual, where the SQL statements are actually documented. There currently is no information in the Reference Manual that explains the syntax diagrams. How does the rest of the Derby community feel about us analyzing the current organization of the Getting Started Guide and the Working With Derby information, with the intent to propose a reorganization of the information? Are there other areas in the Derby Library that seem to be organized poorly?
[jira] Commented: (DERBY-2334) Accept the new collation attribute and store it in services.properties file. Will be used for language based collation ordering.
[ https://issues.apache.org/jira/browse/DERBY-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473858 ] Daniel John Debrunner commented on DERBY-2334: -- I think the BaseMonitor is the incorrect location for checking and setting the collation. I can see why you put it there, to match territory, but territory is an attribute that affects the whole system, such as all error messages. Collation is limited to the SQL subsystem though, much like the handling the log_device is limited to the store code. I think the code related to collation should be in the data dictionary. I think this would mean that the collation setting would be stored as a database property and not in service.properties, I think that means the same api, just not adding the name for the collation property into the list in PropertyUtil. Two other thoughts; Having the database property being different to the JDBC attribute might be a good idea, e.g. derby.database.collation, like territory is not stored using territory, but derby.serviceLocale. I'll try remember why I think this is a better model. :-) The name TERRITORY_BASED_COLLATION is somewhat awkward. Since it's being set using collation=TERRITORY_BASED_COLLATION it doesn't really need the _COLLATION. It would good for the term to be something that described how the collation was being done, not what other attribute it is based upon. I wonder if there is a standard term for this, UNICODE might be it from SQL Standard (and UniCode10) but more checking is needed. I wonder if Java defines its locale based collators in terms of any standard terms. The code can proceed with the current term, but it might be good to change it before 10.3 is released. Accept the new collation attribute and store it in services.properties file. Will be used for language based collation ordering. Key: DERBY-2334 URL: https://issues.apache.org/jira/browse/DERBY-2334 Project: Derby Issue Type: Sub-task Components: JDBC Affects Versions: 10.3.0.0 Reporter: Mamta A. Satoor Assigned To: Mamta A. Satoor Attachments: DERBY2334_AcceptCollationAttribute_diff_V01.txt, DERBY2334_AcceptCollationAttribute_stat_V01.txt A user would specify the collation desired (UCS_BASIC or TERRITORY_BASED_COLLATION) through the new jdbc url attribute - collation. This attribute will be used to determine collation ordering for character datatypes, -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned, should have thrown an error -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[ https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yip Ng updated DERBY-2351: -- Description: When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected was: When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned, should have thrown an error -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2342) convert importExport.java to junit
[
https://issues.apache.org/jira/browse/DERBY-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473861
]
Suresh Thalamati commented on DERBY-2342:
-
Thanks for converting this test to junit , Andrew. Patch looks good to me., I
don't know much about junit..
Just couple of trivial comments/questions.
In the old tests harness I thought the policy was test files shoud not be
accessed under priveleged blocks. In the new junit framework
is it necessary to add priveleged blocks to access test files also ?
some comments :
File :
++ java/testing/org/apache/derbyTesting/junit/BaseTestCase.java (working copy)
1)
+ assertEquals(f1, f2);
+ } catch (FileNotFoundException e) {
+ fail(FileNotFoundException in
assertEquals(File,File): + e.toString());
+ } catch (IOException e) {
+ fail(IOException in assertEquals(File,
File): + e.toString());
+ }
It might be better to throw the exception up or a print stack trace also.
If it ever fails on some platform , stacks will be helpful.
2)
+
+ return new Boolean(true);
Any reson for returning a Boolean object from the run() method ? I
thought you are actaully passing it up ,
but looks like it just getting ignored. May you should just return null.
+++
java/testing/org/apache/derbyTesting/functionTests/tests/tools/ImportExportTest.java
(revision 0)
1)
+ Derby - Class org.apache.derbyTesting.functionTests.tests.tools.importExport
-- Name is copyright notices is correct.
2)
+ ps.setString(3, (fromTable==null ? fromTable : extinout/ +
fromTable + .dat ));
support files exttinout /extin are hard coded in this test. May be you want
to use the methods defined in SupportFilesSetup class.
to access/create support files. p SupportFilesSetup : ublic static File
getReadOnly(String name) ..etc.
3) This test is easy to understand. you may want to add still some comments
to the test , especially the data verification case.
Thanks
-suresh
convert importExport.java to junit
--
Key: DERBY-2342
URL: https://issues.apache.org/jira/browse/DERBY-2342
Project: Derby
Issue Type: Improvement
Components: Tools
Affects Versions: 10.3.0.0
Reporter: Andrew McIntyre
Assigned To: Andrew McIntyre
Fix For: 10.3.0.0
Attachments: derby-2342-v1.diff, derby-2342-v1.stat,
derby-2342-v2.diff
Convert org.apache.derbyTesting.functionTests.tests.tools.importExport to
junit. New test is called ImportExportTest.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[ https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473862 ] Bryan Pendleton commented on DERBY-2351: Hi Yip. I haven't spent any time looking at this yet, just wanted to mention a hunch that this might be related to DERBY-1861. If you apply the patch for DERBY-1861, does the behavior change at all? ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[ https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473864 ] Yip Ng commented on DERBY-2351: --- Hi Bryan. Let me try to apply the DERBY-1861 patch and see if it is related. Meanwhile, it looks like the error is not limited to order by with expression, I get the same behavior with column reference. ij select distinct c2 from t1 order by c1; C2|C1 -- c |1 b |2 a |3 ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[ https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473866 ] Yip Ng commented on DERBY-2351: --- I applied the DERBY-1861 patch but the result is the same, so it may or may not be related. I remembered that in some other databases(DB2 and Oracle), this construction is not allowed but I can't recall for now that if this is an implementation or a standards restriction. ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[ https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473868 ] Bryan Pendleton commented on DERBY-2351: Thanks for checking, Yip! It was worth a try... :) ORDER BY with expression with distinct in the select list returns incorrect result -- Key: DERBY-2351 URL: https://issues.apache.org/jira/browse/DERBY-2351 Project: Derby Issue Type: Bug Components: SQL Affects Versions: 10.2.2.0, 10.3.0.0 Environment: Any Reporter: Yip Ng When distinct is in the select list and the query has order by with expression, the resultset produced contains an additional column. ij create table t1 (c1 int, c2 varchar(10)) 0 rows inserted/updated/deleted ij insert into t1 values (1,'a'),(2,'b'),(3,'c'); 3 rows inserted/updated/deleted select distinct c1, c2 from t1 order by c1; C1 |C2 -- 1 |a 2 |b 3 |c 3 rows selected ij select distinct c1, c2 from t1 order by c1+1; C1 |C2|3 =returns 3 columns, incorrect result returned -- 1 |a |2 2 |b |3 3 |c |4 3 rows selected -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Regression Test Report - tinderbox_trunk16 508656 - Sun DBTG
[Auto-generated mail] *tinderbox_trunk16* 508656/2007-02-17 00:32:31 CET Failed TestsOK Skip Duration Suite --- *Jvm: 1.6* SunOS-5.10_i86pc-i386 0431431 0 111.81% derbyall F:1,E:068016800 0 791.30% org.apache.derbyTesting.functionTests.suites.All Details in http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/testing/Limited/testSummary-508656.html Attempted failure analysis in http://dbtg.thresher.com/derby/test/tinderbox_trunk16/jvm1.6/FailReports/508656.html --- Changes in http://dbtg.thresher.com/derby/test/tinderbox_trunk16/UpdateInfo/508656.txt ( All results in http://dbtg.thresher.com/derby/test/ )
[jira] Commented: (DERBY-2351) ORDER BY with expression with distinct in the select list returns incorrect result
[
https://issues.apache.org/jira/browse/DERBY-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473871
]
Yip Ng commented on DERBY-2351:
---
Ah, I think I see why this particular case should throw an exception. The
result is not predictable (and not portable) if the ORDER BY sort key is not in
the SELECT list. Consider the following example:
create table person (name varchar(10), age int);
insert into person values ('John', 10);
insert into person values ('John', 30);
insert into person values ('Mary', 20);
SELECT DISTINCT name FROM person ORDER BY age;
So which row to display for John? The John (age 10) or the other John (age
30)?
Result may be:
John
Mary
or
Mary
John
Derby should not allow this and return an error.
ORDER BY with expression with distinct in the select list returns incorrect
result
--
Key: DERBY-2351
URL: https://issues.apache.org/jira/browse/DERBY-2351
Project: Derby
Issue Type: Bug
Components: SQL
Affects Versions: 10.2.2.0, 10.3.0.0
Environment: Any
Reporter: Yip Ng
When distinct is in the select list and the query has order by with
expression, the resultset produced contains an additional column.
ij create table t1 (c1 int, c2 varchar(10))
0 rows inserted/updated/deleted
ij insert into t1 values (1,'a'),(2,'b'),(3,'c');
3 rows inserted/updated/deleted
select distinct c1, c2 from t1 order by c1;
C1 |C2
--
1 |a
2 |b
3 |c
3 rows selected
ij select distinct c1, c2 from t1 order by c1+1;
C1 |C2|3 =returns 3
columns, incorrect result returned
--
1 |a |2
2 |b |3
3 |c |4
3 rows selected
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Dan's svn commit: r508418 - Mapping of DOUBLE to Types.INTEGER
Dan, your commit this morning had following change for Given a valid SQL
type return the corresponding
JDBC type identifier from java.sql.Types.
+if (DOUBLE.equals(type) || DOUBLE PRECISION.equals(type))
+return Types.INTEGER;
Is Types.INTEGER the correct return value? Reference manual has a table for
Mapping of java.sql.Types to SQL types and it doesn't have INTEGER for SQL
type DOUBLE.
Mamta
On 2/16/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
Author: djd
Date: Fri Feb 16 06:24:03 2007
New Revision: 508418
URL: http://svn.apache.org/viewvc?view=revrev=508418
Log:
Improve the JDBC.assertSameContents to use the assertEquals() methods for
Blob and Clob in BaseJDBCTestCase.
Create a SQLXML constant in JDBC for the tests to use rather than using
engine code.
Add an initial version of utility method to DatabaseMetaDataTest that
returns a JDBC type identifier from java.sql.Types
given a SQL type name.
Modified:
db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java
db/derby/code/trunk/java/testing/org/apache/derbyTesting/junit/BaseJDBCTestCase.java
db/derby/code/trunk/java/testing/org/apache/derbyTesting/junit/JDBC.java
Modified:
db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java
URL:
http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java?view=diffrev=508418r1=508417r2=508418
==
---
db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java
(original)
+++
db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/DatabaseMetaDataTest.java
Fri Feb 16 06:24:03 2007
@@ -20,6 +20,7 @@
*/
package org.apache.derbyTesting.functionTests.tests.jdbcapi;
+import java.io.IOException;
import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
@@ -79,6 +80,7 @@
* metadata continues to work at various points in the upgrade.
*/
public class DatabaseMetaDataTest extends BaseJDBCTestCase {
+
/*
** Escaped function testing
*/
@@ -786,10 +788,11 @@
/**
* Execute dmd.getTables() but perform additional checking
* of the ODBC variant.
+ * @throws IOException
*/
private ResultSet getDMDTables(DatabaseMetaData dmd,
String catalog, String schema, String table, String[]
tableTypes)
-throws SQLException
+throws SQLException, IOException
{
checkGetTablesODBC(catalog, schema, table, tableTypes);
return dmd.getTables(catalog, schema, table, tableTypes);
@@ -799,8 +802,9 @@
* Test getTables() without modifying the database.
*
* @throws SQLException
+ * @throws IOException
*/
-public void testGetTablesNoModify() throws SQLException {
+public void testGetTablesNoModify() throws SQLException, IOException
{
DatabaseMetaData dmd = getDMD();
@@ -876,8 +880,9 @@
* Test getTables() with modifying the database.
*
* @throws SQLException
+ * @throws IOException
*/
-public void testGetTablesModify() throws SQLException {
+public void testGetTablesModify() throws SQLException, IOException {
int totalTables = createTablesForTest(false);
@@ -994,9 +999,10 @@
/**
* Execute and check the ODBC variant of getTables which
* uses a procedure to provide the same information to ODBC clients.
+ * @throws IOException
*/
private void checkGetTablesODBC(String catalog, String schema,
-String table, String[] tableTypes) throws SQLException
+String table, String[] tableTypes) throws SQLException,
IOException
{
String tableTypesAsString = null;
if (tableTypes != null) {
@@ -1356,8 +1362,6 @@
*/
public void testGetTypeInfo() throws SQLException
{
- // SQLXML is the constant used to represent XML data type in derby
- final int SQLXML = JDBC40Translation.SQLXML;
// Client returns BOOLEAN type from the engine as SMALLINT
int BOOLEAN = Types.BOOLEAN;
if (usingDerbyNetClient())
@@ -1430,7 +1434,7 @@
Types.INTEGER, Types.LONGVARBINARY, Types.LONGVARCHAR,
Types.NUMERIC, Types.REAL, Types.SMALLINT,
Types.TIME, Types.TIMESTAMP, Types.VARBINARY,
- Types.VARCHAR, SQLXML
+ Types.VARCHAR, JDBC.SQLXML
};
// Rows are returned from getTypeInfo in order of
@@ -1517,7 +1521,7 @@
case Types.VARCHAR:
precision = 32672;
break;
- case SQLXML:
+ case JDBC.SQLXML:
precision = 0;
break;
}
@@ -1600,7 +1604,7 @@
case Types.VARCHAR:
searchable = DatabaseMetaData.typeSearchable;
[jira] Updated: (DERBY-2342) convert importExport.java to junit
[ https://issues.apache.org/jira/browse/DERBY-2342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew McIntyre updated DERBY-2342: --- Attachment: derby-2342-v3.diff Thanks very much for the review, Suresh! Attaching a patch which incorporates most of your suggestions. I've used SupportFilesSetup.getReadWrite() to get the Files to pass to assertEquals(File,File). For the other cases where the directory is hardcoded, we need to pass a String to the import or export procedure. I could get a File or URL with getReadWrite or getReadWriteURL, but then I'd either have to get the String representation of the File or URL and do some manipulation of that String to get it into the format that the import/export procedures are expecting. I'm not sure that it is worth the trouble to do that when the location of the support file directories is unlikely to change. Patch is still not quite ready to commit, though, as I'm seeing some failures on jdk16 in client mode. convert importExport.java to junit -- Key: DERBY-2342 URL: https://issues.apache.org/jira/browse/DERBY-2342 Project: Derby Issue Type: Improvement Components: Tools Affects Versions: 10.3.0.0 Reporter: Andrew McIntyre Assigned To: Andrew McIntyre Fix For: 10.3.0.0 Attachments: derby-2342-v1.diff, derby-2342-v1.stat, derby-2342-v2.diff, derby-2342-v3.diff Convert org.apache.derbyTesting.functionTests.tests.tools.importExport to junit. New test is called ImportExportTest. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
