[jira] Commented: (DERBY-216) expand largeCodeGen.java test
[ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12429314 ] Daniel John Debrunner commented on DERBY-216: - The insert case is included in largeCodeGen now. expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Issue Type: Sub-task Components: Test Affects Versions: 10.1.2.1 Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Kathey, I ran the lang/largeCodeGen.java with PRINT_FAILURE_EXCEPTIONset to true as follows: java -Dverbose=true -Djvmflags=-mx512M -ms512M -Dframework=DerbyNetClientorg.apache.derbyTesting.functionTests.harness.RunTest lang/largeCodeGen.java and got the following MasterFileName = master/largeCodeGen.out15a16,18 java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error . SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e9. Caused by: org.apache.derby.client.am.SqlException: Statement too complex. Tryrewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e9. ... 4 more19 del PASS: IN clause with 97000 parameters20 del PASS: PREPARE: IN clause with 98000 parameters21 del PASS: IN clause with 98000 parameters22 delbrawny src/NightlyBuildResults.2006-05-10 lschanges.txt ibm142_largeData test.txt update.txtbrawny src/NightlyBuildResults.2006-05-10 tail -f test.txt 27a35,37 java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error . SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:fillResultSet code_length (69127 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e11. Caused by: org.apache.derby.client.am.SqlException: Statement too complex. Tryrewriting the query to remove complexity. Eliminating many duplicate _expression_s or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s)exceeded: method:fillResultSet code_length (69127 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e11. ... 3 more28 add java.sql.SQLException: Java exception: ': java.lang.OutOfMemoryError'. Caused by: org.apache.derby.client.am.SqlException : Java exception: ': java.lang.OutOfMemoryError'. ... 3 moreTest Failed.Thanks, Raman On 5/5/06, Kathey Marsden (JIRA) derby-dev@db.apache.org wrote: [ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12378176 ]Kathey Marsden commented on DERBY-216:--With revision 37644 I fixed the run to run differences and at that time the test passed with both embedded and client on Windows.I never tried on Linux. With revision 377609 Dan was able to move the number of parameters in the in list query from 3400 to 97000 (amazing). I don't know if it passed with client at that time.Raman's testing shows that client fails before 97000 with client. Ramanwhat is the highest number it goes to with client and what is the error and stack trace when it fails? In the test there is aboolean PRINT_FAILURE_EXCEPTIONthat you will have to change to see the failure. expand largeCodeGen.java test -Key: DERBY-216URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example:large in clauselarge insert statement that inserts many rowssql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code.Those cases may pass but the hope is to get rid of these calls in favor of splittingthe code in a centralized way, so add the tests to largeCodeGen even if they don't fail. --This message is automatically generated by JIRA.-If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa-For more information on JIRA, see:http://www.atlassian.com/software/jira -- Ramandeep Kaur[EMAIL PROTECTED]
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Kathey, please ignore previous stack trace. I am writing it here again:- MasterFileName = master/largeCodeGen.out15a16,18 java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203ax d010x50a9065e9. Caused by: org.apache.derby.client.am.SqlException: Statement too complex. Tryrewriting the query to remove complexity. Eliminating many duplicate _expression_s or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s)exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e9 . ... 4 more19 del PASS: IN clause with 97000 parameters20 del PASS: PREPARE: IN clause with 98000 parameters21 del PASS: IN clause with 98000 parameters22 del FAILED QUERY: IN clause with 99000 parameters. 22a22,29 FAILED QUERY: IN clause with 97000 parameters. java.sql.SQLException: The parameter position '31,465' is out of range. The number of parameters for this prepared statement is '31,464'. at org.apache.derby.client.am.PreparedStatement.setInt(PreparedStatement.java(Compiled Code)) Caused by: org.apache.derby.client.am.SqlException: The parameter position '31,465' is out of range. The number of parameters for this prepared statement is '31,464'. at org.apache.derby.client.am.PreparedStatement.checkForValidParameterIndex(PreparedStatement.java(Compiled Code)) at org.apache.derby.client.am.PreparedStatement.checkSetterPreconditions (PreparedStatement.java(Inlined Compiled Code)) at org.apache.derby.client.am.PreparedStatement.setIntX(PreparedStatement.java(Inlined Compiled Code)) ... 5 more27a35,37 java.sql.SQLException : Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error . SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:fillResultSet code_length (69127 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e11. Caused by: org.apache.derby.client.am.SqlException: Statement too complex. Tryrewriting the query to remove complexity. Eliminating many duplicate _expression_s or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s)exceeded: method:fillResultSet code_length (69127 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e11 . ... 3 more28 add java.sql.SQLException: Java exception: ': java.lang.OutOfMemoryError'. Caused by: org.apache.derby.client.am.SqlException: Java exception: ': java.lang.OutOfMemoryError '. ... 3 moreTest Failed. On 5/10/06, Ramandeep Kaur [EMAIL PROTECTED] wrote: Kathey, I ran the lang/largeCodeGen.java with PRINT_FAILURE_EXCEPTIONset to true as follows: java -Dverbose=true -Djvmflags=-mx512M -ms512M -Dframework=DerbyNetClientorg.apache.derbyTesting.functionTests.harness.RunTest lang/largeCodeGen.java and got the following MasterFileName = master/largeCodeGen.out15a16,18 java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error . SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e9. Caused by: org.apache.derby.client.am.SqlException: Statement too complex. Tryrewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error. SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:e1 code_length (65577 65535) in generated class org.apache.derby.exe.ac46a08075x010bx203axd010x50a9065e9. ... 4 more19 del PASS: IN clause with 97000 parameters20 del PASS: PREPARE: IN clause with 98000 parameters21 del PASS: IN clause with 98000 parameters22 delbrawny src/NightlyBuildResults.2006-05-10 lschanges.txt ibm142_largeData test.txt update.txtbrawny src/NightlyBuildResults.2006-05-10 tail -f test.txt 27a35,37 java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error . SQLSTATE: XBCM4: Java class file format limit(s) exceeded: method:fillResultSet code_length
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Attaching files. Thanks, Raman largeCodeGen.out Description: Binary data PASS: PREPARE: Logical operators with 200 parameters PASS: Logical operators with 200 parameters PASS: PREPARE: Logical operators with 300 parameters PASS: Logical operators with 300 parameters PASS: PREPARE: Logical operators with 400 parameters PASS: Logical operators with 400 parameters PASS: PREPARE: Logical operators with 500 parameters PASS: Logical operators with 500 parameters PASS: PREPARE: Logical operators with 600 parameters PASS: Logical operators with 600 parameters PASS: PREPARE: Logical operators with 700 parameters PASS: Logical operators with 700 parameters PASS: PREPARE: Logical operators with 800 parameters PASS: Logical operators with 800 parameters FAILED QUERY: Logical operators with 900 parameters. PASS: PREPARE: IN clause with 3400 parameters PASS: IN clause with 3400 parameters PASS: PREPARE: IN clause with 97000 parameters PASS: IN clause with 97000 parameters PASS: PREPARE: IN clause with 98000 parameters PASS: IN clause with 98000 parameters FAILED QUERY: IN clause with 99000 parameters. PASS: PREPARE: SELECT with 800 unions PASS: EXECUTE SELECT with 800 unions Row data check ok PASS: PREPARE: SELECT with 900 unions PASS: EXECUTE SELECT with 900 unions Row data check ok FAILED QUERY: SELECT with 1000 unions. FAILED QUERY: SELECT with 1 unions. largeCodeGen.diff Description: Binary data largeCodeGen.fail Description: Binary data largeCodeGen.tmp Description: Binary data
[jira] Commented: (DERBY-216) expand largeCodeGen.java test
[ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12378154 ] Sunitha Kambhampati commented on DERBY-216: --- I remember the last time I ran the largeData with ibm142 a couple months ago, this suite passed OK. Now, I ran the largeData suite with the ibm 1.5 jvm and the largeCodeGen test in embedded mode fails on linux 2.6 kernel, rhel4. There is some discussion on the list about this test failing with client but passes with embedded. I see in the earlier comment - Kathey mentioned that the failed cases (?) (i think ) varies for different jvm to jvm, run to run. -- Has this test ever run successfully before ? -- Is it expected that the test output will vary ? if so, then how do we ensure no regression has happened - is there some way to know that. ? Should this test be enabled as part of largeData ? Thanks. expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Sunitha Kambhampati (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12378154 ] Sunitha Kambhampati commented on DERBY-216: --- I remember the last time I ran the largeData with ibm142 a couple months ago, this suite passed OK. Now, I ran the largeData suite with the ibm 1.5 jvm and the largeCodeGen test in embedded mode fails on linux 2.6 kernel, rhel4. There is some discussion on the list about this test failing with client but passes with embedded. I see in the earlier comment - Kathey mentioned that the failed cases (?) (i think ) varies for different jvm to jvm, run to run. -- Has this test ever run successfully before ? -- Is it expected that the test output will vary ? if so, then how do we ensure no regression has happened - is there some way to know that. ? Should this test be enabled as part of largeData ? Thanks. expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. Yes the test passed for me and Raman for the embedded. But no idea about getting different o/p for different jvms. Thanks Manjula
[jira] Commented: (DERBY-216) expand largeCodeGen.java test
[ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12378176 ] Kathey Marsden commented on DERBY-216: -- With revision 37644 I fixed the run to run differences and at that time the test passed with both embedded and client on Windows. I never tried on Linux. With revision 377609 Dan was able to move the number of parameters in the in list query from 3400 to 97000 (amazing). I don't know if it passed with client at that time. Raman's testing shows that client fails before 97000 with client. Raman what is the highest number it goes to with client and what is the error and stack trace when it fails? In the test there is a boolean PRINT_FAILURE_EXCEPTION that you will have to change to see the failure. expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
I haven't followed the largeCodeGen test much, does it take an extremely long time or need a lot of memory? I am not sure what amount of time is the cutover from not being appropriate in the nightly suite. Otherwise we should just have add it somewhere else. I have run the large data test recently, and I think it took 6 hours and 17gb on a 1.8 laptop. So far I have not made it through a whole suite (basically I have only had overnight free on the machine and it has not finished). It would be nice if when this test succeeds if it dropped it's tables so you would not be left with 17gb in the test directory. If anyone has the resources to run the suite repeatedly, that would great. It now will catch some serious blob/clob regressions we have had in the past. Kathey Marsden (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12364516 ] Kathey Marsden commented on DERBY-216: -- I pulled the largeCodeGen test into the largeData suite. I suppressed the exception output for failed cases by default. It varies from run to run and jvm to jvm. There is a boolean PRINT_FAILURE_EXCEPTION that can be turned on for debugging. I am curious. Is there anyone out there that runs the largeData test from time to time? What are the system requirements? expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail.
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Mike Matrigali wrote: I haven't followed the largeCodeGen test much, does it take an extremely long time or need a lot of memory? I am not sure what amount of time is the cutover from not being appropriate in the nightly suite. Otherwise we should just have add it somewhere else. I have run the large data test recently, and I think it took 6 hours and 17gb on a 1.8 laptop. So far I have not made it through a whole suite (basically I have only had overnight free on the machine and it has not finished). It would be nice if when this test succeeds if it dropped it's tables so you would not be left with 17gb in the test directory. If anyone has the resources to run the suite repeatedly, that would great. It now will catch some serious blob/clob regressions we have had in the past. Kathey Marsden (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12364516 ] Kathey Marsden commented on DERBY-216: -- I pulled the largeCodeGen test into the largeData suite. I suppressed the exception output for failed cases by default. It varies from run to run and jvm to jvm. There is a boolean PRINT_FAILURE_EXCEPTION that can be turned on for debugging. I am curious. Is there anyone out there that runs the largeData test from time to time? What are the system requirements? expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. Hi Mike, I can run the test on one of my machines which is about 2G RAM and ~20GB disk space. Will let you know the results Thanks Manjula
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Mike Matrigali wrote: I haven't followed the largeCodeGen test much, does it take an extremely long time or need a lot of memory? I am not sure what amount of time is the cutover from not being appropriate in the nightly suite. Otherwise we should just have add it somewhere else. It drags my latop with 1Gb of memory to a halt, due to large memory usage. I'm not sure how long it takes, at least 15mins, I usually kill it as I have already gathered what I need to know from the first portion of it. In fact it's the very last query that takes the time. It maybe that once I fix some of the code generation issues the test can be moved into the standard language suite. Dan.
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
Mike Matrigali wrote: I haven't followed the largeCodeGen test much, does it take an extremely long time or need a lot of memory? I am not sure what amount of time is the cutover from not being appropriate in the nightly suite. largeCodeGen takes about 5 minutes on my laptop, but I have heard reports that it has taken longer for others. I am not sure what the cutoff should be. Kathey
Re: [jira] Commented: (DERBY-216) expand largeCodeGen.java test
ok, looks like it belongs in that suite. I just wanted to make sure putting stuff in this suite is the exception. Even with this test it sounds like it would be nice if the 1st part was a standard test and the problem query was in this suite. Daniel John Debrunner wrote: Mike Matrigali wrote: I haven't followed the largeCodeGen test much, does it take an extremely long time or need a lot of memory? I am not sure what amount of time is the cutover from not being appropriate in the nightly suite. Otherwise we should just have add it somewhere else. It drags my latop with 1Gb of memory to a halt, due to large memory usage. I'm not sure how long it takes, at least 15mins, I usually kill it as I have already gathered what I need to know from the first portion of it. In fact it's the very last query that takes the time. It maybe that once I fix some of the code generation issues the test can be moved into the standard language suite. Dan.
[jira] Commented: (DERBY-216) expand largeCodeGen.java test
[ http://issues.apache.org/jira/browse/DERBY-216?page=comments#action_12364516 ] Kathey Marsden commented on DERBY-216: -- I pulled the largeCodeGen test into the largeData suite. I suppressed the exception output for failed cases by default. It varies from run to run and jvm to jvm. There is a boolean PRINT_FAILURE_EXCEPTION that can be turned on for debugging. I am curious. Is there anyone out there that runs the largeData test from time to time? What are the system requirements? expand largeCodeGen.java test - Key: DERBY-216 URL: http://issues.apache.org/jira/browse/DERBY-216 Project: Derby Type: Sub-task Components: Test Reporter: Kathey Marsden the largeCodeGen test needs to be expanded to include other cases that genreate large amounts of byte code. For example: large in clause large insert statement that inserts many rows sql statements with large constant values It is best if the verious tests just use a variable that can be bumped higher and higher for testing and if individual cases are isolated. Possible approaches, think of ways to make sql statements really big that will take different code paths. Look in the code for instances of statementNumHitLimit and create cases that pass through that code. Those cases may pass but the hope is to get rid of these calls in favor of splitting the code in a centralized way, so add the tests to largeCodeGen even if they don't fail. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
