[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristian Waagan updated DERBY-2667: --- Attachment: derby-2667-WriteExceptionsToFileAsTheyHappen.diff 'derby-2667-WriteExceptionsToFileAsTheyHappen.diff' is a patch that makes BaseTestCase also write the stack trace to file when an error occurs. The file will be written to the same location as derby.log, and currently it will overwrite any existing file. I have only tested it lightly, but it seems to work fine for me. Is it something we could consider adding? The reason why I wrote it, is that it is sometimes very annoying to wait for suites.All to finish when you get an error/failure on the first line of dots... Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Myrna van Lunteren updated DERBY-2667: -- Comment: was deleted Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Attachment: DERBY-2667_diff_02_21.txt DERBY-2667_stat_02_21.txt Thanks for your comments. Here is the new patch. Please ereview Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Derby Info: [Patch Available] Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Myrna van Lunteren updated DERBY-2667: -- Derby Info: (was: [Patch Available]) Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Attachment: DERBY-2667_diff_2_19.txt DERBY-2667_stat_02_19.txt Thanks for the comments. Please find the modified patch and I hope I tried to address all your comments in that patch. Please review Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Attachment: DERBY-2667_diff_02_15.txt DERBY-2667_stat_02_15.txt Thanks for the comments Dan. Please review the new patch attached. I hope I answered all your comments in that patch. Please review and if everything looks good, please commit Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dyre Tjeldvoll updated DERBY-2667: -- Derby Info: (was: [Patch Available]) Removing the patch flag since the latest review comments indicate that another rev of the patch is required. Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Attachment: DERBY-2667_diff_02_06.txt DERBY-2667_stat_02_06.txt Attaching a patch which will copy the derby.log file of the failed test to the dir user.dir/fail/clientname/testclass/testname. Please review and if looks useful please commit Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjula Kutty updated DERBY-2667: - Derby Info: [Patch Available] Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.1.4 Reporter: Kathey Marsden Priority: Minor Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ole Solberg updated DERBY-2667: --- Attachment: JUnitMethodTrace_Extra.diff.txt JUnitMethodTrace.diff.txt In the regression testing and also in more experimental testing I have felt a need for more details in the JUnit test reports - e.g. to convince myself that a test was really executed. I have done this by modifying 'java/testing/org/apache/derbyTesting/junit/BaseTestCase.java' to print the test method name, and have also included a printout of the time (in ms.) used by the test method. I use -Dderby.tests.trace=true to turn this on. Attachement JUnitMethodTrace.diff.txt. 'java/testing/org/apache/derbyTesting/unitTests/junit/FormatableBitSetTest.java' currently uses JUnit TestCase and would thus not give the desired extra information: The JUnitMethodTrace_Extra.diff.txt patch changes that by using 'BaseTestCase'. Attachement JUnitMethodTrace_Extra.diff.txt Example output: . testSetCharacterStream used 200ms . testGetAsciiStream used 664ms . testGetCharacterStream used 638ms . testGetCharacterStreamWithUnicode used 817ms . testTriggersWithClobColumn used 2453ms . testGetSubString used 431ms E. testGetSubStringWithUnicode used 855ms . testPositionString used 713ms . testPositionStringWithUnicode used 739ms . testPositionClob used 4204ms . testPositionClobWithUnicode used 4832ms . testSmallClobFields used 82ms . testGetClobFromIntColumn used 302ms F. testSetClobToIntColumn used 412ms F. testRaisingOfExceptionsClob used 523ms F. testSetClob used 1116ms . testPositionAgressive used 9073ms . testClobAfterClose used 587ms . testLockingClob used 60695ms . testLockingWithLongRowClob used 60178ms . . . . Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.0.0 Reporter: Kathey Marsden Priority: Minor Attachments: JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-2667: -- Attachment: MemRunner.java TimeRunner.java I'm attaching the two test runners I have written and granting licence to ASF in case someone wants to play with them. MemRunner runs all the tests in suites.All and reports the increase/decrease in memory footprint for each fixture. TimeRunner runs all the tests in suites.All and reports how many milliseconds each fixture took. Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.0.0 Reporter: Kathey Marsden Priority: Minor Attachments: MemRunner.java, TimeRunner.java Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Matrigali updated DERBY-2667: -- What I am looking for is a runner that produces text based output that is appropriate for offline nightly test suites. I like to run whatever test runner the systems that are running regression tests against the code line are. Currently I believe this is sun jvm nightly, sun jvm tinderbox and ibm jvm nightly. This was if I match my environment it is likely the tool will work. While it is great that people can pick/choose their runner - it would be nice to have at least one runner recommended as the default runner that developers could use and nightly regression tests would also use. When I started I first tried the ant option, but immediately ran into memory issues, but noticed nightly runs using the other tools just worked. What I am looking for is: o a tool that will run all the tests without failing, and is tested to continue to do so every night. o complete set of stack traces, including any nested exceptions. Dan points out this is likely a derby issue o the ability to run all the tests and then come back and have a shot at figuring out what went wrong and when. Things that would help are: o printing each test as it runs. Sometimes when you get errors in teardown the stack gives you no idea what test the teardown actually is failing for. o preserve the persistent information when a failure occurs. Most important is the derby.log as it may have more information. But a copy of the database is also interesting, especially in cases of non reproducible errors where this may have been our one chance to gather info, or to allow someone to debug a system specific problem when they don't have access to that type of system. This may be a test issue rather than the runner. I wonder if we just need to write some generic tearDown that on error or failure copies current state somewhere Create more robust junit TestRunner for running derby tests Key: DERBY-2667 URL: https://issues.apache.org/jira/browse/DERBY-2667 Project: Derby Issue Type: Improvement Components: Test Affects Versions: 10.3.0.0 Reporter: Kathey Marsden Priority: Minor Provide a more full featured TestRunner for Derby testing. junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
