Re: GCJ Mauve testing
Andrew John Hughes wrote: > What is the plan for merging to trunk? If we're sure it fixes bugs and doesn't introduce new ones, once it's stable we can merge. We're close, I think. Andrew.
Re: GCJ Mauve testing
2008/10/18 Matthias Klose <[EMAIL PROTECTED]>: > Andrew John Hughes schrieb: >> I've just run Mauve against both the 4.3 tree and our >> Classpath 0.98 merge tree (effectively equivalent to Classpath >> CVS + GCJ-specific changes). Overall: >> >> 4.3: >> 281 of 3177 tests failed. 123926 total calls to harness.check() failed. >> >> 4.4: >> 239 of 3177 tests failed. 143832 total calls to harness.check() failed. >> >> There will be some new tests that compile with 4.4 but not 4.3 (e.g. >> java.util.Scanner tests). > > Here is another run, done with trunk and branch 20081017. > Thanks for this. > 4.3: > 277 of 3175 tests failed. 139350 total calls to harness.check() failed. > > trunk: > 232 of 3175 tests failed. 122871 total calls to harness.check() failed. > > merge branch (as 4.4 above): > 229 of 3175 tests failed. 122914 total calls to harness.check() failed. > >> Additional FAILs are as follows: >> >> +FAIL: gnu.java.security.util.TestOfIntegerUtil >> +FAIL: java.sql.Date.DateTest >> +FAIL: java.lang.Character.Blocks15 >> +FAIL: java.lang.InheritableThreadLocal.simple >> +FAIL: java.text.SimpleDateFormat.parse >> +FAIL: java.util.Calendar.ampm >> +FAIL: java.util.Scanner.LotsOfPMInts >> +FAIL: java.util.Scanner.Inputs >> +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert >> +FAIL: >> javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground > > comparing 4.3 with the merge branch: > > 56 progressions and > > +FAIL: java.awt.TextField.PaintTest > +FAIL: java.lang.ref.PhantomReference.phantom > +FAIL: java.text.SimpleDateFormat.parse > +FAIL: java.util.Calendar.ampm > +FAIL: java.util.Scanner.FindWithinHorizon > +FAIL: java.util.logging.Logger.getParent > +FAIL: javax.swing.JButton.constructors > +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert > > The following three go away if the cacerts file is in place. > > +FAIL: gnu.java.security.jce.TestOfHttps > +FAIL: gnu.java.security.util.TestOfIntegerUtil > +FAIL: java.net.URLConnection.getHeaderFields > > comparing the trunk with the merge branch (0.97.2 vs. 0.98pre): > > 9 progressions and > > +FAIL: gnu.java.security.util.TestOfIntegerUtil > +FAIL: java.awt.Component.clickModifiers > +FAIL: java.text.SimpleDateFormat.Localization > +FAIL: java.util.Calendar.ampm > +FAIL: java.util.Currency.US > +FAIL: java.util.Scanner.FindWithinHorizon > +FAIL: java.util.logging.Logger.getParent > > Full logs at http://people.ubuntu.com/~doko/java/mauve-results/ > > libjava/testsuite/libjava.mauve doesn't work with current mauve, so all > builds were installed, and > mauve was run on the installed build. > > Matthias > > The Scanner tests can be written off, as they won't have compiled with either 4.3 or trunk (neither have java.util.Scanner). The merge branch seems to be an improvement, but there are clearly some new localisation issues. What is the plan for merging to trunk? -- Andrew :-) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
Re: GCJ Mauve testing
Andrew John Hughes schrieb: > I've just run Mauve against both the 4.3 tree and our > Classpath 0.98 merge tree (effectively equivalent to Classpath > CVS + GCJ-specific changes). Overall: > > 4.3: > 281 of 3177 tests failed. 123926 total calls to harness.check() failed. > > 4.4: > 239 of 3177 tests failed. 143832 total calls to harness.check() failed. > > There will be some new tests that compile with 4.4 but not 4.3 (e.g. > java.util.Scanner tests). Here is another run, done with trunk and branch 20081017. 4.3: 277 of 3175 tests failed. 139350 total calls to harness.check() failed. trunk: 232 of 3175 tests failed. 122871 total calls to harness.check() failed. merge branch (as 4.4 above): 229 of 3175 tests failed. 122914 total calls to harness.check() failed. > Additional FAILs are as follows: > > +FAIL: gnu.java.security.util.TestOfIntegerUtil > +FAIL: java.sql.Date.DateTest > +FAIL: java.lang.Character.Blocks15 > +FAIL: java.lang.InheritableThreadLocal.simple > +FAIL: java.text.SimpleDateFormat.parse > +FAIL: java.util.Calendar.ampm > +FAIL: java.util.Scanner.LotsOfPMInts > +FAIL: java.util.Scanner.Inputs > +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert > +FAIL: > javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground comparing 4.3 with the merge branch: 56 progressions and +FAIL: java.awt.TextField.PaintTest +FAIL: java.lang.ref.PhantomReference.phantom +FAIL: java.text.SimpleDateFormat.parse +FAIL: java.util.Calendar.ampm +FAIL: java.util.Scanner.FindWithinHorizon +FAIL: java.util.logging.Logger.getParent +FAIL: javax.swing.JButton.constructors +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert The following three go away if the cacerts file is in place. +FAIL: gnu.java.security.jce.TestOfHttps +FAIL: gnu.java.security.util.TestOfIntegerUtil +FAIL: java.net.URLConnection.getHeaderFields comparing the trunk with the merge branch (0.97.2 vs. 0.98pre): 9 progressions and +FAIL: gnu.java.security.util.TestOfIntegerUtil +FAIL: java.awt.Component.clickModifiers +FAIL: java.text.SimpleDateFormat.Localization +FAIL: java.util.Calendar.ampm +FAIL: java.util.Currency.US +FAIL: java.util.Scanner.FindWithinHorizon +FAIL: java.util.logging.Logger.getParent Full logs at http://people.ubuntu.com/~doko/java/mauve-results/ libjava/testsuite/libjava.mauve doesn't work with current mauve, so all builds were installed, and mauve was run on the installed build. Matthias
Re: GCJ Mauve testing
2008/10/7 Andrew Haley <[EMAIL PROTECTED]>: > Andrew John Hughes wrote: >> 2008/10/6 Andrew Haley <[EMAIL PROTECTED]>: >>> Andrew John Hughes wrote: >>> FAIL: java.lang.InheritableThreadLocal.simple Test timed out. Use -timeout [millis] option to change the timeout value. >>> This is strange, given that gcj has its own InheritableThreadLocal that >>> hasn't changed. I'll look. >>> >>> Andrew. >>> >> >> I did another Mauve run against the merge branch and gained some passes, >> but also some new failures: >> >> +PASS: gnu.java.security.util.TestOfIntegerUtil >> +PASS: java.awt.event.MouseEvent.modifiersEx >> +PASS: java.awt.event.MouseEvent.modifiers >> +PASS: java.util.logging.Logger.global >> +PASS: javax.swing.plaf.metal.MetalScrollBarUI.constructor >> +PASS: javax.swing.plaf.metal.MetalScrollBarUI.installDefaults >> +FAIL: java.util.logging.Logger.hierarchyChecks >> +FAIL: java.util.logging.Logger.securityChecks >> +FAIL: java.util.logging.SocketHandler.getFormatter >> +FAIL: java.util.Scanner.DoubleFloat >> +FAIL: java.util.Scanner.FishString >> +FAIL: java.util.Scanner.Radix >> +FAIL: javax.sound.sampled.AudioProperties >> +FAIL: javax.swing.JTable.TableRobot >> >> Odd to say the least. > > Curious. It's not immediately obvious how the change could have affected > those, but I'll have a look. > > Andrew. > > I think some of them may be spurious. gnu.java.security.util.TestOfIntegerUtil particularly is based on timing. I'll post the test log shortly. -- Andrew :-) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
Re: GCJ Mauve testing
Andrew John Hughes wrote: > 2008/10/6 Andrew Haley <[EMAIL PROTECTED]>: >> Andrew John Hughes wrote: >> >>> FAIL: java.lang.InheritableThreadLocal.simple >>> Test timed out. Use -timeout [millis] option to change the timeout value. >> This is strange, given that gcj has its own InheritableThreadLocal that >> hasn't changed. I'll look. >> >> Andrew. >> > > I did another Mauve run against the merge branch and gained some passes, > but also some new failures: > > +PASS: gnu.java.security.util.TestOfIntegerUtil > +PASS: java.awt.event.MouseEvent.modifiersEx > +PASS: java.awt.event.MouseEvent.modifiers > +PASS: java.util.logging.Logger.global > +PASS: javax.swing.plaf.metal.MetalScrollBarUI.constructor > +PASS: javax.swing.plaf.metal.MetalScrollBarUI.installDefaults > +FAIL: java.util.logging.Logger.hierarchyChecks > +FAIL: java.util.logging.Logger.securityChecks > +FAIL: java.util.logging.SocketHandler.getFormatter > +FAIL: java.util.Scanner.DoubleFloat > +FAIL: java.util.Scanner.FishString > +FAIL: java.util.Scanner.Radix > +FAIL: javax.sound.sampled.AudioProperties > +FAIL: javax.swing.JTable.TableRobot > > Odd to say the least. Curious. It's not immediately obvious how the change could have affected those, but I'll have a look. Andrew.
Re: GCJ Mauve testing
2008/10/6 Andrew Haley <[EMAIL PROTECTED]>: > Andrew John Hughes wrote: > >> FAIL: java.lang.InheritableThreadLocal.simple >> Test timed out. Use -timeout [millis] option to change the timeout value. > > This is strange, given that gcj has its own InheritableThreadLocal that > hasn't changed. I'll look. > > Andrew. > I did another Mauve run against the merge branch and gained some passes, but also some new failures: +PASS: gnu.java.security.util.TestOfIntegerUtil +PASS: java.awt.event.MouseEvent.modifiersEx +PASS: java.awt.event.MouseEvent.modifiers +PASS: java.util.logging.Logger.global +PASS: javax.swing.plaf.metal.MetalScrollBarUI.constructor +PASS: javax.swing.plaf.metal.MetalScrollBarUI.installDefaults +FAIL: java.util.logging.Logger.hierarchyChecks +FAIL: java.util.logging.Logger.securityChecks +FAIL: java.util.logging.SocketHandler.getFormatter +FAIL: java.util.Scanner.DoubleFloat +FAIL: java.util.Scanner.FishString +FAIL: java.util.Scanner.Radix +FAIL: javax.sound.sampled.AudioProperties +FAIL: javax.swing.JTable.TableRobot Odd to say the least. -- Andrew :-) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
Re: GCJ Mauve testing
Andrew John Hughes wrote: > FAIL: java.lang.InheritableThreadLocal.simple > Test timed out. Use -timeout [millis] option to change the timeout value. This is strange, given that gcj has its own InheritableThreadLocal that hasn't changed. I'll look. Andrew.
Re: GCJ Mauve testing
2008/9/27 Andrew John Hughes <[EMAIL PROTECTED]>: > I've just run Mauve against both the 4.3 tree and our > Classpath 0.98 merge tree (effectively equivalent to Classpath > CVS + GCJ-specific changes). Overall: > > 4.3: > 281 of 3177 tests failed. 123926 total calls to harness.check() failed. > > 4.4: > 239 of 3177 tests failed. 143832 total calls to harness.check() failed. > > There will be some new tests that compile with 4.4 but not 4.3 (e.g. > java.util.Scanner tests). > > Additional FAILs are as follows: > > +FAIL: gnu.java.security.util.TestOfIntegerUtil > +FAIL: java.sql.Date.DateTest > +FAIL: java.lang.Character.Blocks15 > +FAIL: java.lang.InheritableThreadLocal.simple > +FAIL: java.text.SimpleDateFormat.parse > +FAIL: java.util.Calendar.ampm > +FAIL: java.util.Scanner.LotsOfPMInts > +FAIL: java.util.Scanner.Inputs > +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert > +FAIL: > javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground > > The two Scanner tests we can discount as regressions as they simply won't > have run with 4.3. > > The remaining failures: > > FAIL: gnu.java.security.util.TestOfIntegerUtil > line 71: IntegerUtil.valueOf(String) MUST be faster than > Integer.valueOf(String) [1] -- boolean passed to check was fa\ > lse > FAIL: java.sql.Date.DateTest > uncaught exception: > java.lang.OutOfMemoryError > FAIL: java.lang.Character.Blocks15 > uncaught exception: > java.lang.OutOfMemoryError > FAIL: java.lang.InheritableThreadLocal.simple > Test timed out. Use -timeout [millis] option to change the timeout value. > FAIL: java.text.SimpleDateFormat.parse > line 132: parse pattern -MM-dd HH:mm z [3] -- Objects were not equal. > Use -debug for more information. > line 132: parse pattern -MM-dd HH:mm Z [3] -- Objects were not equal. > Use -debug for more information. > FAIL: java.util.Calendar.ampm > line 72: 12:00 AM couldn't be parsed [1] -- forced fail > line 72: 12:10 AM couldn't be parsed [1] -- forced fail > line 57: 12:10 AM couldn't be parsed [2] -- Objects were not equal. Use > -debug for more information. > line 72: 0:00 AM couldn't be parsed [1] -- forced fail > line 57: 0:00 AM couldn't be parsed [2] -- Objects were not equal. Use > -debug for more information. > line 72: 0:10 AM couldn't be parsed [1] -- forced fail > line 72: 12:00 PM couldn't be parsed [1] -- forced fail > line 72: 12:10 PM couldn't be parsed [1] -- forced fail > line 57: 12:10 PM couldn't be parsed [2] -- Objects were not equal. Use > -debug for more information. > line 72: 0:00 PM couldn't be parsed [1] -- forced fail > line 57: 0:00 PM couldn't be parsed [2] -- Objects were not equal. Use > -debug for more information. > line 72: 0:10 PM couldn't be parsed [1] -- forced fail > FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert > line 746: EmptyStackException must be thrown [1] -- forced fail > line 759: EmptyStackException must be thrown [3] -- got 4 but expected 3 > line 765: EmptyStackException must be thrown [7] -- got 7 but expected 15 > line 767: EmptyStackException must be thrown [8] -- got 12 but expected 15 > line 768: EmptyStackException must be thrown [9] -- got 15 but expected 16 > line 771: EmptyStackException must be thrown [10] -- got > javax.swing.text.AbstractDocument$DefaultDocumentEvent but ex\ > pected null > FAIL: > javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground > line 54: [2] -- got javax.swing.plaf.ColorUIResource but expected > java.awt.Color > > The first is odd, and CACAO+Classpath CVS fails this also. OpenJDK can't > compile it. CACAO + Classpath CVS > also fails with java.text.SimpleDateFormat.parse, java.util.Calendar.ampm and > javax.swing.text.DefaultStyledDocument.ElementBuffer.insert, which suggest a > common Classpath issue. > java.sql.Date.DateTest, java.lang.Character.Blocks15, > java.lang.InheritableThreadLocal.simple and > javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground > are > all GCJ-specific failures. OpenJDK passes all but the first, which is GNU > Classpath specific. > > Full logs are available at: > > http://fuseyism.com/classpath/mauve-gcj-4.3.log > http://fuseyism.com/classpath/mauve-gcj-4.4.log > -- > Andrew :) > > Support Free Java! > Contribute to GNU Classpath and the OpenJDK > http://www.gnu.org/software/classpath > http://openjdk.java.net > PGP Key: 94EFD9D8 (http://subkeys.pgp.net) > Fingerprint = F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 > Changed to: http://fuseyism.com/classpath/mauve-gcj-4.3.log.gz http://fuseyism.com/classpath/mauve-gcj-4.4.log.gz due to size... -- Andrew :-) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
GCJ Mauve testing
I've just run Mauve against both the 4.3 tree and our Classpath 0.98 merge tree (effectively equivalent to Classpath CVS + GCJ-specific changes). Overall: 4.3: 281 of 3177 tests failed. 123926 total calls to harness.check() failed. 4.4: 239 of 3177 tests failed. 143832 total calls to harness.check() failed. There will be some new tests that compile with 4.4 but not 4.3 (e.g. java.util.Scanner tests). Additional FAILs are as follows: +FAIL: gnu.java.security.util.TestOfIntegerUtil +FAIL: java.sql.Date.DateTest +FAIL: java.lang.Character.Blocks15 +FAIL: java.lang.InheritableThreadLocal.simple +FAIL: java.text.SimpleDateFormat.parse +FAIL: java.util.Calendar.ampm +FAIL: java.util.Scanner.LotsOfPMInts +FAIL: java.util.Scanner.Inputs +FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert +FAIL: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground The two Scanner tests we can discount as regressions as they simply won't have run with 4.3. The remaining failures: FAIL: gnu.java.security.util.TestOfIntegerUtil line 71: IntegerUtil.valueOf(String) MUST be faster than Integer.valueOf(String) [1] -- boolean passed to check was fa\ lse FAIL: java.sql.Date.DateTest uncaught exception: java.lang.OutOfMemoryError FAIL: java.lang.Character.Blocks15 uncaught exception: java.lang.OutOfMemoryError FAIL: java.lang.InheritableThreadLocal.simple Test timed out. Use -timeout [millis] option to change the timeout value. FAIL: java.text.SimpleDateFormat.parse line 132: parse pattern -MM-dd HH:mm z [3] -- Objects were not equal. Use -debug for more information. line 132: parse pattern -MM-dd HH:mm Z [3] -- Objects were not equal. Use -debug for more information. FAIL: java.util.Calendar.ampm line 72: 12:00 AM couldn't be parsed [1] -- forced fail line 72: 12:10 AM couldn't be parsed [1] -- forced fail line 57: 12:10 AM couldn't be parsed [2] -- Objects were not equal. Use -debug for more information. line 72: 0:00 AM couldn't be parsed [1] -- forced fail line 57: 0:00 AM couldn't be parsed [2] -- Objects were not equal. Use -debug for more information. line 72: 0:10 AM couldn't be parsed [1] -- forced fail line 72: 12:00 PM couldn't be parsed [1] -- forced fail line 72: 12:10 PM couldn't be parsed [1] -- forced fail line 57: 12:10 PM couldn't be parsed [2] -- Objects were not equal. Use -debug for more information. line 72: 0:00 PM couldn't be parsed [1] -- forced fail line 57: 0:00 PM couldn't be parsed [2] -- Objects were not equal. Use -debug for more information. line 72: 0:10 PM couldn't be parsed [1] -- forced fail FAIL: javax.swing.text.DefaultStyledDocument.ElementBuffer.insert line 746: EmptyStackException must be thrown [1] -- forced fail line 759: EmptyStackException must be thrown [3] -- got 4 but expected 3 line 765: EmptyStackException must be thrown [7] -- got 7 but expected 15 line 767: EmptyStackException must be thrown [8] -- got 12 but expected 15 line 768: EmptyStackException must be thrown [9] -- got 15 but expected 16 line 771: EmptyStackException must be thrown [10] -- got javax.swing.text.AbstractDocument$DefaultDocumentEvent but ex\ pected null FAIL: javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground line 54: [2] -- got javax.swing.plaf.ColorUIResource but expected java.awt.Color The first is odd, and CACAO+Classpath CVS fails this also. OpenJDK can't compile it. CACAO + Classpath CVS also fails with java.text.SimpleDateFormat.parse, java.util.Calendar.ampm and javax.swing.text.DefaultStyledDocument.ElementBuffer.insert, which suggest a common Classpath issue. java.sql.Date.DateTest, java.lang.Character.Blocks15, java.lang.InheritableThreadLocal.simple and javax.swing.table.JTableHeader.AccessibleJTableHeader.AccessibleJTableHeaderEntry.getForeground are all GCJ-specific failures. OpenJDK passes all but the first, which is GNU Classpath specific. Full logs are available at: http://fuseyism.com/classpath/mauve-gcj-4.3.log http://fuseyism.com/classpath/mauve-gcj-4.4.log -- Andrew :) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint = F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
