Re: mauve comparison with tgolem
Hi Christian, On Tue, 2005-11-29 at 21:20 +0100, Christian Thalinger wrote: On Tue, 2005-11-29 at 15:04 +0100, Mark Wielaard wrote: * Changed implementation of VMClassLoader.getPackage(s) : new method VMClassLoader.getBootPackages should be implemented by the vm, and sould return a string array of boot package names (java.lang, java.net, ...). Feedback from vm implementors for usability and relevance of the getBootPackages method would be greatly appreciated. Why isn't there a default implementation which returns the GNU classpath boot package names? For CACAO, there are not others. Probably because it was thought that it was easier for the runtime to provide this list. Since GNU Classpath doesn't come with a default bootstrap class loader (they are mostly different between runtimes) there is no easy hook to collect them all. We could of course have a little script that collects them all during configure/build time and then makes them available somewhere in some generated source code. Can you write something like that? Cheers, Mark ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
On Tue, 2005-11-29 at 15:04 +0100, Mark Wielaard wrote: FAIL: gnu.testlet.java.lang.Package.getPackage: checking package for 'java.lang' (number 1) Needs runtime support. See NEWS file: * Changed implementation of VMClassLoader.getPackage(s) : new method VMClassLoader.getBootPackages should be implemented by the vm, and sould return a string array of boot package names (java.lang, java.net, ...). Feedback from vm implementors for usability and relevance of the getBootPackages method would be greatly appreciated. Why isn't there a default implementation which returns the GNU classpath boot package names? For CACAO, there are not others. TWISTI ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
Hi Edwin, On Mon, 2005-11-28 at 14:54 +0100, Edwin Steiner wrote: On Sat, Nov 26, 2005 at 02:01:53AM +0100, Mark Wielaard wrote: If it is in addition to what there is now already just the plain FAIL lines would be ideal to have. No extras, just the facts! :) You can now get a text-only list of common mauve fails at: http://www.complang.tuwien.ac.at/cacaojvm//tgolem/latest/mauve_common_fails.txt Very interesting list. Thanks. Quick analysis for those test that I know why they fail. Maybe others can take a look at the failures and quickly see whether there is a simple explanation or not for these failures. FAIL: gnu.testlet.java.io.File.newFile: getname returns the argument (number 3) Strange... I highly suspect that PlatformHelper is to blame since it tries to bridge any windows vs posix style file path issues. In practice is seems to fail miserably however... FAIL: gnu.testlet.java.lang.Character.classify12 (number 1) FAIL: gnu.testlet.java.lang.Character.getType (number 11) FAIL: gnu.testlet.java.lang.Character.getType (number 20) FAIL: gnu.testlet.java.lang.Character.getType (number 22) These fail because GNU Classpath provides Character based on a newer version of Unicode then what is being tested in Mauve. FAIL: gnu.testlet.java.lang.Package.getPackage: checking package for 'java.lang' (number 1) Needs runtime support. See NEWS file: * Changed implementation of VMClassLoader.getPackage(s) : new method VMClassLoader.getBootPackages should be implemented by the vm, and sould return a string array of boot package names (java.lang, java.net, ...). Feedback from vm implementors for usability and relevance of the getBootPackages method would be greatly appreciated. FAIL: gnu.testlet.java.nio.channels.FileChannel.map (number 1) Seems like a real bug, but fixing it (throwing the right exception) seems to trigger another bug. Needs investigation. FAIL: uncaught exception loading gnu.testlet.java.lang.Class.pkg.test1: java.lang.IllegalAccessException: gnu.testlet.java.lang.Class.pkg.test1 has an inaccessible constructor FAIL: uncaught exception loading gnu.testlet.java.lang.Class.pkg.test3: java.lang.IllegalAccessException: gnu.testlet.java.lang.Class.pkg.test3 has an inaccessible constructor FAIL: uncaught exception loading gnu.testlet.java.lang.Class.pkg.test4: java.lang.IllegalAccessException: gnu.testlet.java.lang.Class.pkg.test4 has an inaccessible constructor These are not tests and should be marked as such. I'll submit a patch. FAIL: uncaught exception loading gnu.testlet.java.lang.Number.NewNumber: java.lang.IllegalAccessException: gnu.testlet.java.lang.Number.NewNumber has an inaccessible constructor FAIL: uncaught exception loading gnu.testlet.java.lang.reflect.Other: java.lang.IllegalAccessException: gnu.testlet.java.lang.reflect.Other has an inaccessible constructor Likewise. Hopefully we can analyze the others also in the near future. Comments and suggestions appreciated. Cheers, Mark ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
On Sat, Nov 26, 2005 at 02:01:53AM +0100, Mark Wielaard wrote: If it is in addition to what there is now already just the plain FAIL lines would be ideal to have. No extras, just the facts! :) You can now get a text-only list of common mauve fails at: http://www.complang.tuwien.ac.at/cacaojvm//tgolem/latest/mauve_common_fails.txt (It's sorted alphabetically. Could also do smarter sorting, if necessary.) If you just want to look at it, these also work: http://www.cacaojvm.org/tgolem http://www.cacaojvm.org/tgolem/latest/mauve_common_fails.txt But there is some strange frame setup in place for these URLs. I hope this meets your expectations, otherwise just yell ;) -Edwin ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
Hi Cacao Team, On Tue, 2005-11-22 at 12:33 +0100, Christian Thalinger wrote: http://www.cacaojvm.org/tgolem The mauve report is the comparison table. The common column shows all PASSes and FAILs which are identical on all JVMs/architectures. Each machine is listed with PASS, FAIL and MISS. MISSed mostly can have two reasons: the testlet crashed and the remaining test were not run or the testlet has been removed from mauve cvs. Maybe this helps someone. This is definitely very useful! The common failures would be good to investigate more closely by all GNU Classpath hackers. And possibly write bug reports (or better patches for fixes) for. In principle the mauve framework can support xfail files, although I don't believe anybody used that for a long time. When all common failures are inspected/identified correctly we could make a good xfails file so it is more clear what issues are regressions and what issues are runtime specific. Would it be possible to split out the mauve results pages? Or at least have a page with just a list of all common FAILS? Any idea about the GUI (Free Swing/AWT) tests? Is there a specific reason they are not enabled at the moment? Thanks, Mark ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
On Fri, Nov 25, 2005 at 07:35:46PM +0100, Mark Wielaard wrote: [snip] In principle the mauve framework can support xfail files, although I don't believe anybody used that for a long time. When all common failures are inspected/identified correctly we could make a good xfails file so it is more clear what issues are regressions and what issues are runtime specific. That would be very useful. It would integrate well with the regression testing framework I'm planning for tgolem. Would it be possible to split out the mauve results pages? Or at least have a page with just a list of all common FAILS? Absolutely possible. Are there any specific details I should include in such a page? Any idea about the GUI (Free Swing/AWT) tests? Is there a specific reason they are not enabled at the moment? Yes. All our machines are headless, so there is no X display. We are pondering to use a virtual X server, but we don't have a real solution right now. Any ideas would be appreciated. -Edwin ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
Edwin == Edwin Steiner [EMAIL PROTECTED] writes: Edwin Yes. All our machines are headless, so there is no X display. We are Edwin pondering to use a virtual X server, but we don't have a real Edwin solution right now. Any ideas would be appreciated. Take a look at the batch_run script in Mauve. It already has all the code needed to run the tests against the Xvfb server. Tom ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
On Fri, Nov 25, 2005 at 03:42:25PM -0700, Tom Tromey wrote: Edwin == Edwin Steiner [EMAIL PROTECTED] writes: Edwin Yes. All our machines are headless, so there is no X display. We are Edwin pondering to use a virtual X server, but we don't have a real Edwin solution right now. Any ideas would be appreciated. Take a look at the batch_run script in Mauve. It already has all the code needed to run the tests against the Xvfb server. Thanks, looks good. Xvnc might also be an option. Now we just have to get our sysadmin to install one or the other and X-dev packages on each machine. Some convincing to do... :) -Edwin ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: mauve comparison with tgolem
Hi Edwin, On Fri, 2005-11-25 at 23:42 +0100, Edwin Steiner wrote: Would it be possible to split out the mauve results pages? Or at least have a page with just a list of all common FAILS? Absolutely possible. Are there any specific details I should include in such a page? If it is in addition to what there is now already just the plain FAIL lines would be ideal to have. No extras, just the facts! :) Thanks, Mark ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
mauve comparison with tgolem
Hi! One of our CACAO developers, edwin, is currently developing a test framework called tgolem. It was initially intended for testing CACAO against different stuff, like mauve or tomcat. It generates output for the different architectures we run the tests on and a comparison table for all architectures the CACAO JIT supports and a JamVM run on an i386 machine. This comparison against JamVM ist _not_ a competition, but it's just to see which mauve tests fail from the GNU classpath side and not CACAO. The test runs are made with CVS head versions of all involved software. The link to these pages is: http://www.cacaojvm.org/tgolem The mauve report is the comparison table. The common column shows all PASSes and FAILs which are identical on all JVMs/architectures. Each machine is listed with PASS, FAIL and MISS. MISSed mostly can have two reasons: the testlet crashed and the remaining test were not run or the testlet has been removed from mauve cvs. Maybe this helps someone. TWISTI ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
