Really interesting results! :) It seems that japitools are not used for certification :) As far as I remember TCK does not check for signature compatibility. It's just a number (huge number) of tests...
SY, Alexey 2006/11/5, Stefano Mazzocchi <[EMAIL PROTECTED]>:
Being a sucker for statistics and charts, I've decided to look into japitools myself and regenerate the graph of API coverage progress of harmony. In doing so, I started to familiarize myself with the Japitools and I found a few interesting discoveries: the latest version of the three freely available certified java 5 implementations (Sun's, BEA's and IBM's) are *NOT* 100% compatible. So, here's what I did: 1) download the three JVMs 2) download japitools, "tar xzf" it and "cd japitools" 3) type: ./bin/japize as $name packages \ /path/to/jvms/$name/jre/lib/*.jar \ +java +javax +org -org.apache -org.ietf and substitute $name with the JVM name 4) you'll obtain three $name.japi.gz files (the binary representation of your API footprint) 5) then type "japicompat -s $original.japi.gz $test.japi.gz" where "original" is the JVM that you consider the reference and $test is the one that you want to test. Here are the (a little surprising, I must say) results: -- sun 1.5 vs. bea1.5 --------------------------------------- 99.99% good, 0% missing java.awt.peer: method java.awt.peer.WindowPeer.updateFocusableWindowState(): missing in /home/stefano/data/japi/ibm1.5 -- sun 1.5 vs. ibm1.5 --------------------------------------- Total: 99.93% good, 0% bad, 0.06% missing java.awt.peer: Missing method java.awt.peer.WindowPeer.updateFocusableWindowState(): missing in /home/stefano/data/japi/ibm1.5 javax.xml.datatype: Bad field javax.xml.datatype.DatatypeFactory.DATATYPEFACTORY_IMPLEMENTATION_CLASS: constant [com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/sun1.5, but constant [org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/ibm1.5 org.omg.stub.javax.management.remote.rmi: Missing class org.omg.stub.javax.management.remote.rmi._RMIConnectionImpl_Tie: missing in /home/stefano/data/japi/ibm1.5 class org.omg.stub.javax.management.remote.rmi._RMIServerImpl_Tie: missing in /home/stefano/data/japi/ibm1.5 org.w3c.dom.html: Missing method org.w3c.dom.html.HTMLFrameElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLIFrameElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLObjectElement.getContentDocument(): missing in /home/stefano/data/japi/ibm1.5 method org.w3c.dom.html.HTMLOptionElement.setSelected(boolean): missing in /home/stefano/data/japi/ibm1.5 - o - There is one method that both Bea and IBM don't implement in awt.peer.WindowPeer and apparently, Sun doesn't implement it either.. it's just a stub! Weird. [see more at http://forums.java.net/jive/thread.jspa?messageID=167137] the differences in datatype factory is plausible and the fact that a stub RMI class is missing is not that big of a deal. It's weird though, that the DOM in IBM's is different than the DOM in Sun's... I guess not that many people use the HTML DOM in java or they would have got that ;-) The really crazy things start to happen if you flip things around and you consider the 'clean room rewrites' as the reference implementations: -- bea1.5 vs. sun1.5 -------------------------------------------- Total: 99.98% good, 0.01% missing javax.xml.namespace: Missing class javax.xml.namespace.QName: missing in /home/stefano/data/japi/sun1.5 Uh? Sun forgot to ship the QName class or this is a japitools bug? googling up shows the class in the java1.5 docs so it's more likely it's a bug in japitools http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/namespace/QName.html Gets more interesting with IBM: -- ibm1.5 vs. sun1.5 -------------------------------------------- Total: 99.77% good, 0% bad, 0.22% missing java.lang: Missing method java.lang.StringBuilder.append(java.lang.StringBuilder): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.capacity(): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.charAt(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.codePointAt(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.codePointBefore(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.codePointCount(int, int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.ensureCapacity(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.getChars(int, int, char[], int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.length(): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.offsetByCodePoints(int, int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.setCharAt(int, char): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.setLength(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.subSequence(int, int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.substring(int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.substring(int, int): missing in /home/stefano/data/japi/sun1.5 method java.lang.StringBuilder.trimToSize(): missing in /home/stefano/data/japi/sun1.5 javax.management.remote.rmi: Missing class javax.management.remote.rmi._RMIConnectionImpl_Tie: missing in /home/stefano/data/japi/sun1.5 class javax.management.remote.rmi._RMIServerImpl_Tie: missing in /home/stefano/data/japi/sun1.5 javax.xml.datatype: Bad field javax.xml.datatype.DatatypeFactory.DATATYPEFACTORY_IMPLEMENTATION_CLASS: constant [org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/ibm1.5, but constant [com.sun.org.apache.xerces.internal.jaxp.datatype.DatatypeFactoryImpl] in /home/stefano/data/japi/sun1.5 org.omg.stub.java.lang: Missing package org.omg.stub.java.lang: missing in /home/stefano/data/japi/sun1.5 org.omg.stub.java.security: Missing package org.omg.stub.java.security: missing in /home/stefano/data/japi/sun1.5 org.omg.stub.java.util: Missing package org.omg.stub.java.util: missing in /home/stefano/data/japi/sun1.5 org.w3c.dom.html: Missing method org.w3c.dom.html.HTMLOptionElement.setIndex(int): missing in /home/stefano/data/japi/sun1.5 method org.w3c.dom.html.HTMLTableCellElement.setCellIndex(int): missing in /home/stefano/data/japi/sun1.5 method org.w3c.dom.html.HTMLTableRowElement.setCells(org.w3c.dom.html.HTMLCollection): missing in /home/stefano/data/japi/sun1.5 method org.w3c.dom.html.HTMLTableRowElement.setRowIndex(int): missing in /home/stefano/data/japi/sun1.5 method org.w3c.dom.html.HTMLTableRowElement.setSectionRowIndex(int): missing in /home/stefano/data/japi/sun1.5 org.w3c.dom.xpath: Missing package org.w3c.dom.xpath: missing in /home/stefano/data/japi/sun1.5 - o - First the not-so-shocking things: 1) IBM ships with xpath support with DOM, while Sun does not. Is this true of it's a bug? 2) IBM has a 'stub' subpackage of org.omg that Sun does not have but the real shocker is that japitools reports that IBM has *modified* the java.lang.StringBuilder class and added *15* new methods to it! If this is true, how is it possible for this JVM to pass the TCK if you add methods to a java.lang class? Isn't this the kind of stuff that TCKs are supposed to prevent? Can anybody explain to me if this is true or if this is a japi bug? Also, I found out that Japitools triggers some NullPointerExceptions here and there, so it needs some tuning. I'll investigate more. Stay tuned. -- Stefano.