Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
Looking through the mail thread: [classlib] JAPI data to drive packages to completion and following the link I found that looks like JAPI somehow supports part of 1.5 features (at least generics). It would be interesting to know for sure which of 1.5 features JAPI supports and which not, to understand what JAPI lacks (or be sure it has everything) to test 1.5 source compatibility of HY vs. conformant APIs. Mark, could your please point me to the document (if know any) that describes what of 1.5 features JAPI supports or provide a link to the tool's binaries to try. Thanks for your help, Vladimir On 5/15/06, Vladimir Ivanov [EMAIL PROTECTED] wrote: Thanks Leo for your input. It forces me to think about some aspects of compatibility again. On 5/6/06, Leo Simons [EMAIL PROTECTED] wrote: On Sat, May 06, 2006 at 12:33:52PM +0700, Vladimir Ivanov wrote: Recently I thought about guaranteeing binary and source compatibility between HARMONY API and other compatible J2SE API implementations, what is our goal and how to check it, automation. Let me share my thoughts - for us to understand clearly what we want and how to test it. Thanks, vladimir, very clear! === Some observations === Observation #1: I think, in general, binary compatibility is a weaker requirement then source compatibility and is completely covered by source compatibility. Hmm. For the general form of general, this is not true, which stems from the use of preprocessors and non-deterministic transformations with which the non-java world is full. Eg when you do C development and change the definition of how big an int is, you lose binary compatibility but preserve source compatibility if this definition is inherited. In the specific, you might very well be very right here, which is quite interesting...how'd you come to these observations? Can I read more about them elsewhere? This observation based on another observation that compiler does linking (name resolution) in the same way as runtime (seems, for your example with C language it will not break linking, i.e. binary compatibility in terms of JLS). Of cause, if the compiler does not start linker or rules for compilation and runtime are different it will not work, but I know only some primitive assemblers that violate these rules. Sorry, I can't refer to articles just because I don't know any one related to source compatibility 'in general'. My observation based on the experience only. Observation #2: I think, talking about 1.4, checking of 2-way binary compatibility + throws clause + inheritance hierarchy will guarantee 2-way source compatibility. I did not find any contra examples. + serialized form I can imagine naughty/hacky code that uses reflection would be able to violate that rule too. The AOP toolkits are a good example of pushing the limits, eg aspectwerkz @ codehaus.org. The JVMS defines for java runtime that linking includes verification, preparation and resolution. Conformat compiler generates valid code. The preparation phase is memory allocation + check for AbstractMethodError. The resolution phase include checks for IllegalAccessError, InstantiationError, NoSuchFieldError and NoSuchMethodError. But compiler does all these 5 checks so source compatibility include binary (for java). Correct serialized form is not required for source/ binary compatibility (it is not affect linking/ compilation), so harmony target may be extended to 2-way-source compatibility and 2-way-serialized form compatibility. As for reflection, seems, the linker does the same checks as compiler (elements are mirrored to the wrappers like java.lang.reflect.Method and both checks types of wrappers only). It will be very useful If your provide code example to think how it can be eliminated. === What is our (Harmony) goal? === In terms of these definitions, ideally, I suppose we want that Harmony is 2-way source compatible with the conformant J2SE API implementation (RI API) to make sure that any application compiled with RI API can be compiled with Harmony and vice versa yep, 2-way-source compatible and 2-way-binary compatible. Agree. 2-way-source compatible and 2-way-serialized form compatible === Questions === 2. What more checks should be added to JAPI to guarantee 2-way source compatibility for 1.5? You know, I can't even think of a good way to do implement the checks for the generics, let alone think of more! Seems, it can be done for case with generic because all needed information stored to the class file. For example, additional checks for source compatibility may be implemented to convert information from the class file 'Signature' attribute to the textual representation of method's signature (with parameters rename for generic names). If we all agree that our target is 2-way source compatible and 2-way-serialized form compatible it would be good to define the complete list of
Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
Thanks Leo for your input. It forces me to think about some aspects of compatibility again. On 5/6/06, Leo Simons [EMAIL PROTECTED] wrote: On Sat, May 06, 2006 at 12:33:52PM +0700, Vladimir Ivanov wrote: Recently I thought about guaranteeing binary and source compatibility between HARMONY API and other compatible J2SE API implementations, what is our goal and how to check it, automation. Let me share my thoughts - for us to understand clearly what we want and how to test it. Thanks, vladimir, very clear! === Some observations === Observation #1: I think, in general, binary compatibility is a weaker requirement then source compatibility and is completely covered by source compatibility. Hmm. For the general form of general, this is not true, which stems from the use of preprocessors and non-deterministic transformations with which the non-java world is full. Eg when you do C development and change the definition of how big an int is, you lose binary compatibility but preserve source compatibility if this definition is inherited. In the specific, you might very well be very right here, which is quite interesting...how'd you come to these observations? Can I read more about them elsewhere? This observation based on another observation that compiler does linking (name resolution) in the same way as runtime (seems, for your example with C language it will not break linking, i.e. binary compatibility in terms of JLS). Of cause, if the compiler does not start linker or rules for compilation and runtime are different it will not work, but I know only some primitive assemblers that violate these rules. Sorry, I can't refer to articles just because I don't know any one related to source compatibility 'in general'. My observation based on the experience only. Observation #2: I think, talking about 1.4, checking of 2-way binary compatibility + throws clause + inheritance hierarchy will guarantee 2-way source compatibility. I did not find any contra examples. + serialized form I can imagine naughty/hacky code that uses reflection would be able to violate that rule too. The AOP toolkits are a good example of pushing the limits, eg aspectwerkz @ codehaus.org. The JVMS defines for java runtime that linking includes verification, preparation and resolution. Conformat compiler generates valid code. The preparation phase is memory allocation + check for AbstractMethodError. The resolution phase include checks for IllegalAccessError, InstantiationError, NoSuchFieldError and NoSuchMethodError. But compiler does all these 5 checks so source compatibility include binary (for java). Correct serialized form is not required for source/ binary compatibility (it is not affect linking/ compilation), so harmony target may be extended to 2-way-source compatibility and 2-way-serialized form compatibility. As for reflection, seems, the linker does the same checks as compiler (elements are mirrored to the wrappers like java.lang.reflect.Method and both checks types of wrappers only). It will be very useful If your provide code example to think how it can be eliminated. === What is our (Harmony) goal? === In terms of these definitions, ideally, I suppose we want that Harmony is 2-way source compatible with the conformant J2SE API implementation (RI API) to make sure that any application compiled with RI API can be compiled with Harmony and vice versa yep, 2-way-source compatible and 2-way-binary compatible. Agree. 2-way-source compatible and 2-way-serialized form compatible === Questions === 2. What more checks should be added to JAPI to guarantee 2-way source compatibility for 1.5? You know, I can't even think of a good way to do implement the checks for the generics, let alone think of more! Seems, it can be done for case with generic because all needed information stored to the class file. For example, additional checks for source compatibility may be implemented to convert information from the class file 'Signature' attribute to the textual representation of method's signature (with parameters rename for generic names). If we all agree that our target is 2-way source compatible and 2-way-serialized form compatible it would be good to define the complete list of checks and, for example, extend JAPI by implementing all these checks to make complete 2 way source compatibility (with RI) testing tool. Thanks, Vladimir Ivanov cheers! Leo - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
Vladimir,
- Am I right that source/binary compatibility implies that
package-private implementation also matches? If it does then how does
it fit to clean-room?
- Do we really need to be source/binary compatible? Do you think that
existing compatibility guideline
http://incubator.apache.org/harmony/subcomponents/classlibrary/compat.html
is not enough?
Sorry if it was discussed before and I missed it.
--
Anton Avtamonov,
Intel Middleware Products Division
On 5/6/06, Vladimir Ivanov [EMAIL PROTECTED] wrote:
Recently I thought about guaranteeing binary and source compatibility
between HARMONY API and other compatible J2SE API implementations, what is
our goal and how to check it, automation. Let me share my thoughts - for us
to understand clearly what we want and how to test it.
Here is the summary:
· For 1.4 J2SE API implementation - JAPI tool, being applied in both
ways (RI vs. HARMONY and HARMONY vs. RI), tests 2-way binary compatibility
between two J2SE API implementations and only one small update to check the
inheritance class hierarchy should be added to tests 2-way source
compatibility.
· For 1.5 – JAPI tests 2-way binary compatibility. For checking 2-way
source compatibility JAPI lacks some 1.5 language specific checks in
addition to checking inheritance class hierarchy. We need some add-ons for
JAPI.
Details:
First, definitions…
Binary compatibility – if class A can be linked with a set of classes B and
a set of classes C than sets B and C are binary compatible (from the A's
point of view). For Java it is defined in the JLS chapter 13.
Source compatibility – if a source of class A can be compiled with a set of
classes B and a set of classes C (by conformant compiler) than sets B and C
are source compatible.
1-way binary/source compatibility – any arbitrary class A, linked/compiled
with a set of classes B, can be linked/compiled with a set of classes C.
2-way binary/source compatibility – any arbitrary class A, linked/compiled
with a set of classes B, can be linked/compiled with a set of classes C AND
vice versa – any arbitrary class A, linked/compiled with a set of classes C,
can be linked/compiled with a set of classes B.
Some observations
Observation #1: I think, in general, binary compatibility is a weaker
requirement then source compatibility and is completely covered by source
compatibility.
Observation #2: I think, talking about 1.4, checking of 2-way binary
compatibility + throws clause + inheritance hierarchy will guarantee 2-way
source compatibility. I did not find any contra examples.
Observation #3: In 1.5, however, checking of 2-way binary compatibility+
throws clause + inheritance hierarchy does not guarantee 2-way source
compatibility. Why? - generics, enums, annotations.
Example for generics:
class AT {
public void m(T p) {
}
}
class A {
public void m(Object p) {
}
}
class MyClass {
public static void main() {
new AString().m(abc);
}
}
Both As are 2-way binary compatible, but, MyClass can not be compiled with
non-parameterized A.
What is our (Harmony) goal?
In terms of these definitions, ideally, I suppose we want that Harmony is
2-way source compatible with the conformant J2SE API implementation (RI API)
to make sure that any application compiled with RI API can be compiled with
Harmony and vice versa (as well as linked, that is guaranteed by the
Observation #1).
* Automation: what does JAPI check? (http://www.kaffe.org/~stuart/japi/)
Being run in two ways (for example, RI vs Harmony and Harmony vs RI) JAPI
checks 2-way binary compatibility for both 1.4 and 1.5 language-written
APIs.
It compares method, class signatures in two sets of API binaries and, being
run in two ways, checks the following:
---
Checks | JAPI
---
Classes: modifiers (access+abstract) | +
Classes: add/remove super classes| +/+
Classes: add/remove super interfaces | +/+
Classes: order of super class hierarchy | -
Fields: modifiers (access+final) | +
Fields: change value for constant| +
Fields: change static/transient | +/-
Fields: add/remove | +/+
Methods: modifiers(access+abstract+final)| +
Methods: change static/throws clause | +/+
Methods: add/remove | +/+
Constructors: modifiers(access) | +
Constructors: add/remove | +/+
Interfaces: modifiers| +
Interfaces: add fields/methods | +/+
Interfaces: remove fields/methods| +/+
---
So, talking about 1.4, two runs of JAPI - Harmony vs RI and RI vs Harmony
and small JAPI update (check that it is illegal to re-order class
inheritance, actually, it is checked, but indirectly) will check 2-way
source
Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
On 5/6/06, Anton Avtamonov [EMAIL PROTECTED] wrote:
Vladimir,
- Am I right that source/binary compatibility implies that
package-private implementation also matches?
If it does then how does it fit to clean-room?
No.
- Do we really need to be source/binary compatible?
Do you think that existing compatibility guideline
http://incubator.apache.org/harmony/subcomponents/classlibrary/compat.html
is not enough?
Seems, it is enough. But letter was about how to test it. We have not such
kind of tests.
But if we will use JAPI with some add-ons, for example, we can check '*Comply
with the Java Specification*' automatically.
Thanks,
Vladimir Ivanov
Sorry if it was discussed before and I missed it.
--
Anton Avtamonov,
Intel Middleware Products Division
On 5/6/06, Vladimir Ivanov [EMAIL PROTECTED] wrote:
Recently I thought about guaranteeing binary and source compatibility
between HARMONY API and other compatible J2SE API implementations, what
is
our goal and how to check it, automation. Let me share my thoughts - for
us
to understand clearly what we want and how to test it.
Here is the summary:
· For 1.4 J2SE API implementation - JAPI tool, being applied in both
ways (RI vs. HARMONY and HARMONY vs. RI), tests 2-way binary
compatibility
between two J2SE API implementations and only one small update to check
the
inheritance class hierarchy should be added to tests 2-way source
compatibility.
· For 1.5 – JAPI tests 2-way binary compatibility. For checking 2-way
source compatibility JAPI lacks some 1.5 language specific checks in
addition to checking inheritance class hierarchy. We need some add-ons
for
JAPI.
Details:
First, definitions…
Binary compatibility – if class A can be linked with a set of classes B
and
a set of classes C than sets B and C are binary compatible (from the A's
point of view). For Java it is defined in the JLS chapter 13.
Source compatibility – if a source of class A can be compiled with a set
of
classes B and a set of classes C (by conformant compiler) than sets B
and C
are source compatible.
1-way binary/source compatibility – any arbitrary class A,
linked/compiled
with a set of classes B, can be linked/compiled with a set of classes C.
2-way binary/source compatibility – any arbitrary class A,
linked/compiled
with a set of classes B, can be linked/compiled with a set of classes C
AND
vice versa – any arbitrary class A, linked/compiled with a set of
classes C,
can be linked/compiled with a set of classes B.
Some observations
Observation #1: I think, in general, binary compatibility is a weaker
requirement then source compatibility and is completely covered by
source
compatibility.
Observation #2: I think, talking about 1.4, checking of 2-way binary
compatibility + throws clause + inheritance hierarchy will guarantee
2-way
source compatibility. I did not find any contra examples.
Observation #3: In 1.5, however, checking of 2-way binary compatibility+
throws clause + inheritance hierarchy does not guarantee 2-way source
compatibility. Why? - generics, enums, annotations.
Example for generics:
class AT {
public void m(T p) {
}
}
class A {
public void m(Object p) {
}
}
class MyClass {
public static void main() {
new AString().m(abc);
}
}
Both As are 2-way binary compatible, but, MyClass can not be compiled
with
non-parameterized A.
What is our (Harmony) goal?
In terms of these definitions, ideally, I suppose we want that Harmony
is
2-way source compatible with the conformant J2SE API implementation (RI
API)
to make sure that any application compiled with RI API can be compiled
with
Harmony and vice versa (as well as linked, that is guaranteed by the
Observation #1).
* Automation: what does JAPI check? (
http://www.kaffe.org/~stuart/japi/)
Being run in two ways (for example, RI vs Harmony and Harmony vs RI)
JAPI
checks 2-way binary compatibility for both 1.4 and 1.5 language-written
APIs.
It compares method, class signatures in two sets of API binaries and,
being
run in two ways, checks the following:
---
Checks | JAPI
---
Classes: modifiers (access+abstract) | +
Classes: add/remove super classes| +/+
Classes: add/remove super interfaces | +/+
Classes: order of super class hierarchy | -
Fields: modifiers (access+final) | +
Fields: change value for constant| +
Fields: change static/transient | +/-
Fields: add/remove | +/+
Methods: modifiers(access+abstract+final)| +
Methods: change static/throws clause | +/+
Methods: add/remove | +/+
Constructors: modifiers(access) | +
Constructors: add/remove | +/+
Interfaces: modifiers
Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
On Sat, May 06, 2006 at 12:33:52PM +0700, Vladimir Ivanov wrote: Recently I thought about guaranteeing binary and source compatibility between HARMONY API and other compatible J2SE API implementations, what is our goal and how to check it, automation. Let me share my thoughts - for us to understand clearly what we want and how to test it. Thanks, vladimir, very clear! === Some observations === Observation #1: I think, in general, binary compatibility is a weaker requirement then source compatibility and is completely covered by source compatibility. Hmm. For the general form of general, this is not true, which stems from the use of preprocessors and non-deterministic transformations with which the non-java world is full. Eg when you do C development and change the definition of how big an int is, you lose binary compatibility but preserve source compatibility if this definition is inherited. In the specific, you might very well be very right here, which is quite interesting...how'd you come to these observations? Can I read more about them elsewhere? Observation #2: I think, talking about 1.4, checking of 2-way binary compatibility + throws clause + inheritance hierarchy will guarantee 2-way source compatibility. I did not find any contra examples. + serialized form I can imagine naughty/hacky code that uses reflection would be able to violate that rule too. The AOP toolkits are a good example of pushing the limits, eg aspectwerkz @ codehaus.org. Observation #3: In 1.5, however, checking of 2-way binary compatibility+ throws clause + inheritance hierarchy does not guarantee 2-way source compatibility. Why? - generics, enums, annotations. Yup. === What is our (Harmony) goal? === In terms of these definitions, ideally, I suppose we want that Harmony is 2-way source compatible with the conformant J2SE API implementation (RI API) to make sure that any application compiled with RI API can be compiled with Harmony and vice versa yep, 2-way-source compatible and 2-way-binary compatible. === Automation: what does JAPI check? (http://www.kaffe.org/~stuart/japi/) === You should put some of the info you wrote down in this e-mail up there, its quite valuable :-) === Questions === 1. Can we think of other cases when JAPI does not guarantee 2-way source compatibility, talking about 1.5 language? No, I think you got the main ones in terms of the 1.4-1.5 change. 2. What more checks should be added to JAPI to guarantee 2-way source compatibility for 1.5? You know, I can't even think of a good way to do implement the checks for the generics, let alone think of more! 3. Does somebody know other tools that can check 2-way source compatibility for 1.5 to compare with JAPI? Not me! From seeing what's happening with gump and its transition to use 1.5 for building the ASF stuff, I will assert that sun doesn't know of something like that either, eg, http://www.kaffe.org/~stuart/japi/htmlout/h-jdk14-jdk15.html is rather obvious in that respect, but in fairness most of the mess is all handled within xerces/xml-apis (which is the well known java xml hell problem) so we haven't seen all that many ripples of brokenness... cheers! Leo - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] HARMONY vs. J2SE API source, binary compatibility: JAPI for 1.5 required
On Sat, May 06, 2006 at 10:17:04AM +0400, Anton Avtamonov wrote: - Am I right that source/binary compatibility implies that package-private implementation also matches? I'll suggest you take another look at Vladimir's definition -- source compatibility as he defines it means that, for A and B, you can compile your apps against both A and B. - Do we really need to be source/binary compatible? Yes, by the definitions of souce/binary compatible that Vladimir used. Do you think that existing compatibility guideline http://incubator.apache.org/harmony/subcomponents/classlibrary/compat.html is not enough? I suspect that we will want to extend those guidelines (turning the general bits into specific bits) for some of the things vladimir mentioned. For example, I can imagine that we implement guidelines such as write 1.4 source code when using collections except where it breaks compatibility. cheers! Leo - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
